[OE-core] Packaging kernel sources

[Bruce Ashfield]

On 14-09-10 11:27 AM, Brandt, Todd E wrote:
> I think David brings up a good point about only needing the kernel
> source when something goes wrong. How about a compromise. What if we
> provided a simply utility which pulls in the kernel source and
> recreates the existing kernel image by using git (with the proper
> commit). It could be installed from within the kernel package and be
> generated by the linux-kernel recipe so that it has the proper commit
> hashes (like a simple bash script). That way there's no wasted space.
> I think I might just do that for the heck of it anyway.

We have to respect however the kernel was built, patched, etc. So it
just needs to be whatever was in the ${S} of what was built. Much of
anything else would be recreating the patch process of the kernel
build .. or maybe I'm misunderstanding what you are suggesting.

Bruce




> ________________________________________
> From: Darren Hart [dvhart at linux.intel.com]
> Sent: Wednesday, September 10, 2014 8:13 AM
> To: Richard Purdie; Ashfield, Bruce (Wind River)
> Cc: Openembedded-core at lists.openembedded.org; Brandt, Todd E; Koen Kooi; Tom Zanussi
> Subject: Re: [OE-core] Packaging kernel sources
>
> On 9/10/14, 1:27, "Richard Purdie" <richard.purdie at linuxfoundation.org>
> wrote:
>
>> On Tue, 2014-09-09 at 17:42 -0700, Darren Hart wrote:
>>> I'm working on a project which needs to have the full kernel sources
>>> installed on the target. The kernel-dev package as defined by
>>> kernel.bbclass is heavily pruned to minimize packaging time and size and
>>> is intended to enable building of external modules on the target.
>>>
>>> Is there an accepted best-practice for how to get the full source
>>> packaged
>>> and installed? I can easily write a new recipe,
>>> linux-custom-source_git.bb, to install the sources, for example, without
>>> impacting the packaging time of "virtual/kernel" package.
>>>
>>> It would be nice in some respects for it to all come from the same
>>> recipe
>>> though, but I suspect the impact to the common-case where this is not
>>> need
>>> would be far too great.
>>
>> Personally, I'm leaning towards a couple of big changes in this area:
>>
>> a) "binning" the kernel-dev package and replacing it with some kind of
>> separate full source recipe like this.
>>
>> The benefit is a fully functional on target source which is only built
>> by people who care about it. This means for most users/builds, we no
>> longer need to generate that huge package. The downside is a little more
>> complexity for those that needs this but its not much.
>
> The other downside is that the most common use case (building external
> modules) would now require a lot more disk space than with just kernel-dev
> (something like 150 MB more iirc).
>
>>
>>
>> b) binning the separate kernel staging dir and making it work more like
>> the gcc shared work directory. This means external module builds and the
>> tools like perf and so on would use this shared source directory.
>
> I was thinking along the same lines here as well.
>
>>
>> The benefit would be that we no longer have the huge install step in the
>> main kernel recipe and the populate_sysroot step shinks in size.
>>
>> The downside has more impact here, the problem with shared work is that
>> it cannot be removed once extracted since the system never knows when
>> something else may need to use it. For gcc the argument was that we have
>> so many users (gcc-cross-initial, gcc-cross, gcc-runtime,
>> gcc-cross-canadian, gcc-crosssdk, gcc-crosssdk-initial and so on) that
>> the multiple copies were far worse. For the kernel, we can argue that we
>> have a ton of disk usage from it in the sysroot anyway so this change
>> just makes things more efficient effectively.
>>
>> The other issue is that for shared work dirs, the stamps need to be kept
>> in sync, if they step out, odd things happen (i.e. do_fetch, do_unpack,
>> do_patch task checksums need to match for linux-yocto, perf, kernel
>> modules and anything else using it). We may need to add some better
>> error cases to catch problems. Not an insurmountable problem, just one
>> that will likely need to be addressed.
>
> Good points.
>
>>
>> I do feel the whole situation with the current kernel size is out of
>> control and badly affecting user experience.
>
>
> Yup.
>
> --
> Darren Hart                                     Open Source Technology Center
> darren.hart at intel.com                                       Intel Corporation
>
>
>