[OE-core] [PATCH v2] kernel-devsrc: restructure for out of tree (and on target) module builds

Tue Jul 10 13:09:48 UTC 2018

On 07/10/2018 06:21 AM, Richard Purdie wrote:
> On Mon, 2018-07-09 at 11:53 -0400, Bruce Ashfield wrote:
>> The existing kernel-devsrc package starts with a full copy of the
>> kernel
>> source and then starts to strip out elements that are not required.
>>
>> This results in extra time (I/O) and extra space being taken up in
>> the
>> final package. The main purpose of the kernel-devsrc package has been
>> to
>> build modules against the running kernel, not to include a full copy
>> of
>> the source code for re-building the kernel. The end result was a
>> 600M kernel-devsrc package.
>>
>> This restructuring of the package uses an approach similar to other
>> distros, where the kernel-devsrc package is for building against the
>> running kernel and uses a curated set of copied infrastructure,
>> versus
>> a mass copy of the entire kernel.
>>
>> The differences in this approach versus other is largely due to the
>> architecture support and the split build/source directory of the
>> kernel.
>>
>> The result is a kernel-devsrc package of about 10M, which is capable
>> of running "make scripts" and compiling kernel modules against the
>> running kernel.
>>
>> Along with the changes to the copying of the infrascture, we also
>> have the following changes:
>>
>>   - a better/more explicit listing of dependencies for on-target
>>     builds of "make scripts" or "make modules_prepare"
>>
>>   - The kernel source is installed into /lib/modules/<version>/build
>>     and a symlink created from /usr/src/kernel to the new location.
>>     This aligns with the standard location for module support
>>     code
>>
>>   - There is also a symlink from /lib/modules/<version>/source ->
>> build
>>     to reserve a spot for a new package that is simply the kernel
>>     source. That package is not part of this update.
>>
>> Signed-off-by: Bruce Ashfield <bruce.ashfield at windriver.com>
>> ---
>>
>> v2: drop DEPENDS on perf. We no longer need to depend on perf since
>> the source
>>      is copied before modification.
> 
> It probably won't surprise you to know there were some issues with this
> patch, sadly.
> 

heh. No problem. These are just things I didn't have the configs
to test.

Outside of the multilib issue, I can fix things up. For the multlib,
I have no idea .. since this:

   file /usr/bin/libtool from install of libtool-2.4.6-r0.0.i586 
conflicts with file from package lib64-libtool-2.4.6-r0.0.x86_64

is greek to me :D

I'll look at these one by one, and see if anyone with more bitbake
knowledge wants to help out with that last one.

Bruce

> The main recurring issue (on all arches and poky+poky-lsb) is failure
> of the kernelmodule.KernelModuleTest.test_kernel_module test. The exact
> failure varies by arch, for x86-64:
> 
> https://autobuilder.yocto.io/builders/nightly-x86-64/builds/1154/steps/Running%20Sanity%20Tests/logs/stdio
> 
> |   CHK     include/generated/utsrelease.h
> |   DESCEND  objtool
> | /lib/modules/4.14.48-yocto-standard/build/tools/build/Makefile.build:37: /lib/modules/4.14.48-yocto-standard/build/tools/build/Build.include: No such file or directory
> | make[4]: *** No rule to make target '/lib/modules/4.14.48-yocto-standard/build/tools/build/Build.include'.  Stop.
> | make[3]: *** [Makefile:43: /lib/modules/4.14.48-yocto-standard/build/tools/objtool/fixdep-in.o] Error 2
> | make[2]: *** [/lib/modules/4.14.48-yocto-standard/build/tools/build/Makefile.include:4: fixdep] Error 2
> | make[1]: *** [Makefile:62: objtool] Error 2
> | make: *** [Makefile:1647: tools/objtool] Error 2
> 
> For mips:
> 
> https://autobuilder.yocto.io/builders/nightly-mips/builds/1115
> 
> |   HOSTCC  scripts/sortextable
> | make[1]: *** No rule to make target 'arch/mips/boot/tools/relocs_32.c', needed by 'arch/mips/boot/tools/relocs_32.o'.  Stop.
> | make: *** [arch/mips/Makefile:16: archscripts] Error 2
> 
> (same for mips64)
> 
> For arm:
> 
> https://autobuilder.yocto.io/builders/nightly-arm/builds/1187/
> 
> |   HOSTCC  scripts/sortextable
> | make[1]: *** No rule to make target 'arch/arm/tools/syscall.tbl', needed by 'arch/arm/include/generated/uapi/asm/unistd-common.h'.  Stop.
> | make: *** [arch/arm/Makefile:319: archheaders] Error 2
> 
> For arm64:
> 
> https://autobuilder.yocto.io/builders/nightly-arm64/builds/1101/steps/Running%20Sanity%20Tests/logs/stdio
> 
> |   CHK     include/generated/uapi/linux/version.h
> |   CHK     include/generated/utsrelease.h
> | make[1]: *** No rule to make target 'arch/arm64/kernel/vdso/vdso.lds', needed by 'arch/arm64/kernel/vdso/vdso.so.dbg'.  Stop.
> | make: *** [arch/arm64/Makefile:160: vdso_prepare] Error 2
>   
> 
> There was also a failure in building the build-appliance image:
> 
> https://autobuilder.yocto.io/builders/build-appliance/builds/1110
> 
> and also a failure in one of the multilib builds, probably from
> different dependencies pulled in by kernel-devsrc:
> 
> https://autobuilder.yocto.io/builders/nightly-multilib/builds/1139/steps/BuildImages_4/logs/stdio
> 
> Error: Transaction check error:
>    file /usr/bin/libtool from install of libtool-2.4.6-r0.0.i586 conflicts with file from package lib64-libtool-2.4.6-r0.0.x86_64
>    file /usr/bin/libtoolize from install of libtool-2.4.6-r0.0.i586 conflicts with file from package lib64-libtool-2.4.6-r0.0.x86_64
> 
> I'll retest with this patch removed just so we can unblock the other patches.
> 
> Cheers,
> 
> Richard
> 