[OE-core] [PATCH v2] kernel-devsrc: restructure for out of tree (and on target) module builds

Tue Jul 10 16:38:56 UTC 2018

On 07/10/2018 06:21 AM, Richard Purdie wrote:
> On Mon, 2018-07-09 at 11:53 -0400, Bruce Ashfield wrote:
>> The existing kernel-devsrc package starts with a full copy of the
>> kernel
>> source and then starts to strip out elements that are not required.
>>
>> This results in extra time (I/O) and extra space being taken up in
>> the
>> final package. The main purpose of the kernel-devsrc package has been
>> to
>> build modules against the running kernel, not to include a full copy
>> of
>> the source code for re-building the kernel. The end result was a
>> 600M kernel-devsrc package.
>>
>> This restructuring of the package uses an approach similar to other
>> distros, where the kernel-devsrc package is for building against the
>> running kernel and uses a curated set of copied infrastructure,
>> versus
>> a mass copy of the entire kernel.
>>
>> The differences in this approach versus other is largely due to the
>> architecture support and the split build/source directory of the
>> kernel.
>>
>> The result is a kernel-devsrc package of about 10M, which is capable
>> of running "make scripts" and compiling kernel modules against the
>> running kernel.
>>
>> Along with the changes to the copying of the infrascture, we also
>> have the following changes:
>>
>>   - a better/more explicit listing of dependencies for on-target
>>     builds of "make scripts" or "make modules_prepare"
>>
>>   - The kernel source is installed into /lib/modules/<version>/build
>>     and a symlink created from /usr/src/kernel to the new location.
>>     This aligns with the standard location for module support
>>     code
>>
>>   - There is also a symlink from /lib/modules/<version>/source ->
>> build
>>     to reserve a spot for a new package that is simply the kernel
>>     source. That package is not part of this update.
>>
>> Signed-off-by: Bruce Ashfield <bruce.ashfield at windriver.com>
>> ---
>>
>> v2: drop DEPENDS on perf. We no longer need to depend on perf since
>> the source
>>      is copied before modification.
> 
> It probably won't surprise you to know there were some issues with this
> patch, sadly.
> 
> The main recurring issue (on all arches and poky+poky-lsb) is failure
> of the kernelmodule.KernelModuleTest.test_kernel_module test. The exact
> failure varies by arch, for x86-64:
> 
> https://autobuilder.yocto.io/builders/nightly-x86-64/builds/1154/steps/Running%20Sanity%20Tests/logs/stdio
> 
> |   CHK     include/generated/utsrelease.h
> |   DESCEND  objtool
> | /lib/modules/4.14.48-yocto-standard/build/tools/build/Makefile.build:37: /lib/modules/4.14.48-yocto-standard/build/tools/build/Build.include: No such file or directory
> | make[4]: *** No rule to make target '/lib/modules/4.14.48-yocto-standard/build/tools/build/Build.include'.  Stop.
> | make[3]: *** [Makefile:43: /lib/modules/4.14.48-yocto-standard/build/tools/objtool/fixdep-in.o] Error 2
> | make[2]: *** [/lib/modules/4.14.48-yocto-standard/build/tools/build/Makefile.include:4: fixdep] Error 2
> | make[1]: *** [Makefile:62: objtool] Error 2
> | make: *** [Makefile:1647: tools/objtool] Error 2

I'll try the other configs, but clearly I have something different in
my x86-64 build.

I can't run the self tests directly, but am copying the files onto my
qemu session and running things myself:

root at qemux86-64:/lib/modules/4.14.48-yocto-standard/build# make ARCH=x86 
scripts prepare
Makefile:950: "Cannot use CONFIG_STACK_VALIDATION=y, please install 
libelf-dev, libelf-devel or elfutils-libelf-devel"
   CHK     scripts/mod/devicetable-offsets.h
   SYSTBL  arch/x86/include/generated/asm/syscalls_32.h
   SYSHDR  arch/x86/include/generated/asm/unistd_32_ia32.h
   SYSHDR  arch/x86/include/generated/asm/unistd_64_x32.h
   SYSTBL  arch/x86/include/generated/asm/syscalls_64.h
   SYSHDR  arch/x86/include/generated/uapi/asm/unistd_32.h
   SYSHDR  arch/x86/include/generated/uapi/asm/unistd_64.h
   SYSHDR  arch/x86/include/generated/uapi/asm/unistd_x32.h
   HOSTCC  arch/x86/tools/relocs_32.o
   HOSTCC  arch/x86/tools/relocs_64.o
   HOSTCC  arch/x86/tools/relocs_common.o
   HOSTLD  arch/x86/tools/relocs
   CHK     include/config/kernel.release
   CHK     include/generated/uapi/linux/version.h
   CHK     include/generated/utsrelease.h
Makefile:950: "Cannot use CONFIG_STACK_VALIDATION=y, please install 
libelf-dev, libelf-devel or elfutils-libelf-devel"

root at qemux86-64:/tmp# make
make -C /usr/src/kernel M=/tmp modules
make[1]: Entering directory '/lib/modules/4.14.48-yocto-standard/build'
Makefile:950: "Cannot use CONFIG_STACK_VALIDATION=y, please install 
libelf-dev, libelf-devel or elfutils-libelf-devel"
   Building modules, stage 2.
   MODPOST 1 modules
make[1]: Leaving directory '/lib/modules/4.14.48-yocto-standard/build'
root at qemux86-64:/tmp# insmod hellomod.ko
[  419.211616] hellomod: loading out-of-tree module taints kernel.
[  419.212586] Hello world!

This is on the core-image-kernel-dev image, which is the one that
I created to test all this.

What image is that using ? some sato sdk one ?

Bruce

> 
> For mips:
> 
> https://autobuilder.yocto.io/builders/nightly-mips/builds/1115
> 
> |   HOSTCC  scripts/sortextable
> | make[1]: *** No rule to make target 'arch/mips/boot/tools/relocs_32.c', needed by 'arch/mips/boot/tools/relocs_32.o'.  Stop.
> | make: *** [arch/mips/Makefile:16: archscripts] Error 2
> 
> (same for mips64)
> 
> For arm:
> 
> https://autobuilder.yocto.io/builders/nightly-arm/builds/1187/
> 
> |   HOSTCC  scripts/sortextable
> | make[1]: *** No rule to make target 'arch/arm/tools/syscall.tbl', needed by 'arch/arm/include/generated/uapi/asm/unistd-common.h'.  Stop.
> | make: *** [arch/arm/Makefile:319: archheaders] Error 2
> 
> For arm64:
> 
> https://autobuilder.yocto.io/builders/nightly-arm64/builds/1101/steps/Running%20Sanity%20Tests/logs/stdio
> 
> |   CHK     include/generated/uapi/linux/version.h
> |   CHK     include/generated/utsrelease.h
> | make[1]: *** No rule to make target 'arch/arm64/kernel/vdso/vdso.lds', needed by 'arch/arm64/kernel/vdso/vdso.so.dbg'.  Stop.
> | make: *** [arch/arm64/Makefile:160: vdso_prepare] Error 2
>   
> 
> There was also a failure in building the build-appliance image:
> 
> https://autobuilder.yocto.io/builders/build-appliance/builds/1110
> 
> and also a failure in one of the multilib builds, probably from
> different dependencies pulled in by kernel-devsrc:
> 
> https://autobuilder.yocto.io/builders/nightly-multilib/builds/1139/steps/BuildImages_4/logs/stdio
> 
> Error: Transaction check error:
>    file /usr/bin/libtool from install of libtool-2.4.6-r0.0.i586 conflicts with file from package lib64-libtool-2.4.6-r0.0.x86_64
>    file /usr/bin/libtoolize from install of libtool-2.4.6-r0.0.i586 conflicts with file from package lib64-libtool-2.4.6-r0.0.x86_64
> 
> I'll retest with this patch removed just so we can unblock the other patches.
> 
> Cheers,
> 
> Richard
> 