[OE-core] [PATCH v2] kernel-devsrc: restructure for out of tree (and on target) module builds

Fri Mar 2 19:29:51 UTC 2018

On Fri, Mar 2, 2018 at 1:16 PM, Khem Raj <raj.khem at gmail.com> wrote:
> On Thu, Mar 1, 2018 at 1:37 AM, Burton, Ross <ross.burton at intel.com> wrote:
>> Also fails on musl:
>>
>> ERROR: Nothing RPROVIDES 'glibc-utils' (but
>> /home/pokybuild/yocto-autobuilder/yocto-worker/nightly-musl/build/meta/recipes-kernel/linux/kernel-devsrc.bb
>> RDEPENDS on or otherwise requires it)
>>
>
> I sent patches to add musl-utils which should help this case. Perhaps
> adding glibc-utils to
> deps unconditionally is not right thing here. We can use libc-glibc
> and libc-musl overrides
> to select the right utils.

Yah. I've already removed that in my updated WIP version of the patch,
I'm just trying to complete
the work and get the scripts building on target with musl .. I assume
I can use the c library variable
and select the right package ?

Bruce

>
> Other idea is to create virtual/libc-utils but that seems like an overkill.
>
>> Ross
>>
>> On 1 March 2018 at 08:28, Maxin B. John <maxin.john at intel.com> wrote:
>>>
>>> Hi,
>>>
>>> On Wed, Feb 28, 2018 at 02:20:45PM -0500, Bruce Ashfield wrote:
>>> > The existing kernel-devsrc package starts with a full copy of the kernel
>>> > source and then starts to strip out elements that are not required.
>>> >
>>> > This results in extra time (I/O) and extra space being taken up in the
>>> > final package. The main purpose of the kernel-devsrc package has been to
>>> > build modules against the running kernel, not to include a full copy of
>>> > the source code for re-building the kernel. The end result was a
>>> > 600M kernel-devsrc package.
>>> >
>>> > This restructuring of the package uses an approach similar to other
>>> > distros, where the kernel-devsrc package is for building against the
>>> > running kernel and uses a curated set of copied infrastructure, versus
>>> > a mass copy of the entire kernel.
>>> >
>>> > The differences in this approach versus other is largely due to the
>>> > architecture support and the split build/source directory of the
>>> > kernel.
>>> >
>>> > The result is a kernel-devsrc package of about 10M, which is capable
>>> > of running "make scripts" and compiling kernel modules against the
>>> > running kernel.
>>> >
>>> > Along with the changes to the copying of the infrascture, we also
>>> > have the following changes:
>>> >
>>> >  - a better/more explicit listing of dependencies for on-target
>>> >    builds of "make scripts" or "make modules_prepare"
>>> >
>>> >  - The kernel source is installed into /lib/modules/<version>/build
>>> >    and a symlink created from /usr/src/kernel to the new location.
>>> >    This aligns with the standard location for module support
>>> >    code
>>> >
>>> >  - There is also a symlink from /lib/modules/<version>/source -> build
>>> >    to reserve a spot for a new package that is simply the kernel
>>> >    source. That package is not part of this update.
>>> >
>>> > [YOCTO #12535]
>>> >
>>> > Signed-off-by: Bruce Ashfield <bruce.ashfield at windriver.com>
>>> > ---
>>> >
>>> >  v2: fix arch/arm devsrc builds
>>>
>>> Noticed one kernel-devsrc build failure in autobuilder (qemuarm) :
>>>
>>>
>>> https://autobuilder.yocto.io/builders/nightly-arm/builds/876/steps/BuildImages/logs/stdio
>>>
>>> ERROR: kernel-devsrc-1.0-r0 do_install: Function failed: do_install (log
>>> file is located at
>>> /home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/build/tmp/work/qemuarm-poky-linux-gnueabi/kernel-devsrc/1.0-r0/temp/log.do_install.25822)
>>> ERROR: Logfile of failure stored in:
>>> /home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/build/tmp/work/qemuarm-poky-linux-gnueabi/kernel-devsrc/1.0-r0/temp/log.do_install.25822
>>> Log data follows:
>>> | DEBUG: Executing python function extend_recipe_sysroot
>>> | NOTE: Direct dependencies are
>>> ['/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/meta/recipes-devtools/binutils/binutils-cross_2.30.bb:do_populate_sysroot',
>>> 'virtual:native:/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/meta/recipes-extended/xz/xz_5.2.3.bb:do_populate_sysroot',
>>> 'virtual:native:/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/meta/recipes-devtools/pkgconfig/pkgconfig_git.bb:do_populate_sysroot',
>>> 'virtual:native:/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/meta/recipes-support/lzop/lzop_1.03.bb:do_populate_sysroot',
>>> '/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/meta/recipes-kernel/kern-tools/kern-tools-native_git.bb:do_populate_sysroot',
>>> '/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/meta/recipes-devtools/gcc/gcc-runtime_7.3.bb:do_populate_sysroot',
>>> '/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/meta/recipes-devtools/gcc/gcc-cross_7.
>>>  3.bb:do_populate_sysroot',
>>> 'virtual:native:/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/meta/recipes-devtools/bison/bison_3.0.4.bb:do_populate_sysroot',
>>> 'virtual:native:/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/meta/recipes-devtools/pseudo/pseudo_1.8.2.bb:do_populate_sysroot',
>>> '/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/meta/recipes-core/glibc/glibc_2.26.bb:do_populate_sysroot',
>>> '/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/meta/recipes-kernel/kmod/kmod-native_git.bb:do_populate_sysroot',
>>> 'virtual:native:/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/meta/recipes-extended/bc/bc_1.06.bb:do_populate_sysroot',
>>> '/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/meta/recipes-devtools/quilt/quilt-native_0.65.bb:do_populate_sysroot',
>>> 'virtual:native:/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/meta/recipes-connectivity/openssl/openssl_1.0.2n.bb:d
>>>  o_populate_sysroot']
>>> | NOTE: Installed into sysroot: ['pkgconfig-native', 'lzop-native',
>>> 'kern-tools-native', 'pseudo-native', 'kmod-native', 'bc-native',
>>> 'openssl-native', 'gtk-doc-native', 'lzo-native', 'makedepend-native',
>>> 'cryptodev-linux-native', 'util-macros-native', 'xproto-native']
>>> | NOTE: Skipping as already exists in sysroot: ['binutils-cross-arm',
>>> 'xz-native', 'gcc-runtime', 'gcc-cross-arm', 'bison-native', 'glibc',
>>> 'quilt-native', 'gettext-minimal-native', 'gnu-config-native',
>>> 'automake-native', 'autoconf-native', 'libtool-native',
>>> 'linux-libc-headers', 'libgcc', 'zlib-native', 'texinfo-dummy-native',
>>> 'flex-native', 'libmpc-native', 'gmp-native', 'mpfr-native', 'm4-native']
>>> | DEBUG: sed -e
>>> 's:^[^/]*/:/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/build/tmp/work/qemuarm-poky-linux-gnueabi/kernel-devsrc/1.0-r0/recipe-sysroot-native/:g'
>>> /home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/build/tmp/sysroots-components/x86_64/pkgconfig-native/fixmepath
>>> /home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/build/tmp/sysroots-components/x86_64/openssl-native/fixmepath
>>> /home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/build/tmp/sysroots-components/x86_64/gtk-doc-native/fixmepath
>>> | xargs sed -i -e
>>> 's:FIXMESTAGINGDIRTARGET:/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/build/tmp/work/qemuarm-poky-linux-gnueabi/kernel-devsrc/1.0-r0/recipe-sysroot:g;
>>> s:FIXMESTAGINGDIRHOST:/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/build/tmp/work/qemuarm-poky-linux-gnueabi/kernel-devsrc/1.0-r0/recipe-sysroot-native:g'
>>> -e 's:FIXME_COMPONENTS_DIR:/home/pokybuild/yocto-autobuilder/y
>>>  octo-worker/nightly-arm/build/build/tmp/sysroots-components:g' -e
>>> 's:FIXME_HOSTTOOLS_DIR:/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/build/tmp/hosttools:g'
>>> -e
>>> 's:FIXME_PKGDATA_DIR:/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/build/tmp/pkgdata/qemuarm:g'
>>> -e
>>> 's:FIXME_PSEUDO_LOCALSTATEDIR:/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/build/tmp/work/qemuarm-poky-linux-gnueabi/kernel-devsrc/1.0-r0/pseudo/:g'
>>> -e
>>> 's:FIXME_LOGFIFO:/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/build/tmp/work/qemuarm-poky-linux-gnueabi/kernel-devsrc/1.0-r0/temp/fifo.25822:g'
>>> | DEBUG: Python function extend_recipe_sysroot finished
>>> | DEBUG: Executing shell function do_install
>>> | cp: with --parents, the destination must be a directory
>>> | Try 'cp --help' for more information.
>>> | WARNING:
>>> /home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/build/tmp/work/qemuarm-poky-linux-gnueabi/kernel-devsrc/1.0-r0/temp/run.do_install.25822:1
>>> exit 1 from 'rm -rf $kerneldir/build/include'
>>> | ERROR: Function failed: do_install (log file is located at
>>> /home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/build/tmp/work/qemuarm-poky-linux-gnueabi/kernel-devsrc/1.0-r0/temp/log.do_install.25822)
>>> NOTE: recipe kernel-devsrc-1.0-r0: task do_install: Failed
>>> ERROR: Task
>>> (/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/meta/recipes-kernel/linux/kernel-devsrc.bb:do_install)
>>> failed with exit code '1'
>>>
>>> Best Regards,
>>> Maxin
>>> --
>>> _______________________________________________
>>> Openembedded-core mailing list
>>> Openembedded-core at lists.openembedded.org
>>> http://lists.openembedded.org/mailman/listinfo/openembedded-core
>>
>>
>>
>> --
>> _______________________________________________
>> Openembedded-core mailing list
>> Openembedded-core at lists.openembedded.org
>> http://lists.openembedded.org/mailman/listinfo/openembedded-core
>>
> --
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core at lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core

-- 
"Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end"