[OE-core] [PATCH] kernel: Add support for multiple kernel packages
Denys Dmytriyenko
denis at denix.org
Tue Dec 15 16:58:26 UTC 2015
On Mon, Dec 14, 2015 at 03:02:44PM -0500, Bruce Ashfield wrote:
> On Mon, Dec 14, 2015 at 2:17 PM, Bruce Ashfield <bruce.ashfield at gmail.com>
> wrote:
>
> >
> >
> > On Mon, Dec 14, 2015 at 2:06 PM, Haris Okanovic <haris.okanovic at ni.com>
> > wrote:
> >
> >> From: Gratian Crisan <gratian.crisan at ni.com>
> >>
> >> Templetize kernel package name:
> >>
> >> Add a "weak" variable KERNEL_PACKAGE_NAME used as the base name for kernel
> >> packages. It defaults to the old hard-coded name 'kernel' but it can be
> >> redefined by recipes that provide additional kernel packages in order
> >> to avoid build conflicts.
> >>
> >> Change hard-coded 'kernel' references to KERNEL_PACKAGE_NAME in
> >> kernel bbclass-es.
> >>
> >> Build alternate kernels from WORKDIR instead of STAGING_KERNEL_DIR:
> >>
> >> Prior to this change, kernel recipes would all fetch source to
> >> STAGING_KERNEL_DIR as defined by bitbake/distro confs. This broke
> >> parallel builds when more than one kernel recipes are defined in
> >> the distribution, since they all attempted to fetch() and patch()
> >> in a shared source dir.
> >>
> >> With this change, alternate kernel recipes fetch source into their
> >> ${WORKDIR} so that they may build in parallel to each other and the
> >> default kernel recipe, which still fetches to STAGING_KERNEL_DIR.
> >>
> >
> >
> > But what's the system level use case for this capability ? Honestly, it
> > just
> > looks complicated .. and I'm not seeing exactly multiple kernels are being
> > built in parallel in a single build. We've been forcing everything
> > explicitly
> > to be in a single, shared location .. and I see that this is both
> > complicating
> > and a possible source of errors.
> >
> > It's probably that I'm just misunderstanding the point of the change.
> >
> > If I had to guess, you are building recovery, or kexec/kdump kernels along
> > side the main kernel ?
> >
>
> Aha. I see the debug kernel reference in the comments (as one example).
>
> As a follow up. If these kernel's are building via kernel.bbclass, how are
> you
> avoiding the artifacts all ending up in STAGING_KERNEL_BUILDDIR ? I did
> a quick scan, and couldn't see if it was being considered.
And the next question would be - how can you build/rebuild and package
out-of-tree modules against all the alternatives?
I've looked at doing the same for our needs and ran out of time and patience
to cover all the corner cases... :)
As of the use cases - we have a need to provide multiple variants of the same
kernel per platform:
1. Production
2. All debugging options enabled
3. RT patch applied
4. Any other major feature, like LPAE
It is common for Linux distros to provide multiple kernel variants:
https://wiki.ubuntu.com/Kernel/Dev/Flavours
So, it's not entirely unheard of...
--
Denys
> >> Testing:
> >>
> >> Built linux-yocto-4.1.13 for qemux86 and verified it produced kernel
> >> image and modules packages. Added kernel recipe with non-default
> >> KERNEL_PACKAGE_NAME and verified it produces kernel image and modules
> >> packages with alternate name next to default kernel.
> >>
> >> Signed-off-by: Gratian Crisan <gratian.crisan at ni.com>
> >> Signed-off-by: Haris Okanovic <haris.okanovic at ni.com>
> >> Coauthored-by: Haris Okanovic <haris.okanovic at ni.com>
> >> Coauthored-by: Josh Hernstrom <josh.hernstrom at ni.com>
> >> Natinst-ReviewBoard-ID: 120348
> >> Natinst-ReviewBoard-ID: 120447
> >> ---
> >> meta/classes/kernel-module-split.bbclass | 9 ++--
> >> meta/classes/kernel.bbclass | 71
> >> ++++++++++++++++++--------------
> >> meta/conf/documentation.conf | 1 +
> >> meta/recipes-kernel/linux/linux-dtb.inc | 2 +-
> >> 4 files changed, 49 insertions(+), 34 deletions(-)
> >>
> >> diff --git a/meta/classes/kernel-module-split.bbclass
> >> b/meta/classes/kernel-module-split.bbclass
> >> index e1a70e6..7415ec8 100644
> >> --- a/meta/classes/kernel-module-split.bbclass
> >> +++ b/meta/classes/kernel-module-split.bbclass
> >> @@ -28,7 +28,7 @@ do_install_append() {
> >>
> >> PACKAGESPLITFUNCS_prepend = "split_kernel_module_packages "
> >>
> >> -KERNEL_MODULES_META_PACKAGE ?= "kernel-modules"
> >> +KERNEL_MODULES_META_PACKAGE ?= "${KERNEL_PACKAGE_NAME}-modules"
> >>
> >> python split_kernel_module_packages () {
> >> import re
> >> @@ -179,14 +179,17 @@ python split_kernel_module_packages () {
> >> # Avoid automatic -dev recommendations for modules ending with
> >> -dev.
> >> d.setVarFlag('RRECOMMENDS_' + pkg, 'nodeprrecs', 1)
> >>
> >> + kernel_package_name = d.getVar("KERNEL_PACKAGE_NAME", True)
> >> + kernel_version = d.getVar("KERNEL_VERSION", True)
> >> +
> >> module_deps = parse_depmod()
> >> module_regex = '^(.*)\.k?o$'
> >> - module_pattern = 'kernel-module-%s'
> >> + module_pattern = '%s-module-%%s' % kernel_package_name
> >>
> >> postinst = d.getVar('pkg_postinst_modules', True)
> >> postrm = d.getVar('pkg_postrm_modules', True)
> >>
> >> - modules = do_split_packages(d, root='/lib/modules',
> >> file_regex=module_regex, output_pattern=module_pattern, description='%s
> >> kernel module', postinst=postinst, postrm=postrm, recursive=True,
> >> hook=frob_metadata, extra_depends='kernel-%s' % (d.getVar("KERNEL_VERSION",
> >> True)))
> >> + modules = do_split_packages(d, root='/lib/modules',
> >> file_regex=module_regex, output_pattern=module_pattern, description='%s
> >> kernel module', postinst=postinst, postrm=postrm, recursive=True,
> >> hook=frob_metadata, extra_depends='%s-%s' % (kernel_package_name,
> >> kernel_version))
> >> if modules:
> >> metapkg = d.getVar('KERNEL_MODULES_META_PACKAGE', True)
> >> d.appendVar('RDEPENDS_' + metapkg, ' '+' '.join(modules))
> >> diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
> >> index b75a462..cc27364 100644
> >> --- a/meta/classes/kernel.bbclass
> >> +++ b/meta/classes/kernel.bbclass
> >> @@ -3,7 +3,6 @@ inherit linux-kernel-base kernel-module-split
> >> PROVIDES += "virtual/kernel"
> >> DEPENDS += "virtual/${TARGET_PREFIX}binutils virtual/${TARGET_PREFIX}gcc
> >> kmod-native depmodwrapper-cross bc-native"
> >>
> >> -S = "${STAGING_KERNEL_DIR}"
> >> B = "${WORKDIR}/build"
> >> KBUILD_OUTPUT = "${B}"
> >> OE_TERMINAL_EXPORTS += "KBUILD_OUTPUT"
> >> @@ -11,6 +10,7 @@ OE_TERMINAL_EXPORTS += "KBUILD_OUTPUT"
> >> # we include gcc above, we dont need virtual/libc
> >> INHIBIT_DEFAULT_DEPS = "1"
> >>
> >> +KERNEL_PACKAGE_NAME ??= "kernel"
> >> KERNEL_IMAGETYPE ?= "zImage"
> >> INITRAMFS_IMAGE ?= ""
> >> INITRAMFS_TASK ?= ""
> >> @@ -34,6 +34,17 @@ python __anonymous () {
> >> image_task = d.getVar('INITRAMFS_TASK', True)
> >> if image_task:
> >> d.appendVarFlag('do_configure', 'depends', ' ${INITRAMFS_TASK}')
> >> +
> >> + # The default kernel recipe fetches it's source to
> >> + # STAGING_KERNEL_DIR as defined by bitbake/distro confs.
> >> + # Alternate kernel recipes (E.g. debug kernels) fetch source into
> >> + # their work dir so that they may build in parallel.
> >> + if d.getVar("KERNEL_PACKAGE_NAME", True) == "kernel":
> >> + d.setVar("S", d.getVar("STAGING_KERNEL_DIR", True))
> >> + else:
> >> + workdir = d.getVar("WORKDIR", True)
> >> + kernelSourceDir = os.path.join(workdir, 'kernel-source')
> >> + d.setVar("S", kernelSourceDir)
> >> }
> >>
> >> # Here we pull in all various kernel image types which we support.
> >> @@ -79,9 +90,9 @@ base_do_unpack_append () {
> >>
> >> inherit kernel-arch deploy
> >>
> >> -PACKAGES_DYNAMIC += "^kernel-module-.*"
> >> -PACKAGES_DYNAMIC += "^kernel-image-.*"
> >> -PACKAGES_DYNAMIC += "^kernel-firmware-.*"
> >> +PACKAGES_DYNAMIC += "^${KERNEL_PACKAGE_NAME}-module-.*"
> >> +PACKAGES_DYNAMIC += "^${KERNEL_PACKAGE_NAME}-image-.*"
> >> +PACKAGES_DYNAMIC += "^${KERNEL_PACKAGE_NAME}-firmware-.*"
> >>
> >> export OS = "${TARGET_OS}"
> >> export CROSS_COMPILE = "${TARGET_PREFIX}"
> >> @@ -270,9 +281,9 @@ do_shared_workdir_setscene () {
> >>
> >> emit_depmod_pkgdata() {
> >> # Stash data for depmod
> >> - install -d ${PKGDESTWORK}/kernel-depmod/
> >> - echo "${KERNEL_VERSION}" >
> >> ${PKGDESTWORK}/kernel-depmod/kernel-abiversion
> >> - cp ${B}/System.map
> >> ${PKGDESTWORK}/kernel-depmod/System.map-${KERNEL_VERSION}
> >> + install -d ${PKGDESTWORK}/${KERNEL_PACKAGE_NAME}-depmod/
> >> + echo "${KERNEL_VERSION}" >
> >> ${PKGDESTWORK}/${KERNEL_PACKAGE_NAME}-depmod/${KERNEL_PACKAGE_NAME}-abiversion
> >> + cp ${B}/System.map
> >> ${PKGDESTWORK}/${KERNEL_PACKAGE_NAME}-depmod/System.map-${KERNEL_VERSION}
> >> }
> >>
> >> PACKAGEFUNCS += "emit_depmod_pkgdata"
> >> @@ -287,7 +298,7 @@ do_shared_workdir () {
> >> # Store the kernel version in sysroots for module-base.bbclass
> >> #
> >>
> >> - echo "${KERNEL_VERSION}" > $kerneldir/kernel-abiversion
> >> + echo "${KERNEL_VERSION}" >
> >> $kerneldir/${KERNEL_PACKAGE_NAME}-abiversion
> >>
> >> # Copy files required for module builds
> >> cp System.map $kerneldir/System.map-${KERNEL_VERSION}
> >> @@ -361,28 +372,28 @@ EXPORT_FUNCTIONS do_compile do_install do_configure
> >>
> >> # kernel-base becomes kernel-${KERNEL_VERSION}
> >> # kernel-image becomes kernel-image-${KERNEL_VERISON}
> >> -PACKAGES = "kernel kernel-base kernel-vmlinux kernel-image kernel-dev
> >> kernel-modules"
> >> +PACKAGES = "${KERNEL_PACKAGE_NAME} ${KERNEL_PACKAGE_NAME}-base
> >> ${KERNEL_PACKAGE_NAME}-vmlinux ${KERNEL_PACKAGE_NAME}-image
> >> ${KERNEL_PACKAGE_NAME}-dev ${KERNEL_PACKAGE_NAME}-modules"
> >> FILES_${PN} = ""
> >> -FILES_kernel-base = "/lib/modules/${KERNEL_VERSION}/modules.order
> >> /lib/modules/${KERNEL_VERSION}/modules.builtin"
> >> -FILES_kernel-image = "/boot/${KERNEL_IMAGETYPE}*"
> >> -FILES_kernel-dev = "/boot/System.map* /boot/Module.symvers*
> >> /boot/config* ${KERNEL_SRC_PATH} /lib/modules/${KERNEL_VERSION}/build"
> >> -FILES_kernel-vmlinux = "/boot/vmlinux*"
> >> -FILES_kernel-modules = ""
> >> -RDEPENDS_kernel = "kernel-base"
> >> +FILES_${KERNEL_PACKAGE_NAME}-base =
> >> "/lib/modules/${KERNEL_VERSION}/modules.order
> >> /lib/modules/${KERNEL_VERSION}/modules.builtin"
> >> +FILES_${KERNEL_PACKAGE_NAME}-image = "/boot/${KERNEL_IMAGETYPE}*"
> >> +FILES_${KERNEL_PACKAGE_NAME}-dev = "/boot/System.map*
> >> /boot/Module.symvers* /boot/config* ${KERNEL_SRC_PATH}
> >> /lib/modules/${KERNEL_VERSION}/build"
> >> +FILES_${KERNEL_PACKAGE_NAME}-vmlinux = "/boot/vmlinux*"
> >> +FILES_${KERNEL_PACKAGE_NAME}-modules = ""
> >> +RDEPENDS_${KERNEL_PACKAGE_NAME} = "${KERNEL_PACKAGE_NAME}-base"
> >> # Allow machines to override this dependency if kernel image files are
> >> # not wanted in images as standard
> >> -RDEPENDS_kernel-base ?= "kernel-image"
> >> -PKG_kernel-image =
> >> "kernel-image-${@legitimize_package_name('${KERNEL_VERSION}')}"
> >> -RDEPENDS_kernel-image += "${@base_conditional('KERNEL_IMAGETYPE',
> >> 'vmlinux', 'kernel-vmlinux', '', d)}"
> >> -PKG_kernel-base =
> >> "kernel-${@legitimize_package_name('${KERNEL_VERSION}')}"
> >> -RPROVIDES_kernel-base += "kernel-${KERNEL_VERSION}"
> >> -ALLOW_EMPTY_kernel = "1"
> >> -ALLOW_EMPTY_kernel-base = "1"
> >> -ALLOW_EMPTY_kernel-image = "1"
> >> -ALLOW_EMPTY_kernel-modules = "1"
> >> -DESCRIPTION_kernel-modules = "Kernel modules meta package"
> >> -
> >> -pkg_postinst_kernel-base () {
> >> +RDEPENDS_${KERNEL_PACKAGE_NAME}-base ?= "${KERNEL_PACKAGE_NAME}-image"
> >> +PKG_${KERNEL_PACKAGE_NAME}-image =
> >> "${KERNEL_PACKAGE_NAME}-image-${@legitimize_package_name('${KERNEL_VERSION}')}"
> >> +RDEPENDS_${KERNEL_PACKAGE_NAME}-image +=
> >> "${@base_conditional('KERNEL_IMAGETYPE', 'vmlinux',
> >> '${KERNEL_PACKAGE_NAME}-vmlinux', '', d)}"
> >> +PKG_${KERNEL_PACKAGE_NAME}-base =
> >> "${KERNEL_PACKAGE_NAME}-${@legitimize_package_name('${KERNEL_VERSION}')}"
> >> +RPROVIDES_${KERNEL_PACKAGE_NAME}-base +=
> >> "${KERNEL_PACKAGE_NAME}-${KERNEL_VERSION}"
> >> +ALLOW_EMPTY_${KERNEL_PACKAGE_NAME} = "1"
> >> +ALLOW_EMPTY_${KERNEL_PACKAGE_NAME}-base = "1"
> >> +ALLOW_EMPTY_${KERNEL_PACKAGE_NAME}-image = "1"
> >> +ALLOW_EMPTY_${KERNEL_PACKAGE_NAME}-modules = "1"
> >> +DESCRIPTION_${KERNEL_PACKAGE_NAME}-modules = "Kernel modules meta
> >> package"
> >> +
> >> +pkg_postinst_${KERNEL_PACKAGE_NAME}-base () {
> >> if [ ! -e "$D/lib/modules/${KERNEL_VERSION}" ]; then
> >> mkdir -p $D/lib/modules/${KERNEL_VERSION}
> >> fi
> >> @@ -393,18 +404,18 @@ pkg_postinst_kernel-base () {
> >> fi
> >> }
> >>
> >> -pkg_postinst_kernel-image () {
> >> +pkg_postinst_${KERNEL_PACKAGE_NAME}-image () {
> >> update-alternatives --install
> >> /${KERNEL_IMAGEDEST}/${KERNEL_IMAGETYPE} ${KERNEL_IMAGETYPE}
> >> /${KERNEL_IMAGEDEST}/${KERNEL_IMAGETYPE}-${KERNEL_VERSION}
> >> ${KERNEL_PRIORITY} || true
> >> }
> >>
> >> -pkg_postrm_kernel-image () {
> >> +pkg_postrm_${KERNEL_PACKAGE_NAME}-image () {
> >> update-alternatives --remove ${KERNEL_IMAGETYPE}
> >> ${KERNEL_IMAGETYPE}-${KERNEL_VERSION} || true
> >> }
> >>
> >> PACKAGESPLITFUNCS_prepend = "split_kernel_packages "
> >>
> >> python split_kernel_packages () {
> >> - do_split_packages(d, root='/lib/firmware',
> >> file_regex='^(.*)\.(bin|fw|cis|dsp)$', output_pattern='kernel-firmware-%s',
> >> description='Firmware for %s', recursive=True, extra_depends='')
> >> + do_split_packages(d, root='/lib/firmware',
> >> file_regex='^(.*)\.(bin|fw|cis|dsp)$',
> >> output_pattern='${KERNEL_PACKAGE_NAME}-firmware-%s', description='Firmware
> >> for %s', recursive=True, extra_depends='')
> >> }
> >>
> >> do_strip() {
> >> diff --git a/meta/conf/documentation.conf b/meta/conf/documentation.conf
> >> index 1b5c071..fb5e03f 100644
> >> --- a/meta/conf/documentation.conf
> >> +++ b/meta/conf/documentation.conf
> >> @@ -250,6 +250,7 @@ KERNEL_FEATURES[doc] = "Includes additional metadata
> >> from the Yocto Project kern
> >> KERNEL_IMAGETYPE[doc] = "The type of kernel to build for a device,
> >> usually set by the machine configuration files and defaults to 'zImage'."
> >> KERNEL_MODULE_AUTOLOAD[doc] = "Lists kernel modules that need to be
> >> auto-loaded during boot"
> >> KERNEL_MODULE_PROBECONF[doc] = "Lists kernel modules for which the build
> >> system expects to find module_conf_* values that specify configuration for
> >> each of the modules"
> >> +KERNEL_PACKAGE_NAME[doc] = "Name prefix for kernel packages. Defaults to
> >> 'kernel'."
> >> KERNEL_PATH[doc] = "The location of the kernel sources. This variable is
> >> set to the value of the STAGING_KERNEL_DIR within the module class
> >> (module.bbclass)."
> >> KERNEL_SRC[doc] = "The location of the kernel sources. This variable is
> >> set to the value of the STAGING_KERNEL_DIR within the module class
> >> (module.bbclass)."
> >> KFEATURE_DESCRIPTION[doc] = "Provides a short description of a
> >> configuration fragment. You use this variable in the .scc file that
> >> describes a configuration fragment file."
> >> diff --git a/meta/recipes-kernel/linux/linux-dtb.inc
> >> b/meta/recipes-kernel/linux/linux-dtb.inc
> >> index 772adcb..c27a3cb 100644
> >> --- a/meta/recipes-kernel/linux/linux-dtb.inc
> >> +++ b/meta/recipes-kernel/linux/linux-dtb.inc
> >> @@ -2,7 +2,7 @@
> >> FILES_kernel-devicetree = "/${KERNEL_IMAGEDEST}/devicetree*"
> >>
> >> python __anonymous () {
> >> - d.appendVar("PACKAGES", " kernel-devicetree")
> >> + d.appendVar("PACKAGES", " ${KERNEL_PACKAGE_NAME}-devicetree")
> >> }
> >>
> >> normalize_dtb () {
> >> --
> >> 2.6.2
> >>
> >> --
> >> _______________________________________________
> >> Openembedded-core mailing list
> >> Openembedded-core at lists.openembedded.org
> >> http://lists.openembedded.org/mailman/listinfo/openembedded-core
> >>
> >
> >
> >
> > --
> > "Thou shalt not follow the NULL pointer, for chaos and madness await thee
> > at its end"
> >
>
>
>
> --
> "Thou shalt not follow the NULL pointer, for chaos and madness await thee
> at its end"
> --
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core at lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core
More information about the Openembedded-core
mailing list