[OE-core] [PATCH] kernel.bbclass, image.bbclass: Implement kernel INITRAMFS dependency and bundling
Andrea Adami
andrea.adami at gmail.com
Tue Aug 20 16:41:19 UTC 2013
On Tue, Aug 20, 2013 at 6:28 PM, Bruce Ashfield
<bruce.ashfield at windriver.com> wrote:
> On 13-08-20 11:03 AM, Andrea Adami wrote:
>>
>> On Tue, Aug 20, 2013 at 3:51 PM, Bruce Ashfield
>> <bruce.ashfield at windriver.com> wrote:
>>>
>>> On 13-08-19 03:11 PM, Jason Wessel wrote:
>>>>
>>>>
>>>> This patch aims to fix the following two cases for the INITRAMFS
>>>> generation.
>>>> 1) Allow an image recipe to specify a paired INITRAMFS recipe such
>>>> as core-image-minimal-initramfs. This allows building a base
>>>> image which always generates the needed initramfs image in one
>>>> step
>>>> 2) Allow building a single binary which contains a kernel and
>>>> the initramfs.
>>>>
>>>> A key requirement of the initramfs is to be able to add kernel
>>>> modules. The current implementation of the INITRAMFS_IMAGE variable
>>>> has a circular dependency when using kernel modules in the initramfs
>>>> image.bb file that is caused by kernel.bbclass trying to build the
>>>> initramfs before the kernel's do_install rule.
>>>>
>>>> The solution for this problem is to have the kernel's
>>>> do_bundle_initramfs_image task depend on the do_rootfs from the
>>>> INITRAMFS_IMAGE and not some intermediate point. The image.bbclass
>>>> will also sets up dependencies to make the initramfs creation task run
>>>> last.
>>>>
>>>> The code to bundle the kernel and initramfs together has been added.
>>>> At a high level, all it is doing is invoking a second compilation of
>>>> the kernel but changing the value of CONFIG_INITRAMFS_SOURCE to point
>>>> to the generated initramfs from the image recipe.
>>>
>>>
>>>
>>> In case it wasn't obvious with Jason cc'ing me, I've seen the patch
>>> before, and I'm good with the change.
>>>
>>> So definitely: Acked-by: Bruce Ashfield <bruce.ashfield at windriver.com>
>>>
>>> I'm adding Andrea to the cc list as well, since we've been talking
>>> about initramfs creation for years now, and I'd want his opinion on
>>> whether this looks good, and would work for meta-handheld.
>>>
>>> Cheers,
>>>
>>> Bruce
>>>
>>>
>>>>
>>>> Signed-off-by: Jason Wessel <jason.wessel at windriver.com>
>>>> ---
>>>> meta/classes/image.bbclass | 12 ++++++
>>>> meta/classes/kernel.bbclass | 96
>>>> +++++++++++++++++++++++++++++++++++++------
>>>> meta/conf/local.conf.sample | 20 +++++++++
>>>> 3 files changed, 116 insertions(+), 12 deletions(-)
>>>>
>>>> diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
>>>> index 116bd22..78c25db 100644
>>>> --- a/meta/classes/image.bbclass
>>>> +++ b/meta/classes/image.bbclass
>>>> @@ -140,6 +140,10 @@ python () {
>>>> d.setVar('MULTILIB_VENDORS', ml_vendor_list)
>>>>
>>>> check_image_features(d)
>>>> + initramfs_image = d.getVar('INITRAMFS_IMAGE', True) or ""
>>>> + if initramfs_image != "":
>>>> + d.appendVarFlag('do_build', 'depends', "
>>>> %s:do_bundle_initramfs"
>>>> % d.getVar('PN', True))
>>>> + d.appendVarFlag('do_bundle_initramfs', 'depends', "
>>>> %s:do_rootfs"
>>>> % initramfs_image)
>>>> }
>>>>
>>>> #
>>>> @@ -614,3 +618,11 @@ do_package_write_deb[noexec] = "1"
>>>> do_package_write_rpm[noexec] = "1"
>>>>
>>>> addtask rootfs before do_build
>>>> +# Allow the kernel to be repacked with the initramfs and boot image
>>>> file
>>>> as a single file
>>>> +do_bundle_initramfs[depends] += "virtual/kernel:do_bundle_initramfs"
>>>> +do_bundle_initramfs[nostamp] = "1"
>>>> +do_bundle_initramfs[noexec] = "1"
>>>> +do_bundle_initramfs () {
>>>> + :
>>>> +}
>>>> +addtask bundle_initramfs after do_rootfs
>>>> diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
>>>> index e039dfc..8cf66ce 100644
>>>> --- a/meta/classes/kernel.bbclass
>>>> +++ b/meta/classes/kernel.bbclass
>>>> @@ -9,6 +9,7 @@ INHIBIT_DEFAULT_DEPS = "1"
>>>> KERNEL_IMAGETYPE ?= "zImage"
>>>> INITRAMFS_IMAGE ?= ""
>>>> INITRAMFS_TASK ?= ""
>>>> +INITRAMFS_IMAGE_BUNDLE ?= ""
>>>>
>>>> python __anonymous () {
>>>> kerneltype = d.getVar('KERNEL_IMAGETYPE', True) or ''
>>>> @@ -19,7 +20,15 @@ python __anonymous () {
>>>>
>>>> image = d.getVar('INITRAMFS_IMAGE', True)
>>>> if image:
>>>> - d.setVar('INITRAMFS_TASK', '${INITRAMFS_IMAGE}:do_rootfs')
>>>> + d.appendVarFlag('do_bundle_initramfs', 'depends', '
>>>> ${INITRAMFS_IMAGE}:do_rootfs')
>>>> +
>>>> + # NOTE: setting INITRAMFS_TASK is for backward compatibility
>>>> + # The preferred method is to set INITRAMFS_IMAGE, because
>>>> + # this INITRAMFS_TASK has circular dependency problems
>>>> + # if the initramfs requires kernel modules
>>>> + image_task = d.getVar('INITRAMFS_TASK', True)
>>>> + if image_task:
>>>> + d.appendVarFlag('do_configure', 'depends', '
>>>> ${INITRAMFS_TASK}')
>>>> }
>>>>
>>>> inherit kernel-arch deploy
>>>> @@ -72,9 +81,82 @@ KERNEL_SRC_PATH = "/usr/src/kernel"
>>>>
>>>> KERNEL_IMAGETYPE_FOR_MAKE = "${@(lambda s: s[:-3] if s[-3:] == ".gz"
>>>> else s)(d.getVar('KERNEL_IMAGETYPE', True))}"
>>>>
>>>> +copy_initramfs() {
>>>> + echo "Copying initramfs into ./usr ..."
>>>> + # Find and use the first initramfs image archive type we find
>>>> + rm -f ${B}/usr/${INITRAMFS_IMAGE}-${MACHINE}.cpio
>>>> + for img in cpio.gz cpio.lzo cpio.lzma cpio.xz; do
>>>> + if [ -e
>>>> "${DEPLOY_DIR_IMAGE}/${INITRAMFS_IMAGE}-${MACHINE}.$img" ]; then
>>>> + cp
>>>> ${DEPLOY_DIR_IMAGE}/${INITRAMFS_IMAGE}-${MACHINE}.$img ${B}/usr/.
>>>> + case $img in
>>>> + *gz)
>>>> + echo "gzip decompressing image"
>>>> + gunzip -f
>>>> ${B}/usr/${INITRAMFS_IMAGE}-${MACHINE}.$img
>>>> + break
>>>> + ;;
>>>> + *lzo)
>>>> + echo "lzo decompressing image"
>>>> + lzop -df
>>>> ${B}/usr/${INITRAMFS_IMAGE}-${MACHINE}.$img
>>>> + break
>>>> + ;;
>>>> + *lzma)
>>>> + echo "lzma decompressing image"
>>>> + lzmash -df
>>>> ${B}/usr/${INITRAMFS_IMAGE}-${MACHINE}.$img
>>>> + break
>>>> + ;;
>>>> + *xz)
>>>> + echo "xz decompressing image"
>>>> + xz -df
>>>> ${B}/usr/${INITRAMFS_IMAGE}-${MACHINE}.$img
>>>> + break
>>>> + ;;
>>>> + esac
>>>> + fi
>>>> + done
>>>> + echo "Finished copy of initramfs into ./usr"
>>>> +}
>>>> +
>>>> +INITRAMFS_BASE_NAME =
>>>> "${KERNEL_IMAGETYPE}-initramfs-${PV}-${PR}-${MACHINE}-${DATETIME}"
>>>> +INITRAMFS_BASE_NAME[vardepsexclude] = "DATETIME"
>>>> +do_bundle_initramfs () {
>>>> + if [ ! -z "${INITRAMFS_IMAGE}" -a x"${INITRAMFS_IMAGE_BUNDLE}" =
>>>> x1 ]; then
>>>> + echo "Creating a kernel image with a bundled
>>>> initramfs..."
>>>> + copy_initramfs
>>>> + if [ -e ${KERNEL_OUTPUT} ] ; then
>>>> + mv -f ${KERNEL_OUTPUT} ${KERNEL_OUTPUT}.bak
>>>> + fi
>>>> +
>>>>
>>>> use_alternate_initrd=CONFIG_INITRAMFS_SOURCE=${B}/usr/${INITRAMFS_IMAGE}-${MACHINE}.cpio
>>>> + kernel_do_compile
>>>> + mv -f ${KERNEL_OUTPUT} ${KERNEL_OUTPUT}.initramfs
>>>> + mv -f ${KERNEL_OUTPUT}.bak ${KERNEL_OUTPUT}
>>>> + # Update install area
>>>> + echo "There is kernel image bundled with initramfs:
>>>> ${B}/${KERNEL_OUTPUT}.initramfs"
>>>> + install -m 0644 ${B}/${KERNEL_OUTPUT}.initramfs
>>>> ${D}/boot/${KERNEL_IMAGETYPE}-initramfs-${MACHINE}.bin
>>>> + echo "${B}/${KERNEL_OUTPUT}.initramfs"
>>>> + cd ${B}
>>>> + # Update deploy directory
>>>> + if [ -e "${KERNEL_OUTPUT}.initramfs" ]; then
>>>> + echo "Copying deploy kernel-initramfs image and
>>>> setting up links..."
>>>> + initramfs_base_name=${INITRAMFS_BASE_NAME}
>>>> +
>>>> initramfs_symlink_name=${KERNEL_IMAGETYPE}-initramfs-${MACHINE}
>>>> + install -m 0644 ${KERNEL_OUTPUT}.initramfs
>>>> ${DEPLOY_DIR_IMAGE}/${initramfs_base_name}.bin
>>>> + cd ${DEPLOY_DIR_IMAGE}
>>>> + ln -sf ${initramfs_base_name}.bin
>>>> ${initramfs_symlink_name}.bin
>>>> + fi
>>>> + fi
>>>> +}
>>>> +do_bundle_initramfs[nostamp] = "1"
>>>> +addtask bundle_initramfs after do_compile
>>>> +
>>>> kernel_do_compile() {
>>>> unset CFLAGS CPPFLAGS CXXFLAGS LDFLAGS MACHINE
>>>> - oe_runmake ${KERNEL_IMAGETYPE_FOR_MAKE} ${KERNEL_ALT_IMAGETYPE}
>>>> CC="${KERNEL_CC}" LD="${KERNEL_LD}" ${KERNEL_EXTRA_ARGS}
>>>> + # The $use_alternate_initrd is only set from
>>>> + # do_bundle_initramfs() This variable is specifically for the
>>>> + # case where we are making a second pass at the kernel
>>>> + # compilation and we want to force the kernel build to use a
>>>> + # different initramfs image. The way to do that in the kernel
>>>> + # is to specify:
>>>> + # make ...args...
>>>> CONFIG_INITRAMFS_SOURCE=some_other_initramfs.cpio
>>>> + oe_runmake ${KERNEL_IMAGETYPE_FOR_MAKE} ${KERNEL_ALT_IMAGETYPE}
>>>> CC="${KERNEL_CC}" LD="${KERNEL_LD}" ${KERNEL_EXTRA_ARGS}
>>>> $use_alternate_initrd
>>>> if test "${KERNEL_IMAGETYPE_FOR_MAKE}.gz" =
>>>> "${KERNEL_IMAGETYPE}";
>>>> then
>>>> gzip -9c < "${KERNEL_IMAGETYPE_FOR_MAKE}" >
>>>> "${KERNEL_OUTPUT}"
>>>> fi
>>>> @@ -219,18 +301,8 @@ kernel_do_configure() {
>>>> cp "${WORKDIR}/defconfig" "${B}/.config"
>>>> fi
>>>> yes '' | oe_runmake oldconfig
>>>> -
>>>> - if [ ! -z "${INITRAMFS_IMAGE}" ]; then
>>>> - for img in cpio.gz cpio.lzo cpio.lzma cpio.xz; do
>>>> - if [ -e
>>>> "${DEPLOY_DIR_IMAGE}/${INITRAMFS_IMAGE}-${MACHINE}.$img" ]; then
>>>> - cp
>>>> "${DEPLOY_DIR_IMAGE}/${INITRAMFS_IMAGE}-${MACHINE}.$img" initramfs.$img
>>>> - fi
>>>> - done
>>>> - fi
>>>> }
>>>>
>>>> -do_configure[depends] += "${INITRAMFS_TASK}"
>>>> -
>>>> do_savedefconfig() {
>>>> oe_runmake savedefconfig
>>>> }
>>>> diff --git a/meta/conf/local.conf.sample b/meta/conf/local.conf.sample
>>>> index 2b078d0..17733ab 100644
>>>> --- a/meta/conf/local.conf.sample
>>>> +++ b/meta/conf/local.conf.sample
>>>> @@ -146,6 +146,26 @@ EXTRA_IMAGE_FEATURES = "debug-tweaks"
>>>> USER_CLASSES ?= "buildstats image-mklibs image-prelink"
>>>>
>>>> #
>>>> +# Kernel image features
>>>> +#
>>>> +# The INITRAMFS_IMAGE image variable will cause an additional recipe to
>>>> +# be built as a dependency to the what ever rootfs recipe you might be
>>>> +# using such as core-image-sato. The initramfs might be needed for
>>>> +# the initial boot of of the target system such as to load kernel
>>>> +# modules prior to mounting the root file system.
>>>> +#
>>>> +# INITRAMFS_IMAGE_BUNDLE variable controls if the image recipe
>>>> +# specified by the INITRAMFS_IMAGE will be run through an extra pass
>>>> +# through the kernel compilation in order to build a single binary
>>>> +# which contains both the kernel image and the initramfs. The
>>>> +# combined binary will be deposited into the tmp/deploy directory.
>>>> +# NOTE: You can set INITRAMFS_IMAGE in an image recipe, but
>>>> +# INITRAMFS_IMAGE_BUNDLE can only be set in a conf file.
>>>> +#
>>>> +#INITRAMFS_IMAGE = "core-image-minimal-initramfs"
>>>> +#INITRAMFS_IMAGE_BUNDLE = "1"
>>>> +
>>>> +#
>>>> # Runtime testing of images
>>>> #
>>>> # The build system can test booting virtual machine images under qemu
>>>> (an emulator)
>>>>
>>>
>>
>> I plan to test this as soon as possible.
>>
>> Though, it seems to me a bit too special-cased: you only change
>> CONFIG_INITRAMFS_SOURCE=some_other_initramfs.cpio.
>> I mean, our necessity is to have a *different* second kernel embedding
>> the initramfs and in fact we have linux-yocto as standard kernel and a
>> linux-yocto-tiny derived special-purpose kernel+cpio.
>
>
> This is exactly what I was hoping to hear as feedback, and yes,
> this solution does stop short of the kexec solution or a two
> kernel variant. It does allow you to have a kernel module for a
> driver in your initramfs, and a unique filesystem for that
> initramfs.
>
> Your document/steps are one of the cases that I had in mind, it
> would be nice to make it simpler as well .. if possible.
>
>
>>
>> (in case you missed it:
>> http://kexecboot.org/documentation/crosscompiling/oe-yocto )
>>
>> About building all in one-pass, well, we were there and we did step back.
>> It was preferred to build only the 'production' kernel when building
>> images and to build separately the special kernel+cpio.
>> The logic was to create a dummy recipe and let the option to build it
>> automatically i.e. adding it as # EXTRA_IMAGEDEPENDS += "dummy" in
>> your machine.conf or another conf file
>
>
> Remind me on this part. When you assemble and include that second
> kernel into the production kernel's initramfs, how are the build
> artifacts shared ? i.e. it's just a second build with a shared
> deploy directory ? (your my-installer.bb recipe) ?
>
Yes, exactly.
In the specific case you can manually do 'bitbake zaurus-installer' or
just uncomment the
following line in zaurus.inc (belongs to the machine config file).
# If set here, each image will also build linux-yocto-tiny-kexecboot
and the updater
# EXTRA_IMAGEDEPENDS += "zaurus-installer"
This will build all the needed parts.
Regards
Andrea
> Cheers,
>
> Bruce
>
>
>>
>> I'll test soon anyway, thx.
>>
>> Cheers
>>
>> Andrea
>>
>
More information about the Openembedded-core
mailing list