[OE-core] [PATCH 5/6] kernel-yocto: streamline patch, configuration and audit phases
Bruce Ashfield
bruce.ashfield at windriver.com
Tue Aug 30 13:05:35 UTC 2016
On 2016-08-30 05:05 AM, André Draszik wrote:
> My kmeta stuff is located directly inside two meta-* layers (i.e. not in an
> extra git repo), where the 2nd layer's kernel recipes .bbappends to the 1st
> one with additional kmeta things.
>
> This used to work fine, but since this commit it doesn't anymore as only the
> kmeta stuff from the 2nd layer is being copied into workdir.
>
> Have I been doing something that wasn't guaranteed / supposed to work in the
> first place? Or should that still work? Is it worth fixing or should I
> rethink my approach?
That was complexity that I had tried to remove, but thinking on it a
bit, I think I can restore that without adding much complexity.
I'm just finishing work on the 4.8 kernel today, and can look at this
tomorrow.
Do you have any public layers that I can reference to pull together a
test case ?
Bruce
>
>
> Cheers,
> Andre'
>
>
> On Mo, 2016-08-15 at 14:26 -0400, Bruce Ashfield wrote:
>> We've been running with a set of kern-tools that were designed to work
>> with build systems that knew nothing about git, trees, commits, etc.
>>
>> As such, there's been a set of shims/wrappers in place to work with
>> within bitbake/oe-core. These were the *me scripts: createme, updateme,
>> patchme and configme.
>>
>> With this commit, we strip that legacy code and use the tools directly.
>> This means less complexity, fewer corner cases .. and no surprises
>> when the tools are arunning. As another benefit, the tools consume
>> much less time during a typical build and have no noticeable impact
>> on the overall build time.
>>
>> Existing .scc files, features, and processing are not impacted as
>> these tools are compatible with existing feature descriptions and
>> kerne configuration fragments.
>>
>> The audit of kernel configuration fragments is now detached
>> from the linux-yocto build structure and process. This means that
>> they can eventually be tweaked to offer kernel audit to any type of
>> kernel build and configuration process.
>>
>> Additionally, the kernel symbol audit phase can now resolve symbol
>> dependencies and offer guidance when a symbol is missing:
>>
>> WARNING: linux-yocto-4.4.15+gitAUTOINC+b030d96c7b_f5e2c49d58-r0
>> do_kernel_configcheck: [kernel config]: specified values did not make it
>> into the kernel's final configuration:
>>
>> ---------- CONFIG_BT_6LOWPAN -----------------
>> Config: CONFIG_BT_6LOWPAN
>> From: /home/bruce/poky/build/tmp/work-shared/qemux86-64/kernel-
>> source/.kernel-meta/configs/standard/features/bluetooth/bluetooth.cfg
>> Requested value: CONFIG_BT_6LOWPAN=y
>> Actual value:
>>
>> Config 'BT_6LOWPAN' has the following conditionals:
>> BT_LE && 6LOWPAN (value: "n")
>> Dependency values are:
>> BT_LE [y] 6LOWPAN [n]
>>
>> Signed-off-by: Bruce Ashfield <bruce.ashfield at windriver.com>
>> ---
>> meta/classes/kernel-yocto.bbclass | 143 ++++++++------
>> -------
>> .../kern-tools/kern-tools-native_git.bb | 4 +-
>> 2 files changed, 55 insertions(+), 92 deletions(-)
>>
>> diff --git a/meta/classes/kernel-yocto.bbclass b/meta/classes/kernel-
>> yocto.bbclass
>> index a9d4205..8650e55 100644
>> --- a/meta/classes/kernel-yocto.bbclass
>> +++ b/meta/classes/kernel-yocto.bbclass
>> @@ -119,77 +119,42 @@ do_kernel_metadata() {
>> patches="${@" ".join(find_patches(d))}"
>> feat_dirs="${@" ".join(find_kernel_feature_dirs(d))}"
>>
>> - # add any explicitly referenced features onto the end of the
>> feature
>> - # list that is passed to the kernel build scripts.
>> - if [ -n "${KERNEL_FEATURES}" ]; then
>> - for feat in ${KERNEL_FEATURES}; do
>> - addon_features="$addon_features --feature $feat"
>> - done
>> - fi
>> -
>> # check for feature directories/repos/branches that were part of
>> the
>> # SRC_URI. If they were supplied, we convert them into include
>> directives
>> # for the update part of the process
>> - if [ -n "${feat_dirs}" ]; then
>> - for f in ${feat_dirs}; do
>> + for f in ${feat_dirs}; do
>> if [ -d "${WORKDIR}/$f/meta" ]; then
>> - includes="$includes -I${WORKDIR}/$f/meta"
>> + includes="$includes -I${WORKDIR}/$f/meta"
>> elif [ -d "${WORKDIR}/$f" ]; then
>> - includes="$includes -I${WORKDIR}/$f"
>> + includes="$includes -I${WORKDIR}/$f"
>> fi
>> - done
>> - fi
>> + done
>> + for s in ${sccs}; do
>> + sdir=$(dirname $s)
>> + includes="$includes -I${sdir}"
>> + done
>>
>> - # updates or generates the target description
>> - updateme ${updateme_flags}
>> -DKDESC=${KMACHINE}:${LINUX_KERNEL_TYPE} \
>> - ${includes} ${addon_features} ${ARCH}
>> ${KMACHINE} ${sccs} ${patches}
>> - if [ $? -ne 0 ]; then
>> - bbfatal_log "Could not update ${machine_branch}"
>> - fi
>> + # expand kernel features into their full path equivalents
>> + bsp_definition=$(spp ${includes} --find -DKMACHINE=${KMACHINE}
>> -DKTYPE=${LINUX_KERNEL_TYPE})
>> + meta_dir=$(kgit --meta)
>> +
>> + # run1: pull all the configuration fragments, no matter where
>> they come from
>> + scc --force -o ${S}/${meta_dir}:cfg,meta ${includes}
>> ${bsp_definition} ${sccs} ${patches} ${KERNEL_FEATURES}
>> +
>> + # run2: only generate patches for elements that have been passed
>> on the SRC_URI
>> + scc --force -o ${S}/${meta_dir}:patch --cmds patch ${includes}
>> ${sccs} ${patches} ${KERNEL_FEATURES}
>> }
>>
>> do_patch() {
>> cd ${S}
>>
>> - # executes and modifies the source tree as required
>> - patchme ${KMACHINE}
>> - if [ $? -ne 0 ]; then
>> - bberror "Could not apply patches for ${KMACHINE}."
>> - bbfatal_log "Patch failures can be resolved in the linux
>> source directory ${S})"
>> - fi
>> -
>> - # check to see if the specified SRCREV is reachable from the
>> final branch.
>> - # if it wasn't something wrong has happened, and we should error.
>> - machine_srcrev="${SRCREV_machine}"
>> - if [ -z "${machine_srcrev}" ]; then
>> - # fallback to SRCREV if a non machine_meta tree is being
>> built
>> - machine_srcrev="${SRCREV}"
>> - # if SRCREV cannot be reached something is wrong.
>> - if [ -z "${machine_srcrev}" ]; then
>> - bbfatal "Neither SRCREV_machine or SRCREV was
>> specified!"
>> - fi
>> - fi
>> -
>> - if [ -n "${KMETA_AUDIT}" ]; then
>> - current_branch=`git rev-parse --abbrev-ref HEAD`
>> - machine_branch="${@ get_machine_branch(d, "${KBRANCH}" )}"
>> - if [ "${current_branch}" != "${machine_branch}" ]; then
>> - bbwarn "After meta data application, the kernel tree
>> branch is ${current_branch}."
>> - bbwarn "The SRC_URI specified branch ${machine_branch}."
>> - bbwarn ""
>> - bbwarn "The branch will be forced to ${machine_branch},
>> but this means the board meta data"
>> - bbwarn "(.scc files) do not match the SRC_URI
>> specification."
>> - bbwarn ""
>> - bbwarn "The meta data and branch ${machine_branch} should
>> be inspected to ensure the proper"
>> - bbwarn "kernel is being built."
>> - git checkout -f ${machine_branch}
>> - fi
>> - fi
>> -
>> - if [ "${machine_srcrev}" != "AUTOINC" ]; then
>> - if ! [ "$(git rev-parse --verify ${machine_srcrev}~0)" =
>> "$(git merge-base ${machine_srcrev} HEAD)" ]; then
>> - bberror "SRCREV ${machine_srcrev} was specified,
>> but is not reachable"
>> - bbfatal "Check the BSP description for incorrect
>> branch selection, or other errors."
>> + meta_dir=$(kgit --meta)
>> + (cd ${meta_dir}; ln -sf patch.queue series)
>> + if [ -f "${meta_dir}/series" ]; then
>> + kgit-s2q --gen -v --patches .kernel-meta/
>> + if [ $? -ne 0 ]; then
>> + bberror "Could not apply patches for
>> ${KMACHINE}."
>> + bbfatal_log "Patch failures can be resolved in
>> the linux source directory ${S})"
>> fi
>> fi
>> }
>> @@ -258,26 +223,37 @@ do_kernel_metadata[depends] = "kern-tools-
>> native:do_populate_sysroot"
>>
>> do_kernel_configme[dirs] += "${S} ${B}"
>> do_kernel_configme() {
>> - bbnote "kernel configme"
>> - export KMETA=${KMETA}
>> + set +e
>>
>> - if [ -n "${KCONFIG_MODE}" ]; then
>> - configmeflags=${KCONFIG_MODE}
>> - else
>> - # If a defconfig was passed, use =n as the baseline,
>> which is achieved
>> - # via --allnoconfig
>> + # translate the kconfig_mode into something that merge_config.sh
>> + # understands
>> + case ${KCONFIG_MODE} in
>> + allnoconfig)
>> + config_flags="-n"
>> + ;;
>> + alldefconfig)
>> + config_flags=""
>> + ;;
>> + *)
>> if [ -f ${WORKDIR}/defconfig ]; then
>> - configmeflags="--allnoconfig"
>> + config_flags="-n"
>> fi
>> - fi
>> + ;;
>> + esac
>>
>> cd ${S}
>> - PATH=${PATH}:${S}/scripts/util
>> - configme ${configmeflags} --reconfig --output ${B}
>> ${LINUX_KERNEL_TYPE} ${KMACHINE}
>> +
>> + meta_dir=$(kgit --meta)
>> + configs="$(scc --configs -o ${meta_dir})"
>> + if [ -z "${configs}" ]; then
>> + bbfatal_log "Could not find configuration queue
>> (${meta_dir}/config.queue)"
>> + fi
>> +
>> + ARCH=${ARCH} merge_config.sh -O ${B} ${config_flags} ${configs} >
>> ${meta_dir}/cfg/merge_config_build.log 2>&1
>> if [ $? -ne 0 ]; then
>> bbfatal_log "Could not configure ${KMACHINE}-
>> ${LINUX_KERNEL_TYPE}"
>> fi
>> -
>> +
>> echo "# Global settings from linux recipe" >> ${B}/.config
>> echo "CONFIG_LOCALVERSION="\"${LINUX_VERSION_EXTENSION}\" >>
>> ${B}/.config
>> }
>> @@ -295,36 +271,23 @@ python do_kernel_configcheck() {
>> kmeta = "." + kmeta
>>
>> pathprefix = "export PATH=%s:%s; " % (d.getVar('PATH', True),
>> "${S}/scripts/util/")
>> - cmd = d.expand("cd ${S}; kconf_check -config %s/meta-series ${S}
>> ${B}" % kmeta)
>> +
>> + cmd = d.expand("scc --configs -o ${S}/.kernel-meta")
>> + ret, configs = oe.utils.getstatusoutput("%s%s" % (pathprefix, cmd))
>> +
>> + cmd = d.expand("cd ${S}; kconf_check --report -o ${S}/%s/cfg/
>> ${B}/.config ${S} %s" % (kmeta,configs))
>> ret, result = oe.utils.getstatusoutput("%s%s" % (pathprefix, cmd))
>>
>> config_check_visibility = int(d.getVar( "KCONF_AUDIT_LEVEL", True )
>> or 0)
>> bsp_check_visibility = int(d.getVar( "KCONF_BSP_AUDIT_LEVEL", True )
>> or 0)
>>
>> # if config check visibility is non-zero, report dropped
>> configuration values
>> - mismatch_file = "${S}/" + kmeta + "/" + "mismatch.cfg"
>> + mismatch_file = d.expand("${S}/%s/cfg/mismatch.txt" % kmeta)
>> if os.path.exists(mismatch_file):
>> if config_check_visibility:
>> with open (mismatch_file, "r") as myfile:
>> results = myfile.read()
>> bb.warn( "[kernel config]: specified values did not make
>> it into the kernel's final configuration:\n\n%s" % results)
>> -
>> - # if config check visibility is level 2 or higher, report non-
>> hardware options
>> - nonhw_file = "${S}/" + kmeta + "/" + "nonhw_report.cfg"
>> - if os.path.exists(nonhw_file):
>> - if config_check_visibility > 1:
>> - with open (nonhw_file, "r") as myfile:
>> - results = myfile.read()
>> - bb.warn( "[kernel config]: BSP specified non-hw
>> configuration:\n\n%s" % results)
>> -
>> - bsp_desc = "${S}/" + kmeta + "/" + "top_tgt"
>> - if os.path.exists(bsp_desc) and bsp_check_visibility > 1:
>> - with open (bsp_desc, "r") as myfile:
>> - bsp_tgt = myfile.read()
>> - m = re.match("^(.*)scratch.obj(.*)$", bsp_tgt)
>> - if not m is None:
>> - bb.warn( "[kernel]: An auto generated BSP description was
>> used, this normally indicates a misconfiguration.\n" +
>> - "Check that your machine (%s) has an associated
>> kernel description." % "${MACHINE}" )
>> }
>>
>> # Ensure that the branches (BSP and meta) are on the locations specified
>> by
>> diff --git a/meta/recipes-kernel/kern-tools/kern-tools-native_git.bb
>> b/meta/recipes-kernel/kern-tools/kern-tools-native_git.bb
>> index c3ded98..6877a0c 100644
>> --- a/meta/recipes-kernel/kern-tools/kern-tools-native_git.bb
>> +++ b/meta/recipes-kernel/kern-tools/kern-tools-native_git.bb
>> @@ -1,10 +1,10 @@
>> SUMMARY = "Tools for managing Yocto Project style branched kernels"
>> LICENSE = "GPLv2"
>> -LIC_FILES_CHKSUM =
>> "file://git/tools/kgit;beginline=5;endline=9;md5=d8d1d729a70cd5f52972f8884
>> b80743d"
>> +LIC_FILES_CHKSUM =
>> "file://git/tools/kgit;beginline=5;endline=9;md5=a6c2fa8aef1bda400e2828845
>> ba0d06c"
>>
>> DEPENDS = "git-native"
>>
>> -SRCREV = "a6a6f6e1e738d2ecd26b675e9bec867105703416"
>> +SRCREV = "4b5de9011b31ff92ba2b6e052ab818b09affd905"
>> PR = "r12"
>> PV = "0.2+git${SRCPV}"
>>
>> --
>> 2.5.0
>>
More information about the Openembedded-core
mailing list