[OE-core] [master][PATCH v3] esdk: Introduce mechanism to keep nativesdk* sstate in esdk
Khem Raj
raj.khem at gmail.com
Wed Sep 25 23:40:24 UTC 2019
actually this is not the culprit but setting SDK_INCLUDE_NATIVESDK =
"1" does help avoid this issue, the problem is in master I dont have
time to bisect it but it worked fine few weeks ago. For now I will
just set SDK_INCLUDE_NATIVESDK and move on
On Wed, Sep 25, 2019 at 3:21 PM Khem Raj <raj.khem at gmail.com> wrote:
>
> I think this is the reason why extensible sdk is not building for me,
> key is I am not setting SDK_INCLUDE_NATIVESDK
>
> The file local.conf.bak that it is not able to find to copy is
> actually inside sdk-ext/image/tmp-renamed-sdk/conf, So I wonder if
> renaming is happening inbetween copying ?
>
> Summary: 1 task failed:
> /mnt/b/yoe/build/tmp/work/qemuriscv64-yoe-linux-musl/yoe-simple-image/1.0-r0/sdk-ext/image/tmp-renamed-sdk/layers/openembedded-core/meta/recipes-core/meta/package-index.bb:do_package_index
> Summary: There was 1 ERROR message shown, returning a non-zero exit code.
> ERROR: yoe-simple-image-1.0-r0 do_populate_sdk_ext: Error executing a
> python function in exec_python_func() autogenerated:
>
> The stack trace of python calls that resulted in this exception/failure was:
> File: 'exec_python_func() autogenerated', lineno: 2, function: <module>
> 0001:
> *** 0002:copy_buildsystem(d)
> 0003:
> File: '/mnt/b/yoe/sources/openembedded-core/meta/classes/populate_sdk_ext.bbclass',
> lineno: 444, function: copy_buildsystem
> 0440: sdk_ext_type = d.getVar('SDK_EXT_TYPE')
> 0441: if (sdk_ext_type != 'minimal' or sdk_include_toolchain
> or derivative) and not sdk_include_nativesdk:
> 0442: # Create the filtered task list used to generate the
> sstate cache shipped with the SDK
> 0443: tasklistfn = d.getVar('WORKDIR') + '/tasklist.txt'
> *** 0444: create_filtered_tasklist(d, baseoutpath, tasklistfn,
> conf_initpath)
> 0445: else:
> 0446: tasklistfn = None
> 0447:
> 0448: if os.path.exists(builddir + '/cache/bb_unihashes.dat'):
> File: '/mnt/b/yoe/sources/openembedded-core/meta/classes/populate_sdk_ext.bbclass',
> lineno: 180, function: create_filtered_tasklist
> 0176: # Clean out residue of running bitbake, which
> check_sstate_task_list()
> 0177: # will effectively do
> 0178: clean_esdk_builddir(d, sdkbasepath)
> 0179: finally:
> *** 0180: os.replace(sdkbasepath + '/conf/local.conf.bak',
> sdkbasepath + '/conf/local.conf')
> 0181:
> 0182:python copy_buildsystem () {
> 0183: import re
> 0184: import shutil
> Exception: FileNotFoundError: [Errno 2] No such file or directory:
> '/mnt/b/yoe/build/tmp/work/qemuriscv64-yoe-linux-musl/yoe-simple-image/1.0-r0/sdk-ext/image//opt/yoe/3.0/conf/local.conf.bak'
> -> '/mnt/b/yoe/build/tmp/work/qemuriscv64-yoe-linux-musl/yoe-simple-image/1.0-r
> 0/sdk-ext/image//opt/yoe/3.0/conf/local.conf'
>
> ERROR: Logfile of failure stored in:
> /mnt/b/yoe/build/tmp/work/qemuriscv64-yoe-linux-musl/yoe-simple-image/1.0-r0/temp/log.do_populate_sdk_ext.1484013
>
> On Wed, Sep 18, 2019 at 10:43 AM Jaewon Lee <jaewon.lee at xilinx.com> wrote:
> >
> > When doing a devtool build-sdk from within an esdk all nativesdk
> > components would be rebuilt. This patch introduces SDK_INCLUDE_NATIVESDK
> > flag to toggle the inclusion of nativesdk packages when creating the
> > esdk sstate
> >
> > Currently locked-sigs.inc is generated during do_sdk_depends which
> > doesn't pull in nativesdk packages. Generating another locked-sigs.inc
> > in do_populate_sdk_ext and pruning it to only nativesdk* packages by
> > using a modified version of the already existing function
> > prune_locked_sigs and merging it with the current locked-sigs.inc
> > Also adding SDK_INCLUDE_NATIVESDK tasklistfn to the logic surrounding
> > setting tasklist file to not prune esdk sstate during creation
> >
> > Fixes [YOCTO #13261]
> >
> > Signed-off-by: Jaewon Lee <jaewon.lee at xilinx.com>
> > ---
> > changes in v2:
> > change to commit message to include reason
> > got rid of some tabs
> > rebased to apply on master
> > changes in v3:
> > fix patchwork failure for format of short commit message
> > ---
> > meta/classes/populate_sdk_ext.bbclass | 28 +++++++++++++++++++++++++++-
> > meta/lib/oe/copy_buildsystem.py | 8 ++++++--
> > 2 files changed, 33 insertions(+), 3 deletions(-)
> >
> > diff --git a/meta/classes/populate_sdk_ext.bbclass b/meta/classes/populate_sdk_ext.bbclass
> > index 800e117..086f55d 100644
> > --- a/meta/classes/populate_sdk_ext.bbclass
> > +++ b/meta/classes/populate_sdk_ext.bbclass
> > @@ -20,6 +20,7 @@ SDK_EXT_task-populate-sdk-ext = "-ext"
> > SDK_EXT_TYPE ?= "full"
> > SDK_INCLUDE_PKGDATA ?= "0"
> > SDK_INCLUDE_TOOLCHAIN ?= "${@'1' if d.getVar('SDK_EXT_TYPE') == 'full' else '0'}"
> > +SDK_INCLUDE_NATIVESDK ?= "0"
> >
> > SDK_RECRDEP_TASKS ?= ""
> >
> > @@ -401,9 +402,27 @@ python copy_buildsystem () {
> > excluded_targets = get_sdk_install_targets(d, images_only=True)
> > sigfile = d.getVar('WORKDIR') + '/locked-sigs.inc'
> > lockedsigs_pruned = baseoutpath + '/conf/locked-sigs.inc'
> > + #nativesdk-only sigfile to merge into locked-sigs.inc
> > + sdk_include_nativesdk = (d.getVar("SDK_INCLUDE_NATIVESDK") == '1')
> > + nativesigfile = d.getVar('WORKDIR') + '/locked-sigs_nativesdk.inc'
> > + nativesigfile_pruned = d.getVar('WORKDIR') + '/locked-sigs_nativesdk_pruned.inc'
> > +
> > + if sdk_include_nativesdk:
> > + oe.copy_buildsystem.prune_lockedsigs([],
> > + excluded_targets.split(),
> > + nativesigfile,
> > + True,
> > + nativesigfile_pruned)
> > +
> > + oe.copy_buildsystem.merge_lockedsigs([],
> > + sigfile,
> > + nativesigfile_pruned,
> > + sigfile)
> > +
> > oe.copy_buildsystem.prune_lockedsigs([],
> > excluded_targets.split(),
> > sigfile,
> > + False,
> > lockedsigs_pruned)
> >
> > sstate_out = baseoutpath + '/sstate-cache'
> > @@ -414,7 +433,7 @@ python copy_buildsystem () {
> >
> > sdk_include_toolchain = (d.getVar('SDK_INCLUDE_TOOLCHAIN') == '1')
> > sdk_ext_type = d.getVar('SDK_EXT_TYPE')
> > - if sdk_ext_type != 'minimal' or sdk_include_toolchain or derivative:
> > + if (sdk_ext_type != 'minimal' or sdk_include_toolchain or derivative) and not sdk_include_nativesdk:
> > # Create the filtered task list used to generate the sstate cache shipped with the SDK
> > tasklistfn = d.getVar('WORKDIR') + '/tasklist.txt'
> > create_filtered_tasklist(d, baseoutpath, tasklistfn, conf_initpath)
> > @@ -657,9 +676,16 @@ fakeroot python do_populate_sdk_ext() {
> > d.setVar('SDKDEPLOYDIR', '${SDKEXTDEPLOYDIR}')
> > # ESDKs have a libc from the buildtools so ensure we don't ship linguas twice
> > d.delVar('SDKIMAGE_LINGUAS')
> > + if d.getVar("SDK_INCLUDE_NATIVESDK") == '1':
> > + generate_nativesdk_lockedsigs(d)
> > populate_sdk_common(d)
> > }
> >
> > +def generate_nativesdk_lockedsigs(d):
> > + import oe.copy_buildsystem
> > + sigfile = d.getVar('WORKDIR') + '/locked-sigs_nativesdk.inc'
> > + oe.copy_buildsystem.generate_locked_sigs(sigfile, d)
> > +
> > def get_ext_sdk_depends(d):
> > # Note: the deps varflag is a list not a string, so we need to specify expand=False
> > deps = d.getVarFlag('do_image_complete', 'deps', False)
> > diff --git a/meta/lib/oe/copy_buildsystem.py b/meta/lib/oe/copy_buildsystem.py
> > index cb663b2..31a84f5 100644
> > --- a/meta/lib/oe/copy_buildsystem.py
> > +++ b/meta/lib/oe/copy_buildsystem.py
> > @@ -177,7 +177,7 @@ def generate_locked_sigs(sigfile, d):
> > tasks = ['%s:%s' % (v[2], v[1]) for v in depd.values()]
> > bb.parse.siggen.dump_lockedsigs(sigfile, tasks)
> >
> > -def prune_lockedsigs(excluded_tasks, excluded_targets, lockedsigs, pruned_output):
> > +def prune_lockedsigs(excluded_tasks, excluded_targets, lockedsigs, onlynative, pruned_output):
> > with open(lockedsigs, 'r') as infile:
> > bb.utils.mkdirhier(os.path.dirname(pruned_output))
> > with open(pruned_output, 'w') as f:
> > @@ -187,7 +187,11 @@ def prune_lockedsigs(excluded_tasks, excluded_targets, lockedsigs, pruned_output
> > if line.endswith('\\\n'):
> > splitval = line.strip().split(':')
> > if not splitval[1] in excluded_tasks and not splitval[0] in excluded_targets:
> > - f.write(line)
> > + if onlynative:
> > + if 'nativesdk' in splitval[0]:
> > + f.write(line)
> > + else:
> > + f.write(line)
> > else:
> > f.write(line)
> > invalue = False
> > --
> > 2.7.4
> >
> > --
> > _______________________________________________
> > Openembedded-core mailing list
> > Openembedded-core at lists.openembedded.org
> > http://lists.openembedded.org/mailman/listinfo/openembedded-core
More information about the Openembedded-core
mailing list