[OE-core] core-image-tiny-initramfs failing to parse because of WIC (WKS_FULL_PATH, )
Martin Jansa
martin.jansa at gmail.com
Sat Feb 18 00:16:00 UTC 2017
On Wed, Feb 15, 2017 at 10:58:49AM +0200, Ed Bartosh wrote:
> Hi Martin,
>
> Sorry for the delay. I missedd your message somehow.
>
> I can't reproduce this. Tried with latest poky and latest oe-core +
> bitbake. I built core-image-tiny-initramfs from scratch and bitbake -e
> busybox works for me just fine.
>
> Can you provide more details about your setup, please?
Hi Ed,
I was also having troubles reproducing it in different setup as well, until I've
found out that my local.conf had this line:
IMAGE_FSTYPES_qemux86 = "vmdk"
which prevented ever using wic (USING_WIC variable was always empty).
Now with this line removed I can share the complete exception reproduced
with just oe-core layer included:
$ bitbake -e core-image-tiny-initramfs | tee env.core-image-tiny-initramfs5
NOTE: Started PRServer with DBfile: /OE/build/oe-core/cache/prserv.sqlite3, IP: 127.0.0.1, PORT: 38869, PID: 21506
Parsing recipes...ERROR: ExpansionError during parsing /OE/build/oe-core/openembedded-core/meta/recipes-core/images/core-image-tiny-initramfs.bb
Traceback (most recent call last):
File "/OE/build/oe-core/bitbake/lib/bb/data_smart.py", line 412, in DataSmart.expandWithRefs(s='\tout="${IMGDEPLOYDIR}/${IMAGE_NAME}"\n\twks="${WKS_FULL_PATH}"\n\tif [ -z "$wks" ]; then\n\t\tbbfatal "No kickstart files from WKS_FILES were found: ${WKS_FILES}. Please set WKS_FILE or WKS_FILES appropriately."\n\tfi\n\n\tBUILDDIR="${TOPDIR}" wic create "$wks" --vars "${STAGING_DIR}/${MACHINE}/imgdata/" -e "${IMAGE_BASENAME}" -o "$out/" ${WIC_CREATE_EXTRA_ARGS}\n\tmv "$out/$(basename "${wks%.wks}")"*.direct "$out${IMAGE_NAME_SUFFIX}.wic"\n\trm -rf "$out/"\n', varname='IMAGE_CMD'):
try:
> s = __expand_var_regexp__.sub(varparse.var_sub, s)
try:
File "/OE/build/oe-core/bitbake/lib/bb/data_smart.py", line 111, in VariableParse.var_sub(match=<_sre.SRE_Match object; span=(43, 59), match='${WKS_FULL_PATH}'>):
else:
> var = self.d.getVarFlag(key, "_content")
self.references.add(key)
File "/OE/build/oe-core/bitbake/lib/bb/data_smart.py", line 773, in DataSmart.getVarFlag(var='WKS_FULL_PATH', flag='_content', expand=True, noweakdefault=False, parsing=False):
cachename = var + "[" + flag + "]"
> value = self.expand(value, cachename)
File "/OE/build/oe-core/bitbake/lib/bb/data_smart.py", line 436, in DataSmart.expand(s="${@wks_search('${WKS_FILES}'.split(), '${WKS_SEARCH_PATH}') or ''}", varname='WKS_FULL_PATH'):
def expand(self, s, varname = None):
> return self.expandWithRefs(s, varname).value
File "/OE/build/oe-core/bitbake/lib/bb/data_smart.py", line 426, in DataSmart.expandWithRefs(s="${@wks_search('directdisk.wks core-image-tiny-initramfs.wks'.split(), '/OE/build/oe-core/openembedded-core/meta/recipes-core/images:${@':'.join('%s/wic' % p for p in '/OE/build/oe-core:${TMPDIR}:/OE/build/oe-core/openembedded-core/meta'.split(':'))}:${@':'.join('%s/scripts/lib/wic/canned-wks' % l for l in '/OE/build/oe-core:${TMPDIR}:/OE/build/oe-core/openembedded-core/meta:/OE/build/oe-core/openembedded-core'.split(':'))}') or ''}", varname='WKS_FULL_PATH'):
except Exception as exc:
> raise ExpansionError(varname, s, exc) from exc
bb.data_smart.ExpansionError: Failure expanding variable WKS_FULL_PATH, expression was ${@wks_search('directdisk.wks core-image-tiny-initramfs.wks'.split(), '/OE/build/oe-core/openembedded-core/meta/recipes-core/images:${@':'.join('%s/wic' % p for p in '/OE/build/oe-core:${TMPDIR}:/OE/build/oe-core/openembedded-core/meta'.split(':'))}:${@':'.join('%s/scripts/lib/wic/canned-wks' % l for l in '/OE/build/oe-core:${TMPDIR}:/OE/build/oe-core/openembedded-core/meta:/OE/build/oe-core/openembedded-core'.split(':'))}') or ''} which triggered exception SyntaxError: invalid syntax (WKS_FULL_PATH, line 1)
Summary: There was 1 ERROR message shown, returning a non-zero exit code.
The strange part of this exception is the unexpanded ${TMPDIR} in the WKS_SEARCH_PATH value,
that lead me to following 2 lines in conf/bblayers.conf:
TMPDIR := "/OE/build/oe-core"
BBPATH = "${TOPDIR}:${TMPDIR}"
I'm not sure why it appears unexpanded in WKS_SEARCH_PATH, I see you do some magick with all these
WKS_* variables, especially in that anonymous python, but the exception is actually from
expanding IMAGE_CMD_wic () function and it goes away if I remove following line from it:
wks="${WKS_FULL_PATH}"
And it also goes away if I use immediate expansion when setting that BBPATH in bblayers.conf.
Interesting variables from bitbake -e with immediate expansion:
# $BBPATH [4 operations]
# immediate /OE/build/oe-core/conf/bblayers.conf:6
# "${TOPDIR}:${TMPDIR}"
# postdot /OE/build/oe-core/openembedded-core/meta/conf/layer.conf:2
# ":${LAYERDIR}"
# set data_smart.py:911 [expandVarref]
# "/OE/build/oe-core:/OE/build/oe-core:/OE/build/oe-core/openembedded-core/meta"
# set /OE/build/oe-core/openembedded-core/meta/conf/documentation.conf:94
# [doc] "Used by BitBake to locate .bbclass and configuration files. This variable is analogous to the PATH variable."
# pre-expansion value:
# "/OE/build/oe-core:/OE/build/oe-core:/OE/build/oe-core/openembedded-core/meta"
BBPATH="/OE/build/oe-core:/OE/build/oe-core:/OE/build/oe-core/openembedded-core/meta"
# $WKS_SEARCH_PATH
# set? /OE/build/oe-core/openembedded-core/meta/classes/image_types_wic.bbclass:10
# "${THISDIR}:${@':'.join('%s/wic' % p for p in '${BBPATH}'.split(':'))}:${@':'.join('%s/scripts/lib/wic/canned-wks' % l for l in '${BBPATH}:${COREBASE}'.split(':'))}"
WKS_SEARCH_PATH="/OE/build/oe-core/openembedded-core/meta/recipes-core/images:/OE/build/oe-core/wic:/OE/build/oe-core/wic:/OE/build/oe-core/openembedded-core/meta/wic:/OE/build/oe-core/scripts/lib/wic/canned-wks:/OE/build/oe-core/scripts/lib/wic/canned-wks:/OE/build/oe-core/openembedded-core/meta/scripts/lib/wic/canned-wks:/OE/build/oe-core/openembedded-core/scripts/lib/wic/canned-wks"
# $WKS_FULL_PATH
# set /OE/build/oe-core/openembedded-core/meta/classes/image_types_wic.bbclass:11
# "${@wks_search('${WKS_FILES}'.split(), '${WKS_SEARCH_PATH}') or ''}"
WKS_FULL_PATH="/OE/build/oe-core/openembedded-core/scripts/lib/wic/canned-wks/directdisk.wks"
And the same without immediate expansion, but with 'wks="${WKS_FULL_PATH}"' line removed so that it doesn't fail to parse:
# $BBPATH [4 operations]
# set /OE/build/oe-core/conf/bblayers.conf:6
# "${TOPDIR}:${TMPDIR}"
# postdot /OE/build/oe-core/openembedded-core/meta/conf/layer.conf:2
# ":${LAYERDIR}"
# set data_smart.py:911 [expandVarref]
# "${TOPDIR}:${TMPDIR}:/OE/build/oe-core/openembedded-core/meta"
# set /OE/build/oe-core/openembedded-core/meta/conf/documentation.conf:94
# [doc] "Used by BitBake to locate .bbclass and configuration files. This variable is analogous to the PATH variable."
# pre-expansion value:
# "${TOPDIR}:${TMPDIR}:/OE/build/oe-core/openembedded-core/meta"
BBPATH="/OE/build/oe-core:/OE/build/oe-core-glibc:/OE/build/oe-core/openembedded-core/meta"
# $WKS_SEARCH_PATH
# set? /OE/build/oe-core/openembedded-core/meta/classes/image_types_wic.bbclass:10
# "${THISDIR}:${@':'.join('%s/wic' % p for p in '${BBPATH}'.split(':'))}:${@':'.join('%s/scripts/lib/wic/canned-wks' % l for l in '${BBPATH}:${COREBASE}'.split(':'))}"
WKS_SEARCH_PATH="/OE/build/oe-core/openembedded-core/meta/recipes-core/images:/OE/build/oe-core/wic:/OE/build/oe-core-glibc/wic:/OE/build/oe-core/openembedded-core/meta/wic:/OE/build/oe-core/scripts/lib/wic/canned-wks:/OE/build/oe-core-glibc/scripts/lib/wic/canned-wks:/OE/build/oe-core/openembedded-core/meta/scripts/lib/wic/canned-wks:/OE/build/oe-core/openembedded-core/scripts/lib/wic/canned-wks"
# $WKS_FULL_PATH
# set /OE/build/oe-core/openembedded-core/meta/classes/image_types_wic.bbclass:11
# "${@wks_search('${WKS_FILES}'.split(), '${WKS_SEARCH_PATH}') or ''}"
WKS_FULL_PATH="/OE/build/oe-core/openembedded-core/scripts/lib/wic/canned-wks/directdisk.wks"
Let me know if sharing "bitbake -e core-image-tiny-initramfs" from these 2 tests would be useful for you.
Regards,
>
> On Thu, Feb 09, 2017 at 09:39:30AM +0100, Martin Jansa wrote:
> > Running e.g. bitbake -e busybox with latest oe-core triggers this badly
> > looking exception:
> >
> > Can we add some weak assignment for people who don't care about wic
> > or core-image-tiny-initramfs to continue using oe-core?
> >
> > ERROR: ExpansionError during parsing /OE/oe-core/meta/recipes-core/images/
> > core-image-tiny-initramfs.bb
> > Traceback (most recent call last):
> > File "/OE/bitbake/lib/bb/data_smart.py", line 412, in
> > DataSmart.expandWithRefs(s='\tout="${IMGDEPLOYDIR}/${IMAGE_NAME}"\n\twks="${WKS_FULL_PATH}"\n\tif
> > [ -z "$wks" ]; then\n\t\tbbfatal "No kickstart files from WKS_FILES were
> > found: ${WKS_FILES}. Please set WKS_FILE or WKS_FILES
> > appropriately."\n\tfi\n\n\tBUILDDIR="${TOPDIR}" wic create "$wks" --vars
> > "${STAGING_DIR}/${MACHINE}/imgdata/" -e "${IMAGE_BASENAME}" -o "$out/"
> > ${WIC_CREATE_EXTRA_ARGS}\n\tmv "$out/$(basename "${wks%.wks}")"*.direct
> > "$out${IMAGE_NAME_SUFFIX}.wic"\n\trm -rf "$out/"\n', varname='IMAGE_CMD'):
> > try:
> > > s = __expand_var_regexp__.sub(varparse.var_sub, s)
> > try:
> > File "/OE/bitbake/lib/bb/data_smart.py", line 111, in
> > VariableParse.var_sub(match=<_sre.SRE_Match object; span=(43, 59),
> > match='${WKS_FULL_PATH}'>):
> > else:
> > > var = self.d.getVarFlag(key, "_content")
> > self.references.add(key)
> > File "/OE/bitbake/lib/bb/data_smart.py", line 773, in
> > DataSmart.getVarFlag(var='WKS_FULL_PATH', flag='_content', expand=True,
> > noweakdefault=False, parsing=False):
> > cachename = var + "[" + flag + "]"
> > > value = self.expand(value, cachename)
> >
> > File "/OE/bitbake/lib/bb/data_smart.py", line 436, in
> > DataSmart.expand(s="${@wks_search('${WKS_FILES}'.split(),
> > '${WKS_SEARCH_PATH}') or ''}", varname='WKS_FULL_PATH'):
> > def expand(self, s, varname = None):
> > > return self.expandWithRefs(s, varname).value
> >
> > File "/OE/bitbake/lib/bb/data_smart.py", line 426, in
> > DataSmart.expandWithRefs(s="${@wks_search('directdisk.wks
> > core-image-tiny-initramfs.wks'.split(),
> > '/OE/oe-core/meta/recipes-core/images:${@':'.join('%s/wic' % p for p in
> > '/OE:${TMPDIR}:/OE/oe-core/meta'.split(':'))}:${@':'.join('%s/scripts/lib/wic/canned-wks'
> > % l for l in '/OE:${TMPDIR}:/OE/oe-core/meta:/OE/oe-core'.split(':'))}') or
> > ''}", varname='WKS_FULL_PATH'):
> > except Exception as exc:
> > > raise ExpansionError(varname, s, exc) from exc
> >
> > bb.data_smart.ExpansionError: Failure expanding variable WKS_FULL_PATH,
> > expression was ${@wks_search('directdisk.wks
> > core-image-tiny-initramfs.wks'.split(),
> > '/OE/oe-core/meta/recipes-core/images:${@':'.join('%s/wic' % p for p in
> > '/OE:${TMPDIR}:/OE/oe-core/meta'.split(':'))}:${@':'.join('%s/scripts/lib/wic/canned-wks'
> > % l for l in '/OE:${TMPDIR}:/OE/oe-core/meta:/OE/oe-core'.split(':'))}') or
> > ''} which triggered exception SyntaxError: invalid syntax (WKS_FULL_PATH,
> > line 1)
>
> --
> Regards,
> Ed
--
Martin 'JaMa' Jansa jabber: Martin.Jansa at gmail.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 195 bytes
Desc: Digital signature
URL: <http://lists.openembedded.org/pipermail/openembedded-core/attachments/20170218/bda3a638/attachment-0002.sig>
More information about the Openembedded-core
mailing list