<div dir="auto">Looks like "readlink -f" is failing. Can you try to capture the output and figure out why?</div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, Aug 20, 2019, 2:04 AM Martin Jansa <<a href="mailto:martin.jansa@gmail.com">martin.jansa@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">* this might be related to:<br>
commit d2fcaeb153fdc3f8d7143ea823139f1537055ff1<br>
Author: Douglas Royds <<a href="mailto:douglas.royds@taitradio.com" target="_blank" rel="noreferrer">douglas.royds@taitradio.com</a>><br>
Date: Thu Dec 20 11:59:47 2018 +1300<br>
<br>
icecc: Don't generate recipe-sysroot symlinks at recipe-parsing time<br>
<br>
* it's still a bit unclear when and why this happends, but I'm seeing<br>
random tasks sometimes failing with:<br>
<br>
WARNING: Exception during build_dependencies for set_icecc_env<br>
WARNING: Error during finalise of /build/meta-oe/meta-python/recipes-devtools/python/<a href="http://python-markupsafe_1.0.bb" rel="noreferrer noreferrer" target="_blank">python-markupsafe_1.0.bb</a><br>
ERROR: Traceback (most recent call last):<br>
File "/build/bitbake/lib/bb/data_smart.py", line 411, in expandWithRefs<br>
s = __expand_python_regexp__.sub(varparse.python_sub, s)<br>
File "/build/bitbake/lib/bb/data_smart.py", line 136, in python_sub<br>
value = utils.better_eval(codeobj, DataContext(self.d), {'d' : self.d})<br>
File "/build/bitbake/lib/bb/utils.py", line 421, in better_eval<br>
return eval(source, ctx, locals)<br>
File "Var <set_icecc_env>", line 1, in <module><br>
File "/build/oe-core/meta/classes/icecc.bbclass", line 287, in icecc_get_and_check_tool<br>
link_path = icecc_get_tool_link(t, d)<br>
File "/build/oe-core/meta/classes/icecc.bbclass", line 246, in icecc_get_tool_link<br>
return subprocess.check_output("readlink -f %s" % tool, shell=True).decode("utf-8")[:-1]<br>
File "/usr/lib/python3.6/subprocess.py", line 336, in check_output<br>
**kwargs).stdout<br>
File "/usr/lib/python3.6/subprocess.py", line 418, in run<br>
output=stdout, stderr=stderr)<br>
subprocess.CalledProcessError: Command 'readlink -f /build/BUILD/work/qemux86-oe-linux/python-markupsafe/1.0-r0/recipe-sysroot-native/usr/bin/x86_64-oe-linux/x86_64-oe-linux-g++' returned non-zero exit status 1.<br>
<br>
The above exception was the direct cause of the following exception:<br>
<br>
Traceback (most recent call last):<br>
File "/build/bitbake/bin/bitbake-worker", line 239, in child<br>
the_data = bb_cache.loadDataFull(fn, appends)<br>
File "/build/bitbake/lib/bb/cache.py", line 327, in loadDataFull<br>
bb_data = self.load_bbfile(virtualfn, appends, virtonly=True)<br>
File "/build/bitbake/lib/bb/cache.py", line 340, in load_bbfile<br>
datastores = parse_recipe(bb_data, bbfile, appends, mc)<br>
File "/build/bitbake/lib/bb/cache.py", line 303, in parse_recipe<br>
bb_data = bb.parse.handle(bbfile, bb_data)<br>
File "/build/bitbake/lib/bb/parse/__init__.py", line 107, in handle<br>
return h['handle'](fn, data, include)<br>
File "/build/bitbake/lib/bb/parse/parse_py/BBHandler.py", line 142, in handle<br>
return ast.multi_finalize(fn, d)<br>
File "/build/bitbake/lib/bb/parse/ast.py", line 386, in multi_finalize<br>
finalize(fn, d)<br>
File "/build/bitbake/lib/bb/parse/ast.py", line 351, in finalize<br>
bb.parse.siggen.finalise(fn, d, variant)<br>
File "/build/bitbake/lib/bb/siggen.py", line 147, in finalise<br>
taskdeps = self._build_data(fn, d)<br>
File "/build/bitbake/lib/bb/siggen.py", line 118, in _build_data<br>
tasklist, gendeps, lookupcache = bb.data.generate_dependencies(d)<br>
File "/build/bitbake/lib/bb/data.py", line 388, in generate_dependencies<br>
deps[dep], values[dep] = build_dependencies(dep, keys, shelldeps, varflagsexcl, d)<br>
File "/build/bitbake/lib/bb/data.py", line 317, in build_dependencies<br>
value, parsedvar = d.getVarFlag(key, "_content", False, retparser=True)<br>
File "/build/bitbake/lib/bb/data_smart.py", line 802, in getVarFlag<br>
parser = self.expandWithRefs(value, cachename)<br>
File "/build/bitbake/lib/bb/data_smart.py", line 424, in expandWithRefs<br>
raise ExpansionError(varname, s, exc).with_traceback(tb) from exc<br>
File "/build/bitbake/lib/bb/data_smart.py", line 411, in expandWithRefs<br>
s = __expand_python_regexp__.sub(varparse.python_sub, s)<br>
File "/build/bitbake/lib/bb/data_smart.py", line 136, in python_sub<br>
value = utils.better_eval(codeobj, DataContext(self.d), {'d' : self.d})<br>
File "/build/bitbake/lib/bb/utils.py", line 421, in better_eval<br>
return eval(source, ctx, locals)<br>
File "Var <set_icecc_env>", line 1, in <module><br>
File "/build/oe-core/meta/classes/icecc.bbclass", line 287, in icecc_get_and_check_tool<br>
link_path = icecc_get_tool_link(t, d)<br>
File "/build/oe-core/meta/classes/icecc.bbclass", line 246, in icecc_get_tool_link<br>
return subprocess.check_output("readlink -f %s" % tool, shell=True).decode("utf-8")[:-1]<br>
File "/usr/lib/python3.6/subprocess.py", line 336, in check_output<br>
**kwargs).stdout<br>
File "/usr/lib/python3.6/subprocess.py", line 418, in run<br>
output=stdout, stderr=stderr)<br>
bb.data_smart.ExpansionError: Failure expanding variable set_icecc_env, expression was if [ "${@use_icecc(bb, d)}" = "no" ]<br>
then<br>
return<br>
fi<br>
ICECC_VERSION="${@icecc_version(bb, d)}"<br>
if [ "x${ICECC_VERSION}" = "x" ]<br>
then<br>
bbwarn "Cannot use icecc: could not get ICECC_VERSION"<br>
return<br>
fi<br>
<br>
ICE_PATH="${@icecc_path(bb, d)}"<br>
if [ "x${ICE_PATH}" = "x" ]<br>
then<br>
bbwarn "Cannot use icecc: could not get ICE_PATH"<br>
return<br>
fi<br>
<br>
ICECC_BIN="${@get_icecc(d)}"<br>
if [ -z "${ICECC_BIN}" ]; then<br>
bbwarn "Cannot use icecc: icecc binary not found"<br>
return<br>
fi<br>
if [ -z "$(which patchelf patchelf-uninative)" ]; then<br>
bbwarn "Cannot use icecc: patchelf not found"<br>
return<br>
fi<br>
<br>
# Create symlinks to icecc in the recipe-sysroot directory<br>
mkdir -p ${ICE_PATH}<br>
if [ -n "${KERNEL_CC}" ]; then<br>
compilers="${@get_cross_kernel_cc(bb,d)}"<br>
else<br>
compilers="x86_64-oe-linux-gcc x86_64-oe-linux-g++"<br>
fi<br>
for compiler in $compilers; do<br>
ln -sf ${ICECC_BIN} ${ICE_PATH}/$compiler<br>
done<br>
<br>
ICECC_CC="${@icecc_get_and_check_tool(bb, d, "gcc")}"<br>
ICECC_CXX="${@icecc_get_and_check_tool(bb, d, "g++")}"<br>
# cannot use icecc_get_and_check_tool here because it assumes as without target_sys prefix<br>
ICECC_WHICH_AS="${@bb.utils.which(os.getenv('PATH'), 'as')}"<br>
if [ ! -x "${ICECC_CC}" -o ! -x "${ICECC_CXX}" ]<br>
then<br>
bbwarn "Cannot use icecc: could not get ICECC_CC or ICECC_CXX"<br>
return<br>
fi<br>
<br>
ICE_VERSION=`$ICECC_CC -dumpversion`<br>
ICECC_VERSION=`echo ${ICECC_VERSION} | sed -e "s/@VERSION@/$ICE_VERSION/g"`<br>
if [ ! -x "/build/BUILD/work/qemux86-oe-linux/python-markupsafe/1.0-r0/recipe-sysroot-native/usr/bin/icecc-create-env" ]<br>
then<br>
bbwarn "Cannot use icecc: invalid ICECC_ENV_EXEC"<br>
return<br>
fi<br>
<br>
ICECC_AS="`${ICECC_CC} -print-prog-name=as`"<br>
# for target recipes should return something like:<br>
# /OE/tmp-eglibc/sysroots/x86_64-linux/usr/libexec/arm920tt-oe-linux-gnueabi/gcc/arm-oe-linux-gnueabi/4.8.2/as<br>
# and just "as" for native, if it returns "as" in current directory (for whatever reason) use "as" from PATH<br>
if [ "`dirname "${ICECC_AS}"`" = "." ]<br>
then<br>
ICECC_AS="${ICECC_WHICH_AS}"<br>
fi<br>
<br>
if [ ! -f "${ICECC_VERSION}.done" ]<br>
then<br>
mkdir -p "`dirname "${ICECC_VERSION}"`"<br>
<br>
# the ICECC_VERSION generation step must be locked by a mutex<br>
# in order to prevent race conditions<br>
if flock -n "${ICECC_VERSION}.lock" \<br>
/build/BUILD/work/qemux86-oe-linux/python-markupsafe/1.0-r0/recipe-sysroot-native/usr/bin/icecc-create-env "${ICECC_CC}" "${ICECC_CXX}" "${ICECC_AS}" "${ICECC_VERSION}"<br>
then<br>
touch "${ICECC_VERSION}.done"<br>
elif ! wait_for_file "${ICECC_VERSION}.done" 30<br>
then<br>
# locking failed so wait for ${ICECC_VERSION}.done to appear<br>
bbwarn "Timeout waiting for ${ICECC_VERSION}.done"<br>
return<br>
fi<br>
fi<br>
<br>
# Don't let ccache find the icecream compiler links that have been created, otherwise<br>
# it can end up invoking icecream recursively.<br>
export CCACHE_PATH="$PATH"<br>
export CCACHE_DISABLE="1"<br>
<br>
export ICECC_VERSION ICECC_CC ICECC_CXX<br>
export PATH="$ICE_PATH:$PATH"<br>
<br>
bbnote "Using icecc path: $ICE_PATH"<br>
bbnote "Using icecc tarball: $ICECC_VERSION"<br>
which triggered exception CalledProcessError: Command 'readlink -f /build/BUILD/work/qemux86-oe-linux/python-markupsafe/1.0-r0/recipe-sysroot-native/usr/bin/x86_64-oe-linux/x86_64-oe-linux-g++' returned non-zero exit status 1.<br>
<br>
ERROR: Task (virtual:multilib:lib32:/build/meta-oe/meta-python/recipes-devtools/python/python-markupsafe_1.0.bb:do_patch) failed with exit code '1'<br>
<br>
Signed-off-by: Martin Jansa <<a href="mailto:Martin.Jansa@gmail.com" target="_blank" rel="noreferrer">Martin.Jansa@gmail.com</a>><br>
---<br>
meta/classes/icecc.bbclass | 6 +++++-<br>
1 file changed, 5 insertions(+), 1 deletion(-)<br>
<br>
diff --git a/meta/classes/icecc.bbclass b/meta/classes/icecc.bbclass<br>
index edb0e10434..63d8b4dfee 100644<br>
--- a/meta/classes/icecc.bbclass<br>
+++ b/meta/classes/icecc.bbclass<br>
@@ -243,7 +243,11 @@ def icecc_get_external_tool(bb, d, tool):<br>
<br>
def icecc_get_tool_link(tool, d):<br>
import subprocess<br>
- return subprocess.check_output("readlink -f %s" % tool, shell=True).decode("utf-8")[:-1]<br>
+ try:<br>
+ return subprocess.check_output("readlink -f %s" % tool, shell=True).decode("utf-8")[:-1]<br>
+ except subprocess.CalledProcessError as e:<br>
+ bb.note("icecc: one of the tools probably disappeared during recipe parsing, cmd readlink -f %s returned %d:\n%s" % (tool, e.returncode, e.output.decode("utf-8")))<br>
+ return tool<br>
<br>
def icecc_get_path_tool(tool, d):<br>
# This is a little ugly, but we want to make sure we add an actual<br>
-- <br>
2.17.1<br>
<br>
-- <br>
_______________________________________________<br>
Openembedded-core mailing list<br>
<a href="mailto:Openembedded-core@lists.openembedded.org" target="_blank" rel="noreferrer">Openembedded-core@lists.openembedded.org</a><br>
<a href="http://lists.openembedded.org/mailman/listinfo/openembedded-core" rel="noreferrer noreferrer" target="_blank">http://lists.openembedded.org/mailman/listinfo/openembedded-core</a><br>
</blockquote></div>