[OE-core] Mis-generation of shell script (run.do_install)?

[Jason Andryuk]

On Tue, Dec 11, 2018 at 10:02 AM Richard Purdie
<richard.purdie at linuxfoundation.org> wrote:
>
> On Tue, 2018-12-11 at 08:42 -0500, Jason Andryuk wrote:
> > Has anyone ever seen a generated shell script missing functions?
> >
> > I have an OpenXT/OpenEmbedded setup where I had run many successful
> > builds.  I made a change and then re-ran the build - it failed in
> > binutil's do_install with autotools_do_install command not found.
> >
> > core2-64-oe-linux/binutils/2.28-r0/temp/run.do_install.11776: line
> > 124: autotools_do_install: command not found
> >
> > Sure enough, autotools_do_install is not in run.do_install.
> >
> > I had not changed binutils or any relevant variable, as far as I can
> > tell.  If I run with '-e' I see the full autotools_do_install
> > function in the output.  For some reason, the generated script wasn't
> > including autotools_do_install.
> >
> > I tried binutils -c cleansstate, but that didn't work.  I tried
> > pruning the sstate-cache dir, but that didn't work.  I tried deleting
> > tmp-glibc and sstate-cache, but it had the same error when I rebuilt.
> >
> > Modifying binutils do_install by adding a comment and `true` lets it
> > builds.
> >
> > I saw something similar one other time where the generated script was
> > missing a function.  I can't recall the details, but it was a
> > different package and MACHINE.
> >
> > Any suggestions on debugging this?
>
> It sounds like pysh in bitbake wasn't able to see a dependency on the
> function in question. Creating a small/reproducible test case would be
> how I'd approach it, there are tests on the pysh code in bitbake-
> selftest for example.
>
> Once I had a test case which failed, I'd then use that to debug and see
> if I could figure out a fix.

Thanks, Richard.

I wasn't sure how to tie into the pysh stuff, but that got me poking
around in bitbake/lib/bb/codeparser.py .  Adding debug messages to
parse_shell(), I see that do_install is found in the CodeParserCache,
bb_codeparser.dat, but the returned `execs` do not include
autotools_do_install.  Strangely, it includes a path to python -
...core2-32-oe-linux/python-async/0.6.2-r0/recipe-sysroot-native/usr/bin/python-native/python.
It looks like `execs` could be for `distutils_do_install`.  And again,
strangely, python-async is not in my tmp-glibc.  It must have been
built at some point which left the entry in bb_codeparser.dat.

I built python-async, but its distutils_do_install hash value does not
match the one in the cache.

Moving cache/bb_codeparser.dat out of the way, bitbake complains:
ERROR: When reparsing
/home/build/openxt/build/repos/openembedded-core/meta/recipes-devtools/binutils/binutils_2.28.bb.do_install,
the basehash value changed from 80812e0772cf901b51790c205564070d to
493152cd3740c5420d0bf7a5d09df001. The metadata is not deterministic
and this needs to be fixed.

`cleanall` does not clear out the message, but the package builds.

Regards,
Jason