[OE-core] Mis-generation of shell script (run.do_install)?

[Richard Purdie]

On Wed, 2019-01-16 at 08:55 -0500, Jason Andryuk wrote:
> On Tue, Jan 8, 2019 at 1:26 PM <richard.purdie at linuxfoundation.org>
> wrote:
> > On Tue, 2018-12-18 at 12:45 -0500, Jason Andryuk wrote:
> > > I can definitively state I have a hash in bb_codeparser.dat with
> > > an
> > > incorrect shellCacheLine entry and I don't know how it got there.
> > > 
> > > The bad hash is 3df9018676de219bb3e46e88eea09c98.  I've attached
> > > a
> > > file with the binutils do_install() contents which hash to that
> > > value.
> > > 
> > > The bad 3df9018676de219bb3e46e88eea09c98 entry in the
> > > bb_codeparser.dat returned
> > > DEBUG: execs [
> > > DEBUG: execs rm
> > > DEBUG: execs install
> > > DEBUG: execs test
> > > DEBUG: execs sed
> > > DEBUG: execs rmdir
> > > DEBUG: execs bbfatal_log
> > > DEBUG: execs mv
> > > DEBUG: execs /home/build/openxt-compartments/build/tmp-
> > > glibc/work/core2-32-oe-linux/python-async/0.6.2-r0/recipe-
> > > sysroot-
> > > native/usr/bin/python-native/python
> > > DEBUG: execs find
> > 
> > This is useful data (along with the attachment), thanks.
> > 
> > I agree that this looks likely to have come from a core2-32 tuned
> > machine (e.g. genericx86) from python-async do_install.
> > 
> > How old was this build directory? Can you remember any details of
> > the
> > update history for it?
> 
> I think the build directory was from the beginning of October 30th,
> and I guess I hit the collision December 10th or so.
> 
> > I'd be very interested to try and reproduce that hash. I locally
> > blacklisted your collision from my cache and tried to reproduce
> > this. I
> > can generate a matching hash for the binutils do_install but I
> > can't
> > produce one matching the above.
> 
> I tried around December 18th to generate the collision again.  I set
> up a new container with an identical openxt path.  There, python-
> async was built, but it did not have the colliding hash.  When core2-
> 64 binutils was built, it had the expected hash.
> 
> > Can you remember the history of this build directory and which
> > updates
> > it may have had? The python-async recipe is confined to OE-Core so
> > its
> > probably the revision history for the oe-core repo which is most
> > interesting. Anything in the .git/logs directory for that which
> > would
> > help us replay the different versions you might have built?
> 
> oe-core is checked out at 819aa151bd634122a46ffdd822064313c67f5ba5
> It's a git submodule locked at a fixed revision, and it had not
> changed in the build directory.
> 
> OpenXT builds 8 or 9 different MACHINEs and images in sequence in the
> same build directory.  Maybe 6 are core2-32 and two are core2-64. The
> 32bit ones run first.

The hash we don't have is from a core2-32 MACHINE. I'm wondering which
configurations you might have parsed for a core2-32 MACHINE between
October and December?

Was TMPDIR ever cleaned? If not, do you have the python-async WORKDIR
for core2-32? The TMPDIR/logs directory may also have useful hints
about the configurations built...

> I think the problem first manifest after I added an additional local
> layer to BBLAYERS.  At that time, I started building an additional
> MACHINE.  Along with the mis-generated run.do_install script, bitbake
> was complaining about the binutils base hash mismatch which triggered
> the re-build.  The first 64bit MACHINE included TUNE-CCARGS +=
> "-mstackrealign" while the second did not.  Could that be a reason
> why bitbake complained about the base hash mismatch?

By the time the binutils error happens, the error is kind of lost in
history and must have been added some time prior to that.

We know its a build of python-async for a core2-32 MACHINE. Did you
also try building those with the "-mstackrealign" option? Were there
any other changes you can think of that would have applied to the
core2-32 MACHINE builds?

Cheers,

Richard