[OE-core] Cache unihash ... doesn't match BB_UNIHASH ...

[Alex Kiernan]

On Fri, Feb 7, 2020 at 8:31 AM Richard Purdie
<richard.purdie at linuxfoundation.org> wrote:
>
> On Fri, 2020-02-07 at 08:16 +0000, Alex Kiernan wrote:
> > Hi
> >
> > We've got a couple of multiconfig builds with different
> > configurations in one bitbake run (whether it needs to be like this a
> > different question - I suspect we had something that worked and
> > stopped there!)
> >
> > However, for sometime we've been seeing errors like this on master
> > (the recipes vary):
> >
> > ERROR: mc:forensic:ca-certificates-20190110-r0
> > do_deploy_source_date_epoch: Cache unihash
> > 37d2546973a2b7c16fbcb1a8dd4939e4a1bba8c3d6d2dfb6e7204ca08d437ace
> > doesn't match BB_UNIHASH
> > 09baf78bd9316acdd308980c42ea4ed1eacea61bcc0d840c36293d1a5a22db69
>
> This is an internal sanity test. It means the hash that it had in the
> task/worker context didn't match what the server thought the unihash
> should be. There have been changes in this area to try and fix various
> things, I'm wondering if we introduced a bug with multiconfig handling.
>
> There are some patches in master-next related to this area so it would
> be interesting to know if that made things better/worse.
>

I can pull those in, if it'd be useful to test?

> > Coupled with deadlock messages:
> >
> > WARNING: Runqeueue deadlocked on deferred tasks, forcing task
> > mc:factory:virtual:native:/var/lib/jenkins/workspace/nanohub_master/b
> > uild/../poky/meta/recipes-
> > support/sqlite/sqlite3_3.31.1.bb:do_populate_sysroot
>
> This happens if a build has multiple different components with matching
> hashes. It builds the "first" one and defers the others since they
> could become available in sstate after the first one builds.
>
> You should see messages from runqueue.py:
>
> bb.note("Deferring %s after %s" % (tid, sqdata.hashes[h]))
>
> The behaviour should be to therefore prioritise one multiconfig. I
> worry that the sorting isn't quite right and its building bits of
> several multiconfigs and deadlocking?
>
> Do you have the logs with the Deferring XXX after YYY messages in you
> could share? I suspect its a sorting problem.
>

Sure - I'll mail it over separately.

-- 
Alex Kiernan