[OE-core] Debug from failing hashequiv builds - server side problem?

[Joshua Watt]

On Sun, Dec 22, 2019 at 3:25 PM Richard Purdie
<richard.purdie at linuxfoundation.org> wrote:
>
> On Sun, 2019-12-22 at 10:22 -0600, Joshua Watt wrote:
> > On Sun, Dec 22, 2019, 10:17 AM Joshua Watt <jpewhacker at gmail.com>
> > wrote:
> > >
> > > On Sun, Dec 22, 2019, 10:09 AM Richard Purdie <
> > > richard.purdie at linuxfoundation.org> wrote:
> > > > On Sun, 2019-12-22 at 10:00 -0600, Joshua Watt wrote:
> > > > It won't help the other native/cross to target boundary mapping
> > > > issue.
> > > >
> > > > See above, I'm wondering if we could abuse the method field to
> > > > make
> > > > this work? Certainly we could test that...
> > >
> > > That seems reasonable for a quick test. The method is supposed to
> > > be the output hash calculation method, so my first instinct would
> > > be that it is better in the long run to add another field e.g.
> > > "class" or something.
> >
> > Perhaps based on ${NATIVELSBSTRING} like sstate.bbclass path? That
> > has nice congruency with sstate file paths, which I like
>
> If only that were the right thing.
>
> We need:
>
>     if bb.data.inherits_class('native', d):
>         d.setVar('SSTATE_PKGARCH', d.getVar('BUILD_ARCH', False))
>     elif bb.data.inherits_class('cross', d):
>         d.setVar('SSTATE_PKGARCH', d.expand("${BUILD_ARCH}_${TARGET_ARCH}"))
>
> i.e. SSTATE_PKGARCH but only for native/cross.

Ok, so I've had a little bit of time to think about this.

>From a more theoretical standpoint, I think that hash equivalence only
works if all of the possible variables that can impact the output hash
are present in the taskhash. Otherwise, you can end up in cases
described above where the same taskhash results in diverging output
hashes that will never converge back together (in this case because
the build host arch differs). For most target recipes this should
already be true because we certainly want to rebuild the recipe if any
of the inputs that could possibly affect the output change. One of the
great parts about hash equivalence is that it sort dynamically and
automatically figures out if these changes don't actually change
anything and accounts for it so we don't have to manually figure out
which variables for which recipes could be whitelisted (not that we
would embark on that madness!).

However, for some recipes (e.g. native and cross), the taskhash is
incomplete in this case because we don't actually care about the
specific binary output from these recipes; we actually care about the
behaviour of these recipes when they are run (e.g. we don't care if
gcc-cross running on x86_64 or aarch is the exact same binary as long
as it produces the exact same cross compiled target binary in both
cases). Hash equivalence sort of falls down a little bit here because
there is no sane way to hash the "behaviour" of a native or cross
recipe. If there were we could set the output hash algorithm for
native and cross recipes to "OEOminiscentBehaviourHash" and all would
be well :) In lieu of that, the next best thing is to make sure that
the inputs we give to the hash equivalence server are the full set
that affect the output hash, just like we do for target builds. This
works because we do expect that on a given build host arch, the output
should be the same from build to build.

So, the real question is how do we make sure that the inputs to the
hash equivalence server are all the ones that affect the output hash?
I think there are a few options:
 1) Make up yet another hash (yahash) that is equivalent to the
taskhash + the hash of variables "unwhitelisted" for the purpose of
hash equivalence. Instead of dealing with taskhashes, the hash
equivalence deals with yahashes
 2) Just like #1, except that the yahash is only the unwhitelisted
variables and hash equivalence uses both the taskhash and the yahash
for lookups.
 3) Implement some sort of "namespace" field in the hash equivalence
tables that is used to filter out entries appropriately (basically,
the solution proposed by Richard).

I don't really have a strong opinion either way.... #3 seems much
easier, but the thing that worries me is if we are going to find other
variables that should be unwhitelisted and do #1 or #2 anyway.

As stated, I think that one of the advantages of hash equivalence is
that we can be a little more aggressive with what we "unwhitelist" and
let the hash equivalence process itself figure out if variable
actually makes a difference in the output.

>
> Cheers,
>
> Richard
>