[OE-core] Debug from failing hashequiv builds - server side problem?

[Joshua Watt]

On Sun, Dec 22, 2019, 10:17 AM Joshua Watt <jpewhacker at gmail.com> wrote:

>
>
> On Sun, Dec 22, 2019, 10:09 AM Richard Purdie <
> richard.purdie at linuxfoundation.org> wrote:
>
>> On Sun, 2019-12-22 at 10:00 -0600, Joshua Watt wrote:
>> > On Sun, Dec 22, 2019, 6:49 AM Richard Purdie <
>> > richard.purdie at linuxfoundation.org> wrote:
>> > > On Sun, 2019-12-22 at 12:08 +0000, Richard Purdie wrote:
>> > >
>> > > At query time in a clean build, the hashserver cannot know which of
>> > > the two output hashes it needs to return the value for.
>> >
>> > In the case of multiple taskhashes mapping to different output
>> > hashes, the server is supposed to simply return the oldest unihash.
>> > If it's not doing that, there might be a bug.
>>
>>
>> The problem here is a single taskhash mapping to two different outputs.
>>
>> m4-native:do_populate_sysroot (on aarch64)
>> m4-native:do_populate_sysroot (on x86-64)
>>
>> When we query based on the input hash we'll get the first built.
>>
>> We then rebuild as its not in sstate and can match a previous task
>> output, we then get a new hash.
>>
>> If we start a new build with no cache, we lookup the input hash and get
>> the first built task of the wrong arch again.
>>
>> > > I can think of two possible options:
>> > >
>> > > a) When we report after running a task, we check if that input hash
>> > > already has a value and then reuse it for the output hash mapping.
>> >
>> > I don't quite follow this one, can you be a little more precise with
>> > what hashes you are referring to (e.g. taskhash, unihash, outhash)?
>>
>> We lookup a taskhash, get unihash A but its not in sstate (wrong arch).
>> We build this thing, send the outhash, currently we get unihash B.
>>
>> I'm saying we map that outhash to unihash A since the server already
>> has an entry for it.
>>
>> > > b) We start adding some kind of suffix to the reported hashes for
>> > > native output which is used within the hash equiv server but not
>> > > sstate.
>>
>> I was thinking about this further and I had a slightly evil idea. What
>> if we set the method to XXX:<native arch>"? (where XXX is the current
>> value).
>>
>> This would namespace the two native arches separately and I think then
>> avoid the problems?
>>
>> > Just for clarification, this is because "native" can either be x86_64
>> > or aarch64 but the actual arch (HOST_ARCH ?) isn't part of the
>> > taskhash calculation?
>>
>> Correct.
>>
>> > This sounds related to the gcc-cross issues we had with the eSDK.
>>
>> The previous problem was on the boundary between cross/native and
>> target. This is a pure native (or cross) issue.
>>
>> > Is there a reason that the host arch isn't part of the taskhash?
>>
>> Yes. Should the target output depend upon which arch it was built on?
>> (answer, no it shouldn't so the native hashes have to match).
>>
>> > I think it might be possible to report additional inputs in the
>> > hashes reported to the server. The server doesn't really care if it's
>> > the exact taskhash, as long as the client is consistent. Perhaps that
>> > would help with the gcc-cross issue as well?
>>
>> It won't help the other native/cross to target boundary mapping issue.
>>
>> See above, I'm wondering if we could abuse the method field to make
>> this work? Certainly we could test that...
>>
>
> That seems reasonable for a quick test. The method is supposed to be the
> output hash calculation method, so my first instinct would be that it is
> better in the long run to add another field e.g. "class" or something.
>

Perhaps based on ${NATIVELSBSTRING} like sstate.bbclass path? That has nice
congruency with sstate file paths, which I like


>
>> Cheers,
>>
>> Richard
>>
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openembedded.org/pipermail/openembedded-core/attachments/20191222/91ed556b/attachment.html>