[oe] Race issues (lzma-native)

[Graham Gower]

On 22 November 2010 17:24, Frans Meulenbroeks
<fransmeulenbroeks at gmail.com> wrote:
> 2010/11/22 Graham Gower <graham.gower at gmail.com>:
>> On 22 November 2010 10:42, Andrea Adami <andrea.adami at gmail.com> wrote:
>>> I though tthe issue had disappeared (in the meanwhile I remove an
>>> obsolete dependency on zlib)
>>> but yesterday again...
>>>
>>> run.SRC_DISTRIBUTECOMMAND was malformed, broken at line 122 (export
>>> docdir="/oe/bu ).
>>>
>>> On second build the size is much bigger (1102 lines).
>>>
>>> Retrying..
>>> Andrea
>>>
>>> _______________________________________________
>>> Openembedded-devel mailing list
>>> Openembedded-devel at lists.openembedded.org
>>> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-devel
>>>
>>
>> With pstaging disabled, do_fetch is called twice. The second do_fetch
>> races with do_unpack. Presumably we see a problem in lzma-native
>> because it does oe.process.run() in do_unpack_append. I can't see
>> exactly *what* is racing though.
>>
>> I previously spoke to Kergoth about it on IRC and he suggested making
>> bb.build.exec_func("do_fetch", d) conditional on whether the do_fetch
>> stamp file exists... I've not gotten around to doing that (bad me).
>> Want to try it?
>
> Hm.
> I don't know the exact details, but this looks like a bandage, not
> like a fix for the root problem.

The conditional call of do_fetch is probably what we want, but I'd
prefer to see this race fixed before we do that to ensure we aren't
masking any other related races.

> Why is do_fetch called twice in the first place. I would expect this
> not to happen.

See src_distribute.bbclass and refer to do_distribute_sources().
do_fetch is only called twice if pstaging is disabled (which is not
the default).

-Graham