[OE-core] [PATCH] bzip2-native: fix problems when bzip2-native is installed in parallel

[Yao Zhao]

On 12-07-24 03:07 PM, Richard Purdie wrote:
> On Tue, 2012-07-24 at 13:39 -0500, Mark Hatle wrote:
>> On 7/24/12 8:57 AM, Burton, Ross wrote:
>>> On 24 July 2012 14:49, Yao Zhao <yao.zhao at windriver.com> wrote:
>>>> when bzip2-native is installed in parallel to sysroot, it is possible that
>>>> some packages are using bzip2 to unpack, there are chances that bzip2 is
>>>> installed to sysroot but libbz2.so.0 not installed yet because parallel
>>>> installation.
>>>> link bzip2 and bzip2recover statically to avoid this problem and don't lose
>>>> parallel installation. libbz2.so is still available.
>>> Is it me, or is this officially getting silly?  This probably happens
>>> for *every* binary in the sysroot that links to a library, which is
>>> probably a fair proportion of them.  Statically linking every single
>>> one and then special-casing further problems where a static link isn't
>>> sufficient (see pythonnative) just isn't going to scale.
>> The problem is that there are a handful of things that are needed, and for some
>> reason (valid or not) packages are not "requiring", either because they assume
>> the system is valid or they simply don't know they have the requirement.
>>
>> Both python and perl may be used by a number of auto* utilities as well as by
>> packages themselves.  We could attempt to add DEPENDS for all of the cases, but
>> is it really worth it?  I suspect we'll end up specify the DEPENDS for 90% of
>> the things in the system, simply due to them incidentally running some system
>> level python or perl script.  If we make the python/perl into external items
>> that only get loaded into the environment when we're building python/perl
>> modules then things become increasingly easy.
>>
>> As for the bzip2 issue.. This is another of those "we assume it's valid
>> binaries"..  And the problem appears to be when it gets build, it's not valid
>> for a small window of time.
>>
>>   From what Yao has explained to me (offline) the issue is that the
>> assume_provided for bzip2-native works just fine, we use the system
>> version...but then python (or other) utilities come in and say they need
>> "bzip2-full-native".  Which triggers the build.. and opens a small window of
>> time where bzip2 is being installed, where it isn't functional -- something else
>> comes in and has a requirement of bzip2-native, which is satisfied by the
>> assume_provided and we get into the race where it executes a broken binary.
>>
>> The underlying issue is simply that if we're installing tooling that is either
>> assume-provided (based on an alternative 'full' version) or incidental usage
>> like python or perl we need to take precautions to ensure that the build system
>> never sees the "broken" version during the short install window triggering the
>> race condition.
>>
>> I'll let Yao comment further on this particular issue, but there is an overall
>> class of issues with -native that we need to track to get rid of these subtle races.
> The set of problems are cases where we have things in ASSUME_PROVIDED.
> It is near impossible to have an empty set there and sometimes we need
> to build things ourselves which are already listed there. python-native
> and perl-native are the pathological cases but as also have issues with
> bzip2, tar and likely undiscovered yet, now git.
>
> I'd been searching for bzip2-replacement-native rather than full which
> is why I didn't see this earlier. I suspect in the python/bzip2 case,
> not installing the bzip2 binary might well solve many of the problems as
> python really wants a native libbz2 and the host bzip2 binary is ok.
Hi Richard,

Can I provide a patch to change bzip2-full-native to bzip2-native to 
finish this problem?
I have tested that the bzip2-native is not built any more.

thanks,
yao
> The races themselves are over binaries in PATH. As far as I
> know/understand things we're close to having them resolved. What I am
> considering as an improvement is instead of the pythonnative/perlnative
> bbclass files, having a variable like "NATIVEPATHADDITIONS" which would
> take values like "python-native", "perl-native" and would simply add
> ${STAGING_BINDIR}/xxx to PATH for each item listed. The bbclass files
> are useful for the dependency additions too though which sometimes are
> not trivial. This would at least remove the case by case bbclass
> addition issue we currently have for the simple cases though.
>
> Cheers,
>
> Richard
>
>
>
>
>
>