[OE-core] util-macros.do_package failed Re: Build failure

[Richard Purdie]

On Fri, 2014-04-04 at 21:01 +0100, Burton, Ross wrote:
> On 4 April 2014 18:18, Robert Yang <liezhi.yang at windriver.com> wrote:
> > Maybe we can let the do_package depend on base-passwd:do_populate_sysroot
> > to fix the problem.
> 
> I keep on thinking that we need to effectively have a readers/writer
> lock on the sysroot, so that e.g. do_compile() takes a read lock and
> do_populate_sysroot() takes a write lock.  This way very slim races
> such as this (and aclocal disappearing, and binaries being re-written
> whilst being executed, etc) just can't happen.  By having a
> readers/writer lock the stalls for the actual populate of the sysroot
> shouldn't have too great an impact, and would let us remove other
> workarounds we've added (such as the baroque aclocal-copy).

We've tried this before and "shouldn't" doesn't work out too well in
practice.

The aclocal-copy business is actually now an improvement since it only
accesses things it depends upon which is a good thing as far as I'm
concerned. Other ways we could possibly improve:

a) Atomically moves files into the sysroot (and I guess move and delete
for atomic removal). This means they can't "half" be there.

b) Move libs in before binaries. This way a binary can't get run before
its dependencies are there.

This all continues to hack around the fact its a shared area though and
I'd imagine regardless of what tricks you play and even if you add in
locks, there will still be determinism issues.

The conclusion you then reach is that the only way you can ultimately
solve things is a sysroot per build item. You could do something that
would perform moderately well with a directory per sysroot item, then
hardlink these into individual sysroots per workdir.

In many ways, the aclocal-copy change is an experiment for that, we
should now have the pieces where we could implement such a thing...

Cheers,

Richard