[OE-core] [PATCH] Allow for simultaneous do_rootfs tasks with rpm

[Stephano Cetola]

On 07/20, Richard Purdie wrote:
> On Thu, 2016-07-14 at 14:20 -0700, Stephano Cetola wrote:
> > Give each rootfs its own RPM channel to use.  This puts the RPM
> > metadata
> > in a private subdirectory of $WORKDIR, rather than living in
> > DEPLOY_DIR
> > where other tasks may race with it.
> > 
> > This allows us to reduce the time that the rpm.lock is held to only
> > the
> > time needed to hardlink the RPMs, allowing the majority of the rootfs
> > operation to run in parallel.
> > 
> > [ YOCTO #9255 ]
> > 
> > Signed-Off-By: Steven Walter <stevenrwalter at gmail.com>
> > Signed-off-by: Stephano Cetola <stephano.cetola at linux.intel.com>
> > ---
> >  meta/classes/rootfs_rpm.bbclass |  5 -----
> >  meta/lib/oe/package_manager.py  | 17 ++++++++++++++---
> >  2 files changed, 14 insertions(+), 8 deletions(-)
> 
> Sadly, much as I'd love to merge this, testing shows we have some
> issues.
> 
> Firstly, it means we no longer generate indexes in tmp/deploy/rpm and
> this breaks -c testimage since some of the tests connect and look for
> packages there, e.g. https://autobuilder.yoctoproject.org/main/builders
> /nightly-qa-logrotate/builds/851 but many others would have also
> failed.
> 

After digging into this I think that I can change the tests so that they
point at the correct location:

e.g.
tmp/work/qemux86-oe-linux/core-image-sato/1.0-r0/rpms/qemux86/repodata

based on the target triplet, image, PV (I'm guessing here), and arch
variables. I realized when digging into this that using "rpms" in the
path (which was a result of this patch) is wrong. It should be "rpm".

Let me know if this sounds right or if I'm dreaming.

> I've also started wondering what happens if we have some scenario where
> we have:
> 
> bitbake core-image-sato epiphany
> <rootfs for core-image-sato starts>
> <rebuild for epiphany or one of its dependencies happens>
> <old package from epiphany or one of its dependencies is removed from
> tmp/deploy/rpm>
> <core-image-sato index references this now removed package>
> 
> I'd hope the system should handle this since it should only be
> referencing things that are direct task dependencies but with
> rpm/smart, who knows :/. We should at least see how this fails (if it
> fails).
> 

When I build these two this morning I got a bunch of failures on
webkit's configure stage. I'll have to look into those.

> It could get particularly ugly if we use globbing for the rootfs
> construction (e.g. *-locale) and the list of packages changed depending
> on what else was built. I'd guess we'd already have this problem but
> this patch could magnify any problem.
> 
> We are going to have to at least solve the testimage package index
> problem before we can merge it.
> 
> Cheers,
> 
> Richard
> 
>