[OE-core] [PATCH] Allow for simultaneous do_rootfs tasks with rpm

[Stephano Cetola]

On 07/27, Richard Purdie wrote:
> On Tue, 2016-07-26 at 20:43 -0700, Stephano Cetola wrote:
> > On 07/20, Richard Purdie wrote:
> > > On Thu, 2016-07-14 at 14:20 -0700, Stephano Cetola wrote:
> > > > Give each rootfs its own RPM channel to use.  This puts the RPM
> > > > metadata
> > > > in a private subdirectory of $WORKDIR, rather than living in
> > > > DEPLOY_DIR
> > > > where other tasks may race with it.
> > > > 
> > > > This allows us to reduce the time that the rpm.lock is held to
> > > > only
> > > > the
> > > > time needed to hardlink the RPMs, allowing the majority of the
> > > > rootfs
> > > > operation to run in parallel.
> > > > 
> > > > [ YOCTO #9255 ]
> > > > 
> > > > Signed-Off-By: Steven Walter <stevenrwalter at gmail.com>
> > > > Signed-off-by: Stephano Cetola <stephano.cetola at linux.intel.com>
> > > > ---
> > > >  meta/classes/rootfs_rpm.bbclass |  5 -----
> > > >  meta/lib/oe/package_manager.py  | 17 ++++++++++++++---
> > > >  2 files changed, 14 insertions(+), 8 deletions(-)
> > > 
> > > Sadly, much as I'd love to merge this, testing shows we have some
> > > issues.
> > > 
> > > Firstly, it means we no longer generate indexes in tmp/deploy/rpm
> > > and
> > > this breaks -c testimage since some of the tests connect and look
> > > for
> > > packages there, e.g. 
> > > https://autobuilder.yoctoproject.org/main/builders
> > > /nightly-qa-logrotate/builds/851 but many others would have also
> > > failed.
> > > 
> > 
> > After digging into this I think that I can change the tests so that
> > they
> > point at the correct location:
> > 
> > e.g.
> > tmp/work/qemux86-oe-linux/core-image-sato/1.0
> > -r0/rpms/qemux86/repodata
> > 
> > based on the target triplet, image, PV (I'm guessing here), and arch
> > variables. I realized when digging into this that using "rpms" in the
> > path (which was a result of this patch) is wrong. It should be "rpm".
> > 
> > Let me know if this sounds right or if I'm dreaming.
> 
> Its not the right fix. The tests can be run against a pre-existing
> image which wasn't "built" by the same build directory. tmp/deploy/xxx
> is therefore the correct location for the packages/index, we just need
> to ensure something has built the indexes there.
> 
> Cheers,
> 
> Richard

I see what you are saying. I'll look into smart and createrepo to see
if there is a "smart" way to update the indexes without requiring a
lock. Is there a place in OE where we can inject code to be run "after
all rootfs"?

-S