[oe] The coreutils-native race...

[Chris Larson]

On Sun, Apr 25, 2010 at 11:30 PM, Koen Kooi <k.kooi at student.utwente.nl>wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On 25-04-10 21:57, Tom Rini wrote:
> > On Sun, 2010-04-25 at 21:28 +0200, Koen Kooi wrote:
> >> -----BEGIN PGP SIGNED MESSAGE-----
> >> Hash: SHA1
> >>
> >> On 25-04-10 19:48, Tom Rini wrote:
> >>> Hey all.  I thought I would try and explain what Chris has been up to
> >>> with at least some of the base.bbclass changes (the ones related to
> >>> md5sum and cp).
> >>>
> >>> Right now, with a big enough BB_NUM_THREADS we can get into a race
> where
> >>> coreutils-native is installing programs and elsewhere we are in a
> >>> do_fetch and either trying to use 'cp' or 'md5sum', and blam, we try
> and
> >>> invoke the program while it's being installed (and see things like
> >>> sh: /path/to/staging/i686-linux/usr/bin/cp: Textfile is busy).
> >>>
> >>> There's a few ways out of this:
> >>> 1) Don't rely on 'cp' and 'md5sum' anymore but use python for it.
> >>> 2) Make an oe_cp and oe_md5sum to go with oe_sha256sum
> >>> 3) IIRC, the big part of coreutils-native was a fully functional,
> >>> always, 'install'.  We could just copy the install we build or provide
> >>> an install wrapper (oe_install) or so
> >>> 4) ???
> >>
> >> Even thought I loathe python, option 1 sounds like a nice way to go
> >> since it doesn't involve butchering recipes.
> >
> > I think we're OK, except in the case of when we can't rely on a built-in
> > md5sum (and then the race comes back) and just need to figure out if the
> > libpam-base-files recipe does things (a) optimally and (b) if that's a
> > common thing.
> >
> > Currently, with a quick glance at the recipe, I don't see any
> > distro-specific overrides, so I'd be inclined to call it lazy recipe
> > writing, until someone points out history of why that's a good thing..
>
> I have no problem with changing the libpam-base-files SRC_URI to
>
> file://pam.d/
>
> or
>
> file://foo.pam \
> file://bar.pam \
> ...
> etc
>
> But I would like to see it documented in the OE manual that file://dir/*
> can't be used, and why.
>

Agreed.  In this particular case, it was never, as far as I'm aware,
*directly* supported, but was one of many cases where we depend on
implementation details - it happens that the argument was directly shoved
onto a 'cp' command line, and that was executed via a shell, so the shell
could expand the glob.  I'll take on adjusting such cases and updating the
docs.

The issue I see with trying to support a globbing mechanism for this is that
the semantics of such a thing are not obvious.  Does it grab everything in
every 'dir' that exists in filespath, or only the first?  It's unclear what
the user intends by that, whereas a 'file://dir/' is rather more clear.
-- 
Christopher Larson
clarson at kergoth dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Maintainer - Tslib
Senior Software Engineer, Mentor Graphics