[OE-core] [PATCH] base.bbclass: avoid 'find -ignore_readdir_race -delete'

[Matthias Schiffer]

On Tue, 2018-11-06 at 00:25 -0800, Andre McCurdy wrote:
> On Tue, Nov 6, 2018 at 12:23 AM Matthias Schiffer
> <matthias.schiffer at ew.tq-group.com> wrote:
> > 
> > On Mon, 2018-11-05 at 09:36 +0100, Matthias Schiffer wrote:
> > > On Fri, 2018-11-02 at 20:28 -0700, Andre McCurdy wrote:
> > > > On Fri, Nov 2, 2018 at 8:43 AM Richard Purdie
> > > > <richard.purdie at linuxfoundation.org> wrote:
> > > > > 
> > > > > On Fri, 2018-11-02 at 15:19 +0000, Peter Kjellerstedt wrote:
> > > > > > > -----Original Message-----
> > > > > > > From: openembedded-core-bounces at lists.openembedded.org
> > > > > > > <openembedded-
> > > > > > > core-bounces at lists.openembedded.org> On Behalf Of
> > > > > > > matthias.schiffer at ew.tq-group.com
> > > > > > > Sent: den 2 november 2018 15:39
> > > > > > > To: openembedded-core at lists.openembedded.org
> > > > > > > Subject: [OE-core] [PATCH] base.bbclass: avoid 'find -
> > > > > > > ignore_readdir_race -delete'
> > > > > > > 
> > > > > > > From: Matthias Schiffer <
> > > > > > > matthias.schiffer at ew.tq-group.com>
> > > > > > > 
> > > > > > > Until recent versions of findutils, -ignore_readdir_race
> > > > > > > did
> > > > > > > not
> > > > > > > work
> > > > > > > correctly with -delete. This led to spurious build
> > > > > > > failures
> > > > > > > when
> > > > > > > files
> > > > > > > disappeared; specifically this was seen due to
> > > > > > > do_configure
> > > > > > > and
> > > > > > > do_populate_lic running concurrently for packages with
> > > > > > > ${B}
> > > > > > > ==
> > > > > > > ${WORKDIR}:
> > > > > > > 
> > > > > > >    find: '.../sstate-build-populate_lic': No such file or
> > > > > > > directory
> > > > > > > 
> > > > > > > As the find command of the host system is used here, we
> > > > > > > can't
> > > > > > > ensure that
> > > > > > > the used version contains the fix. Work around the issue
> > > > > > > by
> > > > > > > passing
> > > > > > > the
> > > > > > > output of find to 'rm -f' instead of using -delete.
> > > > > > > 
> > > > > > > Signed-off-by: Matthias Schiffer <
> > > > > > > matthias.schiffer at ew.tq-group.com
> > > > > > > > 
> > > > > > > 
> > > > > > > ---
> > > > > > > 
> > > > > > > It would be great if this patch could be backported to
> > > > > > > all
> > > > > > > versions
> > > > > > > back to Rocko, as our current setup is still based on
> > > > > > > Rocko.
> > > > > > > 
> > > > > > >  meta/classes/base.bbclass | 2 +-
> > > > > > >  1 file changed, 1 insertion(+), 1 deletion(-)
> > > > > > > 
> > > > > > > diff --git a/meta/classes/base.bbclass
> > > > > > > b/meta/classes/base.bbclass
> > > > > > > index df11c8b270..5ffd8396e3 100644
> > > > > > > --- a/meta/classes/base.bbclass
> > > > > > > +++ b/meta/classes/base.bbclass
> > > > > > > @@ -303,7 +303,7 @@ base_do_configure() {
> > > > > > >                     if [ "${CLEANBROKEN}" != "1" -a \( -e
> > > > > > > Makefile
> > > > > > > -o -e makefile -o -e GNUmakefile \) ]; then
> > > > > > >                             oe_runmake clean
> > > > > > >                     fi
> > > > > > 
> > > > > > You might want to include a comment here explaining why rm
> > > > > > -f
> > > > > > is
> > > > > > used instead of -delete, or someone might come along and
> > > > > > optimize
> > > > > > the
> > > > > > code...
> > > > > 
> > > > > We should detail which versions of findutils have the issue
> > > > > as
> > > > > ultimately it would be nice to use the simpler command. I'm
> > > > > curious
> > > > > how
> > > > > many of the distros we support have the problem.
> > > > 
> > > > There's a test case in the patch proposed to fix the issue:
> > > > 
> > > >   https://savannah.gnu.org/bugs/?52981
> > > >   https://savannah.gnu.org/bugs/download.php?file_id=43119
> > > > 
> > > > Regarding the proposed patch for base.bbclass, xargs should
> > > > generally
> > > > always be called with --no-run-if-empty, unless there's a clear
> > > > reason
> > > > not to.
> > > 
> > > Do we generally assume that we have a GNU xargs with --no-run-if-
> > > empty
> > > support? Calling rm -f without further arguments at least doesn't
> > > cause
> > > an error, so --no-run-if-empty is only a minor optimization in
> > > this
> > > case.
> > 
> > 
> > I've grepped bitbake, openembedded-core and meta-openembedded for
> > --no-
> > run-if-empty, and there is only a single instance in the whole code
> > base - which is conditionalized so that the option is not passed on
> > Darwin. I take from this that --no-run-if-empty is not acceptable
> > in
> > OpenEmbedded to ensure portability.
> 
> Did you grep for the short form too, ie -r ?

Thanks for the pointer; including the short form, I've found a second
instance. Still, in the vast majority of xargs commands it is avoided,
so I guess I should the same to stay consistent with the existing
scripts.

> 
> > > 
> > > I'll send an updated patch with the proposed changes later today.
> > > 
> > > Kinds regards,
> > > Matthias
> > >