[OE-core] [PATCH] base.bbclass: avoid 'find -ignore_readdir_race -delete'

[Matthias Schiffer]

On Mon, 2018-11-05 at 09:36 +0100, Matthias Schiffer wrote:
> On Fri, 2018-11-02 at 20:28 -0700, Andre McCurdy wrote:
> > On Fri, Nov 2, 2018 at 8:43 AM Richard Purdie
> > <richard.purdie at linuxfoundation.org> wrote:
> > > 
> > > On Fri, 2018-11-02 at 15:19 +0000, Peter Kjellerstedt wrote:
> > > > > -----Original Message-----
> > > > > From: openembedded-core-bounces at lists.openembedded.org
> > > > > <openembedded-
> > > > > core-bounces at lists.openembedded.org> On Behalf Of
> > > > > matthias.schiffer at ew.tq-group.com
> > > > > Sent: den 2 november 2018 15:39
> > > > > To: openembedded-core at lists.openembedded.org
> > > > > Subject: [OE-core] [PATCH] base.bbclass: avoid 'find -
> > > > > ignore_readdir_race -delete'
> > > > > 
> > > > > From: Matthias Schiffer <matthias.schiffer at ew.tq-group.com>
> > > > > 
> > > > > Until recent versions of findutils, -ignore_readdir_race did
> > > > > not
> > > > > work
> > > > > correctly with -delete. This led to spurious build failures
> > > > > when
> > > > > files
> > > > > disappeared; specifically this was seen due to do_configure
> > > > > and
> > > > > do_populate_lic running concurrently for packages with ${B}
> > > > > ==
> > > > > ${WORKDIR}:
> > > > > 
> > > > >    find: '.../sstate-build-populate_lic': No such file or
> > > > > directory
> > > > > 
> > > > > As the find command of the host system is used here, we can't
> > > > > ensure that
> > > > > the used version contains the fix. Work around the issue by
> > > > > passing
> > > > > the
> > > > > output of find to 'rm -f' instead of using -delete.
> > > > > 
> > > > > Signed-off-by: Matthias Schiffer <
> > > > > matthias.schiffer at ew.tq-group.com
> > > > > > 
> > > > > 
> > > > > ---
> > > > > 
> > > > > It would be great if this patch could be backported to all
> > > > > versions
> > > > > back to Rocko, as our current setup is still based on Rocko.
> > > > > 
> > > > >  meta/classes/base.bbclass | 2 +-
> > > > >  1 file changed, 1 insertion(+), 1 deletion(-)
> > > > > 
> > > > > diff --git a/meta/classes/base.bbclass
> > > > > b/meta/classes/base.bbclass
> > > > > index df11c8b270..5ffd8396e3 100644
> > > > > --- a/meta/classes/base.bbclass
> > > > > +++ b/meta/classes/base.bbclass
> > > > > @@ -303,7 +303,7 @@ base_do_configure() {
> > > > >                     if [ "${CLEANBROKEN}" != "1" -a \( -e
> > > > > Makefile
> > > > > -o -e makefile -o -e GNUmakefile \) ]; then
> > > > >                             oe_runmake clean
> > > > >                     fi
> > > > 
> > > > You might want to include a comment here explaining why rm -f
> > > > is
> > > > used instead of -delete, or someone might come along and
> > > > optimize
> > > > the
> > > > code...
> > > 
> > > We should detail which versions of findutils have the issue as
> > > ultimately it would be nice to use the simpler command. I'm
> > > curious
> > > how
> > > many of the distros we support have the problem.
> > 
> > There's a test case in the patch proposed to fix the issue:
> > 
> >   https://savannah.gnu.org/bugs/?52981
> >   https://savannah.gnu.org/bugs/download.php?file_id=43119
> > 
> > Regarding the proposed patch for base.bbclass, xargs should
> > generally
> > always be called with --no-run-if-empty, unless there's a clear
> > reason
> > not to.
> 
> Do we generally assume that we have a GNU xargs with --no-run-if-
> empty
> support? Calling rm -f without further arguments at least doesn't
> cause
> an error, so --no-run-if-empty is only a minor optimization in this
> case.


I've grepped bitbake, openembedded-core and meta-openembedded for --no-
run-if-empty, and there is only a single instance in the whole code
base - which is conditionalized so that the option is not passed on
Darwin. I take from this that --no-run-if-empty is not acceptable in
OpenEmbedded to ensure portability.

> 
> I'll send an updated patch with the proposed changes later today.
> 
> Kinds regards,
> Matthias
>