[OE-core] [PATCH 0/2] Avoid build failures due to setscene errors

[Peter Kjellerstedt]

> -----Original Message-----
> From: Andre McCurdy [mailto:armccurdy at gmail.com]
> Sent: den 29 augusti 2017 22:38
> To: Peter Kjellerstedt <peter.kjellerstedt at axis.com>
> Cc: OE Core mailing list <openembedded-core at lists.openembedded.org>
> Subject: Re: [OE-core] [PATCH 0/2] Avoid build failures due to setscene
> errors
> 
> On Tue, Aug 29, 2017 at 1:00 PM, Peter Kjellerstedt
> <peter.kjellerstedt at axis.com> wrote:
> > Occasionally, we see errors on our autobuilders where a setscene task
> > fails to retrieve a file from our global sstate cache. It typically
> > looks something like this:
> >
> > WARNING: zip-3.0-r2 do_populate_sysroot_setscene: Failed to fetch URL
> > file://66/sstate:zip:core2-64-poky-linux:3.0:r2:core2-64:3:\
> > 66832b8c4e7babe0eac9d9579d1e2b6a_populate_sysroot.tgz;\
> > downloadfilename=66/sstate:zip:core2-64-poky-linux:3.0:r2:core2-
> 64:3:\
> > 66832b8c4e7babe0eac9d9579d1e2b6a_populate_sysroot.tgz, attempting
> > MIRRORS if available
> > ERROR: zip-3.0-r2 do_populate_sysroot_setscene: Fetcher failure:
> > Unable to find file
> > file://66/sstate:zip:core2-64-poky-linux:3.0:r2:core2-64:3:\
> > 66832b8c4e7babe0eac9d9579d1e2b6a_populate_sysroot.tgz;\
> > downloadfilename=66/sstate:zip:core2-64-poky-linux:3.0:r2:core2-
> 64:3:\
> > 66832b8c4e7babe0eac9d9579d1e2b6a_populate_sysroot.tgz anywhere. The
> > paths that were searched were:
> >     /home/pkj/.openembedded/sstate-cache
> 
> To trigger this, do you have SSTATE_MIRRORS pointing to
> "/home/pkj/.openembedded/sstate-cache" and SSTATE_DIR pointed
> somewhere else? Or are they both pointing to the same local directory?
> Or something else?

No, the directory above is actually what is in SSTATE_DIR. 
SSTATE_MIRRORS is set to:

SSTATE_MIRRORS ?= "\
file://.* file:///n/oe/sstate-cache/PATH;downloadfilename=PATH"

where /n/oe is an NFS mount where we share a global sstate cache. 

The only way I have figured out to manually simulate the problem is 
by modifying the code in sstate_checkhashes() in sstate.bbclass and 
commenting out the call to fetcher.checkstatus(). Then as long as 
there actually is no sstate files for the task in either the global 
or the local sstate cache, I will get the above. 

I do not know what triggers it on the autobuilder though. My guess is 
that somehow the sstate tgz file disappears between the call to 
sstate_checkhashes() and when bitbake actually tries to download the 
file. 

We do have a daily job that cleans up the global sstate cache and 
removes files that have not been accessed in the last ten days, but 
it seems unlikely that it should remove a file that just happens to 
be required again, and do it at exactly the time when that task is 
building.

> > ERROR: zip-3.0-r2 do_populate_sysroot_setscene: No suitable staging
> > package found
> > WARNING: Setscene task
> > (meta/recipes-extended/zip/zip_3.0.bb:do_populate_sysroot_setscene)
> > failed with exit code '1' - real task will be run instead

//Peter