[oe] Build looping with "Checking sstate mirror object availability...done"?

[Alex Kiernan]

On Wed, Dec 11, 2019 at 5:31 AM Alex Kiernan <alex.kiernan at gmail.com> wrote:
>
> Our local master jenkins build looks like it's started looping with:
>
> 00:56:28.368  Checking sstate mirror object availability...done.
> 00:56:28.368  Checking sstate mirror object availability...done.
> 00:56:28.368  Checking sstate mirror object availability...done.
> 00:56:28.752  Checking sstate mirror object availability...done.
> 00:56:28.752  Checking sstate mirror object availability...done.
> 00:56:29.136  Checking sstate mirror object availability...done.
> 00:56:29.136  Checking sstate mirror object availability...done.
> 00:56:29.136  Checking sstate mirror object availability...done.
> 00:56:29.520  Checking sstate mirror object availability...done.
> 00:56:29.520  Checking sstate mirror object availability...done.
> 00:56:29.904  Checking sstate mirror object availability...done.
> 00:56:29.904  Checking sstate mirror object availability...done.
> 00:56:29.904  Checking sstate mirror object availability...done.
> 00:56:30.288  Checking sstate mirror object availability...done.
> 00:56:30.288  Checking sstate mirror object availability...done.
>
> The only changes from it's previous build are these two:
>
> Layers Update: poky
> * poky 9d606cf...f48ef0b (2):
>  > build-appliance-image: Update to master head revision
>  > sstate: Ensure SSTATE_PKG is reloaded when handling siginfo
>
> Its sstate-cache is on NFS (well AWS EFS) if that's relevant?
>
> At the point I killed it, it was just over an hour in, but was only a
> 100 tasks into the 7392 - a full rebuild is normally ~3.5 hours, but
> this didn't seem to really be making any meaningful progress.
>

Just to fill this back in, it completed after ~8 hours, looks like it
was something like a mass read/modify/write (which over NFS I'm not
surprised was awful) and we're now back where we were in terms of
build times.

-- 
Alex Kiernan