[OE-core] Long delays with latest bitbake (was: [PATCH 1/7] insane.bbclass: in file-rdeps do not look into RDEPENDS recursively)

[Richard Purdie]

On Mon, 2019-08-12 at 20:26 +0000, Peter Kjellerstedt wrote:
> Comparing that build to a corresponding do-nothing build with Thud,
> the time difference matches those three minutes where I have no idea
> what bitbake is doing now that it didn’t need to do before…
>  
> Hopefully these time degradations can be solved, because the current
> state of bitbake is barely usable. We also need to look into possible
> ways to improve the cooker output when it is running the setscene
> tasks so it makes some kind of sense again.

We talked on irc and you pointed at the commit things started to go
wrong. Just to summarise things for the benefit of the list, this is
some quick testing I did:

"bitbake -p; time bitbake core-image-minimal -n"

30.0s 6c7c0cefd34067311144a1d4c01986fe0a4aef26
30.6s a0d941c787cf3ef030d190903279d311bc05d752
40.3s 7df31ff36892c2f9c65326b06b4c70 
42.2s a0542ed3ff700eca35f9195f743c9e28bcd50f3e 
45.4s 9983b07fffd19082abded7c3f15cc77d306dd69c 
76.9s master-next

So basically the original changes showed a 25% hit but the performance
of -next is dire.

This is with no hash equiv server configured.

It will vary depending on the target used (numbers with -sato for the
above would be interesting for comparision) and how much was or is in
sstate, they type of sstate mirror configured and so on.

I really need to focus on getting the new code functioning correctly
before we attempt to optimise but if nobody tests the new code due to
performance problems we have a different issue. We also have a scaling
problem with the hash server itself I need to fix to stop the
autobuilder throwing weird errors. I'm therefore a bit challenged on
where to start with it all :/.

Cheers,

Richard