[bitbake-devel] [PATCH] runqueue: Add output for -S option for listing the changepoints compared with an sstate cache

[Richard Purdie]

On Wed, 2014-01-01 at 23:38 +0000, Richard Purdie wrote:
> Nothing changed in the data -S writes out. 
> 
> Yes, it does currently crash in some cases which is obviously a bug
> however assuming we fix that you can ignore what -S prints on the screen
> or writes to a file or whatever we end up doing and just do what you're
> doing now.
> 
> On the other hand we have a huge hole in the current model when you
> don't have the two environments to compare. You have the current build
> and you want to know why the cache isn't being used. That is a valid
> user question which shouldn't need another build to compare against.
> 
> So can we find the "least delta" fast? Its not actually that hard
> computationally or on resources, at least in the experiments I've made.
> We know the hashes of the current target's tasks and we can quickly tell
> which are in the cache and which are not (using the same function sstate
> uses for that purpose).
> 
> That gives you a set of delta points with the sstate cache but you also
> know the recipe name and other details about the target of interest and
> that they should have common parent tasks. Based on that information you
> can come up with a list of possibilities of least difference and sort
> those by date (which bitbake-diffsigs -t uses) or whatever other
> mechanism we end up choosing to display the data.
> 
> I agree its not always going to give useful information but its a start
> in the right direction and my local tests suggest its perhaps good
> enough to give the user the hints they want about cache reuse (or lack
> of).
> 
> I agree we're not there yet, equally, to be quite honest the current
> debug of sstate sucks and we need to do something about it. Whilst you
> and I may understand it and are able to debug issues, many users are
> struggling with it :(.

I appreciate its taken me a while to loop back around on this but I do
now have an idea. What I'm proposing we do is add an optional parameter
to -S. Without any parameter it would behave as it did classically.

With a parameter, it could trigger the current "debug sstate" behaviour
to stdout and it also allows it to trigger an sstate siggen specific
behaviour as appripriate. I have some prototype code with writes out a
"locked-sigs.inc" file for example, triggered from this same code block.

I think this should let us do more creative things with sstate (even
from the OE siggen class) yet also let it remain useful for different
people.

I don't have any code to do this yet but at least we have a plan
assuming no strong objections.

Cheers,

Richard