[bitbake-devel] [PATCH] runqueue: Add output for -S option for listing the changepoints compared with an sstate cache

[Richard Purdie]

On Mon, 2013-12-30 at 15:34 -0700, Chris Larson wrote:
> I can see that, but I think you're making the semantics of the
> command-line argument confusing by trying to combine the two
> operations into one. I also think that trying to find the "least
> delta" is extremely weird, and not always likely to produce anything
> worthwhile. My SSTATE_DIR has weeks if not months of output from
> hundreds of builds of different combinations of machine, distro, layer
> selection, upstream vs mel, as well as between tons of build
> environments with slight local alterations. Unless I'm
> misunderstanding something, how exactly is it going to know what I
> want it to compare against, and how it is going to keep finding the
> "least delta" fast, amongst all that data? With the current tools, I
> can explicitly use -S in one env and whatchanged in another (with
> appropriate STAMPS_DIR) to compare between two exact build
> environments, rather than letting bitbake try to automagically find
> what I meant. Now that the two are combined into one, I'm not seeing
> how to do what I'm currently doing. Can you explain this?

Nothing changed in the data -S writes out. 

Yes, it does currently crash in some cases which is obviously a bug
however assuming we fix that you can ignore what -S prints on the screen
or writes to a file or whatever we end up doing and just do what you're
doing now.

On the other hand we have a huge hole in the current model when you
don't have the two environments to compare. You have the current build
and you want to know why the cache isn't being used. That is a valid
user question which shouldn't need another build to compare against.

So can we find the "least delta" fast? Its not actually that hard
computationally or on resources, at least in the experiments I've made.
We know the hashes of the current target's tasks and we can quickly tell
which are in the cache and which are not (using the same function sstate
uses for that purpose).

That gives you a set of delta points with the sstate cache but you also
know the recipe name and other details about the target of interest and
that they should have common parent tasks. Based on that information you
can come up with a list of possibilities of least difference and sort
those by date (which bitbake-diffsigs -t uses) or whatever other
mechanism we end up choosing to display the data.

I agree its not always going to give useful information but its a start
in the right direction and my local tests suggest its perhaps good
enough to give the user the hints they want about cache reuse (or lack
of).

I agree we're not there yet, equally, to be quite honest the current
debug of sstate sucks and we need to do something about it. Whilst you
and I may understand it and are able to debug issues, many users are
struggling with it :(.

Cheers,

Richard