[OE-core] [PATCH] sstate: Add sstate usage summary to the build

[richard.purdie at linuxfoundation.org]

On Fri, 2018-08-24 at 23:24 +0200, Martin Jansa wrote:
> This is nice addition, but maybe it would be worth explaining
> somewhere what these numbers actually mean.
> 
> e.g. my current build shows:
> Sstate summary: Wanted 181 Found 39 Missed 142 Current 279 (21%
> match, 69% complete)
> 
> From quick look at the code my understanding is that:
> 1) Wanted = Found + Missed
> 2) Wanted is number of sstate covered tasks needed for current build,
> Found is how many of them will be covered by existing sstate (fetched
> from SSTATE_MIRROR or local) and match is just Found/Wanted
> 3) Current is most confusing for me as it is a sum of tasks covered
> either by normal stamps or setscene and complete is then percentage
> of the tasks which either doesn't need to be executed at all or will
> be covered by sstate

Wanted is the number of artefacts the build ideally wants from sstate

Found is the number of artefacts present in sstate locally or it thinks
are fetchable from any mirrors

Missed = Wanted - Found

Current is the number of stamps which bitbake viewed as "current", i.e.
it needed to do nothing. Since they're current, it wouldn't bother
querying sstate or considering them any further.

"% match" tells you how much of the sstate queried, matched

"% complete" tells you how much of the build is already current or
completed

The reason the current/complete numbers are there are because a build
directory where a build has already partly completed could show a "0%
match" against sstate but its not immediately clear that things which
have already run are never checked for in sstate. A build which was 90%
complete would likely have a high "% match" for a fresh tmpdir so this
was added to try and give people a real idea of how much work remains
in a given build "up front" when you start the build.

Does that help explain it? I'm open to better terminology but its hard
to find something short and simple which conveys this...

> 4) Neither of these is about not-sstate-covered tasks which for this
> build was 1983 and 1574 were already covered by stamp files (as shown
> at the end of the build shown bellow)

The % current is close. 1574/1983 = 79% which is approx equal (but
greater than) 69%. The difference is that 69% is before any sstate was
fetched, there was a 21% sstate match which meant X tasks were then
covered. Therefore after setscene ran, the number of completed tasks
would be higher and at 1574. We couldn't easily compute 1574 in
advance.

>Am I close?
> 
> I'm used to meta/classes/buildstats-summary.bbclass output
> (unfortunately after the build) which just shows 3 numbers per sstate
> covered task and this new summary is a bit confusing for me.

People wanted to know about coverage in advance of the build running
and we have a different set of numbers to work with unfortunately.
Statistics afterwards are much easier! :)

I'm open to better ways of presenting it to make it clearer what it
means but we don't really want to print too much either. It really is
meant so you can tell at a glance roughly how much work a build will
need to do.

Cheers,

Richard