[OE-core] [PATCH] sstate: Add sstate usage summary to the build

[Martin Jansa]

This is nice addition, but maybe it would be worth explaining somewhere
what these numbers actually mean.

e.g. my current build shows:
Sstate summary: Wanted 181 Found 39 Missed 142 Current 279 (21% match, 69%
complete)

>From quick look at the code my understanding is that:
1) Wanted = Found + Missed
2) Wanted is number of sstate covered tasks needed for current build, Found
is how many of them will be covered by existing sstate (fetched from
SSTATE_MIRROR or local) and match is just Found/Wanted
3) Current is most confusing for me as it is a sum of tasks covered either
by normal stamps or setscene and complete is then percentage of the tasks
which either doesn't need to be executed at all or will be covered by sstate
4) Neither of these is about not-sstate-covered tasks which for this build
was 1983 and 1574 were already covered by stamp files (as shown at the end
of the build shown bellow)

Am I close?

I'm used to meta/classes/buildstats-summary.bbclass output (unfortunately
after the build) which just shows 3 numbers per sstate covered task and
this new summary is a bit confusing for me.

NOTE: Tasks Summary: Attempted 1983 tasks of which 1574 didn't need to be
rerun and 1 failed.
NOTE: Writing buildhistory
NOTE: Uploading the report
/OE/build/oe-core/tmp-glibc/log/error-report/error_report_20180824202134.txt
to errors.yoctoproject.org:80
NOTE: Report submitted: 200 Your entry can be found here:
http://errors.yoctoproject.org/Errors/Build/66557/
NOTE: Build completion summary:
NOTE:   do_populate_sysroot: 8.1% sstate reuse(3 setscene, 34 scratch)
NOTE:   do_package: 0.0% sstate reuse(0 setscene, 33 scratch)
NOTE:   do_packagedata: 0.0% sstate reuse(0 setscene, 32 scratch)
NOTE:   do_package_write_ipk: 0.0% sstate reuse(0 setscene, 32 scratch)
NOTE:   do_populate_lic: 0.0% sstate reuse(0 setscene, 1 scratch)


On Tue, Aug 7, 2018 at 4:49 PM Richard Purdie <
richard.purdie at linuxfoundation.org> wrote:

> Currently the user has no indication of how much sstate was already present
> or that would be used by the build. This change adds some summary
> information
> so that the user can see how much reuse is occurring. To fully work it
> needs some extra information from a recent bitbake commit but this is
> optional.
>
> When combined with bitbake --dry-run this feature can be used to check
> if sstate would be reused in a build.
>
> [YOCTO #12749]
>
> Signed-off-by: Richard Purdie <richard.purdie at linuxfoundation.org>
> ---
>  meta/classes/sstate.bbclass | 12 ++++++++++++
>  1 file changed, 12 insertions(+)
>
> diff --git a/meta/classes/sstate.bbclass b/meta/classes/sstate.bbclass
> index 350d3107f59..6743becf071 100644
> --- a/meta/classes/sstate.bbclass
> +++ b/meta/classes/sstate.bbclass
> @@ -892,6 +892,18 @@ def sstate_checkhashes(sq_fn, sq_task, sq_hash,
> sq_hashfn, d, siginfo=False):
>              evdata['found'].append( (sq_fn[task], sq_task[task],
> sq_hash[task], sstatefile ) )
>          bb.event.fire(bb.event.MetadataEvent("MissedSstate", evdata), d)
>
> +    # Print some summary statistics about the current task completion and
> how much sstate
> +    # reuse there was. Avoid divide by zero errors.
> +    total = len(sq_fn)
> +    currentcount = d.getVar("BB_SETSCENE_STAMPCURRENT_COUNT") or 0
> +    complete = 0
> +    if currentcount:
> +        complete = (len(ret) + currentcount) / (total + currentcount) *
> 100
> +    match = 0
> +    if total:
> +        match = len(ret) / total * 100
> +    bb.plain("Sstate summary: Wanted %d Found %d Missed %d Current %d
> (%d%% match, %d%% complete)" % (total, len(ret), len(missed), currentcount,
> match, complete))
> +
>      if hasattr(bb.parse.siggen, "checkhashes"):
>          bb.parse.siggen.checkhashes(missed, ret, sq_fn, sq_task, sq_hash,
> sq_hashfn, d)
>
> --
> 2.17.1
>
> --
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core at lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openembedded.org/pipermail/openembedded-core/attachments/20180824/895130a1/attachment-0002.html>