[bitbake-devel] [PATCH] runqueue: Add output for -S option for listing the changepoints compared with an sstate cache

[Martin Jansa]

On Wed, Jan 01, 2014 at 11:38:01PM +0000, Richard Purdie wrote:
> So can we find the "least delta" fast? Its not actually that hard
> computationally or on resources, at least in the experiments I've made.
> We know the hashes of the current target's tasks and we can quickly tell
> which are in the cache and which are not (using the same function sstate
> uses for that purpose).

I've finally got some numbers to support my claim that it "feels" slower
with this added functionality.

My script is calling
openembedded-core/scripts/sstate-diff-machines.sh --machines="qemuarm qemux86 qemux86copy" --targets=world --tmpdir=tmp-eglibc/;

which in turn runs
bitbake -S world
three times (once for each machine).

The script on jenkins doing world builds is completely crazy (I've
killed it in last run assuming it was stuck, but it wasn't just terribly
slow).
http://jenkins.nas-admin.org/job/oe_world_compare_signatures/11/
Running 21 hours already and doing 2nd machine from 3 (maybe because
longer SSTATE_MIRROR?). There is one python process using 100% cpu.

In local builds without SSTATE_MIRROR it's not so bad, but still
significantly slower with each release (dylan-dora-master).

Master builds have that WIP patch included in order to finish.

It's not very accurate, because it was running on my desktop and each one has
different metadata, but at least the layers included are similar (only
systems which got separated in dora and removal
of meta-webos-backports which isn't needed for dora and newer).

At least I've executed each test twice.

dylan:
real   9m37.271s
user   6m37.104s
sys    3m12.625s

real   10m5.787s
user   6m37.832s
sys    3m23.614s

dora:
real   26m11.374s
user   9m7.325s
sys    3m0.048s

real   28m10.606s
user   8m27.895s
sys    2m57.497s

master:
real   38m17.641s
user   17m46.905s
sys    5m12.061s

real    36m12.748s
user    16m32.961s
sys     4m43.103s

master-without-S-changes:
real    27m34.228s
user    8m6.811s
sys     2m53.724s

real    27m49.752s
user    8m25.026s
sys     2m50.921s

-- 
Martin 'JaMa' Jansa     jabber: Martin.Jansa at gmail.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature
URL: <http://lists.openembedded.org/pipermail/bitbake-devel/attachments/20140110/8c83566b/attachment-0002.sig>