[bitbake-devel] [PATCH] use multiple processes to dump signatures.

[Richard Purdie]

On Thu, 2017-01-12 at 11:33 -0800, Jianxun Zhang wrote:
> I appreciate your multiple hints on the road for this issue! I do
> _feel_ this patch needs some improvement or tests, but I cannot
> identify them in my sight. Well, things shouldn’t be such easy with a
> V1 due to the complexity.

I think your testing was fine, I just wanted to double check a few
things with the patch which I've now done and things seem fine.

> That’s why I have to hand over the burden to you. :-) Feel free to
> let me know any new possibilities.
> 
> I am actually interested in the data you shared. I improved my sig-
> dumping over 10 times (14.4 sec to 1 sec). Your test should covers
> other steps, so the improvement on sig dumping is less significant,
> 325.08 to 93.93 sec.
> 
> Maybe we should find some time to investigate any room to speed up
> the rest in test suite...

I knew that improvement would help this set of tests significantly and
230s or ~4 minutes is a good saving to have per test. oe-selftest
currently takes up to around 8 hours and if we can knock 4 minutes off
say 10 tests that is a good saving! :)

I very much agree that we need to look at other ways of speeding up oe-
selftest. I actually found another problem Ross sent a patch for
earlier today which improve the test load time significantly and should
take another ~10mins off the 8 hours and makes running this manually
less annoying. We also have plans to use memory resident bitbake to
help. There are lots of places in oe-selftest it runs "bitbake -e XXX"
to get variables an the time taken to do this mounts up quickly. If
bitbake is memory resident, we should be able to get answers much
faster.

There is also the option of parallelising oe-selftest too, currently
the tests run in series but running in parallel would help too.

So plenty more work to do but this is a good start (and there are
several other use cases for this codepath I'm happy to see speeded up
too!).

Cheers,

Richard