[OE-core] Accelerate do_package

[Richard Purdie]

On Fri, 2012-10-12 at 13:43 +0200, Samuel Stirtzel wrote:
> Hi,
> after re-building Qt the third time today, I was curious why
> do_package takes this long.

Try:

http://git.yoctoproject.org/cgit.cgi/poky-contrib/commit/?h=rpurdie/t1&id=e355769c1dc2dd5c464f2fcfe0991f797613b43e

The trouble is this "breaks" certain rpm per file dependency pieces and
I've not found time to dive into finding an alternative way to do that.

> From my investigation it looks like the library stripping is the culprit,
> it lets my system idle with 5% CPU usage over a long time.

Its the fork/exec overhead from the above that does this.

> Would it be possible to accelerate do_package with parallelisation?
> Likely I could look into this sometime soon, but I'd rather hear your
> thoughts on this first.

Parallelisation is good but there is a better piece of low hanging fruit
here.

> PS:
> Also sstate creation and rootfs packing could be made faster with
> pbzip2 [1] / pigz [2] (we have a recipe for pigz), or is anthing
> preventing this?

The trouble is circular dependencies and being consistent about which
util gets used where without races and dependency loops.

Its also not the big time sync people think it is, I tested a patch
which "offlined" the whole sstate archive creation and it doesn't make
much difference to overall build time.

Cheers,

Richard (not around much over the next few days)