[OE-core] Improving Build Speed

[Richard Purdie]

On Thu, 2013-11-21 at 09:04 +0100, Ulf Samuelsson wrote:
> >>       Why restrict PARALLEL_MAKE to anything less than the number of H/W
> >> threads in the machine?
> >>
> >>       Came up with a construct PARALLEL_HIGH which is defined alongside
> >> PARALLEL_MAKE in conf/local.conf
> >>
> >>       PARALLEL_MAKE = "-j8"
> >>       PARALLEL_HIGH = "-j24"
> >>
> >>       In the appropriate recipes, which seems to be processed by bitbake
> >> in solitude I do:
> >>
> >>       PARALLEL_HIGH ?= "${PARALLEL_MAKE}"
> >>       PARALLEL_MAKE  = "${PARALLEL_HIGH}"
> >>
> >>       This means that they will try to use each H/W thread.
> > Please benchmark the difference. I suspect we can just set the high
> > number of make for everything. Note that few makefiles are well enough
> > written to benefit from high levels of make (webkit being an notable
> > exception).
> >
> It looks like it is shaving off  ~2 minutes from a build which normally 
> takes ~84 minutes.
> 
> First build
> PARALLEL_MAKE = "-j12"
> PARALLEL_HIGH = "-j24"
> BB_NUMBER_THREADS = "24"
> real    83m24.093s
> 
> Second build
> PARALLEL_MAKE = "-j12"
> PARALLEL_HIGH = "-j12"
> BB_NUMBER_THREADS = "24"
> real    85m12.007s

but what if you set both to -j24?

What I'm trying to understand is if we really need two different
variables?

Note you can also do:

PARALLEL_MAKE = "-j12"
PARALLEL_MAKE_pn-webkit-gtk = "-j24"

so I'm still not convinced we want to start having PARALLEL_HIGH as it
will just confuse users IMO.

Cheers,

Richard