[bitbake-devel] [PATCH 2/2] runqueue: Introduce load balanced task spawning

[Andreas Müller]

On Wed, Aug 15, 2018 at 2:43 PM, Alexander Kanavin
<alex.kanavin at gmail.com> wrote:
> 2018-08-14 13:03 GMT+02:00 Alexander Kanavin <alex.kanavin at gmail.com>:
>> Both make and ninja have -l option:
>>
>>       -l [load], --load-average[=load]
>>             Specifies  that  no  new  jobs (commands) should be
>> started if there are others jobs running and the load average is at
>> least load (a floating-point number).  With no argument, removes a
>>             previous load limit.
>>
>>        -l N   do not start new jobs if the load average is greater than N
>>
>> Maybe that could be appended to PARALLEL_MAKE, which is far less
>> invasive than any other approach?
>
> I've done some tests, and yes -l does help. We currently have a nasty
> quadratic growth rate with cpu cores as input, which hits especially
> badly when the amount of cores is high, and a lot of long, heavy (e.g.
> c++) do_compile tasks run at once. This potentially means n*n compiler
> instances, where n is how many cpu cores are available. '-l' option
> does neatly limit that to a constant amount of compilers per core.
>
> However, this does not solve the other resource problem: running out
> of available RAM and going into swap thrashing. Neither make nor ninja
> can currently watch the RAM, even though it is not complicated:
>>>> import psutil
>>>> psutil.virtual_memory()
> svmem(total=16536903680, available=7968600064, percent=51.8,
> used=16347615232, free=189288448, active=11750494208,
> inactive=2882383872, buffers=3158528000, cached=4620783616)
>
> I think we should teach both to do that, and then replace a static
> 'number of jobs' limit in PARALLEL_MAKE with limits on CPU load and
> RAM usage.
>
> Alex
1. I like -l - have to try either!!
2. Quadratic explosion: I think it you should reduce number of
parallel bitbake threads. From what I have seen: When running heavy
compiles with -j = number-of-cores it takes only 2-3 do_compiles to
have CPU load at constant ~100%. I think this should be independent of
number of cores. Try 4-5 parallel bitbake threads - I bet that speeds
up your builds and reduces swap floods.

Andreas