[bitbake-devel] [PATCH 2/2] runqueue: Introduce load balanced task spawning

[Andreas Müller]

On Tue, Aug 14, 2018 at 12:37 AM, Andre McCurdy <armccurdy at gmail.com> wrote:
> On Mon, Aug 13, 2018 at 2:30 PM, Andreas Müller <schnitzeltony at gmail.com> wrote:
>> On Mon, Aug 13, 2018 at 11:20 PM, Alexander Kanavin
>> <alex.kanavin at gmail.com> wrote:
>>> 2018-08-13 23:04 GMT+02:00 Andreas Müller <schnitzeltony at gmail.com>:
>>>> To get most out of build host, bitbake now detects if the CPU workload is low.
>>>> If so, additional tasks are spawned. Maximum 'dynamic' tasks are set by
>>>> BB_NUMBER_THREADS_LOW_CPU.
>>>
>>> So the best improvement is going from 114.5 minutes to 113.5 minutes?
>>> I don't think it's worth the trouble. Maybe it's time to invest in 16
>>> (or even 32!) core amd threadripper? :)
>>>
>> Indeed - but I have to wait till next holiday to assemble such kind of machine.
>>
>> I still can't believe the results are that disappointing...
>
> I wonder if you've experimented with the opposite approach, ie
> spawning fewer tasks when CPU load is very high? If a single
> do_compile task can fully load all CPUs, not running other tasks in
> parallel with it (especially another do_compile) might give some
> benefit?
>
> Dynamically boosting and dynamically lowering BB_NUMBER_THREADS based
> on overall CPU load both seem like logical things to do.

I think the patch does this of you interpret parameters differently.
So if you use BB_NUMBER_THREADS = 10 currently and want to go down to
2 in case heavy load set

BB_NUMBER_THREADS = 2
BB_NUMBER_THREADS_LOW_CPU = 10

Andreas