[OE-core] [PATCH] local.conf.sample: Add automatic defaults for BB_NUMBER_THREADS and PARALLEL_MAKE

[Ulf Samuelsson]

1 feb 2014 kl. 10:21 skrev Mike Looijmans <mike.looijmans at topic.nl>:

> On 01/29/2014 01:56 PM, Richard Purdie wrote:
>> On Wed, 2014-01-29 at 13:09 +0100, Ulf Samuelsson wrote:
>>> We discussed this 2.3 months ago.
>>> Did some studies on my dual hex-core machine (24 H/W treads) while
>>> building a cloud9-gnome-image derivative.
>>> This did about 7500 tasks.
>>> 
>>> Enabled the CPU supervisors in the panel.
>>> 
>>> Everything seems to be ok with BB_NUMBER_THREADS = "24" for about 4-4500
>>> tasks.
>>> 
>>> Then the CPUs are mostly inactive and only 1-2 running for ~500 tasks.
>>> Then parallellism  is resumed until about task 7000, and again
>>> only a few CPUs are active.
>> 
>> This is likely whilst the lib and toolchain is getting built.
>> 
>>> I believe that some tools use "make" within the Makefile,
>>> and they are written badly, and do not use "-j <n>" for
>>> that part of the build.
>> 
>> Which recipes were building at this point? It would be interesting to
>> track them down.
> 
> I would suspect "recursive" makes. If you use autotools with the evil "SUDIRS=..." construction, it will WAIT for that dir to finish before doing anything else.
> It's much better to construct a giant makefile in the root, autotools will happily do that once properly instructed. I've seen first time builds going from half an hour to two minutes, and incremental builds taking only one or two seconds instead of several minutes just because I removed the recursion.
> 
>>> Got my build down to 83 minutes.
>>> 
>>> Since I have 96 GB of RAM, I tried creating an 80 GB tmpfs for the build,
>>> and copied the download and the recipes to the ram.
>>> 
>>> That shaved only 2 monutes from the build, and some stuff,
>>> still built using only a single CPU.
> 
> That confirms what I have already suspected - it is pointless to buy an SSD, building OE is mostly CPU limited and hardly I/O related.

To be fair, the 83 minute build was on a RAID with two striped 600 GB, 15k RPM, 3 Gbps SAS disks (luckily on the machine I got on eBay :-)
hdparm -t -T give about 390 MB/s.

If I run a single late model SATA-III disk (on a SATA-II) I get 120-150 MB/s
with the same test.

Using 15k RPM SAS disks will give 2 x the number of seeks per second
vs the SATA disks, (170 vs 85 IIRC)
Using RAID does not increase this value.

> 
> I guess the only way to really speed up the build would be to have multiple machines participate in it. Single machines aren't really getting any faster.
> 
>> There are certainly dependency bottlenecks in the build such as the
>> toolchain, compiler, gettext, gtk+ and so where large numbers of things
>> need those dependencies to get built before they can proceed. Not sure
>> what we can do to help this though.
> 
> Move them to the front as far as possible I guess. And any package they depend on as well... It should try to set up the shortest tree to be able to build the crosscompiler and build that first...
> 

If I understand things correctly, bitbake will calculate a priority for a certain package based on
how many packages are depending on this package.
If you could add a value to the priority, specified in the recipe
then you could make the build of the package occur earlier.

If bitbakes can track the loading of the CPU, then maybe some parts
in the later part of each recipe can be delayed until loading is low (maybe already doing this).

If the toolchain/native stuff could be pulled down from packages as part of a build
it would be real cool.
Today you can install cross compilers for certain Linux distributions.
Maybe there should be a possibility to generate automatically the .rpm/.deb etc.
and make that the default way to get stuff done for those distributions.

Best Regards
Ulf Samuelsson


> Mike.
> 
> 
> 
> 
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core at lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core