[OE-core] Improving Build Speed

[Enrico Scholz]

Ulf Samuelsson <angstrom-dev-AoFPY8dbyRPQT0dZR+AlfA at public.gmane.org>
writes:

> PARALLEL_MAKE = "-j6"
> BB_NUMBER_THREADS = "24"

I define

| PARALLEL_MAKE = "\
|   -j ${@int(os.sysconf(os.sysconf_names['SC_NPROCESSORS_ONLN'])) * 2} \
|   -l ${@int(os.sysconf(os.sysconf_names['SC_NPROCESSORS_ONLN'])) * 150/100} \
| "
| 
| BB_NUMBER_THREADS ?= "\
|   ${@int(os.sysconf(os.sysconf_names['SC_NPROCESSORS_ONLN'])) * 150/100}"

in my global configuration (note the '-l').  I would like to limit it by
the available RAM size (e.g. one -j per GB) but BB_NUMBER_THREADS makes
it difficultly to express it.  There are also dependencies on the used
filesystem (e.g. btrfs performance seems to degrade rapidly with higher
-j).

It would be perfect when bitbake takes the role of the toplevel
jobserver[1] but that's probably very difficultly to implement and
might interfere with recursive make.


> and was quicker, but it seemed to be a little flawed.  At several
> times during the build, the CPU frequtil showed that most of the cores
> went down to minimum frequency (2,93 GHz -> 1,6 GHz)

Capturing resource usage (--> getrusage(2)) will give more details
(e.g. about i/o load). E.g. see

             https://www.cvg.de/people/ensc/oe-metrics.html



Enrico