[bitbake-devel] Should multiconfig save build time?

[Richard Purdie]

On Mon, 2019-07-08 at 14:27 -0400, Alan Kozlay wrote:
> We're looking for things we can do to improve the multiconfig build
> time.
> 
> Even on our build servers it still takes a long time.
> For instance, our build servers each have 64 core, 128G RAM with 4
> NVMe SSD HDs in a striped raid (all configured with bells and
> whistles to make it super fast).
> A full fresh build on one of those servers still takes 6 hours.

Given we have no idea what these mulitconfigs are, how much overlap
there is or how you're running it, its hard to know whether 6 hours is
good or bad.

> I need to also clarify that we do indeed define different
> DISTRO_FEATURES between multiconfigs, so multiconfig is necessary.
> 
> (1) We're attempting to use the ccache (via ccache.bbclass), but it's
> causing some build errors that we're investigating.

That shouldn't cause errors and would likely be some ccache problem.

> (2) Can we at least set the root native work directory to be the same
> for all multiconfigs?

It depends if the configurations are different or not. WORKDIR isn't
your problem here, you really need the builds to have the same sysroot-
components dir for the native case too, else this won't work. Its more
about the output from the task than the directory they were built in.

> Would each multiconfig honor the previously built native tools?
> Or, would it just rebuild them, anyway?
> Is this safe or tested or supported?

multiconfig is configured so that if it sees overlapping stamp files it
would handle that correctly. The tricky part is you'd have to split
some directories which aren't usually split such as sysroot-components. 
I've never seen that attempted and am not sure if it would work or not.

> (3) We're considering a network shared sstate-cache.  But it's not
> entirely clear if a multiconfig build is benefited by the sstate-
> cache directory.
> Under what circumstances does a multiconfig leverage sstate-cache
> when building for another multiconfig?

Right now today for a single multiconfig build it won't.

> What if the recipe would be affected by a change in DISTRO_FEATURES
> between multiconfigs?
> Are native packages included in the sstate-cache or just target
> packages?

native and target are included in sstate-cache.

> We welcome all thoughts and ideas.

With current bitbake you'd be best off splitting your multiconfig build
into two parts a first stage where you build common shared components
and a second where you leverage them.

I appreciate that isn't a good solution but its what will work right
now.

The initial work in adapting bitbake to better handle situations like
this is being done, as we speak and there are patches starting it in
master-next.

Right now those patches also won't change anything for you, they re-
architect the codebase so we have a chance of implementing what you
want. It is however at least a start. I'm working on this effectively
alone with the day to day patch review, email and bugs to handle at the
same time :(. As a project we know this is important but we struggle to
find people willing to contribute development resources to make these
kinds of changes happen.

Next step is to write a regression test suite for runqueue. Once I've
done that I'll start to think about optimisation work.

Cheers,

Richard