[bitbake-devel] Should multiconfig save build time?

[Alan Kozlay]

All,

In regards to our original June 28th post:

We're looking for things we can do to improve the multiconfig build time.

Even on our build servers it still takes a long time.
For instance, our build servers each have 64 core, 128G RAM with 4 NVMe SSD
HDs in a striped raid (*all configured with bells and whistles to make it
super fast*).
A full fresh build on one of those servers still takes 6 hours.

I need to also clarify that we do indeed define different DISTRO_FEATURES
between multiconfigs, so multiconfig is necessary.

(1) We're attempting to use the ccache (via ccache.bbclass), but it's
causing some build errors that we're investigating.
(2) Can we at least set the root native work directory to be the same for
all multiconfigs?

   - Would each multiconfig honor the previously built native tools?
   - Or, would it just rebuild them, anyway?
   - Is this safe or tested or supported?

(3) We're considering a network shared sstate-cache.  But it's not entirely
clear if a multiconfig build is benefited by the sstate-cache directory.

   - Under what circumstances does a multiconfig leverage sstate-cache when
   building for another multiconfig?
   - What if the recipe would be affected by a change in DISTRO_FEATURES
   between multiconfigs?
   - Are native packages included in the sstate-cache or just target
   packages?

We welcome all thoughts and ideas.

-Alan

On Sun, Jun 30, 2019 at 5:38 PM Richard Purdie <
richard.purdie at linuxfoundation.org> wrote:

> On Fri, 2019-06-28 at 13:31 -0400, Alan Kozlay wrote:
> > We have need to generate 6 slighly different images, with different
> > dedicated applications and use cases, but they're all destined for
> > the same hardware.
> > We have multiconfig set to build these 6 variants.  Clearly, they'll
> > all share a majority of underlying packages.
> > While the bitbake initialization takes a long time (with
> > multiconfig), we were under the impression that it could save time
> > during the actual build process because it could have the potential
> > to reuse packages already built from mc A for mc B if the inputs to
> > the recipes are unchanged (if same xxx_FEATURES, same arch, etc.).
> >
> > This was a guess on our part since we initially didn't assign
> > different tmp directories for each conf.  So, the <WORKDIR> didn't
> > change between mc builds.  That actually was causing some issues
> > until we realized that separate tmp dirs was recommended for them.
> > Since they all now have separate tmp dirs, they clearly are not going
> > to share work dirs, so, now we're questioning our original assumption
> > about any build-time speed benefit.
> >
> > So, is mc expected to be any faster than, say, 6 separate invocations
> > of non-mc bitbake whilst switching out the local.conf between them?
>
> Ultimately, yes, it would be expected to be faster.
>
> With the code as it stands today, no, it won't be. Its a known open
> issue:
>
> https://bugzilla.yoctoproject.org/show_bug.cgi?id=10682
>
> What would be faster is to build one of the configs initially which
> will populate the sstate cache, then build the other 5 together in one
> multiconfig.
>
> We obviously want to get that issue fixed ultimately but its not an
> easy thing to do due to the way our codebase works.
>
> Cheers,
>
> Richard
>
>

-- 
Alan Kozlay
VES Solutions, LLC
Alan Kozlay <alan.kozlay at ves.solutions>
DESK 443-595-7734
CELL 443-752-0323
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openembedded.org/pipermail/bitbake-devel/attachments/20190708/4c354173/attachment.html>