[OE-core] Build Times and Performance - Discussion + new ideas

[Richard Purdie]

On Fri, 2011-09-30 at 01:11 +0000, McClintock Matthew-B29882 wrote:
> On Tue, Sep 27, 2011 at 12:07 PM, Richard Purdie
> <richard.purdie at linuxfoundation.org> wrote:
> > https://wiki.yoctoproject.org/wiki/Build_Performance
> >
> > As detailed there I've setup a partition formatted with ext4 with no
> > journal, no barriers and most importantly a long commit time. The intent
> > here is to allow things to exist in memory and not block on disk IO.
> 
> Can you comment on how much these settings effect build times vs.
> saying running on ext3 which our build server happens to be using?

I don't have absolute numbers but there are the following performance
limiting factors:

- journal - turned off means faster but more data risk
- ext4 has extents (which if nothing else allow deletion of build 
  directories faster). I suspect these are a big win for us given how 
  much direct file writing we do during unpack
- noatime changes - reduces amount of writes needed
- commit time changes - not forced to only keep 5 seconds of data in 
  memory maximum so less process blocking on IO.

You can do some of these things on ext3 (or just enable the features on
your existing ext3 disk and start using them, you don't need to
reformat).

If the disk is your system and build disk, I'd get more worried and
probably not recommend the changes. I'd recommend a separate build disk
where at all possible.

Cheers,

Richard