[oe] Why is OE so slow?

[Gary Thomas]

I'm using OpenEmbedded to build Angstrom.  Recent trees
in the org.openembedded.dev branch have some 8370 recipes.
When I initiate a build, it takes *forever* just to get started.

I have [reasonably] powerful build machines (4 way 3GHz Zeon) and
it takes over 10 minutes CPU time just to reach 'Preparing runqueue'.
It then takes an additional 10 minutes before it starts to
execute the run queue.  Of course, I have the Psyco JIT installed.

* What takes so long?  Obviously, there are some processes in
   these steps which are non-linear.  I also build Poky which
   has about 10% of the recipes (~850) and these steps are *much*
   more than 10 times as fast.  In fact, building Poky, the first
   task in the run queue is usually started within 30 seconds...

   Also, the "cache" (whatever it holds) does not affect these times
   very much.  The times above are for a first run (rebuild the cache).
   If the cache does not have to be rebuilt, the first phase only takes
   5.5 minutes (still very long), while the second phase (preparing
   the run queue) still takes 10+ minutes (maybe longer now that it
   has to consider partial results from the working build tree)

* Is there any way to tune/improve this?  Is there something I've
   missed?  Why don't the online builders, e.g. narcissus, suffer
   such delays?

I'd really like to use OE more, but having to wait 25 minutes
just to get it going is [at least] off-putting.

-- 
------------------------------------------------------------
Gary Thomas                 |  Consulting for the
MLB Associates              |    Embedded world
------------------------------------------------------------