[oe] RFC: SRC_URI_OVERIDES_PACKAGE_ARCH

[Richard Purdie]

On Sat, 2007-08-11 at 15:11 +0200, Koen Kooi wrote:
> Richard Purdie schreef:
> > The problem is that translating SRC_URI to localpaths is computationally
> > expensive. In the SRCREV case, it involves hitting the network which is
> > very slow. If we use the localpath calls at all, we hit the database
> > which will slow down parsing.
> > 
> > We have several options:
> >  * restore the old behaviour, accept the parsing slowdown
> 
> If people want to know how much the slowdown is, add:
> 
> SRC_URI_OVERIDES_PACKAGE_ARCH = "1"
> 
> to your local.conf. On a dual opteron with U320 scsi disks it becomes unbearably slow.
> 
> I think checking only file:// paths is the best intermediate solution.

With bitbake 1.8.7 and SRC_URI_OVERIDES_PACKAGE_ARCH just set in recipes
that need it parsing OE.dev takes:

real    5m49.043s
user    4m35.649s
sys     0m11.057s

Whereas if we check file:// urls:

real    9m8.550s
user    5m55.542s
sys     0m45.847s

Its possible that machine was more loaded when I ran the second test but
it was reproducibly taking 9 mins vs 5.5 mins with just that change. Its
safe to say there is a 20%+ speed hit, probably worse.

For comparison bitbake 1.8.6 with SRC_URI_OVERIDES_PACKAGE_ARCH just set
in recipes that need it:

real    6m56.088s
user    4m50.558s
sys     0m11.933s

Whereas if we check file:// urls:

real    6m45.358s
user    4m52.466s
sys     0m12.225s

So we have what looks like a speedup in 1.8.7 if we can sort the
SRC_URI_OVERIDES_PACKAGE_ARCH problem...

I suspect I should throw away the sql based primary cache and go back to
the ugly global variable as its tons faster...

Cheers,

Richard