[oe] RFC: SRC_URI_OVERIDES_PACKAGE_ARCH

[Richard Purdie]

Hi,

In trying to make SRCREV work efficiently I recently accidentally
changed the behaviour of SRC_URI_OVERIDES_PACKAGE_ARCH.

The old behaviour is that if SRC_URI_OVERIDES_PACKAGE_ARCH is unset (the
default) or set to anything other than 0 the auto PACKAGE_ARCH mechanism
is enabled.

This code goes through SRC_URI translating each URL into a local path
(where the fetched file would be located). It then checks if this local
path matches any standard MACHINE specific path. Examples are things
like:

PN/files/MACHINE/some_conf_file
PN-PV/MACHINE/some_conf_file
PN/MACHINE/some_conf_file

If its a machine specific path, the .bb file is marked as MACHINE
specific with PACKAGE_ARCH=MACHINE_ARCH.

I'd assumed this was opt in and changed the default case, thereby
breaking some recipes.

The problem is that translating SRC_URI to localpaths is computationally
expensive. In the SRCREV case, it involves hitting the network which is
very slow. If we use the localpath calls at all, we hit the database
which will slow down parsing.

We have several options:
 * restore the old behaviour, accept the parsing slowdown
 * limit the parsing to file:// urls only since they're the only ones 
   this applies to anyway. We avoid the network hits, we still hit the 
   database so see some slowdown
 * make it opt in, recipes which might use this feature have to set 
   SRC_URI_OVERIDES_PACKAGE_ARCH=1.
 
The whole OE <-> bitbake fetcher interaction and the associated caching
done in the fetcher needs some serious thought yet again. Ideally it
needs a total rewrite but I'm not sure I can do it whilst maintaining
backwards compatibility :-(.

For reference, I've been trying to postpone the parsing of SRC_URI until
the latest possible moment (fetch/unpack tasks) to try and avoid the
overhead.

I'll keep thinking but I'm open to opinions of the above options. How
much do we value this automatic handling over parsing speed?

Cheers,

Richard