[oe] [RFC] scalability

[Chris Larson]

On Mon, Jul 26, 2010 at 3:32 PM, C Michael Sundius <msundius at sundius.com>wrote:

> On Mon, Jul 26, 2010 at 2:39 PM, Chris Larson <clarson at kergoth.com> wrote:
>
> > The way bitbake works today, it can't build *anything* unless it's parsed
> > *everything*, because anything could have PROVIDES / RPROVIDES what
> you're
> > wanting to build.  If you want to shift things to a server, which I agree
> > would be nice, you'd have to have a daemon running there to make
> available
> > the full dependency information.
> >
> > Well, I suppose I could think of several other ways to do it. but yes, in
> a
> sense some external way of gaining the list of providers before you parse
> the recipe is the idea. for a big portion of the recipes that is simply
> done
> from looking at the name of the recipe, for the rest is grep-ing for
> "PROVIDES" in the file... .inc files makes it more complicated.
>

No.  Naming goes by the PN variable, which *happens* to default to based on
the filename, but isn't required.  There are recipes in the repository today
which override it, iirc.  You can't get the information you want without
parsing all the recipes, as I said before.

It seems to me that parsing the recipes == creating a database that provides
> info to bitbake. are you saying that this database is kept on a server for
> all to use? could parsing a single recipe mean creating a (partial)
> database
> who's contents could be added to another? or do they depend upon a
> context... my point is: could recipes be "pre-compiled" into ity-bity
> databses of sorts and then selected by a client and pulled in by a list
> (contained in your -local- distro conf file). Could these recipes once
> pulled in, be amended by local description?
>

You can already amend recipes with what we have today, in *multiple* ways.
 As I said before, you can't determine what you're going to build without
parsing all the recipes which are available.  What I'd personally like to
see is a tool which parses the world, then *imports* the metadata and
associated files of the recipe you want to build and all of its dependencies
into your local "project", which your bitbake then parses alone for
subsequent executions.  The problem with this, of course, is dealing with
local modifications to those files, and ensuring  there's an update
mechanism.  Maybe if I get bored I'll try throwing something like that
together -- it's quite like the 'newcollection' bbclass which already exists
in OE.
-- 
Christopher Larson
clarson at kergoth dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Maintainer - Tslib
Senior Software Engineer, Mentor Graphics