[oe] RFC: Improve our default conf file setup

[Chris Larson]

On Sat, Feb 20, 2010 at 3:58 PM, Richard Purdie <rpurdie at rpsys.net> wrote:

> On Fri, 2010-02-19 at 18:39 -0700, Chris Larson wrote:
> > On Thu, Feb 18, 2010 at 9:58 AM, Richard Purdie <rpurdie at rpsys.net>
> wrote:
> > > On Thu, 2010-02-18 at 09:13 -0700, Chris Larson wrote:
> > > > With COLLECTIONS (and BBLAYERS could work similarly), all the overlay
> > > > priorities and bbpath ordering are automatically determined based on
> the
> > > > order of entries in the variable.  Documented as highest-to-lowest.
> > > >
> > > > Note that I'm not necessarily arguing against the use of a per-layer
> > > > configuration file, as this seems like it would be quite useful, but
> I
> > > > question the need for the boilerplate.
> > >
> > > Its certainly possible to put defaults in which would handle the
> > > "standard" case. My main dislike is having to hardcode behaviour and
> > > variable handing into bitbake in that case.
> >
> > Not hardcoding *any* behavior into bitbake is how we got into the mess
> that
> > we're in,
>
> In many ways, its helped make bitbake what it is IMO. Bitbake and OE
> have a fairly clearly modularised structure where you can replace
> individual elements and this is a major attraction of it. Yes it causes
> some pain at times but it also makes many things possible too.
>
> >  and is likely the source of a large amount of confusion on the
> > part of our userbase, and is likely the cause of some of the performance
> > issues as well.  Flexibility is good, but there's a point at which the
> > benefits no longer outweigh the drawbacks.
>
> The performance issues seem to stem from two things, the anonymous
> methods and the way our data store works as far as I can tell. We could
> do with finding a better way to handle the methods I agree but they also
> have massive benefits too.
>
> As for confusion of the userbase, this i still more a documentation
> problem. Those who really understand it don't document it. The
> documentation I have written is for Poky and few people bother to read
> it, even if most of it is generic :(. Starting to hardcode behaviour is
> unlikely to lead to much improvement for the users IMO.
>
> > > The idea that adding an empty layer.conf file breaks a previously
> > > working overlay seems counter intuitive too...
> > >
> >
> > The layer itself deciding where it goes in the BBPATH is not obvious or
> nice
> > or clear to me the way it seems to be to you.  How can a given layer
> claim
> > to know where it should go in the bbpath, what its priority should be?
>
> Think about how people are going to use layers. Its never likely that a
> random collection of layers are going to work together without some
> (possibly minor) effort. Layers are going to end up advertised as being
> enhancements for OE, Poky or some specific choice.
>
> The "policy" on where in BBPATH (prepend or append) and actual priority
> numbers are going to be a policy decision for those projects, just like
> the bitbake.conf.
>
> I'm quite happy if a standard emerges. My point is that policy should
> not be in bitbake if we can help it.
>
> >   I
> > think it makes a great deal more sense to let the order/information in
> > bblayers.conf define that, as the user has direct control over that.  Can
> > you think of any cases where a given layer could possibly know better
> than
> > the user, or where defining the priority via BBLAYERS is insufficient?
> >  Regarding BBFILES, there's nothing to hardcode.  Add the layer to
> BBFILES,
> > let bitbake find the recipes, and you have BBMASK to remove the ones you
> > don't want.
>
> I disagree but its pointless to take this further and spoil what
> otherwise is a useful patch, we'll just add the default behaviour you
> describe.



I see a few problems here.  First, there's more to a notion of priority than
just 'prepend' and 'append'.  It's unlikely that a given layer will either
way to override everything else, or be overridden by everything else.
 Second, the final resulting BBPATH will end up depending upon ordering
anyway, because the prepend/appends will be applied based on the order of
the parsing of the bblayer.conf files.  Third, you seem to be retaining the
separation between configuration file / class priority (BBPATH) and
collection priority (BBFILE_COLLECTIONS, priority number).  Users see a
layer/overlay/collection as a single entity, not two components whose
interactions with everyone else may vary.

If you're going to use a priority numbering scheme or something similar,
defined by the layer, then everything should obey it, in my opinion, and
BBPATH_prepend/append is not sufficient to reflect that priority scheme.
-- 
Christopher Larson
clarson at kergoth dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Maintainer - Tslib
Senior Software Engineer, Mentor Graphics