[Openembedded-architecture] bblayers.conf - Past, Present and Future?

[Richard Purdie]

We currently have a few different ways people setup and build
combinations of OE layers. Koen's presentation and ELC covers some of
this, we've also heard representations at OEDEM/OEDAM and I think
everyone would agree that its worth seeing if there is some way we
could improve the situation.

We have various tools/options:

* TEMPLATECONF
* combo-layer
* repo
* git submodules

and we all have views, some of them quite strong about different
elements of these.

At the end of the day, our users want/need something which sets up the
system for them to use and "works", they don't really care about the
details. 

To summarise the general problem, we need some way of saying "get this
set of git repositories and check them out in this layout, then setup a
bblayers.conf file like this".

In parallel, we're seeing people trying to use OE in CI systems. To
make buildbot work for the Yocto Project, we have a ton of
configuration and "fetching" code and I know Jenkins integration runs
into the same issues. We'd love to be able to throw away the need to
have CI specific layer addition code, or the bulk of the custom git
fetcher we ended up having to add there for example.

The key question is perhaps "how this should work?" and then we could
work backwards from there.

The starting point is probably to have bitbake (as in the repo) and a
configuration file available. A command like "bitbake-setup
<configfile>", perhaps with a way to specify a default configfile would
perhaps be the starting point. As a minimum this would then need to
fetch the appropriate git repos (if not already present) and generate a
bblayers.conf file.

Having a "bitbake-setup update" command to pull in changes from
upstream would also seem appropriate. For git cloned layers, this is
simple, it would also need to have a way to understand how to make
changes to the original configuration file and pull in changes to
bitbake itself. This would likely depend on the way the original
starting point was created and is where some of the complexity would
start to creep in but it should be manageable.

One good thing is we already have fetcher functionality within bitbake
so we can comparatively easily have a url syntax which would support a
wide range of layer fetching/checkout needs.

The main dilemma would be the format of the configfile. Do we use a
bblayers.conf.sample style format, extended to allow for clone
urls/checkout locations and so on? Do we create a new format .conf file
which could generate a bblayers.conf in bitbake's "conf" format? or do
we create something like a json file, similar to what toaster ended up
doing to partially solve some of this problem? I'm actually torn, part
of likes the idea of a json file, part of me says we should try and be
consistent to our current file format.

I'm going to put this idea out here without going more into the
details, I'd like to see what people think. Is this a direction we
should look further into? Would a "bitbake-setup" solve some of our
problems? Is there a better way to proceed?

Cheers,

Richard