[Openembedded-architecture] bblayers.conf - Past, Present and Future?

[Ioan-Adrian Ratiu]

On Thu, 12 May 2016 13:40:01 +0300
Ioan-Adrian Ratiu <adrian.ratiu at ni.com> wrote:

> Hello
> 
> Since no one from the git submodules camp answered, I'll take a stab at it.
> 
> On Wed, 11 May 2016 12:42:27 +0100
> Richard Purdie <richard.purdie at linuxfoundation.org> wrote:
> 
> > We currently have a few different ways people setup and build
> > combinations of OE layers. Koen's presentation and ELC covers some of
> > this, we've also heard representations at OEDEM/OEDAM and I think
> > everyone would agree that its worth seeing if there is some way we
> > could improve the situation.
> > 
> > We have various tools/options:
> > 
> > * TEMPLATECONF
> > * combo-layer
> > * repo
> > * git submodules
> > 
> > and we all have views, some of them quite strong about different
> > elements of these.
> > 
> > At the end of the day, our users want/need something which sets up the
> > system for them to use and "works", they don't really care about the
> > details. 
> > 
> > To summarise the general problem, we need some way of saying "get this
> > set of git repositories and check them out in this layout, then setup a
> > bblayers.conf file like this".
> > 
> > In parallel, we're seeing people trying to use OE in CI systems. To
> > make buildbot work for the Yocto Project, we have a ton of
> > configuration and "fetching" code and I know Jenkins integration runs
> > into the same issues. We'd love to be able to throw away the need to
> > have CI specific layer addition code, or the bulk of the custom git
> > fetcher we ended up having to add there for example.
> > 
> > The key question is perhaps "how this should work?" and then we could
> > work backwards from there.
> > 
> > The starting point is probably to have bitbake (as in the repo) and a
> > configuration file available. A command like "bitbake-setup
> > <configfile>", perhaps with a way to specify a default configfile would
> > perhaps be the starting point. As a minimum this would then need to
> > fetch the appropriate git repos (if not already present) and generate a
> > bblayers.conf file.
> > 
> > Having a "bitbake-setup update" command to pull in changes from
> > upstream would also seem appropriate. For git cloned layers, this is
> > simple, it would also need to have a way to understand how to make
> > changes to the original configuration file and pull in changes to
> > bitbake itself. This would likely depend on the way the original
> > starting point was created and is where some of the complexity would
> > start to creep in but it should be manageable.
> > 
> > One good thing is we already have fetcher functionality within bitbake
> > so we can comparatively easily have a url syntax which would support a
> > wide range of layer fetching/checkout needs.
> > 
> > The main dilemma would be the format of the configfile. Do we use a
> > bblayers.conf.sample style format, extended to allow for clone
> > urls/checkout locations and so on? Do we create a new format .conf file
> > which could generate a bblayers.conf in bitbake's "conf" format? or do
> > we create something like a json file, similar to what toaster ended up
> > doing to partially solve some of this problem? I'm actually torn, part
> > of likes the idea of a json file, part of me says we should try and be
> > consistent to our current file format.
> > 
> > I'm going to put this idea out here without going more into the
> > details, I'd like to see what people think. Is this a direction we
> > should look further into? Would a "bitbake-setup" solve some of our
> > problems? Is there a better way to proceed?  
> 
> Generating bblayers.conf would be useful because right now we use git
> submodules to fetch/manage/track layers (it works really well btw) but
> we hardcode bblayers.conf inside the base git repo which fetches the
> layer submodules. local.conf & other configs are created from templates
> when we call oe-init-build-env using a simple wrapper in which basically
> we set BUILDDIR, BITBAKEDIR and TEMPLATECONF.
> 
> So my main idea is to have the contents of bblayers.conf generated so I
> can remove that hardcoded file. Someone mentioned doing this from
> the "bitbake-setup" args - yes, that would work and be really neat. My
> ideal setup would not start from a pre-existing bblayers.conf file.

Of course, an alternative to those args would be to make bitbake-setup
aware of the submodules present through a location variable which
points to them. After all these are implementation details, so there are
a lot of different ways to do it.

Ioan-Adrian

> 
> I'm not too keen on bitbake fetching and managing layer repos because
> that's what git submodules are for, right? :)
> 
> Ioan-Adrian
> 
> > 
> > Cheers,
> > 
> > Richard
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > _______________________________________________
> > Openembedded-architecture mailing list
> > Openembedded-architecture at lists.openembedded.org
> > http://lists.openembedded.org/mailman/listinfo/openembedded-architecture  
>