[Openembedded-architecture] bblayers.conf - Past, Present and Future?

[Mark Hatle]

On 5/11/16 6:42 AM, Richard Purdie wrote:
> We currently have a few different ways people setup and build
> combinations of OE layers. Koen's presentation and ELC covers some of
> this, we've also heard representations at OEDEM/OEDAM and I think
> everyone would agree that its worth seeing if there is some way we
> could improve the situation.
> 
> We have various tools/options:
> 
> * TEMPLATECONF
> * combo-layer
> * repo
> * git submodules
> 
> and we all have views, some of them quite strong about different
> elements of these.
> 
> At the end of the day, our users want/need something which sets up the
> system for them to use and "works", they don't really care about the
> details. 
> 
> To summarise the general problem, we need some way of saying "get this
> set of git repositories and check them out in this layout, then setup a
> bblayers.conf file like this".
> 
> In parallel, we're seeing people trying to use OE in CI systems. To
> make buildbot work for the Yocto Project, we have a ton of
> configuration and "fetching" code and I know Jenkins integration runs
> into the same issues. We'd love to be able to throw away the need to
> have CI specific layer addition code, or the bulk of the custom git
> fetcher we ended up having to add there for example.
> 
> The key question is perhaps "how this should work?" and then we could
> work backwards from there.
> 
> The starting point is probably to have bitbake (as in the repo) and a
> configuration file available. A command like "bitbake-setup
> <configfile>", perhaps with a way to specify a default configfile would
> perhaps be the starting point. As a minimum this would then need to
> fetch the appropriate git repos (if not already present) and generate a
> bblayers.conf file.
> 
> Having a "bitbake-setup update" command to pull in changes from
> upstream would also seem appropriate. For git cloned layers, this is
> simple, it would also need to have a way to understand how to make
> changes to the original configuration file and pull in changes to
> bitbake itself. This would likely depend on the way the original
> starting point was created and is where some of the complexity would
> start to creep in but it should be manageable.
> 
> One good thing is we already have fetcher functionality within bitbake
> so we can comparatively easily have a url syntax which would support a
> wide range of layer fetching/checkout needs.
> 
> The main dilemma would be the format of the configfile. Do we use a
> bblayers.conf.sample style format, extended to allow for clone
> urls/checkout locations and so on? Do we create a new format .conf file
> which could generate a bblayers.conf in bitbake's "conf" format? or do
> we create something like a json file, similar to what toaster ended up
> doing to partially solve some of this problem? I'm actually torn, part
> of likes the idea of a json file, part of me says we should try and be
> consistent to our current file format.

What I have wanted is to checkout bitbake (or bitbake & oe-core) and then setup
my bblayers.conf file with urls to all of the repositories I want.  On the first
invocation bitbake detects they are urls and not present locally and pulls them
down into a specific cache location (i.e. like the downloads dir).

I don't mind configuring the files, and in the end it all starts with the
bblayers.conf to me.  I think the tricky part is what about the local.conf file?
 Do we setup the (template?) file -after- pulling down the layers, or does there
need to be something configured before?

I'm not to enthusiastic about a json files in this case.  I think it's added
complications, while we already have a reasonable format in the bblayers.conf --
we just need to get the system to understand a more complex format [if it makes
sense].

> I'm going to put this idea out here without going more into the
> details, I'd like to see what people think. Is this a direction we
> should look further into? Would a "bitbake-setup" solve some of our
> problems? Is there a better way to proceed?

I'd like the process to start with the bblayers.conf file..  I'd have no
objections to a bitbake-setup that does the fetch behavior -- and then a
specific [manual] step after [if necessary] to do further configuration for what
was pulled down.

I think a key thing here, if the toaster can pull remote content, I want a
command line way to do the same... and I'd like that mechanism to work from the
beginning.

--Mark

> Cheers,
> 
> Richard
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> _______________________________________________
> Openembedded-architecture mailing list
> Openembedded-architecture at lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-architecture
>