[Openembedded-architecture] bblayers.conf - Past, Present and Future?

[Mark Hatle]

On 5/11/16 4:47 PM, Philip Balister wrote:
> Richard, thanks for kicking this off.
> 
> I've read the other replies in the thread and see there is a very
> diverse set of solutions to this problem.
> 
> Before we worry about solutions, we should figure out what we have in
> common. My mind is coming back to an earlier thread (maybe started by
> Trevor) about how we describe a build. Does this seem like a good place
> to start?

I think that is a good start.

>From what I've read.  I'd suggest:

1) Fetching of layers (initial and update)
2) Identifying the characteristics of the fetch (commit, branch, etc)
3) Setting up the bblayers.conf file
4) Setting up the local.conf file

(I know that is a bit simplistic, but I think captures the high level view.)

> Once we know how to describe a build, then we can think about how to use
> this over the entire user base. So, if we could agree on one file to
> rule them all, then people can develop tooling based on their specific
> needs.


> Philip
> 
> On 05/11/2016 07:42 AM, Richard Purdie wrote:
>> We currently have a few different ways people setup and build
>> combinations of OE layers. Koen's presentation and ELC covers some of
>> this, we've also heard representations at OEDEM/OEDAM and I think
>> everyone would agree that its worth seeing if there is some way we
>> could improve the situation.
>>
>> We have various tools/options:
>>
>> * TEMPLATECONF
>> * combo-layer
>> * repo
>> * git submodules
>>
>> and we all have views, some of them quite strong about different
>> elements of these.
>>
>> At the end of the day, our users want/need something which sets up the
>> system for them to use and "works", they don't really care about the
>> details. 
>>
>> To summarise the general problem, we need some way of saying "get this
>> set of git repositories and check them out in this layout, then setup a
>> bblayers.conf file like this".
>>
>> In parallel, we're seeing people trying to use OE in CI systems. To
>> make buildbot work for the Yocto Project, we have a ton of
>> configuration and "fetching" code and I know Jenkins integration runs
>> into the same issues. We'd love to be able to throw away the need to
>> have CI specific layer addition code, or the bulk of the custom git
>> fetcher we ended up having to add there for example.
>>
>> The key question is perhaps "how this should work?" and then we could
>> work backwards from there.
>>
>> The starting point is probably to have bitbake (as in the repo) and a
>> configuration file available. A command like "bitbake-setup
>> <configfile>", perhaps with a way to specify a default configfile would
>> perhaps be the starting point. As a minimum this would then need to
>> fetch the appropriate git repos (if not already present) and generate a
>> bblayers.conf file.
>>
>> Having a "bitbake-setup update" command to pull in changes from
>> upstream would also seem appropriate. For git cloned layers, this is
>> simple, it would also need to have a way to understand how to make
>> changes to the original configuration file and pull in changes to
>> bitbake itself. This would likely depend on the way the original
>> starting point was created and is where some of the complexity would
>> start to creep in but it should be manageable.
>>
>> One good thing is we already have fetcher functionality within bitbake
>> so we can comparatively easily have a url syntax which would support a
>> wide range of layer fetching/checkout needs.
>>
>> The main dilemma would be the format of the configfile. Do we use a
>> bblayers.conf.sample style format, extended to allow for clone
>> urls/checkout locations and so on? Do we create a new format .conf file
>> which could generate a bblayers.conf in bitbake's "conf" format? or do
>> we create something like a json file, similar to what toaster ended up
>> doing to partially solve some of this problem? I'm actually torn, part
>> of likes the idea of a json file, part of me says we should try and be
>> consistent to our current file format.
>>
>> I'm going to put this idea out here without going more into the
>> details, I'd like to see what people think. Is this a direction we
>> should look further into? Would a "bitbake-setup" solve some of our
>> problems? Is there a better way to proceed?
>>
>> Cheers,
>>
>> Richard
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> _______________________________________________
>> Openembedded-architecture mailing list
>> Openembedded-architecture at lists.openembedded.org
>> http://lists.openembedded.org/mailman/listinfo/openembedded-architecture
>>
> _______________________________________________
> Openembedded-architecture mailing list
> Openembedded-architecture at lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-architecture
>