[Openembedded-architecture] bblayers.conf - Past, Present and Future?

[Randy Witt]

On 05/11/2016 02:57 PM, Mark Hatle wrote:
> On 5/11/16 4:47 PM, Philip Balister wrote:
>> Richard, thanks for kicking this off.
>>
>> I've read the other replies in the thread and see there is a very
>> diverse set of solutions to this problem.
>>
>> Before we worry about solutions, we should figure out what we have in
>> common. My mind is coming back to an earlier thread (maybe started by
>> Trevor) about how we describe a build. Does this seem like a good place
>> to start?
>
> I think that is a good start.
>
> From what I've read.  I'd suggest:
>
> 1) Fetching of layers (initial and update)
> 2) Identifying the characteristics of the fetch (commit, branch, etc)
> 3) Setting up the bblayers.conf file
> 4) Setting up the local.conf file
>
> (I know that is a bit simplistic, but I think captures the high level view.)

This is why I created bug 
https://bugzilla.yoctoproject.org/show_bug.cgi?id=9588. If we can figure out 
what constitutes a "build" then we can come up with a reference consumer which 
is bug, https://bugzilla.yoctoproject.org/show_bug.cgi?id=9589.

I mention this because adding information to the bug would really be helpful.

>> Once we know how to describe a build, then we can think about how to use
>> this over the entire user base. So, if we could agree on one file to
>> rule them all, then people can develop tooling based on their specific
>> needs.
>
>
>> Philip
>>
>> On 05/11/2016 07:42 AM, Richard Purdie wrote:
>>> We currently have a few different ways people setup and build
>>> combinations of OE layers. Koen's presentation and ELC covers some of
>>> this, we've also heard representations at OEDEM/OEDAM and I think
>>> everyone would agree that its worth seeing if there is some way we
>>> could improve the situation.
>>>
>>> We have various tools/options:
>>>
>>> * TEMPLATECONF
>>> * combo-layer
>>> * repo
>>> * git submodules
>>>
>>> and we all have views, some of them quite strong about different
>>> elements of these.
>>>
>>> At the end of the day, our users want/need something which sets up the
>>> system for them to use and "works", they don't really care about the
>>> details.
>>>
>>> To summarise the general problem, we need some way of saying "get this
>>> set of git repositories and check them out in this layout, then setup a
>>> bblayers.conf file like this".
>>>
>>> In parallel, we're seeing people trying to use OE in CI systems. To
>>> make buildbot work for the Yocto Project, we have a ton of
>>> configuration and "fetching" code and I know Jenkins integration runs
>>> into the same issues. We'd love to be able to throw away the need to
>>> have CI specific layer addition code, or the bulk of the custom git
>>> fetcher we ended up having to add there for example.
>>>
>>> The key question is perhaps "how this should work?" and then we could
>>> work backwards from there.
>>>
>>> The starting point is probably to have bitbake (as in the repo) and a
>>> configuration file available. A command like "bitbake-setup
>>> <configfile>", perhaps with a way to specify a default configfile would
>>> perhaps be the starting point. As a minimum this would then need to
>>> fetch the appropriate git repos (if not already present) and generate a
>>> bblayers.conf file.
>>>
>>> Having a "bitbake-setup update" command to pull in changes from
>>> upstream would also seem appropriate. For git cloned layers, this is
>>> simple, it would also need to have a way to understand how to make
>>> changes to the original configuration file and pull in changes to
>>> bitbake itself. This would likely depend on the way the original
>>> starting point was created and is where some of the complexity would
>>> start to creep in but it should be manageable.
>>>
>>> One good thing is we already have fetcher functionality within bitbake
>>> so we can comparatively easily have a url syntax which would support a
>>> wide range of layer fetching/checkout needs.
>>>
>>> The main dilemma would be the format of the configfile. Do we use a
>>> bblayers.conf.sample style format, extended to allow for clone
>>> urls/checkout locations and so on? Do we create a new format .conf file
>>> which could generate a bblayers.conf in bitbake's "conf" format? or do
>>> we create something like a json file, similar to what toaster ended up
>>> doing to partially solve some of this problem? I'm actually torn, part
>>> of likes the idea of a json file, part of me says we should try and be
>>> consistent to our current file format.
>>>
>>> I'm going to put this idea out here without going more into the
>>> details, I'd like to see what people think. Is this a direction we
>>> should look further into? Would a "bitbake-setup" solve some of our
>>> problems? Is there a better way to proceed?
>>>
>>> Cheers,
>>>
>>> Richard
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> Openembedded-architecture mailing list
>>> Openembedded-architecture at lists.openembedded.org
>>> http://lists.openembedded.org/mailman/listinfo/openembedded-architecture
>>>
>> _______________________________________________
>> Openembedded-architecture mailing list
>> Openembedded-architecture at lists.openembedded.org
>> http://lists.openembedded.org/mailman/listinfo/openembedded-architecture
>>
>
> _______________________________________________
> Openembedded-architecture mailing list
> Openembedded-architecture at lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-architecture
>