[Openembedded-architecture] bblayers.conf - Past, Present and Future?

[Richard Purdie]

On Wed, 2016-05-11 at 16:28 +0200, Jérémy Rosen wrote:
> As a "final user" of Yoco, I would be extremely interested in that 
> sort of tool.

Firstly, I'd like to say thanks for replying to this as end user
feedback is really helpful. In particular you've confirmed there that
there is a real world problem. That is something I'm struggling to
convince some others of.

> On 11/05/2016 13:42, Richard Purdie wrote:
> > We currently have a few different ways people setup and build
> > combinations of OE layers. Koen's presentation and ELC covers some
> > of
> > this, we've also heard representations at OEDEM/OEDAM and I think
> > everyone would agree that its worth seeing if there is some way we
> > could improve the situation.
> > We have various tools/options:
> > 
> > * TEMPLATECONF
> big advantage : it allows to set local.conf
> 
> most board makers have a huge readme with a dozen of changes you have
> to 
> do manually to your local.conf
> TEMPLATECONF is way less known that it deserves to be.
> 
> one of the (minor) drawback limitation of TEMPLATECONF is that it
> needs 
> a .templateconf file in the toplevel directory to
> point to an alternative local.conf
> The toplevel directory is usually coming from openembedde-core.git so
> you need to fork that repo if you want to automate the process
> completely

That is a valid point and is something I'd like to try and address but
I'm not sure how exactly.

We also have had significant issues updating some of these kinds of
files previously. I think we made sigificant progress in the recent
release in that regard and now have a lot more capability in how we
update config files.

Even if people don't use TEMPLATECONF, I'd like to encourage people to
take a look at those mechanisms as I believe them to be generally
useful.


> I think the big thing to remember here is that a tool like bitbake
> -setup 
> should not just have a .xml as a source, but also the possiblity to 
> fetch some files to tune the generated repo (bitbake infrastructure
> can 
> certainly help for that)

Agreed, setting up the layers is the first step but some kind of
configuration or customisation beyond that is often needed. Exactly how
we store and obtain that information is a good question.

> > * git submodules
> i'm not very fond of that one. Again, I stumble on the fact that my 
> toplevel yocto directory is already a git repo (poky.git or 
> openembedded-core.git) so I need to fork to handle submodules, which 
> complicates quality management quite a bit

I don't hear many people like submodules, I mainly mention them for
completeness.

> > Having a "bitbake-setup update" command to pull in changes from
> > upstream would also seem appropriate. For git cloned layers, this
> > is
> > simple, it would also need to have a way to understand how to make
> > changes to the original configuration file and pull in changes to
> > bitbake itself. This would likely depend on the way the original
> > starting point was created and is where some of the complexity
> > would
> > start to creep in but it should be manageable.
> there are two update path that I can see, both are usefull for
> different 
> use-cases
> 
> * update all the meta- to their latest version and test them (later, 
> generate a new conffile and push it)
> * re-fetch the conffile and update all meta- to that new version, 
> because upsttream has provided a new version and we are a user, not a
> yocto dev
> 
> since a git url can point to a commit or a branch, it might be
> possible 
> to include the first use-case into the second : if the conffile
> points 
> to a branch, update that branch. if it points to a commit or tag,
> update 
> to that commit or tag

Right, this could be in the form of some kind of update option and a
reset option for example.



> > One good thing is we already have fetcher functionality within
> > bitbake
> > so we can comparatively easily have a url syntax which would
> > support a
> > wide range of layer fetching/checkout needs.
> > 
> > The main dilemma would be the format of the configfile. Do we use a
> > bblayers.conf.sample style format, extended to allow for clone
> > urls/checkout locations and so on? Do we create a new format .conf
> > file
> > which could generate a bblayers.conf in bitbake's "conf" format? or
> > do
> > we create something like a json file, similar to what toaster ended
> > up
> > doing to partially solve some of this problem? I'm actually torn,
> > part
> > of likes the idea of a json file, part of me says we should try and
> > be
> > consistent to our current file format.
> no strong opinion, I like the yocto format with all its _append and 
> OVERRIDES etc... I don't know if it would fit that tool's need well,
> but 
> it avoids inventing a second syntax..

What we don't want to do here is invent yet another file format which
is why I compared with json which we do already use for one piece of
code. Personally I'd probably lean to the bitbake syntax since the user
will encounter it elsewhere in the system...

Thanks again for the reply, its nice to hear a different voice,
particularly with real world experience! :)

Cheers,

Richard