[Openembedded-architecture] layerindex upvoting Was: [OE-core] [oe] OE Changelog since 2016-05-15 until 2016-05-22

[Paul Eggleton]

(moving this to the architecture list where it belongs)

On Sat, 04 Jun 2016 17:11:49 Trevor Woerner wrote:
> On Fri 2016-05-27 @ 08:25:23 AM, Paul Eggleton wrote:
> > On Thu, 26 May 2016 22:18:34 Andreas Müller wrote:
> > > Maybe a bit off-topic but layer index should have some mechanism of
> > > removing rotten layers.
> > 
> > We've discussed this on-and-off at OE developer meetings for a while now.
> > FWIW I can remove them manually - actually I just got around to deleting
> > meta-picosam9 which has been unmaintained for years.
> > 
> > Part of the problem though is determining what "rotten" actually means.
> > Additionally, there's an element of leaving things around in case someone
> > comes to fork/revive them again. We do show a "last updated" date but it
> > would be good to figure out some other way of determining and indicating
> > "layer health" - to date I don't think we've managed to figure out
> > anything practical for that.
> 
> It's nice to see this topic come up. For the last couple OEDEMs and OEDAMs
> I'm usually the person to brings this up and it usually gets bandied about
> for a while. There's certainly lots of interest in general and from the
> toaster people in particular since old layers are likely to cause a build
> to fail which leaves a bad impression on newbies.

It's important for everyone, I think we all acknowledge we have a problem here 
- the difficult part is coming up with a practical solution.
 
> Currently the best idea we've come up with is to have an opt-in class that
> people can enable in their builds which will "phone home" to the layer index
> indicating what was build (target, distro) which which layers. Hopefully
> we'll think of a way of giving some sort of relevant status in the layer
> index; maybe with different coloured balls (a la Jenkins) or something to
> click on for the data. I.e. someone built a poky image for a raspi2 using
> master 2 days ago and it succeeded, but the 3 people who tried since have
> all seen failures. Maybe toaster could use this info somehow too.
>
> Ideally, as a start, we would only record nodistro, core-image-minimal
> builds. If people fed in their results with various branches and various
> (BSP) layers doing this build in a month we'd have something meaningful.

One caveat is that it's hard to tie a given build failure definitively to a 
particular layer, given that most people are likely to have fairly complex 
multi-layer setups even if they build with DISTRO="nodistro". I guess we'd 
just have to be willing to accept that any issues in dependent layers are also 
going to show up as failures in multiple layers, but I think to get anywhere 
with this it'll need to be the submitter who nominates the layer the failure 
should go against (not so much as a manual process for every failure, but more 
a case of "I'm testing this layer, so a failure here is a failure relating to 
that layer").

Something I think we need if we're do this properly is some uniformity around 
how we capture the information about failures (or for that matter, successes). 
We've already gone down this path quite a way with errors.yoctoproject.org in 
terms of capturing the information, presenting it, matching it up to similar 
errors, etc. Some minor problems there though:

 1) It doesn't capture the layers enabled in the configuration. Wouldn't be 
too hard to add though.

 2) There's no means of capturing a success - again, not hard to add assuming 
we adjust the default filters at the same time.

 3) Perhaps more seriously, there are types of error that it isn't possible to 
capture using an event handler in a class. For example, if there's something 
invalid in a layer's conf/layer.conf, it's way too early for classes to even 
be involved, bitbake just errors out immediately. There's a bugzilla entry 
tracking this:

  https://bugzilla.yoctoproject.org/show_bug.cgi?id=7583

I think we need to solve the problems with the error reporting above before 
proceeding here and then that avoids us duplicating that infrastructure. Once 
that's taken care of we could fairly easily add the basic ability for 
authorised users to submit statuses to be shown in the layer index (pass / 
fail and a reference to an errors.yoctoproject.org entry) and then have a 
select set of users submit their results and see how well it works before 
expanding it.

Cheers,
Paul


-- 

Paul Eggleton
Intel Open Source Technology Centre