[Openembedded-architecture] Layer index duplicates tool

[Paul Eggleton]

On Mon, 17 Oct 2016 15:44:09 Sean Hudson wrote:
> On 10/17/2016 3:26 PM, Paul Eggleton wrote:
> > On Mon, 17 Oct 2016 15:16:55 Sean Hudson wrote:
> >> On 10/16/2016 2:34 PM, Paul Eggleton wrote:
> >>> At OEDEM I see in the minutes that there was a discussion about
> >>> duplicate recipes and that we should have a way of finding them.
> >>> 
> >>> I wanted to draw people's attention to the Duplicates function of
> >>> the layer index, which I suspect has still gone unnoticed even
> >>> though it's been there for quite some time. You need to create an
> >>> account and log in in order to access it; once you do you should
> >>> see a "Tools" drop-down at the top right from which you can
> >>> select "Duplicates".
> >>> 
> >>> By default it will show you duplicate recipes and classes across
> >>> all layers, but often that won't be what you want to see - for
> >>> example, meta-debian intentionally duplicates quite a few recipes
> >>> which makes the list longer than you'd want. If this is the case,
> >>> hit the "Filter layers" button in the top right to select the
> >>> layers you do want to see.
> >> 
> >> Thanks for the tip! Is that capability available via the REST API?
> >> One of the other parts of that discussion included identifying ways
> >> to add layer checks, e.g. finding duplicates, to automated builds.
> > 
> > It's not accessible via the API, but I would accept patches to add
> > it. Filing an enhancement bug would be a good first step.
> 
> I will make a note to create a bug and perhaps look at the code to send
> some patches.
> 
> > However "automated builds" implies that in some cases you might
> > actually be using a layer or revision of a layer that's not actually
> > the same as what the index knows about. If that's the scenario then
> > you may need to use something local to the build instead -
> > "bitbake-layers show-overlayed" will do that, but of course only
> > within the layers you have specified in bblayers.conf.
> 
> Good point.  I will need to think through the scenario a bit more.
> Essentially, the thought was when someone was attempting to add a layer
> to the index, automation would identify duplicates and help flag
> 'anti-social' behavior.  In addition, would want a way to enable devs to
> do the same before submission.

Sure, we can look at doing that - but there are still two problems with 
checking on submission:

1) It doesn't pick up items added after submission. We could do a regular 
report to catch those perhaps (or right now, run the existing Duplicates tool 
on a regular basis).

2) It doesn't prevent people doing needless duplication, it only detects it 
after the fact - the only way to solve that is if people check the layer index 
first before starting work, and engage with the existing layer maintainers if 
there's some reason why the existing recipe isn't suitable (too old, etc.) It 
would be worth thinking if there are ways we can encourage that.

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre