[bitbake-devel] [PATCH] fetch2: add ability to skip recipes when get_srcrev fails via BB_FETCH_SKIP_ON_SRCREV_ERROR

[Richard Purdie]

On Fri, 2019-07-26 at 15:46 +0000, chris.laplante--- via bitbake-devel
wrote:
> > -----Original Message-----
> > From: Alexander Kanavin <alex.kanavin at gmail.com>
> > Sent: Friday, July 26, 2019 11:06 AM
> > To: LAPLANTE,CHRIS (Agilent USA) <chris.laplante at agilent.com>
> > Cc: bitbake-devel <bitbake-devel at lists.openembedded.org>
> > Subject: Re: [bitbake-devel] [PATCH] fetch2: add ability to skip
> > recipes when get_srcrev fails via BB_FETCH_SKIP_ON_SRCREV_ERROR
> > 
> > I think it's better to rearrange the layer itself, and split it
> > into a privileged and a non-privileged one.
> > 
> 
> Yes, that's the "cleanest" solution, but it doesn’t necessarily scale
> well when you add more and more disjoint sets of users that can/can't
> access particular recipes. 
> 
> Also, just to add some more context to how we are using this: for
> each of our "skippable" recipes, we have alternative providers that
> pull pre-compiled artifacts from "pool" repositories. It is nice to
> keep the providers' recipes in one place.
> 
> > This patch seems to enable skipping the recipe, regardless of what
> > the actual issue with fetching was (which may or may not be 'access
> > denied'). Skipping on any error doesn't seem right to me as it may
> > mask other types of problems.
> 
> I see your point (in general), but note that this only skips the
> recipe if an error occurs in get_srcrev during the parsing stage (and
> we are not executing in a task). The original exception will still be
> displayed to the user. 
> 
> But it certainly would be better to be more selective with which kind
> of errors get swallowed. I will try to look into that.

I have some quite strong feelings on when recipes should or shouldn't
be skipped.

Skipped recipes are problematic in that they "disappear" from the user
view and it can be hard to understand why. I've therefore always wanted
a strongly deterministic and obvious approach to when it happens and
when it doesn't.

Alex just commented on another thread where he didn't like recipe
dependencies needing to be marked as skipped individually rather than
some automatic magic. That was deliberate as it would be far too easy
for bitbake to mark nearly everything as skipped should something
simple go wrong. I've seen that happen and I could barely unravel it
despite knowing what was happening.

This SRCREV change feels similar, I can just imagine the bug reports
for user A able to build just fine, user B can't even see the recipe
and its all down to say their git config in their homedir.

You can likely do something similar to this by checking access earlier
in the recipe with an immediate expansion calling your custom function
which traps FetchError and have a similar effect to what you want. I
think this is a sufficiently niche and also specialised use case that
we don't want to encourage it or "pollute" the core with code for it.

Cheers,

Richard