[bitbake-devel] [PATCH] fetch2: add ability to skip recipes when get_srcrev fails via BB_FETCH_SKIP_ON_SRCREV_ERROR

[Mark Hatle]

On 7/27/19 5:26 AM, Richard Purdie wrote:
> On Fri, 2019-07-26 at 15:46 +0000, chris.laplante--- via bitbake-devel
> wrote:
>>> -----Original Message-----
>>> From: Alexander Kanavin <alex.kanavin at gmail.com>
>>> Sent: Friday, July 26, 2019 11:06 AM
>>> To: LAPLANTE,CHRIS (Agilent USA) <chris.laplante at agilent.com>
>>> Cc: bitbake-devel <bitbake-devel at lists.openembedded.org>
>>> Subject: Re: [bitbake-devel] [PATCH] fetch2: add ability to skip
>>> recipes when get_srcrev fails via BB_FETCH_SKIP_ON_SRCREV_ERROR
>>>
>>> I think it's better to rearrange the layer itself, and split it
>>> into a privileged and a non-privileged one.
>>>
>>
>> Yes, that's the "cleanest" solution, but it doesn’t necessarily scale
>> well when you add more and more disjoint sets of users that can/can't
>> access particular recipes. 
>>
>> Also, just to add some more context to how we are using this: for
>> each of our "skippable" recipes, we have alternative providers that
>> pull pre-compiled artifacts from "pool" repositories. It is nice to
>> keep the providers' recipes in one place.
>>
>>> This patch seems to enable skipping the recipe, regardless of what
>>> the actual issue with fetching was (which may or may not be 'access
>>> denied'). Skipping on any error doesn't seem right to me as it may
>>> mask other types of problems.
>>
>> I see your point (in general), but note that this only skips the
>> recipe if an error occurs in get_srcrev during the parsing stage (and
>> we are not executing in a task). The original exception will still be
>> displayed to the user. 
>>
>> But it certainly would be better to be more selective with which kind
>> of errors get swallowed. I will try to look into that.
> 
> I have some quite strong feelings on when recipes should or shouldn't
> be skipped.
> 
> Skipped recipes are problematic in that they "disappear" from the user
> view and it can be hard to understand why. I've therefore always wanted
> a strongly deterministic and obvious approach to when it happens and
> when it doesn't.
> 
> Alex just commented on another thread where he didn't like recipe
> dependencies needing to be marked as skipped individually rather than
> some automatic magic. That was deliberate as it would be far too easy
> for bitbake to mark nearly everything as skipped should something
> simple go wrong. I've seen that happen and I could barely unravel it
> despite knowing what was happening.
> 
> This SRCREV change feels similar, I can just imagine the bug reports
> for user A able to build just fine, user B can't even see the recipe
> and its all down to say their git config in their homedir.
> 
> You can likely do something similar to this by checking access earlier
> in the recipe with an immediate expansion calling your custom function
> which traps FetchError and have a similar effect to what you want. I
> think this is a sufficiently niche and also specialised use case that
> we don't want to encourage it or "pollute" the core with code for it.

I agree, I don't really like auto skipping in this case as it will make it
difficult to determine or resolve issues from different users.

But I do wonder if this is a specific URL/SRCREV that isn't available to this
user, or is this related to AUTOREV?

I've had people request this type of behavior or an alternative behavior for
AUTOREV packages in the past from me.

In the AUTOREV case, what was requested of me (and I declined) was try the
network, if we couldn't do that then look for a local mirror and use that
instead.  Only fail if we didn't have a local mirror.   Depending on how this
would be implemented it may suffer from similar non-deterministic behavior.

--Mark

> Cheers,
> 
> Richard
> 
> 
> 
>