[OE-core] Modifying SRC_URI from anonymous python

[Christopher Larson]

On Thu, Nov 17, 2016 at 4:06 PM, Richard Purdie <
richard.purdie at linuxfoundation.org> wrote:

> On Thu, 2016-11-17 at 13:28 -0800, Andre McCurdy wrote:
> > I have a supplier who provides recipes which set SRC_URI to their
> > private git servers. To make those same recipes usable by others (ie
> > me), some anonymous python is used to transform the default SRC_URI
> > (elements which contain private git URLs are replaced, patches and
> > other files are left as-is).
> >
> > This apparently worked in OE 2.0 but from 2.1 onwards the anonymous
> > python which modifies SRC_URI races with the anonymous python in
> > base.bbclass which parses SRC_URI to determine additional
> > do_fetch/do_unpack dependencies and whether or not to call
> > fetch2.get_srcrev().
>
> I suspect we made some changes around that time to ensure determinism
> in the order certain things happened.
>
> >  Specifically I get failures because
> > fetch2.get_srcrev() sees the original SRC_URI and tries to resolve
> > AUTOREV from a repo to which I don't have access.
> >
> > The proposed solution from the supplier is this patch to
> > base.bbclass:
> >
> > @@ -598,7 +598,7 @@ python () {
> >              d.appendVarFlag('do_unpack', 'depends', '
> > file-native:do_populate_sysroot')
> >
> >      if needsrcrev:
> > -        d.setVar("SRCPV", "${@bb.fetch2.get_srcrev(d)}")
> > +        d.setVar("SRCPV", "${@bb.fetch2.get_srcrev(d)}",
> > parsing=True)
> >
> >      set_packagetriplet(d)
>
>
> parsing=True is a bitbake internal thing, its not documented and
> shouldn't be used outside the bitbake codebase. In hindsight, naming
> could have been better.
>
> > After reading the setVar source I'm not very clear how or why this
> > works, but it looks dubious. What is parsing=True intended to do?
>
> Its internal bitbake data store stuff, don't use or rely on it. It
> might happen to work here but its just luck.
>
> > Is it documented somewhere that modifying SRC_URI from anonymous
> > python isn't allowed? I've now seen two suppliers both independently
> > run into the same problem when updating to OE 2.1.
>
> We have an open bug related to anonymous python ordering. Its hard to
> fix easily as something can indicate it wants to run last, but then
> everything wants to run last. As Chris mentions, you can use an earlier
> event handler as one way to work around it.


I think our best bet there is to move toward more explicit event handler
naming and dependency handling amongst event handlers.
-- 
Christopher Larson
clarson at kergoth dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Maintainer - Tslib
Senior Software Engineer, Mentor Graphics
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openembedded.org/pipermail/openembedded-core/attachments/20161117/053353da/attachment-0002.html>