[OE-core] [PATCH] base.bbclass: add named SRCREVs to the sstate hash

Mon May 27 16:56:22 UTC 2019

Hi Michael,

sorry I wasn't clear in the reply, I meant that it's easy to work around
this issue, but it's error prone.

Your patch does it automatically and I don't see any cases where we
wouldn't want all the SRCREVs included in do_fetch signature, so I like
your patch.

+1 from me.

Cheers,

On Mon, May 27, 2019 at 6:34 PM <Michael.Ho at bmw.de> wrote:

> Hi Martin,
>
>
>
> It seems a little tricky though to need to know about the sstate hashing
> in order to use named
> source revisions and it could be a bit confusing that using a normal
> SRCREV doesn’t require any
> extra effort but named SRCREVs require additional lines of metadata. As
> you can see in the
> commit message of my patches, the vulkan-demos recipe in poky also has
> this issue so it might
> be quite easy to forget.
>
>
>
> Thanks for the feedback. If the patch idea isn’t good, let me know if you
> think I should instead
> post a patch for the documentation to try to more explicitly note the need
> for the additional vardeps
> statements when using named SRCREVs.
>
>
>
> Thanks.
>
>
>
> Kind regards,
>
> Michael
>
>
>
> --
>
> *BMW Car IT GmbH*
> Michael Ho
> Spezialist Entwicklung – Build and Release Engineering
> Lise-Meitner-Straße 14
> 89081 Ulm
>
> Tel.: +49-731-37804-071
>
> Mobil: +49-152-54980-471
> Fax: +49-731-37804-001
> Mail: michael.ho at bmw-carit.de
> Web: http://www.bmw-carit.de
> -------------------------------------------------------------------------
> BMW Car IT GmbH
> Geschäftsführer: Kai-Uwe Balszuweit und Christian Salzmann
> Sitz und Registergericht: München HRB 134810
> -------------------------------------------------------------------------
>
>
>
>
>
> *From: *Martin Jansa <martin.jansa at gmail.com>
> *Date: *Monday, 27 May 2019 at 5:44 pm
> *To: *"Ho Michael, JC-3UL" <Michael.Ho at bmw.de>
> *Cc: *Chris Larson <kergoth at gmail.com>, Patches and discussions about the
> oe-core layer <openembedded-core at lists.openembedded.org>
> *Subject: *Re: [OE-core] [PATCH] base.bbclass: add named SRCREVs to the
> sstate hash
>
>
>
> In those recipes which don't include SRCPV in PV (for whatever reason) we
> usually add them explicitly with:
>
> do_fetch[vardeps] += "SRCREV_common"
>
> it's not ideal as people sometimes forget about this. Good practice is to
> add this next to the SRCREV_FORMAT (if you set one, to clearly show which
> revs are and aren't included).
>
>
>
> On Mon, May 27, 2019 at 5:37 PM <Michael.Ho at bmw.de> wrote:
>
> Hi Christopher,
>
>
>
> Thank you for the feedback. Since SRCPV is not always enforced (correct me
> if I’m wrong), it seems easy to trip over this bug when just following the
> documentation on multiple named source uri’s (also if you purposely don’t
> want to include the secondary source revisions in the package version).
> I’ll give it another shot to make it more efficient. If it’s not expected
> to be an encounterable bug case you can ignore the patch.
>
>
>
> Thanks.
>
>
>
> Kind regards,
>
> Michael
>
>
>
> --
>
> *BMW Car IT GmbH*
> Michael Ho
> Spezialist Entwicklung – Build and Release Engineering
> Lise-Meitner-Straße 14
> 89081 Ulm
>
> Tel.: +49-731-37804-071
>
> Mobil: +49-152-54980-471
> Fax: +49-731-37804-001
> Mail: michael.ho at bmw-carit.de
> Web: http://www.bmw-carit.de
> -------------------------------------------------------------------------
> BMW Car IT GmbH
> Geschäftsführer: Kai-Uwe Balszuweit und Christian Salzmann
> Sitz und Registergericht: München HRB 134810
> -------------------------------------------------------------------------
>
>
>
>
>
> *From: *Christopher Larson <kergoth at gmail.com>
> *Date: *Wednesday, 8 May 2019 at 5:17 pm
> *To: *"Ho Michael, JC-3UL" <Michael.Ho at bmw.de>
> *Cc: *Patches and discussions about the oe-core layer <
> openembedded-core at lists.openembedded.org>
> *Subject: *Re: [OE-core] [PATCH] base.bbclass: add named SRCREVs to the
> sstate hash
>
>
>
> Does SRCPV not already cover this in the majority of cases? SRCREV_FORMAT
> controls how the multiple revs end up in PV, and the change to PV results
> in rebuilding anyway. And iterating over d.keys() is inefficient, *if*
> you’re going to do this, operate on SRCREV, gather up the name= parameters
> from scm urls, and use that to drive it.
>
>
>
> On Wed, May 8, 2019 at 7:10 AM Michael Ho <Michael.Ho at bmw.de> wrote:
>
> Several fetchers support named sources that require setting a SRCREV with
> the source name as a suffix. These named SRCREV variables are not captured
> in the sstate hash calculation because they're only referenced within the
> bitbake fetcher function.
>
> Add a snippet to the base.bbclass anonymous python to add all named SRCREV
> variables to the vardeps of do_fetch to capture them in the sstate hash
> calculation.
>
> Testing of the bug can be shown by running the following bitbake commands
> with this patch set not applied:
>
> bitbake vulkan-demos | tee
> sed -i 's/SRCREV_gli = ".*"/SRCREV_gli = "xxx"/' \
>   ../meta/recipes-graphics/vulkan/vulkan-demos_git.bb
> bitbake vulkan-demos | tee;
>
> Results in no errors despite a broken SRCREV because the vulkan-demos is
> considered unchanged.
>
> After applying this patch the above commands instead result in a fetcher
> error which is correct.
> ---
>  meta/classes/base.bbclass | 10 ++++++++++
>  1 file changed, 10 insertions(+)
>
> diff --git a/meta/classes/base.bbclass b/meta/classes/base.bbclass
> index 1636c6e..84a27f5 100644
> --- a/meta/classes/base.bbclass
> +++ b/meta/classes/base.bbclass
> @@ -638,6 +638,16 @@ python () {
>
>      if needsrcrev:
>          d.setVar("SRCPV", "${@bb.fetch2.get_srcrev(d)}")
> +        # Gather all SRCREV_* references to add to the sstate hash
> calculation
> +        # This may capture SRCREVs not used but it's difficult to try to
> restrict it
> +        # to only what is needed
> +        for dkey in d.keys():
> +            if dkey.startswith("SRCREV_"):
> +                # This anonymous python snippet is called multiple times
> so we
> +                # need to be careful to not double up the appends here
> and cause
> +                # the base hash to mismatch the task hash
> +                if dkey not in (d.getVarFlag("do_fetch", "vardeps") or
> '').split():
> +                    d.appendVarFlag("do_fetch", "vardeps", "
> {}".format(dkey))
>
>      set_packagetriplet(d)
>
> --
> 2.7.4
>
> --
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core at lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core
>
>
>
>
> --
>
> Christopher Larson
> kergoth at gmail dot com
> Founder - BitBake, OpenEmbedded, OpenZaurus
> Senior Software Engineer, Mentor Graphics
>
> --
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core at lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openembedded.org/pipermail/openembedded-core/attachments/20190527/c4ae1a33/attachment-0001.html>