[OE-core] [PATCH] base.bbclass: add named SRCREVs to the sstate hash
Michael.Ho at bmw.de
Michael.Ho at bmw.de
Tue May 28 09:31:11 UTC 2019
Hi Martin,
Ah okay, sorry, I misread and thought you meant it was a known issue and it is expected to set vardeps manually.
Thanks for the +1.
Kind regards,
Michael Ho
--
BMW Car IT GmbH
Michael Ho
Spezialist Entwicklung – Build and Release Engineering
Lise-Meitner-Straße 14
89081 Ulm
Tel.: +49-731-37804-071
Mobil: +49-152-54980-471
Fax: +49-731-37804-001
Mail: michael.ho at bmw-carit.de<mailto:michael.ho at bmw-carit.de>
Web: http://www.bmw-carit.de<http://www.bmw-carit.de/>
-------------------------------------------------------------------------
BMW Car IT GmbH
Geschäftsführer: Kai-Uwe Balszuweit und Christian Salzmann
Sitz und Registergericht: München HRB 134810
-------------------------------------------------------------------------
From: Martin Jansa <martin.jansa at gmail.com>
Date: Monday, 27 May 2019 at 6:56 pm
To: "Ho Michael, JC-3UL" <Michael.Ho at bmw.de>
Cc: Chris Larson <kergoth at gmail.com>, Patches and discussions about the oe-core layer <openembedded-core at lists.openembedded.org>
Subject: Re: [OE-core] [PATCH] base.bbclass: add named SRCREVs to the sstate hash
Hi Michael,
sorry I wasn't clear in the reply, I meant that it's easy to work around this issue, but it's error prone.
Your patch does it automatically and I don't see any cases where we wouldn't want all the SRCREVs included in do_fetch signature, so I like your patch.
+1 from me.
Cheers,
On Mon, May 27, 2019 at 6:34 PM <Michael.Ho at bmw.de<mailto:Michael.Ho at bmw.de>> wrote:
Hi Martin,
It seems a little tricky though to need to know about the sstate hashing in order to use named
source revisions and it could be a bit confusing that using a normal SRCREV doesn’t require any
extra effort but named SRCREVs require additional lines of metadata. As you can see in the
commit message of my patches, the vulkan-demos recipe in poky also has this issue so it might
be quite easy to forget.
Thanks for the feedback. If the patch idea isn’t good, let me know if you think I should instead
post a patch for the documentation to try to more explicitly note the need for the additional vardeps
statements when using named SRCREVs.
Thanks.
Kind regards,
Michael
--
BMW Car IT GmbH
Michael Ho
Spezialist Entwicklung – Build and Release Engineering
Lise-Meitner-Straße 14
89081 Ulm
Tel.: +49-731-37804-071
Mobil: +49-152-54980-471
Fax: +49-731-37804-001
Mail: michael.ho at bmw-carit.de<mailto:michael.ho at bmw-carit.de>
Web: http://www.bmw-carit.de<http://www.bmw-carit.de/>
-------------------------------------------------------------------------
BMW Car IT GmbH
Geschäftsführer: Kai-Uwe Balszuweit und Christian Salzmann
Sitz und Registergericht: München HRB 134810
-------------------------------------------------------------------------
From: Martin Jansa <martin.jansa at gmail.com<mailto:martin.jansa at gmail.com>>
Date: Monday, 27 May 2019 at 5:44 pm
To: "Ho Michael, JC-3UL" <Michael.Ho at bmw.de<mailto:Michael.Ho at bmw.de>>
Cc: Chris Larson <kergoth at gmail.com<mailto:kergoth at gmail.com>>, Patches and discussions about the oe-core layer <openembedded-core at lists.openembedded.org<mailto:openembedded-core at lists.openembedded.org>>
Subject: Re: [OE-core] [PATCH] base.bbclass: add named SRCREVs to the sstate hash
In those recipes which don't include SRCPV in PV (for whatever reason) we usually add them explicitly with:
do_fetch[vardeps] += "SRCREV_common"
it's not ideal as people sometimes forget about this. Good practice is to add this next to the SRCREV_FORMAT (if you set one, to clearly show which revs are and aren't included).
On Mon, May 27, 2019 at 5:37 PM <Michael.Ho at bmw.de<mailto:Michael.Ho at bmw.de>> wrote:
Hi Christopher,
Thank you for the feedback. Since SRCPV is not always enforced (correct me if I’m wrong), it seems easy to trip over this bug when just following the documentation on multiple named source uri’s (also if you purposely don’t want to include the secondary source revisions in the package version). I’ll give it another shot to make it more efficient. If it’s not expected to be an encounterable bug case you can ignore the patch.
Thanks.
Kind regards,
Michael
--
BMW Car IT GmbH
Michael Ho
Spezialist Entwicklung – Build and Release Engineering
Lise-Meitner-Straße 14
89081 Ulm
Tel.: +49-731-37804-071
Mobil: +49-152-54980-471
Fax: +49-731-37804-001
Mail: michael.ho at bmw-carit.de<mailto:michael.ho at bmw-carit.de>
Web: http://www.bmw-carit.de<http://www.bmw-carit.de/>
-------------------------------------------------------------------------
BMW Car IT GmbH
Geschäftsführer: Kai-Uwe Balszuweit und Christian Salzmann
Sitz und Registergericht: München HRB 134810
-------------------------------------------------------------------------
From: Christopher Larson <kergoth at gmail.com<mailto:kergoth at gmail.com>>
Date: Wednesday, 8 May 2019 at 5:17 pm
To: "Ho Michael, JC-3UL" <Michael.Ho at bmw.de<mailto:Michael.Ho at bmw.de>>
Cc: Patches and discussions about the oe-core layer <openembedded-core at lists.openembedded.org<mailto:openembedded-core at lists.openembedded.org>>
Subject: Re: [OE-core] [PATCH] base.bbclass: add named SRCREVs to the sstate hash
Does SRCPV not already cover this in the majority of cases? SRCREV_FORMAT controls how the multiple revs end up in PV, and the change to PV results in rebuilding anyway. And iterating over d.keys() is inefficient, *if* you’re going to do this, operate on SRCREV, gather up the name= parameters from scm urls, and use that to drive it.
On Wed, May 8, 2019 at 7:10 AM Michael Ho <Michael.Ho at bmw.de<mailto:Michael.Ho at bmw.de>> wrote:
Several fetchers support named sources that require setting a SRCREV with
the source name as a suffix. These named SRCREV variables are not captured
in the sstate hash calculation because they're only referenced within the
bitbake fetcher function.
Add a snippet to the base.bbclass anonymous python to add all named SRCREV
variables to the vardeps of do_fetch to capture them in the sstate hash
calculation.
Testing of the bug can be shown by running the following bitbake commands
with this patch set not applied:
bitbake vulkan-demos | tee
sed -i 's/SRCREV_gli = ".*"/SRCREV_gli = "xxx"/' \
../meta/recipes-graphics/vulkan/vulkan-demos_git.bb<http://vulkan-demos_git.bb>
bitbake vulkan-demos | tee;
Results in no errors despite a broken SRCREV because the vulkan-demos is
considered unchanged.
After applying this patch the above commands instead result in a fetcher
error which is correct.
---
meta/classes/base.bbclass | 10 ++++++++++
1 file changed, 10 insertions(+)
diff --git a/meta/classes/base.bbclass b/meta/classes/base.bbclass
index 1636c6e..84a27f5 100644
--- a/meta/classes/base.bbclass
+++ b/meta/classes/base.bbclass
@@ -638,6 +638,16 @@ python () {
if needsrcrev:
d.setVar("SRCPV", "${@bb.fetch2.get_srcrev(d)}")
+ # Gather all SRCREV_* references to add to the sstate hash calculation
+ # This may capture SRCREVs not used but it's difficult to try to restrict it
+ # to only what is needed
+ for dkey in d.keys():
+ if dkey.startswith("SRCREV_"):
+ # This anonymous python snippet is called multiple times so we
+ # need to be careful to not double up the appends here and cause
+ # the base hash to mismatch the task hash
+ if dkey not in (d.getVarFlag("do_fetch", "vardeps") or '').split():
+ d.appendVarFlag("do_fetch", "vardeps", " {}".format(dkey))
set_packagetriplet(d)
--
2.7.4
--
_______________________________________________
Openembedded-core mailing list
Openembedded-core at lists.openembedded.org<mailto:Openembedded-core at lists.openembedded.org>
http://lists.openembedded.org/mailman/listinfo/openembedded-core
--
Christopher Larson
kergoth at gmail dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Senior Software Engineer, Mentor Graphics
--
_______________________________________________
Openembedded-core mailing list
Openembedded-core at lists.openembedded.org<mailto:Openembedded-core at lists.openembedded.org>
http://lists.openembedded.org/mailman/listinfo/openembedded-core
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openembedded.org/pipermail/openembedded-core/attachments/20190528/a9648f9a/attachment-0001.html>
More information about the Openembedded-core
mailing list