[bitbake-devel] [PATCH] Remove whitelisted vars from non-task deps

[Christopher Larson]

From: Christopher Larson <chris_larson at mentor.com>

Though the value of variables in the BB_BASEHASH_WHITELIST is kept out of the
checksums, dependency on them is not, at least for variables and non-task
functions. In the code, the whitelist is removed from the overall task dep
list, but not the individual variable deps. The result of this is that
functions like sysroot_stage_all and oe_runmake end up with whitelisted
variables like TERM listed in their dependencies, which means that doing
a 'unset TERM' before building will result in all checksums for tasks that
depend on those changing, and shared state reuse not behaving correctly.

This is only really a potential issue for variables from the environment, as
it's the existance/removal of the variable that's an issue, not its value, and
the other whitelisted variables are set in our metadata. This which means in
practical terms the only cases where this is likely to be an issue are in
environments where one of the following are unset: TERM, LOGNAME, HOME, USER,
PWD, SHELL. This may seem like an unlikely circumstance, but is in fact a real
issue for those of us using autobuilders. Jenkins does not set TERM when
executing shell, which means shared state archives produced by your jenkins
server would not be fully reused by an actual user.

Fixed by removing the whitelisted elements from the individual variable deps,
not just the accumulated result.

Signed-off-by: Christopher Larson <chris_larson at mentor.com>
---
 lib/bb/siggen.py |    9 +++++----
 1 files changed, 5 insertions(+), 4 deletions(-)

diff --git a/lib/bb/siggen.py b/lib/bb/siggen.py
index 8c79b17..7f5ae93 100644
--- a/lib/bb/siggen.py
+++ b/lib/bb/siggen.py
@@ -89,6 +89,7 @@ class SignatureGeneratorBasic(SignatureGenerator):
                 bb.error("Task %s from %s seems to be empty?!" % (task, fn))
                 data = ''
 
+            gendeps[task] -= self.basewhitelist
             newdeps = gendeps[task]
             seen = set()
             while newdeps:
@@ -98,12 +99,12 @@ class SignatureGeneratorBasic(SignatureGenerator):
                 for dep in nextdeps:
                     if dep in self.basewhitelist:
                         continue
+                    gendeps[dep] -= self.basewhitelist
                     newdeps |= gendeps[dep]
                 newdeps -= seen
 
-            alldeps = seen - self.basewhitelist
-
-            for dep in sorted(alldeps):
+            alldeps = sorted(seen)
+            for dep in alldeps:
                 data = data + dep
                 if dep in lookupcache:
                     var = lookupcache[dep]
@@ -113,7 +114,7 @@ class SignatureGeneratorBasic(SignatureGenerator):
                 if var:
                     data = data + str(var)
             self.basehash[fn + "." + task] = hashlib.md5(data).hexdigest()
-            taskdeps[task] = sorted(alldeps)
+            taskdeps[task] = alldeps
 
         self.taskdeps[fn] = taskdeps
         self.gendeps[fn] = gendeps
-- 
1.7.7