[oe] [RFC] BB_STAMP_BLACKLIST

[C Michael Sundius]

I sent this to the bitbake-dev list but thought that the discussion would be
useful to others in the wider OE world...

Also I'm realizing that maybe I should have called it a Request For Comment
:]

all comments are appreciated.

Mike

On Wed, Feb 3, 2010 at 12:59 PM, C Michael Sundius <msundius at sundius.com>wrote:

> A while ago I was asking about a way to make sure dependent packages were
> rebuilt if their dependencies were rebuilt.
>
> Because we're doing lots of development of proprietary software built
> around APIs of all sorts we're finding that dependent packages need to be
> rebuilt after its dependencies are rebuilt (and re-staged). based on some of
> your comments about how it could cause a giant cascade of rebuilding if
> such  an option was turned on universally, I came up w/ the following patch
> to allow a list of recipes identified to cause their dependents to be
> rebuilt if their time stamps were out of date.
>
> To my understanding currently such a check is not ever doen unless the
> BB_STAMP_POLICY is set to full or whitelist. For the whitelist case,
> BB_STAMP_WHITELIST contains a list of recipes that should not be checked
> (i.e. they are always considered up to date). My patch does the opposite by
> setting BB_STAMP_BLACKLIST a list of recipes are considered suspect and
> should be checked (in check_stamp_task()) and flagged to be run if they are
> out of date with respect to their dependencies.
>
> The usecase for this is when you are working on an api for a partiular
> module and want to make sure other modules that use that api are recompiled
> each time you change it.
>
> Well maybe this is a long winded description/justification, but here's the
> patch. Could you all take a look and tell me what you think?
>
> Mike
>
> diff --git a/lib/bb/runqueue.py b/lib/bb/runqueue.py
> index 3a43889..4f1f40f 100644
> --- a/lib/bb/runqueue.py
> +++ b/lib/bb/runqueue.py
> @@ -169,6 +169,7 @@ class RunQueue:
>          self.scheduler = bb.data.getVar("BB_SCHEDULER", cfgData, 1) or
> "speed"
>          self.stamppolicy = bb.data.getVar("BB_STAMP_POLICY", cfgData, 1)
> or "perfile"
>          self.stampwhitelist = bb.data.getVar("BB_STAMP_WHITELIST",
> cfgData, 1) or ""
> +        self.stampblacklist = bb.data.getVar("BB_STAMP_BLACKLIST",
> cfgData, 1) or ""
>
>      def reset_runqueue(self):
>          self.runq_fnid = []
> @@ -671,6 +672,17 @@ class RunQueue:
>              stampfnwhitelist.append(fn)
>          self.stampfnwhitelist = stampfnwhitelist
>
> +        # Create a blacklist usable by the stamp checks
> +        stampfnblacklist = []
> +        for entry in self.stampblacklist.split():
> +            entryid = self.taskData.getbuild_id(entry)
> +            if entryid not in self.taskData.build_targets:
> +                continue
> +            fnid = self.taskData.build_targets[entryid][0]
> +            fn = self.taskData.fn_index[fnid]
> +            stampfnblacklist.append(fn)
> +        self.stampfnblacklist = stampfnblacklist
> +
>          #self.dump_data(taskData)
>
>          self.state = runQueueRunInit
> @@ -779,6 +791,9 @@ class RunQueue:
>              if self.stamppolicy == "whitelist":
>                  stampwhitelist = self.stampfnwhitelist
>
> +        stampblacklist = []
> +        stampblacklist = self.stampfnblacklist
> +
>          fn = self.taskData.fn_index[self.runq_fnid[task]]
>          taskname = self.runq_task[task]
>          stampfile = "%s.%s" % (self.dataCache.stamp[fn], taskname)
> @@ -799,7 +814,7 @@ class RunQueue:
>                  fn2 = self.taskData.fn_index[self.runq_fnid[dep]]
>                  taskname2 = self.runq_task[dep]
>                  stampfile2 = "%s.%s" % (self.dataCache.stamp[fn2],
> taskname2)
> -                if fn == fn2 or (fulldeptree and fn2 not in
> stampwhitelist):
> +                if fn == fn2 or (fulldeptree and fn2 not in
> stampwhitelist) or fn2 in stampblacklist:
>                      try:
>                          t2 = os.stat(stampfile2)[stat.ST_MTIME]
>                          if t1 < t2:
>
>