[bitbake-devel] [PATCH 1/3] recipes: anonymous functions with priorities

[Patrick Ohly]

The execution order of anonymous functions inside a recipe is
deterministic (executed in the order in which the functions get
parsed), but class authors cannot know in which order classes get
inherited and thus cannot know whether the class functions run before
or after functions in other classes. That is a problem for
rm_work.bbclass, which must add its own task after all other
classes have added theirs.

As an extension of the current syntax, now any function whose name
*starts* with "__anonymous" is considered an anonymous function;
previously, anything with *exactly* that name was an anonymous
function.

In contrast to those which are named just "__anonymous" or have no
name at all, the "__anonymous_<something>" name must be globally
unique and can be used to set a "__anonprio" variable flag for the
function.

The default priority is 100. Functions with higher values run later.

For example, rm_work.bbclass can use this to ensure that it runs
later than other anonymous functions:

python __anonymous_rm_work() {
    bb.build.addtask(....)
}
__anonymous_rm_work[priority] = "1000"

The priorities influence parsing results and thus need to be included
in the base hash. That part was necessary to trigger re-parsing when
modifying the priority in rm_work.bbclass, which gets inherited
globally via INHERIT.

Related-to: [YOCTO #10584]

Signed-off-by: Patrick Ohly <patrick.ohly at intel.com>
---
 lib/bb/data_smart.py |  3 +++
 lib/bb/parse/ast.py  | 19 ++++++++++++++++---
 2 files changed, 19 insertions(+), 3 deletions(-)

diff --git a/lib/bb/data_smart.py b/lib/bb/data_smart.py
index 79d591a..4b1e9fb 100644
--- a/lib/bb/data_smart.py
+++ b/lib/bb/data_smart.py
@@ -1002,6 +1002,9 @@ class DataSmart(MutableMapping):
                 for i in bb_list:
                     value = d.getVar(i, False) or ""
                     data.update({i:value})
+                    priority = d.getVarFlag(i, '__anonprio', False)
+                    if priority is not None:
+                        data.update({'%s[__anonprio]' % i: priority})
 
         data_str = str([(k, data[k]) for k in sorted(data.keys())])
         return hashlib.md5(data_str.encode("utf-8")).hexdigest()
diff --git a/lib/bb/parse/ast.py b/lib/bb/parse/ast.py
index 853dda8..6255cc4 100644
--- a/lib/bb/parse/ast.py
+++ b/lib/bb/parse/ast.py
@@ -175,8 +175,15 @@ class MethodNode(AstNode):
     def eval(self, data):
         text = '\n'.join(self.body)
         funcname = self.func_name
-        if self.func_name == "__anonymous":
-            funcname = ("__anon_%s_%s" % (self.lineno, self.filename.translate(MethodNode.tr_tbl)))
+        if self.func_name.startswith("__anonymous"):
+            if self.func_name == "__anonymous":
+                # If truly anonymous, then we have to create a unique name.
+                funcname = ("__anon_%s_%s" % (self.lineno, self.filename.translate(MethodNode.tr_tbl)))
+            else:
+                # The recipe already has a unique name. Let's use that to ensure
+                # that any flags set for that name (like __anonymous_my_func[priority] = "1000")
+                # can be found later.
+                funcname = self.func_name
             self.python = True
             text = "def %s(d):\n" % (funcname) + text
             bb.methodpool.insert_method(funcname, text, self.filename, self.lineno - len(self.body))
@@ -352,7 +359,13 @@ def finalize(fn, d, variant = None):
     bb.data.expandKeys(d)
     bb.data.update_data(d)
     code = []
-    for funcname in d.getVar("__BBANONFUNCS", False) or []:
+    # Anonymous functions with lower priority get executed first.
+    # sorted() is stable, so for entries with the same priority,
+    # execution is in the order of definition.
+    anonymous = sorted(d.getVar("__BBANONFUNCS", False) or [],
+                       key=lambda f: int(d.getVarFlag(f, '__anonprio', True) or 100))
+    d.setVar("__BBANONFUNCS", anonymous)
+    for funcname in anonymous:
         code.append("%s(d)" % funcname)
     bb.utils.better_exec("\n".join(code), {"d": d})
     bb.data.update_data(d)
-- 
2.1.4