[OE-core] Using external source trees with OE-Core

[Richard Purdie]

Someone recently asked me about using external source trees with
OE-Core. I was aware of srctree.bbclass in OE-Classic and did start
looking at it but it has various elements I wasn't so keen on. I
therefore wondered if I could improve upon it. I did start from that
code base but came up with:

http://git.yoctoproject.org/cgit.cgi/poky-contrib/commit/?h=rpurdie/a0&id=ac8a54c18c33aaab6758c34a3d6093d3d3384cee
[patch inline below]

which I've called externalsrc.bbclass since it behaves a bit
differently. In particular, all the usual task targets are available.

As an example usage with the above patch applied:

cd ~
tar -xvzf $DL_DIR/libfm-0.1.17.tar.gz
(creates libfm-0.1.17 in my homedir)

edit libfm_0.1.17.bb and add:

inherit externalsrc
S = "/home/richard/libfm-0.1.17"

bitbake libfm

and it does what you'd expect. You can also -c clean and it will wipe
out WORKDIR but it won't touch ${S}. I picked libfm effectively at
random. It won't remove the autoreconf changes from configure but I'm
not sure I care much about that.

Admittedly, I did have to fix one Makefile.am which did '-I../' instead
of "-I$(srcdir)/../" but that is a bug in libfm.

Obviously this won't work quite as well with some recipes like
linux-yocto which manipulate ${S} a lot more but it should work in most
cases. I also found it very hard to remove tasks from the anonymous
python, we probably need to improve the API for this.

Opinions on including this class?

Cheers,

Richard



>From ac8a54c18c33aaab6758c34a3d6093d3d3384cee Mon Sep 17 00:00:00 2001
From: Richard Purdie <richard.purdie at linuxfoundation.org>
Date: Fri, 24 Feb 2012 12:29:36 +0000
Subject: externalsrc.bbclass: Add class for handling external source trees

This is loosly based upon srctree.bbclass from OE-Classic but with some
changes appropriate to OE-Core.

Signed-off-by: Richard Purdie <richard.purdie at linuxfoundation.org>
---
diff --git a/meta/classes/externalsrc.bbclass b/meta/classes/externalsrc.bbclass
new file mode 100644
index 0000000..7e00ef8
--- a/dev/null
+++ b/meta/classes/externalsrc.bbclass
@@ -0,0 +1,53 @@
+# Copyright (C) 2012 Linux Foundation
+# Author: Richard Purdie
+# Some code and influence taken from srctree.bbclass:
+# Copyright (C) 2009 Chris Larson <clarson at kergoth.com>
+# Released under the MIT license (see COPYING.MIT for the terms)
+#
+# externalsrc.bbclass enables use of an existing source tree, usually external to 
+# the build system to build a piece of software rather than the usual fetch/unpack/patch
+# process.
+#
+# To use, set S to point at the directory you want to use containing the sources
+# e.g. S = "/path/to/my/source/tree"
+#
+# If the class is to work for both target and native versions (or with multilibs/
+# cross or other BBCLASSEXTEND variants), its expected that setting B to point to 
+# where to place the compiled binaries will work (split source and build directories).
+# This is the default but B can be set to S if circumstaces dictate.
+#
+
+SRC_URI = ""
+SRCTREECOVEREDTASKS ?= "do_patch do_unpack do_fetch"
+B = "${WORKDIR}/${BPN}-${PV}/"
+
+def remove_tasks(tasks, deltasks, d):
+    for task in tasks:
+        deps = d.getVarFlag(task, "deps")
+        for preptask in deltasks:
+            if preptask in deps:
+                deps.remove(preptask)
+        d.setVarFlag(task, "deps", deps)
+    # Poking around bitbake internal variables is evil but there appears to be no better way :(
+    tasklist = d.getVar('__BBTASKS') or []
+    for task in deltasks:
+        d.delVarFlag(task, "task")
+        if task in tasklist:
+            tasklist.remove(task)
+    d.setVar('__BBTASKS', tasklist)
+
+python () {
+    tasks = filter(lambda k: d.getVarFlag(k, "task"), d.keys())
+    covered = d.getVar("SRCTREECOVEREDTASKS", True).split()
+
+    for task in tasks:
+        if task.endswith("_setscene"):
+            # sstate is never going to work for external source trees, disable it
+            covered.append(task)
+        else:
+            # Since configure will likely touch ${S}, ensure only we lock so one task has access at a time
+            d.appendVarFlag(task, "lockfiles", "${S}/singletask.lock")
+
+    remove_tasks(tasks, covered, d)
+}
+