[OE-core] [PATCH 2/2] toaster: add class to dump toaster-tracked data

[Paul Eggleton]

From: Alexandru DAMIAN <alexandru.damian at intel.com>

Adding a new bbclass that will collect and send relevant
data from the task context to the Toaster UI.

This bbclass consists of postfuncs that get executed
right after the main task func, and in the same context.
This allows data gathering in a synchronous manner during
the build, guaranteeing data integrity. This approach also
preserves the task signatures.

The data is moved to the UI through the event system.

There is no performance impact if the class is disabled.

License is MIT.

Signed-off-by: Alexandru DAMIAN <alexandru.damian at intel.com>
Signed-off-by: Paul Eggleton <paul.eggleton at linux.intel.com>
---
 meta/classes/toaster.bbclass | 110 +++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 110 insertions(+)
 create mode 100644 meta/classes/toaster.bbclass

diff --git a/meta/classes/toaster.bbclass b/meta/classes/toaster.bbclass
new file mode 100644
index 0000000..7dbb384
--- /dev/null
+++ b/meta/classes/toaster.bbclass
@@ -0,0 +1,110 @@
+#
+# Toaster helper class
+#
+# Copyright (C) 2013 Intel Corporation
+#
+# Released under the MIT license (see COPYING.MIT)
+#
+# This bbclass is designed to extract data used by OE-Core during the build process,
+# for recording in the Toaster system.
+# The data access is synchronous, preserving the build data integrity across
+# different builds.
+#
+# The data is transferred through the event system, using the MetadataEvent objects.
+#
+# The model is to enable the datadump functions as postfuncs, and have the dump
+# executed after the real taskfunc has been executed. This prevents task signature changing
+# is toaster is enabled or not. Build performance is not affected if Toaster is not enabled.
+#
+# To enable, use INHERIT in local.conf:
+#
+#       INHERIT += "toaster"
+#
+#
+#
+#
+
+# 1. Dump package file info data
+
+python toaster_package_dumpdata() {
+    """
+    Dumps the data created by emit_pkgdata
+    """
+    # replicate variables from the package.bbclass
+
+    packages = d.getVar('PACKAGES', True)
+    pkgdest = d.getVar('PKGDEST', True)
+
+    pkgdatadir = d.getVar('PKGDESTWORK', True)
+
+
+    # scan and send data for each package
+    import ast
+    import fnmatch
+
+    lpkgdata = {}
+    for pkg in packages.split():
+
+        subdata_file = pkgdatadir + "/runtime/%s" % pkg
+        lpkgdata = {}
+
+        sf = open(subdata_file, "r")
+        line = sf.readline()
+        while line:
+            (n, v) = line.rstrip().split(":", 1)
+            if pkg in n:
+                n = n.replace("_" + pkg, "")
+            lpkgdata[n] = v.strip()
+            line = sf.readline()
+            pkgsplitname = os.path.join(pkgdest, pkg)
+            # replace FILES_INFO data with a dictionary of file name - file size
+            if n == 'FILES_INFO':
+                filesizedata = {}
+                val = v.strip().replace('\\\'', '\'')
+                dictval = ast.literal_eval(val)
+                for parent, dirlist in dictval.items():
+                    idx = parent.find(pkgsplitname)
+                    if idx > -1:
+                        parent = parent[idx+len(pkgsplitname):]
+                    else:
+                        bb.error("Invalid path while looking for file ", parent)
+                    for basename in dirlist:
+                        fullpath = os.path.join(parent, basename)
+                        try:
+                            filesizedata[fullpath] = os.stat(pkgsplitname + fullpath).st_size
+                        except OSError:
+                            # we may hit a symlink that is not pointing correctly over package-split
+                            filesizedata[fullpath] = 0
+                lpkgdata[n] = filesizedata
+
+        # Fire an event containing the pkg data
+        bb.event.fire(bb.event.MetadataEvent("SinglePackageInfo", lpkgdata), d)
+}
+
+# 2. Dump output image files information
+
+python toaster_image_dumpdata() {
+    """
+    Image filename for output images is not standardized.
+    image_types.bbclass will spell out IMAGE_CMD_xxx variables that actually
+    have hardcoded ways to create image file names in them.
+    So we look for files starting with the set name.
+    """
+
+    deploy_dir_image = d.getVar('DEPLOY_DIR_IMAGE', True);
+    image_name = d.getVar('IMAGE_NAME', True);
+
+    image_info_data = {}
+
+    for dirpath, dirnames, filenames in os.walk(deploy_dir_image):
+        for fn in filenames:
+            if fn.startswith(image_name):
+                image_info_data[dirpath + fn] = os.stat(os.path.join(dirpath, fn)).st_size
+
+    bb.event.fire(bb.event.MetadataEvent("ImageFileSize",image_info_data), d)
+}
+
+
+do_package[postfuncs] += "toaster_package_dumpdata "
+
+do_rootfs[postfuncs] += "toaster_image_dumpdata "
-- 
1.8.1.2