[OE-core] [RFC PATCH] devicetree.bbclass: User/BSP device tree source compilation class

Fri Apr 6 05:52:10 UTC 2018

Acked-by: Martin Hundebøll <mnhu at prevas.dk>

Thanks Nathan; this could be very useful for us.

// Martin

On 2018-04-05 12:48, Nathan Rossi wrote:
> This bbclass implements the device tree compilation for user provided
> device trees. In order to use this class, it should be inherited in a
> BSP recipe which provides the sources. The default setup enables
> inclusion of kernel device tree sources (though can be disabled by the
> recipe by overriding DT_INCLUDE or KERNEL_INCLUDE).
> 
> This provides an additional mechanism for BSPs to provide device trees
> and device tree overlays for their target machines. Whilst still
> enabling access to the kernel device trees for base SoC includes and
> headers.
> 
> This approach to providing device trees has benefits for certain use
> cases over patching the device trees into the kernel source.
> 
> * device trees are separated from kernel source, allows for selection of
> kernel and or kernel versions without needing to explicitly patch the
> kernel (or appending to the kernel recipes).
> 
> * providing device trees from separate sources, from the layer,
> generated by the recipe or other recipes.
> 
> This class also implements some additional features that are not
> available in the kernel-devicetree flow. This includes population of
> device tree blobs into the sysroot which allows for other recipes to
> consume built dtbs (e.g. U-Boot with EXT_DTB compilation), device tree
> overlay compilation and customizing DTC compilation args (boot
> cpu/padding/etc.).
> ---
> This patch is an RFC for adding devicetree.bbclass to oe-core. With the
> plan to submit it for inclusion in the next release.
> 
> In addition to these changes it is my intention to also submit
> documentation to the Yocto BSP guide that covers both methods (kernel
> source patch and this bbclass) for how to provide user device trees for a
> BSP layer.
> 
> The meta-xilinx layer has a recipe (device-tree.bb) that behaves exactly
> like this class for some time. It has been very useful for out-of-kernel
> device tree compilation as well as for a mechanism so that users of
> Xilinx SoCs can provide their customizations (FPGA devices) and board
> specific configuration. This compilation flow also makes sense for other
> BSPs, which is the reason for creating this bbclass for common shared
> use by other BSPs.
> 
> Examples of this classes usage can be seen for meta-xilinx:
> https://github.com/nathanrossi/meta-xilinx/blob/nrossi/devicetree-classify/meta-xilinx-bsp/recipes-bsp/device-tree/device-tree.bb
> 
> And for a custom BSP:
> https://github.com/nathanrossi/gps-clock-project/blob/wip-devicetree-classify/meta-gps-clock/recipes/device-tree/device-tree.bb
> ---
>   meta/classes/devicetree.bbclass | 140 ++++++++++++++++++++++++++++++++++++++++
>   1 file changed, 140 insertions(+)
>   create mode 100644 meta/classes/devicetree.bbclass
> 
> diff --git a/meta/classes/devicetree.bbclass b/meta/classes/devicetree.bbclass
> new file mode 100644
> index 0000000000..dbc83f2a1d
> --- /dev/null
> +++ b/meta/classes/devicetree.bbclass
> @@ -0,0 +1,140 @@
> +# This bbclass implements device tree compliation for user provided device tree
> +# sources. The compilation of the device tree sources is the same as the kernel
> +# device tree compilation process, this includes being able to include sources
> +# from the kernel such as soc dtsi files or header files such as gpio.h. In
> +# addition to device trees this bbclass also handles compilation of device tree
> +# overlays.
> +#
> +# The output of this class behaves similar to how kernel-devicetree.bbclass
> +# operates in that the output files are installed into /boot/devicetree.
> +# However this class on purpose separates the deployed device trees into the
> +# 'devicetree' subdirectory. This prevents clashes with the kernel-devicetree
> +# output. Additionally the device trees are populated into the sysroot for
> +# access via the sysroot from within other recipes.
> +
> +SECTION ?= "bsp"
> +
> +# The default inclusion of kernel device tree includes and headers means that
> +# device trees built with them are at least GPLv2 (and in some cases dual
> +# licensed). Default to GPLv2 if the recipe does not specify a license.
> +LICENSE ?= "GPLv2"
> +LIC_FILES_CHKSUM ?= "file://${COMMON_LICENSE_DIR}/GPL-2.0;md5=801f80980d171dd6425610833a22dbe6"
> +
> +INHIBIT_DEFAULT_DEPS = "1"
> +DEPENDS += "dtc-native"
> +
> +inherit deploy kernel-arch
> +
> +COMPATIBLE_MACHINE ?= "^$"
> +
> +PACKAGE_ARCH = "${MACHINE_ARCH}"
> +
> +SYSROOT_DIRS += "/boot/devicetree"
> +FILES_${PN} = "/boot/devicetree/*.dtb /boot/devicetree/*.dtbo"
> +
> +S = "${WORKDIR}"
> +B = "${WORKDIR}/build"
> +
> +# Default kernel includes, these represent what are normally used for in-kernel
> +# sources.
> +KERNEL_INCLUDE ??= " \
> +        ${STAGING_KERNEL_DIR}/arch/${ARCH}/boot/dts \
> +        ${STAGING_KERNEL_DIR}/arch/${ARCH}/boot/dts/* \
> +        ${STAGING_KERNEL_DIR}/scripts/dtc/include-prefixes \
> +        "
> +
> +DT_INCLUDE[doc] = "Search paths to be made available to both the device tree compiler and preprocessor for inclusion."
> +DT_INCLUDE ?= "${DT_FILES_PATH} ${KERNEL_INCLUDE}"
> +DT_FILES_PATH[doc] = "Defaults to source directory, can be used to select dts files that are not in source (e.g. generated)."
> +DT_FILES_PATH ?= "${S}"
> +
> +DT_PADDING_SIZE[doc] = "Size of padding on the device tree blob, used as extra space typically for additional properties during boot."
> +DT_PADDING_SIZE ??= "0x3000"
> +DT_RESERVED_MAP[doc] = "Number of reserved map entires."
> +DT_RESERVED_MAP ??= "8"
> +DT_BOOT_CPU[doc] = "The boot cpu, defaults to 0"
> +DT_BOOT_CPU ??= "0"
> +
> +DTC_FLAGS ?= "-R ${DT_RESERVED_MAP} -p ${DT_PADDING_SIZE} -b ${DT_BOOT_CPU}"
> +DTC_PPFLAGS ?= "-nostdinc -undef -D__DTS__ -x assembler-with-cpp"
> +DTC_OFLAGS ?= "-@ -H epapr"
> +
> +python () {
> +    if d.getVar("KERNEL_INCLUDE"):
> +        # auto add dependency on kernel tree, but only if kernel include paths
> +        # are specified.
> +        d.appendVarFlag("do_compile", "depends", " virtual/kernel:do_configure")
> +}
> +
> +def expand_includes(varname, d):
> +    import glob
> +    includes = set()
> +    # expand all includes with glob
> +    for i in (d.getVar(varname) or "").split():
> +        for g in glob.glob(i):
> +            if os.path.isdir(g): # only add directories to include path
> +                includes.add(g)
> +    return includes
> +
> +def devicetree_source_is_overlay(path):
> +    # determine if a dts file is an overlay by checking if it uses "/plugin/;"
> +    with open(path, "r") as f:
> +        for i in f:
> +            if i.startswith("/plugin/;"):
> +                return True
> +    return False
> +
> +def devicetree_compile(dtspath, includes, d):
> +    import subprocess
> +    dts = os.path.basename(dtspath)
> +    dtname = os.path.splitext(dts)[0]
> +    bb.note("Processing {0} [{1}]".format(dtname, dts))
> +
> +    # preprocess
> +    ppargs = d.getVar("BUILD_CPP").split()
> +    ppargs += (d.getVar("DTC_PPFLAGS") or "").split()
> +    for i in includes:
> +        ppargs.append("-I{0}".format(i))
> +    ppargs += ["-o", "{0}.pp".format(dts), dtspath]
> +    bb.note("Running {0}".format(" ".join(ppargs)))
> +    subprocess.run(ppargs, check = True)
> +
> +    # determine if the file is an overlay or not (using the preprocessed file)
> +    isoverlay = devicetree_source_is_overlay("{0}.pp".format(dts))
> +
> +    # compile
> +    dtcargs = ["dtc"] + (d.getVar("DTC_FLAGS") or "").split()
> +    if isoverlay:
> +        dtcargs += (d.getVar("DTC_OFLAGS") or "").split()
> +    for i in includes:
> +        dtcargs += ["-i", i]
> +    dtcargs += ["-o", "{0}.{1}".format(dtname, "dtbo" if isoverlay else "dtb")]
> +    dtcargs += ["-I", "dts", "-O", "dtb", "{0}.pp".format(dts)]
> +    bb.note("Running {0}".format(" ".join(dtcargs)))
> +    subprocess.run(dtcargs, check = True)
> +
> +python devicetree_do_compile() {
> +    includes = expand_includes("DT_INCLUDE", d)
> +    listpath = d.getVar("DT_FILES_PATH")
> +    for dts in os.listdir(listpath):
> +        if not dts.endswith(".dts"):
> +            continue # skip non-.dts files
> +        dtspath = os.path.join(listpath, dts)
> +        devicetree_compile(dtspath, includes, d)
> +}
> +
> +devicetree_do_install() {
> +    for DTB_FILE in `ls *.dtb *.dtbo`; do
> +        install -Dm 0644 ${B}/${DTB_FILE} ${D}/boot/devicetree/${DTB_FILE}
> +    done
> +}
> +
> +devicetree_do_deploy() {
> +    for DTB_FILE in `ls *.dtb *.dtbo`; do
> +        install -Dm 0644 ${B}/${DTB_FILE} ${DEPLOYDIR}/devicetree/${DTB_FILE}
> +    done
> +}
> +addtask deploy before do_build after do_install
> +
> +EXPORT_FUNCTIONS do_compile do_install do_deploy
> +
> 

-- 
MARTIN HUNDEBØLL, Prevas A/S
Software Developer

Hedeager 3, DK-8200 Aarhus N
Phone +45 87438070
Mobile +45 25562438
Martin.Hundeboll at prevas.dk
www.prevas.com