[bitbake-devel] [OE-core] RFC: exposing information about the SRC_URI(s)/branch via buildhistory (or similar mechanism)

[chris.laplante at agilent.com]

> > I think this supports my point about being more interested in patches
> > allowing people to extend/customise buildhistory than just adding X.
> >
> > Whilst we want to have good defaults, there are always going to be
> > niche cases for people wanting to extend it...
> 
> 
> Agreed. Then we can implement our BRANCH scheme without polluting the core code with it.

Richard, et al. - 

I have created the cpl/buildhistory branch on poky-contrib to share a draft of my ideas for making buildhistory more modular and extensible. Would appreciate any comments.

The first change was to break up buildhistory.bbclass. Previously it was ~1000 lines, one of the biggest bbclasses in meta/classes. Now, buildhistory just contains common functionality and plumbing for the buildhistory features. Each feature listed in BUILDHISTORY_FEATURES causes a class to be automatically inherited, with a naming convention of "buildhistory-{feature}". I created buildhistory-package, buildhistory-task, buildhistory-sdk, and buildhistory-image.

Users could extend buildhistory by inheriting from buildhistory directly (e.g. if they have a completely new type of buildhistory to implement), or they can inherit from a feature class. For example, buildhistory-my-custom-packaging.bbclass. Then, they would use something like the following:

    BUILDHISTORY_FEATURES_remove = "package"
    BUILDHISTORY_FEATURES_append = " my-custom-packaging"

Alternatively, they could just add "buildhistory-my-custom-packaging" to INHERIT. 

Some features need to do additional things in the buildhistory event handler, e.g. "task" writes task signatures during commit. For this, I created BUILDHISTORY_COMMIT_PREFUNCS, which nicely decouples "task" from the base buildhistory class.

The only other big change (for now) on this branch is to abstract the buildhistory-packaging feature a bit. I do this by introducing several "hooks":

    BUILDHISTORY_PACKAGE_HOOKS - Python functions or BB functions run at the end of buildhistory_emit_pkghistory. Currently used to invoke buildhistory_list_pkg_files. 
    BUILDHISTORY_PACKAGE_RECIPEHISTORY_HOOKS - Python functions invoked in write_recipehistory, providing an opportunity to write additional lines to the per-recipe "latest" file. 
    BUILDHISTORY_PACKAGE_PACKAGEHISTORY_HOOKS - Python functions invoked in write_packagehistory providing an opportunity to write additional lines to the per-package "latest" file.
    BUILDHISTORY_PACKAGE_SRCREV_HOOKS - Python functions invoked in write_latest_srcrev, called once per srcrev-supporting SRC_URI. 
    BUILDHISTORY_PACKAGE_TAGSRCREV_HOOKS - Python functions invoked in write_latest_srcrev, called once per tag srcrev SRC_URI.

Additionally, I have re-implemented the latest_srcrev functionality in terms of BUILDHISTORY_PACKAGE_SRCREV_HOOKS and BUILDHISTORY_PACKAGE_TAGSRCREV_HOOKS.

An example usage of BUILDHISTORY_PACKAGE_SRCREV_HOOKS could be the following:

    def branch_hook(d, name, srcrev, ud, total_count, f): 
        if total_count == 1:
            f.write('BRANCH = "{0}"\n'.format(ud.branches[name]))
        else:
            f.write('BRANCH_{0} = "{1}"\n'.format(name, ud.branches[name]))

    BUILDHISTORY_PACKAGE_SRCREV_HOOKS_append = " branch_hook"


Things I'm currently unsure about:
    1. Variable naming - I thought about calling them _PREFUNCS or _POSTFUNCS instead of _HOOKS, or something like that. But the problem is that some of them can only be Python functions, not BB functions. Which leads me to:
    2. Most hooks are intended to be Python functions that are found via looking in globals(). If someone has a better solution I'd be interested. But I also don't think this one is necessarily bad.
    3. I'm trying to err on the side of not adding too many hooks for the first release. Hopefully the set I have chosen to add is seen as useful.
    4. I wonder if I should create a "buildhistory" directory under "classes" into which I would move the feature classes. To avoid polluting the "classes" directory.

Additional stuff to explore:
    1. Adding a buildhistory-logs feature class, which archives the log.do_{TASK} for a user-configurable selection of tasks. Default would be compile, configure, and install. 
    2. Adding other extension points where it makes sense.

Thanks,
Chris