[OE-core] [PATCH] recipe_links.bbclass: introduction

[Mark Hatle]

On 5/29/18 2:22 PM, Khem Raj wrote:
> On Mon, May 28, 2018 at 6:38 AM, Mark Asselstine
> <mark.asselstine at windriver.com> wrote:
>> This is a new class which can be used (for example via USER_CLASSES in
>> local.conf) to make your build more development friendly. When
>> included this class will create symlinks to the various bb and
>> bbappend files in WORKDIR.
>>
>> Normally when you are debugging or extending a package's recipe files
>> a developer will employ one of a few indirect techniques to determine
>> where bb and bbappends files associated with a recipe exist. For
>> example they might use bitbake-layers show-recipes or similar, or
>> simply rely on their experience to guide them. Even after working with
>> openembedded for serveral years now I find these techniques tedious
>> and time consuming, and sometimes even hit and miss.
>>
>> Since the whereabouts of these files are already stored in various
>> files at parse time we can create symlinks to simplify the task of
>> finding these files, making them available in WORKDIR for easy
>> inpsection and in a convenient location if using devshel for instance.
>>
>> For now this work is completely optional but we could conceivable make
>> this the default behavior if folks find it is convenient and the cost
>> of performing these operations across all builds is minimal enough.
>>
>> recipe_links can safely be added to USER_CLASSES for existing builds,
>> care has been taken to avoid this action causing anything to be
>> rebuilt. After this has been added you can either 'bitbake <recipe> -C
>> unpack' or 'bitbake <recipe> -c create_recipe_links' to cause the
>> links to be created in the WORKDIR for the specified recipe.
>>
> 
> I think this is not complementing devtool workflow, even though its
> not going to hurt it.
> however, devtool workflow is what we should be targeting as primary
> workflow for OE
> and I do not see a benefit of this in that regard.

I think the same could be said about other bbclasses that are not enabled.
devtool is not the only workflow, and a lot of people like to work directly
using 'bitbake -c devshell' or similar still.

As an optional user class, I do think this is reasonable and helpful to many folks.

(it also does not clash with the devtool workflow at all, so even if enabled a
devtool user won't be negatively impacted by it.)

--Mark

> 
>> Signed-off-by: Mark Asselstine <mark.asselstine at windriver.com>
>> ---
>>  meta/classes/recipe_links.bbclass | 79 +++++++++++++++++++++++++++++++++++++++
>>  meta/conf/documentation.conf      |  1 +
>>  2 files changed, 80 insertions(+)
>>  create mode 100644 meta/classes/recipe_links.bbclass
>>
>> diff --git a/meta/classes/recipe_links.bbclass b/meta/classes/recipe_links.bbclass
>> new file mode 100644
>> index 0000000..ea97605
>> --- /dev/null
>> +++ b/meta/classes/recipe_links.bbclass
>> @@ -0,0 +1,79 @@
>> +# Create symlink in WORKDIR to the various bb and
>> +# bbappend files used to define the package ops
>> +
>> +# Create a symlink given src and dst paths
>> +def create_link(d, src, dst):
>> +    if os.path.islink(dst):
>> +        try:
>> +            os.unlink(dst)
>> +        except OSError as err:
>> +            bb.error("  Failed to cleanup old link %s: %s"
>> +                         % (os.path.basename(dst), os.strerror(err.errno)))
>> +            return False
>> +
>> +
>> +    try:
>> +        os.symlink(src, dst)
>> +    except OSError as err:
>> +        bb.error("  Failed to create file link for %s: %s"
>> +                     % (src, os.strerror(err.errno)))
>> +        return False
>> +
>> +    return True
>> +
>> +# Ensure the work is scheduled. We do this as a func
>> +# to avoid sig changes causing things to be rebuilt
>> +# when the class is added/removed after the fact.
>> +do_unpack[postfuncs] += "create_recipe_links "
>> +do_unpack[vardepsexclude] += "create_recipe_links "
>> +python create_recipe_links() {
>> +    import re
>> +    import glob
>> +
>> +    workdir = d.getVar('WORKDIR')
>> +
>> +    # Main recipe file
>> +    bb.note("Add main recipe file link:")
>> +    bb_path = d.getVar('FILE')
>> +    bb_filename = os.path.basename(bb_path)
>> +    bb_destname = "%s/%s" % (workdir, bb_filename)
>> +    bb.note("  Linking %s" % bb_path)
>> +    if not create_link(d, bb_path, bb_destname):
>> +        return False
>> +
>> +    # Cleanup old bbappends links
>> +    bb.note("Removing old bbappend links:")
>> +    pn = d.getVar('PN')
>> +    files = glob.glob('%s/[0-9][0-9]_%s.bbappend' % (workdir,pn))
>> +    files += glob.glob('%s/[0-9][0-9]_%s_*.bbappend' % (workdir, pn))
>> +    for filename in files:
>> +        bb.note("  Removing: %s" % filename)
>> +        try:
>> +            os.unlink(filename)
>> +        except OSError as err:
>> +            bb.error("  Failed to cleanup old link %s: %s" %
>> +                         (os.path.basename(filename), os.strerror(err.errno)))
>> +            return False
>> +
>> +    # Add bbappends links
>> +    bb.note("Adding bbappend links:")
>> +    included_files = d.getVar('BBINCLUDED').split()
>> +    bbappend_re = re.compile( r".*/%s(_[^/]*)?\.bbappend$" % re.escape(pn))
>> +    for filename in included_files:
>> +        if bbappend_re.match(filename):
>> +            bb.note("  Linking %s" % filename)
>> +            destname = "00_%s" % (os.path.basename(filename))
>> +            while os.path.exists("%s/%s" % (workdir, destname)):
>> +                 destname = str(int(destname[:2]) + 1).zfill(2) + destname[2:]
>> +            if not create_link(d, filename, "%s/%s" % (workdir, destname)):
>> +                return False
>> +
>> +    return True
>> +}
>> +
>> +# In addition to the func we want to make things able to be (re)run
>> +# easily by the user so ensure it is available as a task.
>> +addtask do_create_recipe_links
>> +python do_create_recipe_links() {
>> +    bb.build.exec_func("create_recipe_links", d)
>> +}
>> diff --git a/meta/conf/documentation.conf b/meta/conf/documentation.conf
>> index 254f8f1..2f6e686 100644
>> --- a/meta/conf/documentation.conf
>> +++ b/meta/conf/documentation.conf
>> @@ -21,6 +21,7 @@ do_devpyshell[doc] = "Starts an interactive Python shell for development/debuggi
>>  do_devshell[doc] = "Starts a shell with the environment set up for development/debugging"
>>  do_diffconfig[doc] = "Compares the old and new config files after running do_menuconfig for the kernel"
>>  do_fetch[doc] = "Fetches the source code"
>> +do_create_recipe_links[doc] = "Creates links to bb and bbappend files in a package's WORKDIR"
>>  do_install[doc] = "Copies files from the compilation directory to a holding area"
>>  do_install_ptest_base[doc] = "Copies the runtime test suite files from the compilation directory to a holding area"
>>  do_kernel_checkout[doc] = "Checks out source/meta branches for a linux-yocto style kernel"
>> --
>> 2.7.4
>>
>> --
>> _______________________________________________
>> Openembedded-core mailing list
>> Openembedded-core at lists.openembedded.org
>> http://lists.openembedded.org/mailman/listinfo/openembedded-core