[OE-core] [PATCH 3/3] image_types.bbclass: support template .wks.in files for wic

Tue May 24 17:45:27 UTC 2016

On Tue, May 24, 2016 at 9:07 AM, Christopher Larson <kergoth at gmail.com>
wrote:

>
> On Tue, May 24, 2016 at 7:26 AM, Christopher Larson <kergoth at gmail.com>
> wrote:
>
>>
>> On Tue, May 24, 2016 at 2:13 AM, Richard Purdie <
>> richard.purdie at linuxfoundation.org> wrote:
>>
>>> On Mon, 2016-05-23 at 13:34 -0700, Christopher Larson wrote:
>>> > From: Christopher Larson <chris_larson at mentor.com>
>>> >
>>> > These files are treated as the contents of a bitbake variable, so
>>> > usual
>>> > bitbake variable references are supported. I considered using another
>>> > templating mechanism, for example the one used by yocto-layer, but
>>> > then we'd
>>> > end up largely mapping metadata variables to template fields anyway,
>>> > which is
>>> > a pointless indirection. Let bitbake expand the variables directly
>>> > instead.
>>> >
>>> > This feature lets us, for example, reference ${APPEND} in --append,
>>> > and avoid
>>> > hardcoding the serial console tty in the wks file, and let the user's
>>> > changes
>>> > to APPEND affect wic the way they do the other image construction
>>> > mechanisms.
>>> >
>>> > The template is read in and set in a variable at parse time, so
>>> > changes to the
>>> > variables referenced by the template will result in rebuilding the
>>> > image.
>>> >
>>> > Signed-off-by: Christopher Larson <chris_larson at mentor.com>
>>> > ---
>>> >  meta/classes/image_types.bbclass | 35
>>> > +++++++++++++++++++++++++++++++++++
>>> >  1 file changed, 35 insertions(+)
>>> >
>>> > diff --git a/meta/classes/image_types.bbclass
>>> > b/meta/classes/image_types.bbclass
>>> > index dc681ae..caf8757 100644
>>> > --- a/meta/classes/image_types.bbclass
>>> > +++ b/meta/classes/image_types.bbclass
>>> > @@ -206,6 +206,16 @@ IMAGE_CMD_wic () {
>>> >  }
>>> >  IMAGE_CMD_wic[vardepsexclude] = "WKS_FULL_PATH WKS_FILES"
>>> >
>>> > +python process_wks_template () {
>>> > +    """Write out expanded template contents to WKS_FULL_PATH."""
>>> > +    template_body = d.getVar('_WKS_TEMPLATE', True)
>>> > +    if template_body:
>>> > +        wks_file = d.getVar('WKS_FULL_PATH', True)
>>> > +        with open(wks_file, 'w') as f:
>>> > +            f.write(template_body)
>>> > +}
>>> > +do_image_wic[prefuncs] += 'process_wks_template'
>>> > +
>>> >  # Rebuild when the wks file or vars in WICVARS change
>>> >  USING_WIC = "${@bb.utils.contains_any('IMAGE_FSTYPES', 'wic ' + '
>>> > '.join('wic.%s' % c for c in '${COMPRESSIONTYPES}'.split()), '1', '',
>>> > d)}"
>>> >  WKS_FILE_CHECKSUM = "${@'${WKS_FULL_PATH}:%s' %
>>> > os.path.exists('${WKS_FULL_PATH}') if '${USING_WIC}' else ''}"
>>> > @@ -302,3 +312,28 @@ IMAGE_TYPES_MASKED ?= ""
>>> >  # The WICVARS variable is used to define list of bitbake variables
>>> > used in wic code
>>> >  # variables from this list is written to <image>.env file
>>> >  WICVARS ?= "BBLAYERS DEPLOY_DIR_IMAGE HDDDIR IMAGE_BASENAME
>>> > IMAGE_BOOT_FILES IMAGE_LINK_NAME IMAGE_ROOTFS INITRAMFS_FSTYPES
>>> > INITRD ISODIR MACHINE_ARCH ROOTFS_SIZE STAGING_DATADIR
>>> > STAGING_DIR_NATIVE STAGING_LIBDIR TARGET_SYS"
>>> > +
>>> > +python () {
>>> > +    """Read in and set up wks file template for wic."""
>>> > +    if d.getVar('USING_WIC', True):
>>> > +        wks_file_u = d.getVar('WKS_FULL_PATH', False)
>>> > +        wks_file = d.expand(wks_file_u)
>>> > +        base, ext = os.path.splitext(wks_file)
>>> > +        if ext == '.in' and os.path.exists(wks_file):
>>> > +            wks_out_file = os.path.join(d.getVar('WORKDIR', True),
>>> > os.path.basename(base))
>>> > +            d.setVar('WKS_FULL_PATH', wks_out_file)
>>> > +            d.setVar('WKS_TEMPLATE_PATH', wks_file_u)
>>> > +            d.setVar('WKS_FILE_CHECKSUM',
>>> > '${WKS_TEMPLATE_PATH}:True')
>>> > +
>>> > +            try:
>>> > +                with open(wks_file, 'r') as f:
>>> > +                    body = f.read()
>>> > +            except (IOError, OSError) as exc:
>>> > +                pass
>>>
>>> I'm a little nervous about determinism here. Shouldn't it either find
>>> the referenced file or error?
>>
>>
>> That's a fair point, I think I was concerned about parse time errors,
>> since they tend to blow up into a lot of unpleasantness for the user, since
>> parse time errors in classes can cause errors for multiple recipes. If
>> no file is read in at parse time, there'll be no contents to write out,
>> and we could then error when trying to build the image with wic, as no
>> wks file will exist.
>>
>> Of course, the file being missing will change checksums for the image
>> tasks, but I think that's reasonable.
>>
>> Thoughts?
>>
>
> I'll do a bit more testing on the behaviors and get back to you on that
> later. Thanks.

If the WKS_FILE doesn't exist, whether we're using templates or not, it'll
fail in do_image_wic:

ERROR: core-image-sato-1.0-r0 do_image_wic: No kickstart files from
WKS_FILES were found: /scratch/dogwood/minnowmax/build/mkefidiskf.wks.in
core-image-sato.wks. Please set WKS_FILE or WKS_FILES appropriately.
ERROR: core-image-sato-1.0-r0 do_image_wic: Function failed: do_image_wic
(log file is located at
/scratch/dogwood/minnowmax/build/tmp/work/intel_corei7_64-mel-linux/core-image-sato/1.0-r0/temp/log.do_image_wic.12181)

If the template doesn't exist, then no template handling will be done at
all, so do_image_wic won't depend on any of the variables referenced in the
template, and the task will fail. If it does, then we will depend on its
contents and referenced variables, and the task will not fail.

I don't really see a problem with determinism there. Either the file exists
at parse time or it doesn't, and the behavior seems reasonable in either
case.
-- 
Christopher Larson
kergoth at gmail dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Maintainer - Tslib
Senior Software Engineer, Mentor Graphics
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openembedded.org/pipermail/openembedded-core/attachments/20160524/35fd741b/attachment-0002.html>