[OE-core] [for-master][RFC PATCH 0/3] Support template files for wic wks

[Christopher Larson]

On Sun, May 8, 2016 at 9:05 AM, Jeremy ROSEN <jeremy.rosen at smile.fr> wrote:

> Le 02/05/2016 21:23, Christopher Larson a écrit :
>
>> From: Christopher Larson <chris_larson at mentor.com>
>>
>> This adds support for .wks.in files as WKS_FILE.
>>
>> These files are treated as the contents of a bitbake variable, so usual
>> bitbake variable references are supported. I considered using another
>> templating mechanism, for example the one used by yocto-layer, but then
>> we'd
>> end up largely mapping metadata variables to template fields anyway,
>> which is
>> a pointless indirection. Let bitbake expand the variables directly
>> instead.
>>
>> This feature lets us, for example, reference ${APPEND} in --append, and
>> avoid
>> hardcoding the serial console tty in the wks file, and let the user's
>> changes
>> to APPEND affect wic the way they do the other image construction
>> mechanisms.
>>
>> The template is read in and set in a variable at parse time, so changes
>> to the
>> variables referenced by the template will result in rebuilding the image.
>>
>> It's been suggested that the name should better reflect how its contents
>> are
>> used, i.e. .bbin or so. I've also considered including something like
>> 'bbv' or
>> similar, to indicate its contents are that of a bitbake value. I think
>> this
>> should be discussed.
>>
>> I'd love to hear questions or comments on this. I think the value is
>> clear,
>> and it's something we need to solve, but I'd like to hear if others agree
>> or
>> disagree on that, and whether this implementation is appropriate.
>>
>>
>>
>
> We have been discussing this a bit with Chris on IRC and i'll add a couple
> of thoughts we had so that discussion isn't completely lost.
>
> As a final user of yocto, I find the need for templating in wic very
> usefull, it would allow me to specify disk size in the machine
> specification and adapt the wic image accordingly, or use different disk
> names ( mmcblk vs sda)
>
> however my need is more generic and I think it would be interesting to
> consider a bigger problem : using yocto variables in configuration files.
> (I mean files in /etc/ here, not yocto configuration)
>
> It is a pretty common problem to have to tailor configuration files for
> particular machines, distro features, optional dependencies etc... this is
> possible to do using OVERRIDES but it can only be done at the whole-file
> level, not within a configuration file. Moreover it makes some complex
> cases very tricky. let's present an example
>
> It is pretty common to need to include the build-machine's ip address in a
> configuration file. This can be usefull to send logs if the build machine
> is also the developer's machine. The ip address of the build machine can be
> easily obtained via python and put into the bitbake environement, but using
> that variable to set a value in a random file in /etc is pretty tricky.
>
> similarly, I like to include my public ssh key as an authorized key for
> root on my builds, il makes testing way easier
>
> I think having a generic templating mechanism that would recognise some
> source files as needing template extension would be awesome. I don't
> understand fully how bitbake/yocto works, so I can't design a complete
> archtecture, but here are a couple of thought we had with Chris on IRC
>
> * recognise *.in in the same way *.patch are recognized. Is there a risk
> of improper expansion ? maybe specify that templating is needed as a
> parameter in SRC_URI
> * should expansion be in do_fetch or do_patch or somewhere else... images
> tend to skip do_patch, but do_fetch is on the bitbake side IIUC. would that
> be a problem ?
> * interaction with sstate and rebuilding needs to be considered... changes
> need to be detected post-templating obviously
>
> Overall, I really like the idea of this patch, there are lot of things in
> a .wks that need expansion, but I think there is even more in every other
> configuration file of the system.
>
> Hope my random thoughts helped



This is an interesting concept, certainly a generic templating mechanism
for source files may be of use. I expect we could get this merged, and open
a new yocto feature request in bugzilla to add a more generic mechanism,
possibly based on this sort of thing.

The main concern I'd have is making sure changes to the referenced
variables, or changes to the list of referenced variables, changes
appropriate checksums, so as was the case in this patch series, they'd have
to be read in at parse time to gather that info. As long as the files are
relatively small, it should be reasonable to do so, and I expect the common
case will be small config files, so I doubt it'd be a blocking issue.
-- 
Christopher Larson
clarson at kergoth dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Maintainer - Tslib
Senior Software Engineer, Mentor Graphics
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openembedded.org/pipermail/openembedded-core/attachments/20160512/59f4cd8c/attachment-0002.html>