[OE-core] RFC: Maintain backwards compatibility or not for module-base.bbclass

[Peter Kjellerstedt]

I will aggregate my responses to Koen, Bruce and Phil below.

> -----Original Message-----
> From: Koen Kooi [mailto:koen at dominion.thruhere.net]
> Sent: den 16 januari 2014 20:19
> To: Peter Kjellerstedt
> Cc: OE Core (openembedded-core at lists.openembedded.org); Phil Blundell
> Subject: Re: [OE-core] RFC: Maintain backwards compatibility or not for
> module-base.bbclass
> 
> Op 16 jan. 2014, om 14:58 heeft Peter Kjellerstedt
> <peter.kjellerstedt at axis.com> het volgende geschreven:
> 
> > Background: Back in September, Richard made a commit to
> > linux-libc-headers.inc describing why one should not fork the
> > linux-libc-headers recipe:
> >
> > http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=babbf7a46acaefd9b36031483cafce053f607e66
> >
> > As a result I created a local bbclass for our layer called
> > kernel-headers, which provides a recipe with kernel headers
> > that match the actually used version of the Linux kernel.
> > This was needed for packages that need to access hardware
> > specific features that are not present in the generic kernel
> > headers provided by linux-libc-headers.
> >
> > My intention for this class was that it should be generic
> > enough to be able to upstream it to OE Core.
> >
> > Now, the other day a colleague of mine had a build failure due
> > to this class. It turned out that even though the class adds a
> > dependency on virtual/kernel and then uses the files that are
> > installed to ${STAGING_KERNEL_DIR} when running oe_runmake
> > headers_install, the command could fail because the
> > ${STAGING_KERNEL_DIR}/scripts was not populated. After asking
> > Richard about this, I learned that this is due to problems
> > with the sstate cache and not knowing whether a 32 bit host or
> > a 64 bit host was used to generate the files. Thus I also
> > learned that the scripts are actually built as a result of
> > building modules.
> >
> > Since I did not want my class to depend on modules having been
> > built, I looked into modules.bbclass and modules-base.bbclass.
> > There I found the function do_make_scripts() which is
> > responsible for building  the kernel scripts. However, the
> > current setup doesn't lend itself very well to use the
> > modules-base.bbclass for something other than modules.
> >
> > My idea then was to break this part out into a separate class,
> > kernel-scripts, which I did. You can find both the
> > kernel-scripts and kernel-headers classes here:
> >
> > http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=pkj/kernel-headers
> >
> > When I showed this to Richard he noted that my change was not
> > backwards compatible (as I no longer provide the
> > do_make_scripts() function from the module-base.bbclass).
> > However, there is nothing besides module.bbclass in OE Core
> > and meta-oe that use the module-base.bbclass.
> 
> Can't do_make_script run as an sstate post function?

I think that is exactly what the reverted change Bruce mentions 
below did.

> regards,
> 
> Koen

> -----Original Message-----
> From: Bruce Ashfield [mailto:bruce.ashfield at gmail.com]
> Sent: den 16 januari 2014 19:41
> To: Peter Kjellerstedt
> Cc: OE Core (openembedded-core at lists.openembedded.org); Phil Blundell
> Subject: Re: [OE-core] RFC: Maintain backwards compatibility or not for
> module-base.bbclass
> 
> On Thu, Jan 16, 2014 at 8:58 AM, Peter Kjellerstedt
> <peter.kjellerstedt at axis.com> wrote:
> > Anyway, I made a modified version that does maintain backwards
> > compatibility for module-base.bbclass here:
> >
> > http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=pkj/kernel-headers-backwards-compatible
> >
> > This time Richard complained about the extra class
> > (kernel-scripts-base.bbclass), and noted that there was no
> > way to win... He then suggested that I take the question of
> > whether we need to maintain backwards compatibility for
> > modules-base.bbclass to the mailing list.
> 
> FWIW: I agree that have to many small, single purpose
> kernel-*.bbclasses is a pain, since it provides granularity, but 
> more opportunity for varying behaviour during kernel builds.
> 
> I have uses of the module-base.bbclass and an expectation that it 
> will generate the scripts, largely around the SDK and some custom 
> kernel recipes. So they only inherit module.bbclass, and would be 
> impacted if that functionality was changed to require another inherit.

As long as they inherit module (and not add task dependencies on 
do_make_scripts) they should not be affected. It is only if you 
inherit module-base directly and expect the do_make_scripts() 
function to exist that you are affected.

> Speaking of that, we through something like this late last year 
> with automatically restoring the scripts into the sysroot, which 
> ended up being reverted:
> 
> see b2c948d56241ff7cdea2e9e68b740f305c72f5ca in oe-core
> 
> At least the module (and your scripts) class avoid the sstate 
> problems and compiler dependencies that we hit with that solution.

Interesting. It looks like it would have been a good solution 
if it had worked... 

> What are the alternatives to more classes, isn't this something 
> that could be a .inc routine ? And modules simply includes it, and 
> you can do the same .. but I suppose a .inc versus a class inherit 
> is largely semantics in the difference.

Tbh, I think keeping it as a class makes it more straight forward.
And class or include file is merely semantics.

> bottom line, my rambling says that backwards compatibility matters 
> here, and that if we can avoid a new class, that would also be a 
> good thing.

But I think that it should be easy enough for any external party 
to adopt to the new class if they do indeed have a direct 
dependency on module-base and expecting to find do_make_scripts() 
there. It is not like we would change it in the middle of Dora 
anyway. Thus I advocate for the change above on the kernel-headers 
branch that does not maintain backwards compatibility.

> Cheers,
> 
> Bruce

> -----Original Message-----
> From: openembedded-core-bounces at lists.openembedded.org
> [mailto:openembedded-core-bounces at lists.openembedded.org] On Behalf Of
> Phil Blundell
> Sent: den 16 januari 2014 22:39
> To: Peter Kjellerstedt
> Cc: OE Core (openembedded-core at lists.openembedded.org)
> Subject: Re: [OE-core] RFC: Maintain backwards compatibility or not for
> module-base.bbclass
> 
> On Thu, 2014-01-16 at 14:58 +0100, Peter Kjellerstedt wrote:
> > So, here I am now. I do not know who else use the
> > do_make_scripts() function from module-base.bbclass and in what
> > way, and whether restructuring the functionality into the new
> > kernel-scripts.bbclass without maintaining backwards
> > compatibility would be a problem. If you know anything about
> > this, please let me know.
> 
> I'm not entirely clear why you couldn't maintain compatibility 
> by moving the task in question to a new class but keeping its 
> name the same (i.e. refrain from renaming "do_make_scripts" to 
> "do_kernel_scripts") and having module-base.bbclass simply 
> inherit the newly-added class.  That seems like it ought to be 
> fairly straightforward and uncontroversial.

Well, that is basically what my backwards compatible branch above 
does. However, it introduces two new classes since it needs to 
maintain the split between where the addtask is done and the 
actual function.

> That said, though, I don't think there is any pressing need to 
> maintain backwards compatibility around module-base.bbclass.  In 
> fact, I think it would probably be fine for module-base.bbclass 
> to simply go away altogether and have its functionality subsumed 
> into module.bbclass; the split between those two classes is 
> mostly a relic of yesteryear and I can't think of any good 
> purpose that it serves nowadays.  So I would be happy enough to 
> see that (and the tangly mess that is the kernel classes in 
> general) cleaned up irrespective of what happens with
> do_make_scripts.

I won't argue with that. However, I do not feel that I am 
comfortable enough about these classes to do it. I had a quick 
look around in OE core, and I believe you are right. The number 
of references to "module-base" were limited, and in most cases 
it should just be a matter of replacing them with "module".

> p.

//Peter