[OE-core] backfill mechanism

[Peter Kjellerstedt]

Be careful with the _remove operator in common configuration files. 
Once it has been used there is absolutely no way of getting the 
removed value back in, for testing or for real. 

E.g., if you were to change the:

DISTRO_FEATURES_BACKFILL_CONSIDERED += "sysvinit"

in poky-tiny.conf to: 

DISTRO_FEATURES_remove = "sysvinit"

then it would be impossible to use the poky-tiny distro in a 
configuration where you also would like to enable sysvinit.

Also, after having looked at the Git history for when the backfill 
feature was implemented, I do not think DISTRO_FEATURES_remove will 
work, for the same reason DISTRO_FEATURES_append is not used to add 
the backfilled values (the first implementation actually did use 
_append). The reason it was changed is because the _append (and 
_remove) are evaluated too late. I.e., code like this that use a 
conditional based on DISTRO_FEATURES to do inherit would not 
work as intended for a backfilled value:

inherit ${@bb.utils.contains('DISTRO_FEATURES', 'sysvinit', '', 'systemd', d)}

A better solution for how to replace the backfill considered 
mechanism would be to use individual variables for the backfilled 
values. E.g., the current definition in bitbake.conf:

MACHINE_FEATURES_BACKFILL = "rtc qemu-usermode" 

could be replaced with something like:

MACHINE_FEATURES_BACKFILL = " \
    ${MACHINE_FEATURES_BACKFILL_qemu-usermode} \
    ${MACHINE_FEATURES_BACKFILL_rtc} \
"
MACHINE_FEATURES_BACKFILL_qemu-usermode = "qemu-usermode"
MACHINE_FEATURES_BACKFILL_rtc = "rtc"

Then to do the equivalent of:

MACHINE_FEATURES_BACKFILL_CONSIDERED += "rtc"

one would instead do:

MACHINE_FEATURES_BACKFILL_rtc = ""

Alternatively one could use variable flags:

MACHINE_FEATURES_BACKFILL[qemu-usermode] = "1"
MACHINE_FEATURES_BACKFILL[rtc] = "1"

and then simply set it to 0 if one does not want a feature:

MACHINE_FEATURES_BACKFILL[rtc] = "0"

That would of course take a little bit of Python code to 
convert the variable flags into actual features:

MACHINE_FEATURES_BACKFILL = "${@' '.join(sorted(key for key in d.getVarFlags('MACHINE_FEATURES_BACKFILL').keys() if d.getVarFlag('MACHINE_FEATURES_BACKFILL', key) == '1'))}"

(By explicitly testing for "1" we also avoid converting 
MACHINE_FEATURES_BACKFILL[doc] into a feature.)

I also have a hunch that something like this would be needed:

MACHINE_FEATURES_BACKFILL[vardepvalue] = "${MACHINE_FEATURES_BACKFILL}"

which probably leads to recursion unless the key "vardepvalue" is 
explicitly avoided in the if test above. Meh...

//Peter

> -----Original Message-----
> From: openembedded-core-bounces at lists.openembedded.org
> [mailto:openembedded-core-bounces at lists.openembedded.org] On Behalf Of
> Slater, Joseph
> Sent: den 21 november 2017 20:11
> To: Andre McCurdy <armccurdy at gmail.com>
> Cc: openembedded-core at lists.openembedded.org
> Subject: Re: [OE-core] backfill mechanism
> 
> The problem is that when backfilling is done, TUNE_FEATURES, etc, all
> have values associated with the base arch, not the current multilib.
> The "generic" solution, below, should work, although I would still use
> overrides when available.
> 
> Beyond what actually works, if the special backfill processing can be
> eliminated and regular constructs used I think that would be desirable.
> 
> Joe
> ________________________________________
> From: Andre McCurdy [armccurdy at gmail.com]
> Sent: Monday, November 20, 2017 7:24 PM
> To: Slater, Joseph
> Cc: openembedded-core at lists.openembedded.org
> Subject: Re: [OE-core] backfill mechanism
> 
> On Fri, Nov 17, 2017 at 3:32 PM, Slater, Joseph
> <joe.slater at windriver.com> wrote:
> > The backfill mechanism is not compatible with multilib.
> 
> Did anyone ever explain why?
> 
> > This could possibly
> > be fixed, but the backfill_considered functionality is also obscure,
> so I
> > think in at least the machine related .inc files we should replace
> lines
> > like
> >
> > MACHINE_FEATURES_BACKFILL_CONSIDERED_append = "
> > ${@bb.utils.contains('TUNE_FEATURES', 'n32', 'qemu-usermode', '',
> d)}"
> >
> > with lines like
> >
> > MACHINE_FEATURES_remove_mipsarchn32 = " qemu-usermode"
> 
> That only works for TUNE_FEATURES which happen to have a corresponding
> over-ride, so may not be a generic solution. Does something like:
> 
>   MACHINE_FEATURES_remove = "${@bb.utils.contains('TUNE_FEATURES',
> 'n32', 'qemu-usermode', '', d)}"
> 
> work for multilib?
> 
> > There are two advantages:  the second line works for multilib, and it
> is far
> > more readable.
> >
> > Joe
> >
> --
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core at lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core