[OE-core] [PATCHv3 1/1] native/nativesdk: Use fixed DISTRO_FEATURES

[Peter Kjellerstedt]

> -----Original Message-----
> From: Richard Purdie [mailto:richard.purdie at linuxfoundation.org]
> Sent: den 11 april 2017 19:16
> To: Peter Kjellerstedt <peter.kjellerstedt at axis.com>; Jussi Kukkonen
> <jussi.kukkonen at intel.com>; openembedded-core at lists.openembedded.org
> Subject: Re: [OE-core] [PATCHv3 1/1] native/nativesdk: Use fixed
> DISTRO_FEATURES
> 
> On Tue, 2017-04-11 at 15:32 +0000, Peter Kjellerstedt wrote:
> > >
> > > -----Original Message-----
> > > From: openembedded-core-bounces at lists.openembedded.org
> > > [mailto:openembedded-core-bounces at lists.openembedded.org] On Behalf
> > > Of
> > > Jussi Kukkonen
> > > Sent: den 11 april 2017 16:36
> > > To: openembedded-core at lists.openembedded.org
> > > Subject: [OE-core] [PATCHv3 1/1] native/nativesdk: Use fixed
> > > DISTRO_FEATURES
> > >
> > > There seems to be little advantage to letting distro features
> > > affect
> > > native builds. There is a significant disadvantage: a change to
> > > DISTRO_FEATURES will trigger a lot of unnecessary native tasks. In
> > > a
> > > test like this:
> > >   $ bitbake core-image-minimal
> > >   # append " systemd" to DISTRO_FEATURES
> > >   $ bitbake core-image-minimal
> > > The latter build takes 44 minutes (28%) of cpu-time less with this
> > > patch (skipping 135 native tasks). Sadly wall clock time was not
> > > affected as glibc remains the bottleneck.
> > >
> > > Set native distro features to DISTRO_FEATURES_NATIVE appended with
> > > an intersection of DISTRO_FEATURES and
> > > DISTRO_FEATURES_FILTER_NATIVE.
> > > Current default values (baitbake.conf) are
> > > * DISTRO_FEATURES_FILTER_NATIVE ?= "api-documentation" (as gtk-doc-
> > > native
> > > has much less dependencies when built without it)
> > > * DISTRO_FEATURES_NATIVE ?= "x11" (to enable native UIs even if
> > > target
> > > does not containe them)
> > >
> > > Do the variable setting in native_virtclass_handler() because
> > > otherwise
> > > it could still be overridden by appends and the feature
> > > backfilling.
> > > Shuffle the early returns so DISTRO_FEATURES gets set as long as
> > > the packagename ends with "-native".
> > >
> > > Add similar variables for nativesdk.
> > >
> > > Signed-off-by: Jussi Kukkonen <jussi.kukkonen at intel.com>
> > > ---
> > >  meta/classes/native.bbclass    | 14 ++++++++++----
> > >  meta/classes/nativesdk.bbclass |  6 ++++++
> > >  meta/conf/bitbake.conf         |  9 +++++++++
> > >  3 files changed, 25 insertions(+), 4 deletions(-)
> > >
> > > diff --git a/meta/classes/native.bbclass
> > > b/meta/classes/native.bbclass
> > > index 1919fbc..aec1087 100644
> > > --- a/meta/classes/native.bbclass
> > > +++ b/meta/classes/native.bbclass
> > > @@ -121,14 +121,20 @@ PATH_prepend = "${COREBASE}/scripts/native-
> > > intercept:"
> > >  SSTATE_SCAN_CMD ?= "${SSTATE_SCAN_CMD_NATIVE}"
> > >
> > >  python native_virtclass_handler () {
> > > -    classextend = e.data.getVar('BBCLASSEXTEND') or ""
> > > -    if "native" not in classextend:
> > > -        return
> > > -
> > >      pn = e.data.getVar("PN")
> > >      if not pn.endswith("-native"):
> > >          return
> > >
> > > +    # Set features here to prevent appends and distro features
> > > backfill
> > > +    # from modifying native distro features
> > > +    features = set(d.getVar("DISTRO_FEATURES_NATIVE").split())
> > > +    filtered = set(bb.utils.filter("DISTRO_FEATURES",
> > > d.getVar("DISTRO_FEATURES_FILTER_NATIVE"), d).split())
> > > +    d.setVar("DISTRO_FEATURES", " ".join(features | filtered))
> > You should sort the list of features to make it deterministic.
> 
> Do we sort DISTRO_FEATURES anywhere else?

No, but it is never worked upon via set() as these are here...

> I thought we only accessed DISTRO_FEATURES with functions which have
> support in bitbake (contains/filter) which means that should be
> unnecessary?

Will the lack of sorting not affect the task hashes (and bitbake -e) if 
these are set via set() operations, which may result in the order of 
the features listed in the final DISTRO_VARIABLE from varying based on 
how Python happens to pull the features out of the set()?

> Cheers,
> 
> Richard

//Peter