[bitbake-devel] [PATCH 1/1] data_smart.py: make use of expand cache in getVar()

[Xu, Dongxiao]

Hi Richard,

> -----Original Message-----
> From: Xu, Dongxiao
> Sent: Monday, August 29, 2011 9:16 PM
> To: Richard Purdie
> Cc: bitbake-devel at lists.openembedded.org
> Subject: RE: [bitbake-devel] [PATCH 1/1] data_smart.py: make use of expand
> cache in getVar()
> 
> > -----Original Message-----
> > From: Richard Purdie [mailto:richard.purdie at linuxfoundation.org]
> > Sent: Monday, August 29, 2011 9:13 PM
> > To: Xu, Dongxiao
> > Cc: bitbake-devel at lists.openembedded.org
> > Subject: RE: [bitbake-devel] [PATCH 1/1] data_smart.py: make use of
> > expand cache in getVar()
> >
> > On Mon, 2011-08-29 at 21:07 +0800, Xu, Dongxiao wrote:
> > > Hi Richard,
> > >
> > > > -----Original Message-----
> > > > From: Richard Purdie [mailto:richard.purdie at linuxfoundation.org]
> > > > Sent: Monday, August 29, 2011 8:55 PM
> > > > To: Xu, Dongxiao
> > > > Cc: bitbake-devel at lists.openembedded.org
> > > > Subject: Re: [bitbake-devel] [PATCH 1/1] data_smart.py: make use
> > > > of expand cache in getVar()
> > > >
> > > > On Mon, 2011-08-29 at 15:33 +0800, Dongxiao Xu wrote:
> > > > > Currently if passing expand=True to getVar() function, it will
> > > > > pass the handling to getVarFlag(), which doesn't get any benefit
> > > > > from the expand cache.
> > > > >
> > > > > Call the expand() function separately in getVar() to make use of
> > > > > the expand cache, which can decrease the parsing time by 40%.
> > > > > (from current 49s to 27s)
> > > > >
> > > > > Signed-off-by: Dongxiao Xu <dongxiao.xu at intel.com>
> > > > > ---
> > > > >  lib/bb/data_smart.py |    7 ++++++-
> > > > >  1 files changed, 6 insertions(+), 1 deletions(-)
> > > > >
> > > > > diff --git a/lib/bb/data_smart.py b/lib/bb/data_smart.py index
> > > > > 301f9e3..d8ba24f 100644
> > > > > --- a/lib/bb/data_smart.py
> > > > > +++ b/lib/bb/data_smart.py
> > > > > @@ -268,7 +268,12 @@ class DataSmart(MutableMapping):
> > > > >          self.dict[var]["content"] = value
> > > > >
> > > > >      def getVar(self, var, expand=False, noweakdefault=False):
> > > > > -        return self.getVarFlag(var, "content", expand, noweakdefault)
> > > > > +        value = self.getVarFlag(var, "content", False,
> > > > > + noweakdefault)
> > > > > +
> > > > > +        # Call expand() separately to make use of the expand cache
> > > > > +        if expand and value:
> > > > > +            return self.expand(value, var)
> > > > > +        return value
> > > > >
> > > >
> > > > Why can't we have getVarFlag use the expand cache?
> > >
> > > For expand cache, the format is a list: expand_cache[varname].
> > > It doesn't contain information of flag values.
> >
> > Good point.
> >
> > I'm left wondering why we don't key that cache on the unexpanded value
> > rather than a specific variable name (or name+flag).
> >
> > I'll take this patch but its something we should consider...
> 
> Thanks, I will step into this point further and see if (name+flag) type of cache
> could benefit us more.

Sorry I must misunderstand your point just now...

For caching of unexpanded value, there are too many/uncertainty of types, like:

${TUNE_ARCH}
${@bb.parse.BBHandler.vars_from_file(bb.data.getVar('FILE',d),d)[0] or 'defaultpkgname'}
${@bb.utils.contains("TUNE_FEATURES", "m32", "${X86ARCH32}", "" ,d)}${@bb.utils.contains("TUNE_FEATURES", "mx32", "${X86ARCH64}", "" ,d)}${@bb.utils.contains("TUNE_FEATURES", "m64", "${X86ARCH64}", "" ,d)}

For simple variables, it is easy to cache, however for the expressions, it may not suitable to cache.

Besides, I also tried the (name+flag) types of cache, but it doesn't bring benefit (from 27s to 28s). Therefore it is suppose that most of the calling should be getVar(). Those specifically getting flag value should be not much.

Thanks,
Dongxiao

> 
> Thanks,
> Dongxiao
> 
> >
> > Cheers,
> >
> > Richard
> >
> >