[bitbake-devel] [PATCH 00/16] update_data() removal patchset

[Richard Purdie]

Removing the need for finalization or update_data is something we've
idly thought about for a long time. Basically, the whole idea of a
specific finalization phase to the data store causes a lof of issues and
horrible corner cases. For example, should bitbake call update_data,
then expandKeys, or the other way around? Or should we keep calling
update_data until nothing changes?

Through careful thought of some operations and working around the corner
case problems, we've avoided most of the issues however there are some
things we keep observing:

* We keep needing "expanded" data stores in different parts of the 
  system alongside their unexpanded versions
* Few people understand why/when exactly they should call update_data()
* Its near impossible to set PREFERRED_PROVIDERS of recipes which are 
  dynamically named (gcc-cross-${TARGET_ARCH}) along with overrides. 
  This is something meta-darwin/mingw/baremetal could really use.

The alternative is a datastore which dynamically adjusts when anything
changes so that the work is done at getVar time rather than precaching
things at update_data time.

What is missing from the branch as yet is removal of the "expanded" data
stores from use within bitbake and starting to remove the update_data
calls (which are now just empty calls). This can happen once the code
is merged/stablising.

The following changes since commit f883cf240266ee7be2cbd8971a8164cf4df9e372:

  cooker: Improve DATE/TIME handling (2015-07-08 00:06:01 +0100)

are available in the git repository at:

  git://git.openembedded.org/bitbake noupdatedata
  http://cgit.openembedded.org/cgit.cgi/bitbake/log/?h=noupdatedata

Richard Purdie (16):
  data_smart: Defer append/prepend handling
  data_smart: Remove need for update_data calls
  data_smart: Improve override handling
  data_smart: Fix cache clearance problems
  data_smart: Cache overrides and fix data store
  data_smart: Move override handling to getVar
  parse/ast/data_smart: Add parsing flag to getVar/setVar
  data_smart: VariableHistory: Ignore override duplicates and overlap
    with CoW functions
  data_smart: Seperate out the override cache
  data_smart: Tweak OVERRIDES value cache for performance
  data_smart: Improve override recursion handling
  data_smart: Cache set(self.overrides)
  data_smart: Fix keys() missing overrides
  data_smart: Fix data expansion cache issues
  data_smart: Fix appendVar/prependVar
  tests/data: Add new data tests

 lib/bb/data_smart.py  | 337 ++++++++++++++++++++++++++++++--------------------
 lib/bb/parse/ast.py   |  14 +--
 lib/bb/tests/data.py  |  47 +++++++
 lib/bb/tests/parse.py |  46 +++++++
 4 files changed, 306 insertions(+), 138 deletions(-)

-- 
2.1.0