[bitbake-devel] [PATCH] parse/cache/cooker: Preserve order in the file inclusion list

[Martin Jansa]

On Mon, Nov 19, 2012 at 03:01:20PM +0000, Richard Purdie wrote:
> The data returned by get_file_depends() may me used in contexts like
> checksums where order is important. The current usage of sets means
> that some of the checksums can change in circumstances they should not.
> 
> This patch changes to use lists, thereby removing the problem.

Should this cover also BBINCLUDED variable in parsing cache?

Sometimes I see all recipes reparsed without any reason (no change in
metadata).

Comparing output of bitbake -p (with added patch to dump parse data,
shows order of BBINCLUDED changing.

e.g. from yesterday:
OE om-gta02 at shr ~/shr-core $ diff 20121118.BBINCLUDED 20121119.BBINCLUDED
12c12
< /OE/shr-core/openembedded-core/meta/conf/machine/include/tune-arm920t.inc
---
> /OE/shr-core/openembedded-core/meta/classes/mirrors.bbclass
45a46
> /OE/shr-core/meta-smartphone/meta-shr/conf/distro/shr.conf
51c52
< /OE/shr-core/openembedded-core/meta/classes/mirrors.bbclass
---
> /OE/shr-core/openembedded-core/meta/conf/machine/include/tune-arm920t.inc
54d54
< /OE/shr-core/meta-smartphone/meta-shr/conf/distro/shr.conf
60d59
< /OE/shr-core/openembedded-core/meta/conf/distro/include/tcmode-default.inc
62a62
> /OE/shr-core/openembedded-core/meta/conf/distro/include/tcmode-default.inc

Cheers,

> 
> Signed-off-by: Richard Purdie <richard.purdie at linuxfoundation.org>
> ---
> diff --git a/bitbake/lib/bb/cache.py b/bitbake/lib/bb/cache.py
> index 619b9ee..a3c073a 100644
> --- a/bitbake/lib/bb/cache.py
> +++ b/bitbake/lib/bb/cache.py
> @@ -405,12 +405,12 @@ class Cache(object):
>          """Parse the specified filename, returning the recipe information"""
>          infos = []
>          datastores = cls.load_bbfile(filename, appends, configdata)
> -        depends = set()
> +        depends = []
>          for variant, data in sorted(datastores.iteritems(),
>                                      key=lambda i: i[0],
>                                      reverse=True):
>              virtualfn = cls.realfn2virtual(filename, variant)
> -            depends |= (data.getVar("__depends", False) or set())
> +            depends = depends + (data.getVar("__depends", False) or [])
>              if depends and not variant:
>                  data.setVar("__depends", depends)
>  
> diff --git a/bitbake/lib/bb/cooker.py b/bitbake/lib/bb/cooker.py
> index e299059..98d6957 100644
> --- a/bitbake/lib/bb/cooker.py
> +++ b/bitbake/lib/bb/cooker.py
> @@ -692,8 +692,8 @@ class BBCooker:
>          # Generate a list of parsed configuration files by searching the files
>          # listed in the __depends and __base_depends variables with a .conf suffix.
>          conffiles = []
> -        dep_files = self.configuration.data.getVar('__depends') or set()
> -        dep_files.union(self.configuration.data.getVar('__base_depends') or set())
> +        dep_files = self.configuration.data.getVar('__base_depends') or []
> +        dep_files = dep_files + (self.configuration.data.getVar('__depends') or [])
>  
>          for f in dep_files:
>              if f[0].endswith(".conf"):
> diff --git a/bitbake/lib/bb/parse/__init__.py b/bitbake/lib/bb/parse/__init__.py
> index 7b9c47e..4293d09c 100644
> --- a/bitbake/lib/bb/parse/__init__.py
> +++ b/bitbake/lib/bb/parse/__init__.py
> @@ -73,8 +73,7 @@ def update_mtime(f):
>  def mark_dependency(d, f):
>      if f.startswith('./'):
>          f = "%s/%s" % (os.getcwd(), f[2:])
> -    deps = d.getVar('__depends') or set()
> -    deps.update([(f, cached_mtime(f))])
> +    deps = (d.getVar('__depends') or []) + [(f, cached_mtime(f))]
>      d.setVar('__depends', deps)
>  
>  def supports(fn, data):
> @@ -134,8 +133,8 @@ def vars_from_file(mypkg, d):
>  def get_file_depends(d):
>      '''Return the dependent files'''
>      dep_files = []
> -    depends = d.getVar('__depends', True) or set()
> -    depends = depends.union(d.getVar('__base_depends', True) or set())
> +    depends = d.getVar('__base_depends', True) or []
> +    depends = depends + (d.getVar('__depends', True) or [])
>      for (fn, _) in depends:
>          dep_files.append(os.path.abspath(fn))
>      return " ".join(dep_files)
> 
> 
> 
> _______________________________________________
> bitbake-devel mailing list
> bitbake-devel at lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/bitbake-devel

-- 
Martin 'JaMa' Jansa     jabber: Martin.Jansa at gmail.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature
URL: <http://lists.openembedded.org/pipermail/bitbake-devel/attachments/20121119/3e5276b8/attachment-0001.sig>