[bitbake-devel] [PATCH] cooker: bitbake memres server should not always reload the cache

[Alex Damian]

Please hold merging this change.

I think we need to discuss a bit on this. My expectation is that the
Bitbake server will reinit its state between builds, re-parsing the
configuration files and reload the cache.
I am not sure why we would consider maintaining the hot cache copy in
memory between builds.

The patch for bug
https://bugzilla.yoctoproject.org/show_bug.cgi?id=5535will re-create
the exact same scenario that we try to change through this
patch, so
I think we need to clarify the issue before moving forward.

Thanks,
Alex


On Wed, Mar 26, 2014 at 8:05 AM, Cristiana Voicu
<cristiana.voicu at intel.com>wrote:

> Due to commit dd15648fc2654b8d7c3e00ea7ab3dbf04f24f24b, after an async
> command is executed, the state of the server is set back to state.initial,
> so the cache is always reloaded.
> This patch adds a sanity check on the validity of the cache and then only
> reload if something has changed.
> depends_cache variable is needed in this case, so the del statement needs
> to be removed.
>
> [YOCTO #5603]
> Signed-off-by: Cristiana Voicu <cristiana.voicu at intel.com>
> ---
>  bitbake/lib/bb/cache.py  |    2 --
>  bitbake/lib/bb/cooker.py |   32 ++++++++++++++++++++++----------
>  2 files changed, 22 insertions(+), 12 deletions(-)
>
> diff --git a/bitbake/lib/bb/cache.py b/bitbake/lib/bb/cache.py
> index 318781b..da9ad5d 100644
> --- a/bitbake/lib/bb/cache.py
> +++ b/bitbake/lib/bb/cache.py
> @@ -612,8 +612,6 @@ class Cache(object):
>                      cache_class_name = cache_class.__name__
>                      file_dict[cache_class_name].close()
>
> -        del self.depends_cache
> -
>      @staticmethod
>      def mtime(cachefile):
>          return bb.parse.cached_mtime_noerror(cachefile)
> diff --git a/bitbake/lib/bb/cooker.py b/bitbake/lib/bb/cooker.py
> index 07202e3..c90bb44 100644
> --- a/bitbake/lib/bb/cooker.py
> +++ b/bitbake/lib/bb/cooker.py
> @@ -1265,21 +1265,33 @@ class BBCooker:
>              raise bb.BBHandledException()
>
>          if self.state != state.parsing:
> -            self.parseConfiguration ()
> +            check_cache = False
> +            if self.parser:
> +                self.parser.bb_cache.checked = set()
> +                for filename in self.parser.filelist:
> +                    appends = self.collection.get_file_appends(filename)
> +                    if not self.parser.bb_cache.cacheValid(filename,
> appends):
> +                        check_cache = True
> +                        break
> +            else:
> +                check_cache = True
>
> -            ignore = self.data.getVar("ASSUME_PROVIDED", True) or ""
> -            self.recipecache.ignored_dependencies = set(ignore.split())
> +            if check_cache:
> +                self.parseConfiguration ()
> +
> +                ignore = self.data.getVar("ASSUME_PROVIDED", True) or ""
> +                self.recipecache.ignored_dependencies =
> set(ignore.split())
>
> -            for dep in self.configuration.extra_assume_provided:
> -                self.recipecache.ignored_dependencies.add(dep)
> +                for dep in self.configuration.extra_assume_provided:
> +                    self.recipecache.ignored_dependencies.add(dep)
>
> -            self.collection =
> CookerCollectFiles(self.recipecache.bbfile_config_priorities)
> -            (filelist, masked) =
> self.collection.collect_bbfiles(self.data, self.event_data)
> +                self.collection =
> CookerCollectFiles(self.recipecache.bbfile_config_priorities)
> +                (filelist, masked) =
> self.collection.collect_bbfiles(self.data, self.event_data)
>
> -            self.data.renameVar("__depends", "__base_depends")
> +                self.data.renameVar("__depends", "__base_depends")
>
> -            self.parser = CookerParser(self, filelist, masked)
> -            self.state = state.parsing
> +                self.parser = CookerParser(self, filelist, masked)
> +                self.state = state.parsing
>
>          if not self.parser.parse_next():
>              collectlog.debug(1, "parsing complete")
> --
> 1.7.9.5
>
> --
> _______________________________________________
> bitbake-devel mailing list
> bitbake-devel at lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/bitbake-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openembedded.org/pipermail/bitbake-devel/attachments/20140328/f106e93f/attachment-0002.html>