[OE-core] package: Add cachedpath optimisation

[Chris Larson]

On Thu, Mar 14, 2013 at 6:57 PM, Richard Purdie <
richard.purdie at linuxfoundation.org> wrote:

> Currently, various standard library operations like os.walk(),
> os.path.isdir() and os.path.islink() each call stat or lstat which
> involves a syscall into the kernel. There is no caching since they could
> conceivably have changed on disk. The result is that for something like
> the do_package task of the kernel we're spending over two minutes making
> 868,000 individual stat calls for 23,000 files. This is suboptimal.
>
> This patch adds lib/oe/cachedpath.py which are a set of replacement
> functions for these operations which use cached stat data rather than
> hitting the kernel each time. It gives a nice performance improvement
> halving the build time of the kernel do_package.
>
> Signed-off-by: Richard Purdie <richard.purdie at linuxfoundation.org>
>

Have you considered having the initcache() return a cache object, either
with the rest as methods on it, or pass that in? I think it'd be
substantially cleaner than messing with globals, particularly given with
globals we tend to have issues with cache lifetime and invalidation — if
the cache was returned, it would go away when the object gets collected.
-- 
Christopher Larson
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openembedded.org/pipermail/openembedded-core/attachments/20130314/9c385c42/attachment-0002.html>