[oe-commits] Richard Purdie : package: Add cachedpath optimisation

[git at git.openembedded.org]

Module: openembedded-core.git
Branch: master
Commit: 556dee0c4d6d8a87c0cddbd2f60fe5917d009f18
URL:    http://git.openembedded.org/?p=openembedded-core.git&a=commit;h=556dee0c4d6d8a87c0cddbd2f60fe5917d009f18

Author: Richard Purdie <richard.purdie at linuxfoundation.org>
Date:   Thu Mar 14 17:26:20 2013 +0000

package: Add cachedpath optimisation

Currently, various standard library operations like os.walk(),
os.path.isdir() and os.path.islink() each call stat or lstat which
involves a syscall into the kernel. There is no caching since they could
conceivably have changed on disk. The result is that for something like
the do_package task of the kernel we're spending over two minutes making
868,000 individual stat calls for 23,000 files. This is suboptimal.

This patch adds lib/oe/cachedpath.py which are a set of replacement
functions for these operations which use cached stat data rather than
hitting the kernel each time. It gives a nice performance improvement
halving the build time of the kernel do_package.

Signed-off-by: Richard Purdie <richard.purdie at linuxfoundation.org>

---

 meta/classes/base.bbclass    |    2 +-
 meta/classes/package.bbclass |   56 ++++++-----
 meta/lib/oe/cachedpath.py    |  235 ++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 268 insertions(+), 25 deletions(-)

Diff:   http://git.openembedded.org/?p=openembedded-core.git/?a=commitdiff;h=556dee0c4d6d8a87c0cddbd2f60fe5917d009f18