[OE-core] [PATCH V2 0/2] Improve performance for native/cross/crosssdk's cleansstate
Robert Yang
liezhi.yang at windriver.com
Fri Nov 23 02:37:55 UTC 2018
* V2
- Teach sstate.bbclass rather than edit nopackages.bbclass to fix the problem.
* V1
These 2 patches can improve performance a lot for native/cross/crosssdk's
cleansstate:
This test is based on more than 600,000 sstate files
* Without disk caches
# echo 3 >/proc/sys/vm/drop_caches
$ bitbake -p
$ time bitbake quilt-native -ccleansstate
- Before:
real 4m53.815s
user 0m0.820s
sys 0m0.128s
- After
real 0m58.483s
user 0m0.744s
sys 0m0.108s
Saved 293 - 58 = 235s (80% improvement)
* With disk caches (26.6s -> 5.8s, 78% improvement)
$ bitbake -p
$ time bitbake quilt-native -ccleansstate
- Before:
real 0m26.682s
user 0m0.748s
sys 0m0.088s
- After
real 0m5.846s
user 0m0.716s
sys 0m0.116s
Saved 26.6 - 5.8 = 20.8s (78% improvement)
======
I have another idea is put sstate files into the directory
named by the recipe, e.g.:
sstate-cache/quilt-native/sstate:quilt-native::0.65:r0::3:773341447192feb6c1519cace0e6be35_populate_lic.tgz
sstate-cache/quilt-native/sstate:quilt-native::0.65:r0::3:773341447192feb6c1519cace0e6be35_populate_lic.tgz.siginfo
sstate-cache/quilt-native/sstate:quilt-native::0.65:r0::3:ab4a82a07b10926bb42d2f37242413c7_patch.tgz.siginfo
sstate-cache/quilt-native/sstate:quilt-native::0.65:r0::3:e8a4c952a66942653e36f289eaf68ca5_fetch.tgz.siginfo
sstate-cache/quilt-native/sstate:quilt-native::0.65:r0::3:fd35240c437cefa66e24cfe65d3fb335_unpack.tgz.siginfo
This can make it easier to manage sstate files, and can reduce the load of
glob.glob(), currently, the remove command is something like:
$ rm -f glob.glob(sstate-cache/*/sstate:foo.*bar.*)
If we put them into recipe named directory, then we can run:
$ rm -f glob.glob(sstate-cache/<recipe>/sstate:foo.*bar.*)
Which should be faster, what's your opinion, please ?
The following changes since commit fc5418e7bbdecfb27bafe595084e0fd0f991a388:
meta/icecc.bbclass: Update system blacklists (2018-11-21 11:48:18 +0000)
are available in the git repository at:
git://git.openembedded.org/openembedded-core-contrib rbt/sstate
http://cgit.openembedded.org/openembedded-core-contrib/log/?h=rbt/sstate
Robert Yang (2):
sstate.bbclass: Only remove sstate file when task is existed
sstate.bbclass: set SSTATE_EXTRAPATHWILDCARD explicitly
meta/classes/sstate.bbclass | 7 ++++---
1 file changed, 4 insertions(+), 3 deletions(-)
--
2.7.4
More information about the Openembedded-core
mailing list