[oe-commits] [openembedded-core] 08/21: sstate.bbclass: Only remove sstate file when task is existed

[git at git.openembedded.org]

This is an automated email from the git hooks/post-receive script.

rpurdie pushed a commit to branch master-next
in repository openembedded-core.

commit c3d881a52e784eb7fea1820411a44c63d802be82
Author: Robert Yang <liezhi.yang at windriver.com>
AuthorDate: Fri Nov 23 10:37:56 2018 +0800

    sstate.bbclass: Only remove sstate file when task is existed
    
    This can improve the performance a lot for "bitbake <recipe-native/cross/crosssdk>
    -ccleansstate" when there are a lot of sstate files.
    
    For example:
    * Before
      $ bitbake quilt-native -ccleansstate
      - Check log.do_cleansstate:
      Removing /sstate-cache/*/*/sstate:quilt-native:x86_64-linux:0.65:r0:x86_64:3:*_package.tgz*
      Removing /sstate-cache/*/*/sstate:quilt-native:x86_64-linux:0.65:r0:x86_64:3:*_package_qa.tgz*
      Removing /sstate-cache/*/*/sstate:quilt-native:x86_64-linux:0.65:r0:x86_64:3:*_package_write_rpm.tgz*
      Removing /sstate-cache/*/*/sstate:quilt-native:x86_64-linux:0.65:r0:x86_64:3:*_packagedata.tgz*
      Removing /sstate-cache/*/sstate:quilt-native::0.65:r0::3:*_populate_lic.tgz*
      Removing /sstate-cache/*/*/sstate:quilt-native:x86_64-linux:0.65:r0:x86_64:3:*_populate_sysroot.tgz*
    
      There are no package tasks for quilt-native, so the first 4 lines doesn't
      make any sense, but the glob pattern "sstate-cache/*/*" is very time
      consuming when there are no disk caches. E.g., I have more than 600,000
      sstate files:
      - Without disk caches
      # echo 3 >/proc/sys/vm/drop_caches
      $ time python3 -c 'import glob; glob.glob("/sstate-cache/*/*/sstate:quilt-native:x86_64-linux:0.65:r0:x86_64:3:*_package.tgz*")'
        real    4m32.583s
        user    0m5.768s
        sys     0m12.892s
    
      - With disk caches (e.g., run it in the second time)
      $ time python3 -c 'import glob; glob.glob("/sstate-cache/*/*/sstate:quilt-native:x86_64-linux:0.65:r0:x86_64:3:*_package.tgz*")'
        real    0m5.128s
        user    0m2.772s
        sys     0m2.308s
    
      So the 4 removing *package* commands cost more than 20s or 272s in theory.
    
    * After
      $ bitbake quilt-native -ccleansstate
      - Check log.do_cleansstate:
      Removing /sstate-cache/*/sstate:quilt-native::0.65:r0::3:*_populate_lic.tgz*
      Removing /sstate-cache/*/*/sstate:quilt-native:x86_64-linux:0.65:r0:x86_64:3:*_populate_sysroot.tgz*
    
      We can see that it saved 20s or 272s in theory.
    
    Signed-off-by: Robert Yang <liezhi.yang at windriver.com>
    Signed-off-by: Richard Purdie <richard.purdie at linuxfoundation.org>
---
 meta/classes/sstate.bbclass | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/meta/classes/sstate.bbclass b/meta/classes/sstate.bbclass
index 94fde6d..8b48ab4 100644
--- a/meta/classes/sstate.bbclass
+++ b/meta/classes/sstate.bbclass
@@ -423,8 +423,9 @@ def sstate_clean_cachefile(ss, d):
     import oe.path
 
     sstatepkgfile = d.getVar('SSTATE_PATHSPEC') + "*_" + ss['task'] + ".tgz*"
-    bb.note("Removing %s" % sstatepkgfile)
-    oe.path.remove(sstatepkgfile)
+    if d.getVarFlag('do_%s' % ss['task'], 'task'):
+        bb.note("Removing %s" % sstatepkgfile)
+        oe.path.remove(sstatepkgfile)
 
 def sstate_clean_cachefiles(d):
     for task in (d.getVar('SSTATETASKS') or "").split():

-- 
To stop receiving notification emails like this one, please contact
the administrator of this repository.