[oe-commits] [openembedded-core] 16/44: devtool: upgrade: improve performance and show progress when adding files

[git at git.openembedded.org]

This is an automated email from the git hooks/post-receive script.

rpurdie pushed a commit to branch master-next
in repository openembedded-core.

commit 8b184f6c874b60324ee107af53853687173d3434
Author: Paul Eggleton <paul.eggleton at linux.intel.com>
AuthorDate: Thu Sep 28 16:46:14 2017 +1300

    devtool: upgrade: improve performance and show progress when adding files
    
    When devtool upgrade is upgrading to a new version where the source is
    fetched as an archive (e.g. a tarball), we create a single commit in the
    git repository that is the upgrade from the old version to the new. We
    do this by extracting the old source, committing it, deleting all files,
    copying in the new files, running git add on each new/changed/deleted
    file, and then committing the result. When a lot of files have changed
    in an upgrade (such as QEMU 2.8.1.1 -> 2.10.0) the penultimate step of
    running git add it can take quite a long time; in order to reduce this
    and show some feedback to the user, run git add with batches of 100
    files at once and also show a progress bar. In a local test with the
    aforementioned QEMU upgrade it took the time down from over 7 minutes
    down to about 13 seconds.
    
    Fixes [YOCTO #11948].
    
    Signed-off-by: Paul Eggleton <paul.eggleton at linux.intel.com>
    Signed-off-by: Ross Burton <ross.burton at intel.com>
---
 scripts/lib/devtool/upgrade.py | 11 +++++++++--
 1 file changed, 9 insertions(+), 2 deletions(-)

diff --git a/scripts/lib/devtool/upgrade.py b/scripts/lib/devtool/upgrade.py
index 073002b..441dd35 100644
--- a/scripts/lib/devtool/upgrade.py
+++ b/scripts/lib/devtool/upgrade.py
@@ -251,8 +251,15 @@ def _extract_new_source(newpv, srctree, no_patch, srcrev, srcbranch, branch, kee
         _copy_source_code(tmpsrctree, srctree)
 
         (stdout,_) = __run('git ls-files --modified --others --exclude-standard')
-        for f in stdout.splitlines():
-            __run('git add -A "%s"' % f)
+        filelist = stdout.splitlines()
+        pbar = bb.ui.knotty.BBProgress('Adding changed files', len(filelist))
+        pbar.start()
+        batchsize = 100
+        for i in range(0, len(filelist), batchsize):
+            batch = filelist[i:i+batchsize]
+            __run('git add -A %s' % ' '.join(['"%s"' % item for item in batch]))
+            pbar.update(i)
+        pbar.finish()
 
         useroptions = []
         oe.patch.GitApplyTree.gitCommandUserOptions(useroptions, d=rd)

-- 
To stop receiving notification emails like this one, please contact
the administrator of this repository.