[bitbake-devel] [PATCH v2 1/2] gitsm.py: use download cache for git submodules too

[Mikko Rapeli]

Currently download caches for recipes using gitsm are filled also with
the submodule repositories but when the git submodule tree is reused from
cache, only the base repository is cloned. After a source URL rewrite,
'git submodule update --init --recursive' is called in the tree
which downloads the needed submodules from upstream URL's.
This wastes bandwidth since download cache already had the needed
commits for the submodule repositories.

With this change, also the submodule repositories are copied to the
work space from download cache and the download from upstream can
be avoided. Using plain 'cp -a' since git cloning doesn't work for the
submodule directories.

Note that using BB_NO_NETWORK does not stop the submodule downloads
from 'git submodule update --init --recursive'. Thus tested by
breaking access to remote repository via /etc/hosts line like:

127.0.0.1       localhost github.com

Signed-off-by: Mikko Rapeli <mikko.rapeli at bmw.de>
---
 bitbake/lib/bb/fetch2/gitsm.py | 8 ++++++++
 1 file changed, 8 insertions(+)

v2: send to bitbake-devel

v1: was sent to oe-core

diff --git a/bitbake/lib/bb/fetch2/gitsm.py b/bitbake/lib/bb/fetch2/gitsm.py
index 0aff100..20a9c40 100644
--- a/bitbake/lib/bb/fetch2/gitsm.py
+++ b/bitbake/lib/bb/fetch2/gitsm.py
@@ -131,5 +131,13 @@ class GitSM(Git):
         Git.unpack(self, ud, destdir, d)
 
         if self.uses_submodules(ud, d, ud.destdir):
+            # Copy also submodule trees from download cache instead of
+            # downloading again from the upstream repository.
+            # For some reason git does not clone them.
+            clone_modules = os.path.join(ud.clonedir, "modules")
+            if os.path.exists(clone_modules):
+                dest_modules = os.path.join(ud.destdir, ".git")
+                runfetchcmd("cp -a " + clone_modules + " " + dest_modules, d)
+
             runfetchcmd(ud.basecmd + " checkout " + ud.revisions[ud.names[0]], d, workdir=ud.destdir)
             runfetchcmd(ud.basecmd + " submodule update --init --recursive", d, workdir=ud.destdir)
-- 
1.9.1