[bitbake-devel] [PATCH] fetch2/gitsm: fix config file concurrent update race

[liu.ming50 at gmail.com]

From: Ming Liu <liu.ming50 at gmail.com>

A following issue was observed with gitsm:
| git -c core.fsyncobjectfiles=0 config submodule... failed with exit code 255, output:
| error: could not lock config file config: File exists

Since all git submodules share a config file and Git creates a lock
file (.git/config.lock) to synchronize access to it, but Git only tries
exactly once and returns a error if it's already locked. This leads to
a race condition if there are many 'git config submodule' run in
different processes.

It could be fixed by introducing a bitbake file lock to protect config
file from concurrent update from submodules.

Reported-by: Stefan Agner <stefan.agner at toradex.com>
Signed-off-by: Ming Liu <liu.ming50 at gmail.com>
---
 lib/bb/fetch2/gitsm.py | 19 +++++++++++++++----
 1 file changed, 15 insertions(+), 4 deletions(-)

diff --git a/lib/bb/fetch2/gitsm.py b/lib/bb/fetch2/gitsm.py
index f45546b..bb49d5c 100644
--- a/lib/bb/fetch2/gitsm.py
+++ b/lib/bb/fetch2/gitsm.py
@@ -39,6 +39,13 @@ from   bb.fetch2 import Fetch
 from   bb.fetch2 import BBFetchException
 
 class GitSM(Git):
+    def urldata_init(self, ud, d):
+        super().urldata_init(ud, d)
+        # Protects the config file from concurrent updates by submodules
+        # e.g. 'git config submodule' run in different processes may lead
+        # to a race condition.
+        ud.gitsmlock = os.path.join(ud.clonedir, "gitsm-config.lock")
+
     def supports(self, ud, d):
         """
         Check to see if a given url can be fetched with git.
@@ -183,11 +190,15 @@ class GitSM(Git):
 
             local_path = newfetch.localpath(url)
 
-            # Correct the submodule references to the local download version...
-            runfetchcmd("%(basecmd)s config submodule.%(module)s.url %(url)s" % {'basecmd': ud.basecmd, 'module': module, 'url' : local_path}, d, workdir=ud.destdir)
+            lf = bb.utils.lockfile(ud.gitsmlock)
+            try:
+                # Correct the submodule references to the local download version...
+                runfetchcmd("%(basecmd)s config submodule.%(module)s.url %(url)s" % {'basecmd': ud.basecmd, 'module': module, 'url' : local_path}, d, workdir=ud.destdir)
 
-            if ud.shallow:
-                runfetchcmd("%(basecmd)s config submodule.%(module)s.shallow true" % {'basecmd': ud.basecmd, 'module': module}, d, workdir=ud.destdir)
+                if ud.shallow:
+                    runfetchcmd("%(basecmd)s config submodule.%(module)s.shallow true" % {'basecmd': ud.basecmd, 'module': module}, d, workdir=ud.destdir)
+            finally:
+                bb.utils.unlockfile(lf)
 
             # Ensure the submodule repository is NOT set to bare, since we're checking it out...
             try:
-- 
2.7.4