[bitbake-devel] gitsm shared DL_DIR and race conditions

[Mark Hatle]

On 1/9/19 7:48 AM, Stefan Agner wrote:
> Hi,
> 
> We came across race conditions while fetching a repository using gitsm:
> ERROR: aktualizr-native-1.0+gitAUTOINC+d00d1a04cc-7 do_fetch: Fetcher
> failure: Fetch command export PSEUDO_DISABLED=1; export
> PATH="/workdir/oe/layers/openembedded-core/scripts/native-intercept:/workdir/oe/layers/openembedded-core/scripts:/workdir/oe/tmp/work/x86_64-linux/aktualizr-native/1.0+gitAUTOINC+d00d1a04cc-7/recipe-sysroot-native/usr/bin/x86_64-linux:/workdir/oe/tmp/work/x86_64-linux/aktualizr-native/1.0+gitAUTOINC+d00d1a04cc-7/recipe-sysroot-native/usr/bin:/workdir/oe/tmp/work/x86_64-linux/aktualizr-native/1.0+gitAUTOINC+d00d1a04cc-7/recipe-sysroot-native/usr/sbin:/workdir/oe/tmp/work/x86_64-linux/aktualizr-native/1.0+gitAUTOINC+d00d1a04cc-7/recipe-sysroot-native/usr/bin:/workdir/oe/tmp/work/x86_64-linux/aktualizr-native/1.0+gitAUTOINC+d00d1a04cc-7/recipe-sysroot-native/sbin:/workdir/oe/tmp/work/x86_64-linux/aktualizr-native/1.0+gitAUTOINC+d00d1a04cc-7/recipe-sysroot-native/bin:/workdir/oe/bitbake/bin:/workdir/oe/tmp/hosttools";
> export HOME="/home/yocto"; git -c core.fsyncobjectfiles=0 config
> submodule.tests/tuf-test-vectors.url
> /workdir/downloads/git2/github.com.advancedtelematic.tuf-test-vectors.
> failed with exit code 255, output:
> error: could not lock config file config: File exists
> 
> We share the same DL_DIR located on a NFS across multiple builders. We
> are using latest state of the 1.40 branch  (thud) of bitbake.
> 
> It seems that two git config invocations raced in this case. Is there
> locking required in the current gitsm implementation?

Did not expect git config to have locking issue... but it is showing that two
fetches appear to have happened together... gitsm uses the regular git fetcher
for that bit, and then after the fetch is complete updates the config (like git
submodule init would) to point to downloaded components.  It's this
configuration step that appears to have failed.

It should be fairly trivial to catch this issue and retry.

--Mark

> --
> Stefan
>