<div dir="ltr"><div dir="ltr"><div>Hi, Richard:</div><div><br></div><div>Involve in Mark.</div><div><br></div><div>Please correct me if I am wrong, but according to the code, seems these upper level locks could not protect this race condition, because it's inside the GitSM unpack function, the call chain is:</div><div><br></div><div>Inside GitSM:</div><div>```</div><div>def unpack</div><div> -> for module in submodules:</div><div> -> unpack_submodules </div><div> -> runfetchcmd</div><div> -> bb.process.run</div><div> -> git config submodule</div><div>```</div><div><br></div><div>since 'git config submodule' run concurrently and try to modify a same git config file, hence the race might happen, but it's hard to be reproduced since 'git config submodule' finishes quite quickly.</div><div><br></div><div>//Ming Liu</div></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">Richard Purdie <<a href="mailto:richard.purdie@linuxfoundation.org">richard.purdie@linuxfoundation.org</a>> 於 2019年1月25日 週五 下午1:24寫道：<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">On Fri, 2019-01-25 at 12:35 +0100, Ming Liu wrote:<br>
> Hi, Stefan:<br>
> <br>
> Could you help confirm this? <br>
> <br>
> Hi, Richard:<br>
> <br>
> May I know how has the issue been fixed?<br>
><br>
> I updated the tip to:<br>
> ```<br>
> commit 8c8ecec2a722bc2885e2648d41ac8df07bdf660d<br>
> gitsmy.py: Fix unpack of submodules of submodules<br>
> <br>
> If the submodule is in a subdirectory, it needs to have that<br>
> structure<br>
> preserved. This means the unpack path needs to be in the<br>
> 'dirname' of the<br>
> final path -- since the unpack directory name is specified in the<br>
> URI.<br>
> <br>
> Additional specific test cases were added to ensure this is<br>
> working properly<br>
> based on two recent error reports.<br>
> <br>
> Signed-off-by: Mark Hatle <<a href="mailto:mark.hatle@windriver.com" target="_blank">mark.hatle@windriver.com</a>><br>
> Signed-off-by: Richard Purdie <<a href="mailto:richard.purdie@linuxfoundation.org" target="_blank">richard.purdie@linuxfoundation.org</a><br>
> ><br>
> ```<br>
> on top of that commit, I could see there are "git config submodule"<br>
> being called concurrently in different processes, which does have a<br>
> race condition, and will lead to the error observed by Stefan,<br>
> although it's not easy to be reproduced.<br>
<br>
There are locks at a higher level in the fetchers, in<br>
lib/bb/fetch2/__init__.py the download() method has:<br>
<br>
if ud.lockfile:<br>
lf = bb.utils.lockfile(ud.lockfile)<br>
<br>
and the unpack() method has similar locks.<br>
<br>
Mark and I believe there were some code paths where the locks were<br>
being bypassed in the gitsm fetcher but Mark reworked the code so that<br>
we were both happy that there should always be a lock held.<br>
<br>
We need to fix any races properly ensuring they're covered by the main<br>
fetcher locks, adding new ones would only fix a symptom of the real<br>
problem.<br>
<br>
Cheers,<br>
<br>
Richard<br>
<br>
<br>
<br>
</blockquote></div>