<html>
<head>
<meta content="text/html; charset=UTF-8" http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
What happens if two separate bitbake invocations both simultaneously
attempt to download the same file?<br>
<br>
--rich<br>
<br>
On 2/22/12 15:41 , Christopher Larson wrote:
<blockquote cite="mid:E9FAD6E1E68141CCA2CF166F1762B320@gmail.com"
type="cite">
<div style="font-family: Helvetica; font-size: 13px; ">I share it
as well. I don't see locks being a big issue for most things, as
the worst case is a re-download, not use of a corrupt or
incomplete file, due to the completion stamps.<br>
</div>
<div>
<div><br>
</div>
<div>-- </div>
<div>Christopher Larson</div>
<div><br>
</div>
</div>
<p style="color: #A0A0A8;">On Wednesday, February 22, 2012 at 4:39
PM, Tom King wrote:</p>
<blockquote type="cite"
style="border-left-style:solid;border-width:1px;margin-left:0px;padding-left:10px;">
<span>
<div>
<div>I have a single downloads directory with symlinks for
every target build I have to that directory. As long as I
have the source already in there I don't fetch it again
<div><br>
</div>
<div>Tom<br>
<br>
<div>
On Wed, Feb 22, 2012 at 1:33 PM, Rich Pixley <span
dir="ltr"><<a moz-do-not-send="true"
href="mailto:rich.pixley@palm.com">rich.pixley@palm.com</a>></span>
wrote:<br>
<blockquote type="cite">
<div>
<div bgcolor="#FFFFFF" text="#000000"> Is it
reasonable to expect to share DL_DIR between
multiple builds?<br>
<br>
That is, are downloads properly locked so that
multiple concurrent downloads of the same file
won't collide? And if so, are they NFS safe
locks?<br>
<br>
Or must each build download it's own copies of
every component?<br>
<br>
I ask because our, (Palm), branch has been, but
it's a bit of a nuisance to pick a suitable
locking mechanism that is both functional and
performant.<br>
<ul>
<li>Flock/lockf aren't reliably supported in
heterogeneous environments.</li>
<li>The old lockfiles don't work over NFS.
Lock directories apparently do, but trying
to clean these up after interruptions or
failures is a losing battle so if we do use
these, we end up with orphan locks
periodically.</li>
<li>With NFS as a possibility, we can't assume
any kernel local IPC mechanism, so sysV ipc
is out.</li>
<li>It seems like a pretty huge overhead to
try to create any sort of zeroconf overhead.</li>
</ul>
<p>Our solution so far has been to use NFS lock
directories for multiple machine builds and
flock/lockf for single machine builds. This
mostly works, though it's not ideal, and it's
significantly faster than downloading/copying
all of the component source multiple times
even over local mirrors.<span><font
color="#888888"><br>
</font></span></p>
<span><font color="#888888"> --rich<br>
</font></span></div>
<br>
_______________________________________________<br>
Openembedded-core mailing list<br>
<a moz-do-not-send="true"
href="mailto:Openembedded-core@lists.openembedded.org">Openembedded-core@lists.openembedded.org</a><br>
<a moz-do-not-send="true"
href="http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core"
target="_blank">http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core</a><br>
<br>
</div>
</blockquote>
</div>
<br>
</div>
</div>
<div>
<div>_______________________________________________</div>
<div>Openembedded-core mailing list</div>
<div><a moz-do-not-send="true"
href="mailto:Openembedded-core@lists.openembedded.org">Openembedded-core@lists.openembedded.org</a></div>
<div><a moz-do-not-send="true"
href="http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core">http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core</a></div>
</div>
</div>
</span> </blockquote>
<div> <br>
</div>
</blockquote>
</body>
</html>