[OE-core] sharing DL_DIR?

[Rich Pixley]

Is it reasonable to expect to share DL_DIR between multiple builds?

That is, are downloads properly locked so that multiple concurrent 
downloads of the same file won't collide?  And if so, are they NFS safe 
locks?

Or must each build download it's own copies of every component?

I ask because our, (Palm), branch has been, but it's a bit of a nuisance 
to pick a suitable locking mechanism that is both functional and performant.

  * Flock/lockf aren't reliably supported in heterogeneous environments.
  * The old lockfiles don't work over NFS. Lock directories apparently
    do, but trying to clean these up after interruptions or failures is
    a losing battle so if we do use these, we end up with orphan locks
    periodically.
  * With NFS as a possibility, we can't assume any kernel local IPC
    mechanism, so sysV ipc is out.
  * It seems like a pretty huge overhead to try to create any sort of
    zeroconf overhead.

Our solution so far has been to use NFS lock directories for multiple 
machine builds and flock/lockf for single machine builds.  This mostly 
works, though it's not ideal, and it's significantly faster than 
downloading/copying all of the component source multiple times even over 
local mirrors.

--rich
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openembedded.org/pipermail/openembedded-core/attachments/20120222/78923089/attachment-0002.html>