[OE-core] File conflicts in DL_DIR

[Christopher Larson]

On Wed, Oct 12, 2016 at 10:01 AM, Phil Blundell <pb at pbcl.net> wrote:

> I just ran into an annoying problem with two recipes whose upstreams
> both use the same poor naming scheme for their downloadable tarballs
> which causes the sources to collide in DL_DIR.  One of them is eudev,
> which has:
>
> SRC_URI = "https://github.com/gentoo/${BPN}/archive/v${PV}.tar.gz
>
> This shows up as a source checksum error when the second package finds
> what it thinks are its sources in DL_DIR, but of course they are not
> the files that it wanted.
>
> I couldn't immediately find any easy way to work around this
> difficulty.  Setting "localpath=eudev_v{$PV}.tar.gz" in eudev's SRC_URI
> looks superficially like it ought to do the right thing, but although
> this changes the location that the fetcher will search for the
> downloaded file it doesn't actually cause wget to put the file there so
> this doesn't work either.
>

I think that’s just a bug.. The fetcher should fully obey the url
parameters we support, so we should fix the wget fetcher to properly obey
localpath.

I suppose I could define DL_DIR in terms of ${PN}, which would
> presumably fix this problem at the cost of causing all my previous
> downloads to be repeated.  Or I could stop using eudev, or wait for one
> of the two packages to release a new version. :-)  Any better
> suggestions?
>

One option would be to change DL_DIR, but use PREMIRRORS for compatibility
with existing DL_DIRs, i.e.:
DL_DIR_BASE = “${TOPDIR}/downloads”
DL_DIR = “${DL_DIR_BASE}/${BPN}”
PREMIRRORS_prepend = “file://.*/ file://${DL_DIR_BASE}/\n “

or similar (untested).

Of course, this conflict will occur on a mirror too, so that’s a concern if
you care about mirror population.

Long term, I’d like to see our DL_DIR as a more capable download cache.
Download every file to a unique name based on its checksum, then link those
into a ${BPN}-separated area with their real names. Then add a separate
tool to populate a mirror from DL_DIR.
-- 
Christopher Larson
clarson at kergoth dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Maintainer - Tslib
Senior Software Engineer, Mentor Graphics
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openembedded.org/pipermail/openembedded-core/attachments/20161012/ac83535d/attachment-0002.html>