[OE-core] Possible stale tags in the download directory

[Richard Purdie]

On Wed, 2011-12-07 at 09:27 +0100, Ulf Samuelsson wrote:
> When I look at the "downloads" location I find a lot of files called.
> 
> "11_usagi_fix.patch.done"
> 
> but no  "11_usagi_fix.patch" file in the directory.
> 
> If this is a indication of that a patch has been downloaded,
> what happens if you for some reason or other delete your
> OE-core directory and keep the downloads directory.
> 
> When you start from fresh, all the tags are then stale.
> 
> (Deleting the "downloads" directory seems a bad idea)
> 
> I think that if there should be tags in the "downloads" directory,
> they should only reflect things which are downloaded to the
> "downloads" directory, and nothing else.
> 
> As for the tags in the directory, I think a better approach
> is to download a file "tarball.tar.bz2" to a different filename first
> I.E: "tarball.tar.bz2.in-progress"  and if the download completes
> then move the file to "tarball.tar.bz2".
> Should remove a lot of clutter from the "downloads" directory.

What we've tried to do is simplify the fetcher code paths in fetch2.
There were some many corner/special cases and different code paths it
was near impossible to tell what was going on. One of the side effects
is that local file:// urls do touch the done stamp in the same way as
other downloads. The main reason for those files is now checksum
tracking. If the done stamps were part of tmpdir, we'd keep checksumming
the contents of the downloads at each build rather than once after the
download. The way the implementation works, there is very little risk
from stale stamps, it would just mean the checksum code wouldn't get
triggered and that is likely unneeded for a local file:// url anyway.

So yes, its not ideal but looking at the code I'd be surprised if a
build ever broke due to it.

Cheers,

Richard