[OE-core] Fetch failure for source at kernel.org

[Koen Kooi]

Op 22 sep. 2011, om 10:45 heeft Martin Jansa het volgende geschreven:

> On Thu, Sep 22, 2011 at 10:28:02AM +0200, Koen Kooi wrote:
>> 
>> Op 22 sep. 2011, om 09:03 heeft Richard Purdie het volgende geschreven:
>> 
>>> On Thu, 2011-09-22 at 08:40 +0200, Koen Kooi wrote:
>>>> Op 22 sep. 2011, om 02:26 heeft Richard Purdie het volgende geschreven:
>>>> 
>>>>> On Wed, 2011-09-21 at 21:49 +0200, Koen Kooi wrote:
>>>>>> Op 21 sep. 2011, om 21:43 heeft Mark Hatle het volgende geschreven:
>>>>>> 
>>>>>>> On 9/21/11 2:38 PM, Philip Balister wrote:
>>>>>>>> I'm finally trying to get started with oe-core. I'm using a setup 
>>>>>>>> created with the setup-script created by Koen. Everything looks ok until 
>>>>>>>> it tried to build cpufrequtils. The recipe tries to fetch the source 
>>>>>>>> from a git repo at kernel.org. Obviously, the fetch fails.
>>>>>>>> 
>>>>>>>> What is odd, in the "good old days" I would expect this config to fall 
>>>>>>>> back and use the Angstrom source mirror, but apparently this is not the 
>>>>>>>> case with oe-core.
>>>>>>>> 
>>>>>>>> Is this a bug or a feature? If this is a bug, is there a plan to fix it? 
>>>>>>>> If it is a feature, how are we supposed to do builds if servers go away? 
>>>>>>>> What do we do about GPL compliance?
>>>>>>> 
>>>>>>> This is a feature, as people did not want oe-core tied to a given mirror.
>>>>>> 
>>>>>> This is the "OE core doesn't generate versioned git tarballs" problem
>>>>>> people keep talking about. I know I can edit each and every recipe to
>>>>>> add ';rebasable=true', but I'd like a global solution before the GPL
>>>>>> police starts looking at the angstrom source mirrors.
>>>>> 
>>>>> Is this a versioned tarball problem or just the tarballs not being
>>>>> generated at all?
>>>> 
>>>> versioned tarball problem, since the source mirror won't overwrite
>>>> files with the same name, so unversioned ones are both useless and
>>>> bandwidth wasting.
>>> 
>>> They seem to be working perfectly well for Yocto...
>>> 
>>> They're also not bandwidth wasting
>> 
>> They are, since for the kernel you need to re-upload the 300MB tarball to the source mirror for mirroring to work, even if there were no real changes but bitbake repacked it. I know that versioned tarballs aren't terribly efficient either, but at least they are deterministic.
>> 
>> The angstrom source mirror checks for duplicate filenames to avoid introducing checksum changes due to upstream stupidity or corrupted downloads. 
>> 
>>> since the fetcher is efficient and
>>> will incrementally update something that is incomplete.
>> 
>> Which won't work currently since kernel.org is down.
>> 
>>> Anyhow, this is the first time someone has explained that the mirror
>>> tarball improvements that were made to bitbake have not done everything
>>> they desire. If you want someone on the Yocto team to look at this I'd
>>> suggest a bug report or I'll take patches to bitbake (as always).
>> 
>> But you do acknowledge the problem? FWIW, I've been arguing for a global method of enabled versioned tarballs since I first started using oe-core (called differently back then). I know linux-yocto stretches the fetcher to its limits, but I'd like this change for all the recipes using a single git tree with a single srcrev.
> 
> FWIW: I would also like to have versioned git checkouts in fetcher2 as
> it was in old fetcher. It would be more consitent with svn versioned
> checkouts which are still created.

http://patches.openembedded.org/patch/11971/ adds a global flag to generate such tarballs, but I don't know if the mirroring logic will actually try to fetch those or if fetch2 will actually try to unpack them when present, but at least we can generate the tarballs now :)

regards,

Koen