[oe] [PATCH] base.bbclass: introduce OE source mirror network as last resort for do_fetch

[Martyn Welch]

Chris Larson wrote:
> On Fri, Aug 14, 2009 at 7:29 AM, Rolf
> Leggewie<oe-devel at rolf.leggewie.biz> wrote:
>   
>> Updating recipes and checksums.ini for vanishing upstream hosts or simple
>> restructuring can quickly become a chore for OE devs if taken seriously.
>> The OE user experience suffers severely when at any given point in time 4
>> to 5 recipes on average fail do_fetch for a simple image.
>>
>> The solution is for OE to provide its own caching of upstream sources.
>> mirrors.openembedded.org is a swarm of possibly incomplete source mirrors
>> provided by the community and load-balanced via DNS round robin.  They
>> act as a first line of defense and lower bandwidth load on melo which
>> hosts sources.openembedded.org.  The melo mirror aims to have a complete
>> copy of free-to-publish sources used by the current org.oe.dev branch.
>>
>> Some distributions have provided source mirrors in the past for their
>> users. I think it's time for a general and all-encompassing solution.
>>
>> The infrastructure is being currently put in place.  This patch will
>> turn on the OE sources mirror network by default.
>>     
>
> Looks like a good idea to me.  404s cause a great deal of difficulty
> for the userbase
On that note we just ran a "quick" test on OE, basically:

find openembedded -name "*.bb" -exec bitbake -c fetch -b {} \;

The following files seem to be pulling "index.html" files instead of the 
intended packages, I assume because the file/project account has 
moved/vanished:

recipes/balsa/balsa_2.0.17.bb
recipes/zd1211/zd1211_r83.bb
recipes/zd1211/zd1211_r85.bb
recipes/zd1211/zd1211_r77.bb
recipes/mtools/mtools_3.9.11.bb
recipes/mtools/mtools-native_3.9.11.bb
recipes/qxmp/qxmp_1.0.bb
recipes/murasaki/murasaki_0.8.8.bb
recipes/uim/uim-native_1.3.1.bb
recipes/uim/uim_1.3.1.bb
recipes/tmsnc/tmsnc_0.3.2.bb
recipes/ttf-fonts/ttf-sazanami_20040629.bb
recipes/qpe-gaim/qpe-libgaim_1.5.0.bb

I haven't had a chance to get any further.

Martyn

-- 
Martyn Welch MEng MPhil MIET (Principal Software Engineer)   T:+44(0)1327322748
GE Fanuc Intelligent Platforms Ltd,        |Registered in England and Wales
Tove Valley Business Park, Towcester,      |(3828642) at 100 Barbirolli Square,
Northants, NN12 6PF, UK T:+44(0)1327359444 |Manchester,M2 3AB  VAT:GB 927559189