[OE-core] Fetch time optimization (svn : gcc/eglibc - git : linux-yocto)

[Paul Eggleton]

On Friday 30 March 2012 08:44:56 Samuel Stirtzel wrote:
> this might be a bit off-topic, but another idea would be to add a
> separate threading mechanism for fetching.
> 
> Current threading can help to use the CPU and memory load to it's optimum,
> but sometimes you have to wait for a download to finish..
> Instead there could be a separate set of threads that only download
> the sources and make optimal use of the bandwidth too.
> 
> This would also allow to fetch files when the normal threads are busy
> with configuring/building/packaging recipes.

What you're really suggesting here is a modified BitBake scheduler that 
understands that fetch tasks that require network bandwidth are different from 
other tasks such as compile ones which stress the CPU. It sounds like it might 
be worth investigating at least. FYI, BitBake's schedulers are pluggable and 
not particularly complicated (see bitbake/lib/bb/runqueue.py).

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre