[OE-core] Fetch time optimization (svn : gcc/eglibc - git : linux-yocto)

[Richard Purdie]

On Fri, 2012-03-30 at 12:07 +0200, Samuel Stirtzel wrote:
> Of course this will only reduce the time of recipes if they are build
> for the first time,
> or when the version/URL changes.
> It is not that important, I agree,
> but it would improve the situation for first time users, or new installations.
> 
> Example for 2 threads:
> http://pastebin.com/kviwQZJ3
> It is very likely that the current situation also uses cpu and network
> resources at the same time,
> but it might occur that the build-task has to wait for a download to
> finish or vice versa.
> 
> Ignoring fetch tasks from the thread count would only do half of the
> job and _could_ cause network bottlenecks ;)
> Fetching should be "independent" from the dependency chain.

This simply isn't true and there is also no benefit to splitting them to
be independent. The fetch tasks have dependencies just like any other
task (for example git:// urls depend on git-native being built unless
its in ASSUME_PROVIDED).

> E.g.: it should not wait with the downloads for dependencies to finish building,
> the download sequence should still match the dependency chain sequence.

I'm afraid I don't understand what you mean. I think you will find that
if you exclude the "fetch" tasks from the normal "cpu" thread count you
will get the behaviour you are describing.

Cheers,

Richard