[OE-core] Fetch time optimization (svn : gcc/eglibc - git : linux-yocto)

[Samuel Stirtzel]

2012/3/30 Richard Purdie <richard.purdie at linuxfoundation.org>:
> On Fri, 2012-03-30 at 12:07 +0200, Samuel Stirtzel wrote:
>> Of course this will only reduce the time of recipes if they are build
>> for the first time,
>> or when the version/URL changes.
>> It is not that important, I agree,
>> but it would improve the situation for first time users, or new installations.
>>
>> Example for 2 threads:
>> http://pastebin.com/kviwQZJ3
>> It is very likely that the current situation also uses cpu and network
>> resources at the same time,
>> but it might occur that the build-task has to wait for a download to
>> finish or vice versa.
>>
>> Ignoring fetch tasks from the thread count would only do half of the
>> job and _could_ cause network bottlenecks ;)
>> Fetching should be "independent" from the dependency chain.
>
> This simply isn't true and there is also no benefit to splitting them to
> be independent. The fetch tasks have dependencies just like any other
> task (for example git:// urls depend on git-native being built unless
> its in ASSUME_PROVIDED).
You are right, my mistake.
Adding some line like PARALLEL_FETCH to the config will do the rest.

>
>> E.g.: it should not wait with the downloads for dependencies to finish building,
>> the download sequence should still match the dependency chain sequence.
>
> I'm afraid I don't understand what you mean. I think you will find that
> if you exclude the "fetch" tasks from the normal "cpu" thread count you
> will get the behaviour you are describing.

I was erroneously assuming that the download only starts after all
dependencies finished building,
but of course this was only derivated as the threads where blocked by
the build tasks.
So yes the method you mentioned will work.
>
> Cheers,
>
> Richard
>
>
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core at lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core



-- 
Regards
Samuel