[bitbake-devel] request for comments - dynamic task selection

[Damian, Alexandru]

Hello,

I have a RFC patch that attempts to improve build performance by
dynamically selecting the next-to-run task based on  currently running
tasks. The general idea is if we overload the network with too many fetch
tasks, it's better to start doing unpack tasks while the disks idle instead
of starting new fetch tasks. The concept can be generally applied, e.g.
it's better to schedule package tasks in parallel with compile tasks
instead of running all the compile tasks first and then all package tasks.

This patch just looks at currently running tasks and selects the next
available task that hasn't a similar-type task already running.

I'm seeing a roughly 2% build time reduction when building the
virtual:native components, with just 26 tasks reordered out of 244 tasks
executed. I am attaching a toaster.sqlite file that captures the test -
build no.1 is done with the origin/master source, and build no. 2 is done
with this patch applied.

DO NOT MERGE - this patch exposes dependency problems in OE-Core,
specifically GCC 4.8 recipes. It breaks the OE-Core builds.

​I'm waiting for your feedback. Meanwhile, I'm gonna try to solve the GCC
problems exposed.​


-- 
Alex Damian
Yocto Project
SSG / OTC
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openembedded.org/pipermail/bitbake-devel/attachments/20140220/eefbb340/attachment-0002.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0001-bitbake-runqueue-improve-task-selection-algorithm.patch
Type: text/x-patch
Size: 3158 bytes
Desc: not available
URL: <http://lists.openembedded.org/pipermail/bitbake-devel/attachments/20140220/eefbb340/attachment-0002.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: toaster.sqlite.xz
Type: application/x-xz
Size: 178344 bytes
Desc: not available
URL: <http://lists.openembedded.org/pipermail/bitbake-devel/attachments/20140220/eefbb340/attachment-0002.xz>