[bitbake-devel] [PATCH] build/utils: Add BB_TASK_IONICE_LEVEL support

[Scott Rifenbark]

Hi,

See
http://www.yoctoproject.org/docs/2.0/bitbake-user-manual/bitbake-user-manual.html#var-BB_TASK_IONICE_LEVEL
for the new variable description in the BitBake User Manual.  Note that I
did not include the future implementation information at the bottom of
Richard's explanation.  Also, there is a bit about the 'cfq scheduler' near
the end.  I did not know how to properly format this string (e.g. CFG
Scheduler).

Let me know of any changes or fixes.

Thanks,
Scott

On Sat, Oct 24, 2015 at 3:19 AM, Richard Purdie <
richard.purdie at linuxfoundation.org> wrote:

> Similarly to BB_TASK_NICE_LEVEL, add BB_TASK_IONICE_LEVEL which allows the
> ioprio
> of tasks to be adjusted. This is in response to various qemu runtime
> timeouts
> which have been witnessed on the autobuilder, seemingly due to IO
> starvation (we
> already use NICE_LEVEL to adjust tasks). This has a fairly urgent need to
> deal
> with certain 'random' failures we're seeing on the autobuilders in testing.
>
> The format of the data in the variable is BB_TASK_IONICE_LEVEL =
> "<class>.<prio>".
>
> For <class>, 2 is best effort (the default), 1 is real time and 3 is idle.
> You'd
> need superuser privileges to use realtime. The <prio> value is a default
> of 4,
> and can be set between 0 and 7 with 7 being lowest priority and 0 the
> highest.
> The user can set this freely with normal privileges
>
> Note that in order for this to take effect, you need the cfq scheduler
> selected
> for the backing block device.
>
> We could use nice wrapper functions for ioprio from modules like psutil
> however
> that would complicate bitbake dependencies. This version has some magic
> numbers
> but works on the main 32 and 64 bit x86 build architectures and can easily
> be
> extended if ever needed. When we move to python 3.x, we can likely replace
> this
> with standard calls.
>
> Signed-off-by: Richard Purdie <richard.purdie at linuxfoundation.org>
>
> diff --git a/bitbake/lib/bb/build.py b/bitbake/lib/bb/build.py
> index 948c395..22428a6 100644
> --- a/bitbake/lib/bb/build.py
> +++ b/bitbake/lib/bb/build.py
> @@ -413,6 +413,13 @@ def _exec_task(fn, task, d, quieterr):
>          nice = int(nice) - curnice
>          newnice = os.nice(nice)
>          logger.debug(1, "Renice to %s " % newnice)
> +    ionice = localdata.getVar("BB_TASK_IONICE_LEVEL", True)
> +    if ionice:
> +        try:
> +            cls, prio = ionice.split(".", 1)
> +            bb.utils.ioprio_set(os.getpid(), int(cls), int(prio))
> +        except:
> +            bb.warn("Invalid ionice level %s" % ionice)
>
>      bb.utils.mkdirhier(tempdir)
>
> diff --git a/bitbake/lib/bb/utils.py b/bitbake/lib/bb/utils.py
> index eb219c1..86a5328 100644
> --- a/bitbake/lib/bb/utils.py
> +++ b/bitbake/lib/bb/utils.py
> @@ -1310,3 +1310,27 @@ def signal_on_parent_exit(signame):
>      result = cdll['libc.so.6'].prctl(PR_SET_PDEATHSIG, signum)
>      if result != 0:
>          raise PrCtlError('prctl failed with error code %s' % result)
> +
> +#
> +# Manually call the ioprio syscall. We could depend on other libs like
> psutil
> +# however this gets us enough of what we need to bitbake for now without
> the
> +# dependency
> +#
> +_unamearch = os.uname()[4]
> +IOPRIO_WHO_PROCESS = 1
> +IOPRIO_CLASS_SHIFT = 13
> +
> +def ioprio_set(who, cls, value):
> +    NR_ioprio_set = None
> +    if _unamearch == "x86_64":
> +      NR_ioprio_set = 251
> +    elif _unamearch[0] == "i" and _unamearch[2:3] == "86":
> +      NR_ioprio_set = 289
> +
> +    if NR_ioprio_set:
> +        ioprio = value | (cls << IOPRIO_CLASS_SHIFT)
> +        rc = cdll['libc.so.6'].syscall(NR_ioprio_set, IOPRIO_WHO_PROCESS,
> who, ioprio)
> +        if rc != 0:
> +            raise ValueError("Unable to set ioprio, syscall returned %s"
> % rc)
> +    else:
> +        bb.warn("Unable to set IO Prio for arch %s" % _unamearch)
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openembedded.org/pipermail/bitbake-devel/attachments/20151027/5fef6ebb/attachment-0002.html>