[bitbake-devel] [PATCH] utils: Add workaround for multiprocessing bug
Richard Purdie
richard.purdie at linuxfoundation.org
Thu Aug 21 20:47:25 UTC 2014
Our usage of multitprocessing is problematic. In particular, there is a bug
in python 2.7 multiprocessing where signals are not handled until command
completion instead of immediately.
This adds a workaround into our wrapper function to deal with the issue.
Signed-off-by: Richard Purdie <richard.purdie at linuxfoundation.org>
diff --git a/bitbake/lib/bb/utils.py b/bitbake/lib/bb/utils.py
index 781af3a..11f046a 100644
--- a/bitbake/lib/bb/utils.py
+++ b/bitbake/lib/bb/utils.py
@@ -883,5 +883,17 @@ def process_profilelog(fn):
# Was present to work around multiprocessing pool bugs in python < 2.7.3
#
def multiprocessingpool(*args, **kwargs):
+
+ import multiprocessing.pool
+ #import multiprocessing.util
+ #multiprocessing.util.log_to_stderr(10)
+ # Deal with a multiprocessing bug where signals to the processes would be delayed until the work
+ # completes. Putting in a timeout means the signals (like SIGINT/SIGTERM) get processed.
+ def wrapper(func):
+ def wrap(self, timeout=None):
+ return func(self, timeout=timeout if timeout is not None else 1e100)
+ return wrap
+ multiprocessing.pool.IMapIterator.next = wrapper(multiprocessing.pool.IMapIterator.next)
+
return multiprocessing.Pool(*args, **kwargs)
More information about the bitbake-devel
mailing list