[bitbake-devel] [PATCH 2/2] lib/bb/process: check output of select() before reading extrafiles

[Paul Eggleton]

We're calling select() to find the fds that have data available, so we
really ought to check the return to see if the extra file is in there
before trying to read from it. This is part of the fix for the
performance regression that this code introduced (5-10 minutes extra in
a reasonable size OE build); the rest is down to an issue in the way
that pseudo currently handles FIFOs and will need to be addressed there,
see: https://bugzilla.yoctoproject.org/show_bug.cgi?id=7993

Solution suggested by Richard Purdie <richard.purdie at linuxfoundation.org>

Signed-off-by: Paul Eggleton <paul.eggleton at linux.intel.com>
---
 lib/bb/process.py | 21 +++++++++++----------
 1 file changed, 11 insertions(+), 10 deletions(-)

diff --git a/lib/bb/process.py b/lib/bb/process.py
index d95a03d..7c79785 100644
--- a/lib/bb/process.py
+++ b/lib/bb/process.py
@@ -83,15 +83,16 @@ def _logged_communicate(pipe, log, input, extrafiles):
         bb.utils.nonblockingfd(fobj.fileno())
         rin.append(fobj)
 
-    def readextras():
+    def readextras(selected):
         for fobj, func in extrafiles:
-            try:
-                data = fobj.read()
-            except IOError as err:
-                if err.errno == errno.EAGAIN or err.errno == errno.EWOULDBLOCK:
-                    data = None
-            if data is not None:
-                func(data)
+            if fobj in selected:
+                try:
+                    data = fobj.read()
+                except IOError as err:
+                    if err.errno == errno.EAGAIN or err.errno == errno.EWOULDBLOCK:
+                        data = None
+                if data is not None:
+                    func(data)
 
     try:
         while pipe.poll() is None:
@@ -114,12 +115,12 @@ def _logged_communicate(pipe, log, input, extrafiles):
                     errdata.append(data)
                     log.write(data)
 
-            readextras()
+            readextras(r)
 
     finally:    
         log.flush()
 
-    readextras()
+    readextras([fobj for fobj, _ in extrafiles])
 
     if pipe.stdout is not None:
         pipe.stdout.close()
-- 
2.1.0