[bitbake-devel] [PATCH 0/1] flush stdout/stderr to log file

[Juro Bystricky]

This patch fixes the bug https://bugzilla.yoctoproject.org/show_bug.cgi?id=10785
It was observed that some messages may not make it into the log file.
Alternatively, they may end up in the log file but in wrong order.
It is fairly simple to demostrate this, please see the bug description.

Simplified, the logging code (in proccess.py) looks like this:

def _logged_communicate(pipe, log, input, extrafiles):
   <...>
   while ProcessOpen
       log fifo(s)
       log stdout 
       log stderr

   # Process closed....
   log fifo(s)

The messages are written in a round-robin fashion and if he "Process" is closed somewhere in
the loop, some of the pipes may have still data in them. Also, the "Process" may be already closed
before the code reaches the "while" loop and hence the pipes do not get a chance to be logged/drained. 
Note after "Process" is closed we do make effort to drain/log fifos but not stdout/stderr.

So the fix is this:

   while ProcessOpen
       log fifo(s)
       log stdout 
       log stderr

   # Drain pipes after process closed
   log fifo()
   log stdout
   log stderr

Which can be refactored as:

    def log_all_pipes:
       log fifo(s)
       log stdout 
       log stderr

    while ProcessOpen
       log_all_pipes

   # Drain all pipes after process closed
   log_all_pipes

I extensively tested the patch with the following test cases:

do_test1() {
    bbdebug 1 "breakage"
    echo "this will not be printed"
    printf "nor this\n"
}

do_test2() {
    echo "this will not be printed"
    bbdebug 1 "breakage"
    printf "nor this\n"
}

do_test3() {
    echo "line1"
    bbdebug 1 "breakage1 line2"
    echo "this will not be printed (line3)"
    printf "nor this (line4)\n"
    bbdebug 1 "breakage2 line5"
}

do_test4() {
    bbdebug 1 "breakage1 line2"
    bbdebug 1 "breakage2 line2"
}

Juro Bystricky (1):
  bitbake:process: flush stderr/stdout to log

 bitbake/lib/bb/process.py | 81 +++++++++++++++++++++++++----------------------
 1 file changed, 44 insertions(+), 37 deletions(-)

-- 
2.7.4