[OE-core] [PATCH 1/2] commands: send stderr to a new pipe

[Patrick Ohly]

On Thu, 2017-06-22 at 17:59 +0100, Richard Purdie wrote:
> The problem here is that stdout and stderr are being mixed and that
> mixing is not accounting for newlines. bitbake itself does print
> different messages to both. The mixing depends on kernel page size and
> when the buffers are flushed.

It's a single process were stdout and stderr share the same pipe.
Caching outside of the kernel may be to blame, but kernel behavior
should not have an effect.

My guess is that "Parsing recipes..WARNING:
File /etc/selftest-replaceme-patched..." comes from printing "Parsing
recipes.." to stdout without newline and then printing "WARNING:
File ..." to stderr.

I don't know whether Python logging can coordinate two streams. The
logic would have to be that when stdout/stderr go to the same stream and
the last output to stdout did not end in a new line and there's a
message to be printed on stderr, a newline should be inserted first.

> The key thing I'm not sure about from reading the bug is whether this
> output is from the same bitbake process or different ones?

It's from a single "devtool" invocation, so whatever goes wrong with
output handling is in that process.

> I can think of three things we can do here:
> 
> a) Add some flush calls somewhere assuming this is different calls.
> That somehow seems unlikely
> b) Keep stdout and stderr separate as per Leo's change

That helps testing, but actual users will still see confusing output. So
I would prefer to enhance output handling. However, it's probably more
complicated than running tests with split output.

> c) Have a command version which merges stdout and stderr respecting
> newlines. I suspect this happens on a console but likely not on inter
> process pipes...

In this case it happens for a pipe.

> It does sound like we need to answer some questions on what exactly is
> happening before we can know which solution(s) might work.

Yes.

And please, whatever we do, let's not change oeqa in a way that breaks
existing tests, whether they are in OE-core or some other layer.

-- 
Best Regards, Patrick Ohly

The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.