[OE-core] [PATCH 1/1] oeqa/concurrencytest: fix for locating meta-selftest

[Richard Purdie]

On Thu, 2018-12-20 at 12:19 +0000, richard.purdie at linuxfoundation.org
wrote:
> On Thu, 2018-12-20 at 18:43 +0800, Robert Yang
> wrote:                    bb.utils.mkdirhier(newbuilddir)
> 
> This broke on our autobuilder:
> 
> https://autobuilder.yoctoproject.org/typhoon/#/builders/56/builds/61
> 
> It basically creates a "stampeding herd" of bitbake startups as if
> you
> specify -j 40, it would do this 40 times nearly all at once and that
> causes retry problems.
> 
> We need to move the call outside the for loop as the value doesn't
> change.

I fixed the patch but it raises some questions about the way bitbake is
behaving. I wrote a test script:

#!/usr/bin/env python3

import os
import sys
import subprocess

for i in range(50):
    sys.stdout.flush()
    sys.stderr.flush()
    pid = os.fork()
    if pid == 0:
        ourpid = os.getpid()
        print("Starting %s" % ourpid)
        sys.stdout.flush()
        newsi = os.open(os.devnull, os.O_RDWR)
        os.dup2(newsi, sys.stdin.fileno())
        try:
            output = subprocess.check_output("bitbake -e bash | grep PN=", shell=True, stderr=subprocess.STDOUT).decode("utf-8")
        except subprocess.CalledProcessError as e:
            print("output: %s" % e.output.decode("utf-8"))
            os._exit(e.returncode)
        print("output: %s" % output)
        os._exit(0)

which is a bit crazy shows some errors both on the console and in the
bitbake-cookerdaemon log and shows that closing the UI connections when
we have a client is not the right thing to be doing.

This may give some insight into some of the oe-selftest failures and
I'm looking at how to improve this behaviour...

(added Chen Qi+Randy to cc as well)

Cheers,

Richard