[OE-core] [PATCH 0/1] scripts/oe-selftest: oe-selftest-internal wrapper scripts that isolates execution

[leonardo.sandoval.gonzalez at linux.intel.com]

From: Leonardo Sandoval <leonardo.sandoval.gonzalez at linux.intel.com>

The below is a profiling experiment, running oe-selftest -r (the proposed
implementation, see patch description for more info):

Procedure:

With patch 1/1, multiple oe-selftest jobs can be launched in
parallel. One tool that launch jobs in parallel is GNU Parallel [1], allowing
to construct a simpole pipeline to execute all tests with a pool of four
jobs:

    $ echo $ALLTESTS | time parallel --jobs4 oe-selftest -r

where ALLTESTS is a variable containing all tests cases (modules) found by the
the runner (oe-selftest-internal) (i.e. ALLTESTS="$(oe-selftest -m |
awk '{ print $NF }' | grep -v ':')"). This is the result obtained from the
above command:

    739.57user 120.48system 45:34.61elapsed 31%CPU (0avgtext+0avgdata 124600maxresident)k
    390908inputs+15984336outputs (291major+20227951minor)pagefaults 0swaps


The import point on the above numbers is that isolation the oe-selftest execution per
module and using a parallelization tool, complete oe-selftest runs takes less than an hour,
beating current single-job times observed at main auto-buildes.

Profiling results were obtained on a machine with 88 Intel Xeon with 88 cores

[1] https://www.gnu.org/software/parallel/

The following changes since commit 65d23bd7986615fdfb0f1717b615534a2a14ab80:

  README.qemu: qemuppc64 is not supported (2017-10-16 23:54:31 +0100)

are available in the git repository at:

  git://git.yoctoproject.org/poky-contrib lsandov1/oe-selftest-own-directory
  http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=lsandov1/oe-selftest-own-directory

Leonardo Sandoval (1):
  scripts/oe-selftest: oe-selftest-internal wrapper scripts that
    isolates execution

 scripts/oe-selftest          | 102 +++++++++++++++++++++++--------------------
 scripts/oe-selftest-internal |  75 +++++++++++++++++++++++++++++++
 2 files changed, 129 insertions(+), 48 deletions(-)
 create mode 100755 scripts/oe-selftest-internal

-- 
2.12.3