[OE-core] [PATCH v3] libevent: add granularity to ptest log

[Trevor Gamblin]

On 9/9/19 9:35 AM, Trevor Gamblin wrote:

> From: Trevor Gamblin <trevor.gamblin at windriver.com>
>
> The libevent ptest used to report only a global pass or a fail result.
> Count individual PASS, FAIL, SKIP results. The SKIP results now
> include tests that are disabled in the libevent code.
>
> libevent's ptest output did not comply with the automake-style output
> "result: testname", and reported a FAIL status at the end of the test
> run if any of the libevent tests failed. This patch makes the log
> consistent with the automake style:
>
>      PASS: http/cancel_by_host_no_ns
>      PASS: http/cancel_inactive_server
>      PASS: http/cancel_by_host_no_ns_inactive_server
>      SKIP: http/cancel_by_host_server_timeout
>      SKIP: http/cancel_server_timeout
>
> and provides a summary as follows:
>
>      === Test Summary ===
>      TOTAL: 316
>      PASSED: 300
>      FAILED: 0
>      SKIPPED: 16
>      DURATION: 87
>      END: /usr/lib/libevent/ptest
>
> Signed-off-by: Trevor Gamblin <trevor.gamblin at windriver.com>
> ---
>   .../libevent/libevent/run-ptest               | 43 ++++++++++++-------
>   .../libevent/libevent_2.1.11.bb               |  3 ++
>   2 files changed, 31 insertions(+), 15 deletions(-)
>
> diff --git a/meta/recipes-support/libevent/libevent/run-ptest b/meta/recipes-support/libevent/libevent/run-ptest
> index 0241851c70..79d6da3ee1 100644
> --- a/meta/recipes-support/libevent/libevent/run-ptest
> +++ b/meta/recipes-support/libevent/libevent/run-ptest
> @@ -1,18 +1,31 @@
>   #!/bin/sh
>   
> -fail=0
> -for test in ./test/*
> -do
> -	$test
> -	if [ $? -ne 0 ]
> -	then
> -		fail=1
> -	fi
> -done
> +# run-ptest - 'ptest' test infrastructure shell script that
> +#   wraps the libevent test scripts
> +#
> +# Trevor Gamblin <trevor.gamblin at windriver.com>
> +###############################################################
> +LIBEVENTLIB=@libdir@/libevent
> +LOG="${LIBEVENTLIB}/ptest/libevent_ptest_$(date +%Y%m%d-%H%M%S).log"
>   
> -if [ $fail -eq 0 ]
> -then
> -	echo "PASS: libevent"
> -else
> -	echo "FAIL: libevent"
> -fi
> +cd ${LIBEVENTLIB}/ptest
> +
> +# Run the libevent regress test, but format the pass/fail outputs so
> +# that ptest can count them. Ignore the line containing "tests or
> +# "TESTS" as that line is the libevent total at the end of the test
> +# and is counted as an extra test failure, e.g.
> +# "2/300 TESTS FAILED. (31 skipped)" will be seen as a test failure
> +# rather than a total
> +./test/regress 2>&1| sed -e '/TESTS/d' -e '/tests/d' -e '/OK/ s/^/PASS: / ; /FAILED/ s/^/FAIL: / ; /SKIPPED/ s/^/SKIP: / ; /DISABLED/ s/^/SKIP: /' | cut -f1,2 -d ':' | tee -a ${LOG}
> +
> +passed=`grep PASS ${LOG}|wc -l`
> +failed=`grep FAIL ${LOG}|wc -l`
> +skipped=`grep -E SKIP ${LOG}|wc -l`
> +all=$((passed + failed + skipped))
> +
> +(   echo "=== Test Summary ==="
> +    echo "TOTAL: ${all}"
> +    echo "PASSED: ${passed}"
> +    echo "FAILED: ${failed}"
> +    echo "SKIPPED: ${skipped}"
> +) | tee -a ${LOG}
> diff --git a/meta/recipes-support/libevent/libevent_2.1.11.bb b/meta/recipes-support/libevent/libevent_2.1.11.bb
> index 1e18f0ab2c..f005ab8bda 100644
> --- a/meta/recipes-support/libevent/libevent_2.1.11.bb
> +++ b/meta/recipes-support/libevent/libevent_2.1.11.bb
> @@ -43,4 +43,7 @@ do_install_ptest() {
>   	do
>   		install -m 0755 $file ${D}${PTEST_PATH}/test
>   	done
> +
> +        # handle multilib
> +        sed -i s:@libdir@:${libdir}:g ${D}${PTEST_PATH}/run-ptest
>   }
Noticed that the autobuilder results using v2 were detecting libevent's 
"2/300 TESTS FAILED. (31 skipped)" line at the end of the regress test 
as an extra failure. Fixed that up and reduced the ptest to just running 
"regress", as the other scripts in the test folder are more 
performance-oriented and don't provide a pass/fail result that can be used.