[OE-core] [PATCH v2] libevent: add granularity to ptest log

Fri Sep 6 19:21:41 UTC 2019

On 9/6/19 3:02 PM, Trevor Gamblin wrote:
>
> On 9/6/19 2:51 PM, Trevor Gamblin wrote:
>
>> From: Trevor Gamblin<trevor.gamblin at windriver.com>
>>
>> The libevent ptest used to report only a global pass or a fail result.
>> Count individual PASS, FAIL, SKIP results. The SKIP results now
>> include tests that are disabled in the libevent code.
>>
>> libevent's ptest output did not comply with the automake-style output
>> "result: testname", and reported a FAIL status at the end of the test
>> run if any of the libevent tests failed. This patch makes the log
>> consistent with the automake style:
>>
>>      PASS: http/cancel_by_host_no_ns
>>      PASS: http/cancel_inactive_server
>>      PASS: http/cancel_by_host_no_ns_inactive_server
>>      SKIPPED: http/cancel_by_host_server_timeout
>>      SKIPPED: http/cancel_server_timeout
>>
>> and provides a summary as follows:
>>
>>      === Test Summary ===
>>      TOTAL: 316
>>      PASSED: 300
>>      FAILED: 0
>>      SKIPPED: 16
>>      DURATION: 87
>>      END: /usr/lib/libevent/ptest
>>
>> Signed-off-by: Trevor Gamblin<trevor.gamblin at windriver.com>
>> ---
>>   .../libevent/libevent/run-ptest               | 34 ++++++++++++-------
>>   .../libevent/libevent_2.1.11.bb               |  3 ++
>>   2 files changed, 25 insertions(+), 12 deletions(-)
>>
>> diff --git a/meta/recipes-support/libevent/libevent/run-ptest b/meta/recipes-support/libevent/libevent/run-ptest
>> index 0241851c70..080806dea9 100644
>> --- a/meta/recipes-support/libevent/libevent/run-ptest
>> +++ b/meta/recipes-support/libevent/libevent/run-ptest
>> @@ -1,18 +1,28 @@
>>   #!/bin/sh
>>   
>> -fail=0
>> +# run-ptest - 'ptest' test infrastructure shell script that
>> +#   wraps the libevent test scripts
>> +#
>> +# Trevor Gamblin<trevor.gamblin at windriver.com>
>> +###############################################################
>> +LIBEVENTLIB=@libdir@/libevent
>> +LOG="${LIBEVENTLIB}/ptest/libevent_ptest_$(date +%Y%m%d-%H%M%S).log"
>> +
>> +cd ${LIBEVENTLIB}/ptest
>> +
>>   for test in ./test/*
>>   do
>> -	$test
>> -	if [ $? -ne 0 ]
>> -	then
>> -		fail=1
>> -	fi
>> +    $test 2>&1| sed -e '/OK/ s/^/PASS: / ; /FAILED/ s/^/FAIL: / ; /SKIPPED/ s/^/SKIP: / ; /DISABLED/ s/^/SKIP: /' | cut -f1,2 -d ':' | tee -a ${LOG}
>>   done
>>   
>> -if [ $fail -eq 0 ]
>> -then
>> -	echo "PASS: libevent"
>> -else
>> -	echo "FAIL: libevent"
>> -fi
>> +passed=`grep PASS ${LOG}|wc -l`
>> +failed=`grep FAIL ${LOG}|wc -l`
>> +skipped=`grep -E SKIP ${LOG}|wc -l`
>> +all=$((passed + failed + skipped))
>> +
>> +(   echo "=== Test Summary ==="
>> +    echo "TOTAL: ${all}"
>> +    echo "PASSED: ${passed}"
>> +    echo "FAILED: ${failed}"
>> +    echo "SKIPPED: ${skipped}"
>> +) | tee -a ${LOG}
>> diff --git a/meta/recipes-support/libevent/libevent_2.1.11.bb b/meta/recipes-support/libevent/libevent_2.1.11.bb
>> index 1e18f0ab2c..f005ab8bda 100644
>> --- a/meta/recipes-support/libevent/libevent_2.1.11.bb
>> +++ b/meta/recipes-support/libevent/libevent_2.1.11.bb
>> @@ -43,4 +43,7 @@ do_install_ptest() {
>>   	do
>>   		install -m 0755 $file ${D}${PTEST_PATH}/test
>>   	done
>> +
>> +        # handle multilib
>> +        sed -i s:@libdir@:${libdir}:g ${D}${PTEST_PATH}/run-ptest
>>   }
>
> ptest logs are currently still including some performance test outputs 
> in addition to the PASS/FAIL stuff we are concerned with. That being 
> said, I figured it'd be better to get a working patch in to use on a 
> test run before the weekend. Example of the performance output below:||
>
> |||write callback. should only see this once||
> ||timeout fired, time to end test||
> ||usec used=80, usec passed=1501872, cpu usage=0.01%||
> ||closed_cb: detected socket close with success||
> ||=====expected||
> ||Now= 1567793437.757017||
> ||Inserted:||
> ||  0x405940 [fd  3] Write||
> ||  0x4059d0 [fd  4] Read Persist Timeout=T+1||
> ||  0x405a60 [fd  5] Write Persist Timeout=T+1||
> ||  0x405af0 [fd  6] Read Timeout=T+2.5||
> ||  0x405b80 [fd  -1] Timeout=T+1||
> ||  0x405c10 [fd  -1] Timeout=T+1||
> ||  0x405ca0 [fd  -1] Timeout=T+1||
> ||  0x405d30 [fd  -1] Persist Timeout=T+2.5||
> ||  0x405dc0 [fd  -1] Persist Timeout=T+1||
> ||  0x405e50 [fd  -1] Persist Timeout=T+2.5||
> ||  0x406000 [sig 2] Signal Persist||
> ||Active:||
> ||  0x405ee0 [fd  -1, priority=0] Read active||
> ||  0x405f70 [fd  -1, priority=0] Read Write Timeout active||
> ||  0x4059d0 [fd  4, priority=0] Read active||
> ||======received||
> ||Inserted events:||
> ||  0x405940 [fd  3] Write||
> ||  0x4059d0 [fd  4] Read Persist Timeout=1567793438.756966||
> ||  0x405a60 [fd  5] Write Persist Timeout=1567793438.756966||
> ||  0x405af0 [fd  6] Read Timeout=1567793440.256966||
> ||  0x405308 [fd  8] Read Persist Internal||
> ||  0x406000 [sig 2] Signal Persist||
> ||  0x405800 [fd  -1] Internal Timeout=1567793438.756966||
> ||  0x4058b0 [fd  -1] Internal Timeout=1567793440.256966||
> ||  0x405ca0 [fd  -1] Timeout=1567793438.756966||
> ||  0x405e50 [fd  -1] Persist Timeout=1567793440.256966||
> ||  0x405b80 [fd  -1] Timeout=1567793438.756966||
> ||  0x405c10 [fd  -1] Timeout=1567793438.756966||
> ||  0x405dc0 [fd  -1] Persist Timeout=1567793438.756966||
> ||  0x405d30 [fd  -1] Persist Timeout=1567793440.256966||
> ||Active events:||
> ||  0x4059d0 [fd  4, priority=0] Read active||
> ||  0x405ee0 [fd  -1, priority=0] Read active||
> ||  0x405f70 [fd  -1, priority=0] Read Write Timeout active||
> ||read_cb: read 12||
> ||read_cb: read 0 - means EOF||
> ||waiting for 30 conns||
> ||Average group read bucket level: 0.000000||
> ||Average group write bucket level: 0.000000||
> ||1: 683014.200000 per second||
> ||2: 4389273.600000 per second||
> ||3: 683014.200000 per second||
> ||4: 687929.400000 per second||
> ||5: 683014.200000 per second||
> ||6: 592950.200000 per second||
> ||7: 683014.200000 per second||
> ||8: 687929.400000 per second||
> ||9: 1433587.200000 per second||
> ||10: 17691353.400000 per second||
> ||11: 1433590.400000 per second||
> ||12: 17667606.200000 per second||
> ||13: 1202576.000000 per second||
> ||14: 592950.200000 per second||
> ||15: 592950.200000 per second||
> ||16: 17667606.200000 per second||
> ||17: 1199100.600000 per second||
> ||18: 16940233.600000 per second||
> ||19: 1402460.800000 per second||
> ||20: 17667606.200000 per second||
> ||21: 17670063.800000 per second||
> ||22: 592950.200000 per second||
> ||23: 592950.200000 per second||
> ||24: 17667606.200000 per second||
> ||25: 17670063.800000 per second||
> ||26: 17667606.200000 per second||
> ||27: 17699551.800000 per second||
> ||28: 17697907.000000 per second||
> ||29: 17699551.800000 per second||
> ||30: 1216486.400000 per second||
> ||   total: 230756497.800000 per second||
> || average: 7691883.260000 per second||
> ||  stddev: 8132709.287741 per second||
> ||209005, 20000||
> ||write_cb: write 12||
> ||write_cb: write -1|
>
> At least some of the libevent test scripts output only this content 
> (i.e. they don't run any of the PASS/FAIL/SKIP tests), so I'll look 
> into dropping those from the ptest unless there's a reason we'd want 
> to leave them in.
> ||||||
>
>
Just determined that only the libevent "regress" test is involved in the 
PASS/FAIL tests of interest. I'll submit a v3 on Monday.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openembedded.org/pipermail/openembedded-core/attachments/20190906/9d9ff92c/attachment.html>