[OE-core] Help needed with ptest

[Scott Rifenbark]

Hmm... do we want to expand on the "running ptests" section in the
documentation to include some stuff about viewing and analyzing results?
We currently have this section here ...
https://yoctoproject.org/docs/2.7/mega-manual/mega-manual.html#testing-packages-with-ptest

Scott

On Mon, Jan 28, 2019 at 3:39 AM Richard Purdie <
richard.purdie at linuxfoundation.org> wrote:

> We now have better data than ever before on the status of the project
> from a test results perspective. In particular we can now easily
> collect ptest data.
>
> I triggered the tests for qemux86-64 on the autobuilder yesterday, the
> results are published and you can see them with:
>
> wget
> https://autobuilder.yocto.io/pub/non-release/20190127-9/testresults/qemux86-64-ptest/testresults.json
>
> To show the raw logs you can quickly hack to make then more readable:
>
> cat testresults.json  | sed 's/\\n/\n/g'  > testresults1.json
>
> and there are numerous worrying things when you read them such as if
> you grep for "TIMEOUT:".
>
> I also have a suspicion that some ptest runners are exiting without
> cleaning up all processes resulting in corrupted output later in the
> logs too, I saw something like that locally, I think for perl. More
> investigation is needed for that.
>
> There are multiple areas that could do with help:
>
> a) investigate/fix the timeouts
> b) improve the test results tool Ee Peng has posted to include ptest
> processing tools:
>  i) view the logs
>  ii) compare the results
>  iii) highlight timeouts
>  iv) summarise durations
> c) Split logs into individual ptest runs instead of one block
> d) record the timings for the individual ptest blocks
> e) Add timeout status for tests into the json so we can explicitly test
> for it.
>
> I'd welcome help from anyone in any of the above areas, we probably
> need to create feature enhancement bugs for them.
>
> Cheers,
>
> Richard
>
>
> --
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core at lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openembedded.org/pipermail/openembedded-core/attachments/20190128/264d092a/attachment.html>