[OE-core] [PATCH] resulttool: Add reproducible log extraction

[Joshua Watt]

On 9/6/19 4:49 AM, Richard Purdie wrote:
> On Thu, 2019-09-05 at 08:54 -0500, Joshua Watt wrote:
>> Adds an argument to the log subcommand to extract the raw logs from
>> the
>> reproducible selftest.
>>
>> To prevent ambiguity, the "--raw" argument has been renamed
>> "--raw-ptest", although the old "--raw" argument is kept around for
>> compatibility.
>>
>> [YOCTO #13324]
>>
>> Signed-off-by: Joshua Watt <JPEWhacker at gmail.com>
>> ---
>>   scripts/lib/resulttool/log.py | 33 ++++++++++++++++++++++++++++++---
>>   1 file changed, 30 insertions(+), 3 deletions(-)
>>
>> diff --git a/scripts/lib/resulttool/log.py
>> b/scripts/lib/resulttool/log.py
>> index 25c3396717e..2352c767d91 100644
>> --- a/scripts/lib/resulttool/log.py
>> +++ b/scripts/lib/resulttool/log.py
>> @@ -16,6 +16,16 @@ def show_ptest(result, ptest, logger):
>>       print("ptest '%s' not found" % ptest)
>>       return 1
>>   
>> +def show_reproducible(result, reproducible, logger):
>> +    try:
>> +        print(result['reproducible'][reproducible]['diffoscope.text'
>> ])
>> +        return 0
>> +
>> +    except KeyError:
>> +        print("reproducible '%s' not found" % reproducible)
>> +        return 1
>> +
>> +
>>   def log(args, logger):
>>       results = resultutils.load_resultsdata(args.source)
>>   
>> @@ -40,17 +50,28 @@ def log(args, logger):
>>                           with open(dest, 'w') as f:
>>                               f.write(ptest['log'])
>>
> You might want to consider this in autobuilder context:
>
> http://git.yoctoproject.org/cgit.cgi/yocto-autobuilder-helper/tree/scripts/generate-testresult-index.py#n121
>
> Since the indexer could extract reproducible logs as well as ptest
> ones.

Yes, I've though about that, but so far I've purposely not done anything 
because I'm not sure what to do. Originally, I tried making the QA build 
run diffoscope on the package files that failed to build reproducible, 
but this took way too long and generated way too much data for the logs. 
I think that alternative *might* be to include the binary data from the 
packages so that diffoscope can be run on them offline, but this still 
seems like too much data.

Anyway, until we decide on a reasonable thing to do, I've omitted the 
--dump-reproducible option from resulttool. This does have the 
unfortunate side effect that you can't easily debug non-reproducible 
builds when the AB finds them, short of re-running the test locally, so 
it is something we should resolve soon.

>
> Nathan: We probably need to think about the new testsuite pieces in
> this context too...
>
> Cheers,
>
> Richard
>