[OE-core] [PATCH 01/14] oeqa/core/loader.py: support the 'auto' keyword

[ChenQi]

On 05/31/2018 09:57 PM, Alexander Kanavin wrote:
> 2018-05-31 11:32 GMT+03:00 Chen Qi <Qi.Chen at windriver.com>:
>> In previous OEQA, having 'auto' in TEST_SUITES results in executing
>> as many test cases as possible.
>>
>> This behaviour is broken for now. From the codes in core/loader.py,
>> I can see that it tries to use another keyword 'all'. But in fact,
>> it does not work.
>>
>> I've checked the current manual. The manual says using 'auto'.
>> Below is the current information in manual.
>>
>>    """
>>    Alternatively, you can provide the "auto" option to have all applicable
>>    tests run against the image.
>>
>>    TEST_SUITES_append = " auto"
>>    """
>>
>> So we should restore this behaviour. This patch does so.
> Nice! Can we also try to simplify the setting of DEFAULT_TEST_SUITES
> in testimage.bbclass using this keyword? Right now it's basically
> manually created lists, which tend to go out of date.
>
> Alex
>

Hi Alex,

I agree with you that we need to re-evaluate the settings of 
DEFAULT_TEST_SUITES.

I plan to do this work from next week. And I plan to put such changes in 
another patchset.

Looking at the codes, I can see that the DEFAULT_TEST_SUITES are set 
with consideration of qemu machines and libc.
e.g.
# aarch64 has no graphics
DEFAULT_TEST_SUITES_remove_aarch64 = "xorg"
# musl doesn't support systemtap
DEFAULT_TEST_SUITES_remove_libc-musl = "stap"

And I guess this work would take some time.

Back to this patchset, I'll sent out V2 soon.

Best Regards,
Chen Qi