[OE-core] [PATCH 11/12] testimage: Further cleanup DEFAULT_TEST_SUITES

[akuster808]

On 11/11/18 1:18 PM, richard.purdie at linuxfoundation.org wrote:
> On Sun, 2018-11-11 at 12:52 -0800, akuster808 wrote:
>> On 11/9/18 6:29 AM, Richard Purdie wrote:
>>> Now the test markup of the development tools tests is complete,
>>> this
>>> can be further tweaked to auto run the correct tests.
>>>
>>> Signed-off-by: Richard Purdie <richard.purdie at linuxfoundation.org>
>>> ---
>>>  meta/classes/testimage.bbclass | 8 ++++----
>>>  1 file changed, 4 insertions(+), 4 deletions(-)
>>>
>>> diff --git a/meta/classes/testimage.bbclass
>>> b/meta/classes/testimage.bbclass
>>> index b00d47441b2..4e9494296e1 100644
>>> --- a/meta/classes/testimage.bbclass
>>> +++ b/meta/classes/testimage.bbclass
>>> @@ -40,12 +40,12 @@ TEST_NEEDED_PACKAGES_DIR ?=
>>> "${WORKDIR}/testimage/packages"
>>>  TEST_EXTRACTED_DIR ?= "${TEST_NEEDED_PACKAGES_DIR}/extracted"
>>>  TEST_PACKAGED_DIR ?= "${TEST_NEEDED_PACKAGES_DIR}/packaged"
>>>  
>>> -BASICTESTSUITE = "ping date df ssh scp python perl apt opkg gi
>>> ptest dnf rpm parselogs logrotate connman systemd oe_syslog pam
>>> stap ldd xorg"
>>> -DEVTESTSUITE = "gcc kernelmodule buildcpio buildlzip
>>> buildgalculator "
>>> +BASICTESTSUITE = "\
>>> +    ping date df ssh scp python perl apt opkg gi ptest dnf rpm
>>> parselogs \
>>> +    logrotate connman systemd oe_syslog pam stap ldd xorg
>>> kernelmodule gcc \
>>> +    buildcpio buildlzip buildgalculator"
>> Again Nak. You are forcing folks to have x11, systemd etc in the
>> "Basic". 
> No, I'm not. If the image doesn't have x11 in it, the test will show
> "skipped". If the image doesn't have systemd in it, it will show those
> tests as "skipped". If you just want to test ptest, you can specify
> that, which is what you had to do before this change anyway *just* to
> run ptest.
>
> So I'm not really seeing the problem?

So for meta-security, since it needs python and perl, will it re-run
those tests every time I run the meta-security ptests?  if it does, one
may interpret those changes as a negative to the overall testing experience.

- armin

>
> The only risk with doing this is that a test may be skipped when it
> shouldn't be. The plan is to verify the right set of test results were
> obtained when the QA overall results from a release build are compared
> against a known list of tests which should have run.
>
> Cheers,
>
> Richard
>