[OE-core] [PATCH 11/12] testimage: Further cleanup DEFAULT_TEST_SUITES

[richard.purdie at linuxfoundation.org]

On Sun, 2018-11-11 at 12:52 -0800, akuster808 wrote:
> On 11/9/18 6:29 AM, Richard Purdie wrote:
> > Now the test markup of the development tools tests is complete,
> > this
> > can be further tweaked to auto run the correct tests.
> > 
> > Signed-off-by: Richard Purdie <richard.purdie at linuxfoundation.org>
> > ---
> >  meta/classes/testimage.bbclass | 8 ++++----
> >  1 file changed, 4 insertions(+), 4 deletions(-)
> > 
> > diff --git a/meta/classes/testimage.bbclass
> > b/meta/classes/testimage.bbclass
> > index b00d47441b2..4e9494296e1 100644
> > --- a/meta/classes/testimage.bbclass
> > +++ b/meta/classes/testimage.bbclass
> > @@ -40,12 +40,12 @@ TEST_NEEDED_PACKAGES_DIR ?=
> > "${WORKDIR}/testimage/packages"
> >  TEST_EXTRACTED_DIR ?= "${TEST_NEEDED_PACKAGES_DIR}/extracted"
> >  TEST_PACKAGED_DIR ?= "${TEST_NEEDED_PACKAGES_DIR}/packaged"
> >  
> > -BASICTESTSUITE = "ping date df ssh scp python perl apt opkg gi
> > ptest dnf rpm parselogs logrotate connman systemd oe_syslog pam
> > stap ldd xorg"
> > -DEVTESTSUITE = "gcc kernelmodule buildcpio buildlzip
> > buildgalculator "
> > +BASICTESTSUITE = "\
> > +    ping date df ssh scp python perl apt opkg gi ptest dnf rpm
> > parselogs \
> > +    logrotate connman systemd oe_syslog pam stap ldd xorg
> > kernelmodule gcc \
> > +    buildcpio buildlzip buildgalculator"
> 
> Again Nak. You are forcing folks to have x11, systemd etc in the
> "Basic". 

No, I'm not. If the image doesn't have x11 in it, the test will show
"skipped". If the image doesn't have systemd in it, it will show those
tests as "skipped". If you just want to test ptest, you can specify
that, which is what you had to do before this change anyway *just* to
run ptest.

So I'm not really seeing the problem?

The only risk with doing this is that a test may be skipped when it
shouldn't be. The plan is to verify the right set of test results were
obtained when the QA overall results from a release build are compared
against a known list of tests which should have run.

Cheers,

Richard