[oe-commits] Stefan Stanacar : lib/oeqa: change behaviour for unskippable tests

[git at git.openembedded.org]

Module: openembedded-core.git
Branch: master-next
Commit: fd51cecf8b258d9f839a0ecebde69d09f75dc468
URL:    http://git.openembedded.org/?p=openembedded-core.git&a=commit;h=fd51cecf8b258d9f839a0ecebde69d09f75dc468

Author: Stefan Stanacar <stefanx.stanacar at intel.com>
Date:   Thu Aug 22 15:02:55 2013 +0300

lib/oeqa: change behaviour for unskippable tests

When a test module wants to be skipped because it doesn't
apply to the image but it was nevertheless a required
test (one in TEST_SUITES), we issued an warning that it
was a required test and went on with running the module.
Usually all tests in the module failed (e.g gcc tests on a non-sdk image),
but this allowed us to know that something went wrong with the image
(some package/feature didn't make it).

However, instead of just issuing an warning and running the tests
it's better to throw an exception. The traceback will tell us what's wrong,
and we don't run every single test method.
Output will look like this:
--snip--
| NOTE: Test modules  ['oeqa.runtime.ping', 'oeqa.runtime.ssh', 'oeqa.runtime.gcc']
| NOTE: Found 5 tests
| test_ping (oeqa.runtime.ping.PingTest) ... ok
| test_ssh (oeqa.runtime.ssh.SshTest) ... ok
| ERROR
|
| ======================================================================
| ERROR: setUpModule (oeqa.runtime.gcc)
| ----------------------------------------------------------------------
| Traceback (most recent call last):
|   File "/mnt/back/yocto/poky/meta/lib/oeqa/runtime/gcc.py", line 8, in setUpModule
|     skipModule("Image doesn't have tools-sdk in IMAGE_FEATURES")
|   File "/mnt/back/yocto/poky/meta/lib/oeqa/oetest.py", line 108, in skipModule
|     "\nor the image really doesn't have the requred feature/package when it should." % (modname, reason))
| Exception:
| Test gcc wants to be skipped.
| Reason is: Image doesn't have tools-sdk in IMAGE_FEATURES
| Test was required in TEST_SUITES, so either the condition for skipping is wrong
| or the image really doesn't have the requred feature/package when it should.
|
| ----------------------------------------------------------------------
| Ran 2 tests in 1.036s
|
| FAILED (errors=1)
| NOTE: Sending SIGTERM to runqemu
--snip--

Signed-off-by: Stefan Stanacar <stefanx.stanacar at intel.com>
Signed-off-by: Saul Wold <sgw at linux.intel.com>

---

 meta/lib/oeqa/oetest.py |    4 +++-
 1 files changed, 3 insertions(+), 1 deletions(-)

diff --git a/meta/lib/oeqa/oetest.py b/meta/lib/oeqa/oetest.py
index 4a406e7..c9dc5dc 100644
--- a/meta/lib/oeqa/oetest.py
+++ b/meta/lib/oeqa/oetest.py
@@ -103,7 +103,9 @@ def skipModule(reason, pos=2):
     if modname not in oeRuntimeTest.tc.testsrequired:
         raise unittest.SkipTest("%s: %s" % (modname, reason))
     else:
-        bb.warn("Test %s is required, not skipping" % modname)
+        raise Exception("\nTest %s wants to be skipped.\nReason is: %s" \
+                "\nTest was required in TEST_SUITES, so either the condition for skipping is wrong" \
+                "\nor the image really doesn't have the requred feature/package when it should." % (modname, reason))
 
 def skipModuleIf(cond, reason):