[OE-core] RFC: oe-selftest script to run automated tests against bitbake tools
Paul Eggleton
paul.eggleton at linux.intel.com
Tue Jul 9 10:37:44 UTC 2013
Hi Mihai,
On Monday 08 July 2013 20:33:20 Mihai Lindner wrote:
> The idea of "oe-selftest" would be to have an extra method of testing to
> cover what's not addressed by sanity tests, bitbake-selftest or image
> run-time testing. More information and some example tests are posted in
> the related bug, here:
> https://bugzilla.yoctoproject.org/show_bug.cgi?id=4740
>
> Before starting any work on this, maybe we can gather some thoughts,
> comments, here or in the bugzilla entry, something in the lines of what
> kind of tests should it run or not, what would be the usage scenario,
Some of the kinds of tests I would expect this tool to be able to handle:
* Tests that require configuration changes or addition/modification of testing
recipes. One issue here is that if the script did modify existing
configuration/metadata you could potentially break user's environments if they
run this script without realising what it does. Maybe it should set up its own
temporary build directory to avoid this, and add additional recipes via a
temporary layer? This could help keep the test environment sane in any case.
* As an extension to the above, tests that verify failure output. We often
don't notice when something breaks in the error handling code, and this leads
to regressions where we get python tracebacks instead of a proper error
message.
* yocto-bsp/yocto-layer (although these are not in OE-Core - perhaps this is a
chance to engineer the oe-selftest script so it can be extended by additional
layers? I think this is something we want to allow for anyway)
* buildhistory (by running builds and then comparing using buildhistory-diff)
* create-recipe
* runqemu?
Cheers,
Paul
--
Paul Eggleton
Intel Open Source Technology Centre
More information about the Openembedded-core
mailing list