[OE-core] [PATCH 01/12] build-appliance: Drop test which is already automated

[richard.purdie at linuxfoundation.org]

On Wed, 2018-11-14 at 08:05 +0000, Yeoh, Ee Peng wrote:
> > In that case wouldn't it make more sense to combine these steps 
> > with  
> > some of the other build appliance tests? We could add an extra step
> > where the tester builds build-appliance rather than having multiple
> > tests which start "launch build appliance"?
> 
> In general, Pros & Cons for merging test cases, Pros: reduced the
> number of test cases & making it simpler for reading, Cons: loss of
> lower level tracking on the test status. 
> Example: currently we have multiple test cases for build-appliance
> where each test cases testing the ability to use build-appliance-
> image to bitbake different type of images (recently we detect that
> build-appliance-image was able to build core-image-minimal & core-
> image-sato-sdk but it failed to build the build-appliance-image), if
> we merge multiple test cases into general
> test_build_appliance_can_build_images, then we will loss the
> visibilities on which type of images failed when this test failed,
> unless we looked into the test log if test log were generated.  

If a step fails, QA will write a bug report for the failure. I'd have
thought it would be quite clear in this case what the failing step was?
QA would also always include a failing build log in such a case which
should also make the problem quite clear.

Even if the step was "bitbake core-image-minimal core-image-sato build-
appliance-image -k", the logs should still clearly show what it was
that failed?

Cheers,

Richard