[Openembedded-architecture] LSB is dead; when shall we drop it?

[Alexander Kanavin]

On 04/07/2017 04:59 PM, Mark Hatle wrote:

> I really think this needs to be clarified.  It would be perfectly reasonable to me:
>
> 1) Autobuilder 'tests' LSB within the scope that is defined by the project.
>
> 2) Defects are opened for test failures.  (Test failures do NOT block code
> changes, unless they are specifically related to 'LSB' work.)
>
> 3) People who care about the LSB will resolve the defects.  Resolution is:
>    - release note explaining the issue/limitation with a test-case update
>    - actual code change to resolve a problem
>
>
> The LSB should not be driving the work.  The general open source community is
> what should be driving the work forward, and the LSB work -- since it's a
> conformance test -- should be reacting to that work.
>
> I would not consider the OE passing a random testing to be good or bad.  We run
> tests to gain knowledge and decide what that knowledge means -- that is what
> defined a pass or failure.

I don't agree with this at all. We currently do not allow autobuilder 
tests to fail: if they fail, then the patch that is causing the failure 
is rejected. If you change this approach and allow test failures, this 
really opens a can of worms, and basically subverts the entire process 
which we try to bring as close to continuous integration principles as 
possible.

Manual runtime package testing by QA is a different story: those tests 
can fail, and any failures are filed as bugs and dealt with after the fact.


Alex