[oe] OE recipe tree quality

[Esben Haabendal]

On Fri, Jul 30, 2010 at 10:31 AM, Graeme Gregory <dp at xora.org.uk> wrote:
>  On 30/07/10 09:22, Koen Kooi wrote:
>> On 30-07-10 09:21, Esben Haabendal wrote:
>> > On Thu, Jul 29, 2010 at 2:07 PM, Philip Balister
>> <philip at balister.org> wrote:
>> >>
>> >>
>> >> On 07/29/2010 05:45 AM, Koen Kooi wrote:
>> >>>
>> >>> -----BEGIN PGP SIGNED MESSAGE-----
>> >>> Hash: SHA1
>> >>>
>> >>> On 29-07-10 10:50, Frans Meulenbroeks wrote:
>> >>>>
>> >>>> Dear all,
>> >>>>
>> >>>> Given the discussions on quality that sometimes pop up (and also
>> >>>> triggered
>> >>>> by Robert's message), I decided to kick off a bitbake -k world.
>> >>>
>> >>> Could you first explain to me why 'bitbake world' is a good way to
>> >>> measure quality?
>> >>>
>> >>> I would think that building something like console-image and
>> looking at
>> >>> the following would be a much better metric:
>> >>>
>> >>> * does it build?
>> >>> * are all the rootfs types working?
>> >>> * does the image do what it is supposed to do?
>> >>> * Are all the licenses of the output packages correct?
>> >>> * Do the output packages have any spurious deps?
>> >>> * Is the content of the output packages correct?
>> >>> * Are there any known CVEs in the resulting packages?
>> >>> * Did packaged-staging do its job?
>> >>> * What kind of QA errors and warnings were raised?
>> >>> * Did all recipes pass recipe_sanity?
>> >>> * Did all recipes conform to oe-stylize.py?
>> >>>
>> >>> etc
>> >>>
>> >>> I would actually advocate removing the 'world' feature from bitbake/OE
>> >>> to stop people from wasting time on looking at bitbake world and have
>> >>> them fix actual problems.
>> >>
>> >> bitbake world seems to be the source of pointless listserv
>> discussions. Does
>> >> it serve any purpose?
>>
>> > Pointless or not really depends on how you look at quality.
>>
>> > If you look at it as you, Koen and other OE long-timers, yes, it looks
>> > rather pointless to have bitbake world.
>> > But for those of us who have a different view on what quality is, then
>> > bitbake world serves a purpose.
>>
>> As Thomas points out, as soon as you start blacklisting things (which
>> actually increases quality), bitbake world doesn't work anymore.
>> That alone should be enough to kill it.
> Time to jump in the cage here.

Welcome :-)

> "Quality" is achieved by comparing a set of known specifications against
> a known data set. In the software case this means we need a good set of
> specifications which we are testing against. We also need to know in
> detail what we are testing against this set of specifications.

That is by no means the definitive definition of "Quality" in software.

>From http://en.wikipedia.org/wiki/Software_quality : Another
definition, coined by Gerald Weinberg in Quality Software Management:
Systems Thinking, is "Quality is value to some person." This
definition stresses that quality is inherently subjective - different
people will experience the quality of the same software very
differently. One strength of this definition is the questions it
invites software teams to consider, such as "Who are the people we
want to value our software?" and "What will be valuable to them?"

We should not assume that a community such as OE will agree 100% of
what software quality is, but we have to agree on what type of
software quality definitions we will work towards.

I am happy to see that I am not completely alone with my views on what
could improve the software quality of OE, but at the same time I
respect the fact that a lot of the most active OE developers have
different views on software quality.

But I believe everybody should respect that there are different
opnions on what quality is, and none of them should have complete
monopoly, hence the need for a continous discussion on software
quality of OE.

/Esben