[OE-core] Bug reporting and good bug reports

[Paul Eggleton]

On Wednesday 07 January 2015 16:36:37 Khem Raj wrote:
> > On Jan 7, 2015, at 1:25 AM, Richard Purdie
> > <richard.purdie at linuxfoundation.org> wrote:
> > 
> > I was informed on irc yesterday that bug reports are hard and that
> > debugging via irc is easier. I think I need to remind people why good
> > bug reports are important and how they do actually help immensely.
> > 
> > I do actually believe that not everything has to have bug report. If you
> > mention an issue, someone says "hey, I know how to fix that" and sends a
> > patch out, a bug report is wasted overhead IMO. I know some programme
> > managers who disagree strongly with me and would suggest *every* bugfix
> > commit should have a defect tracking item. We're not going there I
> > understand the idea, its not practical.
> > 
> > That said, if its not immediately clear what the problem is, it should
> > become a bug report. Why?
> > 
> > Firstly, the random selection of people on irc at the time probably
> > aren't the right people to fix it. Telling those people to read 48 hours
> > of irc log for the details is disrespectful of their time.
> > 
> > It also happens that the first people referred to a bug may not be the
> > person who actually can fix it. If someone else needs to come to a bug,
> > having a summary of the issue, the salient facts and the current status
> > is immensley useful for handover.
> > 
> > As a case in point, I tried to debug a qt4 build failure yesterday for
> > which there is no bug report. I lost hours building the wrong things,
> > experimenting to try and find the reproducer steps and generally chasing
> > my tail, losing the autobuilder log of the failure, the name of the qt4
> > recipe that was failing (which task was it again?) and so on.
> > 
> > I do now have a set of reproducer steps, its quite simple:
> > 
> > MACHINE=imx53qsb bitbake virtual/kernel
> > MACHINE=imx53qsb bitbake virtual/kernel -c clean
> > MACHINE=imx53qsb bitbake qt4-x11-free
> > 
> > I also have a patch. Where should I share them? How do I ensure everyone
> > with an interest in this defect actually gets the patch? Sure I can
> > create email and send to the people who I think need to know. The
> > bugzilla lets interested parties add themselves to bugs though.
> > 
> > I should also note that QA actually go through bugs in the bugzilla,
> > including closed ones, looking for test cases. We're not great at this
> > yet but it does happen. If there is a well documented test case like
> > that above, we might write an automated QA test for it. Having it
> > documented is therefore a good thing.
> > 
> > I do appreciate writing a bug report is hard, especially if you don't
> > know where the problem is, or how to reproduce it exactly. It takes time
> > and effort. You can however document what you know and discussion can
> > happen in a common place to figure out how to reproduce it. I do except
> > the submitters to fully understand the bug, if they did they'd probably
> > write a patch instead.
> > 
> > So fair warning, I am going to start ignoring things on irc and ask for
> > bug numbers in future, assuming something isn't a 5 minute fix with an
> > immediate patch. I will back and encourage anyone else doing this too.
> 
> What about developer mailing lists ?. isn’t it also a way to report problems
> via emails after all we use emails for patch work flow. Not all people
> working with OE-Core e.g. might be following yocto bugzilla

Mailing lists are great for discussion (e.g. "I have this problem but I'm not 
sure of the right way to solve it") but a terrible way to track actual bugs, 
because mailing lists tend to concentrate on the latest postings; older issues 
that haven't been resolved rapidly disappear with the tide of new postings. Of 
course old issues can build up in a bug tracker, but at least it's trivially 
easy to find open issues where it's much more difficult to find unresolved issues 
on a mailing list.

The point is, the best way to ensure that an issue gets solved at least for 
OE-Core is to file a bug in the YP bugzilla. There is then a triage process and 
in most cases someone to actually assign the issue to so that it can be dealt 
with. There is no such process on the mailing lists.

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre