Increasing bug quality in your organization - yes, it can be done
The subject of poor quality bugs has been touched here before - I wrote a big rant about the particular problem a month ago and Larry Osterman covered some aspects of other bug related problems in one of his posts. In this entry I’ll describe how we handled and are handling poor bug quality related problems in my team (Telephony Application Services). I won’t claim that our bugs are perfect, but at least in daily triage participants are way less sarcastic than about a year ago ;-) IMHO making sure that people in your team report quality bugs isn’t a rocket science; it’s just a question of basic engineering discipline and persistence.
Setting expectations
This is the first thing to do. It needs to be very clearly communicated to everyone what information you expect to see in bugs and how it must be presented. If your bug management tool supports templates then provide one and make sure that everyone knows that they must use this template. If you can’t use bug templates then make sure that you give a list of topics that must be covered in a bug. For junior people it’s always helpful to explain why you need specific parts of the bug report (build number of the product, information about operating system, log files, stack trace etc.), people with years of development/testing experience usually know what this is all about. The typical bug entry consists of the following parts:
- General problem summary.
- Steps for reproducing the problem.
- Expected result.
- Actual result.
- Customer impact.
One thing which may sound a little bit harsh, but IMHO if the person isn’t able to follow specific instructions to report a bug or the person isn’t able in concise technical terms to explain what’s wrong with something and why he/she thinks that this behavior is incorrect then the person shouldn’t be working in the software industry at all. Being able to write a decent bug report which everyone else is able to understand seems to me like a basic skill and precondition for a (successful) career in software quality assurance or testing.
Making it happen
I like to flatter myself with the though that at some point I was the most unpopular person as seen by the people who opened any Telephony Application Services bugs ;-) Achieving good bug quality isn’t so complicated - whenever somebody does something wrong then just send them an e-mail, explain why their bug doesn’t meet your quality bar, and ask them not to do it again.
Our daily component triage meeting has taken place usually at 11:00 AM in my office for the last two years or so. About 10:30 AM I start looking through all the bugs assigned to triage alias and per every bug what I think doesn’t have enough information, I send an e-mail to the person who opened a bug, and ask him/her to fix the problem. Our bug tracking software has a wonderful "Send Mail..." choice from the "File" menu which greatly helps ;-) The usual problems are:
- Steps for reproducing bug aren’t clear enough.
- Bug is missing a product build number.
- URI for log files pointed in bugs is incorrect.
- AV bug doesn’t have enough information attached.
- GUI bug doesn’t have related screen-shot attached.
- The problem description isn’t specific enough.
- ...
Practically this is just lots of basic "bug police" work. I don't like doing this, but it's part of my job responsibilities and ultimately somebody has to perform this task. Of course IRL it’s not so easy to make bug quality problems disappear. I personally divide the people who violate bug quality guidelines and related actions into four different categories:
- Person just didn’t know that some information is needed or it wasn’t clearly communicated to him what information is needed in bug. Perfectly understandable, I’ve been in this position myself. The solution in this case is just to send a polite e-mail and point person to existing bug guidelines and explain why we need this information.
- Person has already been informed of bug etiquette, but he continues to ignore it. In this case I send polite e-mail explaining why we really need to have for example log files from the moment when server crashed, sign the e-mail as "TAS triage team", and insist that bug guidelines will be followed.
- Third category is a phase when development leads start losing patience and making very cynical remarks or developers start sending e-mails and complaining that based on the information in the bug they can’t do much anything. In this case we use powers given to us to send a polite e-mail, include bug opener’s manager’s e-mail on ‘Cc:’ line and strongly insist that bugs will follow the specific guidelines. Usually it ends here.
- We’ve never reached fourth category, but fourth category for our triage team will be that we’ll stop accepting bugs from a person who constantly opens poor quality bugs unless his/her manager signs off on these bugs. I hope we would never have to do this.
Like I said before, IMHO it’s just a question of discipline and holding up to a certain quality bar. Poor quality bugs just waste everyone's time. Anyone who has read "The Mythical Man-Month" remembers the following: "How does a project get to be a year late? One day at a time!"