New resolution type for bugs – "Not a bug"
[GK, 07/10/2004] Please apply 's/Not a bug/Invalid/g' while reading this post. The resolution type is meant to describe the bug quality not the correctness of application's behavior (Thanks, Larry Osterman!) Lesson learned: read and reread the stuff you post ;-)
We’re currently in the process of restructuring our bug database and here’s one thing I have wanted to do a very long time - add a new resolution type called “Not a bug“. I seriously doubt that this proposal will be accepted, but there’s no harm in trying ;-) The current set of values for resolution we’re using is following:
- By Design
- Duplicate
- External
- Fixed
- Not Repro
- Postponed
- Won’t Fix
When I put on my triage hat then I’m quite passionate about making a very clear distinction between different resolution types. The only way to get any meaningful statistics and take action based on that is to make sure that your data is correct. For years we’ve been encountering some bug entries which we just can’t classify under existing resolution types. They are either bug entries just stating some basic facts without having expected result and actual result being specified; bug entries with content which doesn’t make sense to anyone in the room; general suggestions which are too broad to classify under ‘Suggestion’ type etc.
Typically we just assign the active bug we don’t understand back to a bug opener and send a follow-up e-mail asking additional information. There are a couple of problems with that:
- People working in test organization tend to care more about resolved bugs than active bugs (your test organization may vary of course). Based on my experience it takes more time to get a response in regards to active bug than resolved bug.
- Development leads and managers are monitoring constantly active bugs and then we run into "Why SDE/T or STE has active product bug assigned to him? Is he/she going to fix it?" discussion.
- If we would use any other resolution like "Won’t Fix" or "Not Repro" then this would imply that there is actually bug in the product which we decided not to fix or we desperately tried to reproduce the problem, but couldn’t. This will of course start playing tricks with my beloved figures ;-)
Personally I would like to resolve any bug triage team doesn’t understand as “Not a bug” because this will keep everyone honest. Also I would assume that term ‘Not a bug’ will have bigger psychological impact than “Other” or something like this ;-)
The main justification I’ve been using for this is efficiency. Let’s say that there are 20 people in triage meeting and we spend 3 minutes per every triage discussing bugs nobody understands, deciding should we either resolve it, should we ask for the additional information, to whom to assign this bug etc. Practically we just spent 1h in total of people’s time instead of making a quick decision and moving on.
It’s a bug world and I think good way to mock my post is to say that we should be also using tabs instead of spaces, because this will help us to save hard disk space ;-) This is what I call self-critical cynicism ;-)