Gunnar Kudrjavets

Paranoia is a virtue

Complete Idiot’s Guide to opening bad bugs ;-)

Unfortunately the Complete Idiot's Guide site is currently being reconstructed, so if you don’t know what it is, you would have to search elsewhere ;-) But first two major terms to get started:

  • 'Bug' - an unexpected defect, fault, flaw, or imperfection.
  • 'Triage' - the assigning of priority order to projects on the basis of where funds and resources can be best used or are most needed.

Depending on the phase of the milestone, I spend from 1-3 hours daily while attending the triage meeting. This time is spent mainly on analyzing and understanding the new bugs, changing the priority and severity of bugs, asking additional clarification questions from people who discovered the issue or are fixing it, accepting or declining the bug fixes, looking at the bug metrics etc. To summarize: I spend lots of time dealing with bugs every day. After doing this for years, I’ve reached the conclusion that I’m finally ready to write on how to report really badly written bugs in the bug tracking system ;-) Here’s the short list of things you must IMHO do to report a bug which will cause triage team to be confused and force them to ask additional questions from you. Just pick one thing and this will generally cause the desired effect:

  1. Title. Make sure that your title won’t capture the essence of the problem and its importance. Make it two or three sentences long, include contradictory statements, and give as much fuzzy information as possible. This is a good bad title: “When I do X then something seems to go wrong or I receive strange results.” Especially good titles can be determined by the following experiment: more than ten people read the bug title a couple of times and nobody can understand what this is all about.
  2. Steps for reproducing. Best thing to do is not to include them at all. Or specify only two minor steps and then omit the important information like build number or what you actually did. Also don’t mention irrelevant facts like MS Office being installed on the test machine, missing QFE-s or the fact you plugged out the network cable before the problems started to happen.
  3. Customer impact. Avoid in all cases. Just don’t think about how it’ll affect the customer or any major user scenarios. Additional suggestions include making the bug title looking really scary, but forgetting to mention that the customer will be impacted only if all planets are aligned, there are more than three people riding the bicycles outside the building, and your neighbor’s coffee machine must be running for the bug to occur.
  4. Priority. This is a good one also. Cool thing you can do is to make a title look like something really bad is happening and then set the priority to the lowest value as possible. Believe me; this will stun everyone present at the triage meeting. The opposite is also true. Good bad bug is finding the missing comma in the documentation and assigning the highest priority possible to this bug.
  5. Additional information. In case when for example AV (access violation) happens then make sure not to get any stack traces. Avoid attaching debugger to the process, loading symbols, getting a minidump, and alerting a relevant developer. The coolest thing to do is to make a screenshot of AV and include this as the only piece of information in the bug. Don’t even try to look at the source code or try to do the initial diagnosis yourself.
  6. Rare performance and stress bugs. When something which usually happens once a month occurs in your workstation then don’t store any logs, don’t take any notes about what happened, and don’t keep the machine around for somebody to investigate the root cause. Rebooting the machine is definitely the best thing to do or even reinstalling the operating system (just in case).
  7. Misspellings. Every good bad bug must have decent amount of misspellings. Make sure that bug description, title, and steps for reproducing include lots of misspellings. This speller thing is overrated anyway.
  8. Description. Copy and paste hundreds of lines of log file or NT Event Log events to the bug description field. There’s never enough of a good thing. Good approach is also writing an entire essay about the bug. The description should be long enough to make sure that it’ll take at least 10 minutes of reading to get through it.
  9. Blocking issues. When you have an issue which will potentially block something or someone for days or cause team not to meet deadlines, mention this little fact somewhere where nobody will ever notice.

These are my personal Top 9 issues. Everyone is welcome to propose an additional tip to achieve Top 10 ;-)

But to get back from my trip to the sarcastic side, let’s look at why reporting good bugs is extremely important. The main issue here IMHO is the time and the fact that everyone knows: time = money, especially in the software industry. Wasted time can reveal itself under various circumstances:

  • If there’ll be 15 people attending the triage meeting and because of poorly written bug we spend 10 minutes discussing this issue, the total amount of wasted time is 2.5 hours. Convert these 2.5 hours to the money and you’ll see. Now think about how much poorly written bugs you triage every day …
  • A developer tries to reproduce the problem for a couple of hours and fails. He/she goes back to tester only to discover that you have to have this “other application” running at the same time to make the bug to reproduce.
  • It can even get worse. Rebooting the machine where extremely rare timing related issue occurred will manifest itself two months from now when one of the key customers will encounter the same issues in the production environment and the engineer needs to be flown to customer site to troubleshoot the problem.
  • Finally, everyone's favorite. Not bothering to investigate the root cause of the problems leaves one critical buffer overflow to be unnoticed. Three weeks later MSRC (Microsoft Security Response Center) issues another bulletin ...
  • ...

One of my favorite software engineering gurus, Frederick Brooks, once said: “How does a project get to be a year late? ... One day at a time.”

Posted: Mar 05 2004, 02:26 PM by gunnarku | with 12 comment(s)
Filed under:

Comments

Dumky said:

Assign to the wrong person, bounce around or assign to another group.

Others: open dups, open multiple bugs for variations of essentially the same problem, open one bug for many different problems,...
# March 5, 2004 6:16 PM

Ferris Beuller said:

Rare timing issues become even less rare if you give people 2 way machines at least. Come duel cores to the desktop, expect alot more of these to show up more often.
# March 5, 2004 6:20 PM

Gunnar Kudrjavets [MSFT] said:

This is valid observation. We run most of our performance and stress experiments on systems with more than one CPU, so currently this isn't the case of nobody in development and test having multiprocessor machines and discovering the fun of race conditions and deadlocks later in milestone ;-)
# March 5, 2004 6:29 PM

Ferris Beuller said:

Stress testing usually is farmed out to every SDEs SDE/Ts and STEs machines at night is it not, if not why not. It should be a huge distributed stress app.
# March 5, 2004 6:31 PM

Aaron Brethorst said:

#10 Open a User Experience bug and fail to provide any sort of images to back up your claim. That drives me nuts.
# March 5, 2004 8:31 PM

Ferris Beuller said:

Why not emulate this hardware as software and run that on the machines at night?

# March 6, 2004 6:41 AM

Ferris Beuller said:

As for the actual hardware being used, you can use that on the final tests and dogfooding however for stress testing this can be emulated.

# March 6, 2004 6:43 AM

mike said:

These are also excellent guidelines for "How To Write A Bad Forum Post Requesting Help." With the difference that in that case, people just ignore it. :-)
# March 6, 2004 12:24 PM