Archive for August, 2010

Bug Reporting


Having been in the role of Test Lead on many projects both large and small I have seen many, many bug reports. I’ve seen good ones, bad ones and some that I wouldn’t call bug reports at all.

As a Test Lead I have been asked to coordinate the testing effort with experienced testers as well as inexperienced temp resources. From this experience I can say that one of the things that can eat up a lot of your time is bad bug reporting.

On smaller projects it was easier to deal with. There were less stakeholders and the time allotted for testing was manageable so I had time to work with the testers as the bugs started rolling in. On the large, enterprise-wide projects it was much harder. Usually the projects had an absolute deadline for production (for various reasons) and the product was almost always late to test. More time is spent managing the testing effort then triaging bugs in batches at a later time.

I think I should back up a bit and make a statement that these projects were not run in an Agile fashion. These were using a waterfall-type methodology for product delivery and if you’ve every worked with a waterfall-type method you may already understand the issues I faced in the last paragraph. But the issue at hand is bug reporting, which I believe to be the same across any methodology.

Bug reports, at the very least, should contain the following:

Clear and Concise Title
“Got an error” is about the most useless title I have witnessed. It tells me nothing. It tells the developer nothing. Since the title of the bug is usually the first thing a bug reviewer will see it should be a concise record of the problem.

“Java Runtime error when selecting Checkout in Shopping Cart” would be an example of a decent title. A great title could contain (but is not limited to) more information such as test area, test case name and test step. “Checkout – TC 1802 – BUY EGC – Step 4 – Java Runtime error when selecting Checkout in Shopping Cart” I can now glance at the title and I already understand what the problem is.

Of course, depending on what you are using to track bugs you may be limited to the amount of characters you can enter or that can be seen in a quick view of the bug list. If this is the case do your best to offer as much concise information in the title as possible.

Detailed Problem Definition

In the body of the bug report give as much detail about the issue witnessed as possible and tailor the wording to your audience. On some larger projects we would have daily bug review meetings where the priority of the bug would be determined not by developers but by the non-technical stakeholders.

Knowing this we would want to tailor the description of the problem in non-technical terms then drill down to the technical details. If we were on a small project where the bugs would go directly to the developers we might write the description of the problem in more technical terms.

As with the title, “Got an error” is not useful information to anyone. Having already put as much concise information in the title as we could, the description should contain a more detailed review of the problem. This could include:

  • Test Case Name
  • Test Steps with point of failure noted and/or steps to reproduce the issue
  • Detailed problem description – one example of how this differs from last two bullets is if the test step fails only on one type of hardware/OS/Browser etc. and not another. If the developer tries to reproduce the problem where the problem doesn’t exist the bug may get rejected causing project delays and/or bugs pushed to production that shouldn’t have.
  • Error text or detailed description of the problem witnessed if no error is thrown
  • Screen Captures or video of the problem

Every project is different in some way. As a Test Lead I would suggest evaluating what an acceptable bug report should be dependent on the makeup of the resources on hand and the information flow set forth by the project prior to the testing effort. Then be sure your testing team adheres to the template. It will save you and the project a lot of time.

Please feel free to discuss in the comments below.