The Art of Hunting

I’m a hunter by nature. I grew up in a family of outdoorsmen who loved to hunt game and fish whenever they could get the chance. Most of our family vacations revolved around hunting or fishing. Even if they didn’t it was sure bet that a fishing pole was handy just in case the opportunity arose.

I was raised in a county in Northern Michigan where hunting the Whitetail Deer was more of a religion than a pastime. The preparation for the next year’s hunt began the day after the season closed.

I have many fond memories of scouting for sign with my dad before I was legally allowed to hunt on my own.  I was probably five or six years old at the time. We’d get in the jeep and head out to the hunting grounds and start looking for trails, bedding areas, food and water sources, cover and start to form a picture of the deer’s habits. You could see the traces of what they had been doing and there was a good chance they would continue to do so. Once the picture was complete you had a good idea where to set up for the hunt and the chances of it being a success was much greater than someone who just walks into a forest and sits down.

Now that’s not so say that just showing up on opening day and plopping your butt down at the base of a tree and waiting won’t garner a successful hunt. Luck plays a factor in any hunt, but if you took the time to prepare for it and had that mental picture of where and when the deer would appear then you have the advantage.

Deer come in all shapes and sizes. The ultimate goal of the hunt is to get the big buck with the trophy antlers, but bagging a smaller buck or a doe would mean a successful hunt as well. For the true outdoorsman not bagging anything would be considered a successful season for the sheer fact that they were out there in the woods.

Software testing is not much different. Knowledge and preparation can play a key role in hunting software bugs. A little luck on your side doesn’t hurt either.

With a little luck on your side you don’t need to know anything about software bugs to find one. Users do this every day across the globe. They log in to their apps and stumble on a bug. But for the software tester knowing a bit about your game can make you more successful.

The more you can find out about the application you are testing and how it’s intended to be used the better you chances of  assisting in the overall quality of the application by finding the bugs prior to a production release.

I mentioned earlier about finding deer trails while scouting with my dad for a place to hunt. One thing about this kind of sign is that it can be misleading. While it’s true that the majority of the deer have used this trail in the past, it’s also true that other factors play a role in the hunt. During the hunting season, with the added pressures of more humans in their territory and the added fact that “the rut”, the deer’s breeding season, is on the deer is more inclined to travel 10 to 20 yards off the main trail. So if you were to set up your hunt right next to the trail you saw while scouting, there is a good chance the deer will walk right behind you during the hunt.

The same can be said for “Happy Trails” software testing. I’ve worked on many projects where the business owners or project managers, for whatever reasons (usually deadline constraints), insisted that we only focus on the “happy trails” scenarios. Basically we’re talking about only testing the main paths to an applications function. This testing was meant to prove that the application worked, not to find bugs. The problem I have with limiting testing to “Happy Trails” is the same as setting up your stand directly over a deer trail, the chance of finding a significant problem (your game) are greatly reduced.

One real-life example I witnessed was with a retail company that wanted to have a survey printed on every 20^th receipt that was printed. The testing consisted of making sure you could make a sale, accept all forms of tender and that every 20^th receipt printed had the survey attached to it. Because this project was taking place in the fourth quarter (highest volume of sales for the year) thorough testing was disregard in favor of getting the survey out as quick as possible.

When the “Happy Trails” testing was complete all tests had passed and the release was sent into the field. If you’re reading this blog you probably have that feeling in your gut that something went horribly, horribly wrong. Your instincts are correct, because they set up their stand on the trail the bug walked right by them from behind.

About three weeks after the “successfully tested” release was in the field, the company started to notice a large discrepancy between the sales numbers and the actual revenue received. As it turned out, while the testers did test to make sure all tenders were accepted on sales, they neglected to test whether the type of tender used on that 20^th sale had any affect on the changes made to the Point-of-Sale functions.

What was eventually discovered, through more testing, was that if every 20^th transaction was paid for using a credit card, the transaction would complete, the receipt would be printed complete with the attached survey but the credit card information was never sent for settlement.

Now they had a real problem. Instead of making sales, they were giving away product free of charge and asked the people how enjoyable it was to get free stuff!

At least they realized the problem in time to save some money. The credit card reconciliation process took about a month before all was lost. If they could get these transactions to the credit card companies before this time, then the cards could be processed and the company would get its money.

But at what cost? All the stores were required to send their printed journals to the home office where a team of people had to go through each transaction, one-by-one, to find the problem transactions and get them to the credit company for approval.

In my opinion the cost of doing more thorough testing up front surely would have been less than the cost of finding the problem after the fact, the cost of sending all those journals to be reviewed and reviewing them. And who know how many transactions never got reconciled?

Know your game, be as prepared as possible and the odds are in your favor.