Behind the Scenes of Software Testing From the Daily Life of a Bug Hunter

Related Vendors

KIX Service Software GmbH

dataTec AG

Wuxi InfiMotion Technology Co., Ltd.

Software testing is an exciting discipline, not least because of unforeseen events. However, precisely because the workday is unpredictable, the job also presents challenges. Testing coordinator Frank Krahl provides insight into his area of responsibility.

It's Friday afternoon. Normally, my colleagues and I finish our last tasks at this time, look forward to the weekend, and share what we have planned for our days off. Today is different. Instead of "Friday from one o'clock...", we have to go full throttle again. Because my team and I are dealing with a truly extraordinary problem.

In advance, a customer contacted us because he was experiencing hundreds of error messages, but only on Mondays. During the week, we were already able to identify the problem through detective work. Now it's about testing the bug fix and getting the patch ready. Otherwise, our customer would face another Monday with very limited work possible.

Despite the time pressure, we also have to proceed carefully to ensure no further errors slip in with the patch. The minute hand seems to move faster and faster, and we are slowly sweating.

Dream Job: Software Tester?

As a child, I once wanted to become an astronaut, but later I thought that staying on Earth isn't so bad either. That's why I trained as a technical assistant for quality assurance, and I had many ideas about what I could do with it.

Software tester wasn't at the top of the list, though. I wanted to help people and joined support. There, I occasionally came into contact with colleagues from testing. In 2017, I switched to KIX, and both my team leads and I quickly realized - it fits. Today, I coordinate the testing team.

Like me, my colleagues have come here through very different paths - from development, support or sales. And as stressful as initially described, the job is really only in exceptional cases. Of course, we prefer to find bugs before our customers and partners notice them. To do this, we conduct RC, release, or regression tests, in which we put new features and hopefully fixed bugs to the test.

We also use a tool for test automation that takes a lot of work off our hands. However, we still can't put our feet up: We have to feed it with requirements and test scenarios so that it processes regularly recurring steps and informs us of any incidents.

But what do we do once a user reports a problem to us? First, we try to replicate it on a current system with their settings. If that doesn't work, we repeat the process on a system with the same version as our customer's. Errors often occur due to many different configurations. And sometimes it also comes down to every minute on a late Friday afternoon until we have a solution.

I have never regretted the step into testing. I find it particularly interesting that you need to have a certain nose for things. It is not necessary to know a program in every detail - that is more the job of the developers. It is much more important to be open to new ideas in order to solve very specific cases. We often have to think outside the box and look at a problem from the customer's perspective. This way, we learn something new every day, both technically and professionally.

Bugs in the Space Shuttle

What I can only advise newcomers in the testing field: Don't feel personally offended if a bug slips through - it happens! This of course does not mean to approach the job completely carelessly, after all, there are enough examples of bugs that have caused damage worth millions. A few years ago, a chocolate manufacturer wanted to reduce costs through an ERP change. However, due to faulty software, there was an overproduction, tons of unsellable sweets, and a loss of 14 million euros.

A completely error-free software is and remains utopian. In addition to the individual circumstances of the users, it is often due to the code itself. With often hundreds of thousands of lines of code that interact and ideally harmonize with each other, even a small change can lead to problems that did not exist the day before. Some people also refer to very complex constructs as spaghetti code because everything is intertwined.

However, the greatest natural limit of testing is usually its cost-effectiveness. Every company must find a compromise between error costs and error prevention costs, creating a gray area where errors can never be completely avoided. In our company, for example, we work with five error classes, divided from A to E.

If users cannot create a ticket, this would be a class A error for us - and all sirens would go off because we need to fix the problem as quickly as possible. The lowest category, on the other hand, includes more trivial things, such as spelling mistakes or a crooked icon, which in all likelihood do not have a detrimental effect on the business. We also address these errors, but they do not have the highest priority.

To illustrate the financial aspect: In the 1970s, the development of the Space Shuttle along with its associated software began. And even in this high-tech marvel for its time, there were software bugs. Although admittedly not many.

On 420,000 lines of code, NASA had less than one error. How did they achieve that? Well, it was a prestige project of the USA with a huge budget. A written, tested, and documented line of code cost around $1,000 at the time. By today's purchasing power, about $7,000 - or roughly three billion dollars just for the software. Without the rocket, launch pad, and fuel.

Bug, Feature – or Both!?

However, such a bug can also be helpful. It can happen that they save users a few clicks or make the system more stable. There are several examples of this.

Even about 50 years ago, there was a bug in the computer game Space Invaders that might have even contributed to its success: Whenever the player shot down an enemy spaceship, the remaining aliens moved towards him faster because less computing power was required. The developers hadn't planned this, but since it made the game more exciting, they never fixed this bug.

Every now and then, things can get a bit bizarre in the world of bugs. For example, one of our customers had random first and last names generated in contact synchronization. While names like Harald Lee or Montgomery Müller might exist, they seemed a bit strange to us. We quickly found the cause of the problem – instead of the customer data, our demo data was being loaded.

Significantly trickier was the sweat-inducing Monday bug from the beginning: Ultimately, it was a formatting error in a database field that led to the error messages. A small hidden problem, then, but with big impacts. Incidentally, we were able to fix it in time that Friday afternoon. That's what makes the job as a bug hunter so appealing to me – it's never boring.

*Frank Krahl is the Testing Coordinator at the Chemnitz-based IT company KIX Service Software. Together with his team, he runs various testing scenarios daily—from automated workflows to regression tests.