Keep your software tests simple
A big headache for a lot of software testers, especially for those who create automated tests. Or write any sort of code. Is the amount of wasted time spent debugging flaky tests, or tests that are far more complex than they need to be.
Let’s use a hypothetical example of a test that checks whether we have a box filled with a solid red colour for explaining this.
Instead of having separate test cases for checking if the box is filled in with a red chequered pattern, a shade of red that we weren’t even expecting, or that we even have a box to check.
An over-complicated test would include the logic for these extra checks, maybe calling additional test code. In addition to including the original test required of it.
This results in a test that is bloated, difficult to understand, and if something goes wrong. Harder to debug.
The test is no longer something that is adding value to a software development project. Instead, it is now a risk for the testing team.
Why do we prefer complexity?
When faced with two ideas that are complex or simple. Many people have the tendency to ignore the simple option. Convincing themselves that the simple option will never work. And instead, they go with the complex solution.
This is known as complexity bias:
Source
Complexity bias is a logical fallacy that leads us to give undue credence to complex concepts.
A complex problem needs a complex solution, right?
Sometimes. Yes, absolutely. Simple solutions are not the answer to every situation
.
I’m sure you watch the news and are aware of current world affairs. Let’s take the climate emergency that is a big problem for every country right now.
We know a big factor that is contributing to this crisis is the digging up, and burning of fossil fuels, like oil and natural gas. So why don’t we stop digging up these out of the ground and cease our burning of them?
The answer is that it’s just not that easy.
If we applied a simple solution in this situation. They would generate more questions:
- What are we going to do with the thousands of people used by the industry?
- Have we thought about the prices of alternatives for consumers?
- What about the additional cost of new equipment to generate the energy we need?
A simple solution used in the wrong context only adds additional complexity.
We cannot successfully break a complex problem down into its own individual pieces. As doing so ignores the way those pieces interact with others in the system. Interactions which may cause their own errors and impacts.
But software tests need not answer the big problems in the world today.
Most of the time, a test only needs to answer one of two things.
Is this right, or is this wrong?
Introducing unrequired complexity
So-called ‘experts’, love to add complexity to the solutions they produce. It makes them feel needed, valued and enables them to say:
“See! You need me to manage all this complexity”.
But adding complexity, for complexity’s sake is never the best option and only causes a bottleneck for other members of a team. One that leads to frustration, people never being feeling fully confident in the test, and a potential black-hole of wasted time.
I’m reminded of a story I was told a while ago about an American and Russian astronaut.
The American astronaut became frustrated that his pen would work in space. Gravity’s effects were impacting the ink flow and resulted in the American not being able to write in space. So they developed a special pen that was battery operated and forced the ink out of the pen, enabling the American to write normally in zero gravity.
“Look at my pen!”, said the American to the Russian.
“It cost millions to develop a pen that would write up here, and here it is!”
The Russian looked puzzled at this exclamation from the American.
“Why didn’t you just use a pencil?”
How to think simply
I believe that complex solutions in software testing are sometimes born out of a lack of understanding of the problem. Testers may go about trying to find a solution to solve the initial question but think of other scenarios they need to cover. And then before long, the whole thing has got out of hand and beyond the original scope.
So my advice would be to use a bottom-up approach in your thinking process for trying to solve a problem or design a test.
- What are you trying to solve?
- Can you identify the input/outputs?
- What are the variables that also need considering?
Asking yourself questions like these will allow you to build on a foundation of knowledge based on the desired solutions needs.
You can then run various mental scenarios to help you design the most optimal path to take and one that everyone will understand.
