Five reasons why automation is not a magic bullet solution

Manual testing can be quiet bothersome.

Having to write all these test cases and then execute them a hundred times. Can’t we get a computer to do it for us?

Well, in certain circumstances it makes absolute sense to make the use of the computer and code an automated routine for us using Selenium or another automated tool. If all were doing is rerunning a test to check for regressions in the functionality then sure, why not? But this isn’t always the case and automated process can’t perform the same functions that a human can.

Sometimes we just can’t rely on our tests passing to give us confidence the system is working. What if a user does something unexpected? What if they do something that isn’t part of the test case? Does your automated solution test for that?

Below are five reasons you need to still be doing manual tests to be confident about your software testing.

Automated tests can still be wrong.

Any coded solution is just more code, and it could itself contain bugs or errors that would affect its usefulness if it gave a result that was a false positive. With a Selenium test, it could also cause flakiness to appear which can cause frustration if the test has passed previously and a real issue is being hidden from the user.

Automated tests can not explore.

Automated tests can only do what the creator of that test has defined what it can do. It can never replace a human tester who will often ask themselves, “What will happen if I do this?” and then deviate from a particular scenario. They will probably then perform some other functionality that the creator of the automated test didn’t even think about and uncover a defect that would have otherwise gone unnoticed.

Automation takes time.

Time is money for many organisations and taking the time to code an automated test solution for their software that could when its complete, not function in a way that it is beneficial for the company or their test strategy. Might not be the best solution to their problems. They might be better off writing some manual test cases.

Automated tests aren’t always designed in the best way.

If you are using Selenium to perform a series of automated tests for a web application, for example. You finish coding all your tests and press the run button.. they all pass! Great, huh? Well…. while you might be glad to see that all of your coded tests are passing, but are you using a framework for your tests? How are they structured?

I have seen some really badly coded tests over the years, and it has always been because sometimes we can be lazy with our code. If something works once, then we might feel inclined to copy and paste it into a different test. If your tests cases require you to perform certain functions in different tests and you aren’t storing that functionality in its own method then your tests won’t save you time, effort and your business money.

Automated tests can never tell us what is wrong.

The biggest argument against automated testing (or checking), is that the tests can only really tell us something is wrong, but never give us the complete reason it is wrong.

When an automated test fails, it might say that it can’t find x, or that x isn’t intractable. Requiring manual investigation to pinpoint the cause of the failure, time that could have been better utilised if the test were to be manually executed from the beginning.

Automated testing can be a life-saver in many situations. It can provide us with quick results on a broad level. But it can never give us accurate results from the reasons we have seen above.

When used alongside manual testing we can gain a true understanding of applications are being used by the end-users which isn’t the purpose of automated test tools to provide.

• Category: Software Testing