Steps to take when evaluating a test automation tool

The crowded automated tool market is full of tools advertising attractive features and unique technological advances. Deciding on which one to use is rarely a straightforward exercise to complete.

In this post, I will share my tips that will help you look beyond the marketing claims of tool firms. Enable you to evaluate each one based on its abilities. And come to a decision that allows you to reach the intended goal of test automation.

Results.

Decide on the layer you wish to target

Whether you need to test your user interface, your API. Or something else like your database. Or even a combination of all three and more. Choosing a tool that can test everything you need over one that can only solve a piece of your puzzle. Is not only inconvenient to your testing team. It can also mean that extra time gets spent on learning several tools to achieve your intended goal.

Automating a selection of your test cases has the distinct advantage of saving time in executing repetitive steps manually. But the benefit only pays dividends if you choose the tool that can perform the tasks that you need.

Instead of going with the trendiest tool around at the moment. The first question that you need to ask yourself is: “What do I need this tool to do?’’

Consider the skills in your current team

The consequence of introducing a new tool of any kind to an existing team is that there will also be a learning curve that will need tackling. If it is a product that requires a certain amount of programming knowledge to set up and be effective. Is that knowledge already in the team?

If it isn’t, then this will need gaining through additional and outside training. Factoring this into your decision-making process will enable you to estimate the true cost of any investment that you make. It may be the situation that teaching someone to code might have a wider benefit outside the automation project. But if you don’t have the extra time. Consider other options.

Don’t overlook codeless tools

Codeless test automation is not only an attractive solution to many people. It is also a recommended approach if you want to focus on automating your application. Allowing you to achieve robust results in your test automation strategy, while not having to worry about the skills gap in your organisation.

Two of these tools which are highly recommended are Katalon Studio and Test Project.

Both offer cross-platform support, mobile app testing, and are free to use.

I had the chance to see Test Project being demoed at a recent (virtual) meet up and I came away very impressed. Its intuitive layout, workflow and various customization options makes it not only a credible option. But one that has solves many people’s problems. Check out their free trial.

Narrow your list down based on the ability of the tool

At this part of the process. You should know what you need and be shopping around for your options. My advice at this stage would be to eliminate potential tools based not only on their ability to be a suitable fit for your individual needs. But also their reputation in the wider testing community.

I would try to avoid those applications who for example only provide limited functionality, or only offer optional ability to test applications on the web or mobile devices.

It is better to go for a dedicated testing tool. Rather than one that only offers non-dedicated functionality for testers.

Instead, I would look at those tools which ideally do not use solely proprietary solutions. Closing the door to potential modifications, customisations. Or any kind of tinkering to integrate it into my workflow.

It’s also important (to me at least). To choose a tool that comes with those nice to have options. Such as the ability to produce nice graphs and reports which are available after each round of test execution.

Evaluate each one on a minor task

The ultimate stage in choosing the right tool for your organisation is to give it a test task to complete. The task should not be a big operation like complete an end-to-end test in an e-commerce application, for example. But it should be large enough to test how long it takes to both setup and complete. But also have enough complexity to tell if there will need to be further development of the framework. Or additional changes to integrate it into your existing workflow.

Tools rarely come ‘out of the box’ with everything you need. So if you need to download a plugin. Or purchase an add on. What are the costs of that? If the tool does the job, but the price of acquiring the needed add-ons is greater than the tool itself. It might be better to choose something else.

Finally, take your time with automating your test cases. It might be tempting with a new shiny tool to immediately start using it on everything. But neither is that advised or possible.

Automation should be only reserved for those tests that are repeatable, ideally in a regression suite. And not take too long to develop. But that doesn’t mean your entire regression suite should be automated.

Also, you should be continually evaluating the tool. Learning about what it does well. Any features that you didn’t expect. Or possibly even those that if you had known prior. Would have been a deal-breaker.

Using an automation tool shouldn’t be an exercise in which you feel you are wrestling with a program to reach the desired outcome. It should be simple. Easy to use and work with you to reach your intended result.

Whether you choose a coded, or codeless solution. Your goal of deciding what to work with should be rooted in not only its features and reputation. But also its ability to enable you to execute successful rounds of testing.

• Category: Software Testing