Exploratory testing is not just ‘clicking around’ – Part 2
In my last post on exploratory testing. I shared with you the details on why we do exploratory testing. How it is done (not by ‘clicking around’), as well as the mindset of a tester when we are testing in this way.
To provide a recap though:
A tester performs exploratory testing without the help of scripted tests. Or predefined test cases. Exploratory testing relies on the knowledge and experience of the tester to explore a software product. They might have an end goal in mind. But they might not find it, discovering something else instead.
We do exploratory testing is to learn about the software. Once our tests have been formalised and written down. There is no more learning being done. Your tests are now checks.
In this post, I will share with you the ideal time to perform your exploratory tests, the tools that we can use to help guide our sessions. As well as answering some questions you might have around automating exploratory testing, when is enough exploratory testing done, as well as clarifying the advantages and disadvantages of this technique.
When can we do exploratory testing?
In order for a piece of software to be developed in a streamlined, and consistent manner. It should ideally follow a proven methodology that can ensure it gets from point A, to point B, smoothly.
There are several methodologies in use today with the most popular ones being:
- Waterfall
- Agile
- V-Shaped
- Big Bang
Each one comes with its own advantages and disadvantages. Which I’ll save for a future blog post to fully explain.
The main idea to take away is that exploratory testing can be used in any of these models. At any stage of development.
For example. In the unit testing phase, a programmer will be creating unit tests to drive development (if they are using TDD). They will have a goal in their mind, but they do not have the steps figured out to arrive at their destination. There is a lot of learning and exploring going on at this stage.
This is exploratory testing.
The same goes for functional stages of development and testing. You can explore, learn and grow your understanding. But when you write your tests down. You are no longer exploring.
What tools can we use?
Exploratory testing, at is essence does not require a vast amount of outside resources. With a tester relying on their own knowledge, ability to learn and experience to drive their testing.
It can sometimes be seen as a very minimalistic approach to testing.
However, there are some tools that I like to use to help me visualize my thoughts, document my findings, as well as some ‘mental shortcuts’ that really help in devising tests on the fly that are rooted in past experiences.
Mind-maps
I find it extremely helpful at the start of an exploratory testing session to be able to see the areas of the software that are not only available to me as potential start points. But also those which I’d like to explore more of.
Mind-maps are the perfect tool for this purpose as they enable you to get all of your ideas down onto paper in a structured way. Allowing the generation of new ideas and hypothesis.
There are several tools that allow you to create mind-maps. But my favourite tool is XMind.
Document your findings
When you discover something interesting in the course of your exploratory testing. It’s a good idea to make a note of it for later investigation. Also, if you find a bug or an issue that you think a developer needs to be aware of. You need to capture it in the same way as you would in any other instance.
Notepad or post-it notes
This is much of a personal choice, rather than a rule about the method that you use.
However, you need to be using some sort of program/object/activity to be able to note down anything that you find during your exploration.
Personally, I go for the Notepad option myself. I find it easy to have the program open and be able to switch to it if I need to write something down. But again, it’s up to you what you use. Try out different methods before settling on the one you find easiest.
Screen capture tools
In order to create an awesome bug report. High-quality supporting documents need to be supplied along with it. Taking a screen capture, or even recording can make developers lives a lot easier. Especially when it comes to replicating your issue.
My favourite tool for taking screen captures & recordings is Tinytake.
Heuristics
The use of heuristics in software testing allows a tester to call upon proven guidelines that follow ‘the rule of thumb’. Ensuring they are using tried and trusted testing techniques that are efficient, productive and reliable.
However, heuristics are not enforceable ‘rules’. They are fallible and are not appropriate for every context. A tester should always question their own judgement when it comes to applying a heuristic.
A cheat sheet for some common heuristics can be found here.
How much exploratory testing should we do?
Exploratory testing should always be time-bound and depend on your familiarity with the features of an application.
There is no rule for how much or how little. You need to weigh that up against your own understanding. And the risks involved in your project.
As a rule of thumb though. It should not exceed the amount of time spent performing formal testing methods.
Can we automate our exploratory tests?
Short answer: no.
Automation can only run pre-existing checks. It can not explore a system as a user would do.
It would be nice for a program to explore software like a spider, crawling around. Trying out different scenarios and actions. But nothing like that exists currently.
If you’d like to read more about what automation can’t do. I have a separate article here.
What are the advantages and problems with exploratory testing?
Without having to follow rigid test plans or pre-defined checks. Testers are instead empowered with the ability to create their own tests. Think about possible scenarios, use cases, and uncover bugs and issues that would otherwise, gone unnoticed.
There are also a number of other advantages that exploratory testing offers:
- Testing can be performed without formal documentation.
- Discovery of a higher number of bugs.
- Engages more of the testers thinking abilities.
However, that is not to say there are no disadvantages to using the technique.
- A tester should have some prior knowledge of the application.
- Depending on the skill of the tester. There is a high chance of mistakes occurring.
- Exploratory testing is usually performed when there is little time for formal testing. Which may lead to critical bugs being missed.
In my opinion. The positives/out weight the negatives for exploratory testing.
In the hands of a skilled tester. It can be an efficient way to find critical bugs without the need for a lot of prior documentation. Or other steps which can slow down other testing methods.
It is adaptable to any project, regardless of the development structure (agile, waterfall etc). And allows a tester to flex their creative muscles to perform tests based on their intuition, experience and gut-feelings.
However, exploratory testing is not a replacement for other formal test techniques. And it is recommended to keep a healthy balance between the two.
