Exploratory testing is not just ‘clicking around’

I have written a few posts previously that have referenced exploratory testing (here, and here for example). But in those, I have only gone as far as providing a short explanation and not deep-diving into the subject.

Which is a shame because it’s one of my favourite types of testing and the one when performed well. Can be one of the more valuable tools in your testing toolbox.

Hopefully, this collection of posts will right that wrong and you can learn something you didn’t know about testing. My goal in this post is to provide a detailed explanation of the how’s, and why’s of exploratory testing. As well as show how even if we aren’t aware of it, we all use some form of exploratory testing in our daily lives.

By the way, the title of this blog post comes from an awesome talk that Alex Schladebeck gave last year and you should check it out if you want to learn more.

So what is exploratory testing?

I like to think of exploratory testing as getting from A to B, via Z… as well as some other destinations that as testers, we aren’t aware of at the time of test execution.

It’s kinda like the testing equivalent of making a structure out of Lego. You know what you’re doing. But you don’t know the pieces that will go into making your final model.

When Cem Kaner coined the phrase in 1984. He defined it as:

“A style of software testing that emphasizes the personal freedom and responsibility of the individual tester to continually optimize the quality of his/her work by treating test-related learning, test design, test execution, and test result interpretation as mutually supportive activities that run in parallel throughout the project.”

Wikipedia

Starting with a hypothesis of how a certain feature functions, reacts or operates. A tester will then go through a process of execution that prioritises valuable feedback. And will either confirm the original theory. Or cause a shift in our mental model. Creating a new understanding, as well as uncovering any hidden bugs in the process.

The main difference between exploratory testing and other types of formal testing. Is that a tester does not use test scripts or prior written test cases to drive their actions. A tester will likely have an end-goal they would like to achieve or an area of the software they would like to investigate.

But the path they take to achieve that goal and the actions that are executed, are all produced on the fly by the tester who is testing based on their knowledge, and understanding alone.

Why do we do exploratory testing?

There are a number of reasons why we perform exploratory testing of our software. One of the biggest reasons is to combat pesticide paradox. In which it is stated that test-cases will only be able to catch so many errors in our software before they lose their effectiveness.

Exploratory testing is a way of avoiding this problem by using testers experience and creativity to perform tests without them being written down previously. This allows a tester to cover more scenarios and explore parts of the software that wouldn’t normally be covered in scripted tests.

Another reason is that it gives us the opportunity to learn about the unknown parts of our software.

Unknown unknown’s

As then US Secretary of State for Defence, Donald Rumsfeld said in a briefing in February 2008:

There are known knowns. There are things we know that we know. There are known unknowns. That is to say, there are things that we now know we don’t know. But there are also unknown unknowns. There are things we do not know we don’t know.

Take the example of gravity. We all know it’s existence thanks to the story about Isaac Newton and his encounter with an apple. But before his discovery, humans didn’t really understand why the planets were revolving around the Sun in their elliptical orbit.

Gravity was very much a thing back then. We just didn’t know it.

To apply the concept of unknown, unknowns to a piece of software. These are the parts of our applications that we don’t even know exist. Or how they will react in certain situations.

• Will it crash if we provide it with invalid information?
• What will happen if we do something unexpected?
• Will it throw an error message if I do this?

Testing the unknown elements of software diverges from the ‘happy path’, in which traditional testing methods confirm if something is right, or wrong. Instead, exploratory testing embraces the less ‘safe’ parts of applications. Providing testers with the ability to discover new things to test, push the boundaries of their understanding and expose new behaviours that were previously unthought of.

An example of exploratory testing

A very basic example of exploratory testing that we have all performed (even if we didn’t realise it). Is when we receive a new mobile phone and neglect to read the manual.

After learning how to tun the device on. We then discover how the menu is structured, where all our favourite applications are and what all the buttons do.

To an outsider, it may look like we are just ‘playing around’. Or if it’s a piece of software ‘just clicking around’, Hoping a bug will be uncovered if we only just press the right buttons in the right order.

But in reality, we are learning, we are exploring, and we are filling in the gaps of our knowledge.

‘Clicking around’ aimlessly may yield you with intermediate results. But an experienced and skilled exploratory tester will know your applications weak points, parts where the risk of bugs is at its highest. And focus on those areas to uncover high-value errors.

In a follow-up post, I will go into the tools that can be used for exploratory testing, The ideas that can help guide your exploratory testing sessions. As well as making them a bit easier.

As well as answering some questions:

• What is the process of an exploratory test session?
• Can exploratory testing be automated?
• Are there any software products that can use exploratory testing?

Contact me today to understand the value that exploratory testing can help your software development project.

Edit: Check out the second part here.

• Category: Software Testing