Testing Can Be A Risky Business
It’s not uncommon for a software development project to take months or even years to be completed and reach the hands of customers. Or with Duke Nukem Forever.. decades.
And yet, with all that development time and anticipation by your end-users to get to use a piece of software. One incorrect feature or obscure bug can render the entire development process to worthless in less than a minute. This can not only be an infuriating experience for everyone involved. It is a perfect example of why an effective and robust testing strategy is not only crucial. But with software or hardware that is dealing with human life. It would be crazy not to have been included.
Let’s say a new application is being developed to allow customers to purchase a range of items and deliver them to their front door. When designing the test cases we look at the application and its intended use and come to the agreement that we need 122 tests to be confident that the application is built and functions as intended.
But let us throw some constraints into the mix and restrain the time that the testers have to perform their activities. As well as reducing the number of team members available to work through the identified list of test cases.
With deadlines looming and not enough testers to execute the tests. This is a situation in which we can use risk-based testing to not only mitigate the potential negative outcomes. But also ensure that we target our testing efforts towards the areas that have a real impact on the business. And ensuring that we are meeting quality over quantity with our tests.
What is risk?
We can define risk as a potential outcome that could happen and be the reason for a negative consequence occurring.
In the above scenario, we could say that any test cases that we cannot run, could affect the overall quality of the software being developed. Quality is a very broad term and encompasses many attributes covering functional and non-functional aspects.
But if testing is an activity in which we try to enhance a piece of software by finding and reporting bugs. Not being able to run all of our planned test cases comes with the risk that the customer will find errors with unknown impact.
Now, in our example, some risks will be greater than others. If a customer can’t add items to their basket or pay for their shopping. Then the impact would be greater than the wrong font being used or the wrong shade of blue in the footer.
But how do we go about identifying those tests that are riskier than others?
Identifying risk
Identification can occur at various levels in a project. We can look at the entire risk which covers the company’s ability to meet the customer’s needs. The development of a particular feature, or as low as the risk of an individual test case.
Focusing in on test cases, I recommend quantifying the risk associated with each either as a number, a percentage or even as of one of three labels:
- Low: The least risk and potential for negative impact.
- Medium: Some risk is associated with this. Tolerable, but not desirable.
- High: A prime chance of negative outcome.
The important thing to note here is that once you have gone through your tests and identified the ones that have the most risk associated with them. You are not simply presented with a list of tests with a defined order in which to execute each one. But now have the option to manage the identified risks with several options such as:
- Avoiding the risk
- Reducing the risk
- Accepting the risk
You may even see opportunities to transfer the risk to another party altogether through other types of testing. Or asking questions and gaining extra knowledge about the application.
What are the benefits of risk-based testing?
With testing time often being squeezed and the need to perform a high number of tests to achieve confidence often being too large for many teams to accomplish. Taking a risk-based approach provides testers with a way to not only satisfy stakeholders that the system will meet the user’s needs. But also highlight potential areas for adjustments in testing efforts.
If you want to discover more about risk-based testing. I recommend watching this video of a brief talk on the subject given by James Senecal from Tricentis.
To sum up risk-based testing. It’s essentially all about testing smarter. Not harder.
I hope you found this article useful. Leave your thoughts in the comments below ?
Hi, Perhaps I have misread the article, but I read this as is more “prioritisation” and not really risk-based testing – I have a different view of Risk based testing. My view of RBT is that it is a method of using identified and determined risks as the actual basis for the testing (instead of, for example, requirements). Then testing is aimed around demonstrating that the risks aren’t likely to materialise. For example, testing an e-commerce site the #1 risk might be “the user is unable to place an order and pay for it”, or “the user is able to get goods without paying for them” (both resulting in loss of revenue for the company). Note in these examples, and more so the second one, there might not be a specific requirement or user story that actually states this (you would think the functional and technical spec would be designed around the user paying). I use this technique to drive out possible exploratory testing sessions also, and also as a complement to the more requirements based testing – it encourages to look at things from a different viewpoint, wearing a different hat.