Not all bugs are created equal

The bug report is one cornerstone of a software testers work. Not only does it represent the errors or problems we have identified in a piece of software. But for developers on the receiving end of our submissions. They also define a portion of their workload for the day ahead.

It is a tester’s responsibility to report issues and bugs which have a negative on the quality of the product being delivered. But as part of that process. Any reported bug should also receive an associated severity level. This is a measured value which reflects the bug’s impact in the overall application.

To complicate things a little further, each bug also needs its own priority level. Which aims to minimise the impact on the development of a product and those issues with the greatest effect to end users get the required attention.

I will preface this by saying every team is different and you should have a conversation with your internal team before assigning levels to bugs. Some teams, for example, might have tester’s deal with everything related to raising a bug report. But others may only require testers to assign a severity level. Either way, knowing how each part works and how to decide the appropriate priority and severity level will aid your overall knowledge in the handling of bug tickets.

Severity

So you have identified a bug during the testing of a user story. But when raising the report and assigning it a severity level. You need to take the time and properly assess how ‘bad’ the bug is. As technical people, we can sometimes be a bit too quick to come to a resolution without thinking through all the options or evaluating all the causes.

But this is one area that I highly advise taking your time to choose the right level. Not only because it can frustrate developers when they receive a ticket marked as with a high severity level which turns out to be a minor bug. But being familiar with the process needed to assess an issue and discover how it affects the system overall. Is a valuable skill that has a wide range of applications. From estimating workloads to evaluating risk.

Common severity levels

Depending on the impact that bug has. It will receive one of the following levels, (these levels can change slightly between teams).

• Low: Reserved for those bugs whose impact doesn’t noticeably detract from the software functioning as intended.

• Minor: Bugs with this level may cause undesirable results occurring. But still doesn’t negatively affect the software.

• Major: These bugs are affecting the software negatively and are causing unavoidable issues.

• Critical: Bugs which render the software unusable.

How to choose a severity level

Now that we have identified the severity levels available to us. After isolating the bug and we are able to reliably reproduce the results when needed. The process should then pivot to one of examination to determine not only the bug’s overall effect but also the consequences for the software.

How is the end-user effected?

Are users being prevented from carrying on with their journey through the application? Or are they only slightly inconvenienced?

Does the bug occur frequently?

Issues such as typos on a page might not seem that severe and may receive a low rating. But if that text appears on multiple pages and is something the user sees frequently. That might be a trigger to bump the severity level upward on the account of it being annoying to users.

Does the bug occur in a limited set of environments?

If it is a website issue. Does the browser version or screen resolution change the result?

Priority

This is the answer to the question of how urgently a bug needs fixing and retested so it is no longer present or occurring in the application. It’s also a good indicator for developers to see the order of the bugs that they need to work on.

• Low: We assign this level to those bugs that do not require an immediate fix. Or there are other issues that should take precedence.

• Medium: Bugs with this level can be fixed as development progresses.

• High: These bugs need to be fixed at the earliest opportunity. We should reserve this level for those bugs which render the software unusable or with no workaround available.

There can even be one or two levels above high which we reserve for those bugs which cause catastrophic data loss or crashes. Commonly known as ‘Blockers’, bugs with this level attached need fixing before the next release.

Now that we have seen the meaning behind these two terms and what they represent. We can differentiate the two by realising that severity is about the impact and how bad the bug is. Whereas priority is about scheduling and ticket management.

Lets look at some examples of bugs with different severity and priority levels and as part of that. Show how we can realise the appropriate levels for the ticket.

Example of a High Severity and High Priority bug

Given the example of an e-commerce application in which you have tested previously in Chrome with no bugs present. Upon switching to another browser. You find that the menu which runs consistently throughout the website is missing and doesn’t show up. Meaning that users of browsers other than Chrome cannot navigate the website. Or interact with one of the site’s crucial features.

Both severity and priority would be high here because a portion of users are being prevented from using the application. Plus, it is preventing user interaction with a key part of the site.

Example of High Severity and Low Priority

While high severity covers bugs which cause severe disruption, enable a negative user experience and degrade functionality. If it is only restricted to a few users however, then it is likely to be considered as a low priority for fixing.

A detailed example would be a website not appearing correctly in legacy browsers (images not loading, text overlapping, fonts missing etc). While the issue might break functionality and be worthy of a high severity level. It is however only affecting a small subset of users and warrants a low priority.

Example of Low Severity and High Priority

When accessing a website in the latest version of Firefox. You notice that some buttons are overlapping and even though they are still clickable and functioning correctly. The overall look of the page isn’t as visually appealing as it is in Chrome where there are no issues.

The severity in this case, is low because the page still able to function as intended. But because the look of the page isn’t as it should be in the latest version of a popular browser. The priority to fix it is high.

Example of Low Severity and Low Priority

The best example for this is typos in areas that users might not be too concerned about. They aren’t affecting important functionality and are more than likely to less of a concern for end-users. So, in this case, the issue would be low severity and low priority.

Bugs are consistent within the world of software development and taking the time to familiarise yourself between these two terms will not only help you distinguish appropriate levels to assign to your reports. But it will also go a long way to enabling quality software to be produced in the most time efficient way.

Developers can’t handle everything at once, and there may be some room for interpretation of the correct levels to use in different situations. That is why you need to collaborate closely with the everyone in your team. Come to an agreement on the appropriate levels to use in the bug tracking process. And put this into practise so developers lives are that little bit easier.

I hope this article helps! Let me know your thoughts in the comments.

• Category: Software Testing