All About Regression Testing
Regression testing is a term we are all familiar with. But when should we do such testing, what are its benefits and how can we perform regression testing well? In this article, we’ll dive into all the What, When & How’s of regression testing.
Table of Contents — All About Regression Testing
What is regression testing? 👩💻
When you are making a salad, you wash it and add it to a bowl. You know it will taste good, but by adding some seasoning/sauces the taste changes. So, the question is does your pallet approve it or have you messed up a salad entirely? 🥗
Regression testing is very similar to the analogy above. It is about discovering and understanding the risks of the product under development. The responsibility of regression testing is to make sure the product is stable when code changes/updates are added to the source code. The development team should bear in mind that any alterations could cause adverse effects on the whole system and damage the product’s core functionalities. Therefore, with this type of testing, a tester is minimizing risks of dependencies or defects due to code changes.
As our teams grow, so does the product, and complexities rise, which leads to a greater effort and coverage for regression testing. As a team, we need to make sure the regression tests are not massive as this can impact delivery times. You need the right balance. Having a good plan and carrying out regression tests to discover if anything has changed in known areas of an application is a good start. The tests can also give you a better view of whether the code changes impacted the product’s quality. In terms of a timeline, regression testing is a final step to ensure the entire application performs as intended.
When should we do regression testing? 🕵️♂️
- A new requirement is added to an existing feature/implementation
- A new functionality is added
- The code is fixed to solve defects
- The source code is optimised to improve performance
- Patch fixes are added
- The configuration of an application changed
Some examples of regression checks:
- Bug fix regression testing: Retesting bugs to see if they are fixed.
- General functional regression: We retest the product broadly, including areas that worked before, to see whether more recent changes have destabilized the working code.
- Configuration testing: The program is run with a new device or on a new version of the operating system or in conjunction with a new application. The external components that the software under test must interact with.
- Localization testing: The program is modified to present its user interface in a different language and/or following a different set of cultural rules. Localization testing may involve several old tests along with several new tests.
- Smoke testing: A relatively small suite of tests is used to qualify a new build. It helps verify new implementations are built, deployed, and function as intended. You can also include some sanity testing just to make sure new features that are ready to release are working fine and/or check bugs verified are fixed.
I came across a super interesting regression testing heuristic created by Karen Johnson, which I found quite useful. It is known as RCRCRC, a mnemonic that stands for:
- Recent: new features, new areas of code are more vulnerable.
- Core: essential functions must continue to work.
- Risk: some areas of an application pose more risk.
- Configuration sensitive: code that’s dependent on environment settings can be vulnerable.
- Repaired: bug fixes can introduce new issues.
- Chronic: some areas in an application may be perpetually sensitive to breaking.
How can we perform regression testing the right way? ✅
As testers, we are quite busy testing new features as they come… But when we are doing regression testing, it is vital to make sure we have a test suite. It is not an added value to exhaustively re-run all the tests. Have a planned target of what tests to run when performing regression. Some other things that can come in handy are:
Pairing: Pairing is something I fully support and look forward to. Pairing with a developer is useful as you will learn what changes are going into the product. You can then organize your regression plan accordingly. Read more about pairing from Lisa Crispin’s blog.
Understand your product: Do you fully understand your data and/or your application under development? It is very important you understand what is being developed, as this will allow you to create accurate regression tests and verify any high/low-risk areas in the application too. I suggest pairing as a good option to be clear on a product functionality.
Are you reviewing code? Code review is something usually developers do amongst them, but nothing stops testers from being involved. This also helps understand the new code that is going in and how it will impact your testing. You may learn a nugget of information by just questioning and listening which could help your testing strategy.
Release process: I like the idea of developers releasing code and testers deploying that to a test environment often and testing the new code. It is always good to release small and often, as loads of changes go through for a release that means more risks to test for.
If you are following a waterfall or agile methodology, then regression testing is slightly different. In a Waterfall methodology testers perform regression checks on new features plus older ones, basically run a full test suit. Whereas in an Agile team you release in small chunks and more often as the aim is to provide the client with the product sooner and learn from the client feedback. To understand these methodologies you can also read my previous blog.
Regression testing benefits 💪
How does regression testing benefit us all as a team? Remember regression testing is not just the testers’ responsibility, it’s a team responsibility.
- Regression testing executes the selected scenarios and ensures functionalities work smoothly and the application is not broken.
- Regression testing helps capture bugs prior to releasing or helps retest bugs that can impede releasing a new feature. Also something super important, if you have found a bug during regression testing, make sure this bug is for once and for all resolved. If this keeps appearing in regression testing, it means your application has something vital missing.
- Top-notch quality: the aim is to make sure an application is stable as well as getting enhanced continuously.
- Automation of regression test cases allows companies to keep the costs low by minimising the team and involvement of human interaction. In the meantime, the team can focus on more strategic and important tasks. Or do some exploratory testing! ✨
- It allows a timely intervention. If a troubling area is present in the application, then solutions can be worked on as a team just in time, before it is released to production! It also makes the software reusable and maintainable.
Summary — TL;DR
It is vital to have a stable and maintainable product, therefore regression testing becomes a ritual in the team when releasing the product. But to be able to successfully do this there are some things to take into consideration such as good planning, team awareness, visibility, and testing approach.
Therefore pair up with developers and team members to understand the product and its user journeys as this will all help plan a better regression test suite and improve the product quality. As the above image explains, you may have fixed a bug and retested it, but in turn it may have also generated another bug, so it’s crucial to have a detective hat on! 🕵️♀️ Also, avoid any “Swiss cheese” situation and make sure high-risk areas are tested thoroughly during regression.
Remember a team that works together by communicating and collaborating leads to a much smoother testing process. Get involved as early as possible in the SDLC. Finally, regression testing is all but positive for a product as it acts somehow like a product safety net! 💪
