What should be the Quality of Regression Testing?

Regression testing is performed to see if any code changes or the bug fixes have caused any side effects in the software behavior. Yes, the software has the defined behavior! Regression testing is also sometimes  referred as testing  impacted areas of the bug fixes. Usually the regression testing is more than testing the impacted areas of bug fixes. The software is tested randomly to see if all the original functionality of the software are working as expected. Some testers define regression testing as an ad-hoc testing. However the regression testing is more than just testing the affected areas. There are certain things to ensure what needs to be tested as part of regression testing activities. In this post I am going to provide some guidelines on how should a regression test be carried out.

Define the Regression Test Suite

This is the first and the foremost requisite. Its usual practice that there will not be any defined regression test suits to perform. Apart from testing the impacted areas of a given bug fix, it's best practice that there is defined regression test cases that will cover all the software functionality at the high level which need to be executed before the software is claimed to be regression tested. The regression test suite should include all the functionality of the software. This is essential because the software or a software product is going to be deployed or will be used by the customers. Its a final software test before delivering the software to the customers.

Test the Functionality Dependencies

The functionality dependencies are to be identified in the first place. The functionality dependencies are treated as if a particular functionality of the software is changed due to a bug fix, its required and essential that what are the affected functionality of the software. Its quite natural in software system that the functionality are quite related. And its a challenge for the testers to figure out affected functionality and test it. This beyond the impacted area testing. This will not happen effectively without taking inputs from the developers, i.e, the persons who fixed the bugs. They are the best people to guess what could be other software functionality that might have been affected due to a bug fix. Usually developers can trace the impacted areas by taking help of the software design. The software design shows the dependencies of the software functionality. 

Update the Regression Test Suite

In a software development, it so happens that the software behavior may get changed. This will mainly happen if software development process is using methods such as Agile, Scrum. The customers feedback is continuously taken and the software or software product behavior is changed as per the customer inputs. And hence its import to keep the regression test suites up to date to the new software behavior. There should not be any communication gaps for this. It does not make sense to execute a wrong regression test case and raising bugs. Its best practice to change the original software requirement (since the customer wanted the software to behave differently than what was presented to him/her) so that the corresponding test cases are updated to reflect the new behavior.

Automate the Regression Test Suite

As we all know, the regression tests to perform takes time. Automating these test cases helps is the best of best practice. Imagine several test cycles due to several release due bug fixes or reopened bug fixes. Every time a new release, build is handed over for testing, the testing team will have to execute all the defined regression test cases. Automating these test cases help to verify the software behavior quickly to see if the original software functionality has not changed provided the automated test cases are up to date. The reason why I am highlighting the fact to update the test cases is because, its likely that its never planned to do so. So any change in the software requirement should result in changing all the affected test cases including the automated test cases.

Clearly differentiate between Regression Test and Smoke Test

Smoke test is very high level testing and may not cover all the software functionality. There could be defined smoke test suite however. If the software is smoke tested, remember its not regression tested. There has to be different test suites for these categories of software testing altogether Smoke test is performed sometimes as tester acceptance test and hence certainly it does not cover all the software functionality.

Conclusion

Basically regression testing is a challenge for any tester. It sometimes happens that even if the entire software functionality is covered under regression test suite, the bugs can still escape! This happens because a particular functionality may not have been tested at granular level and also its impractical. Hence, the regression tester has to be very experienced testing the software and understand the software functionality and guess the areas to be tested. Trusting on automated regression test suites can be  sometimes dangerous. Human interpretation on affected areas of a bug is very essential. Achieve this through sufficient discussion with the developer who made the changes. There has to be communication between developer and tester to carry out effective regression test.

.