The Problem we have is that the life cycle of a software is a very dynamic. Therefore, every part in code may change or new code may be added. According to this, you should know that every change has risks behind him. The risk occurs when new code modifications may affect other code that already tested.
The Solution for this issue is using “Regression Testing” that provide great solution and coverage that reducing the risks if performed with great design and thinking behind it.
The internet contains many different definitions for this testing type, but the main idea is to make sure that Functionality that already worked and tested by the QA team still works after the new code added.
The main point for this testing type that I hope you all understand at the end of this article is that “Regression Testing is crucial to Re build the confidence that new changes doesn’t affect the great work you did in your original test plan execution.
You must understand that every change in code contain risks, regression testing is what you need to do to reduce the risks and dedicate your time for the new code testing.
Time to use
Well, based on my overview section you can understand that regression tests executed only after the main functionality is tested and approved, furthermore, the Regression tests based on previous tests that executed while the software originally tested.
When to use:
Well it’s easy, you use regression tests whenever you have risk that the original code that already tested may be affected, the risk could be arrive from different modifications, Example:
- New Feature that added to an application.
- Bug Fixes that may cause additional influence
- Changes in original requirements.
- New Requirements added.
- New patches or updated.
- Multiple builds that based on specific version source.
Regression tests Flow
Regression testing is part of predefined Flow, to make regression tests you should understand and master this flow, the following steps will demonstrate my point when new feature is added:
- All tests for a specific code are done (Functionality, System, GUI and more).
- New Feature added, this feature effect the already tested code.
Regression Flow Start her:
Step 1: Analyze and understand the impact
This step is critical for the entire flow, you cannot create a good Regression tests if you skip it, few question you should answer:
- How the feature combined with the current code?
- Which areas are most affected by the change?
- How the new feature affect the Non-Usability part for this feature (Performance, Load, stress and more...)?
- How this change effect predefined tests for the original code?
- How the change effect the testing data of the original code test?
- How this change effect the software documentation?
- How this change effect the current automation flow?
- How this change effect other components that not directly relevant for the change?
Step 2: Select the relevant testing strategy
After understanding the impact and risks, it’s now the time to select the regression testing strategy and testing guidelines, as you probably know you cannot run tests forever. Therefore, your regression tests are limited in time, Strategy examples:
· Tests the entire code– This strategy is the best when you have the Time, Money and highly trained automation team that allowing you to run the entire test plan including Old and new tests. Practical? From those criteria’s you can understand that using this strategy is not practical (Tell me if you have such resources and you’re not working in google…).
· Testing made on Highly Risk areas– this strategy will be used when the testing team identifies the areas that most likely affected from the change and probably contain the bigger risks.
· Bugs, Bugs and Bugs– this testing strategy will hold all test cases that involving bugs, every case that found bug will be part of the regression testing.
Step 3: Creating the actual test plan
Very easy, after you understand and analyze the change (Step 1), selecting the testing strategy (Step 2) it’s now the time to create the actual test plan.
Step 4: Executing regression this steps
Execution can be accomplished in 3 different ways:
Manual testing – executing regression just like every other STD, a tester or group of testers will executing the test plan based on priorities. This execution type is the less preferred for me, I’m not familiar with testers who like to repeat previous tests every testing cycle and after each modification, it’s also have massive costs for the company(Time,Money , Resources.…).
Automating Testing – My preferred way, all tests running automatically without any help from the tester, you need to remember that once regression tests are build ,this test plan will run multiple times with small modifications , automation is simply the best way to executing them.
Use automation and more automation, don’t run those tests manually.
Automation and Manual testing – Combination of the previous execution types, and may be a great solution because some cases cannot be executed with automation.
Step 4: Reporting
The testing results per regression cycle should be reported and analyzed, according to failures we can add/Remove tests and gain the best efficiency per testing cycle.
Guidelines and suggestions
· Prioritize (Guidelines) – regression tests may include thousands of test cases, in some situations you simply don’t have the time to execute them all. Therefore, you must prioritize the highly important test cases that without them the Exit criteria will be failed.
To control your prioritization simply add an additional column on your regression document, Example:
Priority 1 – Must be tested
Priority 2 – Not include any Critical/Showstoppers, Risk reduced if executed.
Priority 3 – Only if time allows
For me this step is the one that makes the differences between testers, prioritization must be made only after a tester really understand the test case and how it’ effect the total risk for the tested software (We all know that some testers will mark many cases with priority 1 just to make sure that tests will be executes to cover their a!@#.
· Documentation (Guidelines) – just like every other testing types, you must create a dedicated Testing document for regression testing, as you probably understand this Doc will be built from other test cases that already tested in the software testing life cycle.
Please remember that creating such doc should be easy when you really understand the targets you want to achieve, The Hard part is the maintenance that for most testers isn’t relevant after the creation and will never be modified when modifications should be made.
· Regression per modification (Suggestion) – you’re a QA, part of your job is not to trust the developer that says to you “The change will not affect the code, nothing should be tested”, if you can, run regression tests per change.
· Functional and Fields tests (Suggestion) – regression tests are simply great test method to get the most efficient coverage for functionality and fields.
· All Areas that affected from the modifications(Suggestion) – make sure that all areas that relevant for the change are covered, in addition, make sure that lower risk areas are not the main cases you use (Short sanity tests will be enough).
- Boundary testing – regression tests are perfect for such tests, it’s also easy to accomplished when using automation (Inputs Vs. expected results) make sure your regression contains such tests for every field that may be affected by the change.
· Cases that found bugs (Suggestion) - Select the cases that failed on the original run and verified after they fixed by the R&D team.
· Duplications (Suggestion) – make sure that every test is unique, regression execution is by default the longest testing type, believe me that duplication is not something you need.
Considerations when building Regression tests
When building a regression test plan there is some main bullets you should remember and follow, please make sure that this once are part of your considerations:
· Coverage - Regression tests are used to cover code (New Change + Old code) , you must consider the coverage you want to achieve and the resources you will spend to accomplish it, in some cases the cost are too large.
· Costs, Costs and Costs – the one question you must answer is this one “How regression tests save time and money…?”
· Maintenance – creating great STD for regression is probably the easy part, the hard part is to make great maintenance, and you must make sure that you understand the impact of the maintenance and how hard it will be to do it.