How to write an effective Test summary report? | David Tzemach

Any testing effort contains several documents prepared as part of the testing process, each document is prepared to answer a specific request/demand during the software development life cycle such as:

• Software Requirements specification (SRS)
• Software High Level Design (HLD).
• Risk Management Plan (SRM).
• Software Test Design (STD)
• Software Test Plan (STP)

For more examples, please use this link containing a great PPT that summarizes the entire list of test documentation:

http://www.slideshare.net/DavidTzemach/test-documentation-throughout-the-sdlc-david-tzemach

What is a test summary report?

As you probably know, the SDLC contains many phases including testing which is used to determine the quality of the application before releasing it to the company clients.

The testing phase contains many activities such as test planning and test execution which have their own specific documentation. 

Once those phases are completed and the testing team approves the quality of the application, they will need to summarize the details, activities and any problems they encountered during the testing process and present it to the senior management, project owners and sometimes even to the company clients.

What sections should be included in a Test Summary Report?

Before examining the content of this report, know that each company may have a different test report format and content, but in general, each report must include a list of mandatory sections and data that I will review in the following paragraphs (Please note that you can change the order of artifacts as you wish). 

#1 –Title

The title should include an informative description about the project name.

#2 – Unique Identifier

Each test report should include an unique identifier that associated to each particular testing cycle, using this identifier we will increase the tractability once needed.

#3 - Objective

This section is used to describe and explain the main activities used during the testing process, besides a short description of what type of testing was conduct (Unit, System, Etc.).

#4 – Description of the Application

Under this section, we should add a very short description of the application that was tested, the main goal here is to let the readers a short review about the application complexity, goals and the business case behind it. 

#5 – References

Add any important links to the documentation used in the testing project and can be used to support this report, the mandatory references are:

• Test plan used during the testing process (STD / STP).
• Software requirements specifications (SRS).
• High-level Design (HLD) of the application.

#6 – Resource allocation  

Add a short description of the resources what were involved in the testing process, the description should include the following details:

• Name and Title
• Area of responsibility.
• Time spent during the project (house per Day/week).
• Planned vs unplanned resource modification during the project.

#7 – The application Testing Scope

This section summarizes the Modules/features that were tested/untested by the testing teams, the general idea is to let the readers understand which areas are covered in the testing process and which are not.

When you create this section, think about the next bullets:

• The integration among the application components.
• The application individual components.
• If a module is not tested, add the cause and reasons. 
• Main flows that were tested.

#8 –  Testing Activities

Define the testing activities and methodologies that were used during the testing process (It will be more efficient if you include a basic information about each activity). 

#Testing Methodologies

• Scrum.
• Waterfall.
• Extreme Programming (EP)

#Testing Activities (In-Scope)

• Functional testing.
• Exploratory testing.
• Usability testing.
• Component and unit testing
• Load and Scale testing.

#Testing Activities (Out of Scope)

• Negative testing.
• Security testing.
• Integration testing.
• Endurance testing.
• Stress Testing.

#Test Coverage

• How many test cases are successfully executed?
• How many test cases are were failed?
• Describe the total number of tests that where planned at the beginning of the project against the actual tests that executed.

#Risks

• Determine the modules that were excluded from testing.
• Determine the modules with higher risk.
• Determine the modules with low risk.

#9 – Defects Statistics and related information

This is the place to provide some statistics about the number of defects found during the testing effort, the information should help the readers to understand the test execution results.

Make sure that your report covers the following statistics:

• The Number of defects divided based on their severity (Blocker, Critical, Major etc.).
• A Total number of defects found during the version and per each testing cycle.
• Defects that were not fixed in the current version (Deferred to future version).
• How many bugs are not resolved and deferred to future versions.
• The Total number of bugs found by automation/manual testing.
• List of the most important bugs found during the version.
• Defects per application module (Defect patterns).
• The Number of defects divided based on their statuses (Invalid, Duplicate, Cannot Reproduce, Etc..).

#10 – Automation 

Add a short description about the automation framework used during the tests, the information should include:

• The number of defects found using the automation framework.
• A general Description of the automation framework.
• The effort invested in creating the automation tests.
• The number execution hours per automation.
• The test coverage of the automation.

#11 – Testing Tools

Add a short description about the 3^rd party / In-house tools used during the tests.

#12 – Test Environments

This area should include the test environments used by the testing team during the testing process, the information should include the following:

• Details about the environment configurations.
• The environment availability during the project.
• Details about any special configuration.
• Details about the operating systems.

#13 – Definition and Approval of the Exit criteria

Description of the testing exit criteria that were defined at the beginning at the testing project, the main goal is to be able to determine if all conditions are fulfilled. Example:

• All bugs with Major and above severity are fixed and verified – Approved/Declined.
• 90% from all tests are moved to automation framework – Approved/Declined.
• All planned test cases are executed – Approved/Declined.
• All important modules should be tested – Approved/Declined.
• All major Risks are removed -– Approved/Declined.
• There is a 90% code coverage– Approved/Declined.
• All development activities are done and no more testing is required – Approved/Declined.