Wednesday, January 15, 2014

STP (Software Test Planning) – Part 3

Section 4: Testing methodologies
Every testing process should base on testing methodology, this section should explain the one we are going to use on current testing process.

Main Bullets:
  • Selected Methodology – describe the testing methodology (Waterfall, Agile-Model…).
  • Benefits – Describe the benefits we are going to achieve when using it.
  • Flow – Display a short flow for the methodology implementation. 

Low detailed Examples:
V model:
Requirements - > STP - > STD->Unit testing - > Integration testing….

Requirements -> Design-> Implementation->Verification…..

My example contains the headers, you should add additional information per section. 

Section 5: Detailed information regarding your test strategy

These sections should provide detailed information regarding the test strategy we are going to use in the testing process, this strategy should be based on the selected methodology, Available resources (Human, Hardware and more), limitations and requirements.

In a very simple words, this section should describe how the testing process will confirm the actual results against the software requirements (SRS).

For my opinion and based on my actual experience, this section should contain the followings bullets: 

Part 1: Documentations
Every Quality Assurance process contains few mandatory docs (Requirements, STP, STD and STR), this section describe the relevant details for each doc type.


Each doc must Contain :
Detailed information based on customer or business requirements
I hope you understand it after reading current doc…
Highly detailed testing scenarios based on STP, this doc includes the test case and the expected results based on Requirements.
Written after the STD compilation (In some cases you can start written it while executing the STD), this doc must include the testing results, opened bugs, Risks, suggestions and more

Part 2: Separate the software components and determine the integrations between them
The first thing you should do is to ‘Separate’ the software components, 
it’s easier to control your testing process when you design it to multiple
components and not to one big elephant called ‘Software’,
using this strategy you can see the list of integration tests 
you need to test for different components.

Let’s give example for a bank account system:
  • Components: User interface, Login procedure, login engine, Database
  • Integrations: DB and Engine, UI and Engine, DB and UI…

Part 3: Determine the testing cycles
This part should provide the number of testing cycles involved on the testing process, each cycle should be described with the testing types and levels you are going to use, Example:

Cycle 1: Exploratory testing + validating the fulfil of the Entry criteria’s

Cycle 2: Functionality testing + Bug Verifications

Cycle 3: Regressions (Tester 1) + Non functionality tests (Tester 2)

Cycle 4: Sanity

Cycle 5: You got the point….

Part 4: Testing types involved in the testing procedure
This section will be used for describing the testing types we are going to use on the current testing procedure.

Unit Testing – performed by the development team before QA can start Integration tests.

Computability testing – software will be tested on Environment A and B, Operating Systems B and C…

System Tests – the tests will be made after all components and integration tests completed.

International Testing – the product will be tested on different OS language packs because it’s going to be installed worldwide.

Stress Tests – tests will be made on servers that possess massive resources, in addition, the tester will use tools A and B.

In Addition, This section is the place to describe also the testing methods that we cannot accomplished due to lack of resources.

Non functionality testing like Load, Performance and stress that you need specific Tools and Hardware to executing them.

Section 6: Completion criteria’s (Exit Criteria)
This section should specify the exact criteria for success,
if one of the criteria’s not accomplished 
the Risk become larger, Example:
  • All bugs with severity greater than ‘Normal’ verified and closed.
  • All test cases on STD with priority greater than one must be executed.
  • Client requirements covered.

Section 7: High level testing details 
You all know that STP is the Key Stone for a great STD.Therefore this section will hold the main areas you need to test and will be expend on STD documentation.
Main Purpose: 

  • All bullets will be used as guidelines when writing the STD
  • Owner can review the bullets and determine if the coverage is good enough.

In some companies this section called TRD (Test Requirement Document), and will be written as completely different doc.

We have new car for testing, I’ll provide few bullets for testing:

·       Functional Testing :
1.      Test the car switch (Electricity on half switch).
2.      Test the car switch (Engine ignition on full switch).
3.      Test the car switch when Power off.

·       Usability Testing:
1.      Test that the drive shift is smooth and easy to manipulate.
2.      Test the dashboard buttons location and size.
3.      Check if the rear camera is easy to use based on the wheel buttons.

STP (Software Test Planning) – Part 2

Your STP structure may be different per company, in some companies the STP may include different sections that not relevant to other companies,but must in the local QA process.

This section will provide my suggestion for the “Key Stones” that every STP must have, feel free to comment my suggestions I’m always eager to learn new ways to improve myself.

Section 1: Project Review
This section will cover the main bullets relevant for the current project, someone who read this section should be able to understand the main concept of this project and the way it’s going to be tested.


Introduction (Project) - Describe the project in few words (You can also describe the main features and benefits we gain when using STP docs).

Software Review - you should add detailed information regarding the tested software, I suggest few fields that cover this task:

  •  Description – Describe the software and the main functionalities.
  • Benefits – what’s the benefits for user when using this software.
  • Requirements – Describe the requirements that used as development guidelines.
  • Targets – What are the main targets for this software?

Document Structure – Here you should add a list of all sections (Timelines, owners, Testing Types, External tools, environment and more...).

Data Resources – add table of resources this STP based on, this table is very important when other owner/tester wants to check your data (SRS, DDD and more…).

Terminology – explain the term’ used on this document, not all readers can understand the meaning of SRS, DDD and TBD….


To be defined
Software Test Result
Software Test Description

Owners – add list of owners, other users who read this doc should know all employees involved in the testing procedure (Product manager, Developers, testers...).


Omer .S.
DB Manager
Dan .e.
QA Manager

Timelines - you should add the timelines for this project, the following table demonstrate it perfectly:

Estimated Times
Entry Criteria
QA team should prepare list of tests executed by development team, in addition QA team will define set of criteria that must be accomplished before the software can be delivered from Dev. to QA.
Start Date :1/1/2013
End Date : 1/2/2013
Detailed design for How we are going to test the software
Start Date :1/2/2013
End Date : 1/10/2013
Exploratory Testing
After Software delivered the QA team will execute Exploratory testing
Start Date :1/10/2013
End Date : 1/12/2013
Functional Testing
Execution for Functionality testing
Load Testing
Load Testing using External Testing Tools
Bug Verification
Closing all opened bugs
Short sanity before Alpha
Testing results
End Date : 2/1/2013

Section 2: Testing Environments 

This section describes the testing environments used to test the software, this section should be very detailed so everyone who wants to read this section can understand the testing architecture and the limitations we have on tests due to missing Hardware/Software for tests.


Testing Type
Detailed Information
Unit Testing
Developer Testing environment
All unit tests will be performed to check units integration before validating components integration (QA Side)
Integration Testing
Testing execution on QA environment
All tests performed on QA testing environment before starting System and Computability testing
Servers : Win 2012 ,Celera,  NTAP
compatibility Testing
QA testing environment
Different DB : Oracle, SQL ...
Different Web Browsers: Firefox, Explorer…
Alpha Testing
Company production
Installation made after all Exit criteria accomplished on last build

Testing Limitations:
Describe the limitations based on available resources, for example:
  • Load Testing cannot be performed due to lake on Cup’s and Storage limitations.
  • Compatibility testing on Exchange 2013 cannot be executed due to missing license.

Section 3: Testing Tools
This section should cover two important issues:
  • Relevant tools that we are going to use in the testing process.
  • Tools we need for testing and not available on current timelines.

This section is very important because testing tools are decisive part in almost every testing process, in addition this is the place to describe the limitations we have if some testing tools will not be available.

In addition, the project owner should understand how the testing tools can help the QA teams to reduce the risks and expand the testing coverage.

STP (Software Test Planning) – Part 1

The Software Test Planning (STP) is one of the most important documents when designing a software testing process, usually each project contains a single STP that based on information allocated from different resources (Reviews, PM, SRS and more).

STP should (Probably ‘must’ is the correct word…) contain all the relevant information regarding Test procedures for specific software, every STP must provide detailed information regarding the test process (Owners, Timelines, Requirements, budgets, procedures and many more…).

STP can be different per company (you probably get 100 different templates from 100 different companies…). Although, each STP must be based on few mandatory criteria’s that would be examined on the next few paragraphs.

STP – When should we start it..?
STP documentation is a big part on every software testing process, but the question is when should we start to write it..? 

The answer is very simple,
STP become relevant only after few criteria’s accomplished, the following flow will demonstrate this point: 

1.    Client should send a request for a software development based on specific requirements.

2.    After the client requests approved by the company owners (every company should make money…).

3.    Product management team (PM) prepare the software design Documentation (SRS) based on user requirements.

4.    Technical teams (Development, DB...) prepare a detailed Design documentation (DDD) based on SRS requirements.

5.     QA teams have the relevant documentations to start the test plan (STP), this step is dedicate to determine the “What” to test in the Quality procedure.

6.    Based on STP, QA teams prepare the actual test plan (How we are going to test the software).

7.    STR should be available after the testing procedure accomplished.

If you’re not familiar with STD/STR docs, I already started new posts that going to be available in the next few weeks.

STP – Allocating the data
As explained the STP is crucial for your testing process and will be used as bridge for other documentations that based on it. Therefore, the research must be highly effective and relevant resources must be investigated and used. 

The following resources should be used: 
Software Requirements Specification (SRS) – we can declare that SRS is probably the most important resource you should use, in other words we can say that SRS is “The” foundation stone for the entire life cycle (From design and until release).

The SRS Written by the company employs (PM could be great example) and based on the software requirements (usually come directly from the client).

The SRS must be detailed as possible because both development and QA teams will use it (Developers for development and QA for testing documentation (STP/STD...)).

Development– every development process contain unique documentation that handles the develop process, we can use it to understand exactly how the developers develop the software and probably discover high risk areas that could affect the entire testing approach.

Sales/Operations Team – they know the clients and all the week points, they may supply great information regarding main points that may affect the client experience.

Company knowledge base – it’s not something new, but the company KB may supply different point of view or different approach for testing. You can search information on Bug tracking system and learn about issues that may affect the current process or ask experienced manager that already worked on similar project and may add useful information.

The following tools may be used to allocate the data from your resources:
Personal reviews – schedule meetings with every engineer/other employs that relevant, in the reviews you must come with predefined Questions and document all the answers. In my personal experience I cannot start my STP before reviewing all team owners that relevant for my project.


Product manager – provide the “key stones” for the entire project, pay attention to his words…

Database/ Software Engineer – different departments that involved in the development process, here you should find useful info regarding developing issues and weak points that may occur.

Product Manager – understand the priority, timelines, Teams involved and more.

Sales/ Support Personal – learn about your clients and what are they expecting from the end product, investigate older release issues, software recalls and every other detail that may affect the current process.

Team Meetings – from previous example we can see that every project contain multiple teams involved, each team has different project and different agendas, your reasonability is to make sure that all teams synchronized with each other so the info you get is the most relevant and accurate. Furthermore, in team meeting you always find new ideas that may affect the product design and your STP afterwards. 

        Personal Investigation - make your own investigation, to accomplish this task you can use your previous experience and KB, just for example:

  • Investigate your bug tracking system, search for relevant bugs that may reproduce on current version.
  • Search on older documentation for relevant test cases that may be used on current tests.
  • Think about previous projects conclusions (STR’s) and try to understand the failures.

My Presentations