Monday, September 25, 2017

The Art of Exploratory Testing | David Tzemach

תמונה קשורה


Exploratory testing is probably one of the most efficient and effective testing method available in the world of quality regarding the SDLC methodology, as such, every tester should become familiar with the basics of this method and how can it be used to improve the efficiency and effectiveness of his testing process.

This article will cover most aspects of this testing method including best practices and some great Tips to helping you to maximize the advantages of this testing methodology.

Enjoy!

Author View 


Exploratory testing, is a slightly different compared to all other testing methods, why? Well, there are many differences between Exploratory and other testing methods, starting from the state of mind and continuing to the execution process.

In exploratory testing, the test design, learning, and the actual testing is done at the same time (Parallel), this logic alone is the exact opposite from all other testing methods that separating these activities into specific phases (Learning and Predefined test Design must be defined prior to executing the tests (Serial)).

Now, some of you will probably raise your hand and say that using this “parallel” state of mind is a mistake, because any test project should start with a structured test design and only after all requirements are reviewed, and that’s fine, but really relevant to the old era of testing, where the SDLC was based on traditional methodologies such as waterfall/V-Model Etc.

Now, that more and more companies are performing the transition to Agile methodologies, there is no more room for testers to work with large, heavy test documentation and weeks of planning, to stay relevant, testers must adapt to the new challenges and demands that come from this transition. For them, to be able to do so, they must become familiar with both Exploratory and Risk Based Testing (See it in my Blog), that mainly used by testing teams throughout the agile iterations.  

In addition, I can point some cool aspects of this methodology that can answer the question of why it’s my preferred testing method: 

Freedom 

ET is allowing me to rise above the regular ‘Strains’ attached to a tester when using other approaches that based on a strict test script.

Responsibility

Due to the nature of this methodology; the responsibility of the tester becomes even more crucial to the success of the entire testing process.

Generate New Ideas

In most cases, I use ET prior to creating the STD doc, this short session of exploratory testing will allow me to gain a better understanding of the product and as a result to create a more efficient test design.

Measure the Risks

Exploratory Testing allowing me to gain a great perspective on the risks involved in the testing process, I can easily find the weak points or areas that should get more attention during the testing process.

Creativity 

To be able to achieve the best results from Exploratory testing, the tester must use his creativity and imagination and always think about new ways to manipulate the system.

Adaptability


Not like other testing methods, the ET method demands the tester to be adaptive while performing the ET session; the tester must have the ability to understand how he can use the system outputs that based on his own manipulations.

When should we use Exploratory Testing (ET)?

Exploratory testing can provide a great platform to handle many testing challenges, such as:

There is No/Partial Documentation 

In a perfect world, the test team will receive a detailed, informative SRS that will help them to design the STD that will be used during the testing process, but in many scenarios this is not the case, sometimes the SRS doc will contain partial information that does not allow the test team to gain a deep understanding of the requirements they will need to test.

To narrow this gap, the test team can run a short ET session on the application and based on their experience and knowledge they will most likely find major issues that not defined correctly or efficiently in the SRS doc. No, I know that for some of you it may seem like a bad practice, but believe me that in some situations there is no other way besides diving into the water by doing this session.


Lack of Time 

In some cases, the test teams do not have time to make a full testing process as they wish to, using ET sessions as an alternative, will help the team to run the top Priority tests, Remove critical issues and Reduce the Risks.


Learning new features 

Exploratory testing has a great advantage in this area because it’s allowing the tester to work with the application without any strains that he would have once he using the traditional test scripts. Testing the application without the traditional restrictions will help the tester to use his senses while investigating the different components of the application.


Feedback and Estimations


How many times did you asked by your manager to provide a time estimations on a new feature that you need to test? Well, using ET, you can work with the new application and that will help you to generate a fast feedback about the overall effort that you need in order to run your tests.

Guidelines and Tips


Exploratory testing is a great testing method with many advantages, but in order for testers to exploit the full potential of this method, I created this useful list of guidelines and Tips. 

Exploratory testing and Documentation


Exploratory testing is not a monkey testing! There is a Hugh important for the tester to document the main testing flows during the execution time, the main reason for that is that it will allow him to present the main flows and areas that were tested during the ET session. 

The Documentation generated during the testing session has a Hugh importance because they allow the tester to present his work and remove any unnecessary questions like “What you did in the last 5 hours…?”

Explore with Different Testers

The main advantage of Exploratory testing is that it allows the testers to use an open mind while executing the tests, therefore each testing session can be different from one tester to another. By using different testers, that will run these sessions, we will most likely gain other testing flows and different perspective about the non-functional aspects of the application such as usability, Performance Etc.

Exploratory Testing is Not Everyone Cup of Tea


Although exploratory testing is slightly different from other testing methods, the main idea remains the same (Learning, Executing and documentation), but still some testers are struggling to use it during their testing process, the main reason for that is mainly focused on the ability of the tester to execute tests without having a well-defined STD that specify how each function should be tested (Inputs And the expected results) or documenting  the main test flows during the test execution.


Exploratory Testing as Additional Testing Layer

Exploratory testing can be used to test small features/Function, but for large projects, it must be combined with other traditional testing methods that will allow the testing teams to guarantee that the main risks are removed. 


Exploratory Testing Should Be Made With Control

Although ET provides a solid ground for the tester to use is imagination and skills, it does not mean that the testing session should not be controlled and monitored, just for example: 


Before Starting ET 

Ask your tester to provide information about the main testing flow that he wants to explore, the ideas for testing and the main goals that he wants to achieve. 


While ET in Progress

For any testing session that is set for more than a few Hours, a review should be made in order to understand the efficiency and coverage of the testing session; the review process itself should be quick so it will not distract the tester, Guidelines:
  • See if testing data that used for test cases may be relevant for future tests.
  • Review opened bugs and how it affects future tests.
  • Measure the timelines based on defects found.
  • Review the test flows made by the tester. 
  • Validate the actual testing coverage.
  • Get answers to open questions.
  • Make Risk analysis.
  • Make sure you do not have showstoppers or critical issues that will affect the efficiency of the testing session.

At the End of the ET Session

Similar to any other testing sessions, we need to review the results of the testing effort, during this review we will cover the following topics: 
  • Review some of the main metrics (Defects severity, Defect Clustering Etc.).
  • The remaining Risks and the mitigated once.
  • Issues that affected the testing session.
  • What was tested and what is not.

The Basic Guidelines

Experienced and Non-Experienced Tester

Both experienced and Non-Experienced Testers can make exploratory testing depending on the scale, complexity, and goals of the testing effort. To achieve more from the ET session, we must use testers with a relevant experience (For learning purposes, we can assignee tester that has not any experience but we take this as a calculated risk but for large complex projects we will want an experienced tester that familiar with this testing method).

Experienced In the Tested Application

Based on my experience, you will achieve more from the ET session if the tester has more experienced in the tested application or the technology that used to develop it, Expected Benefits:

  • The tester will know how the application should work and how to find logical defects. 
  • The tester will have the knowledge about the weakest areas of the tested app. 
  • The tester will have the capability to design more effective testing flows. 
  • The tester will have the knowledge about the Risks that may affect the functionality and user experience. 

Script Exploratory is an Option 

Exploratory testing can be made without a preliminary test design, but it can also be executed using specifically created test scripts that will be used as test direction for the tester during the test execution.

Next Steps Based on Findings  

During the ET session, the tester will use different inputs that to create both positive and Negative behaviors that reflected in the outputs generated by the application, a great software tester would use this data to choose the next step in the execution flow in order to increase the efficiency of the tests. 

Live Documentation 

The documentation process is conducted during the testing session, to make it more efficient, you can use screen recorders that will record the testing scenarios without the need to do it manually. Another option is to use a simple notepad that will be used by the tester to document the main testing flows that will be used to analyze the test coverage at the end of the session. 

Choose the Correct Test Environment

A common mistake among many testers is that the ET session should be executed in a very simple test environment because it’s just a short testing session, but just because of that narrow time frame, the lab should be created to simulate real user experience, that will help the tester to run all the major testing flows that cannot be simulated on a basic testing lab.

Keep the Testing Artifacts

During the exploratory testing session, a new testing data (Scripts, Inputs Etc.) are created and can be re-used in future tests (Especially for automations scenarios), make sure you keep this data to save future testing time in regenerating this data. 

1 comment:

My Presentations