Monday, September 25, 2017

The Art of Exploratory Testing | David Tzemach

תמונה קשורה


Exploratory testing is probably one of the most efficient and effective testing method available in the world of quality regarding the SDLC methodology, as such, every tester should become familiar with the basics of this method and how can it be used to improve the efficiency and effectiveness of his testing process.

This article will cover most aspects of this testing method including best practices and some great Tips to helping you to maximize the advantages of this testing methodology.

Enjoy!

Author View 


Exploratory testing, is a slightly different compared to all other testing methods, why? Well, there are many differences between Exploratory and other testing methods, starting from the state of mind and continuing to the execution process.

In exploratory testing, the test design, learning, and the actual testing is done at the same time (Parallel), this logic alone is the exact opposite from all other testing methods that separating these activities into specific phases (Learning and Predefined test Design must be defined prior to executing the tests (Serial)).

Now, some of you will probably raise your hand and say that using this “parallel” state of mind is a mistake, because any test project should start with a structured test design and only after all requirements are reviewed, and that’s fine, but really relevant to the old era of testing, where the SDLC was based on traditional methodologies such as waterfall/V-Model Etc.

Now, that more and more companies are performing the transition to Agile methodologies, there is no more room for testers to work with large, heavy test documentation and weeks of planning, to stay relevant, testers must adapt to the new challenges and demands that come from this transition. For them, to be able to do so, they must become familiar with both Exploratory and Risk Based Testing (See it in my Blog), that mainly used by testing teams throughout the agile iterations.  

In addition, I can point some cool aspects of this methodology that can answer the question of why it’s my preferred testing method: 

Freedom 

ET is allowing me to rise above the regular ‘Strains’ attached to a tester when using other approaches that based on a strict test script.

Responsibility

Due to the nature of this methodology; the responsibility of the tester becomes even more crucial to the success of the entire testing process.

Generate New Ideas

In most cases, I use ET prior to creating the STD doc, this short session of exploratory testing will allow me to gain a better understanding of the product and as a result to create a more efficient test design.

Measure the Risks

Exploratory Testing allowing me to gain a great perspective on the risks involved in the testing process, I can easily find the weak points or areas that should get more attention during the testing process.

Creativity 

To be able to achieve the best results from Exploratory testing, the tester must use his creativity and imagination and always think about new ways to manipulate the system.

Adaptability


Not like other testing methods, the ET method demands the tester to be adaptive while performing the ET session; the tester must have the ability to understand how he can use the system outputs that based on his own manipulations.

When should we use Exploratory Testing (ET)?

Exploratory testing can provide a great platform to handle many testing challenges, such as:

There is No/Partial Documentation 

In a perfect world, the test team will receive a detailed, informative SRS that will help them to design the STD that will be used during the testing process, but in many scenarios this is not the case, sometimes the SRS doc will contain partial information that does not allow the test team to gain a deep understanding of the requirements they will need to test.

To narrow this gap, the test team can run a short ET session on the application and based on their experience and knowledge they will most likely find major issues that not defined correctly or efficiently in the SRS doc. No, I know that for some of you it may seem like a bad practice, but believe me that in some situations there is no other way besides diving into the water by doing this session.


Lack of Time 

In some cases, the test teams do not have time to make a full testing process as they wish to, using ET sessions as an alternative, will help the team to run the top Priority tests, Remove critical issues and Reduce the Risks.


Learning new features 

Exploratory testing has a great advantage in this area because it’s allowing the tester to work with the application without any strains that he would have once he using the traditional test scripts. Testing the application without the traditional restrictions will help the tester to use his senses while investigating the different components of the application.


Feedback and Estimations


How many times did you asked by your manager to provide a time estimations on a new feature that you need to test? Well, using ET, you can work with the new application and that will help you to generate a fast feedback about the overall effort that you need in order to run your tests.

Guidelines and Tips


Exploratory testing is a great testing method with many advantages, but in order for testers to exploit the full potential of this method, I created this useful list of guidelines and Tips. 

Exploratory testing and Documentation


Exploratory testing is not a monkey testing! There is a Hugh important for the tester to document the main testing flows during the execution time, the main reason for that is that it will allow him to present the main flows and areas that were tested during the ET session. 

The Documentation generated during the testing session has a Hugh importance because they allow the tester to present his work and remove any unnecessary questions like “What you did in the last 5 hours…?”

Explore with Different Testers

The main advantage of Exploratory testing is that it allows the testers to use an open mind while executing the tests, therefore each testing session can be different from one tester to another. By using different testers, that will run these sessions, we will most likely gain other testing flows and different perspective about the non-functional aspects of the application such as usability, Performance Etc.

Exploratory Testing is Not Everyone Cup of Tea


Although exploratory testing is slightly different from other testing methods, the main idea remains the same (Learning, Executing and documentation), but still some testers are struggling to use it during their testing process, the main reason for that is mainly focused on the ability of the tester to execute tests without having a well-defined STD that specify how each function should be tested (Inputs And the expected results) or documenting  the main test flows during the test execution.


Exploratory Testing as Additional Testing Layer

Exploratory testing can be used to test small features/Function, but for large projects, it must be combined with other traditional testing methods that will allow the testing teams to guarantee that the main risks are removed. 


Exploratory Testing Should Be Made With Control

Although ET provides a solid ground for the tester to use is imagination and skills, it does not mean that the testing session should not be controlled and monitored, just for example: 


Before Starting ET 

Ask your tester to provide information about the main testing flow that he wants to explore, the ideas for testing and the main goals that he wants to achieve. 


While ET in Progress

For any testing session that is set for more than a few Hours, a review should be made in order to understand the efficiency and coverage of the testing session; the review process itself should be quick so it will not distract the tester, Guidelines:
  • See if testing data that used for test cases may be relevant for future tests.
  • Review opened bugs and how it affects future tests.
  • Measure the timelines based on defects found.
  • Review the test flows made by the tester. 
  • Validate the actual testing coverage.
  • Get answers to open questions.
  • Make Risk analysis.
  • Make sure you do not have showstoppers or critical issues that will affect the efficiency of the testing session.

At the End of the ET Session

Similar to any other testing sessions, we need to review the results of the testing effort, during this review we will cover the following topics: 
  • Review some of the main metrics (Defects severity, Defect Clustering Etc.).
  • The remaining Risks and the mitigated once.
  • Issues that affected the testing session.
  • What was tested and what is not.

The Basic Guidelines

Experienced and Non-Experienced Tester

Both experienced and Non-Experienced Testers can make exploratory testing depending on the scale, complexity, and goals of the testing effort. To achieve more from the ET session, we must use testers with a relevant experience (For learning purposes, we can assignee tester that has not any experience but we take this as a calculated risk but for large complex projects we will want an experienced tester that familiar with this testing method).

Experienced In the Tested Application

Based on my experience, you will achieve more from the ET session if the tester has more experienced in the tested application or the technology that used to develop it, Expected Benefits:

  • The tester will know how the application should work and how to find logical defects. 
  • The tester will have the knowledge about the weakest areas of the tested app. 
  • The tester will have the capability to design more effective testing flows. 
  • The tester will have the knowledge about the Risks that may affect the functionality and user experience. 

Script Exploratory is an Option 

Exploratory testing can be made without a preliminary test design, but it can also be executed using specifically created test scripts that will be used as test direction for the tester during the test execution.

Next Steps Based on Findings  

During the ET session, the tester will use different inputs that to create both positive and Negative behaviors that reflected in the outputs generated by the application, a great software tester would use this data to choose the next step in the execution flow in order to increase the efficiency of the tests. 

Live Documentation 

The documentation process is conducted during the testing session, to make it more efficient, you can use screen recorders that will record the testing scenarios without the need to do it manually. Another option is to use a simple notepad that will be used by the tester to document the main testing flows that will be used to analyze the test coverage at the end of the session. 

Choose the Correct Test Environment

A common mistake among many testers is that the ET session should be executed in a very simple test environment because it’s just a short testing session, but just because of that narrow time frame, the lab should be created to simulate real user experience, that will help the tester to run all the major testing flows that cannot be simulated on a basic testing lab.

Keep the Testing Artifacts

During the exploratory testing session, a new testing data (Scripts, Inputs Etc.) are created and can be re-used in future tests (Especially for automations scenarios), make sure you keep this data to save future testing time in regenerating this data. 

All you need to know about Regression testing

Overview


The Problem we have is that the life cycle of a software is a very dynamic. Therefore, every part in code may change or new code may be added. According to this, you should know that every change has risks behind him. The risk occurs when new code modifications may affect other code that already tested.


The Solution for this issue is using “Regression Testing” that provide great solution and coverage that reducing the risks if performed with great design and thinking behind it.



The internet contains many different definitions for this testing type, but the main idea is to make sure that Functionality that already worked and tested by the QA team still works after the new code added. 

Author View:

The main point for this testing type that I hope you all understand at the end of this article is that “Regression Testing is crucial to Re build the confidence that new changes doesn’t affect the great work you did in your original test plan execution.

You must understand that every change in code contain risks, regression testing is what you need to do to reduce the risks and dedicate your time for the new code testing. 

Time to use

Well, based on my overview section you can understand that regression tests executed only after the main functionality is tested and approved, furthermore, the Regression tests based on previous tests that executed while the software originally tested.

When to use:

Well it’s easy, you use regression tests whenever you have risk that the original code that already tested may be  affected, the risk could be arrive from different modifications, Example:
  •           New Feature that added to an application.
  •           Bug Fixes that may cause additional influence
              on nearby code.
  •          Changes in original requirements.
  •          New Requirements added.
  •           New patches or updated.
  •           Multiple builds that based on specific version source.

Regression tests Flow

Regression testing is part of predefined Flow, to make regression tests you should understand and master this flow, the following steps will demonstrate my point when new feature is added:

Flow Steps

Predefined Steps:
  • All tests for a specific code are done (Functionality, System, GUI and more).
  • New Feature added, this feature effect the already tested code.


Regression Flow Start her:

Step 1: Analyze and understand the impact
This step is critical for the entire flow, you cannot create a good Regression tests if you skip it, few question you should answer: 

  • How the feature combined with the current code?
  • Which areas are most affected by the change?
  • How the new feature affect the Non-Usability part for this feature (Performance, Load, stress and more...)?
  • How this change effect predefined tests for the original code?
  • How the change effect the testing data of the original code test?
  • How this change effect the software documentation?
  • How this change effect the current automation flow?
  • How this change effect other components that not directly relevant for the change?


Step 2: Select the relevant testing strategy
After understanding the impact and risks, it’s now the time to select the regression testing strategy and testing guidelines, as you probably know you cannot run tests forever. Therefore, your regression tests are limited in time, Strategy examples:


·      Tests the entire code– This strategy is the best when you have the Time, Money and highly trained automation team that allowing you to run the entire test plan including Old and new tests. Practical? From those criteria’s you can understand that using this strategy is not practical (Tell me if you have such resources and you’re not working in google…).

·       Testing made on Highly Risk areas– this strategy will be used when the testing team identifies the areas that most likely affected from the change and probably contain the bigger risks.

·        Bugs, Bugs and Bugs– this testing strategy will hold all test cases that involving bugs, every case that found bug will be part of the regression testing.

Step 3: Creating the actual test plan
Very easy, after you understand and analyze the change (Step 1), selecting the testing strategy (Step 2) it’s now the time to create the actual test plan.

Step 4: Executing regression this steps
Execution can be accomplished in 3 different ways:

Manual testing – executing regression just like every other STD, a tester or group of testers will executing the test plan based on priorities. This execution type is the less preferred for me, I’m not familiar with testers who like to repeat previous tests every testing cycle and after each modification, it’s also have massive costs for the company(Time,Money , Resources.…).

Automating Testing – My preferred way, all tests running automatically without any help from the tester, you need to remember that once regression tests are build ,this test plan will run multiple times with small modifications , automation is simply the best way to executing them.

Personal view:
Use automation and more automation, don’t run those tests manually.

Automation and Manual testing – Combination of the previous execution types, and may be a great solution because some cases cannot be executed with automation.

Step 4: Reporting
The testing results per regression cycle should be reported and analyzed, according to failures we can add/Remove tests and gain the best efficiency per testing cycle.

Guidelines and suggestions

·        Prioritize (Guidelines) – regression tests may include thousands of test cases, in some situations you simply don’t have the time to execute them all. Therefore, you must prioritize the highly important test cases that without them the Exit criteria will be failed.

To control your prioritization simply add an additional column on your regression document, Example:
Priority 1 – Must be tested
Priority 2 – Not include any Critical/Showstoppers, Risk reduced if executed.
Priority 3 – Only if time allows


Personal View:
For me this step is the one that makes the differences between testers, prioritization must be made only after a tester really understand the test case and how it’ effect the total risk for the tested software (We all know that some testers will mark many cases with priority 1 just to make sure that tests will be executes to cover their a!@#.

·        Documentation (Guidelines) – just like every other testing types,  you must create a dedicated Testing document for regression testing, as you probably understand this Doc will be built from other test cases that already tested in the software testing life cycle.
Please remember that creating such doc should be easy when you really understand the targets you want to achieve, The Hard part is the maintenance that for most testers isn’t relevant after the creation and will never be modified when modifications should be made.

·       Regression per modification (Suggestion) – you’re a QA, part of your job is not to trust the developer that says to you “The change will not affect the code, nothing should be tested”, if you can, run regression tests per change.

·        Functional and Fields tests (Suggestion) – regression tests are simply great test method to get the most efficient coverage for functionality and fields.


·        All Areas that affected from the modifications(Suggestion)  – make sure that all areas that relevant for the change are covered, in addition, make sure that lower risk areas are not the main cases you use (Short sanity tests will be enough).
  • Boundary testing – regression tests are perfect for such tests, it’s also easy to accomplished when using automation (Inputs Vs. expected results) make sure your regression contains such tests for every field that may be affected by the change.

·       Cases that found bugs (Suggestion)   - Select the cases that failed on the original run and verified after they fixed by the R&D team.

·        Duplications (Suggestion) – make sure that every test is unique, regression execution is by default the longest testing type, believe me that duplication is not something you need.

Considerations when building Regression tests

When building a regression test plan there is some main bullets you should remember and follow, please make sure that this once are part of your considerations:


·        Coverage  - Regression tests are used to cover code (New Change + Old code) , you must consider the coverage you want to achieve and the resources you will spend to accomplish it, in some cases the cost are too large.

·        Costs, Costs and Costs – the one question you must answer is this one “How regression tests save time and money…?”  


·        Maintenance – creating great STD for regression is probably the easy part, the hard part is to make great maintenance, and you must make sure that you understand the impact of the maintenance and how hard it will be to do it.

Monday, September 18, 2017

Benefits of User Stories in Scrum | Supreme Agile

תוצאת תמונה עבור ‪stories in agile‬‏

Simplicity

User stories are simple and fast to write, both the customer and the team can describe the exact requirements (Functional and Non-Functional) without using, multiple long descriptions of the task that they want to achieve.

This simplicity will contribute to the process because the customer does not need to know the technical aspects of his requirements, the simplicity in writing user stories will help him to use a basic language to reflect his functional needs.

Written by the customer

User stories based on the customer requirements that knows what he wants to get at the end of each iteration. This allows the scrum team to communicate directly with the customer (or with the product owner that represent him) to get a better understanding of the real needs and what they should deliver.

Easy to Prioritize

Once the user stories added to the product backlog, the product owner with the team has the opportunity to understand which stories are more important to the customer. Following this basic approach, the PO and the team can prioritize the important user stories first; this prioritization will guarantee that the team will deliver the most important user stories first which will lead to happy customers.

Communication

User stories are a great way to increase the communication among the team members, and between the team and the product owner, a good user story will enable the relevant stakeholders to make a real conversation about the requirements and expectations of the customer.

Collaboration

The team will review each story during the sprint "Planning" meeting, this will increase the discussion between the team members (Especially when the need to make the requirements analysis) so they will increase the knowledge prior to implementation.  

All in one

Each user story contains all the relevant information that the team members need to start the implementation (Acceptance criteria, Requirement specification and done criteria) without searching different parts of the data between multiple resources.

Commitment

User stories will allow the team to evaluate each story so they can determine the work effort prior to taking any commitment during a sprint.

Tracking

Remember that each user story contains the work effort that the team needs prior to complete it, this effort reflected in Hours that are reflected in the sprint "Burn-Down" chart that allow the PO/SM/scrum team members to determine the amount of work that completed/remained during a specific sprint.


Sunday, September 10, 2017

What are User Stories in Scrum ? | Supreme Agile

תוצאת תמונה עבור ‪user stories scrum‬‏
Once stakeholders realize that a new development is needed (Software, Features Etc.), the agile process begins with the product owner (PO) that will translate the customer requirements into clear definitions of what the new software will do and what services it provides to its users.

This process is not unique and relevant for both Traditional and Agile methodologies, but the main difference is the “Way” and “Process” used by the product owner to translate these requirements, in traditional methodologies such as “Waterfall” the product owner will create a large heavy SRS/SPEC with lengthy requirements and specifications. This process of creating large heavy docs is not suitable for Agile; due to the nature of this methodology, the Product owner should use a lightweight approach of translating the customer requirements into short and clear descriptions of the pieces that needed in order to satisfy and provide value to the client.

Therefore, if in traditional model the product owner will use a SRS/Spec to define the customer requirements, in agile the product owner will use work items captured in the form of “User Stories”. A user story is an alternative way to express a software requirement in a very readable and structured form (based on the client perspective), that will add value to the customer once it is accomplished by the team.

The basic and very “Theoretical” format of a user story is:

Title: <Short and informative name of the user story>
As a <User>
I want to <Description of the action>
So that <I will get this benefit>

In addition to this basic format, user stories should also include the following fields:


Artifact
Description
Creator
The creator of the user story can be any team member.
Definition of Done (DOD)
Checklist that determines the conditions that allow the team to mark a story as completed.
Acceptance Criteria
List of conditions that needed to accomplish prior to the team be able to start this story.
Story ID
The story ID should be unique to differentiate it from other user stories.
Priority
The PO in a way that will increase the value to the customer should prioritize each story.
Effort Estimation
The estimate time (Story Points/Hours) that the team will need in order to deliver the user story.
Owner
Determine the owner of the story.
Value/Business Value
The value of the story for the customer/organization.

User stories and Area of responsibilities


Product owner
Scrum Team
Who can Write stories?
The product owner Is the one that has the main responsibility to gather and manage user stories. However, the scrum team and other stakeholders also have the power to create stories of their own

Type of stories
Stories that based on Organization/Customer Requirements and specification

Mostly technical stories that are not add a direct value to the customer

Who is the Owner of the story?
The PO is the owner of the stories prior to the commitment of the scrum team
Take the ownership during the sprint commitments
Who should Maintain the backlog stories?
The product owner should be the one that is responsible to manage the product backlog, but in order for him to succeed, he must involve the scrum, team as well, this collaboration take part throughout the backlog “refinement” meetings.

Who should Prioritize the story?
The product owner should determine which stories will developed first in order to increase ROI and value for the organization/customer
The team cannot determine the priority, but they still have a decisive part when it comes to technical stories that should developed in order for them to be able to accomplish the functional stories.
Who Will Execute the story?
Not relevant
The team is responsible to make the implementation

Tuesday, September 5, 2017

Definition of DONE (DoD) in Scrum | Supreme Agile

תוצאת תמונה עבור ‪definition of done agile‬‏





In order for a scrum team to be able to understand when a user story from the Sprint Backlog can be marked as "Done", the team will use a technique called "DoD" that is used to determine a set of Requirements, conditions and quality factors of necessary activities that must be completed to ensure that the team will deliver the user story in the highest quality to add value to the customer.

In addition, a good DoD, will help the team members to get a clear understanding of what is expected from them per story, which will reduce the uncertainty and will allow them to focus on the things that matter. Some great examples for that is that DoD is also used to determine if all tasks are written during sprint planning and also as a checklist at the end of the sprint to verify whether all activities were completed by the team 


DoD at various Scrum levels

We can use DoD on different project levels, the simplicity and clarification that a good DoD provides for the team can be added to many other scrum process and Artifacts, Examples:

  1. Definition of Done for Sprint (Collection of goals that the team must develop within the sprint time-frame).
  2. Definition of Done for product release (It’s the PO decision to decide on the criteria of what it takes to release the product).
  3. Definition of Done for a user story (Commitment to the product owner).
  4. Definition of Done for a Task (Tests are passed in 100%, Code is implemented, Code review was made Etc.).

What is a typical DoD will look like?

A DoD may look different between one scrum team or another, but the typical DoD would be similar to:

  1. An HLD will be written and reviewed by the team.
  2. The code will be implemented and code review is made.
  3. The new code should be covered with tests at all application levels (Unit, component, Integration, and System).
  4. The code should be integrated into version 1.0.
  5. A working service should be presented in the review meeting.

Expected Benefits of using DoD


  1. The Definition of Done will help the team to understand what is expected from them once they take commitment on a user story.
  2. The Definition of Done will help the team to make a better time estimation because they can understand the work effort they need to put in order to complete the DoD.
  3. The Definition of Done will help the team to gain a shared understanding to ensure transparency.

Saturday, September 2, 2017

Scrum Master in scrum team | Supreme Agile

תוצאת תמונה עבור ‪agile planning‬‏

Prior to each iteration planning meeting, the product backlog user stories that likely to be delivered in the next iteration have to be prepared by both the team and the product owner, the preparation process includes many different activities and demands the collaboration of the entire Scrum team.

Choosing the Iteration Goal

The first and most important thing in the preparation process is to define the iteration goal that will determine the desired results we want to achieve at the end of the iteration. Setting the iteration goal will allow the scrum team to focus on their commitments the amount of work needed to be done in order to succeed to deliver the desired outcome of the iteration.

Although the team has the responsibility to deliver the technical aspects throughout the iteration, the iteration goal is set by the Product Owner that need to set the goal to the satisfaction of the customer.

Now, although the product owner is responsible to determine the iteration goal, it is highly recommended that he will involve the team in formulating the goal, involving the team will lead to some major benefits, such as: 
  1. Once the team is involved, they will help the PO to set realistic goals that based on the team experience and knowledge in the technical domain.
  2. A team that participates in formulating the iteration goal will ensure clarity.
  3. The involvement of the team will help them to set goals that will provide them some room to maneuver throughout the iteration.
  4. Setting the iteration goal will make it easier to communicate stakeholders what the team is expected to deliver at the end of the iteration.
  5. The team will most likely be more collaborative because they are working toward a common goal.

Backlog Refinement

Once we set the iteration goal, we need to prepare the relevant user stories for the upcoming iteration, the team will accomplish this task using the backlog refinement meeting.

The refinement meeting is an essential part of any planning effort in scrum, teams hold a short backlog refinement meeting prior to the planning meeting (30-60 minutes), during this meeting the team and the product owner will collaborate and deciding which user stories to complete in the upcoming iteration.

During the refinement meeting, the team and the product owner will ensure that the backlog is healthy and prepared for the upcoming planning meeting, A healthy backlog: 

  • Contain a list of informative, user stories that will allow the team to understand what is expected of them, each story should include the requirements, Acceptance, and Definition of Done criteria.
  • User stories are prioritized with the most important items are listed at the top.
  • It's clear so every relevant stakeholder can understand it.
  • Large user stories are broken down into smaller stories so they can be completed in a single iteration.

The number of user stories that the team should prepare for this meeting depends on the velocity and the available capacity of the team in the upcoming iteration. The higher the team velocity, the more user stories should be refined during the meeting (In addition to a few extra stories to give the team some ability to make modifications throughout the iteration).

Ensuring Clarity, Testability, and Size

Once the user stories are chosen to the upcoming iteration, we must ensure that they are Clear and testable so the team can take real commitments during the planning meeting.

Clarity

We can say that a user story is clear only in the case that the entire scrum team members have a common understanding of its areas including the requirements, Acceptance, and Definition of done criteria.

Testability

And what about the user story "Testability"? A user story is testable only if there is a real and effective way to determine whether the requested requirement is fulfilled within the iteration in which it is supposed to be implemented.  

Size

The size of a user story should be small enough so the team can complete it in a single iteration, the team can determine if a user story can be delivered in a single iteration based on the Definition of Done determined for the story and its dependents in other stories.

My Presentations