Wednesday, May 9, 2018

Traditional testing vs. agile testing | Supreme Agile

Throughout my professional career, I find myself often asked what is the difference between the Agile and the traditional methodologies? This is especially true when it relates to testing.

To answer the question, let us begin by looking at similarities between Agile testing and testing in traditional software development (SDLC).

Traditional Testing (Waterfall)

The SDLC waterfall model is the most commonly used methodology in the software development industry for those organizations that still use traditional SDLC methodologies. Let's examine it by reviewing its all the phases prior to go-live (i.e. release) and maintenance.

Phase 1 - Requirements Analysis
All requirements for the software are captured in a specific software requirement specification (SRS) document.

Phase 2 – Design
Development teams will learn the requirements captured in the SRS document and then create the system design specifications (System Architecture, Flows, Hardware, Etc.).

Phase 3 – Coding (or commonly known as Build)
The development team will use the system design document and start to develop the software (Data layers, Services, Procedures Etc.). The basic development cycle is to start with the units, modules that will integrate into advanced development phases to create the system.

Phase 4 – Testing
The system that was developed in the coding phase is now delivered to QA teams so they can run "Functional" and "nonfunctional" tests. During this phase, testers will report software bugs that will need to be resolved by the Development Teams.

As you can see, this model is relatively simple. It is clear that testing happens only at the end, just prior to release. Now, if you not familiar with this approach you may get the impression that there is as much time for testing as there is for coding. 

Well, based on my experience this is not the case. The testing 'Entry' and 'Exit' criteria get squished because in almost all projects the coding phase will take longer than it was expected to end, and from the other side, the testing phase needs to end sooner because the customer will push the organization for early delivery. 

Agile Testing

Agile is an iterative and incremental approach. Agile testing is a software testing practice based on the 12 principles and 4 values of the Agile Manifesto. It covers all layers and all types of testing.

Agile testing involved the entire Agile team including DBA's, Developers and testers. Testing in Agile may look similar to what we saw in traditional testing but it’s not! In Agile, testing is NOT a separate phase, but rather the test team will test each increment of coding once its ready (interwoven of all development phases as we saw in the waterfall model) 

An Agile iteration (Aka: Sprint) might be as short as one week, or as long as a month (Two weeks is the classic amount of time that is used by most organizations). During this time, the team will return the same four phases (See waterfall model) to build a working high-quality product and then move to the next piece that needs to be delivered (The entire team will work together simultaneity to meet the story "Definition of Done (DoD)" that includes testing). 

Another important point is that testers are active participants in the entire software development lifecycle (SDLC) in conjunction with cross-functional team members. (I will publish a dedicated article about model 'T'  as the best way to build great Agile teams). These people will contribute all the necessary effort towards building the software as per the customer requests, with less effort, better software design, increased ROI and the delivery of high-quality software. 

Focus on the things that matter

Testing is a mandatory part of any project no matter how big it is. In agile, testing may refer to Epic, Feature and User story (As you will see, the focus is on stories). No matter where you choose to start, the team will need to ensure it has a high-level understanding of it (Similar to what testers will do in traditional testing).

Agile testing will allow the team to focus on small PBI's without the need to come up with a massive test plan or test strategy for testing as is usually done for traditional approaches. The team can focus on creating testing per PBI instead of creating tests on the basis of one SRS document was created by Product Management/Business analysts before anyone even wrote a single line of code.

Working at the PBI level will allow the team to design and execute the tests after they have a precise knowledge of how Dev will implement the code and what is needed to be done to ensure quality. 

Acceptance Testing

The basic definition of 'Acceptance testing' as it published in the ISTQB guide is "formal testing with respect to user needs, requirements, and business processes conducted to determine whether a system satisfies the acceptance criteria and to enable the user, customers or other authorized entity to determine whether or not to accept the system."

Now that we understand the importance of this tests for the success of the project, let’s see some of the major differences between the two test methodologies. In traditional testing, the acceptance tests are made by the customer and not by the team and only after the software has been released (Do you feel the Pressure? The panic? The Risk?).

In agile testing, acceptance tests are made by the team on an iteration basis, the team will follow a DoD that will ensure that the team will follow all criteria to meet the expected quality. So, this makes a Hugh difference, the team can perform the tests during the iteration and prior to delivering the product to the customer which will reduce the risk and pressure.

Adaptive to changes

As you already know, in a traditional testing environment a test team will have all the knowledge they will need at the start of the project once the SRS and HLD docs are ready. In Agile testing this is not the case, every PBI may involve a different type of challenge for such cases A team should be adaptive and solve it based on the tools, knowledge and the resources available (As an Agile team member, you must be able to meet this type of test environment and adapted to whatever the team's needs). 

Documentation is not the main goal

In traditional testing projects, the test team will create an endless test documentation (STP, STD, Etc.) contains test scenarios, test cases and the expected results of each test. Now when you have weeks and sometimes even months to invest in this exhaustive test documentation it may become reasonable, but in Agile testing, this timeframe is narrowed to few days and sometimes even hours.

Now, if you ever worked in a traditional testing project, you already know that this approach has some major disadvantages, such as:
  • Test teams cannot really create effective test plans for weeks and even months of testing while relying on written documentation.
  • The creation of massive test plans will not worth the time spent to write it down (Think that this time was invested in the actual testing process).
  • Writing down comprehensive test documentation will not benefit with the team morale (I never saw a tester that enjoyed to write thousands of tests that may be worthless once the actual testing will start). 

So, is there a test documentation in Agile? Yes, there is. The difference is in the size, complexity and the amount of time that invested by the team to create it (the team will focus on the essence of the tests rather than a comprehensive writing of its details). 

Instead of writing a detailed functional test scenario that will consume a time that the team does not have to spend, team members will conduct their tests based on more advanced testing methods such as Exploratory Testing (ET) and Risk-based Testing (RBT) which will allow the team to run more efficient and advanced testing process. 

Collective responsibility for the quality

There are so many jokes about the difference between testers and developers when it comes to software quality, the origin of these jokes are related to traditional testing. Let's explain it, as you saw, traditional testing has two different phases that separate testing from coding, as a result, each department is responsible to a single aspect of the software (Developers to create the code and testers that will test it).

In agile this separation does not exist. This is because the entire team is responsible for the quality factor (the main focus of the team is to produce quality software in a limited timeframe that maximizes its value to the customer). In some cases, developers will execute manual tests and automated tests. These developers will work with their testers to guarantee the software delivered is of high quality. 

Quick feedback from testing

The most important difference for testers in an Agile project is the fast feedback generated from testing that is just crucial in the overall effort of pushing the project forward. A good Testing feedback allows the team to understand the quality of the code and change it faster than it will ever be in traditional testing projects. 

Test Requirements 

I think you will agree the foundation for any project are the requirements, no matter what the chosen test methodology is. Below I have listed the main differences between Agile testing and traditional testing. 


Traditional Testing

Agile Testing

Who's responsible to write the project requirements?

Product Manager (PM), Business Analysis (BA)

Product owner with the team will write both the "Functional" & "Nun-Functional" requirements

When the requirements are written?

Once the project start (As we saw in the first phase of the waterfall model)

Every day during the project (Planning meetings, Sprint 0, refrainment meetings Etc.)

Ability to absorb changes

Almost impossible once development starts, each change will have a severe effect on Eng. teams

Requirements are changed frequently to allow the team to provide a better product to the customer


Requirements are created for the whole scope of the project

Requirements are created for a partial scope of the project (Usually enough for 3-5 sprints)

Type of writing

Very detailed including test inputs, steps, and expected results.

In the spirit of the used test techniques such as exploratory testing, there is no need to write detailed information.

Keep the code clean

In traditional testing approach, The Test team will report software defects. These defects will be fixed only after the coding phase is done. Therefore, the more bugs discovered will mean more fixes and more risk added to the project. In Agile, defects are fixed as they are raised in the same Sprint. The ability of the team to handle all defects in short iterations will reduce the risks and increase the team's ability to create clear and more resilient code and ultimately better, quality deliverables.

One last comment, in some cases not all bugs will be fixed in the same iteration. It is a common practice that the Product Owner will review (with the Team) opened bugs and will decide which bugs are more important than others. Using this process, the team will handle an only partial number of bugs that will not affect the DoD and the quality of the developed code. 

No comments:

Post a Comment

My Presentations