Sunday, July 8, 2018

Why automate tests in an agile environment | Supreme Agile

Agile teams will fail if they don’t use frameworks to automate different aspects of the development, such as tests, builds, test environments, and so on. To understand why a lack of automation causes failure, we first need to know that the main focus of all agile teams is to successfully deliver real value to the customer, which can be translated into delivering working software at the end of the sprint (in other words, short releases in a fast-changing environment).

To achieve this goal, an agile team should be able to test their deliverables often, quickly and with great coverage. In this article, I will review some of the most common reasons that agile teams have to achieve this important goal and the challenges they face.

Why automate?

There are multiple reasons why agile teams will want to automate their tests besides the knowledge that everyone else is doing it, including the following:

Manual testing takes too long

This is the most basic reason for an agile team to automate their tests. Manual testing and long regressions simply take too long to complete in an environment that does not provide the same testing time as traditional software development methodologies (such as the waterfall model).

As an application gets bigger and more complex over time, the test matrix grows longer and longer, and the team will not have time during the sprint to manually complete the regression tests needed to guarantee they meet the highest quality standards.

Running a full regression suite during a sprint is just an incorrect practice in an agile environment. You simply cannot do it with manual execution. If your current practices do not contain any automation coverage, don’t let that stop you from starting now.

Manual testing leads to technical debt  

If you execute your regression testing manually, you already know that it takes a huge amount of time that you simply do not have in an average of two weeks of work. In addition, it is common practice for agile teams to run regressions every day, every iteration (which does not make any sense to do manually).

If your regression tests are executed manually, it will not allow the team to keep pace with coding. Then programmers have to free up their time to help with the testing, which leads to technical debt and increases frustration among the team.

Automation techniques improve team knowledge

Automation can allow the team to get a better understanding of the product architecture. Using advanced techniques such as Test-Driven Development (TDD), where code tests are written prior to the actual code, allows programmers to gain a deeper understanding of the requirements, design, and architecture of the product.

Automated tests provide fast, early and frequent feedback

If there is no change in the automated test script, we can expect it to pass until the functionality of the application is changed. When a programmer modifies the application code, he also needs to maintain the automated tests to accommodate them.

If the programmer changed the automated script to support the code changes, and automated tests still fail unexpectedly, a regression bug may have been produced by the code modification. This is a great indicator of the efficiency of the automated test design.

Another thing we need to know is that when the team runs an automated suite of tests for all new and modified code that is checked in, this ensures that regression defects will be caught at an early stage. Bugs caught early are cheaper to fix and will not add major risk in advanced phases of the development process.

Now, what does it mean to receive quick feedback from the automated tests? The main thing here is that once the defect is found early (a matter of minutes to hours from the time that the programmer adds the code), the change is still fresh in the programmer’s mind and thus will be easier to troubleshoot than if it was found days later, during the manual testing process.

Automation as a key factor for continual stable builds  

Due to the nature of agile development, it is hugely important that the team not spend their already limited time on unwanted problems such as partial or corrupted builds that will prevent the team from finishing their work.

Partial or corrupted builds are one of the top time consumers during agile development. There is no way that the team will succeed in delivering their commitments without having an automated CI/CD system that will allow the team to have continual stable builds throughout the iteration.

Automation will help free up the team’s time

The creation of test automation takes time, and of course, there is more time spent on maintenance, but once the team succeeds in creating the greater portion of this system, it will reduce the time they used to spend on executing manual test scripts, running numerous repeatable tests and endless manual regression cycles.

Now that the team has reliable automation that can replace the manual execution of regression cycles, the team will have the energy and time to focus on new test scenarios and learning more about the product and how it works, which increases the overall quality of the team deliverables.

Automated regression tests as a safety net

Knowing that the code has sufficient test coverage by automated regression tests gives the team a great amount of confidence while modifying or adding new code. This confidence is not a small thing in the agile development process where new code is checked-in and tested on a daily basis.

Automated regression tests will provide the team with a safety net that will help them find unexpected defects caused by the newly added or modified code within a matter of minutes (for unit, component, and integration-level tests) or hours (if at a higher functional level, such as system or end-to-end tests).

Now, if the team fails to invest their time in building a solid automated suite of tests at all levels of the application (unit, component, integration, and system) to act as a safety net, the programmers will start to treat the testers themselves as safety nets, which will lead to "mini" waterfalls during the development process.

Teams that have good coverage from automated regression tests can work in a more reliable environment that will allow them to write their code and add it to the main branch fearlessly because they do not need to wonder how this new code will break the build and what hidden defects may arise. The tests will tell them in a very short time frame whether or not they broke anything. 

Manual processes are more vulnerable to human error

Manual processes and manual testing are more vulnerable to human error. That's just a simple fact that we can explain by saying that manual testing is a repetitive process that gets very boring for a team member who needs to follow the same testing scripts over and over again.

And when people get bored, stupid mistakes are more likely to happen, obvious bugs are overlooked, testing scripts are not executed as they were the first time, and as we know, people will cut corners to complete work when facing tight deadlines.

The simple solution is to automate all tests that contribute to the team ROI from sprint to sprint. The direct result is the reduction of human error, risk mitigation, and creating a constant development process that will increase the efficiency of the team.

I already mentioned the word consistency earlier, which is key in the agile development process. Consistency is achieved by the team once they reduce the manual processes and automate processes (testing included), which will reduce the possibility of human error because each process and test is done exactly the same way over and over again. Automated frameworks will not cut corners to achieve deadlines.

Thanks to Tally Helfgott for proofreading :) 

Linkedin Profile

Tuesday, June 26, 2018

What testers should perform to stay relevant in agile teams | Supreme Agile

Let's put it on the table, traditional software testers must step up if they want to remain relevant as testers in agile teams. You cannot run from it, agile is here and most probably will continue to be the leading SDLC methodology that will lead the industry in the upcoming years.

Testers must step up and add more skills to their arsenal to meet the new challenging environment of an agile team. Testing in an agile team has much more than what testers have used to have in traditional testing. It's no longer about working in one bug QA team that testing features once development phase is done.

Agile testing means more communication with different stakeholders (External/Internal), Faster testing with more advanced test methods (Exploratory and Risk-based testing are the classic once) and many other challenges that demand testers to develop themselves to really be able to contribute to the team and stay relevant.

The classic scenario

It usually starts when the organization understands the Hugh benefits that agile has to offer and decide to move forward and make the agile transition in the organization. Once the announcement arrives at the different departments, many questions are a raised and uncertainty becomes the employees worst enemy.

Now, if you’re a tester in a testing team the uncertainty becomes even hire and some of the most classic questions are thrown into the air:
  • How this transition will affect the testing team?
  • What should we do in the new agile teams?
  • How can testers and developers will work in one small team?
  • Who is responsible to ensure that the quality will not affect? 
This is just a narrow list of questions that I see in almost any agile transition, and we can understand why moving from traditional testing to agile testing is not obvious to testers that used to own the responsibility on testing the product that now spreads to the all agile team.

The expectations from testers on an agile team

So now it’s a fact, your organization started the transition to agile and as a tester you have a great amount of uncertainty of what does it really means to be tester in agile team, in the following paragraphs, I will review some of the mandatory tasks that any tester is expected to perform as part of the team.

Collaborate with other team members

Collaboration is one of the key principles of agile, as such, we can expect to see it from any team member and testers are not different. Testers should have the personal capabilities to communicate with other team members inside the team and also with external stockholders that are part of the project.

  • Help other team members to write tests on all level of the application (Starting from unit, component and integration tests that will make the testing process more efficient and reduce the manual testing effort).
  • Conduct test review, create test designs and reduce duplicate tests on the different testing layers.
  • Work together with the team developers to report on design issues that may raise major quality issues. 
  • Help other team members to complete a user story that hasn't progress as planned (It can be testing, writing code, and set-up the test environment). 

Help with Testability

Testability is one of the most important items that testers are expected to contribute to an agile team. Testability will help the team to increase the software quality but no less, it also helps to improve the discipline and communications.

Testability is a major part of any user story that the team will want to deliver and known as one of the five pars of the commonly used "INVEST" model for writing user stories (Each PBI should be independent, negotiable, valuable, estimable and testable).

Testers must be involved in all parts of this model but especially on the testability side, they must help the team to understand how and what they need to be able to test this story, which will increase the contribution of the other team members to the testing effort.

Another aspect of testability is that testers are now working closely with the team developers to ensure and advice on all testability aspects. As part of this discussion, the team will decide the test design, strategy (Automate/Manual) and who execute which tests.

learning is not an option

I think that it not really matters what you do in the software industry or in any other job, but just to be on the safe side I will say that for any member in agile teams and especially for testers learning is not an option it must do. Testers have just too many areas that they must contumely improve at, including testing methods, tools, coding skills, technologies, agile itself and to keep up with the production they are testing. 

Take an active part in defining "Done"

This is just a classic example of how the world of testers has changed when moving to agile teams. In traditional testing, testers had to wait for some time to the completion of the first phase (Requirements Analysis) before they even can start thinking on the test plan, Strategy and ensure all requirements are covered (It's just easier when you get one SRS document containing all the application requirements).

Now, once testers are integrated into agile team that follow the basic values and principles of the agile manifesto, any team members (including testers) are part of the all process of defining user stories, adding them to the project backlog and helping the product owner (the business owner of the product) and the rest of the team to define the relevant criteria that must be fulfilled prior the team can mark user story as "Done."

Definition of done from testers perspective is not so different from any other testing project, if they have the most experience and knowledge among all team members in this medium, they will need to ensure that the DoD will include all quality aspects that guarantee that the user story is delivered at the highest quality

From a quality perspective, a DoD of a user story may include the following:
  • All functional and nonfunctional tests with priority X, Y are executed and approved.
  • All bugs are fixed and verified (Product owner approval is needed for all bugs that are differed in future sprints).
  • All tests are automated to reduce future technical debt.
  • All major flows of the software are tested on production environment
  • The code is covered with 90%+ unit tests.
  • All major risks are mitigated with corresponding tests  

Estimate work

Same as other team members, testers will have to have the skills that should be allowing them to estimate the scope and size of the testing effort of each user story that the team will need to deliver in an upcoming sprint.

The testing effort should be estimated by the testers and the other team members in a way that the team will agree, this is crucial aspect of any estimation in agile team because it is part of the overall estimation of the size of the user story which also includes different activities such as coding, design and sometimes even research.

Provide fast feedback 

One of the most important advantages of agile software development is being able to deliver in short iterations that will allow the organization to face the increased challenges of the software industry. to be able to work in an environment of quick releases, the team must have the ability to get quick feedback on each one of their deliverables and make the necessary changes accordingly.

Again although the entire team is responsible for the quality of the product, in most cases testers will be the one that will deliver the quick feedback about the quality of the written code, testers must have the ability to  test fast and to determine the real health of the application (As fast as defects are reported, the faster the fix can be made).

Testers can share their feedback with the team while using different forms such as:
  • Bug reports. 
  • Code and Test reviews.
  • quick sessions of exploratory testing. 
  • Design and monitor quality KPI's.
  • Pair Programming. 

Fail as long as you learn

One of the biggest advantages in agile teams is that you can fail and that's just fine as long as you use it as another step for continued growth for both the individual and the rest of the team. In traditional development, the room for mistakes Is almost not exists due to the nature of the separate phases of the traditional SDLC that increase the impact of each mistake made by the involved engineers. 

Help the team to automate

When it comes to automation, Agile testing is not different from traditional testing. Automation has a major part in the team able to run a more rebuts and efficient test process while keeping future technical debt to the minimum.

In agile development environment that runs on short iterations, automation is the main option to test all the small increments functionalities that added per spring, these small changes that added per sprint will not allow the team to run the same manual tests over and over again (The software is continually changing) and regression test will become a Hugh risk to the team able to deliver in time.

The only way that an agile team can confront this issue is by using automated frameworks on all layers of the application including unit, Component and system tests and do it as much as possible (Manual tests should reduce to the minimum).

As a tester in the team, you must have the capability to write these types of tests and to contribute to the overall testing effort. That does not mean that the tester will write the automated frameworks (Again, depends on his coding skills) but he most defiantly assists in writing the test scenarios, writing them in code (It is more than accepted that the team developers will assist and educate their testers to do it).

Due to the importance of this item, I will add a real example from one of my consultant activities, and the story goes like this; I once invited to organization with more than 40 working agile teams, 35 from these teams were able to deliver on time but the remaining 5 continually fail to do so. After a week that I was reviewed how these teams have worked, I found many failures in different aspects including many HR issues among the members of the team. But the one thing that was really critical is that in all of these teams there were only two testers on a group of 10 engineers.

The bigger problem was that these testers were expected to write and execute tests without any real knowledge in automated (in-house) framework and as a result, all tests were executed manually and in the good case automated by the team developers.

To overcome this problem, I used this situation as a leverage to resolve many other HR issues that were affecting the team; First, we raised this issue in retrospective and let the testers to share their technical issues in addition to their feeling of lack of help of the rest of the team members.

Second, we let the team decide how they can help their testers to become more professional in the company automated frameworks (Per sprint, a new user story was added for this purpose, to provide the team the necessary time to help their testers).

The result of these two simple items was the trigger to major success in future sprints, after four sprints of investment in the team testers, they gain the respect of other team members, the team started to meet the stories DoD and from delivery of 40% they started to deliver in almost 90% per sprint.

  • As a tester, you will help the team to search and acquire the most suitable testing tools.
  • As a tester, you will help the team to minimize the execution of manual tests.
  • As a tester, you will help the team to reduce regression tests and technical debt.
  • As a tester, you will help the team to create different automated metrics. 
  • As a tester, you will help the team to write, execute and analyze the results of the automated tests.
  • As a tester, you will help the team to decide which tests should be automated (per testing layer).

Help the team to determine different metrics

There is one quote about metrics that I love to use in my teams, the quote is "if you cannot measure it you cannot improve it – William Thomson, Lord Kelvin". This short quote can explain why metrics are so important to any project and especially to agile projects that based on continues improvement of the process, Quality and the team itself.

In traditional testing, the metrics were very important from an organization perspective, such as the number of reported bugs, Number of executed tests etc. Now in agile, we are looking at a new world of metrics that the team must track as part of an agile working environment.

The classic metrics:
  • Sprint Burndown.
  • Release Backlog.
  • Velocity.
  • The frequency of changes once the sprint has started.
  • Team happiness 

Open and Verify defects

It's the bread and butter of any tester to report on new defects and verify them once the bug was fixed. Now, although it doesn't sound new, there are some major differences between traditional and agile testing I summarized in this table:

Traditional Testing
Agile Testing
Fix time
defect fixes might be made by the dev department days, weeks and even months ago
Bugs are fixed during the sprint (Max of 4 weeks, Min of 1 week)
Similar to bug fixes, verifications are made department days, weeks and even months after the fix was made
Verifications are made based on the PBI definition of done (So almost all bugs should be verified within the sprint)

In some cases, defects are moved to a future iteration.
Usually by the project manager/Dev Manager
Prioritization is made by the product owner and the team
Build for verifications
Verifications are made on formal builds that are created every few days.
With a good CI system, builds are available within the hour and there is no need to wait for a formal build

Be ready to handle the frequent changes

One of the biggest differences between agile and traditional testing is the frequency of the changes that the development team and especially their testers should face during the sprint. if you remember the 4 agile values than you know that it’s something that is expected from the team "Welcome changing requirements, even late in development. Agile processes harness change for the customer's competitive advantage."   

In traditional testing, the room for changes in the testing phase are very narrowed and almost does not exists (Each disruption can severely affect the project and jeopardize the whole project) which allow the testers to work on their test strategy without the need to change it once they start to test.

As a tester in an agile team, you must be able to work in an environment that will arise many disruptions and new challenges during development that arrive from new customer requirements, changes in prioritization and many other streams that will increase the risks but without affecting the whole project (Remember that we are talking about short iterations and contuse refrainment of the product backlog that based on the exact demands of the customer). 

Thursday, June 21, 2018

The barriers to successful agile adoption by software Testers | Supreme Agile

I think that almost any major changes in life face barriers to success and in an agile transition, the story is not different. Once the organization decides to start an agile transition, he expected to face multiple barriers that arrive from many areas and especially from the organizational culture.

Let us examine the organizational culture from the eyes of the QA teams. Think about a QA team that has already established a strong organizational culture about quality, this team is now committed to the culture, which makes it extremely hard for them to drop it for the new culture that is a mandatory part from any transition to agile.

In this article, I will focus on some of the most common barriers to adoption of agile development frameworks that you can expect to see in your testers and QA teams. 

The uncertainty of their new role as testers in agile teams

There are many differences between agile and other traditional software development, the role of testers in agile teams is just one of them but yet a very important one. Testers that integrated into agile teams must have the preliminary knowledge of what are the boundaries, expectations, and responsibilities that they will need to follow in their new role. 

This is the only way to allow those testers to reduce the amount of uncertainty prior to the start of the agile transition, a tester that will not gain this important information will most likely have much more difficulties to perform at its highest performance and to be a productive team member.  

It is not just about testing anymore

Agile teams built from a set of individuals that work together to achieve a common goal. Each team member will have to use is knowledge, experience, and technical skills to help the team with the many challenges that involved in the environment of agile development.

To become really valuable team members, testers will have to take more responsibilities and roles than what they used to have in the old traditional environment. Besides the obvious activities that involve in testing, testers will have to understand the true needs of the team and take an active part in resolving them, just for example:
  • Work closely with the analysts and developers right from the very beginning of the project.
  • Help the product owner to design and determine the acceptance criteria.
  • Help the team to move forward to complete the iteration goal.
  • Interact with the product owner to clarify the business requirements.
  • Help the team to estimate stories, define ‘Done’ and assess testability. 

Relying on negative past experience

Hold your horses….past experience is critical for anyone and especially for test engineers when I analyze common pitfalls of integrating testers into agile teams, I can see one common issue that repeats numerous times and that’s software testers that have been through changes that did not succeed.

As a result, from this bad experience, testers are kept focused on their previous unsuccessful patterns and start the agile transition with a negative approach that will defiantly not help them or their new team members. Questions like “Why should I do it again?”, “I already been in this process and saw how it fail” are just classic examples and big red warning signs for this issue. 

Loss of Identity

I can write thousands of words about this topic alone, but for this article let us keep it simple. In traditional testing, test teams and each individual tester are working in an independent group with known barriers, responsibilities, and goals. 

Now that they need to move to small agile teams that built from more developers than testers, this change alone is enough to trigger many fears from the tester's side, for example: 
  • Fear of losing their ability to affect the quality process.
  • Fear of losing their job during the agile transition.
  • Fear of losing their ability to affect the prioritization of quality items.
  • Fear of being less productive in a continually changed environment.
  • Fear that they do not have the relevant skills to work in agile teams.
  • Fear that they will not get the support they will need to do their work. 
  • Fear of losing their QA identity (Now that they need to invest time in many other activities).

Bad Attitude

Is there really need to explain why bad attitude can be devastated for software testers that now need to work together with their associated developers?  By using the wrong attitude, testers will up to face major challenges in integrating into their new teams, here are some of the classic examples:
  1. Testers that does not participate in the day-to-day activities.
  2. Testers that perform only the minimum tasks without taking more responsibilities.
  3. Testers that use bad attitude towards the deliverables of their associated team members.
When the organization faced with a transition to agile frameworks, many engineers have a bad attitude that does not allow them to give a real chance to the new agile process, the results. In most cases, this engineers will leave the organization at the start of the transition or after a few weeks of the implementation process.

Although agile development has many advantaged over traditional methodologies, we need to keep in mind that agile development is not for everyone, but using the right education, training and the removal of many uncertainties can really help to mitigate engineers with the wrong attitude.

Lack of Training (Methodology/Technical)

Training is one of the most important things for testers prior they integrated into their new agile teams; the training sessions should focus on two main aspects:

Methodology – This is relevant to the entire organization that will need to get the necessary training about what is agile and how to work in this new environment. However, for testers there should be an additional educational layer that will help them to succeed in their new teams, this layer should allow the testers to visualize their future in their new agile teams and how can they contribute to the success of the team.

Technical - There is a Hugh difference between traditional and agile testing, testers should get the necessary technical training that will allow them to stay relevant in their new teams. Just, for example, a good training of programming languages will allow the testers to handle many automation challenges such as writing/using the automated framework, writing and debugging tests and share a common language with the rest of their team members.

After many years of involvement and directing agile transitions, lack of training is one of the most common reasons of failures to integrate testers into agile teams, so there is a Hugh important that your testers will gain all the training and knowledge they need prior and during the integration into their new teams. 

My Presentations