Friday, October 23, 2015

Lesson 2 - Apache JMeter Introduction


In this article, we will see how can we implement and configure the Apache JMeter that allowing us to perform different types of performance tests such as Load/Stress on a Web Application.

Prior to the implementation step, we first need to answer the question “What is Apache JMeter..?” , JMeter is an open source Java application that is built to allow the person which use it to perform both basic and advanced performance tests.

JMeter provides a well-structured Graphical interface that supports almost any Operating system (Windows/Unix) that can run java virtual machine (VM).

In addition, if you’re new in the performance testing world, you need to know that performance test activities (Stress, load, pike, endurance testing...) are running (In most cases) for a long period of time, therefore apache JMeter is designed to measure the designed tests on a long execution time, flexibility in the supported scenarios and a great reporting mechanism that allow the user to receive a great structured input about the test results

JMeter Main Features

The main features of JMeter:
  • JMeter provides the ability to reduce the regression testing cycles with automated test execution.
  • JMeter provide a great and intuitive graphical interface that simplifies the user tasks
  • JMeter provides a more reliable test results compared to other open source tools.
  • JMeter is an open source software, everyone can download and use it for free!
  • JMeter can be used to monitor your servers (Database, Web, Ftp…)
  • JMeter provide a Great statistic during and after the test execution.
  • JMeter support multiple protocols (Web, FTP, LDAP, Imap…).
  • JMeter support multiple platforms and architectures.
  • JMeter support multi-threading execution, therefore we can execute multiple tests simultaneously, save time and increase our testing coverage.
  • JMeter support XML files, which allow us to export our tests, edit them in any text editor application and generate new test cases.

How JMeter work

  1. The User configures a new test case and start the test.
  2. JMeter starts to simulate a multiple request to the target server that simulate a "Real" scenario of multiple clients that connecting to the server.
  3. The server receives the virtual requests and respond.
  4. JMeter responds to the server answer.
  5. JMeter allocates statistics, analyze the data and gather the relevant information.
  6. JMeter Finish the test.
  7. The User can extract a detail report.

Wednesday, October 14, 2015

Uncover the truth about Testing documents


I have more than 13 years of experience in the software industry, during this time I participate in many major projects that produce test documentation (STP/D/R) which included thousands of test scenarios/cases for each major project.

Based on my experience I will tell that testing documentation is a major part in every large project that combined with testing efforts, now when Said, I want to add some facts and personal humor that uncovers the truth about test documentation. 

What is the truth that we know as software testers?

  1. The world is divided into two types of testers, the first type will tell you that they test any software without a preliminary test documentation, the second type will tell you that they cannot.  If you belong to the second type, find a new occupation.
  2. Writing a software test case is a simple matter when you understand how the system works, don’t make it a Ritual.
  3. When you write a test plan, please make sure that you cover more than just the basic SRS/Spec requirements.
  4. In any case that you need to explain the obvious steps of a test case, then you failed to write it in the first place.
  5. If you write a test case with million steps, I guarantee this, No one will execute this test as expected from him.
  6. Sometimes we spend Hours on Hours to write test documents with the knowledge that no one will use them.
  7. Ask yourself this, do you really perform your tests based on each step that was written in the test design?
  8. Be truthful to yourself, how many test cases did you mark as “Passed”, without the actual execution..?
  9. Many test documents are written when the person who wrote them doesn't know how the system works.
  10. Be truthful to yourself, do you really know why there is a "case" when you need to write a "Test Case"..?
  11. There is a simple algorithm for test documents, It takes 20H to write a simple doc, the review will take 20H*10, the algorithm changed to 20H*100 if the person that wrote it doesn’t know how to explain the idea behind it.
  12. Some testers as the assumption that they perform a better job when they write a large test case document, based on my experience they wrong! The number of test cases that you write will never indicate your quality!  
  13. After a few months into a project, no one will know about the original test design that you prepared.
  14. How many test cases have you written when the answer to "Is it possible to test this..?" was No!
  15. How many test cases you've been written without a true intention to execute them?
  16. How many test cases are written based on a meaningful use case/client story?
  17. Every test plan/design will include a copy/paste data from older documents.
  18. How many test cases are written without knowing the expected results?
  19. At least 50% of all test plans are not signed off by the relevant owner.
  20. Ask yourself this, how many bugs are found based on a specific test case, and how many bugs are found without a dedicated test case…? I promise you that the results are not decisive.
  21. Many test documents are written without knowing the user needs.
  22. A Use case is not a test case, Test scenarios is not a test case.
  23. There are always two options to write a test case, option 1 write it in a way that will make sense to anyone who read it, Option 2: write it as you want it, what is your preferred way?
  24. It's a fact, no one love to write thousands of test scenarios.
  25. Remember that there is a great testing method that called “Exploratory”, embrace it, I promise you that you can save hours over hours in writing test cases.
  26. In many cases, the person that should approve your test design will fail to answer the question of "What is the business case behind this software..?"
  27. I always prefer a simple and self-explained test design, rather than a Hugh and complex document that no one can understand or use.
  28. Remember to write all the test cases that matter, never reduce your cases because that you think that there isn’t enough time to test them.
  29. How many times you did, you complained during a project, that you put too much effort to write documents instead of actual testing…?
  30. Everyone can write a test case with inputs that match the Spec requirements, your job is to write a test case that can question it.
  31. You will never have the time to execute all the test cases that you write if you know your job you should know how to prioritize them.
  32. No matter how hard you try, there always be a person which will fail to understand the idea behind some of your test scenarios.
  33. No matter how hard you try, you will always fail to update your test documents during a project that continues more than a month.

Saturday, October 3, 2015

Lesson 1 - Apache JMeter in the world of Performance testing


Prior to starting the course of JMeter, I want to start the course with a few basic definitions that are relevant to any person that want to take part in the world of performance testing.  I will explain each category with a very short description and add further information during the course.

What is Performance testing?

Performance testing is a is a non-functional testing technique that we execute when we need to validate how the tested software performs in terms of stability, reliability, responsiveness and scalability under a predefined system workload.

What are the benefits of Performance test types?

  • Comparing the performance attributes against other applications/Old version of the same application.
  • Understand the kind of errors that may affect the system functionality and user experience.
  • Understand the connection between slow performances based on the related defects.
  • Reveal the bottlenecks of the application that will not cover in the functional tests.
  • Understand how different hardware will affect the application performance.
  • Understand the tuning that you need to create a better load balancing.
  • Understand if the software behaves according to the client demands.
  • Reveal functionality errors that occurred during load/stress tests.
  • Get the answer to the basic question of "how many users can use the software at the same time prior to compromise the application performance?"
  • Collect great data about the software scalability and stability.
  • Understand of how the system behaves on different loads.
  • Understand how the system can handle different loads.
  • Reveal defects that come from concurrency scenarios.
  • Understand the system boundaries.
  • Determine the application performance against the market demands, which lead to continues improvement.
  • Provide a great view on the amount of data that the system can process when exceeding the upper limitations.

What are the Key Types of performance testing?

Load/Capacity Testing

Load testing is the process where we want to examine how the system behave when tested under a specific expected load while constantly increasing the system transactions (until the system reaches to the end boundaries). The load can simulated by the number of users that simulate a specific number of operations/transactions under a specific duration.
Load tests will help the tester to uncover bugs in the application related to bottlenecks, CPU/Memory overflows… and get a better knowledge about the system limits when the user uses it for a long period of time (Remember that load and Endurance tests are almost always executed as corresponding with each other).

Stress Testing

Every system has a capacity limit, when the system load exceeds this limit, the application will fail to be responsive to the user request and in some cases it becomes unusable.  Therefore, we need to run Stress tests to understand what the system limits are and how the system handles such situations.

Endurance Testing

Endurance testing is performed when you want to test the application behavior and parameters (Memory, CPU….) under long term loads.  The main purpose of this test, is to revel the bugs that may raise during sustained use of the application functionalities.

Spike Testing

Spike testing is performed when the tester need to test the application against sudden loads (Both increment/Decrement). The goal of pike tests is to determine the behavior of the application when its needs to handle a dramatic change in load.

Volume Testing

Volume testing is done when you want to determine the efficiency of the application when processing a Hugh amount data that processes be the application. Running volume tests will help you to show that the system cannot be used when the amount of data become too heavy for the system to handle.

My Presentations