Wednesday, January 14, 2015

AD Hoc Testing | Best Practices


Ad-Hoc testing, is a different testing approach, in such tests the tester performs is tests without a real planning (There is no time to design and write an appropriate test scenarios with expected results) and in a few cases without a predefined documentation (SRS/Spec that describes the basic requirements and specifications).

When executing “AD-Hoc” testing, the tester can use this approach to test any part of the system (Random or dedicated tests), and uncover any hidden bugs that in some cases will fail to be found in the regular testing cycles.

To achieve the best results during such tests, the tester will use is intuition, skills, experience and knowledge to test the software while improvising to manipulate the software in a way that will uncover the hidden bugs.



When should we use this testing approach?

Based on the reading so far, you can understand that this type of test is slightly different from the conservative testing approach that demands an appropriate documentation that we create based on the client requirements and specifications.

But what can we do, in a case that the project timelines are too demanding and doesn’t allow us to make an exhaustive testing With an appropriate and detailed testing documentation as we supposed to do in the regular testing process? Well, the answer cannot be “I cannot test! You can, and this testing method will allow you to achieve a better result from the first option of not testing at all.

Author View:

Based on my experience in the real testing world, I can say for sure that most project managers will exclude this testing type from the regular testing cycle, the main reason is that Ad-Hoc testing are performed without a real monitoring capability (there is no STD that the manager can track and see the tester progress).

Furthermore, most blogs or written resources will suggest that you perform these tests only at the end of the testing cycles and after you have finished with the formal testing approach, my opinion is completely different, I think that you need to embrace this type of testing and involve it as a regular method in your testing approach, I really think that you will achieve more from your testing effort when you execute Ad-Hoc testing at the beginning, middle and the end of your testing effort.



When should we NOT use the testing approach?   

After you have read my personal opinion you may questioning this question, but sometimes you have specific scenarios or limitations that make it harder to use the Ad-Hoc testing, I decided to add them so you can include them in your calculation before you decide to use this method or not.

When the testing functionality, window Etc., is simple


When there is a simple window that doesn’t include any complex scenarios, in that case you will find 99% of the bugs, Ad-Hoc testing will not help you to find anymore defects and I consider it as a waste of time.


When you have clients that want to see the testing matrix

My experience in the software world (Both from the testing and client side) is really helpful when I want to explain this point sometimes as a client you want to see that the product that you need to spend thousands of dollars, is tested efficiently and all the basic bugs are removed and excluded when you want to operate the software after the purchase.

To accomplish this, you can request the testing matrix that allows you to allow you to understand if the software is good enough for your site.

Therefore, if that's the case, AD-Hoc testing will not help you because you cannot supply such documentation that expected from the client side.

The advantages of “AD-hoc” testing

  • Ad-Hoc testing often used as an additional testing layer of the regular testing cycles.
  • Testers can use their knowledge, without the regular restrictions of script testing.
  • Ad-hoc testing provide quick results.
  • Ad-hoc testing can improve the documentation and vision that failed to build properly in the ordinary testing methods.
  • Ad-Hoc testing involve different testing methods that combined together in different situations (negative/positive/exploratory /functional testing ETC), this combination will help the tester to perform many tests in a short time and find more bugs compared to the regular testing effort.
  • Improvising is more than needed in such type of test, the improvising factor, will allow the tester to concentrate in the software and not in the official documentation, the side effect is that the tester will learn more about the software (Architecture, flow, integrations Etc.).
  • Ad-Hoc testing is one of the most efficient ways to test a software in a case that the testing timelines are narrow and doesn’t allow you to test the software in the formal process.

The Disadvantages of “AD-hoc” testing

  • Ad-hoc testing should be executed by specific testers that have the knowledge and skill in the tested software, if that’s not the case, the effort will fail to be efficient.
  • You cannot assume the time estimation that you need when using AD-Hoc testing.
  • Ad-hoc testing is NOT a valid method to us, during a Beta and acceptance tests.
  • Missing information on how the time spent during the testing activity.
  • Bugs that found are (In most cases) really difficult for reproduction (missing documentation that specify the steps for execution).
  • AD-hoc testing are not enough for testing coverage.
  • There are NO predefined results for AD-hoc testing.
  • Major bugs could remain undetected.
  • Poor documentation (if any…).

Best Practices for AD-has testing

Based on the information that we reviewed until now, we can continue to the next level and provide few guidelines that allow you to achieve more from your Ad-has testing.


Be familiar with the tested software

As previously explained, the main key for success in this type of test, is to control the knowledge and architecture of the tested software.

Comparing to a tester that doesn’t have a great knowledge,
a skillful tester will have the opportunity to rely on his knowledge and skill and think about more specific and relevant testing scenarios, that will cause more opportunities to find great software bugs.

Relay on your experience

Continuing the first paragraph, a tester that has the experience will most likely will find more bugs than a new tester that doesn’t have the same experience in the tested software.

Run as many tests as you can

Another factor that can make the difference between success and failure, is the number of tests that the tester can execute in a narrow timeline. That’s a simple equation, in most cases, the more testing you execute, the more bugs you will find.

Document the basics

Ad-Hoc testing is performed by definition without any documentation, but based on my experience, you must make the minimum documentation for easy tracking and testing coverage. Examples:

-      Document (High-Level) the basic areas that you already tested.
-      Document the bugs that you find.
-      Etc.

Learn from the testing process

Ad-hoc testing is a great way to learn your product, because you perform your tests without a predefined documentation that restricts you from manipulate the software in a way that you want in specific moments. 

While performing AD-Hoc testing you will learn more about the software than the regular documentation review that you based on.

In my opinion, the ability to learn more about the software is critical and should always combine to gather with the main target of finding the software bugs, you always need to remember that knowledge is power, Ad-Hoc testing will give you the opportunity to gain it.

Test on all levels

Ad-Hoc testing, allow you to test every part of the software without any restrictions, based on this approach, you will achieve more from your tests if you use this advantage and perform your tests on different levels of the application (Component - >The integration between them ->System).

Focus on areas that has more risk

Although AD-Hoc testing are performed without any real planning, it doesn’t give the tester the opportunity to taste like a monkey. A good tester will always test based on a Risk analysis that he made before he run is the first test.

Therefore, before you start your tests, perform the basic Risk identification and Risk analysis, only then and after that you understand the areas that has the bigger chance to fold, you are allowed to start your test, otherwise you are testing without a real efficiency.

No comments:

Post a Comment

My Presentations