Wednesday, June 17, 2015

Software Installation Testing - Best Practices

Overview

After more than 10 years of experience that I gain in the software industry, I can say that this type of testing is simply critical no matter the type of software that you are testing.


And why it’s so critical? Think about a software that installation tests are excluded, what are the Risks? Well the answer is huge! Both from the company and client perspective. 

Example: 

You have tested a software for five or six months, you executed a numerous amount of test cases that covered different aspects of the software, the only problem is that the owner of the project has told you that the installation part will be examined “On the road” during the testing cycles (based on the wrong assumption that installing the software during the testing cycles will be enough to cover this section). 

Why this assumption is so wrong? Because you cannot trust that during the testing cycles the software will be installed on all the relevant supported architectures or any other complex scenarios that wasn’t needed during the regular testing cycles. 

Well, now that you’ll all happy that the software is ready to be distributed as Beta, you send it to few clients that install it on one single architecture that doesn’t use during the testing cycles and then Boom!!! The installer failed to proceed with the implementation and the user cannot use the software. 

Think about this situation, what would you say to your client? Stockholders? I hope that, you have a good answer because I don’t.   

To avoid such scenarios you have the option (Based on the scenario above, it’s not really an option…) to perform installation tests during the software testing life cycle that will reduce the Risk for such cases. 

Another aspect of the importance of this test is the “First Impression”, remember that the first interaction between the client and the application is the actual implementation process. 

Yes, the first impression of the user regarding the quality of the application is not based on the great user interface that you have, efficient functionalities or any other great work that you did and relevant to the second phase (Phase 1 - Implementation, Phase 2 - product review…. Phase 3…) when the implementer performs the first review of the product capabilities.

 Now, do you really think that a client that need to pay thousands of dollars on a product, will continue with the purchase when he have problems with the basic steps of phase 1? And has, you probably know, you have only one chance to make a good impression. 

So how can we make sure that the first impression of the user will be the one that make the difference? Just continue to the next paragraphs, understand them and use the suggestions in your daily testing process. 



The Basic Definition 

Installation testing (“implementation, testing” by it's second name) is a testing method that we will use to validate that the application and the corresponding components are installed and working as described in the application documentation (SRS/SPEC/DDD).

The implementation stage is one of the most important phases that involved in the software life cycle, as a matter of fact, in any case of failures in the implementation stage, the life cycle will end before it can be started.

 

Types of installation 

Let’s think about a new application that we need to install and configure, the basic process is to upload the installation menu, add the relevant configuration and wait until the process ends. Well, that’s great, but there are few more ways that we can use to implement the software, this section will describe a few more types of installation that can be really effective during the software life cycle.  




Type of installation
Description
Manual installation
The basic and most common type of installation, the user will install the application will review and navigate be between all forms that are relevant to the installation process. 

Automated Installation
The application installed without any human interference (The main use, is on automated environments that configured with a dedicated test and run via scheduler).

Clean Installation
The environment that host the new application is cleared from any Old version/component of the product (Examples: New environment, New Client Etc.)

Silent installation
The application installed without any notifications that are relevant to the automation process (Will written in a dedicated log file). Example: It department that wants to install a new Antivirus client on 5000 computers during the company working hours. 

Local/Distributed Installation
Installation that we made on specific host Vs distributed installations over LAN/WAN

Unattended Installation(Operating System)
Automated installation is made from a single source that distribute the installation packages to multiple servers. This automation will based on simple TXT/XMl file that contain the installation steps and post configuration. 

Attended Installation (Operating System)
The same as “unattended” installation, but here we don’t have the configuration file and user should attend in the entire installation and configuration steps. 




Activities of installation testing

When people think about “Installation” testing, they may examine it from the narrow and wrong perspective that this type of tests is mainly focused on just the basic installation process (Clean implementation of the product).  

I believe that testing should always perform based on a wider view, this table will help you to understand the different aspects that involve in this kind of tests, and I hope that this will help you improve your knowledge and view, about this highly important testing method. 

Example:
Activity
Details :
Licensing.

Licensing and installation are going together, you need to validate that the installation is Approved/Declines based on the license that you have (The same thing for components).

Repair existing implementation. 

Any scenario that existing version is installed and doesn’t function as expected, A repair is needed to overcome the problem.

Add or Remove components.

Sometimes you want to remove an existing component or add a new one.

Installation on different architectures
Remember that each architecture that you are using to make your implementation tests, will provide few more challenges, a good testing process will validate that the software is ready to use on all the supported architectures while managing the multiple challenges that raised along the way. 

Uninstallation of the software
The client needs to entirely remove and existing instance of the software.

Registry Validation (Installation / Uninstallation )
As you probably know, the registry contains values to each program that installed in the Computer OS, it’s a major task to validate the Registry parameters are added/removed correctly when you install/remove the software. 

File System Validation (Installation / Uninstallation)
The same case like the registry, you need to validate that the software installed with all the expected files and on the accurate path, from the other side you must validate that all files are removed when you uninstall the software.

Distributed Vs Local  installation
Think about a software that has many components, each component should be installed on a specific server , in that case we need to configure the application on one server and connect to each one of the distributes server to install and configure the relevant components.

Based on my experience, sometimes you have software that can be installed on one machine with all components interacting which other, in that case the QA team may run the entire test cycle on this simple configuration while ignoring the more complex tests that are relevant to a distributed configuration.

Negative installation tests
Another great facto that can determine the criteria of success and failure for this kind of tests is the negative side of testing.

You should always had negative testing no matter the type of testing that you are currently running, in installation testing this aspect is even more important, just think about the following examples:

  • Break the installation process in the middle of the installer execution.
  • Install the software without the full necessary permissions.
  • Install the software when the System Disk Space is totally consumed during the installation process.
  • Install the software on a low resources architectures.
  • Etc.

Note!
I will add more information and examples, in my second post regarding “Installation testing checklist”. 





My Presentations