Thursday, May 8, 2014

QA – Graphical User Interface Testing (Advanced)

The Graphical user interface (GUI) is the main platform we use to manipulate a given application, a great UI that was coded and tested efficiently will allow the end user to perform a highly complex operation without a complex set of tasks.

When talking about a GUI testing, we need to take in mind that our goal is to help the user to perform his tasks with the easiest way and most importantly in a way that saves is time, among the different criteria’s that we need to consider:
  • The Graphical interface must simplify complex tasks.
  • The Graphical interface must have a logic in the way it’s built for use.
  •  Multiple functionalities that doesn’t relevant for the end users should be removed, that’s allow the client to concentrating on the functionalities that he really needs without wasting time in figuring the once that not relevant for him.
  • If the user need to learn the software Graphical interface from 10000 pages of “User Guide” than this GUI is simply not a usable interface.
  • Continuing the previous bullet, Graphical interface must contain the functionalities that requested by the client, the GUI that failed to contain the basic requirements and design specifications is not a valid one.

Personal View:
In this kind of test we must make sure that the user interface functionality, feel and look are coded with the exact behavior as defined in the SRS documentation that contains the software requirements and design specifications. The GUI functionality should do what it’s supposed to do and nothing else (Positive and Negative tests), you need to remember that UI functionality like buttons or text Fields can receive Valid and Invalid input from client. Therefore, you must be sure that a UI object will know how to handle this inputs and design the relevant test cases.

The Challenge in GUI testing
Software testing is a very challenging task, among the multiple testing types involved in the process we can find the “Graphical User Interface” tests that raise a major challenge in every software testing Lifecycle.

It’s well known that in the past 30 years the “Graphical Interface” changed dramatically in almost every aspect, today “Graphical User Interface” is written in highly advanced coding languages that allow the company developers to create a highly complex interface that involved different layers of design, functionalities (with millions of available parameters that can be sent as input) that cause the interfaces to behave differently on different platforms that support our software.

The main problem is that every device that compatible with our software may behave differently when integration made with the software, and change the total look and feel when using it.

 In other words the challenges for testing today are determined by two major aspects: 

User interface complexity–As described in the paragraphs above, the software industry can produce a very complex GUI (that contains infinite scenarios and combinations) for the end clients. Therefore, it’s a simple fact that the testing complexity is cut based on this parameter (GUI complexity = More tests).

Supported platforms (Hardware/Software) – not like 30 years, today's applications become relevant to almost every device and software platforms, the more platforms we need to support more testing is needed, just for example, think about an application that should be compatible with the following platforms:

Hardware platforms:
  • Mobile Phones
  • Personal Computers
  • Tablets
  • laptops

 Software platforms:               
  • Web browsers (Firefox, IE, Chrome…).
  • Phones OS (Android, WinOS, MacOS…).
  • Different OS systems (Win 8, Win Servers, UNIX ….)

Well, as you can see the matrix is infinite (and that’s before we described the actual software we need to test) therefore, when testing the “User Graphical interface” we need to calculate and be familiar with all aspects and only then, we can start designing our test plan and coverage matrix (I will expend this point in the next paragraph).

How much testing is enough?
Based on my experience, I can say for sure that many testers have major issues when performing UI testing, the main reason is that when executing such tests, it’s really easy to get carried away and waste a critical testing time that should be saved for additional testing types that cover different layers of the application, but here come the main question, how much testing is enough for UI testing based on the importance of such component?

The simple answer will probably be that just like any other testing types, you can say that you do enough testing only when all the risks you analyzed at the beginning of the testing process are removed. But wait, it’s not easy as it looks, to understand the risks (That involved additional testing aspects like the form Fell/look of the tested forms) you must do a skillful “Risk Analysis” and conclude those aspects, if you failed to do so you probably go to the wrong direction and test the applications while executing thousands of tests that written from the point of view of coverage(that you never have the chance to finish within the testing timelines…) and not from the point of view of an efficient testing(less testing scenarios but the once that written will provide a batter and efficient coverage) that integrates multiple testing objects.

Automation or manual tests..?
To answer this question we need to consider few parameters that I already covered at the beginning of this article (To make it easier to review them here also), the parameters are:

  1. Today “Graphical User Interface” are very complex for testing and contain an infinite supported scenarios (Based on my experience, most GUI bugs are a direct result of a tester who sees the GUI testing in a naïve approach of clicking few buttons and see if the GUI functionality works).
  2. GUI testing is one of the most time consuming tasks in the software industry.
  3. To make an efficient GUI test, the tester should consider additional aspects that not relevant to other testing types (among those aspects we can find the feel and look of the tested software).
  4. Depends on the GUI complexity, writing automation test cases could take an infinite amount of time, as a result, we must select the test cases that really important for the regression cycles, just like manual testing, we cannot cover everything using auto tools.
  5. Continuing the previous parameter, Automation can cover major aspects of the tested GUI, but few GUI cases cannot be automated (can we test different graphical aspects that involve colors that may be changed during the software testing Lifecycle..?).
  6. The GUI is one of those components that could be changed multiple times during the testing life cycle, as a result, we need to save a large amount of time to maintain the written automation, in some testing cycles the maintainer of the automation can take as much as three times from manual testing and as a result become ineffective.
Based on the list above, you can easily understand that you need to find the balance between the test cases that can be automated and the once that should be tested by the human eye. My suggestion for you is to make manual testing of new components, but afterwards move each case to automation tools. Repeating UI tests can be really painful in the @#$. 

The basic steps for GUI testing
GUI Testing may involve few stages that relevant to another testing types that we all familiar with , My example here is really basic and could be adjustable depends on the company needs and testing strategy:
  •  The basic first steps is to do your research (That mostly relevant for the “Static” testing type), few examples:

o   Read the SRS and DDD design docs.
o   Understand the client needs.
o   Understand the business case.
o   Understand the timelines for tests.
o   Do the research for AUTO tools.
  • Perform the relevant “Risk analysis” and “management”.
  • After you finished your research (and understand the risks) and you have all the info, you need to determine the “Coverage” criteria (As explained today GUI contains an infinite number of tests that cannot be covered).
  • Determine the testing strategy (Sections that going to be automated or tested manually).
  • Create your STD with specific test cases (every test in you STD must contain the expected result) that includes tests based on GUI objects (I’ll explain this issue in the next paragraphs), each object must be tested with the relevant events related both Keyboard and mouse (Mouse Clicks, Keyboard shortcuts, menu manipulations, GUI functionality, Overall look….).
  • Deliver the relevant test cases to the automation team.
  • STD execution and verification.
  • Analyze the testing results and coverage to determine the remaining risks (I’m telling you right now, the coverage will not be 100%..., if you performed a flawless testing the coverage will be among 60%).

Recommendations for User interface testing:

The human eye - GUI testing should be made by human beings, the perspective and sense is different from an automation procedures, my recommendation is to test every part of the application and only after you confirmed that it works as expected we can move it to the automation team and forget about those tests.

Personal View: Repeating Graphical tests is not a nice task, GUI testing is perfect for automation and key part from every regression. Therefore, try to automate as many test cases available.

Choose the right direction(Coverage vs. efficiency) – I already mention this point on earlier paragraphs, but it’s a really critical point in GUI testing, as explained Graphical interface testing may involve an infinite testing matrix that you cannot cover, but in the same time graphical bugs are not something that you want to release to your end clients (Think about an enterprise product that have GUI bugs, it’s may severely affect the company reputation because it’s the basic things that must work. In addition, from the user perspective, if you fail in the basic stuff, you probably failed also on the performance and functionality side), to reduce those faults you must plan your testing activity in a way that your testing is efficient and not based on 100% coverage that you cannot accomplish in 100 years of testing (Efficiency = based on appropriate Risk analysis and Risk Management).

Separation of GUI objects - Graphical user interface is a set of objects that located on a host platform, when designing your tests you should consider every object as “Stand Alone” and ask yourself few questions that going to help you design your STD:
The object business case?
Why we need it?
Is the location is the right one?
Available Inputs (If Supported)?
Integrations with different objects? 
What are the outputs we need to get while using that object?

Few Examples for Graphical User interface objects:
Context menus
Help icons
Dialog poxes
Help menu
Legend menu
Tab sequence
Maximize /Minimize.
Combo boxes (Default value as selected value)
Search Boxes
Sorting / Groupings and columns manipulations.
Increase / Decrease

Example for Graphical User interface parameters:

Object visibility:
Controller’s position
Grid size and position
Controller’s size
 Picture size and shape

Object Syntax:
Capital/ Non capital 

The Software industry perspective(The expected standards from a new software) - Every Graphical user interface must be tested with few basic standards that we use in the software industry (although the SRS doc is the one who determines the end design), every tester who involved in GUI testing must be familiar with the software industry basic expectations and
standards from a new software,  this issue is really important both in the static testing procedure (Industry expectations will lead to SRS changes if needed) and the “Dynamic” testing (The tests performed from both the company and the industry perspectives). 

  • Keyboard button “F1” should point to user help guide on windows platforms.
  • Every field that used to find values should be called “Search” and not “Find”.
  • In Windows OS the “Ok” button will be on the left of the “Cancel” button (The opposite behavior from MacOS). 

Rules – GUI testing should be tested with few basic rules that must be considered when designing and executing test plans, Classic Examples:

  • Validation process must be executed - More functionalities are not stating that you built the correct software (Validation) – the validation process is made to guarantee that the software meets the client expectation, too much functionality that not relevant to the client will lead to multiple questions and misunderstanding of the functionalities that he really needs. Therefore, a good validation process should be made to ensure that the client receives the functionalities he needs and the one that not relevant will be removed.

  • Usable – The GUI is the main console that the user going to use when working with the software, your tests must involve additional layer that guarantee that the user can use the software, those tests called “Usability” testing and should be used as additional layer in your testing plan.

Checklists and Guidelines – GUI testing may involve thousands of tests that performed on different layers of the application, sometimes it’s basically impossible to remember the parameters we need to follow in our tests, as a result, we should use a predefined  “Checklist” and “Guidelines” in our test cycles  that ensure that our Graphical interface tested professionally and efficiently.

As explained in the previous paragraphs, we need to see our graphical interface as a list of objects, as a result we need to understand that some checklists are relevant for all objects (Color\Syntax...) but some checklists are relevant for specific objects only (Functionality, Events (Clicks, moving the marker)…).

The following list is designed to handle few mandatory objects that relevant for most applications that are used in the software industry (To create this list I mostly based on Windows operating systems), my recommendation for you is to take a really big breath before you are reading the list because I promise you that this list will change the way you are testing your graphical interface from now on.

Checklists and Guidelines that basic for every application:

  • In any case of “Disabled” field the user cannot use the functionality, in addition “Focus” on that field shouldn’t be available.
  • Every software that supports the “Help” functionality should be triggered from the “F1” button.
  • Is there a default object that highlighted when the user starts the application?
  • Try to start new instance of the software while the first instance already in active state.
  • Web applications should be tested with different resolutions (600*800, 1920*1080…).
  • TAB functionality  should be available with the order of left to right(TAB – Moving forward  / SHIFT + TAB is moving backwards)
  • In most cases GUI forms should have the Minimize/Maximize options.
  • Try to run the application from different locations (Client<->Server).
  • Closing the application shouldn’t occur without an approval notification that allows the user to “Approve” (OK) or “Decline” (Cancel) is operation.
  • In most cases the application integrated with a dedicated Database (Sql, Oracle…), in many situations the application DB requires a value from the user (the expected Value cannot be empty) therefore those fields will be marked with default values (GUI) and if not specified differently by the user those default values are going to be the one that will be used by default.
  • In any case that cause the application to move into “Loading” state it should display a relevant notification for the user that the current state of the program is “Processing”.
  • Text should be justified (otherwise it’s not relevant).
  • It’s basic to try to start the application few times while another instance in “loading” process.
  • The main functionalities of the application should have a corresponding “Shortcut” key that user can trigger from his Keyboard.
  • Make sure that one form buttons doesn’t affect sub-form when it’s not needed or specify.
  • The application name should be displayed on the application main form.
  • Each application should described with the version number.
  • In most cases you should have the ability to perform every operation from the Key board and without the mouse.
  • In any case that the changes made by the user are final, make sure that the “Cancel” button reacts as “Close” button.
  • The “Help” menu should be available in the main screen navigation bar.
  • Applications should start based on few triggers:

§  Double click on the application icon.
§  Double click on the application shortcut.
§  Right Click on the application icon -> Select the application
§  Command line (With a dedicated syntax).
§  Script file (With a dedicated syntax).

Color tests are relevant for all forms, the following list contains the main questions you should ask yourself when testing the GUI from this side:

  • Do the text inserted in text fields are in the correct color?
  • Do the form objects (Drop-Down list, Radio button) displayed with the correct color?
  • Are the Hyperlink colors are in the expected color?
  • If fields become “Grayed-Out”, do we display the correct color?
  • Loading screen displayed with the correct color?
  • Loading process bar in the correct color?
  • Form title and description displayed in the correct color?
  • Form background color is the correct one?
  • Are the buttons are in the right color?
  • Is the sub-screens in the correct color?
  • Are the “Warning” notifications displayed with the relevant color?
  • When a field in focus, do we mark it with different color?
  • If form or field state is set as “Read-Only”, do we grayed them out?
  • In general, are the application colors are usable for the client? 

Every application contains different aspects of syntax, the following list is relevant when testing a software from this side: 
  •  Is the screen text being written with the correct font?
  • Is the screen text being written with the correct size?
  • Is the text in all objects are written with the correct font? 
  • Is the text in all objects are written with the correct size? 
  • Are all the screen texts are aligned correctly?
  • Are all the screen texts in the text box are aligned correctly?
  • Spellcheck tests are basic...
  • Is the first char (If relevant) in a word set as capital?
  • Does the second word should start with Upper/Lower case?

 Checklists and Guidelines for specific Objects:

Validation fields(User Credentials is classic example)
  • All mandatory fields must contain values, if the user failed to insert valid values the validation process will raise a notification that explains the error that allows the user to supply the missing value.
  • If the form contains mandatory fields it’s usually marked with asterisk (*), but that’s not all, most users will not understand the meaning of such mark. Therefore, you should add a note that explains the values that user should enter.
  • It’s highly recommended that the authentication object will use an additional password checker to make sure that the user insert a valid password and save the actual validation time (Enter Password + Re-Enter password).
  • In a password field, it’s a good idea to use a “Password Strength” checker, to notify the user that is password is not secured enough (Mostly used on WEB authentications when creating the first registration).
  • Different “Injection” tests are really important for validation screen, you need to check that your validation screen is protected.
  • If mandatory field remain empty and user press the “Ok” button, the application should mark the empty field to direct the user to his fault.
  • Do the validation fields are “Case Sensitive”?
  • In most cases the validation fields should be limited with MAX and MIN values (Boundary tests are classic for such fields).
  • Negative testing for such fields are crucial, validation must be failed if user insert invalid values (In the worst case scenario user can insert invalid values and the validation pass).
  • In any case of invalid validation, the user should notify that the validation process failed with an appropriate notification.
  • If the validation process limited to a MAX number of failures, the user should receive a valid notification that explains the remaining number of authentication attempts, if the user failed to authenticate within the limited attempts a valid notification should be presented and explain the current state.
  • If the user failed to add an appropriate value in one of the fields and press “Ok” in most cases he cannot edit the field (That cleared from the inserted value) and should re-enter a new value.
  • You must understand if the validation performed on the client side or on the server side it’s really manner when testing web validation screens.
  • In any case that the user press the “Cancel” button the form should be closed and the validation should not take place.
  • Check if the validation fields should support a specific format of values.
  • Does the SRS doc, specified that the authentication support negative values?
  • Does the SRS doc, specified that the authentication support Special characters?
  • What should be happened if user insert a valid valued and the validation completed successfully?
  • Do we make the validation against any kind of Database? If so, you must check if the DB columns allow “Null” values.

Radio Buttons
  • Every button should be available for selection both by using the mouse and keyboard (Arrow keys should allow the navigation between the buttons).
  • When using this object we need more than one available option for selection. Therefore, in any given state only one option should be selected.
  • By default at least one button should be selected. 

Push buttons
  • Esc should active the “cancel” button (if available in form).
  • Every button should be triggered by the mouse when user clicks on it.
  • Continuing the previous bullet, the “Space” key should do the same action.
  • Every button should have the option to trigger with an appropriate keyboard Shortcut (You must make sure that duplicate shortcuts are not existing).
  • Make sure that all buttons are similar in size, shape and size.
Drop down lists / List Boxes /Combo box
  • The “Drop Down list” is an object that doesn’t receive any typing inputs from the user (In other words drop down lists are not text boxes).
  • Pressing the arrow of the drop down should open a menu with all the available options for the user.
  • If the list contains multiple values, the list should be scrollable.
  • Drop Down values must be presented with order, in 90% the order determined alphabetically.
  • The User should have the option to change the values' order using different options of sorting based operations.
  • The list should be responding to user manipulations, when user press a character the focus should move to the first value that start with the same char.
  • When the drop down in focus the Keyboard combination of Ctrl-F4 should open the list of values.
  • When the user selects a value, this value should be displayed on the main Drop Down field.
  • Not like the first two objects, In Combo Box, user should have the option to insert text.
  • Drop Down objects doesn’t need to set with default values.

Text Boxes
  • Double click on the text should highlight the entire syntax.
  • Text box must support Copy/Paste of syntax from different locations.
  • The User have the option to enter text into the box.
  • Testing for text box must involve negative testing that indicates that the field behave correctly when user insert invalid characters.
  • The User should have the option to select specific text when using the Shift + Arrow Keys.
  • Try to use Blank fields and sync the value to DB.
  • In most cases the “Save” button should be grayed out if the user not yet insert syntax to the text box field.
  • Enter syntax in the text box with space at the beginning.
  • Enter syntax in the text box with space at the end.
  • Do we support upper and lower case?
  • Text box should be limited to Max characters.
  • The basic test for this object is to overflow the text box by typing the MAX available of characters.
  • Every text box should be tested with special characters.

Date and Time fields
  • Applications must be tested with OS “Time Zone” changes, different components that integrated with different time zones mail lead to massive failures in the data synchronization and integrity (Believe me I know it personally…).
  • Can you change the date/time (insert day in the year location, insert year in the month location...) order and approve the change?
  • Change Time zones in specific components to see how the application can handle different date formats.
  • Make sure that the application can handle any case of “Leap” years (if your application failed to handle such cases an appropriate notification should raise for the user).
  • A Different date fields contain MAX and Min values, you must make sure that negative values will not raise any error(Boundary tests are classic..) and appropriate notification displayed to the user, Examples: 

Special 1
Value > two chars
Value > two chars
Special 2
0 or 00
0 or 00

No comments:

Post a Comment

My Presentations