Saturday, May 17, 2014

Software Testing – guidelines and principles

Overview

A Few years ago when I started my way in the testing field I thought that testing is all about coverage, I always want to test the full testing matrix no matter the time frame, “Risk Management” was one of those terms that I master now, but back then it’s look like the worst enemy, how can I release a software that I wasn’t able to test the full available scenarios..? For a beginner, I was really afraid that after release this one case that I failed to test is going to be the one who will discover on the client environment.

This issue was my worst fear on every testing cycle no matter the quality of the job I supplied, therefore when I decided that QA is the profession for my life, I make a deep investigation on few main fields that for me every QA/Tester must know and keep in mind before he touch any application, the fields are relevant for “Risk Management” and the main testing principals.

The next few paragraphs will cover the second part “Principles”. In addition, the list below contain principles and guidelines that existing in the software world for more the 30 years, my job is to add my personal view and how this principle can help in your daily job when the time frame is narrow and the pressure is high.

Software Testing Guidelines and Principles


Principle 1: Developers should avoid testing their own software

Before I start the discussion about this principle, I must say that this principle is a great example of the changes we had in the software industry in the last few years, the knowing that QA teams are crucial for the organization success and the understanding that testing is a profession and not a sidekick of a development team.

But this part is about principles, and if I want to analyze this issue from this view I can say that developers cannot test their own code (I don’t talk about unit tests..), it’s a really bad idea for every organization that have respect for his clients because in the end developers are here to build the application and not here to destroy her (that’s our job…).

When developer build the software he can see each function and know how it’s supposed to work, but for a developer to investigate the full system or different integrations it’s simply too complicated and not a good idea, in addition, think about a developer that you familiar with, now ask yourself how this developer response when you as a tester find a “Logical” bug in his code (you know one of those bugs that do the difference between the specialist and the novice testers), the response in 90% will be that he failed to think about it or probably failed to understand how this code affects the code it’s integrated with.

An additional aspect is  related to the philological side and says that testers simply doesn’t want to find the errors in their code because they spent hundreds of hours in hard work to construct something, for a developer to search for his own errors it’s simply against his own nature. In addition, the programmer who builds the application will want to do the best coding as possible like the tester who wants to reduce the defects to the minimum, as a result the programmer may ignore issues in his code that may cause future errors in real-time environments because he simply doesn’t want to expose the basic fact that his code is variable in some scenarios.

Now, think about it… do you really think that all developers are smart enough to report such bugs to the manager, project owner… and demonstrate code parts that coded with errors in the first place..? I don’t think so, a developer will want to show a flow lace code execution and if an error will be found in real environments, it’s the QA fault that they simply failed to do their job, sound familiar anyone? 

Before I end this section I want to add additional thinking point, as you can see the principle is dedicated for the reasons that make developers not useful for testing, but if you think about this little further, that’s doesn’t mean that a developer (that probably learned one or two throughout in his life) cannot make tests on his own software, the major key for this guideline is related to the fact that if that’s the case you will probably should expect for multiple failures that would be discovered on any simple testing cycle performed by the QA teams.

Principle 2: Do not design your tests based on the assumption that the software free from bugs

This principle is simply classic for the testing world, my idea here is to give a real example from the real testing world and demonstrate how the timelines factor may lead to the wrong assumption that the software is free from bugs.

How many times you received a software or Feature for testing that in the optimum world you will need 20 days for testing, but in the narrow timelines for the current  project you have only 2-3 days to finish your tests? 

Well, I’m sure that your answer will point that the timeline factor are relevant for almost any project, and why this point is so important? Because it has a direct impact on the tester judgment that based his tests on the wrong assumptions, the classic failure is the one that usually made by a project owner that will make the classic mistake and tell you that all you need to do is few “End to End” tests + some basic sanity tests because you already test this version on previous testing cycles and the risk now is minor and probably doesn’t exist. This assumption will lead to the second classic mistake that says that all we need to do in this case is to show that the software performed with the main functionality and nothing more, well this is wrong!

Our job as professional is to test any given software from the perspective that any software’s will have its own bugs. Therefore, we must make our tests from the point of view that the bugs are out there and the main task we have is to find them.

You need to remember that when testing a software that has new feature we have three main tasks: 

  • Approving that the software works as defined in the SRS doc.
  • Testing the old code and understand how the new feature effect it.
  • Reduce the bug counter to the minimum necessary. 

Most managers will most likely failed to understand that those three tasks are must be combined together on any testing effort (The timelines issues will cause the manager /tester to focus on the first task and most likely to ignore the other two tasks), any testing cycle that based only on one of them will not be effective and probably put the software on greater danger when deployment occurs on real time environments.

Principle 3: Test results must be inspected thoroughly

Some of you will probably say that this principle is obvious, but although it is, many testers are failing to understand the importance of this issue and as a result they failed to reduce the software failures. It’s basic, in 95% of the total testing cycle testers design their tests based on SRS and DDD documentation, as a result, most of the test scenarios will contain the expected results for a given operation, the tester who execute those tests must follow the expected outcomes and make sure it’s valid against the predefined expected results designed in the test case.

In addition, you always need to remember that validating the end results is only one half of this principle, sometimes the expected results from test cases that written based on pure documentation That in many cases has a missing information regarding complex test scenarios due to the limit technical knowledge of the product manager, as a result you must make a deep investigation to understand if the outcome in the “Dynamic” testing is based on the PM expectations, you should never make the assumption that the result is valid if it’s not written in the original documentation. 

Principle 4: Software Test cases must be designed with unexpected and Negative inputs

That’s classic, when testing a software our STD must include test cases that are written with both invalid and valid inputs, most testers ignore the first part (Invalid Inputs) and goes directly to the second part (Valid inputs), in that way the testers will cover the demands from the SRS documentation and that’s fine, but the Invalid inputs (In many cases) are the once that going to make the difference regarding the software stability and resilience.

Based on my personal experience in testing enterprise software’s I can say for sure that most bugs and the once that I really like to find as a specialist are discovered when my inputs are not the once that expected based on spec. Therefore, a decisive part from my test design is dedicated to write a complex tests with Invalid inputs that really pushing the system to the limits and as a result my developer got additional working hours, you need to remember that when a developer written is code he simply do it (in most cases) like a blind horse, as a result the requested functionality will probably work as expected, but when the same functionality is tested with inputs that our developer failed to understand and think  in the coding stage the errors will pop-up in a way that we couldn’t do with Valid inputs.

Principle 5: Test cases must be written with definitions for the inputs and the expected result

Although this principle may look obvious for most of you, it’s really important to understand the psychology behind it, when a tester that doesn’t familiar with is software or testing new feature based on predefined STD that failed to address the relevant expected results, you may fail to judge the actual results that you receive when executing the test case, some testers will take such output and say, Ok, that’s look like the correct outcome, we can approve this case and continue to the next one, well, such judgment is wrong, a tester will want to see the correct results may be fail to understand that what’s look right is still far away from the PM expectations.

Because testers are humans that make mistakes based on this basic psychology, we must remove this factor from our tests, to make such thing, we must reduce the “Human Eye” factor. Therefore, each test case must contain a detailed list of the expected results and inputs, in a way that a tester will execute the case without the option to think that “what he sees is probably the right thing”.

Now I want to clarify something, the human eye of a tester is crucial, otherwise testing could be made by robots, but when executing a test case that designed specifically based on SRS doc that contain the client design specifications, we cannot have deviated from the expected results, the human eye will help us in different ways such other impact we have while executing the case or any other issues that we see that simply doesn’t look right.

To summarize this principle I want you to think about the next point, if the test case designed with predefined inputs that led to a specific outcome, what the % for failure when a tester runs this case with detailed steps? Well, I think that when a tester can follow a detailed test case that contains the relevant outcome, the % for failures will be reduced to the minimum (based on the assumption that the tester who designed this case made is investigation).

Principle 6: No matter the number of tests you made, you cannot prove that the software is free from bugs

I hate this principle, from the tester perspective it’s simply says that no matter the hard job you do, you still cannot guaranty that your software cleared from all failures, that’s really annoying because you always have the chance that failures will occur after the testing cycle is done and the software delivered to the company clients. The solution? Change the state of mind, although you cannot promise that the software cleared from bugs you can point your finger to the bugs you discover within the testing process and say that those failures are removed and will not affecting the clients.

In addition, you already know that for me a great testing process cannot be accomplished without the knowledge about the risks (Risk Analysis), when a tester knows the risks he can simplify his test plan and cover the areas that most likely contains the critical/Blocker bugs (Risk Management), that leads to an efficient testing process that most likely reduced the risk for bugs after the end of the testing process.

Personal view: Mastering the art of “Risk Analysis” and “Risk Management” is the keyStones for a tester, without them you’re testing cannot be efficient and relevant in the software industry. 

Principle 7: Start your tests early as possible

It's simple fact that the company costs of fixing bugs that found on later stages in the “Software Life Cycle”, will be much higher than the once that found on the earlier stages. To overcome this problem we need to start our tests as early as possible in the “Software life cycle”, why? Because early testing focuses on the “Static” side of testing that is crucial when we want to find the defects on earlier stages of the tested software.

After I made the connection between this principle and the “Static” testing side, we can understand that the benefits are Hugh, just a few examples of the benefits you gain when starting your test early:
  • Find design bugs on earlier stages.
  • Perform the first stage of the validation process.
  • Understand the big picture and the complexity of the project.
  • Determine the test objects and scope.
  • Save the company time and money.
  • Design our test cases per each level of the software life cycle.
  • Understand the user requests and the company business case.

Principle 8: Software testing has two different sides, First, Examine that the software do what it’s supposed to do, Second, examine that the software doesn’t do anything that it’s not supposed to do


This principle is highly important when testing a software, a tester that failed to accomplish one of the two sides of this principle will be probably failed to deliver a high end product. Now let’s simplify things; the first half as I mention is dedicated to the verification that all the requirements and specifications design are functional and working without any deviation from the SRS documentation, the second part I mention is relevant when we want to see how the software performed when the functionality is implemented correctly, but in some situations the software performing in unexpected way that affect the basic requirements we tested on the first half of this principle (Think about an ATM that release money and on the same time update the account of another client, the ATM does what is supposed to do (Releasing money) but in the same time he does something that he is  not supposed to do).

Principle 9: testing is limited in time, you must design effective test cases


It’s obvious that any testing process is limited in time, as a result the test plan should be written only after the tester understands the “big” picture. Let me explain, based on my experience on major testing projects I become familiar with the fact that testers create their test plans without the simple idea about the company business case (Why do we need this software?)  And in the worst case without the full info about the user expectation that leads to poor design of the test plan.

Based on this simple fact, we can say that effective test plan will be determined by the information we gain in the SRS documentation, if the spec defines the precise and full requirements of the software we can design our test cases without the need to add the missing information that in many cases will lead to irrelevant test cases and waste of critical testing time.
To simplify this principle, I added few examples that should lead to a better efficiency when creating the test plan:

  • Test cases should be written only after the tester becomes familiar with the SRS documentation.
  • Missing SRS information should be completed by the PM and NOT by the tester assumptions.
  • It’s critical to understand the user expectations and the company business case.
  • The majority of your Test plan must be written before the actual testing execution.
  • The tester must know the software architecture and supported platforms.


Principle 10: Defect clustering

Defect clustering…? Very simple, every software is built from a few integrated components, based on this principle we can say that specific modules will have most of the defects we will find on the pre-release testing. 

Based on my large experience I can say that this principle is really important because it’s can change the testing effort in a way that if you found such module you probably need to add additional testing effort to remove the risk.

Principle 11: software testing an endless process that should be stopped

We all know that testing procedure that executed on a complex software can take an unlimited amount of time (the testing matrix is endless), as a result we need to decide when to stop the tests and what the consequence of our decision.
The available options for stooping testing are:
  • Timelines
  • Project budget
  • Acceptable level of risk
  • Testing coverage based on requirements

From the list above, you can see that every bullet affects the other once, and why it’s so important? Because when we need to stop our testing, we should understand the outcomes, the classic example is to stop the testing cycles due to pressure timelines and taking the chance that the software will have bugs that may influence the user experience (Lower quality against product release). 

Thursday, May 8, 2014

QA – Graphical User Interface Testing (Advanced)

Overview
The Graphical user interface (GUI) is the main platform we use to manipulate a given application, a great UI that was coded and tested efficiently will allow the end user to perform a highly complex operation without a complex set of tasks.

When talking about a GUI testing, we need to take in mind that our goal is to help the user to perform his tasks with the easiest way and most importantly in a way that saves is time, among the different criteria’s that we need to consider:
  • The Graphical interface must simplify complex tasks.
  • The Graphical interface must have a logic in the way it’s built for use.
  •  Multiple functionalities that doesn’t relevant for the end users should be removed, that’s allow the client to concentrating on the functionalities that he really needs without wasting time in figuring the once that not relevant for him.
  • If the user need to learn the software Graphical interface from 10000 pages of “User Guide” than this GUI is simply not a usable interface.
  • Continuing the previous bullet, Graphical interface must contain the functionalities that requested by the client, the GUI that failed to contain the basic requirements and design specifications is not a valid one.

Personal View:
In this kind of test we must make sure that the user interface functionality, feel and look are coded with the exact behavior as defined in the SRS documentation that contains the software requirements and design specifications. The GUI functionality should do what it’s supposed to do and nothing else (Positive and Negative tests), you need to remember that UI functionality like buttons or text Fields can receive Valid and Invalid input from client. Therefore, you must be sure that a UI object will know how to handle this inputs and design the relevant test cases.

The Challenge in GUI testing
Software testing is a very challenging task, among the multiple testing types involved in the process we can find the “Graphical User Interface” tests that raise a major challenge in every software testing Lifecycle.

It’s well known that in the past 30 years the “Graphical Interface” changed dramatically in almost every aspect, today “Graphical User Interface” is written in highly advanced coding languages that allow the company developers to create a highly complex interface that involved different layers of design, functionalities (with millions of available parameters that can be sent as input) that cause the interfaces to behave differently on different platforms that support our software.

The main problem is that every device that compatible with our software may behave differently when integration made with the software, and change the total look and feel when using it.

 In other words the challenges for testing today are determined by two major aspects: 

User interface complexity–As described in the paragraphs above, the software industry can produce a very complex GUI (that contains infinite scenarios and combinations) for the end clients. Therefore, it’s a simple fact that the testing complexity is cut based on this parameter (GUI complexity = More tests).

Supported platforms (Hardware/Software) – not like 30 years, today's applications become relevant to almost every device and software platforms, the more platforms we need to support more testing is needed, just for example, think about an application that should be compatible with the following platforms:

Hardware platforms:
  • Mobile Phones
  • Personal Computers
  • Tablets
  • laptops

 Software platforms:               
  • Web browsers (Firefox, IE, Chrome…).
  • Phones OS (Android, WinOS, MacOS…).
  • Different OS systems (Win 8, Win Servers, UNIX ….)

Well, as you can see the matrix is infinite (and that’s before we described the actual software we need to test) therefore, when testing the “User Graphical interface” we need to calculate and be familiar with all aspects and only then, we can start designing our test plan and coverage matrix (I will expend this point in the next paragraph).

How much testing is enough?
Based on my experience, I can say for sure that many testers have major issues when performing UI testing, the main reason is that when executing such tests, it’s really easy to get carried away and waste a critical testing time that should be saved for additional testing types that cover different layers of the application, but here come the main question, how much testing is enough for UI testing based on the importance of such component?

The simple answer will probably be that just like any other testing types, you can say that you do enough testing only when all the risks you analyzed at the beginning of the testing process are removed. But wait, it’s not easy as it looks, to understand the risks (That involved additional testing aspects like the form Fell/look of the tested forms) you must do a skillful “Risk Analysis” and conclude those aspects, if you failed to do so you probably go to the wrong direction and test the applications while executing thousands of tests that written from the point of view of coverage(that you never have the chance to finish within the testing timelines…) and not from the point of view of an efficient testing(less testing scenarios but the once that written will provide a batter and efficient coverage) that integrates multiple testing objects.

Automation or manual tests..?
To answer this question we need to consider few parameters that I already covered at the beginning of this article (To make it easier to review them here also), the parameters are:

  1. Today “Graphical User Interface” are very complex for testing and contain an infinite supported scenarios (Based on my experience, most GUI bugs are a direct result of a tester who sees the GUI testing in a na├»ve approach of clicking few buttons and see if the GUI functionality works).
  2. GUI testing is one of the most time consuming tasks in the software industry.
  3. To make an efficient GUI test, the tester should consider additional aspects that not relevant to other testing types (among those aspects we can find the feel and look of the tested software).
  4. Depends on the GUI complexity, writing automation test cases could take an infinite amount of time, as a result, we must select the test cases that really important for the regression cycles, just like manual testing, we cannot cover everything using auto tools.
  5. Continuing the previous parameter, Automation can cover major aspects of the tested GUI, but few GUI cases cannot be automated (can we test different graphical aspects that involve colors that may be changed during the software testing Lifecycle..?).
  6. The GUI is one of those components that could be changed multiple times during the testing life cycle, as a result, we need to save a large amount of time to maintain the written automation, in some testing cycles the maintainer of the automation can take as much as three times from manual testing and as a result become ineffective.
Based on the list above, you can easily understand that you need to find the balance between the test cases that can be automated and the once that should be tested by the human eye. My suggestion for you is to make manual testing of new components, but afterwards move each case to automation tools. Repeating UI tests can be really painful in the @#$. 

The basic steps for GUI testing
GUI Testing may involve few stages that relevant to another testing types that we all familiar with , My example here is really basic and could be adjustable depends on the company needs and testing strategy:
  •  The basic first steps is to do your research (That mostly relevant for the “Static” testing type), few examples:

o   Read the SRS and DDD design docs.
o   Understand the client needs.
o   Understand the business case.
o   Understand the timelines for tests.
o   Do the research for AUTO tools.
  • Perform the relevant “Risk analysis” and “management”.
  • After you finished your research (and understand the risks) and you have all the info, you need to determine the “Coverage” criteria (As explained today GUI contains an infinite number of tests that cannot be covered).
  • Determine the testing strategy (Sections that going to be automated or tested manually).
  • Create your STD with specific test cases (every test in you STD must contain the expected result) that includes tests based on GUI objects (I’ll explain this issue in the next paragraphs), each object must be tested with the relevant events related both Keyboard and mouse (Mouse Clicks, Keyboard shortcuts, menu manipulations, GUI functionality, Overall look….).
  • Deliver the relevant test cases to the automation team.
  • STD execution and verification.
  • Analyze the testing results and coverage to determine the remaining risks (I’m telling you right now, the coverage will not be 100%..., if you performed a flawless testing the coverage will be among 60%).


Recommendations for User interface testing:

The human eye - GUI testing should be made by human beings, the perspective and sense is different from an automation procedures, my recommendation is to test every part of the application and only after you confirmed that it works as expected we can move it to the automation team and forget about those tests.

Personal View: Repeating Graphical tests is not a nice task, GUI testing is perfect for automation and key part from every regression. Therefore, try to automate as many test cases available.

Choose the right direction(Coverage vs. efficiency) – I already mention this point on earlier paragraphs, but it’s a really critical point in GUI testing, as explained Graphical interface testing may involve an infinite testing matrix that you cannot cover, but in the same time graphical bugs are not something that you want to release to your end clients (Think about an enterprise product that have GUI bugs, it’s may severely affect the company reputation because it’s the basic things that must work. In addition, from the user perspective, if you fail in the basic stuff, you probably failed also on the performance and functionality side), to reduce those faults you must plan your testing activity in a way that your testing is efficient and not based on 100% coverage that you cannot accomplish in 100 years of testing (Efficiency = based on appropriate Risk analysis and Risk Management).

Separation of GUI objects - Graphical user interface is a set of objects that located on a host platform, when designing your tests you should consider every object as “Stand Alone” and ask yourself few questions that going to help you design your STD:
The object business case?
Why we need it?
Is the location is the right one?
Available Inputs (If Supported)?
Integrations with different objects? 
What are the outputs we need to get while using that object?


Few Examples for Graphical User interface objects:
Context menus
Buttons
Icons
Help icons
Dialog poxes
Help menu
Links
Legend menu
Tab sequence
Maximize /Minimize.
Checkboxes
Combo boxes (Default value as selected value)
Search Boxes
Sorting / Groupings and columns manipulations.
Increase / Decrease

Example for Graphical User interface parameters:

Object visibility:
Color
Template
Width
Length
Controller’s position
Grid size and position
Controller’s size
 Picture size and shape

Object Syntax:
Size
Length
Spellcheck
Fonts
Capital/ Non capital 

The Software industry perspective(The expected standards from a new software) - Every Graphical user interface must be tested with few basic standards that we use in the software industry (although the SRS doc is the one who determines the end design), every tester who involved in GUI testing must be familiar with the software industry basic expectations and
standards from a new software,  this issue is really important both in the static testing procedure (Industry expectations will lead to SRS changes if needed) and the “Dynamic” testing (The tests performed from both the company and the industry perspectives). 

Examples: 
  • Keyboard button “F1” should point to user help guide on windows platforms.
  • Every field that used to find values should be called “Search” and not “Find”.
  • In Windows OS the “Ok” button will be on the left of the “Cancel” button (The opposite behavior from MacOS). 

Rules – GUI testing should be tested with few basic rules that must be considered when designing and executing test plans, Classic Examples:

  • Validation process must be executed - More functionalities are not stating that you built the correct software (Validation) – the validation process is made to guarantee that the software meets the client expectation, too much functionality that not relevant to the client will lead to multiple questions and misunderstanding of the functionalities that he really needs. Therefore, a good validation process should be made to ensure that the client receives the functionalities he needs and the one that not relevant will be removed.

  • Usable – The GUI is the main console that the user going to use when working with the software, your tests must involve additional layer that guarantee that the user can use the software, those tests called “Usability” testing and should be used as additional layer in your testing plan.


Checklists and Guidelines – GUI testing may involve thousands of tests that performed on different layers of the application, sometimes it’s basically impossible to remember the parameters we need to follow in our tests, as a result, we should use a predefined  “Checklist” and “Guidelines” in our test cycles  that ensure that our Graphical interface tested professionally and efficiently.

As explained in the previous paragraphs, we need to see our graphical interface as a list of objects, as a result we need to understand that some checklists are relevant for all objects (Color\Syntax...) but some checklists are relevant for specific objects only (Functionality, Events (Clicks, moving the marker)…).

The following list is designed to handle few mandatory objects that relevant for most applications that are used in the software industry (To create this list I mostly based on Windows operating systems), my recommendation for you is to take a really big breath before you are reading the list because I promise you that this list will change the way you are testing your graphical interface from now on.

Checklists and Guidelines that basic for every application:


General 
  • In any case of “Disabled” field the user cannot use the functionality, in addition “Focus” on that field shouldn’t be available.
  • Every software that supports the “Help” functionality should be triggered from the “F1” button.
  • Is there a default object that highlighted when the user starts the application?
  • Try to start new instance of the software while the first instance already in active state.
  • Web applications should be tested with different resolutions (600*800, 1920*1080…).
  • TAB functionality  should be available with the order of left to right(TAB – Moving forward  / SHIFT + TAB is moving backwards)
  • In most cases GUI forms should have the Minimize/Maximize options.
  • Try to run the application from different locations (Client<->Server).
  • Closing the application shouldn’t occur without an approval notification that allows the user to “Approve” (OK) or “Decline” (Cancel) is operation.
  • In most cases the application integrated with a dedicated Database (Sql, Oracle…), in many situations the application DB requires a value from the user (the expected Value cannot be empty) therefore those fields will be marked with default values (GUI) and if not specified differently by the user those default values are going to be the one that will be used by default.
  • In any case that cause the application to move into “Loading” state it should display a relevant notification for the user that the current state of the program is “Processing”.
  • Text should be justified (otherwise it’s not relevant).
  • It’s basic to try to start the application few times while another instance in “loading” process.
  • The main functionalities of the application should have a corresponding “Shortcut” key that user can trigger from his Keyboard.
  • Make sure that one form buttons doesn’t affect sub-form when it’s not needed or specify.
  • The application name should be displayed on the application main form.
  • Each application should described with the version number.
  • In most cases you should have the ability to perform every operation from the Key board and without the mouse.
  • In any case that the changes made by the user are final, make sure that the “Cancel” button reacts as “Close” button.
  • The “Help” menu should be available in the main screen navigation bar.
  • Applications should start based on few triggers:

§  Double click on the application icon.
§  Double click on the application shortcut.
§  Right Click on the application icon -> Select the application
§  Command line (With a dedicated syntax).
§  Script file (With a dedicated syntax).

Color:
Color tests are relevant for all forms, the following list contains the main questions you should ask yourself when testing the GUI from this side:

  • Do the text inserted in text fields are in the correct color?
  • Do the form objects (Drop-Down list, Radio button) displayed with the correct color?
  • Are the Hyperlink colors are in the expected color?
  • If fields become “Grayed-Out”, do we display the correct color?
  • Loading screen displayed with the correct color?
  • Loading process bar in the correct color?
  • Form title and description displayed in the correct color?
  • Form background color is the correct one?
  • Are the buttons are in the right color?
  • Is the sub-screens in the correct color?
  • Are the “Warning” notifications displayed with the relevant color?
  • When a field in focus, do we mark it with different color?
  • If form or field state is set as “Read-Only”, do we grayed them out?
  • In general, are the application colors are usable for the client? 

Syntax:
Every application contains different aspects of syntax, the following list is relevant when testing a software from this side: 
  •  Is the screen text being written with the correct font?
  • Is the screen text being written with the correct size?
  • Is the text in all objects are written with the correct font? 
  • Is the text in all objects are written with the correct size? 
  • Are all the screen texts are aligned correctly?
  • Are all the screen texts in the text box are aligned correctly?
  • Spellcheck tests are basic...
  • Is the first char (If relevant) in a word set as capital?
  • Does the second word should start with Upper/Lower case?


 Checklists and Guidelines for specific Objects:

Validation fields(User Credentials is classic example)
  • All mandatory fields must contain values, if the user failed to insert valid values the validation process will raise a notification that explains the error that allows the user to supply the missing value.
  • If the form contains mandatory fields it’s usually marked with asterisk (*), but that’s not all, most users will not understand the meaning of such mark. Therefore, you should add a note that explains the values that user should enter.
  • It’s highly recommended that the authentication object will use an additional password checker to make sure that the user insert a valid password and save the actual validation time (Enter Password + Re-Enter password).
  • In a password field, it’s a good idea to use a “Password Strength” checker, to notify the user that is password is not secured enough (Mostly used on WEB authentications when creating the first registration).
  • Different “Injection” tests are really important for validation screen, you need to check that your validation screen is protected.
  • If mandatory field remain empty and user press the “Ok” button, the application should mark the empty field to direct the user to his fault.
  • Do the validation fields are “Case Sensitive”?
  • In most cases the validation fields should be limited with MAX and MIN values (Boundary tests are classic for such fields).
  • Negative testing for such fields are crucial, validation must be failed if user insert invalid values (In the worst case scenario user can insert invalid values and the validation pass).
  • In any case of invalid validation, the user should notify that the validation process failed with an appropriate notification.
  • If the validation process limited to a MAX number of failures, the user should receive a valid notification that explains the remaining number of authentication attempts, if the user failed to authenticate within the limited attempts a valid notification should be presented and explain the current state.
  • If the user failed to add an appropriate value in one of the fields and press “Ok” in most cases he cannot edit the field (That cleared from the inserted value) and should re-enter a new value.
  • You must understand if the validation performed on the client side or on the server side it’s really manner when testing web validation screens.
  • In any case that the user press the “Cancel” button the form should be closed and the validation should not take place.
  • Check if the validation fields should support a specific format of values.
  • Does the SRS doc, specified that the authentication support negative values?
  • Does the SRS doc, specified that the authentication support Special characters?
  • What should be happened if user insert a valid valued and the validation completed successfully?
  • Do we make the validation against any kind of Database? If so, you must check if the DB columns allow “Null” values.

Radio Buttons
  • Every button should be available for selection both by using the mouse and keyboard (Arrow keys should allow the navigation between the buttons).
  • When using this object we need more than one available option for selection. Therefore, in any given state only one option should be selected.
  • By default at least one button should be selected. 

Push buttons
  • Esc should active the “cancel” button (if available in form).
  • Every button should be triggered by the mouse when user clicks on it.
  • Continuing the previous bullet, the “Space” key should do the same action.
  • Every button should have the option to trigger with an appropriate keyboard Shortcut (You must make sure that duplicate shortcuts are not existing).
  • Make sure that all buttons are similar in size, shape and size.
Drop down lists / List Boxes /Combo box
  • The “Drop Down list” is an object that doesn’t receive any typing inputs from the user (In other words drop down lists are not text boxes).
  • Pressing the arrow of the drop down should open a menu with all the available options for the user.
  • If the list contains multiple values, the list should be scrollable.
  • Drop Down values must be presented with order, in 90% the order determined alphabetically.
  • The User should have the option to change the values' order using different options of sorting based operations.
  • The list should be responding to user manipulations, when user press a character the focus should move to the first value that start with the same char.
  • When the drop down in focus the Keyboard combination of Ctrl-F4 should open the list of values.
  • When the user selects a value, this value should be displayed on the main Drop Down field.
  • Not like the first two objects, In Combo Box, user should have the option to insert text.
  • Drop Down objects doesn’t need to set with default values.


Text Boxes
  • Double click on the text should highlight the entire syntax.
  • Text box must support Copy/Paste of syntax from different locations.
  • The User have the option to enter text into the box.
  • Testing for text box must involve negative testing that indicates that the field behave correctly when user insert invalid characters.
  • The User should have the option to select specific text when using the Shift + Arrow Keys.
  • Try to use Blank fields and sync the value to DB.
  • In most cases the “Save” button should be grayed out if the user not yet insert syntax to the text box field.
  • Enter syntax in the text box with space at the beginning.
  • Enter syntax in the text box with space at the end.
  • Do we support upper and lower case?
  • Text box should be limited to Max characters.
  • The basic test for this object is to overflow the text box by typing the MAX available of characters.
  • Every text box should be tested with special characters.


Date and Time fields
  • Applications must be tested with OS “Time Zone” changes, different components that integrated with different time zones mail lead to massive failures in the data synchronization and integrity (Believe me I know it personally…).
  • Can you change the date/time (insert day in the year location, insert year in the month location...) order and approve the change?
  • Change Time zones in specific components to see how the application can handle different date formats.
  • Make sure that the application can handle any case of “Leap” years (if your application failed to handle such cases an appropriate notification should raise for the user).
  • A Different date fields contain MAX and Min values, you must make sure that negative values will not raise any error(Boundary tests are classic..) and appropriate notification displayed to the user, Examples: 

Value
Day
Month
Min
1
1
Max
31
12
Special 1
Value > two chars
Value > two chars
Special 2
0 or 00
0 or 00

My Presentations