Testing

What is Testing?

Testing is a process of validating and verifying that a technology solution:
  • Meets the requirements that guided its design, configuration and/or development;
  • Works as expected; and
  • Satisfies the needs of the stakeholders.

Why is it important?

Testing is the means to identify unmet requirements and defects prior to Go-Live. It is much more costly to fix issues once a solution is implemented. In addition, you risk a loss of public confidence, particularly if they are the ones identifying the problems. Issues impacting data security can have even more severe consequences, resulting in confidentiality breaches or even identity theft as a result of compromised data.

Why test Commercial Off-The-Shelf (COTS) software that has already been successfully implemented at other organizations?

The solution may work at other organizations, but may not:
  • Meet all of your requirements;
  • Work as you expect; 
  • Be configured to use only the features/functionality that you want to use in the way that you want to use them; and  
  • Be compatible or integrate with your other systems and interfaces.

It is important to test a COTS system prior to implementation, but your testing strategy will be different than testing custom development:

  • Base tests on functional or business processes used in your real world environment,
  • Design tests that are important to how you will use the product,
  • Ensure you test the functionality configured specifically for your business, and
  • Identify where the highest risks are (in terms of the biggest negative impact and greatest likelihood of occurrence) and ensure those areas are thoroughly tested. 

Types of Testing:

There are many types of testing. You need to determine what is appropriate for your project. Here are some types of testing to consider:
  • Unit Testing: This testing is performed by the software developer or for COTS systems by the vendor who configured the solution, to test their own programming &/or configuration prior to providing it to the customer for testing.
  • User Acceptance (UAT):  In this type of testing, Business users create and test scenarios to ensure a solution fulfills their specific business/customer requirements.
  • System Integration Testing (SIT): This is testing to ensure that a solution's components/modules are successfully integrated.  
  • End-to-End Testing: This testing is used to test the flow of data through an entire business process lifecycle. It tests how actual data will flow across applications. For example, a bank might test the lifecycle of a loan from origination to pay-off. 
  • Regression Testing: This testing occurs to verify that changes made to a system application (i.e., to fix a bug or address a new requirement) did not inadvertently introduce any new bugs. 
  • Conversion Testing: This type of testing needs to be performed if you are converting data from an old to a new system. It involves a:
    • Reconciliation of data to ensure all the data was successfully transferred to the new system. 
    • Verification to ensure the converted data is interpreted correctly by the new system and is available for use as needed.  
  • Performance/Load Testing: Generally executed to determine how a system or sub-system performs in terms of responsiveness and stability under a particular workload. It can also serve to investigate, measure, validate or verify other quality attributes of the system, such as scalability, reliability, and resource usage.
  • Security Testing: This testing is performed to verify the solution and any configured controls are adequate to prevent unauthorized internal or external access.

Special Considerations for Testing Web-Based Applications:

When you have a web-based product, you need to keep in mind what browsers might be used. A product may perform or display differently in each browser, so you will want to test in each of the highest volume browsers at a minimum.    
You will want to test at a minimum the most recent released version of each type of browser. You will want to identify which browsers you will want to test. This list might include:
  • Internet Explorer
  • Firefox
  • Safari (for Windows)
  • Chrome
Following is a list of items to take into consideration, when testing in each browser:
  • Each page/screen displays correctly (not too much white space)
  • Ensure that each field on each screen displays & performs correctly
  • Navigation through each screen is consistent
  • Group boxes appear correctly
  • Navigation sidebar displays/performs correctly
  • A task can be performed (i.e. saved) correctly