Operational Acceptance Testing

Ensuring that a server, network or software can be patched once in a productive state.

Patch updates delivered from the vendor are required throughout the life of the software.

Being able to patch the solution without having to undertake a large project is critical for maintaining the software, particularly for security patches.

This testing ensures that you can patch a server in production. This will identify what type of outage would be required, providing confidence that this can occur in production within an outage window, reducing the impact on the business.

Most software systems have installation procedures that when followed render the system available for use. Testing these procedures to achieve an installed software system that may be used is known as installation testing.

When we test a release, you may need to roll back, this is often performed during the delivery lifecycle to ensure that Developers who release their code are able to back out their code using a simple process.

With the increase of CI/CD, this process is usually built into the development pipeline solution and tested as part of the management of these tools. An installation however is a little different, this is usually when you are testing an installation of a solution and ensuring that it can be uninstalled without leaving any residual files.

Logically, it’s fundamental that you make sure that users do not have any trouble when installing your software. If your software’s installation is successful, your customer will be satisfied, but if the installation fails, not only will the software not work on that system, but it could have damaged the user’s system. Of course, this is not a desirable position to be in.

Installation testing has become more complex with time, this owing to the many different ways software can be distributed nowadays. Whereas in the past software was distributed in the physical format of disks, nowadays it is commonplace for software to be installed from the internet, a network location or pushed to the user’s machine.

Ensuring that a solution can be upgraded to a later version once in a productive state. Often when a solution or package is implemented (excluding SaaS or PaaS), it often needs to have regular upgrades to maintain the functionality, performance or security of the solution.

This testing ensures that you can upgrade the solution once it has been released into production. This will identify what type of outage would be required, providing confidence that the upgrade can occur in production within an outage window, reducing the impact on the business.

With the move to cloud and cloud services being reasonably stable, we are seeing this type of testing being shifted to the cloud hosting provider however it is essential that organisations have tested this to ensure business continuity.

Simple backups should be tested frequently, at least one a quarter and whenever this is a major hardware or software change to your backup system. It is important to run a test after upgrading your firmware in your backup system to ensure that the new firmware works as expected.

Often performed as part of the Disaster Recovery testing, this testing provides you with the confidence that you have your system and data restored in the event of a failure.

Often coupled within the Disaster Recovery testing, failover and recovery is essential more so given the level of complex integrations that organisations have.

Failover in computing terms is when the solution switches to a redundant or standby server, system, hardware component or network if there is a failure or abnormal termination on the active server, system, hardware or network. Failover is an automated process an operates without warning.

Failback is the process of restoring the server, system, hardware or network back to it’s original working state.

The purpose of Disaster Recovery (DR) testing is to assure that information technology systems can be restored if an actual disaster occurs. As part of a DR plan, companies typically engage a disaster recovery team, whether this is internally or specialists to recover the systems.  Effective disaster recovery testing can reduce downtime and save your organisation time and money if an actual disaster is experienced. There are a number of various methods that can be applied for a DR Test:

• Paper test: This is where the test plan is read, and recovery plans are reviewed
• Walkthrough test: A group of people come together and walk through the plans to identify issues and identify changes that may be needed.
• Simulation: A group of people come together and walk through business scenarios to better understand whether the plan caters for these various situations. This is ensuring that emergency response plans, call trees are up to date and adequate.
• Parallel test: Recovery systems are established and tested to see if they can perform business transactions to support key processes. Primary systems still carry the full production workload.
• Cutover test: Recovery systems are established to assume the full production workload. All primary systems are disconnected.

Some simple steps can be followed to identify any errors during individual tests ad correct those before you do comprehensive tests. This saves time and the costs associated to errors interrupting a comprehensive test that engages a lot of people.

1. Understand how frequently you should perform each type of test
2. Test individual components – ensuring that the component can restore
3. Perform wider tests including multiple components – ensuring that restoration at a point in time is possible and any interfaces can recover without loss of data
4. Test the entire DR Plan, including all call trees and manual activities