We hope to never experience them, but we do. From ransomware attacks and prolonged power outages to natural disasters and acts of war, disasters vary in form and magnitude.
In the face of an intensified threat landscape and other events that can disrupt the flow of business — as well as employees’ and customers’ lives — credit unions need to ensure IT resiliency to help protect their data and quickly recover if and when an incident or disaster occurs.
According to a 2022 Veeam survey, 40% of servers suffered at least one outage per year, and a 2021 Coveware survey indicated the average downtime after a ransomware attack was 23 days.
The cost of downtime doesn’t just come from loss of current and prospective business. Loss of customer trust and loyalty, loss of employee productivity, compliance issues and penalties, and the time and money it takes to remedy the situation all take their toll. In fact, 25% of respondents to a 2020 ITIC survey indicated the average per-hour cost of server downtime was between $301,000 – $400,000, with 17% reporting their average per-hour cost to be more than $5 million.
The best way to efficiently recover from a disaster, minimize downtime costs, and protect your credit union, its customers, and its data is by being prepared and having a game plan. The following guidelines will get you on the right track to developing your credit union’s unique disaster recovery plan to better face the unpredictable.
Step 1: Identify
Start by identifying your assets, including data, systems, applications, and stakeholders, and their locations. Networking configurations are also essential for proper application communications and recording these will help assist during recovery. This allows you to accurately define recovery strategies and recovery locations for each site in your organization.
Step 2: Define
Risk assessments and business impact analyses (BIA) are crucial in helping you to determine disaster potential and impact, what necessitates activating a DR plan, and the critical revenue or performance indicators. Once this is completed, it will enable you to tier your applications, allowing disaster recovery processes to recover systems in the correct order according to business objectives.
Determining RTO (recovery time objective) and RPO (recovery point objective) is essential to disaster recovery overall. RTO is a measure, in time, of how highly available your applications need to be — the shorter the amount of time, the more refined your technologies and processes need to be. RPO is the acceptable amount of data your organization can lose, also expressed as an amount of time. There is going to be a difference in your recovery response if you can weather losing two weeks’ worth of data versus two days’.
All of this helps you determine your failover plan which puts everything into action. Next, you’ll want to consider your failback plan, or how you’ll get back to your production environment.
Step 3: Document
Document all the information you have defined and gathered thus far. Remember: this will be read during a crisis, so avoid using slang or jargon. Assume that no one has access to key contact information and document this, as it is essential for cross-organization communication and third-party assistance. Set up call trees for each department, and don’t forget to include vendors, support numbers, and any current contractors. Creating checklists for each department or application for use during recovery also helps ensure items are not missed.
Giving a current copy of your disaster recovery plan to your DRaaS provider ensures you always have an accessible off-site copy and have the best response possible.
Step 4: Test
Don’t wait until an actual disaster event occurs before you test out your recovery plan — testing is a critical part of disaster recovery planning. It helps you know if your documentation and processes make sense and are complete, as well as if backups are reliable. Test quarterly, annually, or whenever significant changes occur. Allow others to review or test the plan to help prevent confusion and to find anything that was potentially missed.
Step 5: Refine/Revise/Repeat
A disaster recovery plan is a living document that requires routine updating. Changes such as employee turnover, software, or overall business objectives will affect your plan. After making any changes, have someone review it again — accidentally deleting one paragraph could substantially alter the entire process.
For more details on these steps, a disaster recovery plan checklist, and a business continuity and disaster recovery tabletop exercise you can perform with your team, download the CIO’s Guide to Disaster Recovery Planning.
There’s no better time than now to solidify your credit union’s business continuity and disaster recovery plan. Doing so will greatly help minimize downtime that impacts your organization and your customers and bring efficiency to recovering data and applications in the event of an outage or disaster.