CUs, Fintechs Step Up Disaster Recovery Preparations

Extreme weather-related emergencies precipitated by hurricanes, tornadoes, wildfires, mudslides, flooding, heatwaves, polar vortexes and bombogeneses have prompted credit unions and fintech suppliers to step up their disaster recovery and business continuity planning efforts.

The severity of recent events has heightened the urgency to prepare for the worst. “Weather-related events such as hurricanes, typhoons, cyclones and severe snowstorms are on the rise and have the potential to significantly impact an organization’s ability to continue business as usual and service its clients,” Gillian Catalano, principal, business resiliency services for Finastra, said.

Eric Flick, director of Centurion Disaster Recovery at Jack Henry & Associates, pointed out the company has seen the number of incidents affecting its customer base increase as supported by National Oceanic and Atmospheric Administration research.

“These events often can have ripple effects, felt hundreds of miles away.” Flick cited the record snow that fell this winter in the Twin Cities area of Minnesota, which subsequently caused near record flooding along the Mississippi River.

“We take a proactive response by working with credit union clients year-round on their overall recovery strategy, not just at the time of the incident,” Flick explained. “It’s our job as technology service providers to ensure credit unions can best serve their members when disaster strikes.”

The importance of thorough preparation rose following Superstorm Sandy in 2012. National Geographic called it a “raging freak of nature” and said it exposed the vulnerability of wider areas of the U.S. Sandy caused at least $70 billion in damages; flooding, mudslides and destructive winds along the East Coast; and outages as far west as Michigan.

Michael Terry, who works in IT managed services for Fiserv’s Credit Union Solutions, observed prior to Sandy that the company looked at a possible disaster zone that might impact a handful of clients. “When Superstorm Sandy came in, now we had 50 clients potentially impacted. We have gone from looking at it from a micro level to a macro level in just the last few years.”

Terry explained Fiserv tailors disaster recovery applications to each credit union location. It monitors for floods, tornadoes, hurricanes, earthquakes, wildfires and other events that could affect clients. “We’ll let them know the process to declare a system emergency, start spinning their servers up and stay in contact with them as much as we can.” He also advised once the credit union is ready to fail over, Fiserv cranks up backup systems and contacts third parties usually within 15 to 30 minutes of when the event hits.

Terry said Fiserv has redundancy built into its own systems, “especially after Superstorm Sandy.” At least twice a year, Fiserv completes disaster recovery exercises within the Credit Union Solutions division.

“Our platforms give financial institutions the ability to make nearly instantaneous changes,” Larry Edgar Smith, SVP of product evangelism for Temenos’ Lifecycle Management Suite, said. He added if members need quick access to funds, a credit union could move quickly and create an online loan application within a day.

A financial institution can also pause collections efforts in certain ZIP codes, Kris Frantzen, Temenos’ senior product manager, explained. “Credit unions on our platform can easily provide members access to online and mobile options, giving them anytime/anywhere access despite the circumstances,” Frantzen said.

Venessa Peters, assistant manager, collections for the $2.4 billion, Austin, Texas-based University Federal Credit Union, a Temenos customer since 2006, said the credit union has gotten up close and personal with disaster recovery (DR). In recent years, UFCU has had members become adversely affected by Hurricanes Harvey and Michael, the federal government shutdown and wildfires. “We responded with programs specifically designed to address the needs of those affected. Programs have ranged from loan payment deferrals and temporary adjustments, to loan terms to special low- or no-interest loan products,” she said.

Extra Credit Union ($241 million, Warren, Mich.) needed a backup plan, literally, as it previously had no redundancy or failover strategy. Its solution was to green light implementation of US Signal’s Remote Monitoring and Management platform, Disaster Recovery-as-a-Service and Backup-as-a-Service.

Paul Shammas, IT consultant for Extra Credit Union, said, “We’re regulated just like every other credit union in the state by Michigan’s Department of Insurance and Financial Services. One of the things they speak about is having data geographically distributed.”

Shammas said Extra Credit Union also wanted to make sure it could jumpstart its systems to service members. “If there’s a disaster, I can point to my new environment in less than 15 minutes.”

Extra Credit Union also tests its environment monthly. This past March it set up its whole environment to mimic a natural disaster. “We were able to get everything up and running at our failover branch within minutes,” Shammas said.

Extra Credit Union also has backing from its core provider, the Farmington Hills, Mich.-based CUSO Member Driven Technologies. MDT recently migrated all Western-based clients to its Montana-based second data center, taking a big step in strengthening disaster avoidance and business continuity.

“We provide assurance to our credit unions, including Extra Credit Union; we take it very serious,” Scott Johnston, EVP/COO of MDT, said. “Redundancy is really a mandatory thing these days, due to changing weather conditions.”

“It’s not just the core system anymore, it’s also all the ancillary systems that are just as critical,” Johnston articulated. “If it is a multi-day or longer duration type outage, the credit union still needs to be able to conduct business. We replicate critical services that provide the lifeblood to the credit unions in both data centers.”

Johnston also emphasized MDT constantly rehearses DR. “The critical ingredient there is to test it, and test it and test it.”

The Chicago, Ill.-based, $11.6 billion Alliant Credit Union, a digital credit union with two brick-and-mortar Windy City locations, shored up its business continuity plan by selecting Commvault Complete Backup & Recovery to protect it from unexpected events.

Matthew Tyrer, senior solutions manager for Commvault, said, “Having a solid disaster recovery plan is key to the success of any enterprise, yet our research has found that only 25% of companies are ready for a natural disaster.”

Marty Yurcheshen, senior systems engineer for Alliant, said the credit union performs biannual tests and rotates different teams through to run the practices. In disaster recovery tests, Alliant has recovered data 90% faster and decreased backup time from 30 hours to four hours using Commvault.

Alliant has also taken advantage of Commvault’s Live Sync. Yurcheshen said, “It allows us to do a quick failover. We can actually run two Commvault servers.” One continues to do the backups; the other one can be up and online, but in test mode.

Julio Arevalo, manager, systems engineering for Alliant added, “If you’re going to ask Marty, or myself, is our data secure? Is it retrievable? Can we restore? Yes. We can say with confidence that our data is going to be available for our members.”

The 2019 polar vortex tested Alliant for real. “We ran everybody remotely. Our online banking continued to run, everything stayed up and all of our people were able to work from home so they didn’t have to go out into that weather,” Yurcheshen said.

Disaster recovery and business continuity are an integral part of Finastra’s business resiliency program, Catalano held. “Our plans include strategies to enable the continuity of operations and geographical redundancies during weather-related events.”

Credit unions must do their part as well. “It’s important for our clients to have detailed response and recovery plans in place,” Catalano noted. “Clients should consider developing a business impact analysis to understand what is critical to the business and a threat risk assessment to understand the potential threats to their critical operations.”

Terry maintained that when it comes to credit unions, preparations vary. “Some clients will do as little as connecting tech and checking a few balances; others will go the other way and do a full disaster recovery exercise. When you do your DR test with your vendor, work it as deep as possible.”

Terry also acknowledged these tests will never cover everything. “But if you can cover as much as you can, it’s just going to make things a lot easier.”

Flick added, “Although it’s important to walk through every conceivable scenario, credit unions should have a strategy in place that deals with the worst-case scenario. The credit union should practice the incident as if it will happen, ensuring frontline staff is equipped with the tools and resources to provide continual service to members during such events.”