Hennegan was motivated not just by concern for the executive's well-being, but also by the fact that the president was one of only three people at Gilsbar who were authorized to call SunGard Availability Services, the third-party insurance administration firm's IT hot-site vendor, and declare a disaster.
In addition, there weren't any working landline or cell phones within 50 miles. As a result, it wasn't until 3 p.m. on Wednesday, Aug. 31 -- two days after Katrina struck -- that Gilsbar was able to make a disaster declaration to SunGard. And then the company was thrown a curveball: Although Gilsbar had always run its disaster recovery tests at SunGard's Alpharetta, Ga., facility, it was assigned to a hot-site facility in Chicago -- a 14-hour drive away.
In preparation for this year's hurricane season, Gilsbar has made some changes that are designed to help get its systems up and running again within 48 hours of a disaster. If a big storm is forecast, the company will pre-position an IT team in another location, such as Memphis, with a set of backup tapes, Hennegan said. Those workers will be authorized to declare a disaster and start recovery procedures at SunGard, he added.
Despite the challenges that Gilsbar faced in the aftermath of Katrina, Hennegan said SunGard's hot-site service "is fabulous for us." He also praised SunGard for helping with non-IT needs. "People came up with their families," he said. "Some of them didn't know what their home situation was, and they were emotional wrecks when they got to Chicago." SunGard made sure that everyone was fed and that activities were organized for the children, he said.
But not everyone shares Hennegan's view. For example, the series of hurricanes that hit the East Coast two years ago encouraged Greg Holdburg, manager of disaster recovery and business continuity services at S1 Corp. in Atlanta, to move away from SunGard as a hot-site vendor.
S1, a vendor of banking and financial services software, hosts systems for some of its customers and has service-level agreements with them requiring annual tests of its disaster recovery procedures. But Holdburg said that because of the heavy demand for hot-site space at SunGard's facilities after the hurricanes, S1 risked not being able to complete the mandated tests.
Consequently, the company decided to reopen its disaster recovery contract and call in other vendors to bid against SunGard. It ended up choosing Houston-based VeriCenter Inc., which proposed a hybrid hot-site/in-house solution that provided S1 with a dedicated 3,000-square-foot caged area at a data center in Dallas. S1 supplies its own equipment and can run tests whenever it wants, Holdburg said. "It's ours to do whatever we please with," he noted.
What's more, even after buying the IT gear, S1 expects to save $1.5 million over three years because of the switch, according to Holdburg. That is a result of both reduced hardware costs and the tendency of disaster recovery vendors to overprovision their facilities and then pass the costs on to customers, he said.
One caveat, though, is that users of a service such as VeriCenter's need to have trained staffers available to support their systems, Holdburg said.
The city of Tallahassee, Fla., is also considering dropping its SunGard hot-site service. "Right now, we will take our tapes, get on an airplane and head to Philadelphia," where a SunGard facility is located, said Don DeLoach, Tallahassee's CIO. But it may take longer than 48 hours to restore operations, and DeLoach said he wants a faster recovery process and the ability to get his IT employees out of harm's way as soon as possible.
"We want our data out of here," he said, adding that the city is looking at using Unisys Corp.'s Business Continuance SafeGuard 30m technology, which is designed to provide automatic fail-over of clustered Windows-based applications to a backup facility.