"We would back something up from the night before and shoot those tapes off-site," says Roemmele, SAN team leader at the Livonia, Mich.-based financial services firm. "A few hours later, the Exchange administrator would get a [user] request to restore something from their in-box from the night before."
That meant having the company's off-site vendor return -- for a fee -- the same tape that Quicken Loans had just sent it. Even worse were the episodes when Roemmele's staff asked for the wrong tape or multiple users needed restores from different tapes, forcing an expensive volley between Quicken Loans and its off-site vault.
Roemmele has since purchased DD400 Enterprise Series backup and recovery appliances from Data Domain Inc. in Palo Alto, Calif. They reduce the amount of data that needs to be backed up so dramatically that he now stores two months' worth of Exchange backups on the appliances, from which users can easily recover their own e-mails.
Roemmele's dilemma is all too common. The pressure to quickly recover specific files from backed-up data comes from careless users who delete important files, regulators who demand quick access to records, and corporate lawyers who need a particular e-mail or memo to defend a lawsuit.
Those demands are reflected in IT spending plans. In a survey of more than 300 IT professionals by market research firm IDC, two-thirds of the respondents said backup and recovery/data protection would be a major driver in their spending on storage services in the next 12 months.
Customers are using a combination of technologies to reduce the amount of data that needs backing up and store it in "active" archives that can be accessed more easily than typical off-site tape archives. Those technologies include disk-based storage, incremental backup, data compression or reduction, and WAN optimization.
Active archives
"For years, many customers viewed archiving only as taking backup tapes out of the normal rotation cycle and storing them off-site," says Rob Ensley, director of product marketing for information management products at EMC Corp. in Hopkinton, Mass. "We've seen a significant increase in people creating what we call active archives," keeping archived data on disk storage so it can be quickly accessed for regulatory or litigation purposes.
Oftentimes, the active archive is kept on storage built around ATA or Serial ATA drives, which offer performance and reliability close to that of Fibre Channel arrays but cost significantly less. One user taking that approach is Jamesburg, N.J.-based Argix Direct Inc., which tracks detailed information about shipments it makes to retail stores from its four package-sorting centers.
Until last year, the company had backed up that Microsoft SQL Server data to its headquarters using Backup Exec from Symantec Corp. However, over time, the amount of data grew so large that it threatened to exceed the company's backup window and slowed traffic on the WAN between headquarters and the sorting centers.
Argix now uses EMC's RepliStor software to replicate the data from the sorting centers to the production data-base on Fibre Channel drives on an EMC Clariion CX300 storage system at headquarters. From there, the data is backed up daily to ATA drives on the Clariion and then onto tape for long-term storage. The replication cuts the time and bandwidth needed to move the data to headquarters, and storing data on the Clariion ATA drives lets the IT staff restore critical backed-up data instantly, compared with the hours it took to restore from tape, says Argix CIO Nino Silvano.
Argix has achieved similar results with its Microsoft Exchange environment, archiving any e-mail more than 30 days old onto the Clariion's ATA drives and eventually to tape. Compared with the old process of backing up immediately to tape, this has reduced backup times by 80% and allows "you to access the archived e-mail as if it were still in your mailbox," says Silvano. It also reduces the amount of primary storage required for the production Exchange environment.
Curtis Damhof, network manager at St. Peter's Healthcare Services in Albany, N.Y., is using Axion backup and recovery appliances from Avamar Technologies Inc. to replace backups to tape. The appliances cut his backup window for about 3TB of data from hours to five to 10 minutes, and they let him restore e-mail messages "within an hour," he says. "We couldn't do anything even close to that with tape."
At Cincinnati Thermal Spray Inc., MIS manager Steve Wilson is moving to Symantec Backup Exec 10d to replace the sometimes unreliable tape backups that had been done at the company's four regional locations. He says he likes the fact that Backup Exec 10d can capture changes to applications in near real time. This provides extra protection for about 1TB of data, which includes the highly technical paper trail documenting exactly how the company has applied anticorrosion or wear-resistant coatings to aircraft parts.
As the only storage professional in the 200-person company, Wilson also appreciates that Backup Exec 10d gives "users access to their own files and allows them to create their own restore jobs, and even to see the different versions of those files they can restore from." When he finishes the rollout of Backup Exec 10d, Wilson hopes to eliminate tape backups at the remote offices and back up all the data from them to a 2TB storage server at headquarters. Like other customers, he isn't eliminating tape but is creating a weekly backup to tape from the disk-based archive for long-term storage off-site.
Technology options
Disk-based backup systems use different software to capture incremental backups of frequently changing data. Some, such as Hewlett-Packard Co.'s ProLiant Data Protection Storage Servers, are based on Microsoft Corp.'s Data Protection Manager software. The HP offering provides "near-continuous" backup that fills the gap between daily tape-based backups on the low end and synchronous replication on the high end, says Brad Parks, product marketing manager for network-attached storage at HP.
WAN optimization technologies speed backup traffic over WANs, while wide-area file systems use caching, compression and other methods to consolidate file servers that used to reside in branch offices into the data center. There, both servers and data can be protected more effectively. Vendors such as Data Domain cut backup volumes by breaking the backup stream into tiny bits, comparing each fragment and backing up only those that have changed since the previous backup.
However it's done, the spotlight is on backup and recovery processes. Years ago, "backup and recovery was something you had to do, but it was something like you had to eat your oatmeal," says Doug Chandler, an analyst at IDC. Now, he says, "the pressure's from the business unit, the people who own the application and maybe the general counsel" to not only back up data but to be sure that it can be restored quickly.
Sidebar
A comeback for managed storage services?
Managed storage service providers (MSSP), which store customer data at their own facilities, were one of the high-profile victims of the dot-com bust. But as the need to cost-effectively store and retrieve data grows, those providers are getting another look.
A 2003 Gartner Inc. survey of about 120 IT managers showed that 20 percent were using an MSSP, another 29 percent had used one in the past, 9 percent planned to use such a provider, and 42 percent had no interest, says Gartner analyst Adam Couture. By last year, 50 percent were using an MSSP, 34 percent planned to use one, 10 percent had used one but dropped it, and only 6 percent had no interest.
Ease of recovery is one major reason market research firm Synovate Americas in Chicago stores 25TB to 30TB of data using an online storage management service from Sun Managed Enterprise Services.
When a customer or researcher needs an old research report or an executive needs to restore data to his notebook computer, "we don't want to spend 48 hours getting a tape and loading it," says Prabhakar Sonparote, senior vice president and IT director at Synovate.
Iron Mountain Inc., a physical records management services firm in Boston, jumped into the MSSP market with its acquisitions of Connected Corp. (for backup of desktop and laptop data) and LiveVault Corp. (for backup and recovery of server data). Revenue from those services is expected to rise from $90 million in 2005 to $140 million to $150 million this year, says John Clancy, executive vice president of Iron Mountain Digital.
But Doug Chandler, an analyst at IDC, estimates that MSSPs sold $215 million in services in 2005, less than 5 percent of the overall market for backup hardware, software and services, and that by 2009, the market will have grown to only $450 million. Chandler says the largest, most sophisticated customers will still choose to handle storage themselves, while smaller companies willing to outsource storage will buy it as part of a service package from an established vendor rather than from a relatively unknown MSSP.
He's also skeptical that stand-alone storage providers are large enough to achieve the economies of scale that would let them provide services at lower costs than in-house storage managers could.
MSSPs can be less expensive when a customer considers all the costs of maintaining storage staff and hardware at multiple locations, says Steve Siegel, vice president of marketing at Arsenal Digital Solutions Worldwide Inc. in Cary, N.C.
Sonparote says he's saving 30 percent to 40 percent on capital expenses using an online service, and 100 percent on expenses such as labor.
But Couture warns that many MSSPs price their services differently.
"Some of them charge you for the largest backup you made in a one-month period; some of them charge you for the amount of data you're protecting on the server; some of them charge you for every [gigabyte of storage] you've written to or from," Couture says. He recommends asking competing MSSPs how much it would cost to protect a given amount of data to get "an apples-to-apples comparison."
Scheier is a freelance writer in Boylston, Mass. Contact him at rscheier@charter.net.