Transmitting data from the middle of nowhere

02.12.2008

At times, hydrographic data floods in at 5TB a day, and that data not only needs to be transmitted to disk arrays aboard ship from the sonar equipment, but then replicated to an onshore data center for analysis. That requires either satellite transmission or T1 lines -- and either can be a bottle neck.

Georgion had previously set up a data center on the ship using NetApp NAS arrays. That not only required greater bandwidth to transmit data -- the deduplication algorithm was not as efficient -- but it also required a crew of IT personnel to support the setup. Both were costly.

"There's a real shortage of surveyors in the world," he said. "So if you're going to have them spending three or four hours a day copying data onto disk, verifying it, mailing it back, copying it on the other end, doing all these pieces, you're basically going to be taking away 30% maybe 40% of your total efficiency. You can't survive doing that. Plus, you can't manage it."

According to Georgion, Data Domain figured out how to do something no other vendor has yet been able to do cost effectively: in-line data deduplication, or compression with very miniscule performance overhead hits.

"In this day and age when we're moving 10TB at a time, are we going to move that at 10Mit/sec or 50Mbit/sec per second? No."