But for fast collection, deduplication and transmission of data, Georgion said only one company's appliance fit the bill and that was Data Domain Inc.'s .

Data Domain makes disk-based storage appliances for data backup and disaster recovery using a compression algorithm that is supposed to make sure data being transmitted does not already exist on the host system.

"The main problem people have in this industry is moving data. Once generated, how do you get [it] from point A to point B? Anything that holds, let's say 5TB of storage, is not portable," Georgion said. "So data movement is the number one problem for anyone in the survey or field scientific world."

On average, Fugro achieves a 50% data reduction by using Data Domain's compression algorithm, which Georgian said is "really good considering that our data is largely image based, meaning that it's not typically compressible." "It basically doubles the line speed for the cost. Or in far flung locales doubles the line speed that's available," he said, adding that the appliances send data between 20 and 24 hours a day at 100% bandwidth capacity. For the California coastline survey, Georgion set up one Data Domain appliance on the research boat; it receives its data from a NetApp NAS array that collects transmissions from the sonar head. Once the data is copied onto the Data Domain box, the software automatically compares it against a second Data Domain box in the San Diego data center and, after deduplication, begins sending.

On an average day, Fugro's coastline survey systems create about 100GB of data, "but it just never stops," Georgion said. "It just keeps growing and growing. It's 100GB a day, every day."