Petascale storage may trickle down to you

06.11.2006

-- One promising approach that's now coming into use at the national labs is a technology called object storage, by which clients can access storage devices directly without going through a central file server. Object storage devices have processors attached to them so that lower-level functions, such as space management, can be handled by the devices themselves. And because data objects contain both data and metadata, it's possible to apply fine-grained, highly intelligent controls for security and other purposes. What's more, object-based storage systems tend to be much more scalable than traditional ones.

-- Researchers will also work on protocols and APIs, especially those related to Linux. They will help develop extensions to Posix, the portable operating system interface for Unix, to enable more effective use of file systems in highly parallel computer clusters. Researchers will also work with The Open Group and the Internet Engineering Task Force to make the Network File System protocols for file access more capable in highly parallel systems.

-- The PDSI will explore a number of emerging technologies, such as phase-change RAM, Miller says. PRAM, recently announced by Samsung Electronics Co., offers the speed of dynamic RAM with the nonvolatility of flash memory. Miller says it's the perfect place to put metadata because it can be accessed much more quickly than if it were on disk, thereby making object storage systems much faster.

-- Miller says PRAM might also be used to store indexes used by search engines, greatly accelerating them as well. That increased speed may prove to be of interest to businesses such as oil companies that have huge stores of private data but lack the enormous resources of a company like Google Inc.

-- Few corporations will ever have systems the size of those at the national labs, with tens of thousands of disks, says Miller. But even desktop systems, which will have more and more disk drives over time, will experience some of the challenges the PDSI will address.