Exponential capacity growth of silicon-based electronic devices has been inevitable since William Shockley"s invention of the semiconductor in the late 1950s. Physical limitations in the silicon space aren"t dragons to be slain, they"re chimeras: phantoms to be ignored.

Not so in "meatspace" -- the physical world, or "bricks-and-mortar" as it was known during the dotcom hype several years ago. Ironically, many of the new paradigms predicted by latte-jazzed dotcom gurus during the Age of Irrational Exuberance have manifested, and are now leveraged by niche-players and enterprise-juggernauts alike.

One phenomenon tied to physical realities -- data storage -- has transcended those limitations and followed a curve not dissimilar to Moore"s Law. Twenty years ago, hard drives were finicky, brittle creatures found in "clean-room" environments. Ten years ago, storage power-users measured their might in megabytes. Today, one-inch multi-gigabyte drives spin Cantopop mp3s in teenagers" iPods while external hard drives aimed at consumer desktops top a terabyte.

On the enterprise side, storage has become a commodity. Redundant disk (RAID) arrays can be stacked up like sardine tins. Need more? Just buy it and stack it. But if it"s a commodity, how low can it go? And what about the long-promised interoperability?

Storage stack

IDC Asia-Pacific"s Quarterly Disk Storage Systems Tracker report for March 2005 said the total market for disk storage systems -- including storage which is internal to the server as well as external storage -- in Asia-Pacific ex-Japan increased by 3.6 percent over 2003 to US$2.08 million.

External disk storage revenue grew by 8.5 percent in 2004 to US$1.57 million. However, the external storage capacity showed much stronger growth year-on-year in 2004 in the region. The IDC report said 2004 saw a total of 76,350 terabytes in this sector -- that"s a 65.1 percent increase from 2003"s figure of 46,239TB.

According to IDC, these results indicate that ongoing demand for storage capacity remains strong for all countries throughout the Asia-Pacific region as many organizations build their disk storage systems to take account of the need to store and manage ever-increasing amounts of business and reference data. Cheaper disk storage capacity, ongoing market competition, transition to modular disk storage systems, increasing capacity of hard disk drives and the increasing shipments of lower cost ATA disks in some storage systems were pegged as factors.

"The increased adoption of entry and midrange networked storage systems was a notable feature of the market due to the improved flexibility and cost effectiveness of these systems," said Graham Penn, IDC Asia Pacific associate VP of storage. "This has been well accepted by many users, particularly in midsize and smaller organizations as well as many departments in larger organizations."

IDC"s data indicated that external disk storage systems connected to servers via SAN accounted for 53.2 percent of disk storage systems revenue in 2004. Direct attached storage accounted for 40.5 percent of revenue while network attached storage (NAS) accounted for 6.3 percent.

China, Australia and Korea remained the largest markets in Asia-Pacific ex-Japan. However, the highest growth rates recorded in the Asia-Pacific region in 2004 were India, Hong Kong, Singapore, New Zealand and Malaysia.

Pixie dust

"The price of storage capacity is going down 30 percent [annually], and we believe it will continue to do so for the next ten years, or maybe more," said Hu Yoshida, vice president and CTO for Hitachi Data Systems. Yoshida said that "the technologies on the roadmap -- for example, SATA (Serial ATA) technology, which is a major jump" would ensure that the path to cheaper and higher-capacity would remain clear.

"People don"t realize what a breakthrough the one-inch drive in that Apple iPod is," said Yoshida, "because there used to be a physical limitation where the bits [on the hard drive media] became so small that they would not maintain their magnetic orientation." Yoshida explained that a coating called anti-ferro magnetic coupling made the media more stable: "IBM called it "pixie dust" and Hitachi bought that technology."

Commodity trading

In many ways storage has become a commodity, and this is perhaps best reflected in HP"s approach to selling medium-sized storage products. As explained by Darren Ho, product marketing manager of enterprise products, NSS customer solutions group, HP, the company"s approach is much like that of any retailer -- offering bundled kits complete with software, licenses, and "Wizard" setup assistants, all aimed at booting up a working SAN within an hour or two. "The price per gigabyte keeps dropping," said Ho, "we"re seeing a 30-40 percent drop annually."

Ho added that his firm is continuously improving their storage offerings, through innovations like firmware updates or swapping out new drives in existing systems. "Life cycles of storage systems range from three to five years, based on the environment and data growth requirements of the customer," said Ho. He added that SATA was suitable for low-end storage, while FATA (Fiber Attached Technology Adapted) was better for mid-range applications. "The customer doesn"t need a separate controller-shelf with FATA," said Ho. "This offers better protection and reliability."

Ho added that HP customers continue to demand cheaper storage, and that his firm put effort into helping customers understand that different tiers of storage are the way to go, with mission-critical data stored on FATA systems.

"While there are areas in the market where storage is "commoditizing" (which is usually storage for storage sake or point solutions)," said Wally Tung, executive for storage solution sales, Systems and Technology Group, IBM Asia Pacific. "Storage is becoming more and more a critical focus in many companies with regards to data storage and management, [and] storage customers are seeking new ways to manage and access information regardless of what platform it resides on."

"We are seeing a clear movement among our clients away from point solutions into more integrated offerings," said Tung, "as they put in place processes and systems that meet their needs to manage information for long periods and for regulatory requirements."

Cross-department data

Tung added that his firm"s customers "from large to SMBs" view storage as a strategic element, and want systems that are easy to access across business operations for CRM or SCM purposes, and feature easy integration to meet data growth.

"Like all other technology solutions, the price declines when it becomes accepted by the market," said Gabriel Leung, general manager for Hong Kong, EMC. "[But] the discussion is more than disk arrays and gigabytes. Customers are embracing ILM because terabytes of data stored on disk arrays are growing at close to 70 percent a year, much faster than IT budgets are growing."

"Mid-sized enterprises and SMBs are now facing the same information pressures as large enterprises," said Leung, "such as managing ever-growing amounts of information, streamlining key processes (such as backup, recovery, and archiving), optimizing e-mail and other critical applications, and protecting critical business data. However, the EMC chief said that these smaller businesses were handling these tasks "without the luxury of large IT staffs and budgets."

But not all problems are solved by plummeting price points. Many IT managers see the demand for more data storage capacity outpacing their ability to consolidate their infrastructures and improve systems performance. That conflict, they said, comes even as backup and recovery windows are shrinking. Other storage managers said they are moving away from individual storage-component purchases so they can focus on architectures to handle specific business needs?not just individual user requests for increased storage capacity.

"My last challenge to the industry as a buyer is to see how fast can you make (storage) cheaper and make storage retrieval faster and occupy less space in my data center, because I"m out of room," said Bob Eicholz, vice president of corporate development at Efilm in the US.

The future: going vertical

Yoshida from HDS detailed what he called a "roadmap" to the future of storage technology: "Beyond that is vertical orientation-instead of having the bits arranged horizontally we orient them vertically-so you get better densities," he said. "Already we"ve got [technicians] testing them out, running them in their laptops. That [technology] will take us another order of magnitude."

Seagate and Toshiba are also planning to roll out hard disk drives based on vertical orientation or perpendicular recording technology. The technology promises to significantly boost the capacity of hard drives without affecting size, weight or read/write speeds.

Drives store data in magnetically charged bits. In today"s commercially available drives the bits lay flat on the disk surface. With perpendicular drives, the bits stand upright. Because they take up less space, more room is available on the disc.

"Beyond that we have heat-assisted magnetic recording," said Yoshida. He added that the next technology would be Self Ordered Magnetic Arrays [SOMA], forming "a clear roadmap that will continue to drive that capacity-cost."