The older the equipment is, the less efficiently it uses power, Nelson said. Yet IT managers might be reluctant to upgrade, given the cost of new equipment. However, this approach may actually cost more in the long term.

"We replace our equipment every two years, in a leasing model," he said.

The worlds of physical infrastructure management and IT management are converging, agreed Martin McCarthy, CEO of the 451 Group, which owns the Uptime Institute. Today's data center operators have to think more holistically, he said. "Energy efficiency is ultimately economic efficiency," he said.