But these kinds of metrics are much more difficult to use than the traditional IT time and transaction measures. One of the big challenges that many IT leaders face in developing effective metrics, says Rubin, is determining which activities are under the auspices of the IT department. For instance, some user departments at financial services companies have their own LAN administrators who are outside the purview of the IT organization, says Rubin.

"There's IT everywhere, not just in the IT department but in bank branches and on the plant floor," he says. The business activity supported by IT "drifts back and forth like the eye on the head of a flounder," he adds.

Metrics get even more complicated as IT organizations move to service-oriented architectures, explains Stanley. "In a classical IT world, you measure a particular application," he says. "But when these systems are all highly integrated, what do you actually track? The platform? The services?"

Among its current measurements, Harrah's IT department is gauging how much investments in IT and business projects are helping the hospitality and gaming company to meet its business objectives.

"So let's say we're targeting 15 percent earnings-per-share growth and tracking that to the right IT projects, customer satisfaction with those projects, etc.," says Stanley. "Over the past three or four years, I can actually show the return on invested capital to the business results and sales growth."