"It gives us a way to generate a lot of ideas, kill them fast and cheap [if they don't work out], calculate ROI and involve business leaders," he says.

When making decisions, return on investment isn't necessarily essential or even a consideration, the CIOs say. "A lot of things aren't based on ROI," says Leader. "I don't remember the last ROI argument I made."

Dunn says regardless of the weight ROI is given, figuring it out is a valuable exercise. "Going through the process of finding ROI helps give you a feeling of if it's the right decision," he says.

"ROI is an aspect of it," says Kuttler, "but it's not the only aspect of the business plan."

The CIOs also have advice for vendors: Don't waste time, and be a partner. Being a partner means helping the CIO sell his ideas within his company, setting up tests and working with the IT team to deploy whatever product is involved, says Dunn.