"IT is a business within a business," said Tom Sanzone, senior vice president at Booz Allen Hamilton and a former CIO at Credit Suisse, during a panel discussion. "It should be run that way."

"When you look at IT services you say, can I deliver these services at my scale, at a price the market can bear," Sanzone added.

But at the same time, CIOs can no longer sacrifice quality, other panelists said.

"We have accepted a very poor quality of product for a very long time, and that has been the major filter through which the business perceives us," said Steven John, strategic CIO at cloud ERP (enterprise resource planning) software vendor Workday. "We need to change that."

Global systems integrator Accenture fits its IT spending into "two buckets," one for operating expenses and another for investments in IT, said CIO Frank Modruson. Each year, his teams are urged to find ways to lower operating costs and move the money over to investments, according to Modruson.