Questions are raised about the value of investing in technology, which could comprise 50 percent of a company's capital budget, he said. "The scary thing is, the CIOs just can't answer that question," said Gaughan.

A more linear and smooth approach is needed to categorize assets, said Alan Nugent, senior vice president and general manager for Enterprise Systems management at Computer Associates.

"At the core of the frustration that a lot customers have and people have relative to managing infrastructure and all the relative pieces of it, is we always take an asset-centric approach," such as with managing databases, he said. The industry has not matured enough to look at the entire management stack, translate data about assets, correlate that and pick out pieces of relevant information relevant to specific business problems, Nugent said.

An actual ERP for IT system might be difficult for a single provider to offer but is possible, according to panelist Gary Oliver, president and CEO of Blazent, which offers business intelligence solutions for IT.

"I think it's very difficult for one company to be the provider of all that whole capability," Oliver said.