The survey seems to show a huge gap between private industry tracking of data center use and the federal government's, Weber added. Just 31 percent of survey respondents from federal agencies knew the average load across their data centers, compared to more than 90 percent in the private sector, he said.

The survey isn't all bad news for the federal government, he said. "Look what they could be doing if they start tracking these things and determining how efficiently the assets they have are being utilized," he said. "Look at how much savings they could gain."

The U.S. government operates about 2,100 data centers, outgoing federal government CIO Vivek Kundra said in a released in December. Kundra, leaving OMB for Harvard University later this year, called on agencies to close 800 data centers by 2015.

Kundra, in , said many federal data centers are using about 7 percent of their servers, suggesting that someone at OMB has tracked some data center numbers.

It's difficult to track data center power consumption because government data centers are often located in buildings with other federal offices, said OMB spokesman Kenneth Baer. The electricity consumption records of federal data centers are often part of larger electricity bills, he said.