Payroll processor ADP is taking the same approach. The company has set up an Innovation Lab to manage how it stores, processes and analyzes extremely large data sets.

The idea is to create an environment where subject matter experts from different industries and backgrounds can work together to tackle big data analytics, Roberto Masiero, vice president of ADP's Innovation Labs, said in a keynote address.

In a sense what is happening is reminiscent of the situation when enterprises first started using online analytical processing tools, said William Herridge, managing director of emerging solutions at the Tribune Company.

"When we made the transition to OLAP (online analytical processing), it was hard to get business users to get over their [existing] mindset," of using tabular data, he said. "They didn't have any idea of the value of OLAP till you started showing them," he said. IT organizations face the same challenge with big data, he said.

"We see the value in this, but getting users to understand that value and seeing it is there," is a huge challenge, especially when dealing with concepts such as unstructured data, he said. "Until business users can see some benefits, they are not going to sign on to big data projects," Herridge said.