Big Data Projects Lack Big Budgets

28.09.2012
With corporate data stores bursting at the seams, 65 percent of the 369 companies surveyed by magazine have deployed or plan to deploy big data technology to boost the speed and quality of business decisions. The number-one hurdle is getting the big bucks to pay for it.

The survey indicates that the top challenges for big data initiatives are (in order): limited budgets, lengthy development times, legacy issues (such as integration with existing systems), a skills shortage, and poor data quality.

Tom Soderstrom, CTO of NASA's Jet Propulsion Laboratory (JPL), has a few big data initiatives under way but needs a highly skilled staff to make them work.

Soderstrom says he's forming a group of data scientists to manage high volumes of unstructured data. "They need to know programming, statistics, and how you visualize and communicate data," he says, "and most of all, how to understand the domain of that data."

Like JPL, 33 percent of the surveyed companies are looking to hire people with analytics skills for big data projects. Typical job titles include data analyst, data architect and data scientist, the survey says.

JPL is using Hadoop to process large photos from NASA's mission on Mars. JPL breaks the files into smaller parts so they can be easily viewed on programs like Google Earth. Each photo is analyzed to determine whether NASA needs to do more tests on objects such as rocks.