IDC: big data skills crisis needs vendor action

21.04.2012

"There are very few Hadoop experts around, and there are only very poor tools available for using it. You don't only need experts that know how to master an Hadoop file system, but experts that know how to master an Hadoop file system using bad tools," said Feldman.

Feldman urged the likes of Autonomy, EMC, Teradata and IBM to improve the tools on offer to reduce the impact of the skills crisis.

"If vendors could supply the expertise, if they could keep the software updated, then this would lessen the burden on IT departments in the enterprise," she said.

"Integrating all of the big data pieces into a well formed architecture, so that everything can interact with everything else, that's very difficult. Most people doing this could probably happily hire ten extra people, but they just aren't around."

IDC predicts a compound annual growth rate of 39.4 percent in big data hardware, software and service sales between 2010 and 2015. However, this doesn't take into account open-source offerings, which accounts for a significant amount of enterprise use, but is difficult to measure.