Don't get carried away by Hadoop's 'gee whiz' factor

11.10.2011

"Explore a big data approach like Hadoop only if your data volumes are likely to scale into the high terabytes or even petabytes," level and include a wide variety of data types, the Forrester report noted. "If you overinvest in data , computing, and networking capacity, you'll add to your cost overhead without any concomitant business benefit," it warned.

Forrester also recommended that companies use enterprise-grade platforms and tools for powering their Hadoop environments. Where possible, companies should evaluate the functionality, scalability and maturity of the commercial offerings and tools that build on and extend the Apache Hadoop open source distribution.

Some examples include Hadoop distributions from vendors such as Cloudera, Greenplum, IBM and Hortonworks, the report said. It's unwise to attempt to build an in-house Hadoop stack when there are commercial options, Forrester said.

It's also unwise to completely standardize or commit to one vendor's distribution at such an early stage because it is unclear how the market will evolve over the next few years, Kobielus said.

Companies would also do well to build their own Hadoop centers of excellence, Kobielus said. Given the severe shortage of skilled Hadoop professionals, companies should try and develop Hadoop skills in-house as much as they can.