In a rare insight into the inner-workings of the world's most popular search engineer, Google principal engineer Rob Pike said the problem is not so much the implementation of algorithms, rather that "everything breaks", and on the scale of Google "a lot will break every day".

"If you have 1000 computers in your cluster you can expect one to die every day [and] ours are not that good [so] stuff happens all the time," Pike said.

"The Internet is so big you are going to expect to see failure; the software has to be robust so you can afford to by stuff more often. You can buy tons of cheap crap [and] it's crappier than you might think."

Speaking in Sydney at an Australian Service for Knowledge of Open Source Software (ASK-OSS), a Department of Education, Science and Training-funded initiative to advise about open source software, Pike expressed doubts about the reliability of "expensive" hardware and software, saying commodity hardware with open source will yield a more powerful solution.

"Google is not a computer somewhere, it's a lot of computers running proprietary and open source software," he said.