Google corporate IT builds before buying

20.07.2012

Like most organizations, machine management has been a challenge for Google. Google engineers get their choice of operating systems on their work machines, either Apple's OS X, Google's own Chrome, one of several distributions of Linux, or Microsoft Windows. The Apple machines in particular have been hard to manage, given the limited tools available from Apple and third parties for enterprises. By McWilliams estimate, Google has one of the largest corporate IT deployments of Macs in the world, with over 30,000 units now in use. "That creates a lot of challenges for us," McWilliams said.

To push patches and software updates to the Macs, Google initially used Puppet, . The organization quickly ran into scaling problems, however. It looked at commercial solutions, though most charge about $100 per machine per year. Additional Web servers, file servers and load balancers would also have to be deployed. "It would have cost us several million a year for the infrastructure and licensing," McWilliams said.

The company finally found an answer for its Mac support issues in open source software called , which was developed by an engineer at Walt Disney Animation Studios. McWilliams' team had deployed Munki on Google App Engine, which meant they did not have to manage any additional physical servers to run the software. "We have days where we are pushing out over six terabytes of traffic, or thousands and thousands of updates," he said.

Encrypting Macintosh disks was another task that Google tackled without the use of commercial software. OS X Lion 10.7 offered built-in FDE with a program called FileVault 2, but it had some issues for corporate users. For instance, the software doesn't force users to encrypt the disks, nor does it offer an escrow repository for storing keys, other than one provided by Apple itself. So Google developed its own software in-house, called (an anagram of the phrase "FileVault Escrow"), which provides a companywide escrow service.

"When a Googler forgets the password, an admin can fetch the recovery key, unlock the hard drive and reset the password," McWilliams said. As with Munki, Cauliflower Vest runs on the Google App Engine hosted service.