( on the Human Neuroimaging Lab's groundbreaking research.)
Researchers and post-doctoral students at HNL spend their time answering questions such as: Why are humans inclined to trust strangers in matters of finance? And how does awareness of a brand (such as ) influence our preferences and what does that tell us about the human brain? Behind the scenes is an IT infrastructure with storage systems from four vendors, 30 x86 servers and two high-performance computing clusters.
Managing it all is just one man -- Justin King. The best part? He's a college dropout.
King, 29, attended the University of Texas, but was thoroughly unimpressed by its computer science department. In one class, he remembers there being far more students than computers, and many of them were broken. He left after two years.
"I said you're kidding me -- one of the biggest schools in the nation can't get enough computers to use? On top of that, it was 1999. People were getting [US]$50 million to teach swimming lessons online,"King says.
There's some exaggeration there, but many people have succeeded without a college degree. Bill Gates skipped out of Harvard in 1976, after all.
King had been working in a computer store since high school and figured he could at least get a job doing desktop support. He did just that at the food company with headquarters in Houston. King was hired away nine months later by Read Montague, a neuroscience professor who was starting up a software company called Quaadros.
The company failed, but King followed Montague to Baylor College in 2001. By 2003, Montague had opened his Human Neuroimaging Laboratory, and King's role has only expanded with each passing year.
Montague, despite being a professor, didn't seem to mind King's lack of a college degree.
"I just sort of work like a maniac,"King says. "That's right up Read's alley. He just puts everyone to shame. He works all the time."
Today, King is the sole system administrator for the and the Computational Psychiatry Unit. King, as well as a few software programmers at the lab, have immersed themselves not only in computer technology but also in the science of fMRI experiments, says Montague, who is director of the HNL.
"It's a very rare system administrator ... who understands the nature of the experiments we're doing,"Montague says. "I always included them as much as possible. There's not a cultural gap between the scientists and the computer guys."
It's common for King and the programmers to ask probing questions -- do you really want to store the data that way? What method should be used to anonymize personal information? The intertwined nature of scientists and techies at HNL has helped keep the lab nimble and lean, Montague says.
The list of technologies King manages includes 102TB of Pillar Data Systems storage; 15TB of ; a Dell tape library; four Fibre Channel Qlogic SANbox 5600 switches; about 30 x86 servers, mostly Dell; VMware virtualization; a PostgreSQL database for storing experimental data; and two high-performance computing clusters with 50 nodes each. (Compare .)
There's also a Cisco Catalyst 6509 switch, but luckily for King the Baylor College of Medicine has a networking group, so he doesn't have to manage that piece of technology. King manages Microsoft Exchange e-mail for the HNL and Computational Psychiatry Unit, but the college networking team manages the spam filtering and antivirus needs, taking much of the burden off of King in this area.
King singles out a few products that have helped him simplify his job, including Pillar Data Systems' storage and VMware's server virtualization.
The ability to manage all virtual servers from one interface, and advanced features such as and are the key benefits of using VMware, he says.
Virtualization does raise the threat of having a single point of failure for many workloads, though, he notes.
With VMware, "you've got everything you need in one space,"King says. "At the same time, if one server goes down you're going to lose a bunch of [workloads]."
A lot of suffered from just such a problem recently, when a bug in a software update prevented virtual machines from booting up. King avoided the problem by not upgrading to the new software.
"It always makes sense to wait at least two weeks to install the newest version," he says.
King recently to Pillar from IBM System Storage DS4500, an older product that IBM is no longer selling.
Pillar storage is designed to drive up disk utilization rates by providing extra RAID controllers. It's also billed as "application-aware," meaning it knows the requirements of specific applications and can dynamically reassign resources based on changing priorities.
For example, it's easy to move data from high-performance disk drives to the archive layer, King says. Pillar is able to squeeze lots of performance out of standard SATA drives that are not as expensive as Fibre Channel hardware, he says. (Compare )
"You can selectively decide how fast or slow you want your storage to be," he says. "That's extremely important. We have lots of stuff we don't need on really fast spinning disk, but we need it on nearline availability."
When you're the lone ranger in a one-man IT department, "just finding enough time to stay focused on one thing" is the biggest challenge, King says. "I try to offer as much as I possibly can. We're not an Amazon. If we go down for an hour, we're not losing money by the second. It's an inconvenience but it's not the end of the world."
King provides services to about a half-dozen faculty members, and a few dozen doctoral students and researchers. Because of their high level of technical expertise, King does not have to deal with some of the minor problems that monopolize the time of a typical system administrator.
"The people here ... are all pretty savvy," King says. "I don't have to deal with [users saying] 'I can't figure out how to get my printer installed, or the sound isn't working.' That makes it possible to get other stuff done."
All in a day's work
In a typical day, King arrives in the morning, cracks open a diet Dr. Pepper and checks out his network monitoring software to make sure everything is running smoothly. Then he chips away at his long list of stuff to do, such as planning to consolidate some servers with VMware and get rid of some outdated EMC storage in order to make room for a new Pillar box. Then there's always work to do for scientists operating the fMRI scanners.
Montague's lab developed a unique research method called hyperscanning, which lets one scientist control multiple scanners over the Internet, even if they are thousands of miles away from each other.
One experiment involved squirting Coke and Pepsi into subjects' mouths and measuring brain responses. Another involved a financial game that examined people's willingness to trust people they don't know with money. (.)
King has helped this process by writing programs that pull data from the scanners and make them available via Web interfaces. According to Montague, King has also been critical in coordinating the experiments when time differences are involved. Sometimes the lab uses scanners in the United States and Hong Kong simultaneously.
"He kept [the Hong Kong project] running, and when it didn't run, he fixed it,"Montague says. "He anticipates problems."
When King goes on vacation, Montague says the lab will hand off his work to the programmers and then "pray that nothing breaks."
Working at the HNL is like being at a small company -- everyone just does whatever needs to be done, King says. It's a 40-hour-a-week job, but he's on call during weekends, and when he has needed to stay all night, he's done just that. Being involved in Montague's research projects makes King feel like his work contributes to a higher purpose than just fixing and setting up computers.
"You found a guy with the best job in the world. I feel like that,"King says. "I really enjoy it and I get paid well to do stuff that I really enjoy."