Step 1: The CPD model identified three key areas for improving efficiency. First, it identified a that was floating in the upper half of the data center space. That hot layer started at a height of about 5.5 to 6 feet and extended all the way to the ceiling, some 10 feet from the floor. That meant that the equipment in the top of Scottrade's racks was in the hot air cloud.

There were other problems, too. That hot-air layer was circulating over the tops of the racks, spilling over from the hot aisle, which is supposed to return hot air to the air conditioning system, into the cold aisle, which is supposed to supply only chilled air from the CRAC units. As a result, equipment in the tops of the racks was running warmer than it should have been.

Step 2: The second issue was the configuration of the racks themselves. Not all racks were fully populated, but equipment was always concentrated at the top of the racks, where it was subject to those higher temperatures. In fact, says Patterson, the hottest running servers tended to be mounted at the top of racks, where cooling efficiency was lowest. To address that, Scottrade had lowered the computer room air conditioning system temperature settings, in effect overchilling the rest of the room. "Scottrade was running the overall data center temperatures colder than necessary to keep the temperatures at the top of the racks within acceptable ranges," Graves says.

Step 3: Finally, the balance between the heat load produced by the server racks and the quantity of air supplied to the cold aisle was out of whack. Engineers redistributed the perforated tiles on the aisle floor to match the output required. "A thermal balance was noticed immediately," Graves says.

Air conditioning systems perform most efficiently when the air temperature differentials are higher, so Glumac implemented changes that made the cold aisles colder and the hot aisles a few degrees warmer. "We weren't optimizing the heat-to-cooling ratio that the AC units needed. You have to get that balance," Patterson says.