Our Test Process
For this comparison, we focused on tests that truly reflect the kind of applications that these machines will be used for. We began by installing 64-bit versions of Centos 5.2 and Vista Ultimate; the 32-bit versions cannot adequately address more than 3 GB of RAM, and most professional applications work in 64-bit mode. Subsequently, we ran SPEC CPU2006. The Standard Performance Evaluation Corporation's benchmark is widely used in both industry and academia. This suite tests the CPU, memory systems, and the compiler. We used the integer benchmarks for our test (CINT). These twelve tests are drawn from real world applications. To ensure compatibility and repeatability, we used the Intel C compiler to compile the programs. The tests were done with the baseline tuning to ensure that all the machines were treated equally.
This was followed by a series of tests from the Phoronix Test Suite. We used twenty-one tests from this to test various aspects of the overall performance of the machines. Everything from multi-core performance, to memory bandwidth, to disk speed was put through the wringer. The suite runs around fifty tests, but all the others were in line with the results of the chosen twenty one.
Finally, we used SPECviewperf 10.0 to test a set of common workstation applications. This benchmark uses programs such as Maya, 3D Studio Max, Solid Works, Pro/ENGINEER, and CATIA to run simulations of complex designs. We ran the tests in 64-bit mode and multi-threaded mode, to really take advantage of the four cores. Naturally, we used the newest drivers, and Windows was updated before we started.
Features and Design
These machines are more than just the sum of their parts, and nothing brings that fact home with more force than simply lifting them up. You'd expect them to weigh roughly what a standard desktop does--and you'd be dead wrong. They are a lot heavier. This weight is a testament to the solidity of the materials used in their design. All four abound with clever design touches, from their tool-less designs to the airflow patterns. Sun's Ultra 24 has a mesh grill in front, for optimum airflow. Inside, all the cables are very neatly routed away. This machine had the neatest interior of the lot. There is only one case fan, at the rear of the machine, which pulls air across the components from the ventilated front.
It supports up to four hard-drives, the most in this test. These drives can be either SATA or SAS drives, since the backplane they plug into supports either. The drives plug in via a drive holder that slides into the bay and clicks into place securely. This keeps things very neat, since there are no dangling wires. However, Sun does not provide empty holders for the other drive slots, so you can't just buy a drive and plug it in. The power supply is a robust 530 watts, and is labeled as being 80 percent efficient. On the front of the case, you get two FireWire ports, two USB ports, and audio I/O. Curiously, the FireWire ports were FW 400, not FW 800. In fact, none of the workstations supported FW 800. Round the back, you get four USB ports and 7.1 audio, and that's it. It's all very legacy free. Interestingly, there's one USB port on the motherboard. Dell and HP also have this.
The idea is to keep the security dongles for expensive software inside the case, not outside where anyone can make off with it. This is a good idea. Dell's Precision T3400 is a lot busier on the inside when placed next to Sun's spartan interior. It's slightly cluttered with wires, but a lot of space is taken up by the humongous heatsink on the processor. The cable management could be better. It supports a maximum of two drives, and comes with the necessary drive holder for the second one. The front has a Firewire port, two USB ports, and audio I/O. The rear port cluster has six USB ports, audio I/O (no 7.1 here), a cluster of legacy com, parallel, and PS/2 ports. There are two fans that keep the interior cool, and it's completely tool-less.
The power supply is rated at 375 watts, and there's no mention of an 80 percent efficiency rating. HP's xw4600 is the narrowest of the four workstations. The insides are quite neat, falling in between Dell and Sun in terms of the cable management. It supports two hard drives, and has the appropriate holders in place. The front panel has two USB ports and audio I/O. The rear panel has seven USB ports, a com port, a parallel port, PS/2 ports, 5.1 audio, and uniquely, an eSATA port. Firewire is conspicuous by its absence.
The power supply is rated at 80 percent efficiency, and 475 watts. Lenovo's Thinkstation S10 has a nice handle on the top that makes it easier to carry around. Cable management is quite good, and the cabinet has space for three drives, with the holders already provided. The front panel has one Firewire port, two USB ports, audio I/O, and a multi-format media card reader. At the rear reside eight USB ports, dual Gigabit ports, one com port, and PS/2 ports. The 7.1 audio output stands out with the S/ PDIF output.
Performance
In SPEC CPU2006, both Sun and Lenovo did very well. This is due to their having faster processors: a Core 2 Quad QX6850 for Sun, and a Core 2 Duo E6850 for Lenovo. This test is weighted towards CPU performance, so they outdid the slower HP and Dell with their Q6700 and Q6600 processors respectively. However, in the multi-threaded tests, Lenovo was handily out done by the trio of quads. Sun aced most of the tests, thanks to its speed advantage, while Dell and HP were neck and neck through most. The biggest difference between the quad cores and the dual core was in the Bzip2 Compression test.
This takes advantage of all the cores, and the quad core machines were more than twice as fast as the dual core. Lenovo was also hampered by having only 2 GB of RAM, though it was DDR3 RAM, so memory bandwidth tests were a walk in the park. Still, in the overall analysis, it could not keep up with the quad cores. SPECviewperf depends on the GPU. Sun and Lenovo both had Quadro FX1700s, while HP and Dell used the slower FX570. Interestingly, this did not predict the order of the winners. Sun took first spot, but Lenovo didn't come close and placed third.
This is due to the multithreaded nature of the tests. Sun shone in these tests thanks to a combination of a fast CPU and GPU, while HP and Dell slugged it out. In the Pro/E benchmark, Sun got 60.98, HP followed with 47.41, Lenovo came in third with 41.33 and Dell had 36.53. This was emblematic of this set of tests. In Maya, 3D Studio Max, CATIA and SolidWorks, the results were similar to Overall, the tests showed quite clearly that a fast dual core can out do slower quad cores; except when you use applications that are optimized to use multiple cores. The choice you make will depend on the applications you intend to run.
Conclusion
We had four solid machines, but there's only one winner: the Sun Ultra 24. The most expensive of the lot, the Ultra 24 won both the performance and the design tests. In second place, HP put in a good mix of performance and features. The third quadcore machine in the test, Dell, managed to get past the fast dual-core on the basis of the multi-threaded tests. The Thinkstation S10 is a good choice if the applications that you run aren't going to use more than two cores. The price is quite impressive as well. Indeed, when dealing with machines of this class, the difference between them is measured in fractions. All four have highly customizable configurations, so make sure to buy a machine that is configured optimally for the applications that you intend to run. And yes, a little bit of bargaining won't go amiss.