Production Web and e-mail servers may be Linux"s heartland, but researchers are now leveraging the operating system for testing and debugging large SMP systems.

Speaking at this year"s Linux conference in Canberra, IBM Linux technology center (LTC) engineer Anton Blanchard said Linux is used to debug both hardware and software while testing the latest-generation Power5-based systems.

"There"s no problem with Linux"s support of large CPU counts and memory these days [and] thread priorities are supported," Blanchard said, adding that each chip appears as a four-way SMP system to the software.

During the design phase, the first Power5 CPUs arrived early but the memory buffer chips had not yet been delivered.

Blanchard"s team then discussed running Linux on the machine with all the caches turned off.

After two nights of hacking on Linux and the "bringup tools", the researchers managed to boot Linux on a machine without local memory. And, although very slow - it took two hours to load Linux into memory - Linux booted on the first attempt and a Web server was started.

"All Power5 systems were brought up with Linux which allowed hardware to be exercised earlier and issues to be identified and fixed earlier," Blanchard said. "A bunch of hardware bugs were discovered and de-bugged on Linux; Kernel 2.6 has some useful features for this work."

Power supports multi-chip modules which combine four Power5 chips, providing a "whole bunch" of CPU buses on a single chip improving cross-communication, according to the team.

"In the process a lot of tests on Linux were performed," Blanchard said, adding that the benefits for Linux included improvements to its out-of-memory killer, SCSI layer locking, and the e1000 gigabit net driver.

Also discovered during testing were issues with scalability and performance, random driver locking, and SMT scheduler tuning.

The group"s Linux test machine was a 64-processor (128 threads) Power5 system with 128GB of memory and 640 PCI slots.

To benchmark the system"s performance, the group used the Kernbenchrate application which indicates the number of kernels a machine can compile per second.

The results were impressive with five kernels compiled in about six seconds but 10 kernels compiled in under 10 seconds, he said.

"Managers realize Linux is a good tool as the traditional way of bringing up a system requires the use of firmware," Blanchard said.

Asked about when non-Apple Power-based computers will be available to consumers, Blanchard replied, "next question".

IBM"s LTC Canberra manager, Hugh Blemings, said the local group has established itself in the community for its work on large SMP support in the kernel.

"Linux is playing an increasing role in the development of Power-based systems," Blemings said, adding that the LTC work is being merged upstream into the standard kernel.

Blemings said the group also works on large-page support which can improve the performance of applications that require large amounts of memory.