New Zealand home to largest on-demand supercomputer

18.05.2005
Von Stephen Bell

New Zealand is home to what Scott Houston, former CTO at Weta Digital Ltd., claims is the largest assembly of processors offered for ?on demand? outside use in the world -- the Weta Digital array, now over 1,000 blade processors strong, each clocking 2.8GHz.

Weta Digital creates digital graphics, particularly special effects for movies, and was instrumental in the Oscar-rated success of the Lord of the Rings trilogy.

University of Auckland researchers are using 48 of the processors to simulate the spread of HIV and its progression to Aids among large populations. Research is also in progress on the Weta array on the genetics of crustaceans and the identification of the species -- "and even the individual pod" -- of whale meat samples, Houston says.

Descending from the heights of academic research, the first major project using the array was the design and simulated walk through of a NZ$25 million (US$18 million) super-yacht, he says.

The processors are being applied to land-based architecture through a partnership with Auckland multimedia and design company Urban Voyage. Architects are even busier than film-makers, Houston says, and it could hold up a job to have to send everyone home at the weekend in order to have enough processing cycles for a big rendering task.

Houston sees such partnerships marrying powerful hardware with specialist knowledge and software as a growing part of the supercomputer market.

Supercomputer-based research is potentially handicapped by the need for secure fast communications, particularly where reference data resides overseas, says Houston. There is probably a case for concluding agreements to import ?the basic data? in any domain, such as the sequence of the human genome for biotechnology, so it could be kept available locally.

That the supercomputer is based in New Zealand could have great appeal to members of nonaligned nations, he says. Research establishments and private companies that may have used U.S.-based supercomputer services in the past could well be nervous of surveillance of their traffic into the U.S. as part of the Homeland Security effort, he says.

Grid computing is an emerging potential use of supercomputers and arrays of less powerful machines, with Sun boss Scott McNealy predicting it as the ?fourth big wave? of success for the company, most recently in an address to media and analysts on Sun?s California campus in February.

Some problems, such as weather simulation, cannot be split into discrete parts for grid processing and will still require massive parallel processing on a single site, Houston says; but many problems are amenable to grid processing.

For the links of the grid, many organizations simply used the internet, he says, but Supercomputing Center partner Gen-i and its owner, Telecom, stand ready with virtual private network technology in cases of greater security need.

Federated use of many computers had its early trials on massed ranks of PCs in such projects as SETI, the search for extra-terrestrial intelligent life. ?Given sufficient security, there is no reason why [a supercomputing grid] couldn?t use your processors,? he says.