Nasdaq's Facebook glitch came from race conditions

21.05.2012

Nasdaq characterized the problem as a race condition. A race condition occurs when two or more parts of a program that rely on each other get locked in an infinite loop, halting forward progress of the program as a whole. The race condition in this case took place between the auction process and the final calculation of the IPO price.

While most race conditions can be identified by simply testing programs while in development, in some cases they may not become apparent until the software is stressed under heavy usage, said Scott Sellers, CEO of software provider Azul Systems. Azul offers a version of the JVM (Java Virtual Machine) engineered for extreme scalability.

While Sellers had no direct information about this Nasdaq system, he did say that many of his customers and potential customers -- which include other financial exchanges -- have experienced race conditions only under heavy loads, as multiple threads in the program interact with one another in sometimes unpredictable ways. So the frenzy of trading around the Facebook IPO might have been enough to expose a flaw in the software, he said.

"A surge in demand can be extraordinarily difficult to model in pre-production staging," he said. "It is the corner case conditions that usually occur in high volumes. If [an organization] has not built a platform that is scalable enough, it can get in trouble."

Oddly enough, the IPO of another company, Zynga, also scheduled that day, came off correctly, even though the IPO Cross auction period was similarly heated. In this case however, all auction bids were reconciled before the final trading price was issued, at 12:27 p.m.