NYSE Euronext has been using for the past nine months to store and analyze data to determine where there are slowdowns in the process of taking, executing and acknowledging trades for stocks, bonds and other securities, according to chief data officer Steve Hirsch.
The time for an order to arrive at the NYSE's network, be executed, and then have an acknowledgement, or 'ack,' sent back to the trading party, is already fast: 1 to 2 milliseconds, Hirsch said. But using Greenplum to analyze bottlenecks in this system will help the NYSE "get closer to zero," he said in an interview Thursday.
The Greenplum software includes both data warehousing and BI analytical capabilities. This allows the appliance, which includes a from and fiber-attached storage totaling 100 TB in capacity, to analyze the data itself, rather than sending large amounts of data to a separate server for analysis.
"This makes us hundreds to thousands of times faster. It's incredible," Hirsch said. He cited one task that formerly took 12 hours to run and now takes "just minutes," and another report that formerly took 50 to 60 hours to generate and now "runs in a single hour."
Accelerating trade times is an important but new emphasis for the tradition-bound, 216-year-old NYSE. It only two years ago while maintaining traditional human auctioneering.
This was partly spurred by the NYSE's merger in 2005 and
The NYSE has been using Netezza's appliances slightly longer than GreenPlum's appliance in order to track and audit trades and orders, Hirsch said.
Before selecting these products, Hirsch said he "evaluated most of the products out there," including of Microsoft, and a that includes SAS analytics onboard.
Both GreenPlum and Netezza are far faster than its prior infrastructure, which primarily consisted of Oracle data warehouses paired with BI servers running SAS, Hirsch said. Having separate servers for data warehousing and BI "doesn't work when you are processing terabytes of data per day ... it doesn't scale to our current levels."
Hirsch said that despite the NYSE doing "pretty sophisticated types of analytics," it hasn't run into problems where the Greenplum or Netezza appliances could not perform desired analytical runs.
"We look at Greenplum as the leader in terms of software-based solutions, and Netezza as the leader in hardware-based solutions," he said.
Hirsch declined to say how much he paid for the Greenplum and Netezza appliances. Greenplum has said it of user data, while about $29,000 per TB for its 10100 data appliance.
The NYSE is about to go live with a latency monitoring system that will plug into Greenplum and use Global Positioning System (GPS) signals to synchronize measured times within the network, for better precision and accuracy, as well as scale up to billions of rows of data per day while performing "near-real-time analysis."