Shutdown at Nasdaq Is Traced to Software

The Nasdaq offices in Times Square on Aug. 22, when the exchange shut down for three hours. Jason Szenes/European Pressphoto AgencyThe Nasdaq offices in Times Square on Aug. 22, when the exchange shut down for three hours.

Though the Nasdaq market calls itself home for the stocks of the world’s biggest technology companies, the exchange acknowledged on Thursday that a three-hour halt in trading arose from a problem with its software.

The Nasdaq OMX Group, the parent company, on Thursday released preliminary findings that provided the clearest official insight into what caused the trading halt, being called in trading circles as the “flash freeze.”

Related Links

While stock prices were little affected when the exchange reopened late in the afternoon of Aug. 22, the episode reignited concerns about the fragility of modern markets and their dependence on intricate software systems.

In particular, a series of attempts by a market operated by the NYSE Euronext to connect with the Nasdaq system that reports the prices of recent trades generated a surge of data. That led to a failure of Nasdaq’s backup systems, forcing the market to go offline to fix the problem.

“Our job here is obviously to reduce as much as we can any trouble we have, but we certainly understand that code has this nefarious way of working and then not working,” Robert Greifeld, Nasdaq’s chief executive, said in a telephone interview. “Our job then is to mitigate the impact.”

According to Nasdaq, the trouble began just after 10 a.m. on Aug. 22 when Arca, one of the exchanges run by NYSE Euronext, began having trouble connecting with the system that provides prices for recent trades in Nasdaq stocks, which Nasdaq operates. Arca tried to connect more than 20 times to the system.

As part of its standard procedure, it also sent over a number of zero-dollar quotes, meant to ensure that no stale trades would be sent out to the market.

All told, the data sent greatly exceeded the amount that Nasdaq’s system — known as the Securities Information Processor, or SIP — was intended to handle. People with direct knowledge of the matter said that Nasdaq’s system was not designed to “throttle,” or forcefully slow down, the flood of information. That meant that the data was hitting the SIP at full force.

Ultimately, Nasdaq turned off Arca’s access to the system around 11 a.m.

By 11:17 a.m., Arca’s systems appeared to be working fine.

But the immense amount of data sent by Arca overwhelmed Nasdaq’s SIP, prompting the system to resort to backup servers. A flaw in the backup software then emerged, forcing Nasdaq to shut down the entire system. At 12:14 p.m., the exchange sent a notice to traders notifying them of the complete market halt.

Nasdaq fixed the problem within 30 minutes of halting trading, but it took additional time to contact other markets and regulators.

Trading resumed around 3:30 p.m.

Mr. Greifeld noted specifically that he was not blaming Arca for the shutdown, accepting responsibility for Nasdaq’s share of the problems.

He also pointed to what he described as broader issues affecting the stock market industry as a whole, including information security and data capacity.

“They obviously had issues, and it caused an event,” Mr. Greifeld said, referring to the NYSE exchange.

“We obviously had issues, we should be able to handle that. We were supposed to be able to fail over, and we did not.”

A NYSE Euronext spokesman, Richard Adamonis, declined to comment. But a person close to the exchange said that it was reviewing its backup systems.

At the same time, executives at the Big Board’s parent have complained that Nasdaq initially did not adequately acknowledge responsibility for the problem last week.

On Thursday, Nasdaq said that it was “deeply disappointed” in its performance on Aug. 22 and called it “unacceptable to our members, issuers and the investing public.”

Robert Greifeld, Nasdaq’s chief executive, is set to meet with industry leaders next week to discuss the software problems. Brendan McDermid/ReutersRobert Greifeld, Nasdaq’s chief executive, is set to meet with industry leaders next week to discuss the software problems.

Mr. Greifeld, who has staunchly defended Nasdaq’s response to the market halt amid criticism that the company offered little information about the troubles, added that the market operator was studying additional ways to post information in real time. They include using social media like Twitter; other ways of automatically posting information; and opening up additional phone lines for market participants to hear directly from Nasdaq officials.

The preliminary report on Thursday is far from the last word on the subject. Nasdaq is set to meet with an industry group to discuss the issue next week.

Officials from both exchanges are scheduled to speak with the chairwoman of the Securities and Exchange Commission, Mary Jo White, on Sept. 12.