updated 5/21/2010 11:43:47 AM ET 2010-05-21T15:43:47

Clarence Woods was attending a community college near Baltimore in 1982 and playing drums in a wedding band when one day, to his surprise, the financial world beckoned. "They were so desperate for anyone who knew anything about computers," Woods recalls. "If you could spell the word, you had the job." He swiftly moved from the back office at the brokerage Legg Mason to Equitable Bank, where he installed Quotron stock price machines. In 1985 he made his first purchase as a rookie trader: 100 shares of IBM on the New York Stock Exchange.

On May 6, Woods, now 47, realized how radically his industry has changed. He had just started his own hedge fund after quitting as chief equity trader at MTB Investment Advisors, a $13 billion money manager in Baltimore. He was working at home when the trouble hit. "I was doing some small trades, and had a lot of puts set up, and all of a sudden they just went berserk," he says. "Then I started to panic."

His first thought was that cyber-savvy extremists had infiltrated the fiber-optic network on which automated programs now trade securities tens of thousands of times a second. With no terrorism reports surfacing, Woods shifted his suspicion to the nature of the contemporary market itself: hyper-accelerated, decentralized, and, in important ways, beyond human supervision. "I thought, 'My God, I bet you're sure going to miss the New York Stock Exchange now.'"

A lot of people felt nostalgia for Big Board dominance on May 6. The fleet computers that drive today's securities industry are astounding — and unsettling. "Wall Street is no longer what it was designed to be," Mark Cuban, the tech entrepreneur, veteran investor, and owner of the Dallas Mavericks basketball team, blogged after watching the frantic selloff. "Wall Street is now a huge mathematical game of chess where individual companies are just pawns."

Hysterical Thursday did no apparent long-term harm. Some venerable stocks dropped to a penny apiece before bouncing back. Overall, the Standard & Poor's 500-stock index declined 6.2 percent, from 1,136.16 to 1,065.79, in a 20-minute span—an $862 billion paper loss—before recovering to finish down 3.2 percent.

Still, the brief crash threw up a flare that illuminated a financial topography that was unfamiliar even to many experienced investors. A Bloomberg Businessweek investigation into those harrowing minutes revealed the extent to which the market is now dominated by quick-draw traders who have no intrinsic interest in the fate of companies or industries. Instead, these former mathematicians and computer scientists see securities as a cascade of abstract data. They direct their mainframes to sift the information flows for minute discrepancies, such as when futures contracts fall out of sync with related underlying stocks. High-frequency traders (HFTs), as they're known, set an astonishing pace. On May 6, 19 billion shares were bought and sold; as recently as 1998, 3 billion shares constituted a very busy day.

The HFT wizards argue that all that extra buying and selling provide the liquidity that makes the market more efficient. As long as the machines are humming, electronic bids and offers abound. On May 6, however, we saw what happens when digital networks follow conflicting protocols and some of the mighty computers temporarily power down. Liquidity evaporates. Panic combined with automation leads to much faster panic.

The decline began midmorning as skittishness intensified over the Greek economic debacle spreading elsewhere in Europe. A closely watched gauge of volatility calculated by the Chicago Board Options Exchange hit a high point for the year at 2:08 p.m. The volatility index, or VIX, is derived from options on the S&P 500, and it measures investor perceptions of market risk. When the VIX surged again, in its biggest gain in three years, some high-frequency programs may have automatically slowed their normal pace to limit losses, according to a May 15 research note by Nomura Securities.

Sell orders piled up much faster than buys, an imbalance that worsened over the next hour. During the period of heaviest selling, starting around 2:30 p.m., the NYSE paused electronic trading in certain stocks and switched to computerized auctions conducted by human traders. This caused electronic sell orders to be rerouted to other trading venues, where there were few, if any, buy orders to absorb them. As Mary L. Schapiro, chairman of the Securities & Exchange Commission, put it in congressional testimony five days later, some high-frequency firms "withdrew their liquidity after prices declined rapidly."

During the next few hours of confusion, exchanges began canceling trades in hundreds of stocks. NYSE Arca, an electronic platform operated by the Big Board, erased transactions in 295 companies. A surge in trades rejected by exchanges constitutes another trigger that automatically causes some high-frequency firms to slow down, says Ethan Kahn, a principal at Wolverine Trading, an electronic market-making outfit in Chicago: "You disable. You shut down." Wolverine pared back activity in equity futures because of concerns about the accuracy of data it was receiving, he adds.

SEC investigates
In Washington, the staff at the SEC began reviewing up to 10 terabytes of market data to figure out what happened. Twelve days later, on May 18, the agency conceded that it still couldn't offer a firm answer. That uncertainty in itself suggests the disquieting complexity the stock market now presents.

The SEC and the Commodity Futures Trading Commission issued a preliminary report in which they outlined six hypotheses that could explain the scare. "We continue to believe that the market disruption of May 6 was exacerbated by disparate trading rules and conventions across the exchanges," Schapiro said upon the report's release. As one response, the SEC proposed that exchanges halt trading in individual stocks that swing more than 10 percent during a five-minute period. The new "circuit breaker" rules are subject to commission approval after 10 days of public comment.

While temporarily slowing trading during periods of investor high anxiety makes sense to regulators, at least some high-frequency traders disagree. "I don't think that's the right solution," Wolverine's Kahn told Bloomberg News after the SEC announcement. "It could cause a lot of complications. On a busy day where the market is making major moves, you'd have a handful of [stock] names where it's circuit breaker-on/circuit breaker-off all day."

As this debate unfolds, one danger is that regulators, politicians, and industry executives — already distracted by how to reform Wall Street in the wake of the broader credit crisis of 2008 — will shrug off May 6 as a weird blip requiring no fundamental rethinking of how man, machine, and market interact. Absent so far from the public discussion is any talk about whether the next quickie-crash might coincide with an outside event that shakes investor confidence much more severely: Iran and its nukes, industrial-environmental disaster, North Korean aggression. Or all of the above.

As we just saw when major investment banks suffered blindness to the toxic effects of mammoth leverage, exotic credit derivatives, and a nationwide housing bust, Wall Street's computer models tend to fail when unpredictable disasters overlap.

For generations, the Big Board played the vital role of establishing prices for most major stocks. Even after rudimentary computers arrived in the 1960s, living, breathing people continued to supervise the proceedings. Beginning in the 1970s, Nasdaq, and, later, additional electronic rivals, gradually eroded the NYSE's dominance. Humans could intervene if things got too strange.

That has changed. Today, hedge funds and HFT shops move enormous quantities of stock in fractions of a second. Firms jockey to place their computers near the mainframes of wholly automated trading venues you've never heard of in Jersey City, N.J., and Kansas City, Mo. The speed-of-light traders do this because the distances that their orders travel, measured in feet, can determine profit or loss.

Directives from Washington have encouraged the dispersal of trading. Some 50 exchanges and other electronic venues across the country now compete for securities business. The volume of equity traffic controlled by the NYSE fell from 80 percent in 2005 to 50 percent in 2007 and then to less than 25 percent this year. The exchange floor swarming with brokers in colorful jackets has become little more than a theatrical backdrop for cable TV correspondents.

The vast majority of the action occurs elsewhere. High-frequency traders now account for as much as 60 percent of daily volume, according to Tabb Group, a research firm. The most prolific HFT outfits, such as Getco in Chicago, Tradebot Systems in Kansas City, Mo., or RGM Advisors in Austin, Texas, can individually generate as much as 5 percent or 10 percent of all the stocks traded in the U.S. on a given day.

"The world has totally changed in the last 15 years," says Fred Federspiel, who started Pipeline Trading Systems in 2004. Pipeline, based in New York, belongs to yet another new breed: "dark pools" that allow major trades to take place out of public view. Before getting into finance, Federspiel, who holds a doctorate in nuclear physics, worked at Los Alamos National Laboratory in New Mexico, using particle accelerators to determine if subatomic neutrinos help hold the galaxy together. Trading models built by people with this sort of training tend to be based on a view of the market that is data-driven and news-agnostic.

Wall Street's extreme makeover has achieved its main goals: greater efficiency and much lower commissions for the pension and mutual funds, insurance companies, and endowment managers that invest in equities. Reduced transaction costs benefit teachers, office workers, corporate executives, and retirees from coast to coast.

At the same time, the transformation has created new risks, some of which were momentarily perceptible on May 6. "What happened [that day] is completely unacceptable," says Richard Gorelick, RGM's chief executive officer. Gorelick, who formerly helped run the Internet company, turned 39 on May 6. "I got a lot of birthday calls," he says, "followed by, 'What the hell happened?!'"

Drastic market changes
Manuel A. Henriquez is no Luddite. He has invested in technology companies for 23 years and runs Hercules Technology Growth Capital (HTGL), a publicly traded venture firm in Palo Alto, Calif. On May 6 he gaped as Hercules shares fell from $9.50 to $5.22 in less than 33 minutes. The 46-year-old father of two girls, ages 12 and 10, has $6 million of his own money tied up in Hercules. For a while that afternoon, he thought he had lost half of it. "I literally called my wife and said, 'Give the kids tennis rackets; they're going to have to get scholarships.'"

Hercules shares closed on May 6 at $9.56 after trading at twice their normal volume. If the Henriquez girls take up tennis, it will be for the love of sport. Still, their father remains shaken. "It's like the movie '2001: A Space Odyssey,' " he says. "You can't have Hal the Computer make all the decisions for you. We need to synthesize the human element of logic and say, 'Wait a minute, I've got an order that has never been of this size. This seems to be an anomaly.'"

When Henriquez started investing in the mid-1980s as a college student in Boston, he set aside $20 or $30 a week to buy shares. In that era, broker Richard Rosenblatt and his breed still had tremendous influence over the running of the Big Board, and, by extension, over the entire market in stocks. "I was much younger and pretty quick," Rosenblatt says. "I was the high-frequency trader at the time."

Rosenblatt Securities, launched in 1979, executes buy and sell orders for mutual funds and other money managers. When carrying out trades back then, Rosenblatt could see everyone he was dealing with. Specialists positioned at designated posts on the floor "made markets" in stocks they were assigned. They had the responsibility of buying even when a company's shares were falling. Big investors relied on their brokers to buy low and sell high. Unscrupulous traders could — and sometimes did — put their own interests ahead of those of their clients. Aside from exchange rules and the remote danger of prosecution, the trust among brokers and specialists provided the main impediment to fraud.

Four years before Rosenblatt started his company, though, Congress had signaled that the era of traditional markets wouldn't last forever. Lawmakers directed the SEC to reduce the NYSE's near-monopoly and replace it with "a national market system." The goal: greater competition and lower expenses for investors. Rosenblatt embraced electronic trading and survived. Many of his cohorts did not.

The SEC banned fixed minimum commissions and imposed rules that strengthened the upstart Nasdaq, which had begun operations in New York in 1971. Improved technology contributed to rising volume. Much of the additional trading was generated by firms using computer software to identify momentary pricing discrepancies, as opposed to longer-term investors hunting for corporate value. In the 1980s, "program traders" bought or sold large portfolios of securities in a single order, while simultaneously making offsetting bets on related futures contracts. Designed to hedge risks, program trading at times contributed to greater volatility. On Oct. 19, 1987, it helped accelerate the Black Monday market crash.

Historically, the NYSE and Nasdaq were nonprofits seen as utilities that served the public interest in matching investor resources with corporate enterprise. They evolved into for-profit corporations fighting for survival. Newer profit-making exchanges started explicitly to benefit the firms that ran and patronized them. "They're more competitive, more self-serving, and they've moved more away from the utility concept," says Rosenblatt. "Maybe it's gone too far."

SEC rule changes adopted in 1998, 2005, and 2007 aimed at ever-lower transaction costs by encouraging formation of electronic communication networks, or ECNs, that challenged the NYSE and Nasdaq for market share. Robert Colby, who oversaw the modernization process for 15 years as deputy director of the SEC's Trading & Markets Div., says the march of technology was irresistible. "Electronic trading allowed new automated markets to spring up and compete head-on with established exchanges and market makers," he says.

David Leinweber, a finance professor at the University of California at Berkeley, helped create one of the first algorithmic trading strategies in 1989. Originally called Market Mind, it allowed computers to execute securities orders entirely on their own. Leinweber had studied math and physics as an undergraduate at the Massachusetts Institute of Technology. His preparation for the Market Mind breakthrough also included research he had done at the government-funded RAND Corp. think tank. There, he helped improve communication systems and real-time data analysis for the space shuttle. In the late 1990s he worked as a partner at First Quadrant, a Pasadena (Calif.) investment firm. "I was managing equities all over the world at that time," he says, "and pretty soon there were just no [trading] floors to visit. You switched to trading electronically."

Digital networks such as Island ECN and Archipelago usurped the human trading floors by offering rebates on trading and faster execution. These innovative virtual trading sites lacked the cadres of specialists and market makers obliged to maintain orderly trading. In their place were "liquidity providers" — brokerages willing to post electronic bid and ask quotes, but free of institutional duties.

Seemingly minor advances had profound consequences. The switch in 2001 to decimal share pricing, from sixteenths of a dollar, gave investors greater flexibility. Any firm willing to sell for a penny less than the best available offer price could step in and make the trade. A traditional Nasdaq market maker that had bought at $10 and sold at $10.0625 found itself in a lower-margin business. This favored the new electronic communications networks and the ever-speedier high-volume traders seeking microscopic profits across a multitude of transactions.

Market making moved from exchange floors to computers. On its Web site, Wolverine in Chicago says its servers receive direct data feeds from more than 15 exchanges and execute more than 1.5 million orders a day. The culture of the industry changed, too. Membership on a college sports team no longer constitutes a ticket to an entry-level job. In the past, Wall Street traders "were taller, bigger, more aggressive," says Kahn, who oversees market making at Wolverine. Today's HFT firms, by contrast, look for "more of a quant-computer-type person," says Kahn, who has a finance degree from the Wharton School.

The quants use a range of strategies. One is simultaneously posting bids and offers for ever-changing amounts of a single stock. Prices tend to vary by minuscule amounts on different electronic exchanges, so a stock can be bought at a lower price on one, then sold instantly at a higher price on another. The profit could be as little as a hundredth of a cent per share, which, multiplied by millions of shares a day, adds up to real money.

One thing that apparently happened on May 6 is that when HFT firms reacted to the market's sudden moves by slowing their computers or switching them off, buy orders that had been in place only seconds earlier disappeared, causing disequilibrium. HFT behavior, to be sure, wasn't the only factor that turned a down day in the markets into an abrupt collapse. Conflicting rules among exchanges also played a role.

RGM's Gorelick says his HFT firm, founded in 2001, continued to trade during the market turmoil. Its software developers and IT support people crowded onto the trading floor in Austin and quietly stared at the computer screens. Gorelick says, however, that high-frequency buying and selling wasn't so much the problem. He blames "old-fashioned human panic," worsened by inconsistent policies among exchanges.

Kahn disagrees. "The system broke down" on May 6, he says. "There should be and will be structural changes in the future." New regulation is coming, he adds. "We just don't know what it will be."

Schapiro has said she hasn't assessed any blame yet. HFT firms that pulled back may have acted appropriately, given that they had no legal obligation to do otherwise, she told Congress. In addition to its circuit-breaker proposal, the SEC is expected to consider requiring high-frequency traders to continue to make markets, even during a major selloff.

Some would resist such a mandate. "No one should be forced to provide liquidity when CNBC is showing riots in Greece in the morning and there are worries the bailout of Greece and Portugal will fall apart and they'll default on their debt," says Pipeline's Federspiel.

On another front, some lawmakers have proposed enacting a tiny tax on each equity trade. Such a levy would likely discourage some high-frequency trading, slow the market's pace overall, and raise billions in revenue for the federal government. Some of the tax proceeds could be used to bolster SEC monitoring.

After the frenzy of May 6, Clarence Woods of Baltimore says he ended up more or less where he started. He's moving ahead with plans for his new hedge fund and counts on the SEC to reassess the market: "We'll sit back, see what works and what doesn't." Manuel Henriquez, the Palo Alto venture capitalist, acknowledges "a visceral reaction to pull all the computers out." He doesn't think that's feasible, though. "We need to continue to embrace technology, but understand that technology can bite both ways."

Matt Andresen helped launch Island ECN in 1996 and later oversaw market making at Citadel Investment Group, which he left in March 2009. "The impact of the high-frequency tool has been, I believe, very democratizing," he says. Brokers serving individual investors can execute orders more quickly and less expensively — at least on most days. Then there was May 6. "Whatever the cause of it," Andresen says, "the failure mode was unacceptable."

— With Jeff Kearns, Whitney Kisling, and Peter Coy

Copyright © 2012 Bloomberg L.P.All rights reserved.


Discussion comments


Most active discussions

  1. votes comments
  2. votes comments
  3. votes comments
  4. votes comments

Data: Latest rates in the US

Home equity rates View rates in your area
Home equity type Today +/- Chart
$30K HELOC FICO 3.79%
$30K home equity loan FICO 4.99%
$75K home equity loan FICO 4.69%
Credit card rates View more rates
Card type Today +/- Last Week
Low Interest Cards 13.83%
Cash Back Cards 17.80%
Rewards Cards 17.18%