Relational databases are great, but not for everything, Michael Stonebraker says.
He should know. The former computer science professor at University of California, Berkeley is a bit of a legend in the database world, having created two well-known relational database systems, Ingres and Postgres. Ingres, the company Stonebraker founded, is now part of Computer Associates International. Stonebraker also founded Illustra, a database company that was acquired by Informix, which later was acquired by IBM.
Now Stonebraker has launched a new database software company to tackle one of the toughest jobs in computing -- analyzing huge amounts of streaming data on the fly.
It is a job Stonebraker says ordinary database programs made by IBM, Microsoft and Oracle simply cannot handle.
"Relational databases are one to two orders of magnitude too slow," says Stonebraker, who is chief technology officer at Streambase, a 25-person outfit based in Lexington, Mass. "Big customers have already tried to use relational databases for streaming data and dismissed them. Those products are non-starters in this market."
In a recent pilot program, Streambase was able to analyze 140,000 messages per second, while a leading relational database -- Stonebraker won't say which one -- could handle only 900 messages per second. Streambase has 12 customers now testing its software, all of them financial services companies that need to analyze rapid-fire ticker feeds and other streaming data.
Steam processing engine
Stonebraker calls his product a stream processing engine. On top of that engine, customers write applications to handle specific tasks, using a version of Structured Query Language that traditional database programs use. Streambase's version is called StreamSQL and is designed to handle data on the fly.
Unlike traditional database programs, Streambase analyzes data without storing it to disk, performing queries on data as it flows. Traditional systems bog down because they first store data on hard drives or in main memory and then query it, Stonebraker says.
The company's approach to making sales is pretty simple: "We ask big customers to point us to their hardest problems. Then we say go home and write the application on our own nickel and come back in a week with it running," Stonebraker says.
Streambase grew out of research conducted in computer science departments at the Massachusetts Institute of Technology, where Stonebraker has been teaching, as well as Brown University and Brandeis University. The three universities hold equity stakes in Streambase, alongside venture backers Bessemer Venture Partners and Highland Capital Partners.
Streambase runs on industry standard servers running Intel microprocessors and the Linux operating system, and on Sun Microsystems servers running Solaris. A version that runs on Microsoft Windows will be available soon.
Streambase charges customers annual subscriptions for its software, setting prices based on how many CPUs a customer uses to power the software. Typical deals so far have ranged from $100,000 to $300,000 a year, says Barry Morris, Streambase's chief executive.
For now Streambase is focusing attention on financial services companies, which hope to do things like track how well traders are performing on a real-time basis, rather than aggregating trades at the end of the day and analyzing them overnight.
A bigger opportunity involves processing real-time data feeds generated by sensor networks and RFID tags. A military contractor wants to use Streambase to keep track of soldiers and vehicles in the battlefield. A casino in Las Vegas is considering using Streambase to track the performance of individual gamblers.
"There's just going to be a huge green field of opportunity as everything gets sensor-tagged," Stonebraker says.
Who says there's nothing cool going on in high-tech these days?