As wonderful as Internet search engines are, they have a pretty big flaw. They often deliver too much information, and a lot of it isn't quite what we're looking for. Who really bothers to read the dozens of pages of results that Google generates?
Some intriguing technologies are getting better at bringing order to all that chaos, and could revolutionize how people mine the Internet for information.
Software now emerging analyzes search results and automatically sorts them into categories that, at a glance, present far more information than the typical textual list.
"We enliven the otherwise deadening process of searching for information," said Raul Valdes-Perez, co-founder of Vivisimo Inc., which quickly puts search results into clickable categories.
Pittsburgh-based Vivisimo sells its technology to companies and intelligence agencies, and offers free Web searches at Vivisimo.com.
A superfast librarian
Valdes-Perez describes his company this way: If the Internet is a giant bookstore in which all the books are piled randomly on the floor, then Vivisimo is like a superfast librarian who can instantly arrange the titles on shelves in a way that makes sense.
Consider it a 21st century Dewey Decimal System designed to fight information overload. But unlike libraries, Vivisimo doesn't use predefined categories. Its software determines them on the fly, depending on the search results. The filing is done through a combination of linguistic and statistical analysis, a method that even works with other languages.
A similar process powers Grokker, a downloadable program that not only sorts search results into categories but also "maps" the results in a holistic way, showing each category as a colorful circle. Within each circle, subcategories appear as more circles that can be clicked on and zoomed in on.
It takes a few minutes to get used to Grokker. But the value of its nonlinear approach quickly becomes clear.
Let's say, for example, you're curious about accommodations in France and enter a search for "Paris Hilton."
Google recognizes this as a search in the category of "Regional-Europe-Travel and Tourism-Lodging-Hotels" but still produces page after page with links about celebrity socialite Paris Hilton and her exploits. That's because Google's engine ranks pages largely based on how many other sites link to them, sending the most popular pages to the top.
Finding what's important
If you run the search on Grokker, however, the resulting circle shows all the possible categories of information the Internet offers on a search for "Paris Hilton"—including reviews, maps and online booking sites for the Hilton hotel in Paris, which are all but buried in the Google rankings. Now you've much more quickly found not what is popular among Internet gawkers, but what is genuinely useful to you.
Groxis Inc., the 15-person company that introduced Grokker last year and released an upgraded, $49 second version in December, is not out to replace Google. Grokker is not in itself a search engine—it only analyzes and illustrates search engines' results.
For example, Grokker2 can categorize and map files on your hard drive, arranging them by content, not by the folders you happened to put them in, or listings on Amazon.com. If you use Grokker2 to search the Web, it combines results from six search engines—Yahoo, MSN, AltaVista, Wisenut, Teoma and FAST, a business-focused product by a Norwegian company.
In 2004, Grokker plans to release up to two dozen downloadable plug-ins that will set its colored circles loose on a wider variety of databases, including the Library of Congress, news Web sites and yes, Google itself.
"We now have the capability to `grok' anything," said R.J. Pittman, chief executive of Sausalito, Calif.-based Groxis. Would-be Grokkers, a note of caution: it requires Windows 2000 or XP or Mac OS X.
The Google plug-in is partly a market test; Google and Groxis will analyze how well it works and then consider whether to work on developing a service together, Pittman said.
Google spokesman Nathan Tyler declined to comment on Groxis. Nor would he say whether Google is exploring its own measures of sprucing up search pages with categorization tools like Vivisimo or visualization aids like Grokker.
Search sites get better
Another visualization possibility is offered by TouchGraph LLC, which has a Google plug-in that shows links as an interconnected web, an appropriate image for the World Wide Web.
Such tools have been applied by the Manhattan firm Plumb Design in its Visual Thesaurus, which maps a word's meanings, or in a navigation tool it developed for a Smithsonian Institution exhibit.
Meanwhile, a number of search sites have gotten hip to honing results.
For example Teoma, which is part of Ask Jeeves Inc., suggests ways to refine or narrow a search. That means a Teoma search for "Las Vegas" will serve up roughly the same links as other sites, but it also suggests "Vacation Packages" and "Vegas Wedding Chapel."
"Search has to evolve," Pittman said. "It can't just be Google sitting there with a stash of places they've crawled on the Web. People are becoming more astute and demanding better results, and they're demanding a more powerful search experience. People like to get a landscape of information once they've found out there's one available."