In an engineering lab at the University of Southern California, graduate student Arjun Rihan pulls on a pair of gadget-laden gloves and begins pointing at a computer-generated map of the main campus. As Rihan gestures at the plasma screen, the grid of streets and buildings begins to whirl.
"We have liftoff," Rihan jokes. Within seconds, he has focused the map down to a campus plaza. Tiny figures walk back and forth, and cars drive along the adjacent street. It looks like a realistic video game.
But this is no game. Instead, the scene blends virtual imagery with five video streams from cameras mounted just outside the lab building. Those are real people in the picture.
The USC software project — known as Geospatial Decision Making, or GeoDec — delivers the kind of real-time data layering that was made up for books like "Snow Crash," movies like "Minority Report" and TV shows like "24." Adam Clayton Powell III, director of USC's Integrated Media Systems Center, says that "it's fictional if you're watching '24,' but it's real in here."
More than visualization
GeoDec takes visualization databases such as Google Earth and MSN Virtual Earth to the next level, and indeed both those ventures have taken an interest in the project.
(MSNBC.com content is distributed by MSN. MSNBC itself is a Microsoft - NBC joint venture.)
GeoDec, however, is much more than a visualization program, says computer scientist Cyrus Shahabi, one of the leaders of the USC research team.
"Once you have all the data about the area, in addition to just visualizing what's there with the 3-D models and the video and so on, you can start asking questions," he told MSNBC.com this week. "You might say, 'I want to see all the accidents in this area.'"
Or you might pose more complex queries: What's the optimal time and place for taking a bus from point A via transfer station B to point C? Which trees need to be trimmed because they're getting too close to the power lines? Which routes of travel might leave a target open to a sniper's line of sight? That's where the "decision-making" aspect of GeoDec comes to the fore.
GeoDec could conceivably be applied to fields ranging from urban planning to emergency response to military surveillance — with multiple agents accessing the same multilayered database. "It's Web-based," Rihan said during a demonstration earlier this month, "so you could have somebody in another aircraft or location looking at this on a PDA, so you're looking at the same information."
After the demonstration, a military visitor said privately that such a system could come in handy in Iraq. "We just did not have anything close to this that could take numerous viewpoints and combine them," he told MSNBC.com.
Quick ‘wrap’ for any database
Closer to home, Shahabi said the GeoDec can easily knit together geographic information with other data, such as street addresses, real-estate assessments and phone numbers. It sounds like a daunting database task, but Shahabi said the system is designed to capitalize on already-existing reservoirs of online information.
"You don't really need to manually create a database," he explained. "We utilize all the data sources already out there. We 'wrap' them so it looks like it comes from our database, but it actually comes from the original source."
The program also is designed to blend data from the different sources to create a coherent picture rather than a hodgepodge. The usual method for building a computer landscape requires processing loads of laser-ranging data, then struggling to reconcile inconsistencies in the 3-D maps. GeoDec can crank out 3-D maps automatically, using two different aerial images of the same area to add perspective. And the results are produced much faster.
Exactly how much faster? "I don't feel comfortable giving a number here, since this depends a lot on the accuracy of the sources and how consistent or aligned they are," Shahabi said. "In general, at GeoDec, we are building tools and a general framework to make this type of application-building fast and accurate."
There's lots of work yet to be done to turn GeoDec into a commercial product. "That I think will not happen in academia, and would need a company to take over after we present the vision, publish the research result and build proof-of-concept prototypes," Shahabi said.
Wow factors
But GeoDec already has a couple of wow factors going for it. Those haptic gloves, for example: Gesturing with the gloves just works better than clicking a mouse, particularly when you're sifting through mounds of data displayed on a big screen. There may be still more tricks to come on that front, although Shahabi emphasized that the interface is not the main focus of his team's research.
"Several researchers suggested that a combination of hand gestures and voice commands, in addition to mouse and keyboard, is the ultimate interface," he observed.
The integration of real-time video into a data-rich, virtual-reality environment is perhaps the biggest wow factor — and it also points up GeoDec's potential power as a surveillance tool. Theoretically, a "Minority Report" investigator could follow a target from video to video through the 3-D environment.
During the demo, the only video shown was from the plaza in front of the lab building — not because that's all the video that's available, but because the team is not allowed to use video from a building's security cameras "unless we're in that building," Rihan said.
Shahabi acknowledged that the video angle might raise privacy concerns, but he pointed out that the software itself simply worked with video that's already available to security types.
"It's just a better way of integrating several video streams together so you get a better view of what's happening," he said. "The funny thing is, as we see more and more public sources out there that collect video data in different areas of the city, we can take that video that's already there and superimpose it on the models."
Seeing all that imagery knit together into a virtual world produces a powerful gestalt effect — and that's the kind of effect Shahabi and his colleagues are trying to achieve with the wider GeoDec project as well.
"When you integrate data from different sources, you suddenly can answer questions that you couldn't answer from these various sources individually," he said.
All the members of the GeoDec team are at USC and are affiliated with the Integrated Media Systems Center. The team consists of Cyrus Shahabi, associate professor of computer science, databases (lead); Craig Knoblock, research professor of computer science, artificial intelligence (also with USC's Information Sciences Institute); Ulrich Neumann, associate professor of computer science, graphics; Ramakant Nevatia, professor of computer science, computer vision; Suya You, research assistant professor of computer science, graphics; and Roger Zimmermann, research assistant professor of computer science, systems.