While everyone seems to have a touchscreen in their pocket, the next big thing is having a touchscreen pretty much wherever you want, whenever you want it. This prototype "smart environment" lets you turn any surface into a display or control.
Want email controls on your desk, or volume on the armrest of your chair? No problem. This combination of Kinect depth-sensing camera and wide-angle projector can track a user's hand, and drop a virtual button or other control wherever that user feels like putting it. It takes a bit of doing, but eventually, buttons can appear (and disappear) with a gesture.
This so-called WorldKit system was developed by Robert Xiao, a graduate student at Carnegie Mellon University, in collaboration with professor Scott Hudson, and the work of another grad student, Chris Harrison. (Sharp-eyed readers may recognize the idea from a much earlier version of the system that Harrison was working on, back in 2010.)
What's new is the use of the Kinect/projector combo to track an interface that's not just on your hand or the surface right in front of you, but in larger more challenging spaces: across an entire wall or covering an area of the floor. And it can do more than track your hand — it can count or track items like coins, or provide real-world measurements like size and distance.
Of course, not everyone wants a big projector and Kinect standing behind them, blasting a set of buttons over their shoulder. Xiao thinks he has a solution.
Depth sensors are getting better and projectors just keep getting smaller. We envision an interactive ‘light bulb' — a miniaturized device that could be screwed into an ordinary light fixture and pointed or moved to wherever an interface is needed.
That would certainly make it easier — a next-generation Kinect mini and pico-projector plugged into a power socket that's conveniently already there. But don't expect this prototype design to hit the mainstream any time soon. As today's smartphones and movement-based games show, virtual interfaces like this still have a long way to go.
Devin Coldewey is a contributing writer for NBC News Digital. His personal website is coldewey.cc.