Video games have long promised to offer their players "immersive experiences," but full-blown virtual reality — the holodeck or the matrix — was still a fantasy. Now with the Oculus Rift headset on the horizon, that's no longer the case: soon gamers will be able to slip on a pair of high-tech goggles and truly be immersed in a virtual world.
The only problem is: How will gamers play when they get there? With the Oculus Rift still in development for a full consumer launch, developers are finally realizing the deficiency of contemporary controls. In this brave new world where "The Matrix" suddenly doesn't seem like too remote a possibility, the standard joysticks and gamepads that gamers have been using for decades are defunct.
That's the problem that the Los Gatos, Calif.-based company Sixense is now trying to solve. Previously known for developing the motion-tracking technology inside the Razer Hydra PC gaming controller, Sixense caught the attention of many gamers last month when it unveiled a Kickstarter project for a new motion-tracking controller setup it's calling the "STEM System" that blew past its $250,000 funding goal in less than a day.
The Sixsense folks visited New York recently, and I managed to get some hands-on time with their prototype for the STEM System. It's still in its early stages, but basically STEM works like this: there are two separate wand-like controllers, each with a few buttons and an analog stick attached. Another small box sits on the Oculus Rift headset itself to track your head movements. All these moving parts are clunky at first, to the point where Sixense CEO Amir Rubin assured me that these are still prototypes, not final designs for consumer products.
Once I slipped on the Oculus Rift and picked up the two sticks, however, the virtual reality system worked so well that it didn't really feel like "hands-on time" at all. I found myself in a small cottage by the sea, where I could walk around and pick up various objects — not exactly with my own hands, but with the two limbs that appeared once I successfully calibrated the controllers.
I know what you're thinking — why press a button at all when you can let a Kinect camera track you instead? Well, the current Kinect can only track rudimentary motions at best. The next-generation Kinect that's coming out with the Xbox One is far more precise, but Sixense's Rubin claimed that there's no substitute for the precision you get when a controller is actually in contact with your body. Whether or not that's true, holding (or wearing) a controller does have one crucial benefit over something like the Kinect when it comes to virtual reality: much in the same way that you are no longer bound by the TV or computer screen when wearing a headset, you don't have to be positioned directly in front of the camera for the device to accurately track your movements.
Most impressive was the motion-tracking that STEM brings to the Oculus Rift. First-person PC and console games can already put players in many lush and beautiful environments, but with only a mouse and keyboard or gamepad to account for user input, they can never reflect the nuances of subtle head motions that make truly being in a virtual space feel authentic. The graphics in the two demos that Sixense showed off paled in comparison to a modern console game like "Grand Theft Auto V" or "Call of Duty: Ghosts," but the combination of the Rift headset and STEM System gave the demos an unprecedented [??] precision. At one point, I discovered I could actually crouch down in the virtual house and turn my head upwards to look up the chimney into the sky above.
I left the office dazed and astounded by these visuals. The controls, however, were another matter. When I was so close to being able to reach down to the ground and pick up a ball with my own hand in virtual reality, having to press a button to do so was abstract and arbitrary.
And that's not even to mention the weirdness of having to use an analog stick to move. As Jan Goetgeluk, the CEO of Virtuix, a company currently making a "virtual reality treadmill," put it: "In virtual reality, you don't want to sit down and push buttons on a mouse."
"We all bring a very important piece to the puzzle," Goetgeluk said of Virtuix, Sixense, and Oculus VR. "In virtual reality, there are three elements: the glasses, the motion tracking, and the locomotion device."
And what's the magic combination of the three that will really bring virtual reality to life? Palmer Luckey, the creator of the Oculus Rift, told NBC News in an emailed statement that "there's no silver bullet VR input solution that we know of yet." Different players feel "more grounded" with one device vs. another, and "none of the existing solutions deliver truly low-latency precise motion controlled input."
"There’s still a long road ahead before discovering the ideal input scheme for virtual reality," Luckey added. "But we’ll get there eventually."
Yannick LeJacq is a contributing writer for NBC News who has also covered technology and games for Kill Screen, The Wall Street Journal and The Atlantic. You can follow him on Twitter at @YannickLeJacq and reach him by email at: Yannick.LeJacq@nbcuni.com.