The story of how John Carmack found Palmer Luckey is something more worthy of a book than an article, but it may be this meeting of the minds that changes consumer gaming hardware. Creating a good head-mounted display is a problem of both software and hardware, and suddenly one of the sharpest minds in video game software was working with a young visionary in the world of wearable displays. The result is the Oculus Rift, a wearable display that has a chance at bringing consumer-level virtual reality to the mainstream.
How this happened
Carmack had spent a large amount of both time and money trying to find, or create, a working, high-quality head-mounted display to bring virtual reality back to video games after the launch of id’s last game, Rage. He tried modifying existing hardware and explored the problems with optics and displays used for virtual reality before he finally began to run out of time; id Software needed him back working on video games.
It was around this time Carmack learned of Palmer Luckey through a forum for 3D enthusiasts, and the developer saw promise in Luckey’s designs. If the hardware could deliver everything being discussed online, it might just be the breakthrough Carmack himself had been trying to achieve.
“It was one of those times it’s good to be me,” Carmack said, describing how he contacted Luckey to ask for one of the headsets to try. Luckey sent a prototype, and Carmack was able to code support for Doom 3: BFG Edition in a day, even without the benefit of an SDK. [Update: Carmack contacted me to say most of the coding had been done for previous headsets, and the optical correction was the portion that had to be implemented for the Rift]
Luckey had no idea the unit was going to be shown at E3, and was barely aware that his hobby project had become a large part of the marketing campaign of Doom 3: BFG Edition, but soon Carmack was using the hardware to demo the game to the press. It was a great way to get the press interested in an upcoming id release, and Carmack finally had a wearable, high-quality display with which to experiment.
John Carmack gives a demo for the press. The final design will connect to your devices via a USB and HDMI connection
Carmack later told me he thinks he would have ultimately arrived at something close to Luckey’s design on his own, but Luckey had the right experience with displays and wearable computing, due to his work in VR and head-mounted displays at USC’s Mixed Reality lab, to make the needed breakthroughs. Once the hardware was in Carmack’s hands the developer had a strong, working base to begin his experiments using Doom 3 and, to a smaller extent, Rage.
Suddenly Luckey had a prototype that wasn’t just a fun project for him and his enthusiast friends online, but the beginning of a consumer product. Oculus the company was born soon after E3, and there was soon a Kickstarter to fund the creation of updated hardware to put the display in the hands of developers who could continue to refine the software and create games or write mods for existing games.
The initial goal was $250,000, and in around 48 hours they had broken the million dollar mark. The Kickstarter video features Gabe Newell talking up the Rift, as well as Cliff Bleszinski from Epic Games and David Helgason, the CEO of Unity, offering their support. Markus “Notch” Persson has tweeted that he’s looking forward to playing with the hardware and possibly adding support in Mojang’s current and upcoming games. When it comes to powerful friends, it doesn’t get more impressive; these are the faces behind the engines that power an extensive number of the games we play.
The consumer product may not be launching until 2013, but there already seems to be something close to critical mass in industry support.
My demo
John Carmack handed me a rough prototype for the Rift headset. It may have been held together with duct tape, but it felt snug when I adjusted the straps. The hardware was almost preternaturally light, and the field of view was broad. I moved my head to look around the world of Doom 3.
I looked up, and Carmack noted how few people had seen the skylight that filled my view. I didn’t feel like I was looking at a game world through a display, I felt like I was exploring a physical environment. “This world is pretty cool, there are a lot of neat things here,” Carmack said as I silently looked around the corridor of a game I had played half a dozen times. I noticed things I had never seen before. It’s a subtle effect, as you run down the hallway, your head moves in tiny ways and the view moves with it. It feels like you’re there. The 3D effect isn’t as much as an “effect” as it is when you’re looking at a 3D monitor or theater, it feels correct and subtle.
There was a tiny bot following me around in the game. I lost track of him before I remembered that I could move my head to look around. I peered back, and looked down over my shoulder. I saw the little bot behind my left foot, and he looked up at me. For that moment, we were looking into each other’s eyes. A slight shudder of pleasure ran through my body. This is the future.
Palmer Luckey
Palmer Luckey has over 40 headmounted displays. He told me it may be the largest collection of virtual reality hardware on the planet.
We met at a bar in the Hilton Anatole during QuakeCon, and Luckey ordered a pink lemonade. He spoke in a flood of details and technical information, and if you asked a simple question about displays you may end up listening to the history of a VR company whose leadership is now in jail for fraud. He can remember the model number of any number of portable displays and discuss the strengths and weaknesses of each one.
You don’t speak with him as much as you surf across the many interesting subjects he can discuss off the top of his head. Later in the evening, I took the Oculus crew up to my hotel room to show them my press sample of the 3DS XL, and Luckey split the time between raving about the expanded “sweet spot” in the 3DS XL hardware and discussing the world of enthusiast lasers.
I’m glad I interviewed Luckey away from John Carmack; separately they are fascinating to speak with, but together they must create a room where those of us without beautiful minds could drown trying to keep up.
“Galvanic vestibular stimulation, have you ever heard of it?” he asked me. I shook my head. He described hardware that uses electrical signals to affect the small hairs on the inside of your ear through electrodes placed around your head. Luckey can make people feel like they’re falling, or flying forward, by using the right impulses.
“Moving forward is one of the easier ones. That’s something I’ve played with a lot. That’s something that would be very cool in a headset,” he said. This is the sort of thing he plays with when he’s not innovating in the world of head-mounted displays. Gabe Newell told him he should add a heart-rate monitor. Luckey brings up virtual reality gloves, impact vests to simulate being hit by bullets… they have a whiteboard for future ideas, and it’s always filled.
Carmack told me that he often tells Luckey to focus on releasing this design of the developer kit, and the rest can be added later. Luckey admits that it’s hard to think of consumer products when he’s used to experimenting and adding features and new ideas to each of his projects in his personal lab. It struck me that this is how Ender Wiggin might have turned out if he designed toys instead of going to war.
The $300 developer kit isn’t for you
I was told, again and again, that most gamers shouldn’t bother with the $300 developer kit offered on the company’s Kickstarter page; this is a product aimed at developers, not consumers.
I asked John Carmack if he was worried about consumers buying the product via the Kickstarter. “Yes I am,” he said, cutting me off. “Because this is not intended to be a consumer product.” He stressed that he has no official ties to Oculus, and merely wrote the software that allowed the hardware to work with Doom 3: BFG Edition, and now cheers on Palmer Luckey. (“I’m not a software guy,” Luckey admitted later). “He’s a young guy, this is his first business, I hope that it works out okay. I’m a little concerned that he underpriced the stuff on here.” Carmack is worried about burps in manufacturing, and the company being underfunded.
“We are hoping consumers stay away from it, and we’re doing our best to market it not as a consumer product,” Nate Mitchell, the VP of product for Oculus VR said. The developer kit is exactly that, and the final design will feature a host of improvements. The Kickstarter isn’t there to raise money, not exactly, it’s there so the company can fund a large amount of developer kits and send them to developers who’d like to work with the hardware. The $300 price point for the dev kit leaves little to no profit for Oculus, but the resulting software will give them a powerful advantage when the consumer model is released. If even a tenth of the developers ordering the kits create working mods for the Rift or new games, there will be a flood of software support ready for the consumer launch.
While the headset is light and fits well, the player can’t wear glasses under the current design. If you move the display further from your head you lose field of view. There are things they can do with sharpness in the software, or they can create adjustable optics that remove the need for glasses, but those are problems that will be solved in the retail version. “Astigmatism I could correct for with a fragment program,” Carmack told me without skipping a beat. This is the world we live in; your vision problems can be solved in software.
“These are problems we have plans for in the consumer version, so we don’t want gamers to run out and waste their money when the consumer version is going to be so much better,” Mitchell said. “There’s no software for it right now, we’re working on that.” The developer kits are for people who want to have fun adding VR support for their games while learning how the hardware works. It’s for the geeks who want to make games and modifications for unfinished hardware.
The idea is to get the hardware into the hands of developers for as cheap a cost as possible, as early as possible. Even though the Kickstarter campaign has been successful, there will be no stretch goals that increase the quality of the first batch of hardware. The initial specs are a design they know they can deliver in the numbers needed for the initial run. The work done between the launch of the development kit and the consumer model will make sure the hardware launches with robust software and game support.
How we got here
Both Carmack and Luckey described a number of headsets that offered a lesser experience for more money. So what’s the secret sauce? Where is the jump between how companies used to do things with headsets that cost tens of thousands of dollars and having a $300 prototype that worked this well?
“Mobile hardware,” Luckey said. “That’s the jump.”
From our cell phones to digital cameras, the market for small displays has exploded. That means the technology has matured rapidly, with increasingly powerful displays sold at a decreasing cost. Consumers deal with dozens of types of hardware with high quality, small displays in our daily lives. “We have all of this display technology where people are racing to get to high pixel density in really cheap, lightweight displays,” Luckey said.
Sensory hardware such as accelerometers has also dropped in price; the components have gotten better and cheaper. It just took someone who had the enthusiasm for games and virtual reality to put them together in a way that made sense for those applications.
Detail on Carmack's hacked together prototype
The other magic trick is the fact that image distortion is handled in software, not the optics. Optical experts used to design complex, expensive lens solutions to allow the human eye to focus on a small, rectangular display close to the eye. Earlier head-mounted displays used a six-lens display with very little distortion, but that solution was expensive and you ended up with a product with a low field of view.
These days computing power is a commodity, and image distortion is a known variable. The software that drives games on the Rift actually distorts the image before it goes through the optics, so the inexpensive lenses correct the distortion before it hits your eye. The image is simply warped to the opposite of the low-cost lenses. The processing overhead is minimal, and it allows the Rift to use inexpensive optics in place of expensive lenses. It sounds obvious in retrospect, but computing power was not cheap ten years ago, the time of the last big push for virtual reality.
“The [optical solution] is one of the key elements to this endeavor,” Carmack said.
The display in the developer kit was released in 2008. “That being said, it was the best display at the time. It was designed for ultra-mobile PCs,” Luckey explained. Even now it’s a good display, and it’s hardware Oculus can buy at a low price, without ordering in bulk. It’s a solid, known quantity that’s perfect for a developer product. There will be better displays available in time for the consumer product, although the price of those displays is still up in the air.
While the final SDK won’t be based on John Carmack’s work with the Rift and Doom 3, the developer’s star power and E3 demos certainly gave the hardware a shot in the arm. It’s a fascinating moment in gaming hardware; the right two people met at the right moment in terms of both hardware quality and price. The two men’s enthusiasms and specialties happened to line up in an optimal way; in many ways it seems like Luckey poured gasoline all over the floor, and Carmack began to flick matches at the puddle.
The final product
“We’re not going to commit to a number today, because we’re still so far off from the consumer version,” Mitchell said when I asked about the price of the consumer hardware. What he will say is they’re not going to sacrifice quality; the consumer model may launch anywhere from $100 to $1,000. The developer kit is about finding a product that works and is fun to play with, that can be offered directly to developers for a very low cost. The consumer model will be about offering the best quality experience using the best components available at that time.
They’re also rejecting the idea of multiple models. “At the end of the day, the best experience is probably going to come from a single hardware spec. When people can develop for the Xbox, and they know what the specs are, and know that it’s going to play perfectly every time, that’s a powerful tool. It allows for best possible experience for the consumer,” Mitchell explained.
They also have some powerful help with the consumer Rift design. Jack McCauley created the peripherals for the Guitar Hero series of games, and was instrumental in the creation of the USB specification itself, as well as being an expert in microelectromechanical systems. McCauley is not just working on this as a side-project, he’s a full time member of the Oculus staff.
The developer kit offers a resolution of 640 by 800 for each eye, but both Carmack and the men behind Oculus shrug off the issues of the relatively low resolution display. “The consumer version, through a combination of higher resolution, tighter fill, and maybe some fancy optical tricks… in the consumer version resolution issue will not be a problem people are bitching about,” Luckey told me.
Carmack complained of “resolution bigots” who aren’t taking into account the experience over the resolution, and in fact the way the optics funnel the image leads to smaller, bunched pixels in to the middle of your eye while pushing larger pixels to the peripheral, but he also said better displays will be available before the end of the year. This introduces another problem, however, as anything less than 60 frames per second destroys the immersion from the head tracking, and it takes a significant amount of computing power to deliver 3D graphics at that frame rate while playing modern games. If you want a higher resolution headset, expect high system requirements.
There are other tricks they have up their sleeve for the consumer version they’re not ready to talk about, but it’s clear that the final version is going to be a much higher quality experience than the $300 developer kits being offered on the Kickstarter right now.
“We already have the best tracking you can get in a headmount, but we can make it even better by optimizing the components specifically for virtual reality,” Luckey said. Carmack likewise noted that latency can be improved before launch, even though the developer hardware already offers a smooth experience. There is the possibility of a wireless adapter for the consumer hardware as well, for an added price.
The games
It’s important to get games that people know and love. Something they may already have on their Steam account. Luckey gave an example of the perfect game: Fallout 3 with new content and Rift support. That’s the sort of thing that will get people’s attention.
What about Skyrim support? “There’s already a community driver. It’s not a lot of work,” Luckey said with a wave of his hand.
In an amusing twist, Carmack said that Bethesda’s Todd Howard is surprisingly “3D-phobic,” and would only look into the Rift for a moment before putting it back down. I relayed this story to Luckey, who told me they could simply turn off the 3D effect if Howard wanted to explore the world of Skyrim with the head tracking. They’ve also experimented with playing Mirror’s Edge with the Rift, with only so-so success. A tiny portion of Rage has been modified to work on the headset, and I was told that simply flying around the level with clipping turned off feels amazing.
“I think what you’ll see, soon enough, is some exciting official support,” Mitchell said. “We can’t announce anything right now, to be honest we don’t have anything firmly written down, anywhere, but we’re in an awesome position. We’re having a lot of fun.”
“We’re dreaming big, but delivering realistically,” Luckey said. In 2013, when the consumer edition is released, there is a small but very real chance they’re going to change the world.
More from The Penny Arcade Report
- Tim Willits talks Doom 4, John Carmack, and what id learned from Skyrim’s launch
- Salvaging Baldur’s Gate: the “horror” of getting the original code in shape for the Enhanced Edition
- Between the STACs: how MechWarrior Tactics will monetize free-to-play Mech combat
- The rise and fall of the video game capital of the world, and its dreams of rebirth