Eye tracking and full-room motion sensing: Qualcomm's dev kit reference design for standalone VR could be a peek at where 2019 heads.
I'm standing in front of a mirror, looking at an avatar of a woman that isn't me. "Blink," I'm told. But it's not working for me. One eye is permanently shut.
Maybe it's my glasses? I try lifting the goggles away from my face a bit. Now, it tracks. I roll my eyes around, and my virtual eyes in the mirror roll. I blink, they blink.
Qualcomm makes the Snapdragon processors in most smartphones, but it's also working hard to make the processors in most future VR and AR headsets. The company's previous reference designs for standalone VR headsets were followed by the HTC Vive Focus (a China-only headset), and the upcoming standalone Lenovo Mirage Solo with Daydream and Oculus Go, all with Qualcomm processors (Snapdragon 835 for Vive Focus and Mirage Solo, and the older Snapdragon 820 for Oculus Go).
This year, Qualcomm's newly announced VR dev kit using the latest Snapdragon 845 processor, announced at the Game Developers Conference, is all about eye tracking, full-room motion tracking and adding better graphics and battery life. It's the eye tracking part that's the most interesting... and possibly concerning. And as I wear Qualcomm's VR dev kit in a hotel corridor on a rainy day in San Francisco, I'm already thinking about the possibilities.
A ring of IR LEDs and VGA cameras come standard in the VR headset. Qualcomm envisions eye tracking to be an always-on feature, used in apps for a variety of reasons. The biggest ones are social connection to improve direct eye contact, added control without extra head motion, and better graphics. The better graphics could happen by a technology called foveated rendering, which will aim to push higher end graphics to whatever the eye is directly looking at. Qualcomm promises 30% better graphics performance and 30% better battery life with its new VR headset design, which ships to developers by the third quarter of this year.
The freaky part of eye tracking, however, is that any headset using it can record a heat map of your eye motion, tracking what you look at and learning from it. That type of analytics is part of what will make eye tracking valuable for any company that adopts it. Facebook and Google have both acquired eye tracking companies in the last couple of years, and Qualcomm's dev kit is made to accommodate the possibilities of that increasingly likely VR future.
My VR demo didn't show all that much: I got to see eye tracking in action, but full-room motion tracking wasn't demonstrated. Neither was foveated rendering via eye tracking. A graphics demo from Qualcomm called "Space Dock" had me shoot alien insects in a Starcraft hangar, but I didn't get to do more than lean forward and take a few steps. Qualcomm promises that its standalone Snapdragon 845-based headsets can be used to free-walk in VR up to a 20 foot by 20 foot range.
The headset comes with a little remote control wand that feels just like the one that comes with Samsung Gear VR and Google Daydream View (using 3 DoF tracking). But Qualcomm promises that eventually these headsets could support improved 6 DoF controllers with better motion tracking, more like those that come with PC-connected VR like Oculus Rift, HTC Vive and Microsoft's Windows 10 headsets.
Google, Facebook and HTC are all pursuing a VR future of untethered, mobile, phone-free devices. Right now, they're using Qualcomm Snapdragon processors. And there's a strong chance that their 2019 evolution could end up looking a bit like what I'm wearing right now. Or, for blends of AR and VR in mixed reality, or as Qualcomm calls it, "XR."
The Qualcomm VR dev kit also supports HTC's Vive Wave SDK, and has specs that might indicate where standalone headsets will aim for next:
- Snapdragon 845 processor
- 2,560x1,440 60Hz display
- 4GB DRAM, 64GB storage
- Bluetooth, Wi-Fi, USB-C
- Two monochrome 1,200x800 cameras and fisheye lens for room tracking
- Two monochrome VGA cameras for eye tracking