Koji Gardiner is VP of Hardware Engineering at Jaunt. In this three-part series, Koji speaks to the design process that led to the development of the Jaunt ONE cinematic VR camera. Jaunt ONE is available for rent through Radiant Imagesin Los Angeles and AbelCine in New York City.
With this year’s releases of dedicated VR headsets from Oculus, HTC, and Sony, as well as the influx of smartphone-based viewers like Samsung’s GearVR and Google’s Daydream, the VR ecosystem is exploding. In this current state of heightened VR awareness it can be easy to lose sight of the fact that a mere three years ago, the Oculus Rift was still a Kickstarter-funded project, with its only product a rough-around-the-edges development kit.
Back in 2013, the core focus of Oculus and VR enthusiasts was gaming and computer-generated environments. It was at this time, however, that Jaunt’s founders began brainstorming a different experience in VR, asking themselves, “What if you could capture scenes in the real world and experience them again in a VR headset?”
The thought sparked the efforts that we at Jaunt have been focused on over the past three years: developing technology that enables creatives to capture stereoscopic 360° content for VR.
Capturing high quality VR content requires complex hardware as well as software. My role at Jaunt is to lead the development of professional quality VR capture systems. Jaunt ONE is the culmination of three years of design and development, and I’d like to talk a bit about how we got to where we are today.
Mid 2013: GoProBot
Three years ago, Jaunt’s founders developed their first VR camera: a small camera rig composed of a single GoPro and a simple Lego Mindstorms robot. The robot was a platform with 4 wheels, that could be programmed to move about a circle at a precise speed. The GoPro was mounted on the robot and a 360° video of a static scene was captured. By extracting video frames captured at different points along the arc of rotation, a left and right eye view could be created, thereby enabling a stereo 360° panorama of the scene.
Prototype Zero, the GoProBot
Below, you can see a still panorama of Jaunt CTO Arthur van Hoff, captured with the GoProBot. The top half of the image is the left-eye view and the bottom half is the right-eye view. When viewed together in a VR headset, this provides the illusion of stereoscopic depth.
Though the stereoscopic nature of the captured image was powerfully immersive, the prototype had the major limitation of only being able to capture a static scene. The next step was to develop a way to capture multiple viewpoints within the scene simultaneously and enable video capture in VR.
Enter the Hellraiser camera.
Late 2013: Hellraiser
Using off-the-shelf cameras from Point Grey and a 3D-printed housing design, we created a “ball” with 32 lenses pointing outward in all directions. Because of its resemblance to a certain horror character, I dubbed this design the Hellraiser camera.
The cameras were all tethered to an ethernet switch and PC, which would decode the raw video streams from the cameras and perform a rudimentary stitch. Plug in an Oculus headset and you had 3D 360° video, albeit with very low frame rate and limited fidelity.
Hellraiser prototype with PC cabinet
Video panorama of the Jaunt founders in their original office space, captured with the Hellraiser prototype
We learned a number of lessons from building this rig:
- Assembling a 3D 360° camera system out of multiple individual cameras has merit. While today this feels like an obvious discovery, the feeling of presence and immersion when viewing captured footage in a VR headset was revelatory at the time.
- Stitching multiple video views together seamlessly is incredibly challenging, with every shot captured by the 32 individual cameras needing to match the color, geometry, and detail of its neighbors. Achieving artifact-free stitched footage in a reasonable amount of time would take a lot of work.
- Stuffing that many cameras in that small of a space generates a great deal of heat — the nemesis of modern-day electronics. Overheating of cameras causes all sorts of problems with reliability as well as optical performance of the lenses.
Most importantly, we also quickly realized that the system was in no way practical. The ball of cameras was tethered to a cabinet with the ethernet switch and PC, and couldn’t easily be transported to shoot scenes more interesting than the inside of the office.
In order to achieve our vision for capturing compelling content, we needed a more practical solution.
Follow us on Medium or sign up for our email newsletter to get an update when the next chapter of “How We Built the Jaunt ONE” is published next week.