Hollywood is doing its best to figure out virtual reality, often with less than compelling results.
But a small group of VR producers at this year’s Tribeca Film Festival have successfully innovated their way through experimental territory that big studios have not yet dared to tread, using physical objects and even live actors in their experiences.
Let's start with the very cool and end with the absolutely mind-blowing.
Blackout is an attempt to give all big city commuters the super powers they've always wanted: the power to read the minds of your fellow riders. Using gaze-based tracking, the experience allows you to walk through the subway car and, when you stop and stare at a particular person (a real person who has been captured volumetrically), that person's inner monologue begins playing in your ears, delivering a kind of telepathic eavesdropping effect.
However, instead of giving you a Vive headset and hoping you just "get it," the creators actually constructed a life-sized subway car, complete with seats and handrails that accurately correspond to the the VR experience.
It's incredibly weird to reach up to grab a handrail in VR and actually feel it. But that weirdness makes reading the minds of your fellow travelers that much more realistic, and visceral.
Treehugger lets you do exactly what its name suggests: wrap your arms around a massive, psychedelic tree that pulses with spectral energy in virtual reality. Using the Vive headset, a Subpac vest for haptic feedback, and a new kind of olfactory attachment that delivers the smell of a forest, you're transported into the secret world of a tree's life force.
But you don't just see, hear, and smell the experience, you're also given a giant foam-covered, tree-shaped construct that you can caress and even stick your head into to view the inner workings of the tree in VR. With one HTC Vive tracker puck attached to each hand, the strokes you make to touch the tree's surface have real impact in the VR environment.
The experience is less narrative than meditative in the service of respecting the under-appreciated aspects of forest life. But with four Vive stations situated around the base of the physical tree construct, every person who completed the experience seemed almost spiritually refreshed when they removed their headsets.
Of all the rule-breaking approaches to Shakespeare executed over the years by theaters and filmmakers alike, To Be With Hamlet may be, hands down, the most cutting-edge of them all. Three audience members and I were brought into a large room and fitted with Vive headsets and controllers, after which we were plunged into a VR version of a castle ledge, surrounded by a roiling seascape.
But the two computer generated characters standing before us, delivering the immortal bard's lines with emotion and impact aren't preprogrammed characters—instead they're controlled by two actors wearing motion capture suits many miles away in a downtown Brooklyn studio.
Each gesture, every dramatic flair executed by the actors was transmitted in real time to our group in lower Manhattan. As I watched the three other audience members (represented by small, translucent avatars with the ability to teleport all around the scene as it played out in VR), I felt like I was taking part in something from the future. But it's not sci-fi, it's real and happening in 2017.
Although, the experience that changed my thinking the most about the future of VR and theatrical works forever was Draw Me Close, Jordan Tannahill's short play remembering his time as a child with his mother following her terminal cancer diagnosis. First, I was brought into a small entry room and made to take off my shoes and strap on a Vive tracker on each hand and then the headset. One of the team asked, "Are you OK with light touching?" Uh, sure, no problem.
I had no idea what was coming next.
Once the VR experience starts, a narrator's voice leads you to a door that you can walk to and, surprise, open with your hands in real life (I'd assumed I was going to be standing in that entry room the entire time). Soon you meet your (Tannahill's) mother, and she approaches you in VR as a moving illustration.
Then she moves in to hug you and, oh wow, a real person in the room hugs you at the same time as the VR figure! It turns out that the mother you're looking at is being controlled by an actor wearing a set of Vive sensors who is in the room with you and guides you, through touch, holding, and gentle prompts, through the VR experience in concert with her virtual self.
It's a groundbreaking marriage of reality and VR, and I believe (and you should hope) that it will be copied by others. But it doesn't end at hugs. You mother helps you open a window in VR (that you also feel and open in the real room), she draws pictures with you on the floor, and at one point she even tucks you into a bed in VR that, yes, corresponds to a real bed and set of covers that you can feel. It's an emotional and technologically brilliant experience that takes the concept of immersion into new heights.
Will Hollywood, or even Broadway follow the lead of these incredibly imaginative hybrids of art and tech to make such experiences the norm?
That's up to the artists.
As with all shifts in the history of art, it will take the passion of the creators to pull mainstream audiences — perhaps at times kicking and screaming against these new tech-underpinnings — out into the deep end of what could eventually become the way we all receive the stories that change our lives.