Modern virtual reality has been hailed as the future of Hollywood entertainment, our science fiction fantasies come to life. But there’s an ongoing problem: most VR experiences just aren’t that interesting. Hampered by evolving hardware and a medium with no set rules or audience expectations, most virtual reality experiences come off as either glorified tech demos, or simulacra of other, more established types of content. They’re usually riffing on stories that would be better told as short films or traditional games.
It’s partially a matter of storytelling conventions. Cinema has had more than a century to develop its own language of shots, cuts, and transitions, while storytelling in VR is still in its infancy. Creators are still figuring out what the medium can even do, let alone how to best take advantage. But virtual reality is only one small sliver in the much larger continuum of immersive entertainment. Real-world entertainment experiences have been evolving in their own right, developing their own unique approaches to storytelling. In the process, they aren’t just engaging audiences — they’re showing the way forward for virtual reality.
“YOU’RE DESIGNING FOR A GENERATION OF PEOPLE THAT HAVE GROWN UP ON THE INTERNET.”
“I think there is some [kind of] crossover between these mediums,” Kamal Sinclair, director of the Sundance Film Festival’s New Frontier Lab, tells me over the phone. “Between gaming and interactive, immersive theater and escape rooms — all of it. You basically are designing for a generation of people that have grown up on the internet, which is a very different audience than those that came from linear and passive media. You're designing for a generation that has a complete expectation of agency as a native and organic way of interacting with content and story.”
That sounds like a blur of buzzwords, but after decades of film and television, the ways in which audiences interact with the stories they love has fundamentally changed. Technology has enabled new creative tools and forms of mass distribution, altering the landscape so fundamentally that it’s hard to remember when it wasn’t common to interact with every beloved property via memes, cosplay, video remixes, and other forms of fandom. Interactivity and immersion aren’t just add-ons; they have become essential elements of the entertainment experience.
Escape the Rubicon escape room. Photo: Puzzle Break
THE RISE OF ESCAPE ROOMS
Sinclair dove into the topic further at the 2017 Sundance Film Festival, moderating a panel that brought together experts from various immersive fields to discuss how storytelling has evolved. Their conversation made it clear that nothing demonstrates the public’s rabid interest in real-world interactivity like the explosive growth of escape rooms. Nate Martin, co-founder of the Seattle-based company Puzzle Break, pointed out that escape rooms jumped from five installations in 2013 to a mind-boggling 1,500 just four years later. (How nuts has it gotten? Last month, we checked out a touring Legend of Zelda escape room.) But escape rooms are more than just random collections of puzzles. They also require the connective tissue of narrative, and are the most straightforward example of how experiential storytelling differs from what we see in books, TV, and movies.
“WE CREATE AN ENVIRONMENT WHERE THE PLAYERS ALMOST CREATE THE EXPERIENCE THEMSELVES.”
Traditional stories, Martin explained, revolve around a set beginning, middle, and end. A better way to think about it in the context of escape rooms is to approach the written story solely as a premise — in one of Puzzle Break’s rooms, for example, players are told they’ve been locked in an underwater laboratory, with only an hour to get out — but the actual story that unfolds for players is the tale of their own escape attempt, failed or otherwise. “‘Hey, remember when we found that thing, and then you put your hand on that, and then that trap door opened, and then Steve peed his pants?’” Martin asked. "That's the story we tell in escape rooms. We create an environment where the players almost create the experience themselves.”
“It's very uncomfortable for a lot of traditional filmmakers to think in these terms,” he said during the panel. “Of moments between players that have nothing to do with the amazing story, great narrative, and awesome characters they wrote.” It’s easy to see that tension playing out in VR, almost in real time. Justin Lin’s Help was released with much fanfare, only to quickly vanish from memory because it was simply a fixed story where the audience could look around. Oculus Story Studio won an Emmy for its VR short Henry, a story about a hedgehog who has a hard time making friends. That piece is cute and narratively efficient. It delivers on the promise of a empathetic lead character in Henry himself. But it’s also utterly non-experiential. Viewers are essentially inside a 360-degree Pixar short. Story Studio producer Edward Saatchi recently acknowledged that Henry would have likely been funnier if it had just been told as a traditional short film in the first place. And many VR stories follow the same lines — they’re just traditional productions planted in a new environment, never proving why they need to exist in VR in the first place.
Photo: Wesley Allsbrook / Oculus Story Studio
EXPERIENCE AS STORY
The concept Martin espouses — experiential storytelling, co-authored by the audience — is slowly trickling into VR, and it’s resulting in some of the more interesting new works. Story Studio’s follow-up, Dear Angelica, begins with the thinnest premise: a young girl is thinking about her mother, whom she recently lost to cancer. It doesn’t concern itself with traditional plot beats — instead, it focuses on slowly pulling viewers through its emotional moods.
The resulting experience is as much about what viewers bring with them as it is about what the headset shows. Another outstanding example, Chris Milk’s The Life of Us, which debuted at this year’s Sundance, is a co-op experience where players take on the role of an amoeba as it transforms into various life forms throughout the history of evolution. But players share their journey with another player in real time, creating a crude but effective collaboration. The Life of Us isn’t about the different stages of evolution; it’s the story of how the two players communicate and go on their trip together.
It’s a narrative dance, one where fewer traditional story beats result in more immersive experiences, and it gets to the very heart of what storytelling in the medium of virtual reality can become. “What are the building blocks of telling stories?” Dear Angelica director Saschka Unseld asked when I spoke with him last year. He noted that writers have words, and illustrators have images, “but in VR, I don't really think it's images. I think it's more the thoughts that are in the audience's head. It's states of being.” In other words, it’s about experience and interaction, not plot.
IT’S ABOUT THE AUDIENCE’S EXPERIENCE, NOT PLOT
Given that, it’s natural for VR creators to look to other experiential art forms for inspiration, and over the past year, that process has accelerated. Skybound Entertainment, the studio owned by The Walking Dead creator Robert Kirkman, recently decided to move forward with a VR adaptation of the immersive “haunted play” Delusion, and Oculus has begun bringing together experts across multiple disciplines as part of its DevLab initiative. One of the advisors at last year’s DevLab, which is aimed at honing and incubating projects from indie VR developers, was Third Rail Projects’ co-artistic director Zach Morris, who sees a fundamental link between VR and his own work in immersive theater.
Then She Fell. Photo: Third Rail Projects
PLAY’S THE THING
“We spend much of our time thinking about crafting the audience's experience, and really curating that experience,” Morris tells me over the phone. Third Rail specializes in site-specific immersive theater pieces, where audience members often become characters in their own right. The company’s most famous production, a reinterpretation of the Alice in Wonderland mythos called Then She Fell, has been running in Brooklyn since 2012, and along with Sleep No More, it’s considered one of the genre’s foundational works. Given that audience members have a certain amount of freedom, he says, shows need to make audiences feel like they’re living inside a story, rather than being told one. “We try to really make sure that every moment is designed in such a way that it honors their choices, that it feels really intuitive, and that the story is able to be unlocked by the audience. So they can forget about the mechanics of the form and really fall into it, and really author their own experience.”
When it comes to blocking, or pulling audience focus toward an event, some of the group’s design principles can be mapped directly onto VR projects. But there’s something fundamental about immersive theater shows that Morris says can’t be so easily applied: the simple power of human interaction. “Millennia of hard-wiring in our bodies tell us we want to connect with another human being,” he explains. “And I think no matter how good the rendering is, no matter how good the controls are, no matter how much we're able to fool our nervous systems, there is something fundamentally different about sharing the same air and the same breath as another human being.”
“NO MATTER HOW GOOD THE RENDERING IS, THERE IS SOMETHING FUNDAMENTALLY DIFFERENT ABOUT SHARING THE SAME AIR AS ANOTHER HUMAN BEING.”
It’s hard to deny that the lack of human-driven interaction is a stumbling block. So far, many “cinematic” VR experiences have put viewers in the passive position of watching a staged play, and pieces that are more interactive don’t bother trying nuanced, real-time interaction, due to technological limitations as much as anything else. It renders VR incapable — at the moment — of delivering on its most fantastical, Holodeck-esque promise. But many of those Westworld-style fantasies are possible when playing a one-on-one scene with an actor in Then She Fell, or what I experienced when I sat down with a cult leader in last year’s The Tension Experience. “Weirdly, for the first time in history, the performing arts have a corner on a market,” Morris laughs.
When I tried the demo for the VR project Gnomes & Goblins, from Iron Man and Jungle Book director Jon Favreau, it did a good job of approximating the basics of human-to-human eye contact. (Perhaps that should read “human-to-goblin.”) But while that underscored how important such basics are to creating the feeling of true, believable interaction, it also revealed just how far away we truly are. It’s one thing to interact with a virtual character in a gaming context when cues like eye contact aren’t essential, and interactions can be made with a mouse or keyboard click. But dump a viewer into a one-on-one VR scene with a character they’re supposed to interact with organically, and the uncanny valley yawns open.
Star Wars Land concept drawing. Photo: Lucasfilm / Disney
THE ROAD AHEAD
That’s not going to stop experimentation, however — particularly when creators seem to be zeroing in on interactive, experiential moments as one of the key building blocks of VR storytelling. One of Chris Milk’s next projects is a piece set in the Planet of the Apes universe that will lean heavily on AI to drive interactive character performances, while building further upon the socially connected foundations explored in The Life of Us. Morris says Third Rail Projects is interested in creating an original piece for VR, one that would be tailored specifically to the strengths of the medium, and director Jon Braver is shooting his VR version of Delusion later this year. And that’s saying nothing about the more traditional projects and VR series that are continuing to be produced — or the abstract, artistic pieces that could surprise us with an unexpected breakthrough or innovation.
VR storytelling organizing itself around the principle of experience rather than plot is just one more piece in what is becoming an ever-expanding puzzle of immersive entertainment. There’s also alternative reality gaming, theme parks like the upcoming Star Wars Land, and even interactive books like Doug Dorst and J.J. Abrams’ S. And talking to the creators themselves, it’s clear that audience interest is only growing.
“WHAT THINGS CAN ONLY BE DONE IN VIRTUAL REALITY?”
“The people I have coming to my shows, they're not the same audience we had 10 years ago,” Morris says. “It's a very different audience seeking out this type of work. And they're looking for engagement and experience in virtual reality, and in games, and in immersive or experiential performance — but also in other types of multi-form narrative.”
But in order for each of the forms to distinguish themselves — and succeed long-term — he says it’s going to be a matter of each medium honing in on its unique strengths. “I think the challenge — and really exciting thing — for makers right now is figuring out what things can only be done in immersive theater, or only in virtual reality. It's about figuring out the things virtual reality can do that no other medium heretofore can do.”