The answer is yes, someone has created a Game Of Thrones virtual reality. (How cool is that?)
If you missed it, well, it was probably because you weren’t at SXSW last year. Thrillists got to stand on the North Wall in a fully immersive 360-degree experience. Look up, look down, look left and right. You’re at Castle Black, standing at the edge of a 700-foot icy escarpment. Until a Wildling fires an arrow from below and you die.
But take off that headset and you’re good to go again.
It was just a taste, but reactions to the virtual experience were so positive that a second GOT event has been created as part of a multi-city touring experience.
“Expectations are high,” says Amy Small, Global Head of Virtual Reality at FramestoreVR, the virtual experience were so positive that a second GOT event has been created as part of a multi-city touring experience.
“Expectations are high,” says Amy Small, Global Head of Virtual Reality at FramestoreVR, the production company that created both experiences. “What I find fascinating is how people wanted horror first, but what we learned is not everyone wants that high anxiety-ridden experience.”
The new 2017 Game Of Thrones virtual reality event will move across the country. You’ll still feel like you’re there, but it will be different. Probably calmer.
Virtual Reality (VR) is the hottest technology happening on the planet right now. Global searches in VR have grown ten times in the last year alone. A Google report calls virtual reality the next frontier in advertising, marketing, and storytelling. And Deloitte predicts the birth of a new billion dollar experience industry emerging from hardware, game sales, mobile content and distribution. (Others claim a $150 billion industry.)
Pathways for new VR and augmented reality (AR) technologies are as omnidirectional as the technology itself, impacting films, gaming, online learning, retail, publishing, experiential marketing and more.
So far, reality is on a learning curve.
“We’re just starting to scratch the surface,” says Catherine Day at creative studio m ss ng p eces, which has produced VR for the NBA, Google, Oculus and others. “Last year wasn’t even the beginning. It was the beginning for insiders, but this year was the beginning for consumers—and we’re still testing, iterating.”
“The tools of even just storyboarding are still being created,” agrees Patrick Milling-Smith who is partner at president of Here Be Dragons, which he co-founded in 2014 with director Chris Milk. Among other things, Dragons has done breakthrough work with Beck, TOMS, Mr. Robot and The New York Times’s NYTVR series—you might be familiar with “The Displaced.”
So far, virtual reality comes in three flavors: two kinds of VR and then AR. Immersive 360-degree virtual reality explores the existential being of human beings. Three-hundred-sixty degree reality is plied via Oculus Rift, Samsung, Cardboard and a dozen other companies who want to immerse us front, back, top, down and sideways.
“It’s difficult to define the VR experience, because it’s a spectrum,” says Catherine Day of m ss ng p eces. “You can watch 360 videos on your laptop, then a headset; I don’t know if consumers know there’s a difference. There will be 3D content instead of just 2D spherical video and more and more photoreal VR as compressions for larger amounts of data are optimized.
“Right now,” says Day, “people are still figuring out what VR is.” And. “People in the industry are also getting precious about what they think VR is—it’s not VR unless it’s volumetric. Or it’s not VR unless it’s room-scale. Or it’s not VR unless it’s 4D, with wind blowing in your face, and you have agency so you can tilt your flying to the left and right.”
As users become more familiar, the experience and expectations will adjust. Eventually, we will move out of the world of screens, into a Charlie Brooker fantasm. Everyone will have their own screen and content will be projected into space through intelligent light.
Existing on a not-quite-parallel path is augmented reality (AR), a mixed bag that, in some ways is the most interesting because it adds to what we know to be real.
“Augmented” reality enables people to view physical space through a different medium (in most cases, that medium is a smartphone or tablet). It uses sensors to activate a digital layer of experience that “augments” the space using visuals such as 3D images, descriptive text, video and audio.
AR can include holograms, RFID scanners, and crowd technologies. And the list grows week over week. Anyone bingeing “Black Mirror” on Netflix has been gobsmacked by the consequences of flesh and bone reality meeting technological possibilities of what might be layered onto our real world.
“And then consumers are going to be able to choose,” says Catherine Day. “Do I want to be immersed, or do I want a mix of both? This is where it becomes interesting—the challenge is to navigate bravely.”
IBM and The New York Times Company’s launched their Outthink Hidden experience and T Brand Studio AR app this week, an augmented reality engagement inspired by the 20th Century Fox film, “Hidden Figures.” The film includes the true story of three female African American mathematicians as the heroes at NASA during the 1960s Space Race.
“This technology is moving fast—probably faster than the consumer’s ability to experience it,” says Sebastian Tomich, SVP of advertising and innovation at The New York Times and head of T Brand Studio, who created the app, a collaboration with IBM and agency Ogilvy & Mather.
“AR is a re-definition of the real world,” says Tomich, who predicts that eventually AR will dominate. “When the technology gets to a place where we don’t have to wear a box on your head—where reality is augmented all around you,” he explains.
“And that’s on the horizon.”
(In fact, at CES 2017 this week, people have been talking about virtual and augmented realities merging over the next 36 months.)
“I’m a really big fan of mixed reality and truly excited about the potential of that,” agrees Jon Kamen, co-founder of Radical Media which did ‘Flight’ for The New York Times VR (co-produced with Here Be Dragons). “There’s a lot to do with the excitement over that. There are amazing entertainment applications to it, and it’s being purposed for industrial applications too.”
What’s the opportunity for brands? “I always applaud companies trying to experiment with new things to see how they work,” say Kamen. “One of the projects we’re most proud of, is the Taylor Swift project. It was the extension of an American Express assignment. Amex and Taylor Swift came to us and challenged us to do something different—then she agreed to take it into a 360 [degree] app-based experience. She understood the global phenomenon of her success and that millions of her fans (without access to 360-viewers) wouldn’t be able to view it.”
By releasing the performer’s film from the Cardboard viewer experience, Swifties were able to share the experience with one another, interact with it, gamify it and have more fun with it.
“We also felt we had a responsibility to American Express that the project would appeal to a younger audience,” adds Kamen. “And what made the most sense was that it was content targeted toward mobile. And today’s exceptional smartphone visual quality makes it an amazing platform.”
Another prominent example of augmented reality was the 2016 summer craze, Pokémon Go. The game was downloaded 500 million times over two months and was Apple's most downloaded app in 2016. The interaction was genius. Simplistic. Download the app, and your smartphone mapped out where to get Pokémon points.
The epic interaction captured global headlines and showed how AR could create fun and interesting experiences for anyone carrying a smartphone.
Niantic CEO John Hanke, creator of Pokémon GO, cites AR as being “far more interesting and promising” than VR in a recent ReCode Decode interview with Kara Swisher, because of its ability to integrate into and enhance the real world.
“Being outside, socializing with other people, shopping, playing, having fun. AR can make all those things better,” says Hanke, who also launched Google Maps.
Case in point, the Pokémon GO craze had people wandering the streets, pointing their smartphones into the sky, rather than sitting at home on a couch.
But despite the millions who played Pokémon GO in July 2016 (some are still playing) the game feels more like a Stanford thought experiment than enterprise. As marketers picked up on the potential of intersecting with the craze, Pokémon GO stakeholders seemed unprepared for their own success. Opportunity missed, but it’s probably not over yet.
Actually, the Pokémon GO frenzy is a beautiful new metaphor for commercial retailing: Technology plus retail environment plus new experiences that attract excited, engaged crowds.
Capitalism and free markets send up bubbles of smiley emojis at the thought.
Today, virtual reality is in its infancy and still trying to form words.“Yaw” “Flatties,” “easter eggs,” “pass-thrus,”“spacial audio,” “eye tracking,” “head gaze detection,” “room scale,” “stitching,” “procedural generation”—new words, new technologies, everything is being invented.
When I mentioned that someone just launched an app for creating 360-degree storyboards, one VR producer asked me for the link. They burst out, “We need that!”
“The language of storytelling needs to be developed and there needs to be satisfying content,” agrees Patrick Milling-Smith.
“We’re still figuring out how to tell a story,” says Michael Cable, global technical director at Framestore’s London VR studio. “As more people find better ways to do it in the future, that’s going to change the game a bit.”
A constant reminder is the evolution of film that took place 100 years ago. It took twenty years to go from Eadweard Muybridge’s images of the walking man and running horse, to the invention of the close-up.
“We made a lot of mistakes and learned from those mistakes. You can see vast progress,” says Patrick Milling-Smith of Here Be Dragons. “After demoing 5000-plus people, the emotional response is undeniable. If you put people in the shoes of a refugee girl, people come out of it with tears in their eyes. There’s great joy from kids [as well as from] people in their 80s.”
Patrick Milling-Smith has been trained in theater. His production company SMUGGLER received multiple Cannes Lions Palme d’Or and Grand Prixs, Tony’s, Olivier’s, Emmy’s and a Grammy. Patrick recently produced the Broadway adaptation of the Oscar-winning musical “Once” which garnered 8 Tony Awards including Best Musical. He sits on the board of Punchdrunk, the word’s most immersive theatre company, known for the innovative Sleep No More.
The overwhelming desire is not only to create emotion, but to also satisfy core human desires. To transform the new medium from cheap tricks to truly immersive reality. There are not only elementary (and complex) technological hurdles, there are also emotional ping points.
“You want to get as close as you can to your subjects because you want that kind of intimacy,” explains Milling-Smith. “You want to move that camera. If it’s more documentary, it needs to have less of a presence. With the 360, you need to get oriented, you need to pace the story. You don’t want to bash people in the head. You need to establish that presence.”
Virtual reality disrupts the art of being. “The more you make the viewer feel immersed, the stronger the experience will be,” says Niklas Lindstrom, head of interactive production at advertising agency Droga5. “My strongest sense of immersion have been game engine-based experiences in mixed reality where I am able to use more of my senses—where I have been able to move around in the environment and interact using my hands and physical objects, or using hand controllers instead of passively viewing the content play out, predetermined in 360 degrees.”
The essence of 360VR is to create memories. “The biggest advantage of VR is that, when done right, it is a new, innovative and exciting platform for clients and brands to create powerful immersive experiences that create more lasting memories for the viewer,” says Niklas Lindstrom.
“The nature of the 360 tricks your brain,” says Milling-Smith. “You have location-based memory that reminds you that you were there. You feel like you were there.”
There is historical film precedent for this. Back in the day, both computer and cell animation were challenged with creating verisimilitude. Not just looking like the real world, but actually feeling like the real world.
Walt Disney trotted actual deer into the studios to help animators create more lifelike drawings. Eventually, Disney animators created a broom in Fantasia that acted like a human being. At Pixxar, fifty years later, John Lasseter created a computer-animated lamp whose actions projected human emotion.
360VR producers are trying to create similar evolutions. And they don’t want to wait 20 years before someone imagines their version of the close-up. What’s next after what’s next?
“It’s such a new industry. It’s a totally new medium,” says Catherine Day at m ss ng p eces. The danger is that people are already convinced they know what the medium is. Worse, some are setting expectations and rules. “We haven’t had enough time yet. We need to be fearless and ignore the echo chamber that the industry is creating,” says Day. We need to work from the perspective of how can we make this better.”
“VR is a fairly new medium, [so] the client should be open to the ‘test and learn’ aspects,” adds Niklas Lindstrom at Droga5.
One gift from the world of startup technology is the culture of iteration. Being agile and trying new things, continually making them better is, for now, a better model than the Hollywood mode of instant hit.
“The culture of iteration is what’s going to push us forward,” says Day. She recalls the piece her team produced for the NBA. Just when everyone thought they had figured out what VR could and should be, the m ss ng p eces team disrupted those conventions.
It’s a good example of people prematurely setting expectations and rules: analysts were seeing VR viewer drop-offs after two or three minutes. So it was assumed that VR shouldn’t be longer than three minutes.
But content is king. “Our 2016 NBA Finals piece featuring Lebron James was contracted to be fifteen minutes, which blew some people’s minds,” remembers Day. At first, Day explains, the NBA didn’t know if they wanted to give the film crew 360-degree access. Letting a film crew mingle elbow-to-elbow with players pre-game during the playoffs could be distracting.
“We wanted to give our artists the freedom to try that out,” says Day.
And stroke of luck: The experience featured the Cleveland Cavaliers against the Golden State Warriors, one of the biggest comebacks in NBA playoff history. The events surrounding the historic game were riveting on-screen and the Oculus-backed VR final edit came in at over 25-minutes long.
When you watch the NBA experience, you get the full picture of VR’s potential. Forget court-side seats or “behind the scenes”, you’re actually in the scene, living the moment. Viewers become participants immersed in intimate game-play and exploding court spectacle.
“That interaction was created intuitively by the production team,” says Day. “We had one of the best writers and the best directors, a very experimental technologist—and a guy who lives and breathes documentaries.”
The result was an immersive push into professional basketball that was unprecedented and, in fact, could be imagined for every sport.
“It wasn’t about AIDs or refugees. But I was choking up, I was crying,” recalls Catherine Day. A reveal: the power of virtual reality as game changer. “It convinced me that I needed to stay in the industry,” admits Day.
For many of us, virtual reality became real in June, 2014 with a front-cover story from WIRED magazine about a mysterious company in Orange County, California with the equally mysterious name Oculus Rift. The company had already been purchased by Facebook.
Mark Zuckerberg introduced Oculus Rift in a Facebook post as a builder of virtual reality technology enabled to build useful, entertaining and personal experiences.
“Oculus's mission is to enable you to experience the impossible,” declared Zuckerberg.
Big Z cited clear immediate potential in gaming, sports, education, and reminded Facebookers that what was once the domain of science fiction is now reality. “One day, we believe this kind of immersive, augmented reality will become a part of daily life for billions of people.”
Where Facebook treads, others follow. Suddenly, Snapchat pays between $30 million to $40 million for Israeli startup Cimagine Media, a company that enables consumers to get a virtual look at how appliances and furniture will look in their homes. Augmented-reality startup Magic Leap attracts $1.4 billion in funding for gear currently being prototyped—it’s like Microsoft's HoloLens, but not as far along. Advertisers take a liking to Google TrueView's immersive video platform, which offers 360-degree mobile video to give an “added layer of interactivity [that] enables consumers to engage.” Augmented reality app Blippar is working with Kraft and Coca-Cola to let smartphone users unlock content by pointing their devices at brand packaging. Two days ago, Google announced that it is moving into augmented reality shopping with Gap and BMW.
And we haven't even started talking about Google Tilt Brush.
There are conventions, workshops, panels, debates, and special sessions. A Nielsen study reveals that VR elicited 27% higher emotional engagement than in a 2D environment and 17% higher emotional engagement than a 360-degree video on a flat screen.
What everyone is considering is this: as god-like technologists create a reality that rides through optic fiber pipes thinner than a human hair, compressing excited exabytes pulsing within servers located in New Jersey or outside of Budapest, we ponder whether it is an illumination, a deeper understanding, or simply another shiny object.
The whole thing calls into question what is reality, really? If we can taste, smell, hear, touch and feel—if we believe that we can see and remember and exist and (most of all) belong, then reality is no longer fixed; it is elastic, pliable. We can story another existence to wrap around our brain stem: another departure point where we can eat, pray, love.
Imagine the future of a virtual reality where we might actually have a second life. This is where Bitcoin goes. This is where haptic engineering goes. This is where dreams go: A quadriplegic can hike Annapurna from their Omaha living room. You can stand onstage with The Rolling Stones, swim alongside humpback whales, shop in Shoreditch or stroll down the rue de Jacobs and try on that dress in your size at Chanel. You could buy an apartment in Shanghai with Bitcoin and your persona can reside in a virtual Airbnb, well, anywhere.
And just as sideways technologies sprang from putting man on the moon (superglass, Teflon, satellite television, smoke detectors, battery-powered power tools, and more), new technologies are likely to spring from chambering humans inside the 360VR bubble.
VR could be the ultimate reboot. Skip the messy realities of ‘real’ life and slip through the air to live your best possible self. A sanctuary. A vision—or revisioning? An opportunity to reset life’s opportunities and possibility.
There is a place in our brain that is hardwired to belong. To fit amidst a community of other people. But with VR we can be spared the pain of other people. We are a replicated self without heart or lungs, nevertheless a soul filled with desire, with emotions.
But not yet. Maybe later.
“We’re at this interesting crossroads where it’s pre-railed but you have the ability to fill things in,” says Amy Small back at Framestore. “It can still be somewhat gated, but there are things to achieve, unlock, they are able to digest in a slightly different way. It’s kind of Wild West right now and that gives us the opportunity to create something new and exciting!”
For now, VR/AR are plussed-up realities that seem plummy with possibility. In their report, Deloitte reiterates that VR’s capability is “likely to improve further still over the years as processors improve, screen resolution increases yet further, and content creators learn how to create for the format.”
As famed designer Milton Glaser reminded us in The New York Times last year, “Everything’s a transitional period. There’s no such thing as a non-transitional period.” We’re in it now. You can stand on the North Wall from Game Of Thrones. Or stand on the court with NBA champions. Fiction or reality, it makes no difference, we’re already debating ‘false news’ in the what-is-real? department. So this, too, is part of our bubbling social soup.
Grab a spoon.
Patrick Hanlon is CEO and founder of Thinktopia and author of The Social Code: Designing Community In The Digital Age.