SO HERE’S A STRANGE thing about making VR videos: turns out it’s really hard to show a rough cut. Even once you’ve done the complicated 360-degree shooting, and your computational algorithms have stitched all the footage together into something realistic and immersive, you still need to fine-tune the edits, sound effects, and visuals so you don’t disorient your viewers (or worse). Looking at a two-dimensional version on a laptop doesn’t really do it justice, and if you’re dealing with people in remote locations the chances are basically zero that everyone will have their own Oculus Rift to weigh in on the footage.
Considering all that, Stuart Cripps’ jitters are understandable. The heavyset, soft-spoken British director is working on a VR project for a 20-year-old Irish singer-producer named Jonathan Ng, known to his fans as Eden. The song, “Drugs,” is the second single from Ng’s new EP. Cripps has been working on the video for months, but nobody outside his studio has seen anything. Now, on a blisteringly hot day in early June, two weeks before the video’s scheduled release, he has something to show. It’s just not finished yet.
A dozen people, all involved with the project in some way, file into a post-production office near Universal Studios in L.A. While they crowd onto couches and chairs and try to loop Ng in from his home in Dublin, Cripps walks them through his vision; after a few minutes of high-mindedness—“I imagine us going from dust to dust,” he says, all but staring wistfully out the window—he leads everyone downstairs and gestures towards a swivel chair and an Oculus Rift dev kit. As with so much about VR, the only way to really get the video is to try it.
Outside of games, music is almost certainly the most popular content type in VR right now, which makes sense both technically—right now, VR’s best for quick viewing periods, about the length of your average song—and creatively. Both formats trade in experiences, connection, and immersion. You don’t go to a concert for the sound quality; you go to be part of something. “Today’s adolescents,” music writer/analyst/crank Bob Lefsetz wrote earlier this year, “are enthralled by YouTube stars and others who evidence a personality, there’s a human bond between them and their heroes and there’s this feeling that the famous and the hoi polloi are in it together.” That feeling also happens to be the kind of thing fans will pay for, which is hard to come by in today’s music industry. Meanwhile,one industry forecast pegs VR and AR as a $120 billion business by 2020, and everybody wants a slice. The tech and the content seem like a perfect match. But figuring out how to make it all work? That’s another thing entirely.
Music in the round
Musicians have been experimenting with VR longer than almost anyone. Most of the early projects go one of two ways: Either someone sets up a camera on stage at a concert, so you get to watch the show from the perspective of a weird band member who doesn’t play anything and just kind of stands there, or the band plays while standing in a circle around a camera, as if the show’s just for you. Whether either even counts as virtual reality, or is just “360 video,” depends on who you ask. But everybody’s doing it: Coldplay, Avicii, U2, Bjork, Paul McCartney, Taylor Swift, thecast of Broadway’s Lion King. Run The Jewels’ “Crown” is a particularly cool, in-your-face example.
A few others go a step further, using the immersive power of VR to make music videos more interesting. Ray Lamontagne’s “Hey, No Pressure” video drops you into a kaleidoscopic acid trip of a dance floor, where dancers gyrate and lights flash to the beat of the song. Earlier this year, Dawn Richard dropped a bonkers VR space jam, full of holograms and intergalactic surroundings.
Regardless of genre or feel, all those projects have one thing in common: they don’t sing you a song so much as drop you into the middle of one. “In traditional music videos,” says Chris Milk, a celebrated music video director turned founder of VR company Within, “the viewer is outside of the experience, looking inward. In VR, people are right in the middle of it.” Narrative, story-first music videos—like Michael Jackson’s 13-minute “Thriller” epic, to name the one that literally everyone mentions whenever they bring this up—just don’t work in VR. Not yet, anyway. What VR does best right now is enhance the sort of raw, Rorschach emotion that makes music powerful in the first place.
The best example of VR’s potential so far is probably “Old Friend,” a much-loved piece by animator Tyler Hurd that’s set to a Future Islands song of the same name. You put on your headset, and an outrageous dance party starts all around you. Using the HTC Vive’s controllers and positional tracking, your avatar dances however you dance. Hurd was making the video for less powerful systems like the Gear VR, but he found the Vive’s full-body experience irresistible. “As soon as you start leaning around and looking at things,” he says, “they feel more real. You stop wondering and you just let yourself be there.” He calls his video “an overwhelming barrage of nonsensical joy.” It’s not a game or a story; it’s an experience, a place.
Cripps and the “Drugs” crew wanted to create a similarly visceral effect—and the song, mellow and moody, was a perfect match. “Everyone kind of immediately felt like ‘Drugs’ was the best fit,” Ng says. Cripps and his crew shot with a strange-looking rig: a Canon 5D on top of a Kinect, which together capture real-time three-dimensional data using beta software created by a New York company called DepthKit. The resulting aesthetic is pure wireframed data, like a glitchy Tron landscape crossed with Star Trek’s Holodeck. At one point the picture floats over a mesh surface, panning down and out until you realize it’s an outline of a person playing the keyboard.
Ordinarily, with a music video, the song is the easy part. It’s already done. But making the sound work in VR is a whole other project. So after the rough-cut demo of “Drugs,” the group heads a few miles away to a mansion-slash-office that looks like a stop on an Entourage fan tour. This is the home of Source Sound, a sound-design studio that works on big-name video games and movie trailers, and has become the go-to studio for all things virtual reality. Source started working with Jaunt in 2013, on Jaunt’s first-ever project: a VR concert with Paul McCartney. Since then they’ve worked with everyone from Oculus to Google and YouTube. Tim Gedemer, the owner of Source Sound, gestures grandly to his giant pad and calls it “ground zero of VR and spatial audio.”
Almost immediately after we arrive, Gedemer and Cripps start discussing the “Drugs” video. Cripps doesn’t want to add sound effects to the song, and he doesn’t understand how spatial audio is different from standard stereo. Gedemer smiles, and gives a speech he’s obviously given before. “The first thing,” he says, “is that we need to decouple ourselves from anything we’re used to experiencing.” He puts his index fingers up on either side of his head, each representing a virtual speaker. The audio doesn’t move with you, he says, swinging his head from side to side; it stays where it is as you move.
Cripps wonders, doesn’t that become disorienting? So Gedemer says they could make the drum not responsive to head-tracking, to root the audio a bit. But he’d rather get weird with it: “If we’re going to do it,” he tells Cripps, “we’re going all the way. Let’s not just dip our toe in the water, because that’s not a good VR experience.” Not long after, the rest of us are ushered out again. Cripps and Gedemer have some figuring out to do.
At some point, they’re going to have to show all this to Ng, who’s constantly checking in from Dublin wondering about the progress. He doesn’t have a headset other than Google Cardboard, and so all he’s so seen is a Skype call of a 2D view of a demo of something running on a Rift. Which is maybe not the ideal way to experience VR.
Ng’s not the only person who might want to watch this video but doesn’t have a headset. Far from it: unless you work for Facebook or live with someone who does, chances are you don’t have access to a high-end VR headset. But wait, the news gets worse! Even if you do have a headset, it’s not always easy to find what you’re looking for, since it’s spread over so many apps and stores and propietary platforms. That makes life hard for users, who want all the VR they can get, and hard for the artists and creators, who would really love to reach every corner of the tech’s so-far tiny audience.
Samo is trying to engineer that most elusive of business models, the one the music industry had and lost and has tried desperately to get back: If you want something, you buy it. With money.
That’s what a company called Samo wants to do for the VR industry: connect users across all those platforms, not unlike the way Amazon became the everything store or how Steam put all your PC, Mac and Linux games in one convenient place. Samo’s plan is also to help virtual reality make financial sense. “VR’s the first platform in my lifetime that’s actually sort of disrupted the entire content ecosystem,” says Adam Johnson, who founded Samo in 2015 after working for years on video-streaming tech. A Netflix-style subscription model won’t work, he says, because there’s no back catalog of content to buy. And you can’t do ads, because there aren’t nearly enough people in VR yet. So Samo is trying to engineer that most elusive of business models, the one the music industry had and lost and has tried desperately to get back: If you want something, you buy it. With money.
While Samo is working with Cripps to handle the production for the “Drugs” video, being a production company isn’t Johnson’s aim. He hopes the rest of Hollywood will handle that part. He wants Samo to be a tech company, a pipeline for all things VR music. Rather than figuring out how to make your VR video available on Jaunt, Oculus, Steam, YouTube, Samsung VR, Facebook, Hulu, Amazon, and the untold other options yet to come, Samo hopes you’ll just give them your video and let them put it everywhere in exchange for a small cut of the sale price. That’ll certainly sound good to people like Hurd, who’s found himself constantly telling people not to give their VR content away. “My hope,” he says, “is that people will see value in these types of experiences. When I see people giving stuff away for free, I want to shake them!” (“Old Friends” will cost $3 in the Vive store.)
At least for Ng, this feels like the very beginning of Eden working with VR. It’s really important to him, a native of YouTube and Soundcloud, that everyone can experience what he makes. “I’ve been very conscious,” he says, “that I want it to work for people who don’t have Oculus or a VR headset, and are just watching on a 360 video.” That’s still tough to pull off. SB Projects, which manages Ng along with artists like Justin Bieber and Kanye West, worked with Dodocase to get a few thousand headsets to press, influencers, and fans who bought the VIP ticket package. Ng is already teaching himself some VR-making skills, and thinking about how he can incorporate the tech into his tour and music.
Ultimately, VR’s appeal to the music industry goes way beyond music videos. Imagine getting a front-row seat to a concert halfway around the world, live-streamed to your couch. Or maybe you and your friends all put on your Hololenses and watch an acoustic set right in your own living room. Next time Beyonce makes a visual album (Limeade!), you could be part of it. You could go to a concert, then go see a VR doc of the band practicing—and feel like you’re there too. Music is about connection, closeness, shared experience. VR may not be able to put you on the tour bus—but in all the other ways that matter, it kind of can..