Virtual reality isn’t just for video games. The same technology that can help to create room-scale VR is being used in television and film production as a way to ease the burden of setting up shots and going to far-off locations.
CBS Digital, a self-described “studio agnostic” company stationed inside the CBS Television City in Los Angeles, have developed a new system, called Parallax–named after a camera effect where an object in view can appear in a different position when viewed from different angles–which is helping television directors to view sets in a virtual space. The company hopes that with this technology, crews can save money and focus on things such as acting and storytelling instead of scouting and framing shots.
The company recently invited the press to check out its facilities, including a visual effects studio and a relatively small room covered in the familiar color of green screens. Craig Weiss, executive director at CBS Digital, led the group into the room and pointed up to a bunch of white modules poking down from every part of the ceiling.
Weiss mentioned that they were optical tracking cameras. A man with a large, RED digital camera moved around the room, pointing the lens at the green screens. We watched a scene of the Griffin Conservatory in Los Angeles on a monitor. As the camera panned, so did our perspective.
Optical tracking cameras
Working in the Unreal engine, visual effects experts can set up scenes in real time and block out an entire sequence. They can show it to a director, who can then decide how they want to film a scene. Because an entire location can be recreated digitally, a director can move around a space, checking out different angles and potential shots before making any large decisions. Directors can also test out lenses and technologies like blue and green screens in the virtual world. A house can be built with exact dimensions.
Forensic specialists have been utilizing 3D laser scanners to digitally recreate crime scenes in a hyper-realistic fashion. The same goes for film and television engineers, who can send small scouting teams to locations to capture every angle and piece of a scene. Weiss said that a group captured two blocks in New York City, which took around 12-14 hours. Then the assets were uploaded. The team can also use a stock library of locations and objects to plan out a scene before setting it up in real time.
The executives point to a shot in The Last Man on Earth, where the character played by Will Forte sets off fireworks and accidentally destroys the city of San Francisco. Obviously, the people behind the show didn’t blow up the city, but instead recreated the city digitally and put the actors in front of a blue screen. This was mapped out first in Parallax so that the director could see the optimal way of planning out that sequence.
“For me, it’s like directing a stage play,” said George Bloom, executive producer at CBS Digital. He spent 14 years directing TV shows, trailers, and other content, so as he speaks, he often draws on his experiences directly behind the camera. “At the end of the day, you can just focus on that performance and bring the world here.”
Directors often don’t have visual effects experience (and they shouldn’t need to), so the goal is to ensure those in charge are focusing on the things they’re good at–managing what goes on in front of the camera with actors–instead of figuring out how to set up shots at, say, a hotel in Connecticut when the production is on a studio back lot in LA.
Even when actors get on the set, current technologies can be alienating. Audiences get a kick out of behind-the-scenes footage where actors have to react to a ball on a stick in front of a green screen, for instance, because we can’t imagine how it’ll translate to the screen. CBS Digital hopes that by recreating sets virtually and broadcasting them to the actual physical set, they can erase some of the discomforts.
“I think having a blue screen or a green screen out there is a little disconnecting for everybody–for the actors, for the director–and for the ability for the actors and the director to see what’s going to be there and they can frame it, that all goes away,” Weiss said.
As a natural extension, CBS Digital is also working to make room-scale, walkable experiences in virtual reality. It’s played around with “circular” VR, such as with Facebook 360 and Google Cardboard, but officials hope to release recreated sets and sequences from shows such as Stranger Things and Daredevil for the Oculus Rift and the HTC Vive.
The Stranger Things was full-blown horror. You can watch the stars of the series freak out in the video below.
The guests got to try out the VR headset–an Oculus headset set up to link with the optical tracking cameras. No scares in this one, but there was a moment where you could walk off a balcony multiple stories in the air and keep on walking.
In the end, the goal isn’t to get rid of practical effects or physical sets altogether. As we’ve seen from films like the Star Wars prequels, making everything digitally is not always the smartest move. It’s about balance. The visual effects team might recreate sets in a digital space, but there are decorators and carpenters that still have to build base sets. Now that more platforms are popping up to make television, there’s even more work to be done.
“What’s really a powerful thing that I want to emphasize is that we’re taking a lot of really great technology and simplifying it for filmmakers to allow them to really embrace the medium of virtual sets and visual effects and to make something that is very accessible and understandable to them,” Weiss said. “This is the stuff they love to do.”