Each time I move my arm, the frog on screen extends his. I drop into a low splits and the frog follows suit, even mirroring me when I transition into a handstand. Yes, his hands float a little over the ground, but that’s because he’s been calibrated as someone six inches taller than me. Finally, I take my headset off, and the frog reacts like I've decapitated it, head hanging backward, reminiscent of Shakespeare’s Macbeth. But even in its death throes, it’s still uncannily accurate, following my every move perfectly — with zero cameras involved in this motion capture. Motion capture technology costs big bucks in Hollywood, and is widely used for everything from modeling animation to creating a realistic Gollum. But motion capture requires a big investment for a company, which tends to cut out the little guys. The suit I'm wearing hopes to bridge that gap. Danish company Rokoko is offering this for the wallet-friendly price of $2495, and they've just opened for orders. But where other cheap motion capture outfits provide a disjointed effect, Rokoko's magic sauce is their sensor calibration.
Here's how it works. An actor/designer/developer wears the Smartsuit Pro, a stylish ninja like bodysuit equipped with 19 sensors and a battery pack that’s held at the small of the back.
Rokoko CEO Jakob Balslev recognizes that he’s priced his system low, but says this is intentional. “It's not a price sensitive industry, but we want to let everyone try, we want to democratize motion capture,” he says earnestly. Balslev’s background is in film production and education, and the idea for the Smartsuit Pro originated with a desire to create a way for children to do motion capture so he could represent their bodies in animation.
It didn't take long for him to realize that this was far more complicated and expensive than he thought, but rather than be deterred he turned to innovation to solve this. Realizing that everyone currently carries an incredibly powerful processor — their cell phone — he started investigating how to make motion capture using a similar mechanism. This evolved into the 19 sensor suit we see today, each sensor equipped with a gyroscope accelerometer and magnetometer, similar to the type of sensors found in cell phones. They’re arranged across the body like a spidery nervous system, and link to the hub at the base of my spine.
For Balslev, the Smartsuit Pro is a way to democratize motion capture technology. “It’s not only cheaper but it’s location independent,” Balslev says, gesturing to the large light filled room we’re standing in, inside Upload VR, a San Francisco-based virtual reality incubator and coworking space. “For regular motion capture this room would be impossible — there's far too much light.” Plus, you’d need cameras, infrared sensors and more, as motion capture technology, while incredibly impressive and a great storytelling tool is also fiddly, clunky, and relies on experts to create and feed the data. Here, all we have is the suit, a laptop, and an LCD screen to display the action. “We can also shoot on location — if you want Gollum climbing rocks, you don't need to build a set!” he adds. The Smartsuit Pro uses WiFi to record but also can store sensor data to an integrated SD card, making it useful when on location with poor signal.
Chris Ford, former director of Pixar was so impressed with Rokoko’s product that he signed up to join Rokoko’s board of directors and be part of their executive strategic committee this May, giving the company a much-needed industry stamp of approval. The Smartsuit Pro can be used to record, edit and live stream with their custom software, but it also interfaces with Unity and Unreal game engines and can export files in BVH or FBX format, which makes it easy to import straight into programs like AfterEffects and Blender. Another bonus of recording sensor data instead of traditional motion capture images is that file sizes are kept small and the suit has an overall wireless range of 100 meters.
I wondered why the company chose to go in this direction, rather than utilizing a lighthouse like system as offered by HTC’s Vive — positional tracking that many consumers are already familiar with. “Like the traditional [motion capture] systems, the Vive is optical and comes with occlusion issues where you can block the view of the trackers and lose data and you can never move out of sight of the trackers,” says Balslev. “For the type of precise human motion capture we’re doing at Rokoko, you still need more data points than the Vive can provide. We produce denser data output and the type of professional-grade accuracy that our target audience demands.”
For now, it’s available to order in four sizes, small, medium, large and extra large, and it can be adjusted with velcro straps. Surprisingly, for a working tool, the suit is actually attractive to wear, looking like a sleek bobsled suit, one that’s almost catwalk appropriate. To make sure this was wearable, Balslev enlisted the help of a Danish fashion designer. “I wanted something that you don’t feel ridiculous when you sit at a desk and wear it,” he says. “I want people to feel comfortable.”
That’s something that took some time, as earlier versions weren’t as flattering. For one, when he started designing this the only material available was purple, which did not get good feedback from his test subjects. And then there was the decision if it should be worn under or over jeans — cue awkward moments and weird press briefings till over the jeans was given the thumbs up. Even so, it’s still an inelegant process getting into the suit, and involved me hopping around the room on one foot as I pulled up the lycra-like legs. The suit is heavier than expected when you pick it up off a chair, but feels weightless when on — I attribute the initial surprise at the weight to how light the design looked. Zips conceal all the wires and battery packs, which can be removed before washing.
What’s most impressive about this is not just the simplicity but the breadth of potential applications it can be used in. For one, they’ve included a program that accurately maps the degree of angles of the body, which could be a great tool for rehab, when you’re trying to improve motion and want to track your progress. The same can be said for sports as golfers etc. would benefit from being able to use this tech to assess their swings. Virtual reality seems like another obvious play as this type of tracking could be used to create more realistic avatars in places like AltspaceVr or to enhance gaming with HTC’s Vive, now they've released the Vive Tracker. It will be intersting to see how this progresses, but the possibilities here have only just started to be explored.
“We could develop for Prosumers, or do a collaboration with Nike if they want... there are so many possibilities,” Balslev says. For now, his focus is in motion capture as that’s an established industry and people can see the application benefits immediately, and then he’ll look to grow in other directions. “VR reinvented gaming, and we want to reinvent the keyboard and the mouse,” he says. “This disrupts the motion capture industry and for the first time makes it a creative tool to personalize films or games.”
The next big step for Rokoko is finger tracking which they're hoping to roll out this year. He’s also thinking about extending the Smartsuit Pro line into child sizes; though surprisingly, he says he gets asked about if he could fit it for animals a lot. “I could do that — it’s all about rigging the sensors,” he says. "I could get this working for any kind of animal."