Disney Researchers build off an idea from 2011 to develop a new way to puppeteer digital characters in augmented reality.
In 2011, a Disney Researcher named Karl Willis developed the idea of a ‘MotionBeam’, which gives users the ability to have an AR interaction via a digital beam created by a handheld projector.
A newly published report by Disney explores the recent work from a team of Disney researchers made up of Raphael Anderegg, LoÏc Ciccone, and Robert W. Sumner, and how they approached Willis’s MotionBeam idea as a launching pad for a new extension that uses AR as the mechanism to puppeteer virtual characters in a way that feels more natural than using something like a virtual joystick or conventional thumb controllers.
What they ended up with was ‘PuppetPhone,’ a new form of augmented puppeteering powered by what the Disney Research team refers to as ‘MotionStick’.
Think of it as a digital stick connecting your phone to an action figure. Move your phone to the left and the character moves left. Move it up towards the sky and the character jumps and or floats in the air.
The idea is to capture the same excitement and imagination that comes with creating your own stories with physical action figures, toy spaceships, and cars within an AR environment; your smart phone mimicking the motion of your arms similar to puppeteering.
Since the introduction of the Nintendo Wii, the controller has gone from steering the action you see on the screen, to becoming extensions of your actual arms, allowing you to swing a tennis racquet inside of the game. Of course we are now seeing technology such as the Vive and Rift controllers, which bring even further immersion to the experience. MotionStick takes those same concepts and introduces them to an AR environment using smart phone technology.
To build your scene, you will first use your smart phone to place the AR character in your real-world surrounds. You take control of your character by placing your finger on the screen; it’s this action that creates the MotionStick between your smart phone and the AR character. Once connected, you now have the ability to make your character walk, run, crouch and even jump using a series of simple, natural gestures.
Move your smart phone to the left in a slow manner to have your AR puppet walk casually to the left. If you were to quickly move your phone in circles, however, your puppet would then engage in a frantic jog in a circular pattern.
The AR puppets can even create new puppets within the experience and allow you to switch from character to character; and because your entire touchscreen is a single button, the learning curve is incredibly easy. I mean, gwarsh, even Goofy could do it.
Disney parks — which are already considered an immersive Disney environment — are exploring a future where visitors will be able to have an immersive AR experience within various Disney parks. However, because AR can be delivered through smartphones, the experience isn’t just limited to official Disney locations. Any media – such as a dusty old Pinocchio book sitting in your parent’s basement – has the potential to unlock a fun interactive AR experience controlled via Disney’s MotionStick.
For Disney, AR creation and consumption is a big part of their strategic research towards better storytelling. MotionStick could help Disney connect even further with fans all over the globe by giving them the ability to bring Disney characters to life wherever and whenever they’d like.