Accepted for publication for CHI 2012 (ACM Digital Library)

Use your body as a puppetry controller giving life to a virtual silhouette through acting.

We present a solution that allows real-time interactive control of virtual shadow puppets for performance animation based on body motion.
The interaction with the virtual silhouette with this method is somehow similar to the hand shadow performance, where the performer models his hands to create the illusion of a shape, is like mixing the performance of an actor with the manipulation of a puppeteer.

Abstract

In this work we propose to use our body as a puppetry controller, giving life to a virtual silhouette through acting. A framework was deployed based on Microsoft Kinect using OpenNI and Unity to animate in real-time a silhouette. This was used to perform a set of experiments related to the user’s interaction with human and non-human like puppets. We believe that a performance-driven silhouette can be just as expressive as a traditional shadow puppet with a high degree of freedom, making use of our entire body as an input. We describe our solution that allows real-time interactive control of virtual shadow puppets for performance animation based on body motion. We show through our experiment, performed by non- expert artists, that using our body to control puppets is like mixing the performance of an actor with the manipulation of a puppeteer.