Virtual Marionette

Real-time Puppet Animation with Kinect

Virtual Silhuette

Real-time puppet animation with low budget motion capture solution using the Microsoft Kinect connected to a macbook.

The workflow for this project is based on the OpenNI driver frame work, using SensorKinect, the AVIN hacked PrimeSense driver for kinetic supporting OSX.

The OSCELETON from Sense Bloom routes the skeleton information through open sound control messages. This information reaches the real-time animation software ANIMATA giving life to the Silhouette.

This demonstration is part of a research project on real-time puppet animation called Virtual Marionette.

credits:

Silhouette figure – Luís Silva

Demonstration+Music – Grifu

Video Demonstration

INTERFACE: Microsoft Kinect

FRÁGIL – Puppet Show from Teatro de Marionetas do Porto

Collaboration (Setembro/2010-Janeiro/2011) with Teatro de Marionetas do Porto to create linear and interactive digital animation for the play “Fragile”. This was a challenging project oriented to my research, including the study of the relationship between digital and analog puppetry and linear animation and interactive animation.
The show debuted in January 2011 with great acceptance from the public. This project was very positive both in terms of work experience as well as developed investigation.

As a result for the research highlights the comparison between the relationship of the actor with a linear animation and actor with real-time animation. In both situations, the actors interacted with the animation.
In the first situation, the time for action had been memorized by the actor, been aided by the placement of sound marks to count the times in order to synchronizetheir movements with the animation.
In the second situation, the animation was reacting to real-time action of the actor, as if they were the opposite.
The animation in real time is therefore easier, more intuitive and allows the improvisation in the scene. However one does not preclude the other, there is room for both.

video preview

Development Culture (Videogame) by Ricard Pillosu Development Manager at Crytek

The video games are an important segment in the development of real-time animation and an industry that is driving large investments in this area, with good results. Some of the interfaces in this research study were created for videogames consoles, such as remote control of the WII, Playstation Move and Microsoft Kinect. Thus, participation in the Masterclass was fully justified and profitable.

Organization IT – Instituto Telecomunicações (Veronica Orvalho)

Real Time Video with openframeworks workshop  by Pedro Angelos from Audiência Zero was justified for learning a potential authoring and programming platform for real time processing. This workshop was very important in acquiringknowledge and skills in this area.

Organization: Audiência Zero (Pedro Angelos)

Digital Narratives Conference was an import event in multimedia specially because of the experienced lecturers. Rafael Hohr from The Sunday Times and Vu Nguyen from msNBC.com are a reference in this subject.

although this event was very interesting, none of the subjects had much relevance to this project, but i enjoyed the conference.

Yesterday (29 September.2010) we lost a great friend and a puppet master. One of the greatest reference for this work.

Someone who kept alive the tradition of Portuguese Puppets “Teatro Dom Roberto” and at the same time someone who innovate the art of the marionettes. He also created some of  the greatest puppet television series like: “Árvore dos Patafúrdios” or “Os Amigos de Gaspar”.

I worked with him at Teatro de Marionetas do Porto for “bichos do Bosque”, and was working for them at this time…..

I´ll dedicate this work to you João Paulo.

«É essa marioneta mediadora que, liberta dos constrangimentos funcionais do corpo humano, na forma e no espírito, nos permite com uma enorme liberdade insuflar sinais de vida à própria matéria dos sonhos, aos devaneios mais surreais. Assim entramos numa nova dimensão do jogo teatral e jogamos com prazer o jogo das impossibilidades.»

João Paulo Seara Cardoso, in “Maravilhas teatrais voadoras”, escritos, www.marionetasdoporto.pt

using Arduino as Interface

ARDUINO + MAXMSP + MODUL8 = REAL-TIME MANIPULATION / ANIMATION

arduino

Low budget solution for real-time manipulation for live performance contents.
Using some analog knobs connected to a Arduino board that communicates to MaxMsp witch translates the data to Midi messages mapping those to functions in Modul8.

2010, video example of real-time manipulation

download video at full size (720×576 – 68Mbytes) arduino

or view the stream version above

Video Demonstration

(more…)

Present at 1ª Edition of GetSetArt Festival.

Conference with Multitouch-barcelona discusting interaction design.
Learn something important about real interaction, with touch, with feeling….witch happens when giving life to a puppet.

(more…)

In this section i will present some of the prototype applications build for the project.
The main ideia is to experiment different kinds of marionettes with different aesthetics and different methods of manipulation.

The goal is to build several puppet tools, from the simpler to the most complex prototype with different interfaces, graphic engines, physic engines, and so on.
Testing this tools with different audience/users will be the main target to achieve some conclusions about the way to manipulate puppets in realtime.

Animatic was the first project build for the Masters program in Multimedia at FEUP.

A Virtual puppets system with special focus in the puppeteering art using several ways of interaction as a way to recreate the diferent styles of puppetry, including marionettes and rod puppets.

The main goals of this project are:
– To demonstrate the potential of real-time animation based interfacesß using serveal ways of interaction;
– To open the use of virtual puppetry systems in different areas like in education, performing arts, realtime television programs and even in film animation.

The system makes use of interfaces such as the microphone to create facial expressions, body language or the use of interfaces as the WIImote (WII remote control). The prototype demonstrates the capabilites of this project in commercial and artistics areas, envision a new way of looking at animation in real time.