CALLAS

Galaxies, Puppetwall and Euclide are Proof-of-Concepts of interactive digital theatre,  addressing active participation of the public, turned from passive consumers to active performers and creators. 
The challenge is to provide new formats or practices which support performative interaction, where media is not just created and browsed but animated in real time by the user multimodal and emotional inputs in an embodied and expressive way. See poster.

Application scenarios include enhanced theatrical and play experiences:

Galaxies	PuppetWall
Euclide

 

• Galaxies is part of a theater production about Galileo Galilei made from Studio Azzurro and performed daily between 10th - 12th of July 2008 in the Teatro Arcimboldi of Milan,Italy.
• Puppetwall is a site-specific performative installation for multimodal collective interaction
• Euclide continues the tradition of puppetry introducing high technology systems in the theatre domain

Key research questions include identifying modalities to be used as input, expressive features in each modality, and the interaction loop by creating reactive audio-visual animations that motivate users in communicating expressively and in an embodied way.

Galaxies is an experiment where stars projections are generated and move according to motion of visitors, changing colour depending on emotion detected from their voices.
Video tracking supports the understanding of position and outline of visitors on the stage, directional microphones provide input for acoustic feature analysis. Emotional states (neutral, positive or negative) detected are used to influence the appearance of the galaxies.
The experiment supported the understanding of how visitors reacted to the experience, presented to Interact 2009. 
Suggested reading is :

• Bodily Explorations in Space: Social Experience of a Multimodal Art Installation - Abstract.

In Puppetwall media is animated by direct, gestural interaction with the media elements combined with audio input and recognition of basic emotional indices from talk.
A large touchscreen enables people to collaboratively create and act out stories, using either pre-created or their own media content.   Watch the video here: the prototype, evaluated with improvisational actors and puppeteers, was presented at AVI 2008.
Suggested reading is:

• Exploring Emotions and Multimodality in Digitally Augmented Puppeteering - Abstract.

Euclide is a computer-animated puppet used in CALLAS interactive installations. Hidden in a control room a concealed animator wears a cyber glove connected to a graphic computer; a monitor, connected by cable to the computer, shows the virtual character on the "stage" to the audience. The animator's hand movements activate the virtual character, controlling his mimic, whilst digital effects alter animator's voice and accelerometers sensors support affect recognition. Even if the audience is aware of the existence of the concealed animator (the audience knows where he is hidden), people react to the virtual character as to an independent entity and not a "puppet" in his hands. The contrast between the cold appearance of computer imagery and the lively fun virtual character entices even the most reluctant spectator.
A first prototype was demonstrated at the IED Workshop in 2007, and  at Ubicomp conference in 2009. The subject topic was reference for a training workshop during eNTERFACE 2009 .
Watch the video with puppet Euclide Recent prototypes experimented the Euclide puppet at Museo della Scienza in Naples, through field trials aiming to engage kids in their visit to the museum. Performances such as Bit & Bold have been derived by Studio Azzurro from these activities, repeatedly featured with great success of public (children and young guys) in Theatres in Italy.
Suggested reading is:

• Interactivity of an affective puppet -  Abstract.