CALLAS

Augmented Reality for Art is subject of a dynamically-generated piece of digital art, e-Tree (Emotional Tree), developed in CALLAS as a virtual tree whose growth and evolution reflects the perceived affective response of the spectators. The leading digital artist Maurice Benayoun  collaborated to develop the idea and artistic brief.
The e-Tree provide an interactive session where spectators' reactions are captured, making usage of Multi Keyword Spotting , Real-time emotion recognition from speech and Video Feature Extraction components, combined using a common affective model which then controls properties of the digital artwork. 

Spectators interact and respond to the e-Tree, which aggregates affective reactions over a sustained period of interaction. The concept supports the exploration of "affective feedback loops" where a participant's response to dynamic artwork determines changes and development that occur within the artwork, to which the participant then also responds. The responses of the viewers of the installation are captured using cameras and microphones, which feed CALLAS components that produce affective interpretations of those responses. The affective input is combined into a dimensional affective model (according to the PAD temperament model) which then drives the growth and branching of the tree.

• Small, sustained positive input causes the tree to start to "twitch" and come "alive".
• Larger, sustained positive inputs cause it to branch and grow progressively more quickly.
• Negative inputs cause growth to shrink back, and larger negative inputs cause the tree to lose branches (which re-appear as affective values return to neutral).

The shape and angle of branches reflects the affective conditions when a branch was instigated, and the overall look of the tree at the end of the session reflects the history of affective response over the whole of the interaction. Watch the video of the eTree here.

Initial CALLAS e-Tree was based on an experimental version of ARToolkit, described in following papers presented at ISMAR2007:

• A High-level Event System for Augmented Reality: Abstract
• An Emotionally Responsive AR Art Installation: Abstract

This Scientific Showcase was shown at  ICT2008 [read also article in CALLAS Newsletter1]. See also additional discussion of the artistic concept and technology aspects in the following papers:

• E-Tree: Emotionally Driven Augmented Reality Art: Abstract
• An affective model of user experience for interactive art: Abstract