Skip to main content

Multisensory Expressive Gesture Applications

Objective

The project is centred on the modelling and real-time analysis, synthesis, and networked communication of expressive and emotional content in non-verbal interaction by multi-sensory interfaces, from a multi-modal perspective. Music (incl. singing voice) and movement (incl. dance) are first class channels for conveying expressive and emotional content. A project mission is to deliver products and related applications and services: for each product, the work is organized in three development cycles converging to the final release in a three-phase spiral approach. Each product is developed on the basis of industry standards and architectures, including those developed by partners in the consortium. Since the early stages, the project will deliver working prototypes for early validation, experiments with groups of users. At the end of each of the three basic project cycles, the prototypes will be installed and validated with a larger number of users in public events (e.g., exhibit, artistic and music performance, workshops). The demonstration of project results in public events will also contribute to dissemination. The partners in the consortium already developed an internationally awarded concrete basis of consolidated research and technology which will be directly transferable into the project at its startup.

Objectives:
Strategic and economical objectives include:
(i) consolidation of a European route original in both cultural and scientific content with respect to Affective Computing (USA) and KANSEI Information Processing (Japan);
(ii) dramatic raising of the competitiveness of European Industry of Digital Music Instruments (suffering from the aggressive Japan and USA industry), of Entertainment and Multimedia Industry.

Work description:
The project is centred on the modelling and real-time analysis, synthesis, and networked communication of expressive and emotional content in non-verbal interaction by multi-sensory interfaces, from a multi-modal perspective. Music (including singing voice) and movement (including dance) are first class channels for conveying expressive and emotional content. A project mission is to deliver two pre-products and related applications and services. For each pre-product, the work is organized in three development cycles converging to the final release in a three-phase spiral approach. The results obtained from research will be utilized for the development of the pre-products. Each pre-product is developed on the basis of industry standards and architectures, including those developed by partners in the consortium. Since the early stages, the project will deliver partial prototypes for early validation of techniques and user requirements, including experiments with groups of users. At the end of each of the three basic project cycles, the prototypes will be installed and validated with a larger number of users in public events (international industry exhibits, artistic and music public performances, conferences and workshops). The demonstration of project results in public artistic events will also contribute to dissemination. The partners in the consortium already developed an internationally awarded concrete basis of consolidated research and technology which will be directly transferred into the project since it start-up. During the whole life cycle of the project, the presence of partners from research on expressiveness in music and dance, game and digital music instruments industry, VEs and visual media, sensor fabrics materials, will provide a cross-fertilisation and synergy as a fundamental concrete contribute to the originality of the pre-products definition and development. The Liaison Board of the project will be invited to participate to the public events and organised workshops in the project.

Milestones:
Three milestones at each conclusion of the three cycles of the spiral approach. Each milestone includes a version of the main scientific/technological product (MEGA System Environment) and a public demonstration/artistic event.
Milestone M1: initial prototype of the product and its experimenting in a small-scale public performance;
Milestone M2: second prototype of the product and a second public event;
Milestone M3: Final product and related applications and services, final validation and exploitation of results.

Funding Scheme

CSC - Cost-sharing contracts

Coordinator

UNIVERSITA DEGLI STUDI DI GENOVA
Address
Via Balbi 5
16126 Genova
Italy

Participants (8)

CONSORZIO PISA RICERCHE SCARL
Italy
Address
Piazza Alessandro D'ancona 1
56127 Pisa
EIDOMEDIA
Italy
Address
Via Assab 4/2
16131 Genova
GENERALMUSIC S.P.A.
Italy
Address
Via Delle Rose 12
47040 San Giovanni In Marignano
KUNGLIGA TEKNISKA HOEGSKOLAN
Sweden
Address
Valhallavaegen 79
10044 Stockholm
TELENOR COMMUNICATION AS
Norway
Address
Universitetsgaten 2
0130 Oslo
UNIVERSITA DEGLI STUDI DI PADOVA
Italy
Address
Via Viii Febbraio 2
35122 Padova (Pd)
UNIVERSITEIT GENT
Belgium
Address
Sint Pietersnieuwstraat 25
9000 Gent
UPPSALA UNIVERSITET
Sweden
Address

751 05 Uppsala