CORDIS
EU research results

CORDIS

English EN
IMMERsive multi-modal interactive preSENCE

IMMERsive multi-modal interactive preSENCE

Objective

The main objective of this Integrated Project (IP) is to enable highly realistic multi-modal interactive immersion into virtual and augmented reality environments. Its focus will be on visual, haptic and auditive sensory components, while addressing an even broader range of human senses. Hand-based interaction will serve as a prototypical framework, but the multimodal feedback of all relevant information will be considered.

Three driving scenarios will cover the full spectrum of problems arising when implementing interactive environments: manipulation of an object by a human; direct contact between persons e.g. during a handshake or social dancing; and object mediated collaboration between humans, like jointly carrying a heavy object. These targets can only been reached by a concerted action, unifying competences in cognitive neuroscience, in psychophysics and in technology development. The indispensable scientific fundament will be established by basic research looking for new insights into the psychophysical and neurological mechanisms underlying multimodal interaction.

A novel, data-driven algorithmic paradigm will be investigated for the generation of sensory feedback. During a first recording stage all relevant data characterizing interaction are collected by multimodal sensing suites. The recorded data will be processed in order to convert the raw signals into abstract descriptors. This abstraction will not only enable to faithfully replay the interactive session using the collected data while replacing the real counterpart by a virtual one, but will provide the fundament for prediction schemes to create sensory feedback for interaction which never has been observed before. The resulting prototype systems will be systematically evaluated by novel measures of presence to be developed in the course of the project. The IP will lead directly to trend-setting applications in science, medical therapy, entertainment, training, and education with immense potential.

Leaflet | Map data © OpenStreetMap contributors, Credit: EC-GISCO, © EuroGeographics for the administrative boundaries

Coordinator

TECHNISCHE UNIVERSITAET MUENCHEN

Address

Administrative Contact

Participants (9)

Sort alphabetically

Expand all

CENTRE NATIONAL DE LA RECHERCHE SCIENTIFIQUE

France

EIDGENÖSSISCHE TECHNISCHE HOCHSCHULE ZÜRICH

Switzerland

MAX PLANCK GESELLSCHAFT ZUR FOERDERUNG DER WISSENSCHAFTEN E.V.

Germany

TECHNION - ISRAEL INSTITUTE OF TECHNOLOGY

Israel

THE UNIVERSITY OF BIRMINGHAM

United Kingdom

UNIVERSIDAD POLITECNICA DE MADRID

Spain

UNIVERSITA DI PISA

Italy

UNIVERSITAT POLITECNICA DE CATALUNYA

UNIVERSITE D'EVRY-VAL D'ESSONNE

France

Project information

Grant agreement ID: 027141

  • Start date

    1 January 2006

  • End date

    31 December 2009

Funded under:

FP6-IST

  • Overall budget:

    € 6 123 435

  • EU contribution

    € 5 550 000

Coordinated by:

TECHNISCHE UNIVERSITAET MUENCHEN