Skip to main content
European Commission logo print header

MultimodAl analysis/synthesis systEm for human INTERaction to virtual and augmented environments

Ziel

The project will define new models and will implement advanced tools for audio-video analysis, synthesis and representation in order to provide essential technologies for the implementation of large-scale virtual and augmented environments. The metaphor, which will inspire the project approach, is oriented to make man-machine interaction as natural as possible, based on everyday human communication means like speech, facial expressions and body gestures from both sides. A case study application on Internet will be developed and demonstrated.

Objectives:
The objective of the project is to define new models and implement advanced tools for audio-video analysis, synthesis and representation in order to provide essential technologies for the implementation of large-scale virtual and augmented environments. The work is oriented to make man-machine interaction as natural as possible, based on everyday human communication by speech, facial expressions and body gestures. Man to machine action will be based on coherent analysis of audio-video channels to perform low-level tasks, or high level interpretation and data fusion, speech emotion understanding or facial expression classification. Machine-to-man action, on the other hand, will be based on human-like audio-video feedback simulating a "person in the machine". A common sw platform will be developed by the project for the creation of Internet-based applications. A case study application will be developed, demonstrated and evaluated.

Work description:
The project activity will last three years, the first part devoted to specifications definition and tools development, the last part dedicated to the implementation of demo applications. Project coordination, within WP1, will be carried out in synergy and concentration with other projects working in similar research areas within and outside the IST program. Most of the WPs will be activated at the beginning of the project. In particular, WP2 will take responsibility of defining systems specification with particular reference to the common software platform, which will be developed later on in WP5, that will integrate all the tools into an environment with authoring capability for the creation of applications of shared virtual and augmented reality. Technical work packages WP3 and WP4 will develop a library of tools for managing man-machine interaction with reference to man-to-machine and machine-to-man communication, respectively. WP3 will be oriented to the implementation of audio-video analysis tools, WP4 o the implementation of audio-video synthesis tools. Both WP3 and WP4 will guarantee full compliance to specifications defined in WP2. Activities in WP5 will be focused on the integration of the developed tools into a common software platform and on the implementation of an example of application based on shared virtual and augmented environments.

The integrated sw platform, will be developed progressively through upgraded releases, the first of which at the end of the first year of project activity. Compliance with MPEG-4 and MPEG-7 will be guaranteed by deep project commitment in the standardisation process (WP2). At project conclusion, the Interface consortium will organise an International Workshop for public demonstration and dissemination of the achieved results.

Milestones:
During the 3 year project, major deliverables will be produced that will represent the actual project milestones. In particular, after the first year, the common sw platform with integrated tools, after the second year, the updated sw platform with added authoring functionality and, at the third year, the implementation of a case study application, based on the common sw platform. This last milestone will also include the organisation of a public event with live demonstration of results.

Aufforderung zur Vorschlagseinreichung

Data not available

Koordinator

UNIVERSITA DEGLI STUDI DI GENOVA
EU-Beitrag
Keine Daten
Adresse
VIA BALBI 5
16126 GENOVA
Italien

Auf der Karte ansehen

Gesamtkosten
Keine Daten

Beteiligte (13)