Skip to main content

Vision and Hearing In Action Laboratory

Objective

The objective of VHIALab is the development and commercialization of software packages enabling a robot companion to robustly interact with multiple users. VHIALab builds on the scientific findings of ERC VHIA (February 2014 - January 2019). Solving the problems of audio-visual analysis and interaction opens the door to multi-party and multi-modal human-robot interaction (HRI). In contrast to well investigated single-user spoken dialog systems, these problems are extremely challenging because of noise, interferences and reverberation present in far-field acoustic signals, overlap of speech signals from two or more different speakers, visual clutter due to complex situations, people appearing and disappearing over time, speakers turning their faces away from the robot, etc. For these reasons, today's companion robots have extremely limited capacities to naturally interact with a group of people. Current vision and speech technologies only enable single-user face-to-face interaction with a robot, benefitting from recent advances in speech recognition, face recognition, and lip reading based on close-field microphones and cameras facing the user. As a consequence, although companion robots have an enormous commercialization potential, they are not yet available on the consumer market. The goal of VHIALab is to further reduce the gap between VHIA's research activities and the commercialization of companion robots with HRI capabilities. We propose to concentrate onto the problem of audio-visual detection and tracking of several speakers, to develop an associated software platform, to interface this software with a commercially available companion robot, and to demonstrate the project achievements based on challenging practical scenarios.

Host institution

INSTITUT NATIONAL DE RECHERCHE EN INFORMATIQUE ET AUTOMATIQUE
Net EU contribution
€ 149 866,00
Address
Domaine De Voluceau Rocquencourt
78153 Le Chesnay Cedex
France

See on map

Region
Ile-de-France Ile-de-France Yvelines
Activity type
Research Organisations
Non-EU contribution
€ 0,00

Beneficiaries (1)

INSTITUT NATIONAL DE RECHERCHE EN INFORMATIQUE ET AUTOMATIQUE
France
Net EU contribution
€ 149 866,00
Address
Domaine De Voluceau Rocquencourt
78153 Le Chesnay Cedex

See on map

Region
Ile-de-France Ile-de-France Yvelines
Activity type
Research Organisations
Non-EU contribution
€ 0,00