Community Research and Development Information Service - CORDIS


VHIA Report Summary

Project ID: 340113
Funded under: FP7-IDEAS-ERC
Country: France

Mid-Term Report Summary - VHIA (Vision and Hearing in Action)

The overall goal of the ERC advanced grant VHIA (Vision and Hearing in Action) is twofold: to study the fundamental computational aspects of audio-visual fusion for human-robot interaction, and to develop the corresponding methods, algorithms, software and proof-of-concept demonstrations. This goal necessitates an interdisciplinary team, and one important aspect of the project is to enable researchers with expertise in different topics to carry out collaborative research. The team members cover the following disciplines: computer vision, statistical machine learning, audio signal processing, multi sensor fusion, robot programming, and robotic software. During the first half of the project (30 months) both the research and technological work-packages were achieved as planned, while the link between work-packages has been guaranteed by the fact that each team member has expertise in at least two of the project topics. The following topics were addressed: recognition and tracking of humans and of their activities, estimation of head orientation, gaze and visual focus of attention, localization of single and of multiple sound sources, separation of static and of moving sound sources, spatiotemporal alignment of auditory and visual features, and distributed architecture for human-robot interaction. During the period from February 2014 to July 2016 (30 months) he VHIA members published 8 journal papers and 13 conference papers. VHIA received the best student paper award (second place) at the 2015 IEEE International Conference on Image Processing (ICIP'15), the best student paper award at the 2015 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA'15) and the 2016 award for outstanding contributions to neural systems, established by World Scientific Publishing Co., for the most innovative paper published in the previous year of the International Journal of Neural Systems. The technology developed in VHIA is implemented onto several humanoid robots manufactured by Aldebaran Robotics (Softbank).

Reported by

Follow us on: RSS Facebook Twitter YouTube Managed by the EU Publications Office Top