Periodic Reporting for period 1 - VHIALab (Vision and Hearing In Action Laboratory)
Reporting period: 2018-02-01 to 2019-01-31
The objective of the VHIALab proof of concept project was twofold: first, the demonstration of software packages enabling a robot to acquire audio and visual information and to robustly process and analyze it for the purpose of enabling the robot to interact with several users, and second, to investigate the potential of the technology in terms of economic value and of business opportunities. VHIALab has built on the scientific findings and methodological developments of ERC Advanced Grant ``Vision and Hearing in Action" - VHIA (1/2/2014 - 31/1/2019). By providing solutions to the audio-visual machine perception problem in the context of multi-party human-robot interaction, the project opens the door to a wide range of applications with high societal impact. In particular, the VHIA/VHIALab technology is well suited to address issues associated with Socially Assistive Robots (SARs). With the continuous and accelarated aging of the European population, SARs are likely to play an important role in healthcare and psychological well-being of the elderly, most notably during non-medical phases of the hospital process. There is a consensus that the use of SARs has a positive impact, such as decreased stress and loneliness and improved mood and sociability. Nevertheless, current SAR abilities to interact with users (for example, patients, their family members and hospital staff) are limited to touchscreen- and voice-commands. This has a negative impact on the level of acceptability of using robots by the elder population. Based on this unsatisfactory and limited technological and commercial offer, VHIALab developed software packages that would enable a robot to naturally interact with people based on advanced audio signal processing, computer vision, machine learning and multi-modal fusion methods, as well as based on an advanced robot software and hardware architecture that has been designed, implemented, tested and validated. VHIAlab researchers and engineers worked in close collaboration with three companies: SoftBank Robotics Europe (France), Samsung Electronics (Korea), and ERM Automatismes (France). The VHIALab software packages were embedded into the NAO humanoid robot (SoftBank) and into a companion robot prototype under development (Samsung). The third company (ERM) is a french SME with expertise in the development and commercialization of integrated robotic solutions. Together with ERM and with Broca Hospital (Assistance Publique des Hopitaux de Paris), the VHIALab team investigated the potential of audio-visual machine perception technology to address the gerontological healthcare market and a number of use cases were clearly identified. Although, the VHIALab solutions for SARs in particular, and for HRI in general, are very promising, the conclusion of the market study conducted by the project is that the technology is not mature enough to be massively deployed in hospitals and that further scientific and technological progress is needed. Based on the encouraging results of VHIALab (and of VHIA) it was decided to submit an H2020 RIA proposal under the ICT-10-2018-2020 call "Robotics Call Technology", with ERM, Broca Hospital and with several other partners. If granted, this project will build on the scientific and technological achievements of VHIA and on the economic opportunities revealed by the VHIALab project, such as to enable tomorrow's generation of social robots.