3D-games for TUNing and lEarnINg about hearing aids

Project Information

3D Tune-In

Grant agreement ID: 644051

DOI

10.3030/644051

Project closed

EC signature date 12 December 2014

Start date 1 May 2015

End date 30 April 2018

Funded under

• INDUSTRIAL LEADERSHIP - Leadership in enabling and industrial technologies - Information and Communication Technologies (ICT)

Total cost

€ 3 355 151,85

EU contribution

€ 2 896 175,00

Coordinated by

IMPERIAL COLLEGE OF SCIENCE TECHNOLOGY AND MEDICINE
United Kingdom

Periodic Reporting for period 3 - 3D Tune-In (3D-games for TUNing and lEarnINg about hearing aids)

Reporting period: 2017-05-01 to 2018-04-30

Summary of the context and overall objectives of the project

3D Tune-In (3DTI) is an EU-funded project coordinated by Imperial College London (2015-2018), which brings together the relevant stakeholders from the videogame industry, academic institutions, a large hearing aid manufacturer, and hearing communities, to produce digital games in the field of hearing aid technologies and hearing loss.
Using a participatory-design process involving relevant stakeholders (human factors experts, technology developers, hearing communities, end-users) the 3DTI project aims to:
•Enable end users to explore, review and customise hearing aid devices for different usage scenarios
•Enable individuals with no hearing impairment to understand how hearing loss can compromise everyday activities and how a hearing aid can improve this situation
•Enable gaming SMEs to explore new non-leisure applications in the area of hearing loss and hearing aid technology with support from the scientific community
•Enable hearing aid providers to evaluate and demonstrate the various functionalities of their products to improve their services and increase sales
During the course of the project, several libraries and applications have been developed, including the 3DTI Toolkit (a standard C++ library for audio specialisation and simulation of hearing loss and hearing aids) and Applications (five applications, each of which aim to improve the lives of those affected by hearing loss). All the developed components have been created employing a participatory design approach, a series of AGILE development/evaluation loops and a final summative evaluation.
The dissemination of the project has been particularly successful, including scientific and industry dissemination activities, journals and magazines publications and attendance to several national and international conference venues. Also the exploitation has been particularly successful; the Toolkit has been released open-source, and has already shown a good success in terms of developers and researchers using it in their projects/studies. Each Application’s exploitation plan has been carefully planned, and is currently undergoing; an example is www.hearingamestudio.com a joint effort of the 3DTI Italian partners for the exploitation of their Applications.
All project outputs and material can be found on the project website.

Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far

During the course of the project we have developed the 3DTI Toolkit, a library which provides 3D audio simulations (both for loudspeakers and headphones) with a high level of realism and immersiveness, while allowing for the simulation of hearing aid devices and of different typologies of hearing loss. An easy-to-use test-app interface allows the Toolkit to be used by hearing aid users, audiologist and any other interested party without needing to go through complex code and settings.
Using the Toolkit, it is possible to simulate complex listening environments (e.g. a train station or a noisy restaurant), and demonstrate how much a hearing aid could improve the situation. Users can simulate their specific hearing loss and show others how complicated it is to understand speech when other noises are present. The Toolkit test-app can be downloaded and used for free directly from the project website (http://3d-tune-in.eu/toolkit-developers) while the open-source code can be accessed from our GitHub account (https://github.com/3DTune-In/).
In addition to the Toolkit, 3DTI has produced 5 different applications aimed at different groups of the hearing impaired and non-hearing impaired communities (http://3d-tune-in.eu/applications):
•Musiclarity. A web-based app which supports hearing impaired individuals and hearing aid users in listening music
•Dartanan. A mobile app for children in order to help them learn about the functionalities of their hearing aid while playing a fun and engaging game.
•Play&Tune. A mobile app specifically designed for adult hearing aid users which guides them through the different functionalities of the hearing aid using a simple and engaging interface.
•AudGamPRO. A loudspeaker-based desktop application for replicating the acoustic conditions of real-life scenarios (e.g. restaurant, street, etc.) in order to allow audiologists to verify if and how hearing aids have been fitted properly.
•Darius’ Adventure. A mobile app for children without hearing problems which aims to educate them about living with a hearing impairment, simulating the presence of hearing loss and engaging the child to try different hearing aids and communication strategies in order to improve their hearing.
All the developed components have been created employing a participatory design approach, a series of AGILE development/evaluation loops, and a final summative evaluation. Ultimately, all applications met their original goals and have been released according to the project’s exploitation plans. Furthermore, all project outcomes have been communicated and disseminated according to the original plan.

Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far)

The progress beyond the state of the art achieved during the 3DTI project can be summarised in the following points:
-The 3DTI Toolkit which includes several innovative functionalities in a single open-source library, available also as an executable application (for MacOS, Windows and Linux), and as a wrapper for Unity and Javascript. This research represents a contribution to the state of the art in acoustics and signal processing research, and is currently being published.
-A series of studies on Head Related Transfer Functions selection, adaptation and individualisation has been carried out during the project. This research represents a contribution to the area of acoustics and psychoacoustics research, and is currently being published.
-All 3DTI Applications have been created employing a participatory design approach, a series of AGILE development/evaluation loops, and a final summative evaluation which involved 319 participants. Data from these evaluations represent a contribution to the state of the art in human factors research and serious games, and is currently being published.
In terms of societal engagement, in addition to the participatory design and evaluation stages which involved more than 500 individuals (hearing aid users, hearing impaired individuals, audiologists, developers and general public), the project reached several communities through participation in conferences, workshops and seminars. Three major workshops were organised by 3DTI including a final launch event (Hearing Futures) at the London V&A Museum in April 2018. Attended by more than 100 participants, Hearing Futures sold out in a few days and a follow up event is being organised. A full list of dissemination and societal engagement activities can be found in the Dissemination and Communication Report (D5.4) http://3d-tune-in.eu/Deliverables. The highlight of the last year of the project has been the NHS CSO Healthcare Research Award, won by the project for the category of Healthcare Science Partnering Patients and Citizens (http://3d-tune-in.eu/3d-tune-in-wins-nhs-award)
The consortium is now moving in several directions aiming to carry forward activities started during the 3DTI project (www.hearingamestudio.com; www.pluggy-project.eu; www.scent-project.eu) and to set up new collaborations (Weizmann Institute of Science, Georgia Tech, University of Castilla La Mancha, AmpAmsterdam, Video Art, University College London, Tel-Aviv University) to further develop and exploit the 3DTI Toolkit.

3D Tune-In consortium at the kick-off meeting (London May 2015)

3D Tune-In Toolkit postcard

3D Tune-In postcard

3D Tune-In consortium at the final review meeting (London April 2018)