Spatial Audio Virtualization and Gamification for Hearing Assessment and Enhancement

Project Information

SAV

Grant agreement ID: 101129903

DOI

10.3030/101129903

EC signature date 4 August 2023

Start date 1 October 2023

End date 30 September 2027

Funded under

• Marie Skłodowska-Curie Actions (MSCA)

Total cost

No data

EU contribution

€ 496 800,00

Coordinated by

UNIVERZITA PAVLA JOZEFA SAFARIKA V KOSICIACH
Slovakia

Periodic Reporting for period 1 - SAV (Spatial Audio Virtualization and Gamification for Hearing Assessment andEnhancement)

Reporting period: 2023-10-01 to 2025-09-30

Summary of the context and overall objectives of the project

The SAV (Spatial Audio Virtualization and Gamification for Hearing Assessment and Enhancement) project addresses a crucial gap in current auditory testing and training approaches. Traditional hearing assessments often fail to capture the complex, real-world listening scenarios that people face daily, such as conversing in busy environments or understanding speech with multiple background voices. These limitations mean that hearing assessments do not always reflect the true challenges individuals encounter, leading to incomplete or inadequate support, especially for those with hearing impairments.
To overcome these challenges, SAV combines virtual reality (VR) and gamification to develop innovative, immersive tools that better simulate real-life auditory environments. By incorporating VR and game-like elements into training and testing, SAV aims to increase user engagement and enhance the learning transfer from training in the lab to everyday life. This approach not only mirrors the dynamic nature of actual hearing environments but also engages neural processes essential for speech and spatial perception.
Through collaborations among European and international institutions specializing in cognitive neuroscience, auditory modeling, and VR technology, the project is building a suite of virtual and gamified modules. These will be integrated into existing auditory assessment tools like the Portable Automated Rapid Testing (PART) app. SAV’s goal is to improve hearing assessment accuracy, deepen our understanding of auditory perception mechanisms, and enhance applications that can significantly benefit those with hearing challenges.
The SAV project’s potential impact spans health, cognitive neuroscience, and technological innovation, creating tools that not only address individual hearing needs but also set new standards for assessment and training protocols in clinical and everyday settings.

Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far

During Years 1 and 2, SAV consortium has performed activities that include the development, testing, and refinement of VR-based auditory environments and gamified training modules.
Progress has been made in the gamification of training exercises, where interactive, game-like tasks are used to encourage consistent engagement and enhance auditory skill retention. Research teams have examined the impact of these VR and gamified environments on participants’ spatial and speech perception abilities, using psychophysical and neuroimaging studies to map the neural mechanisms involved.
Early-phase experimental studies have focused on understanding the factors essential for creating a convincing perception of “sound externalization”—the sensation of sound originating outside the head rather than from within, which is crucial for realistic auditory simulations.
The project has successfully tested prototypes of VR/gamified/externalization environments with groups of normal-hearing and hearing-impaired participants, with promising initial results on the efficacy of VR-based auditory training. As a next step, results will inform adjustments to further refine the realism and user experience of the modules, with upcoming tests planned to evaluate the tools’ effectiveness across a broader demographic range.

Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far)

The SAV project advances auditory assessment and training by achieving a level of ecological validity and user engagement previously unaddressed in standard auditory tools. By integrating advanced VR and gamification techniques, SAV creates tools that accurately simulate real-world listening environments and motivate users to improve their auditory skills.
One of SAV’s key innovations is its focus on achieving “sound externalization,” a quality that ensures sounds are perceived as originating from external sources, mirroring how we naturally hear in real environments. This approach allows for a more immersive and realistic testing experience and promotes better transfer of skills from training environments to daily life. Additionally, SAV’s gamified modules tap into motivational and cognitive mechanisms that enhance the learning experience, leveraging game-based rewards to maintain user engagement and stimulate brain areas related to learning and attention.
As the project progresses, SAV aims to refine these innovations and make them accessible for widespread use, potentially leading to significant advancements in hearing aid fitting, auditory therapy, and clinical assessments. The outcomes will support the development of robust, next-generation tools that are applicable not only to individuals with hearing impairments but also to broader audiences seeking cognitive and perceptual enhancement.