Periodic Reporting for period 2 - VRACE (VRACE - Virtual Reality Audio for Cyber Environments)
Período documentado: 2021-03-01 hasta 2023-08-31
The ITN project "VRACE – Virtual Reality Audio for Cyber Environments" establishes a multidisciplinary network that will train the next generation of researchers in the audio part of virtual and augmented reality. An important field of research is the audio part. The importance of sound becomes evident when considering how people orientate themselves in space. Unlike seeing, hearing allows us to perceive instantly from all angles, and plays a leading role in giving us clues where to look at. To support natural orientation in VR, the visual and auditory information has to closely match, as otherwise the illusion is shattered and the experience is not convincing.
With an estimated revenue of $80bn by 2025 ("Virtual & Augmented Reality: Understanding the Race for the Next Computing Platform", Goldman Sachs, Equity Research, Jan. 13th 2016) the demand for trained VR audio experts will increase rapidly. Besides advancing methodologies in this cutting-edge technology, VRACE will train 15 Early Stage Researchers who will multiply and spread this knowledge in industry and academia. VRACE thus gives European industry a competitive edge in this global race.
The main objective of this Training Network is to raise VR to a next level beyond gaming and entertainment by benefiting from the critical mass of expertise gathered in this distinguished consortium. Driven and directed by the industrial partners, the consortium strives to make significant progress on the way towards a physically correct virtual reality and thus towards a true world simulation e.g. empowering engineers to enter VR for virtual testing and engineering. Although this network aims at sound experience design, there will be intermediate simulation results having some impact on the visual part of VR. In order to study sound generation, physical modelling of dynamic systems with moving parts and non-stationary fluid motion will be performed. The predicted motion of solid bodies and the dynamic deformation of objects will serve as the source of radiated sound, but simultaneously requires visual rendering in order to become visible and make VR physically correct.
Due to the interdisciplinary and intersectoral composition of the network, different partners have different but complementing requirements depending on their interests and their fields of expertise. Musicians need convincing sound simulations, room acousticians need tools for realistic room simulations, engineers dealing with Virtual Acoustics are interested in improving the computational efficiency of the whole audio rendering process. Especially for audio rendering it is essential to take human perception into account. This will ensure that no processing power is being wasted by rendering acoustic details which cannot be heard. Furthermore, perception modelling and psychoacoustic experiments may give clues on how sounds contribute to deepen immersion and make virtual environments more convincing.
Within the second reporting period, research intensified, resulting in advances in the following areas:
- Sound source and sound radiation modelling and characterisation, including sound-source separation;
- Room acoustics and outdoors wave propagation modelling and auralisation, including perceptual studies;
- Measurement and modelling of Head Related Transfer Functions and ear canals, and binaural rendering;
- Perception of complex sound sources, audio-visual latency tolerances and transfer plausibility.
Another five training workshops took place, including a public tutorial course and two VRACE special sessions in academic conferences.
20 Journal papers and 10 Open Datasets have been published and more are planned. Research results have been presented in numerous scientific conferences of various fields related to Virtual Reality, Audio Engineering and Acoustics. Exploitable results include code for sound separation, binaural rendering, sound source modelling and wave propagation, HRTF models (measured and simulated) and room impulse response measurements.
Involvement in the European Acoustics Association (EAA) has aided towards structuring existing training capacities for European PhD-level training, with available courses listed in the EAA website. This raises the quality of training programs due to international competition and comparability of training courses, and due to the scientific exchange and increasing international mobility. Discussions for providing an institutional framework for basic research and exchange of know-how has also been initiated.