Final Report Summary - PHDVIRTA (Physically-based Virtual Acoustics)
My research plan (Annex I) had three main tasks:
1. Physically-based room acoustics modeling and auralization
2. Analysis of concert hall acoustics based on directional audio coding and sensory evaluation
3. Augmented reality audio
Room acoustics modeling research concentrated on searching novel and accurate modeling methods for room acoustics. Modeling of reflections from rough and scattering surfaces were studied and novel geometric acoustics modeling methods were found. Moreover, case studies on the acoustics of ancient theaters, the seat-dip effect, and sound reflection on layered wall structures were done with a wave-based modeling technique. Currently, we are working on the hybrid modeling method in which acoustics radiance transfer method is complemented with a FDTD method. In auralization research, we have collected the essential data, such as anechoic symphony orchestra recordings and directivity of musical instruments. Moreover, we have developed novel methods to record sound in a concert hall and reproduce the recordings authentically with spatial sound reproduction in the laboratory environment. The recordings are done via spatial sound impulse responses, which are processed with the invented Spatial Decomposition Method before convolution with anechoic signals.
The main scientific impact has been achieved in studies on concert hall acoustics. We invented the loudspeaker orchestra that simulates a symphony orchestra with 34 loudspeakers. With such a constant sound source we were able to record concert hall acoustics in a way that simultaneous comparison of different halls has been possible in the laboratory conditions. In addition, the measurement system makes possible detailed analysis of spatial impulse responses, which is essential to understand how sound propagates in a concert hall. For example, we have invented time-frequency and spatiotemporal methods to visualize cumulative sound energy distribution. Such visualizations layered with plan and section drawings of the hall enable to see which surfaces reflect the sound and contribute to the total captured sound. The loudspeaker orchestra recordings in real halls led to a unique possibility for subjective evaluation of concert hall acoustics. In subjective listening tests we have used sensory evaluation methods that are originally developed in food and wine industry. Our subjective evaluations have shown that the approach is very sound and reveals information about perceptual issues that were out of the reach with earlier methods. There are still many issues to be solved in near future, but the methodology created in this project will pave the way for both subjective and objective studies to finally get all-encompassing understanding of concert hall acoustics.
Augmented reality audio research has been divided into two research lines. Firstly, we invented several methods to use gestures to control the menu structures, presented with spatial audio, in mobile devices. These innovations led us to study more carefully eyes-free user interaction of mobile devices. This development led to a start-up company Blaast Ltd., founded in June 2010. Secondly we have been concentrating on audio augmented reality applications, such as an audiomemo for outdoor use, teleconferencing, mobile audio messaging, and automatic co-location of teleconference participants.
1. Physically-based room acoustics modeling and auralization
2. Analysis of concert hall acoustics based on directional audio coding and sensory evaluation
3. Augmented reality audio
Room acoustics modeling research concentrated on searching novel and accurate modeling methods for room acoustics. Modeling of reflections from rough and scattering surfaces were studied and novel geometric acoustics modeling methods were found. Moreover, case studies on the acoustics of ancient theaters, the seat-dip effect, and sound reflection on layered wall structures were done with a wave-based modeling technique. Currently, we are working on the hybrid modeling method in which acoustics radiance transfer method is complemented with a FDTD method. In auralization research, we have collected the essential data, such as anechoic symphony orchestra recordings and directivity of musical instruments. Moreover, we have developed novel methods to record sound in a concert hall and reproduce the recordings authentically with spatial sound reproduction in the laboratory environment. The recordings are done via spatial sound impulse responses, which are processed with the invented Spatial Decomposition Method before convolution with anechoic signals.
The main scientific impact has been achieved in studies on concert hall acoustics. We invented the loudspeaker orchestra that simulates a symphony orchestra with 34 loudspeakers. With such a constant sound source we were able to record concert hall acoustics in a way that simultaneous comparison of different halls has been possible in the laboratory conditions. In addition, the measurement system makes possible detailed analysis of spatial impulse responses, which is essential to understand how sound propagates in a concert hall. For example, we have invented time-frequency and spatiotemporal methods to visualize cumulative sound energy distribution. Such visualizations layered with plan and section drawings of the hall enable to see which surfaces reflect the sound and contribute to the total captured sound. The loudspeaker orchestra recordings in real halls led to a unique possibility for subjective evaluation of concert hall acoustics. In subjective listening tests we have used sensory evaluation methods that are originally developed in food and wine industry. Our subjective evaluations have shown that the approach is very sound and reveals information about perceptual issues that were out of the reach with earlier methods. There are still many issues to be solved in near future, but the methodology created in this project will pave the way for both subjective and objective studies to finally get all-encompassing understanding of concert hall acoustics.
Augmented reality audio research has been divided into two research lines. Firstly, we invented several methods to use gestures to control the menu structures, presented with spatial audio, in mobile devices. These innovations led us to study more carefully eyes-free user interaction of mobile devices. This development led to a start-up company Blaast Ltd., founded in June 2010. Secondly we have been concentrating on audio augmented reality applications, such as an audiomemo for outdoor use, teleconferencing, mobile audio messaging, and automatic co-location of teleconference participants.