We performed research on the signal processing considerations regarding the capturing and reproduction signal paths optimized for the best listener experience, in order to achieve the most “faithful” resynthesis of the initial soundscape, placing importance on deriving methodologies which can deal effectively with the challenges of crowded environments (non-measurable number of sound sources, wideband noise-type sounds like applause, emphasis on ambience rather than foreground). We placed emphasis on amplitude panning functions, which are typically employed when rendering directional sound sources using multiple loudspeakers, and the question to investigate has been how the panning effect can be taken specifically into consideration in this “unified” signal processing analysis/synthesis of the soundscape. We concentrated initially on sports events, and especially football as the main use-case for our project.
The panning functions produce an identity mapping between the incident and perceived direction, e.g. the system recreates a virtual acoustic environment which is, to some degree, identical with the original sound field at the location of capturing. In our work, the most innovative point has been that we considered on a non-identity mapping between the incident and the perceived direction, so that the listener is “placed” at the center of the stadium, even though the capturing device is located far away from that center. Indeed, placing the array at the center of the stadium would be impossible in most cases without disturbing the players inside the field. Moreover, the examined formulation allows a flexible segmentation of the acoustic environment so that different arrays may be used in order to capture different segments of the stadium geometry. During the reporting period, we derived beamforming weights that can achieve the directivity patterns that have the desired effect. We also performed a thorough calibration procedure in order to identify potential sources of mismatch that might degrade the overall performance, including the microphone device (microphone array), the loudspeaker array, and the soundcard to be employed in the listening tests.
The project expected impact is the enhancement of the career of the Researcher, as well as the derivation of technological achievements that can improve Europe’s position in the particular area of the project. Regarding the Fellow, this period has been a great exposure to the industrial landscape of a dynamic SME, while the research he performed in this period formed the basis for important results that can be obtained in future research. At the same time, the research foundations laid in this period can form a basis for important research outcomes in the area especially of sport events broadcasting, which is currently under great technological advancements in terms of rendering quality and is open to such innovations that can enhance even more the quality that the viewers enjoy.
