Learning through audio/visual perception
Two European and 2 US institutions have collaborated in the project 'Perceptual, contextual, and cross-modal learning in hearing and vision' (LEARN 2 HEAR & SEE). Researchers conducted behavioural experiments, imaging studies and computational modelling to understand how humans perceive objects and distance using hearing and vision.Multiple experiments and modelling studies were carried out using auditory stimuli to judge distance and perceive objects. The researchers observed better room-specific auditory distance perception in listeners over several days, particularly, when they focused on sound reflections. Easy association between auditory stimuli and room characteristics made their learning process even faster. Scientists successfully determined the areas of the brain involved in distance processing from auditory sources using functional magnetic resonance imaging (fMRI) and virtual acoustics techniques. Modelling studies showed that people judge distance in rooms using room reflection as individual cues rather than binaural cues (i.e. information perceived from both ears). This suggests that people switch between individual cues for spatial perception when moving from room to room. Experiments using visual stimuli revealed that participants quickly but unconsciously learn to develop expectations from visual stimuli. As a result, learning and perception is faster. Unfortunately, when presented with other visual stimuli, biases and hallucinations occurred as people incorrectly perceived these stimuli to be more similar to the expected stimulus. Other studies revealed that people develop long-term expectations based on lifetime exposure to visual stimuli that can lead to biases and hallucinations. However, these expectations can be changed through a series of experimental training sessions. Simulations revealed that participants have a limited capacity to learn from different sets of visual stimuli with complex statistics.Project activities have highlighted how we process and perceive information obtained through auditory and visual stimuli. These results can be used to develop innovative sensory prosthetic devices and virtual reality technologies to help rehabilitate the blind and deaf.
Keywords
Auditory, visual, modelling, functional magnetic resonance imaging, statistics, virtual acoustic, perception, stimuli, bias, auditory cortex
