How the brain processes and integrates different visual inputs
The fovea covers only the central 1 to 2 degrees of the visual field. Peripheral vision is much larger and is vital for orientation and navigation, which would be challenging using only the small foveal aperture. As both visual fields require very different processing, the PERFORM project, supported by the European Research Council, investigated how the brain accommodates such variations in sensory input to achieve cohesive representations of the environment. The project found that the brain does not simply discard peripheral information, when superior foveal information becomes available, but compares inputs from both, assessing their relative reliability. “This process, called transsaccadic perception, is a highly complex and little understood computation. We’ve been able to revive interest in this topic,” adds project coordinator Alexander Schütz from Philipps University Marburg, the project host. Indeed, the project results have been featured in a special issue of the ‘Journal of Vision’.
Exploring transsaccadic integration
Foveal visual processing is optimised for high acuity and colour vision, whereas peripheral processing is optimised for a large visual field, requiring less processing. This means that peripheral vision is used for navigation and situating objects but lacks certainty about detail. To compensate, the eye undertakes rapid movements, known as saccades, allowing processing to fluctuate between both visual fields. “In PERFORM, we adopted insights and mathematical models from the field of multisensory perception and applied them to visual processing, especially the integration of peripheral and foveal information – so-called transsaccadic integration,” explains Schütz. The team undertook a series of psychophysical experiments with children and adults in a lab with black walls, specialised monitors and an eye-tracking device. Subjects’ eye movements were tracked as they reacted to visual stimuli in their periphery and automatically moved their gaze to it – meaning they performed saccades. They were asked to conduct perceptual discrimination tasks including making judgments about the location, orientation or colour of stimuli. The results were compared to a computational model that predicted how participants were expected to weigh peripheral and foveal information.
Visual development and strategies
One key finding was that the precision of the visual and the eye movement system is limited in children. In one experiment, 7- to 12-year-olds performed worse than adults at detecting changes to the position of stimuli, and their eye movements were also found to be more variable. But these children corrected their errors with subsequent eye movements faster than adults, suggesting an inherent compensation for limitations. The team also showed that transsaccadic integration is limited by attention and memory, and only applies to one selected object. “It seemed to break down, or at least, become impaired when we flashed a distractor stimulus or when participants had to memorise another stimulus,” says Schütz. Another finding relates to the so-called foveal rod scotoma. Under dark lighting conditions, humans use rod photoreceptors, but as these are absent in the fovea, it isn’t clear how humans compensate. “We found that this missing information is seamlessly added from surround information without any distortion perceived at the fovea. Although this information is merely inferred, participants trust it more than actual peripheral information,” notes Schütz. This finding has led to the new project, SENCES, which investigates how the brain completes gaps in its sensory information and compares its own compensatory inferences with actual sensory inputs.
Keywords
PERFORM, vision, fovea, eyes, peripheral, saccades, photoreceptors, perception, orientation, navigation, brain
