Periodic Reporting for period 1 - MemoryBasedPercepts (Filling-in the blank: Perceiving through the lens of prior knowledge.)
Reporting period: 2022-03-01 to 2024-02-29
In this project we aimed to understanding how human perception is supported by one’s memory and prior knowledge of the world. Our research relates to an overarching paradox for vision scientists. That is, people subjectively report a rich visual experience, but empirical findings prove that they rather see very little of what is in view. In other words, people feel as if they can see everything that is presented to their eyes. Yet, several cognitive science studies provide striking demonstrations that people can perceive at once much less than they think, due to capacity limits of the visual information in input (e.g. peripheral vision, inattentional blindness). One possible solution for solving the incongruency between the empirical findings and the subjective reports is to posit that our perceptual operations are constantly supported by and integrated with memories, knowledge, and expectation that people have about the world. This project aimed to perform systematic tests on memory-based perception, with the goal of revealing the how of knowledge effects on visual processing. In our experiments, we used as visual stimuli realistic scenes and common objects, capitalizing on the fact that these are highly familiar to the observers, and therefore can evoke the prior knowledge that people have about them. We expected such knowledge to be integrated with visual operations in the brain, and specifically that visual brain regions would be sensitive to the integration between knowledge and perception. The possibility that human vision represents only partially the processing of image driven physical features and that is rather an inferential process based on our subjective knowledge of the world has implications for understanding the nature of the phenomenology of human experiences and a direct link with other disciplines such as philosophy, psychiatry, and aesthetics.
We conducted experiments on the topic of memory-based perception using a wide range of neurocognitive methods including laboratory studies, large-scale online studies, functional magnetic resonance imaging, and brain stimulation studies. Generally, we were able to show that people’s perception of complex visual stimuli (such as scenes, and day-to-day objects) capitalizes on prior knowledge. For example, we show that 1) if participants have a strong memory of what object should be appearing in a scene, then they will be able to see these objects as sharper, even if heavily blurred. 2) When participants have expectations of what might be found beyond the boundaries of a picture, then they will tend to remember this picture with additional information on top of what they previously saw, in line with their expectations. 3) A sharp object in a picture, its positioning and its appearance, can help understanding whether a surrounding blurry background is an indoor or an outdoor scene, thus supporting the construction of a sharper background. 4) When people see objects appearing in a scene in a position coherent to what they expect given a scene’s distance (e.g. a large object in the foreground) they will perform better at recognizing that object. 5) The perception of highly familiar objects, such as other people, is strongly influenced by the knowledge and expectations we have about them. Indeed, the perception of a lifted arm is biased by our knowledge of gravity (a lifted arm should fall down), and by our knowledge of what movements a lifted arm can perform (an arm cannot go all the way behind the back). Crucially, using functional magnetic resonance imaging, we show that visual brain regions contribute to knowledge effects on perception. Further, with brain stimulation, we confirm that the activity of these visual brain regions is causally necessary, thus directly implicated in determining such memory-based effects on perceptual operations.
Our results have been presented to national (the Dutch Society for Brain and Cognition winter conference) and international conferences (Vision Science Society Meeting, US; European Conference for Visual Perception; Concepts, Actions, Objects meeting) and have been partly published in Scientific Journals (e.g. Psychological Science, iScience, Current Biology). Other results are currently in preparation for publication. In addition, we exploited the knowledge generated from this project by establishing a collaboration with a local museum on visual impairment. Here, we arranged for the museum visitors (~20000 people so far), a demonstration on inattentional blindness. In this demonstration, we could directly show to the visitors (by making them participate in a short user-friendly experiment) that in absence of expectations (i.e. of prior knowledge) we may end up missing highly visible objects appearing in front of our eyes. Interestingly, we also let them find out that highly familiar and relevant objects, such as other people, may be missed less often, because our visual system is sensitive to these important stimuli even in absence of attention.
Our results have been presented to national (the Dutch Society for Brain and Cognition winter conference) and international conferences (Vision Science Society Meeting, US; European Conference for Visual Perception; Concepts, Actions, Objects meeting) and have been partly published in Scientific Journals (e.g. Psychological Science, iScience, Current Biology). Other results are currently in preparation for publication. In addition, we exploited the knowledge generated from this project by establishing a collaboration with a local museum on visual impairment. Here, we arranged for the museum visitors (~20000 people so far), a demonstration on inattentional blindness. In this demonstration, we could directly show to the visitors (by making them participate in a short user-friendly experiment) that in absence of expectations (i.e. of prior knowledge) we may end up missing highly visible objects appearing in front of our eyes. Interestingly, we also let them find out that highly familiar and relevant objects, such as other people, may be missed less often, because our visual system is sensitive to these important stimuli even in absence of attention.
Our results progress our understanding of the how of memory-based perception and crucially demonstrate that our perception is not a direct copy of the external world, but rather the result of an inference that the observer performs on the external visual input based on their knowledge. Crucially, we reveal that visual brain regions are directly involved in performing such inference, bringing strong evidence for a view of the visual system being directly involved in cognitive operations. These results have implications for a better understanding of conditions where this inference mechanism is out of balance (e.g. in patients with visual hallucinations, or people without imagery), and may contribute to improving performance of artificial visual systems, as used in self-driving cars, or automated medical screening procedures.