Periodic Reporting for period 1 - VisionAtSecondGlance (Vision at a second glance - how memories interact with and depend on information processing in the human visual cortex)
Reporting period: 2017-09-01 to 2019-08-31
The second aspect of the research program relates to the phenomenon that we are better able to remember images when we actively process the meaning of the image – a mental process referred to as ‘deep encoding’. The ambition of this program is to unravel the so far unknown neural mechanisms of deep encoding and thereby shed light on how the integration of sensory information into existing knowledge aids the formation of long-lasting memories. This insight will lead to a better understanding of the fundamental processes underlying the formation of memories and could inspire the development of new techniques for boosting human memory function.
As a first step towards revealing the neural mechanisms of deep encoding of images, we have developed a method for measuring how similar images of objects are in terms of their meaning. This we have realized by determining how often words describing the depicted objects co-occur inside large text corpora (e.g. all text on Wikipedia). This enables us to assess the extent to which brain response patterns encode the meaning of the images we are looking at and to determine if this measure of deep encoding is enhanced for subsequently remembered images.
In addition, we have developed a behavioral paradigm that can efficiently measure how important thousands of low-level image features are for recognizing an image (e.g. recognizing images as depicting a cat or a dog, biorxiv, Alink & Charest, 2018). Intriguingly, the data recorded with this method revealed that individuals with a greater number of autistic traits rely more on fine-grained image details, which provides evidence for the idea that autism is related to an enhanced ‘eye for detail’.
In addition, this project has resulted in the development of two novel psychophysical techniques: one for quantifying the semantic relationship between images and the feature diagnosticity mapping technique (Alink & Charest, 2018) which enables one to reveal the low-level features most critically important for image recognition. The latter technique has provided the clinically relevant insight that neurotypical individuals with a greater number of autistic traits rely more on visual details when recognizing images – which aids our basic understanding of the autism spectrum disorder and could potentially inspire novel treatment strategies in the future.