Final Report Summary - COLUMNARCODECRACKING (Cracking the columnar-level code in the visual hierarchy: Ultra high-field functional MRI, neuro-cognitive modelling and high-resolution brain-computer interfaces)
Cortical columns had been originally discovered in the somatosensory cortex of cats by Mountcastle in 1957 and later in the visual cortex of monkeys by Nobel prize winners Hubel and Wiesel. They found that neurons within small cortical clusters running from the top to the bottom of cortex respond to the same stimulus property, i.e. they represent the same basic feature such as the orientation of a bar in primary visual cortex. Despite the larger size of columns - when compared to the micrometer size of neurons - cortical columns were for a long time too small to be reliably measured with non-invasive functional brain imaging in the human brain since the spatial resolution of standard functional magnetic resonance imaging (fMRI) using magnetic field strengths of 1.5 – 4 Tesla is in the order of a few millimeters. Cortical columnar organisations within brain areas were thus measured for many decades only in animals using electrical recordings or invasive optical imaging. Only with the advent of ultra-high field (UHF) MRI scanners (7 Tesla and higher) combined with optimized hard and software tools, the measurement of cortical columns became convincingly possible in the human brain. The ColumnarCodeCracking project has substantially advanced this challenging undertaking by pushing the limits of technology and by developing new paradigms, analysis methods and modeling tools. With this progress it is now possible to almost routinely scan specific brain regions with a resolution of at least 0.8 mm in all three spatial dimensions and to make sense of the obtained high-resolution fMRI data.
Besides technological and methodological advancements, our work has led to a deeper understanding of how the brain and mind work by zooming into the fine-grained functional organisation within specialised brain areas. We have, for example, revealed for the first time columnar-level direction-of-motion and disparity maps in human visual cortical area hMT that help to understand how we see objects moving in specific directions and how we see the world around us in 3D. In collaborative studies, we have also seen for the first time columnar maps of sound frequencies in the human auditory cortex and how top-down expectations about the visual world can be read out from upper, but not lower, layers of primary visual cortex.
One of the most exciting results of the project shows that observed activity changes in columnar-level features can be used to predict the content of conscious experiences. In this study we used ambiguous motion stimuli that spontaneously switch between two clearly separate perceptual states reported as motion in either horizontal or vertical direction. We observed that the amplitude of direction-of-motion selective clusters in area hMT dynamically reflect the perceived direction of motion as indicated by button presses, i.e. when the subject perceived an ambiguous stimulus as moving in horizontal direction, the identified "horizontal" columnar-like features showed increased activity whereas "vertical" columnar clusters showed increased activity when the participant perceived the stimulus as moving in the vertical direction.
Another result may pave the way for new brain computer interfaces (BCIs) that could help selected patients in the future suffering from severe motor impairments being even unable to speak. Our 7 Tesla study demonstrated that imagined letter shapes can be literally read-out as 2D images from self-generated brain activity in early visual cortex providing also new insights in the operation of the "mind's eye". This work may lead to BCIs that could enable locked-in patients to communicate with their relatives by simply imagining letters of the alphabet.
Besides technological and methodological advancements, our work has led to a deeper understanding of how the brain and mind work by zooming into the fine-grained functional organisation within specialised brain areas. We have, for example, revealed for the first time columnar-level direction-of-motion and disparity maps in human visual cortical area hMT that help to understand how we see objects moving in specific directions and how we see the world around us in 3D. In collaborative studies, we have also seen for the first time columnar maps of sound frequencies in the human auditory cortex and how top-down expectations about the visual world can be read out from upper, but not lower, layers of primary visual cortex.
One of the most exciting results of the project shows that observed activity changes in columnar-level features can be used to predict the content of conscious experiences. In this study we used ambiguous motion stimuli that spontaneously switch between two clearly separate perceptual states reported as motion in either horizontal or vertical direction. We observed that the amplitude of direction-of-motion selective clusters in area hMT dynamically reflect the perceived direction of motion as indicated by button presses, i.e. when the subject perceived an ambiguous stimulus as moving in horizontal direction, the identified "horizontal" columnar-like features showed increased activity whereas "vertical" columnar clusters showed increased activity when the participant perceived the stimulus as moving in the vertical direction.
Another result may pave the way for new brain computer interfaces (BCIs) that could help selected patients in the future suffering from severe motor impairments being even unable to speak. Our 7 Tesla study demonstrated that imagined letter shapes can be literally read-out as 2D images from self-generated brain activity in early visual cortex providing also new insights in the operation of the "mind's eye". This work may lead to BCIs that could enable locked-in patients to communicate with their relatives by simply imagining letters of the alphabet.