Final Report Summary - EMCOREP (Emergence of complex internal representations in humans)
The EMCOREP project investigates the behavioural and computational foundations of human sensory and cognitive learning in adults and children through three aims. The problem at hand is that while the human brain easily performs the most sophisticated computations including recognition and identification of hundreds of thousands of items from a small fraction of information, logical thinking, or solving problems by generalising any knowledge to new situations, we have no idea about the computational basis of these amazing behaviours nor about their the actual implementation in the brain. Being able to answer these puzzles would unleash an unprecedented ability of creating intelligent equipments and restructure the division of labor between humans and machines by freeing up the capacity of humans from low-level mechanistic work.
While this goal is grandiose, the road toward it can be travelled in tiny steps, and EMCOREP contributes to this journey by focusing on three small but fundamental issues. First, we investigated what kind of computational framework might capture the best the functioning of the brain. Specifically, we had proposed (following some earlier ideas) that cortical processes can be best captured by a probabilistic framework, which assumes that the brain computes with different possible interpretations of the incoming information by weighting these interpretations by their relative “plausibility”. This strategy leads, especially in complex situations, to an optimally efficient behaviour, but it also raises several questions of implementation. Our first aim was to gather evidence through behavioural studies and modelling that, nevertheless, this probabilistic framework is likely to be implemented in the brain. To this end we conducted four experiments demonstrating the manifestation of different aspects of probabilistic behaviour. We showed that humans handle well and differently various types of statistics (single element, joint, and conditional probabilities), that in case of a visual input, they automatically integrate internally stored knowledge to perform “unconscious inference” on the input in order to arrive to a 3-dimensional interpretation of the scene, that the way they learn unknown information follows the path of probabilistic learning rather than simple associative learning, and that they, indeed combine their knowledge according to the optimal probabilistic method. These four studies have been written up for scientific publications, and they provide a univocal support for the probabilistic framework.
To dive into the details of this issue, we also started a fifth line of investigations asking how fundamental is this type of behaviour in the cortex. In principle, it is possible that only the high cognitive aspects of brain functioning follow the probabilistic route and low level sensory processing is hardwired. However, we found that even at the lowest level of visual processing, the rules of probabilistic computation seems to prevail suggesting that the brain implements probabilistic computation from the very first step of sensory information processing. These results were already published partially (Christiansen et al. Journal of Vision 2016), and two further publications are on their way.
The second issue we targeted in EMCOREP following two lines of research is the sufficiency of the probabilistic framework to describe internal representations and their effect in the brain. In the first line, we conducted an experiment in which we explored the hypothesis that as the brain encodes visual information of multi-shape scenes, it will not only represent shape groups as single chunks if the elements of the group appear together statistically, but also these chunks rather than the individual elements will be the units that determine the capacity of the brain, i.e. its limit of how many pieces of information it can hold in memory. This study is under revision for journal publication. In the second line of research, we investigated the effect of this internal representation by looking into how sequentially carried out perceptual decision processes are influenced by previous decisions, that is how prior biases emerge. We found that such strong biases, indeed, emerge, but instead simply following the rule of gradually collected event statistics, perceptual decision making is influenced by these biases through a complex process in which statistics are weighted by significance due to detected changes in the environmental status quo. This work has been completed and presently being written up for journal publication. In the third line, we combined the probabilistic framework with a particular type of representational scheme called sampling-based representation. Our focus was to find evidence that the sampling based representation is a valid biological implementation of the probabilistic framework we propose. By developing a model based on these principles and comparing the behaviour of the model to a large amount of previously published results, we could find a strong correlation between the prediction of the model and cell behaviours in the literature. This confirmed our suggestion that such sampling based computational schemes might be in place i the cortex. This work has been just published int he journal Neuron. In ta separate study, we asked whether this sampling based representation can be put into a hierarchical model to test it against more complex tasks. We developed a hierarchical probabilistic model of visual processing and showed how the behaves of this model can explain neurophysiological findings previously attributed to attentional effects. This work has also been just published recently int he journal Neuron.
Finally, the third issue concerned was the time frame and the context within which this type of probabilistic representation develops. First, we investigated the correlation between abilities to learn and abilities to perform some other high-level cognitive task and found that such a correlation exists. Second, we used simple categorisation task that the subjects had to perform and monitored the P300 and alpha activity during the formulation of the categories. We found that the amplitude of the alpha activity reliably tracked the quality of the formulated categories. In the third study, we started another experiment with children to assess how much adult type of sensory information processing can be found in young children. Our hypothesis is that there is a very prolonged developmental phase, during which childrens’ and adults’ information processing is very different. This naturally raises the question whether there is any systematically across different representations in this type of deviation, that we are in the process of answering.
Together these investigations made a clear step toward clarifying the status of the probabilistic framework in explaining complex cortical functions.
While this goal is grandiose, the road toward it can be travelled in tiny steps, and EMCOREP contributes to this journey by focusing on three small but fundamental issues. First, we investigated what kind of computational framework might capture the best the functioning of the brain. Specifically, we had proposed (following some earlier ideas) that cortical processes can be best captured by a probabilistic framework, which assumes that the brain computes with different possible interpretations of the incoming information by weighting these interpretations by their relative “plausibility”. This strategy leads, especially in complex situations, to an optimally efficient behaviour, but it also raises several questions of implementation. Our first aim was to gather evidence through behavioural studies and modelling that, nevertheless, this probabilistic framework is likely to be implemented in the brain. To this end we conducted four experiments demonstrating the manifestation of different aspects of probabilistic behaviour. We showed that humans handle well and differently various types of statistics (single element, joint, and conditional probabilities), that in case of a visual input, they automatically integrate internally stored knowledge to perform “unconscious inference” on the input in order to arrive to a 3-dimensional interpretation of the scene, that the way they learn unknown information follows the path of probabilistic learning rather than simple associative learning, and that they, indeed combine their knowledge according to the optimal probabilistic method. These four studies have been written up for scientific publications, and they provide a univocal support for the probabilistic framework.
To dive into the details of this issue, we also started a fifth line of investigations asking how fundamental is this type of behaviour in the cortex. In principle, it is possible that only the high cognitive aspects of brain functioning follow the probabilistic route and low level sensory processing is hardwired. However, we found that even at the lowest level of visual processing, the rules of probabilistic computation seems to prevail suggesting that the brain implements probabilistic computation from the very first step of sensory information processing. These results were already published partially (Christiansen et al. Journal of Vision 2016), and two further publications are on their way.
The second issue we targeted in EMCOREP following two lines of research is the sufficiency of the probabilistic framework to describe internal representations and their effect in the brain. In the first line, we conducted an experiment in which we explored the hypothesis that as the brain encodes visual information of multi-shape scenes, it will not only represent shape groups as single chunks if the elements of the group appear together statistically, but also these chunks rather than the individual elements will be the units that determine the capacity of the brain, i.e. its limit of how many pieces of information it can hold in memory. This study is under revision for journal publication. In the second line of research, we investigated the effect of this internal representation by looking into how sequentially carried out perceptual decision processes are influenced by previous decisions, that is how prior biases emerge. We found that such strong biases, indeed, emerge, but instead simply following the rule of gradually collected event statistics, perceptual decision making is influenced by these biases through a complex process in which statistics are weighted by significance due to detected changes in the environmental status quo. This work has been completed and presently being written up for journal publication. In the third line, we combined the probabilistic framework with a particular type of representational scheme called sampling-based representation. Our focus was to find evidence that the sampling based representation is a valid biological implementation of the probabilistic framework we propose. By developing a model based on these principles and comparing the behaviour of the model to a large amount of previously published results, we could find a strong correlation between the prediction of the model and cell behaviours in the literature. This confirmed our suggestion that such sampling based computational schemes might be in place i the cortex. This work has been just published int he journal Neuron. In ta separate study, we asked whether this sampling based representation can be put into a hierarchical model to test it against more complex tasks. We developed a hierarchical probabilistic model of visual processing and showed how the behaves of this model can explain neurophysiological findings previously attributed to attentional effects. This work has also been just published recently int he journal Neuron.
Finally, the third issue concerned was the time frame and the context within which this type of probabilistic representation develops. First, we investigated the correlation between abilities to learn and abilities to perform some other high-level cognitive task and found that such a correlation exists. Second, we used simple categorisation task that the subjects had to perform and monitored the P300 and alpha activity during the formulation of the categories. We found that the amplitude of the alpha activity reliably tracked the quality of the formulated categories. In the third study, we started another experiment with children to assess how much adult type of sensory information processing can be found in young children. Our hypothesis is that there is a very prolonged developmental phase, during which childrens’ and adults’ information processing is very different. This naturally raises the question whether there is any systematically across different representations in this type of deviation, that we are in the process of answering.
Together these investigations made a clear step toward clarifying the status of the probabilistic framework in explaining complex cortical functions.