Final Report Summary - CEMNET (A unified framework for Perceptual Inference in Sensory cortices)
The neural mechanisms that underlie perceptual inference in cortex are a matter of intense research. Neural models implementing Bayesian models have proved useful to uncover neural computations in a variety of paradigms. Still no existing neuro-computational model provides a general framework for understanding perception in naturalistic conditions. Indeed these models suffer from two major shortcomings: i) they cannot perform accurate inference in naturalistic environments where numerous objects interact in complex fashion; ii) they cannot deal with correlated sensory evidence, though such correlations are ubiquitous in sensory systems.
We have developed a unifying model of perception called Constrained Entropy Maximization Network (CEMNet) that provides a theoretical framework for inference in complex naturalistic environments. CEMNet stores an internal model of the environment by representing regularities across stochastic variables as constraints; those constraints shape the response of the network. The project had both a theoretical and an experimental component.
From the theoretical side, we have developed a computational model of CEMNet and studied how it behaves in various perceptual environments. Implementation relies on a biologically plausible neural architecture. It is composed of neural units coding for the presence of features in the environment (marginal probabilities) as well as their conjunction (factor probabilities). Those probabilities are constantly updated through the presence of constraint units that enable a gradient descent over the maximum likelihood model: as sensory evidence is presented, activity spreads within the network and stabilizes when the most likely features of the environment causing observed sensation is found. We simulated the network and showed that, unlike existing neural models, CEMNet can deal with the difficulties of inference in complex environments. Plasticity rules have also been derived that explain how connections between neural units should evolve to learn internal models of the environment.
From the experimental side, we have collected evidence that humans take into account correlations in the structure of sensory signal in perceptual integration. The task required human participants to judge as fast and accurately as possible the dominant orientation of a rapid stream of visual patterns. Crucially, in some blocks we introduced correlations between successive samples. Sophisticated machine learning techniques (including expectation-maximization) allow to infer the impact of each sample onto subject response as well as the time at which participants reach their decision. Using such techniques, we found that subjects adapted to the correlation structure. These results are in line with the CEMNet framework, where regularities at different levels (correlation between successive samples and link with overall category) coexist and guide the perceptual process. They are however at odds with classical models of perceptual integration where all samples are integrated equally. This piece of evidence is key as the classical assumption behind all those models that sensory stimuli are conditionally independent is unlikely to be met in most ecological conditions.
We have also investigated how perceptual decisions are taken when it organizes as a hierarchy of subdecisions. A recent study concluded that perceptual processes in primates mimic this hierarchical structure and perform subdecisions in parallel. Through simulations, we showed that a flat model that directly selects between final choices accounts more parsimoniously for the reported behavioral and neural data. Our results point to the role of experience for building integrated perceptual categories where sensory evidence is merged prior to decision. These conclusions are in accordance with the CEMNet framework where representations are learned that bind together sensory stimuli and their common underlying cause.
Overall our theoretical and experimental work converge to establish CEMnet as a potential unifying framework for understanding perceptual inference. It may open new ways to capture how perception is distorted in illusory events for both healthy and pathological (notably schizophrenic) populations.
We have developed a unifying model of perception called Constrained Entropy Maximization Network (CEMNet) that provides a theoretical framework for inference in complex naturalistic environments. CEMNet stores an internal model of the environment by representing regularities across stochastic variables as constraints; those constraints shape the response of the network. The project had both a theoretical and an experimental component.
From the theoretical side, we have developed a computational model of CEMNet and studied how it behaves in various perceptual environments. Implementation relies on a biologically plausible neural architecture. It is composed of neural units coding for the presence of features in the environment (marginal probabilities) as well as their conjunction (factor probabilities). Those probabilities are constantly updated through the presence of constraint units that enable a gradient descent over the maximum likelihood model: as sensory evidence is presented, activity spreads within the network and stabilizes when the most likely features of the environment causing observed sensation is found. We simulated the network and showed that, unlike existing neural models, CEMNet can deal with the difficulties of inference in complex environments. Plasticity rules have also been derived that explain how connections between neural units should evolve to learn internal models of the environment.
From the experimental side, we have collected evidence that humans take into account correlations in the structure of sensory signal in perceptual integration. The task required human participants to judge as fast and accurately as possible the dominant orientation of a rapid stream of visual patterns. Crucially, in some blocks we introduced correlations between successive samples. Sophisticated machine learning techniques (including expectation-maximization) allow to infer the impact of each sample onto subject response as well as the time at which participants reach their decision. Using such techniques, we found that subjects adapted to the correlation structure. These results are in line with the CEMNet framework, where regularities at different levels (correlation between successive samples and link with overall category) coexist and guide the perceptual process. They are however at odds with classical models of perceptual integration where all samples are integrated equally. This piece of evidence is key as the classical assumption behind all those models that sensory stimuli are conditionally independent is unlikely to be met in most ecological conditions.
We have also investigated how perceptual decisions are taken when it organizes as a hierarchy of subdecisions. A recent study concluded that perceptual processes in primates mimic this hierarchical structure and perform subdecisions in parallel. Through simulations, we showed that a flat model that directly selects between final choices accounts more parsimoniously for the reported behavioral and neural data. Our results point to the role of experience for building integrated perceptual categories where sensory evidence is merged prior to decision. These conclusions are in accordance with the CEMNet framework where representations are learned that bind together sensory stimuli and their common underlying cause.
Overall our theoretical and experimental work converge to establish CEMnet as a potential unifying framework for understanding perceptual inference. It may open new ways to capture how perception is distorted in illusory events for both healthy and pathological (notably schizophrenic) populations.
