Two main lines of studies have been implemented. The first relates to the decomposition elementary operations of perceptual inference in the human brain. The second relates to identifying the very same components in linguistic processes.
1. Decoding elementary computations of perceptual processes
To isolate the elementary computations of perceptual processes, we first recorded the brain activity of 20 healthy subjects while they performed a visibility inference task on a stimulus presented at perceptual threshold. We show that a rich amount of sensory, perceptual and decisional information can be decoded and track from MEG activity. The present study has been published in Neuron (King et al 2016), and received the postdoctoral award from the Cognitive Neuroscience Society 2016, and the William James Prize from the Association for the Scientific Study of Consciousness. The data and code is available on
https://github.com/king(s’ouvre dans une nouvelle fenêtre) partially fulfilling “WP2.3 : Datasets made publically available” and “WP1.2 : Peer-reviewed publication MEG+ECoG + Open Access codes of original analytical tools”.
Second, to identify whether the corresponding neural representations were hierarchically organized, we applied similar decoding analyses to participants’ MEG while they discriminated between ambiguous visual symbols. Our results show that a hierarchy of psychological variables can be decoded from brain activity. These computations are largely similar to those of deep neural networks subject to the same task. The present study is currently being written and has been accepted for a dynamic poster presentation at SFN and a poster presentation at Computational Cognitive Neuroscience Conference 2017 for which it received a travel award.
Finally, we investigated how this hierarchy of computations unfolded in streams of images. Fifteen subjects were presented with ~5,000 visual stimuli presented in rapid sequences while being recorded with EEG. Temporal generalization and source analyses reveal that the information contained in each stimulus is processed by a “visual pipeline”: a long cascade of transient processing stages, which can overall encode multiple stimuli at once. The corresponding study is currently under review at Science.
2. Decoding elementary computations of linguistic processes
Like perception, language can be described at many level representations, ranging from acoustic features, to phonemes, syntax and semantics. To isolate each level of representations, we developed new methods to both encode and decode linguistic representations from 24 subjects’ brain activity, while they were listened to stories. Our results show that we can isolate and track the variable maintenance of acoustic, phonetic, lexical, syntactic and semantic computations. The results suggest a predominantly hierarchical organization of elementary neural computations.
Overall, the above demonstrate that theoretically-derived computations can be isolated, and tracked in the human brain activity. Our results converge on the idea that the same computations apply at multiple level of the perceptual processing hierarchy.
3. Dissemination and exploitation
The above studies have been accepted for presentation in a number of conferences (listed in
https://sites.google.com/site/jeanremiking/(s’ouvre dans une nouvelle fenêtre)). This fulfills “WP1.1: Presentation of MEG results at conference” and “WP1.4 Conference Talk” Amongst these events, I have:
* organized and chaired 3 symposia (ICON 2017, ASSC 2017, Biomag 2016),
* been invited for plenary session at 2 conferences (ASSC 2017 ‘William James Prize’, CNS 2017: ‘Big Ideas’)
Additionally, 1) I have participated in open data-science competitions and ranked first and third, 2) I am an official core developer of MNE, an open-source package in python to analyze temporally resolved neuroimaging data and 3) I have been invited to present my research at Facebook Artificial Intelligence Research.
Finally, I have organized the Supervised Neural Time Series Coding Sprint 2017 in New York where the main developers of machine learning packages for neuroimaging setup the bases for a coherent analytical eco-system. I have also co-organized the workshop “ Interdisciplinary Metacognition & Uncertainty meeting” together with M. Wokke and S. Flemming. This fulfills “WP1.3 Organization of seminar.”.
