Periodic Reporting for period 1 - NeuPheMi (Neurophenomenology of Mental Imagery)
Reporting period: 2016-03-22 to 2018-03-21
Yet, the need for integrative studies “that bridge between low-level (e.g. neuronal, physiological) and high-level (e.g. belief, intention, identity) descriptions” and “go beyond the classical perception-action loop” to also tackle issues such as experience is clearly identified in the EU research priorities. Indeed, a better understanding of the neural correlates of mental imagery in relation to the experience felt by the subject could enhance the quality and efficacy of mental imagery-based clinical protocols and considering mental imagery as a human experience instead of a mental representation only would foster a better understanding of the nature of MI and its relation to cognition and the brain.
Electroencephalographic (EEG) activity reflects the real-time occurrence of brain events and, as such, is well suited for the study of the neural correlates of transient cognitive events such as mental images. A classical approach in EEG studies of mental imagery is to compute the average neural response elicited by a mental image (event-related potentials, ERPs). However, the oscillatory dynamic of the brain processes is lost both in studies using ERPs and in the ones using the low-time resolution functional magnetic resonnance imaging (fMRI) technique. As a result, the literature lacks of a clear picture of the brain dynamics underlying mental imagery.
The first objective of NeuPheMI is thus to study the dynamic of the brain activity during mental imagery. To do so, electroencephalography (EEG) is a technique of choice, as it has a very high temporal resolution.
A second objective is to study the phenomenological experience that accompanies the making of visual mental images. To do so we decided to employ micro-phenomenology, a novel scientific discipline allowing to explore lived experience very finely. Micro-phenomenology relies on non-inducive guided interviews, enabling the interviewee to become aware of her or his subjective experience and to describe it with great precision.
Preliminary data analyses from a first paradigm using naturalistic stimuli were not conclusive.
We created then another paradigm with well-controlled pictures (same luminance, same spatial frequencies, same overall shape) as visual stimuli on which to base the mental imagery trials on. We developed a paradigm allowing us to investigate classification of EEG patterns of activity during visual mental imagery of a face versus a house as well as during visual perception of these stimuli.
To uncover the temporal dynamics of category representations during perception and imagery, we used machine learning algorithms to decode over time the category of the stimuli/ mental images from the EEG signal recorded on 25 participants . The results revealed that during perception, significantly different patterns of activity for faces and houses were present from 140ms to 300ms after stimulus onset with the peak accuracy at 170 ms. In the imagery condition however, we did not found significantly different pattern of activity between faces and houses.
If decoding of visual stimuli category using the EEG activity during visual perception has been studied before , multivariate pattern analysis had not been applied to EEG pattern of activity during visual mental imagery. Thus our results showing that EEG patterns of activity during visual mental imagery of perceptually well-controlled stimuli does not allow for the decoding of semantic category are a novel addition to the literature. Since we were able to successfully decode stimuli category based on the EEG activity of visual perception of the stimuli, we suggest that semantic category representation during visual mental imagery relies on different EEG correlates than those of visual perception.
A paper reporting this experiment is currently in preparation.
Micro-phenomenology is a new scientific discipline that allows the very fine exploration of the lived experience.
While collecting micro-phenomenology data is a well defined process, analysis of a large micro-phenomenology data corpus is not yet well defined.
Therefore the researcher decided to engage in the growing international community using micro-phenomenology to help determine and addressing the issues posed by group data analysis. To this end, the researcher organized an international workshop at the host institution, gathering participants coming from France, Germany and the UK. The researcher is also involved in a group of research aiming at designing and evaluating a cross-validation procedure for the micro-phenomenological interview data. A methodological paper on this topic is in preparation.
The results from the EEG experiment are likely to have a wide impact on the development of real-time EEG clinical applications, such as neurofeedback and brain-computer interfaces, as it highlights the limits of what can be achieved using the EEG signal. It also pushes forward the sate of the art in neuroscience of mental imagery as EEG activity during visual mental imagery had not been studied using multivariate pattern analysis (decoding) methods before.
Micro-phenomenology is a young discipline that is likely to have a growing impact on the study and understanding of human cognition and behaviour. Indeed it becomes more and more acknowlgeded that to understand the functioning of the mind, science cannot rely solely on the study of cerebral activity, but has to create a rigorous method to study human experience. Gathering the developing international community around micro-phenomenology and developing its methodological toolbox is thus utterly important for the future of this discipline.