Periodic Reporting for period 1 - NeuPheMi (Neurophenomenology of Mental Imagery)
Período documentado: 2016-03-22 hasta 2018-03-21
Mental imagery is the human brain capacity to create mental images of past, present, future or invented events or action. Mental imagery engages similar neural networks as does the perception of the corresponding stimuli or execution of the corresponding movement and emerging protocols are taking advantage of this characteristic to propose clinical remediation relying on mental imagery training or mental imagery-based neurofeedback, a brain-computer interface allowing to learn to voluntarily control cortical oscillations. However current knowledge of the cerebral basis of mental imagery mainly relies on problem-solving tasks and mental imagery versus physical perception comparisons. While there seems to be a relationship between the subjective experience of imagery during mental imagery training and subsequent functional enhancement, and while scientific research on mental imagery strongly relies on first-person report of imagery experience, most of the experimental work on mental imagery so far has largely ignored the imagery experience itself.
Yet, the need for integrative studies “that bridge between low-level (e.g. neuronal, physiological) and high-level (e.g. belief, intention, identity) descriptions” and “go beyond the classical perception-action loop” to also tackle issues such as experience is clearly identified in the EU research priorities. Indeed, a better understanding of the neural correlates of mental imagery in relation to the experience felt by the subject could enhance the quality and efficacy of mental imagery-based clinical protocols and considering mental imagery as a human experience instead of a mental representation only would foster a better understanding of the nature of MI and its relation to cognition and the brain.
Electroencephalographic (EEG) activity reflects the real-time occurrence of brain events and, as such, is well suited for the study of the neural correlates of transient cognitive events such as mental images. A classical approach in EEG studies of mental imagery is to compute the average neural response elicited by a mental image (event-related potentials, ERPs). However, the oscillatory dynamic of the brain processes is lost both in studies using ERPs and in the ones using the low-time resolution functional magnetic resonnance imaging (fMRI) technique. As a result, the literature lacks of a clear picture of the brain dynamics underlying mental imagery.
The first objective of NeuPheMI is thus to study the dynamic of the brain activity during mental imagery. To do so, electroencephalography (EEG) is a technique of choice, as it has a very high temporal resolution.
A second objective is to study the phenomenological experience that accompanies the making of visual mental images. To do so we decided to employ micro-phenomenology, a novel scientific discipline allowing to explore lived experience very finely. Micro-phenomenology relies on non-inducive guided interviews, enabling the interviewee to become aware of her or his subjective experience and to describe it with great precision.
Yet, the need for integrative studies “that bridge between low-level (e.g. neuronal, physiological) and high-level (e.g. belief, intention, identity) descriptions” and “go beyond the classical perception-action loop” to also tackle issues such as experience is clearly identified in the EU research priorities. Indeed, a better understanding of the neural correlates of mental imagery in relation to the experience felt by the subject could enhance the quality and efficacy of mental imagery-based clinical protocols and considering mental imagery as a human experience instead of a mental representation only would foster a better understanding of the nature of MI and its relation to cognition and the brain.
Electroencephalographic (EEG) activity reflects the real-time occurrence of brain events and, as such, is well suited for the study of the neural correlates of transient cognitive events such as mental images. A classical approach in EEG studies of mental imagery is to compute the average neural response elicited by a mental image (event-related potentials, ERPs). However, the oscillatory dynamic of the brain processes is lost both in studies using ERPs and in the ones using the low-time resolution functional magnetic resonnance imaging (fMRI) technique. As a result, the literature lacks of a clear picture of the brain dynamics underlying mental imagery.
The first objective of NeuPheMI is thus to study the dynamic of the brain activity during mental imagery. To do so, electroencephalography (EEG) is a technique of choice, as it has a very high temporal resolution.
A second objective is to study the phenomenological experience that accompanies the making of visual mental images. To do so we decided to employ micro-phenomenology, a novel scientific discipline allowing to explore lived experience very finely. Micro-phenomenology relies on non-inducive guided interviews, enabling the interviewee to become aware of her or his subjective experience and to describe it with great precision.
We used multivariate pattern analysis, also called decoding to investigates the characteristics of the EEG signal during visual mental imagery.
Preliminary data analyses from a first paradigm using naturalistic stimuli were not conclusive.
We created then another paradigm with well-controlled pictures (same luminance, same spatial frequencies, same overall shape) as visual stimuli on which to base the mental imagery trials on. We developed a paradigm allowing us to investigate classification of EEG patterns of activity during visual mental imagery of a face versus a house as well as during visual perception of these stimuli.
To uncover the temporal dynamics of category representations during perception and imagery, we used machine learning algorithms to decode over time the category of the stimuli/ mental images from the EEG signal recorded on 25 participants . The results revealed that during perception, significantly different patterns of activity for faces and houses were present from 140ms to 300ms after stimulus onset with the peak accuracy at 170 ms. In the imagery condition however, we did not found significantly different pattern of activity between faces and houses.
If decoding of visual stimuli category using the EEG activity during visual perception has been studied before , multivariate pattern analysis had not been applied to EEG pattern of activity during visual mental imagery. Thus our results showing that EEG patterns of activity during visual mental imagery of perceptually well-controlled stimuli does not allow for the decoding of semantic category are a novel addition to the literature. Since we were able to successfully decode stimuli category based on the EEG activity of visual perception of the stimuli, we suggest that semantic category representation during visual mental imagery relies on different EEG correlates than those of visual perception.
A paper reporting this experiment is currently in preparation.
Micro-phenomenology is a new scientific discipline that allows the very fine exploration of the lived experience.
While collecting micro-phenomenology data is a well defined process, analysis of a large micro-phenomenology data corpus is not yet well defined.
Therefore the researcher decided to engage in the growing international community using micro-phenomenology to help determine and addressing the issues posed by group data analysis. To this end, the researcher organized an international workshop at the host institution, gathering participants coming from France, Germany and the UK. The researcher is also involved in a group of research aiming at designing and evaluating a cross-validation procedure for the micro-phenomenological interview data. A methodological paper on this topic is in preparation.
Preliminary data analyses from a first paradigm using naturalistic stimuli were not conclusive.
We created then another paradigm with well-controlled pictures (same luminance, same spatial frequencies, same overall shape) as visual stimuli on which to base the mental imagery trials on. We developed a paradigm allowing us to investigate classification of EEG patterns of activity during visual mental imagery of a face versus a house as well as during visual perception of these stimuli.
To uncover the temporal dynamics of category representations during perception and imagery, we used machine learning algorithms to decode over time the category of the stimuli/ mental images from the EEG signal recorded on 25 participants . The results revealed that during perception, significantly different patterns of activity for faces and houses were present from 140ms to 300ms after stimulus onset with the peak accuracy at 170 ms. In the imagery condition however, we did not found significantly different pattern of activity between faces and houses.
If decoding of visual stimuli category using the EEG activity during visual perception has been studied before , multivariate pattern analysis had not been applied to EEG pattern of activity during visual mental imagery. Thus our results showing that EEG patterns of activity during visual mental imagery of perceptually well-controlled stimuli does not allow for the decoding of semantic category are a novel addition to the literature. Since we were able to successfully decode stimuli category based on the EEG activity of visual perception of the stimuli, we suggest that semantic category representation during visual mental imagery relies on different EEG correlates than those of visual perception.
A paper reporting this experiment is currently in preparation.
Micro-phenomenology is a new scientific discipline that allows the very fine exploration of the lived experience.
While collecting micro-phenomenology data is a well defined process, analysis of a large micro-phenomenology data corpus is not yet well defined.
Therefore the researcher decided to engage in the growing international community using micro-phenomenology to help determine and addressing the issues posed by group data analysis. To this end, the researcher organized an international workshop at the host institution, gathering participants coming from France, Germany and the UK. The researcher is also involved in a group of research aiming at designing and evaluating a cross-validation procedure for the micro-phenomenological interview data. A methodological paper on this topic is in preparation.
The two main contribution of NeuPheMI are (1) the results of the EEG decoding experiment showing that semantic information about the category of the visual stimulus being imagined is not present within the EEG patterns of activation during mental imagery and (2) the contributions to the development of an international community of researcher using the novel scientific discipline micro-phenomenology as well as research in methodological development of micro-phenomenology data analysis and validation.
The results from the EEG experiment are likely to have a wide impact on the development of real-time EEG clinical applications, such as neurofeedback and brain-computer interfaces, as it highlights the limits of what can be achieved using the EEG signal. It also pushes forward the sate of the art in neuroscience of mental imagery as EEG activity during visual mental imagery had not been studied using multivariate pattern analysis (decoding) methods before.
Micro-phenomenology is a young discipline that is likely to have a growing impact on the study and understanding of human cognition and behaviour. Indeed it becomes more and more acknowlgeded that to understand the functioning of the mind, science cannot rely solely on the study of cerebral activity, but has to create a rigorous method to study human experience. Gathering the developing international community around micro-phenomenology and developing its methodological toolbox is thus utterly important for the future of this discipline.
The results from the EEG experiment are likely to have a wide impact on the development of real-time EEG clinical applications, such as neurofeedback and brain-computer interfaces, as it highlights the limits of what can be achieved using the EEG signal. It also pushes forward the sate of the art in neuroscience of mental imagery as EEG activity during visual mental imagery had not been studied using multivariate pattern analysis (decoding) methods before.
Micro-phenomenology is a young discipline that is likely to have a growing impact on the study and understanding of human cognition and behaviour. Indeed it becomes more and more acknowlgeded that to understand the functioning of the mind, science cannot rely solely on the study of cerebral activity, but has to create a rigorous method to study human experience. Gathering the developing international community around micro-phenomenology and developing its methodological toolbox is thus utterly important for the future of this discipline.