Combining electrophysiology and cognitive computational modeling in research on meaning in language

Informations projet

Modeling ERPs

N° de convention de subvention: 658999

DOI

10.3030/658999

Projet clôturé

Date de signature de la CE 13 Avril 2015

Date de début 1 Août 2015

Date de fin 31 Juillet 2017

Financé au titre de

• EXCELLENT SCIENCE - Marie Skłodowska-Curie Actions

Coût total

€ 159 795,60

Contribution de l’UE

€ 159 795,60

Coordonné par

FREIE UNIVERSITAET BERLIN
Germany

Periodic Reporting for period 2 - Modeling ERPs (Combining electrophysiology and cognitive computational modeling in research on meaning in language)

Période du rapport: 2016-08-01 au 2017-07-31

Résumé du contexte et des objectifs généraux du projet

Language and meaning processing have been investigated with event-related brain potentials (ERPs), providing direct time-resolved measures of electrical brain activity, and with neural network models, providing mechanistic implementations of the assumed processes. However, there has been very little contact between these fields, even though a combination of both methods could be highly beneficial.
Specifically, a brain signal known as the N400 (a signal that was first described as a response to the presentation of a semantically unexpected word in a sentence) has aroused much interest for its promise to shed light on the brain basis of meaning processing. However, in spite of over 1000 studies using the N400 as a dependent variable, the representations and processes that underlie it remain incompletely understood.
The present project aims to provide an implemented theory of the N400’s functional basis and thus a theory of implicit meaning processing in the brain.
Concerning training, the main goal of the project was to provide the researcher with neural network modeling skills in order to link explicit computational models to neural signals.

Travail effectué depuis le début du projet jusqu’à la fin de la période considérée dans le rapport et principaux résultats atteints jusqu’à présent

We linked the N400 to a deep neural network model that predates many current state of the art language processing models by over 20 years and has renewed relevance in the context of recent breakthroughs in the field of deep learning.
Specifically, we provide both support for and formalization of the view that the N400 reflects the stimulus-driven update of a representation of sentence meaning – one that implicitly and probabilistically represents all aspects of meaning as it evolves in real time during comprehension. We do so by presenting an explicit computational model of this process, showing that it can account for a broad range of empirically observed N400 effects which have been difficult to capture within a single theoretical account and have previously been taken to support diverse and sometimes conflicting N400 theories.
We also show that the model does not predict N400 effects in situations where such effects are not observed empirically (e.g. in response to syntactic irregularities), demonstrating the model's specificity. Furthermore, model comparison with a simple recurrent network model (SRN) trained to predict the next word based on the preceding context (which has also been proposed to account for the N400 component) shows that the SRN fails to capture the N400 data pattern in several instances where our model is in line with the empirical data. Thus, to date our model accounts for the widest range of different N400 effects reported in the literature.
With respect to training, the researcher acquired skills in neural network modeling enabling the described research.

Progrès au-delà de l’état des connaissances et impact potentiel prévu (y compris l’impact socio-économique et les conséquences sociétales plus larges du projet jusqu’à présent)

"The present work provides a computationally explicit and precise theoretical formulation of the N400’s functional basis. This theory of the N400 and thus the brain’s implicit representation of meaning can provide a solid foundation for future studies to further delineate the distinct roles of the N400 as well as other signals evoked during language processing, and can serve as the basis for examining the roles of experience and individual differences in deriving meaning from language. It also provides a computationally explicit account of a number of phenomena that have been taken to suggest that language comprehension does not always result in completely accurate representations of the linguistic input, but may sometimes give rise to representations that are plausibility-sensitive and just ""good enough"", as a result of the inherent uncertainty in real language processing situations.
The project thus provides the basis for extensive follow-up research that has resulted in continued funding and ongoing collaborations enabling the researcher to further develop and establish this line of research."

Simulating N400 amplitudes as change in an implicit and probabilistic representation of meaning