Combining electrophysiology and cognitive computational modeling in research on meaning in language

Información del proyecto

Modeling ERPs

Identificador del acuerdo de subvención: 658999

DOI

10.3030/658999

Proyecto cerrado

Fecha de la firma de la CE 13 Abril 2015

Fecha de inicio 1 Agosto 2015

Fecha de finalización 31 Julio 2017

Financiado con arreglo a

• EXCELLENT SCIENCE - Marie Skłodowska-Curie Actions

Coste total

€ 159 795,60

Aportación de la UE

€ 159 795,60

Coordinado por

FREIE UNIVERSITAET BERLIN
Germany

Periodic Reporting for period 2 - Modeling ERPs (Combining electrophysiology and cognitive computational modeling in research on meaning in language)

Período documentado: 2016-08-01 hasta 2017-07-31

Resumen del contexto y de los objetivos generales del proyecto

Language and meaning processing have been investigated with event-related brain potentials (ERPs), providing direct time-resolved measures of electrical brain activity, and with neural network models, providing mechanistic implementations of the assumed processes. However, there has been very little contact between these fields, even though a combination of both methods could be highly beneficial.
Specifically, a brain signal known as the N400 (a signal that was first described as a response to the presentation of a semantically unexpected word in a sentence) has aroused much interest for its promise to shed light on the brain basis of meaning processing. However, in spite of over 1000 studies using the N400 as a dependent variable, the representations and processes that underlie it remain incompletely understood.
The present project aims to provide an implemented theory of the N400’s functional basis and thus a theory of implicit meaning processing in the brain.
Concerning training, the main goal of the project was to provide the researcher with neural network modeling skills in order to link explicit computational models to neural signals.

Trabajo realizado desde el comienzo del proyecto hasta el final del período abarcado por el informe y los principales resultados hasta la fecha

We linked the N400 to a deep neural network model that predates many current state of the art language processing models by over 20 years and has renewed relevance in the context of recent breakthroughs in the field of deep learning.
Specifically, we provide both support for and formalization of the view that the N400 reflects the stimulus-driven update of a representation of sentence meaning – one that implicitly and probabilistically represents all aspects of meaning as it evolves in real time during comprehension. We do so by presenting an explicit computational model of this process, showing that it can account for a broad range of empirically observed N400 effects which have been difficult to capture within a single theoretical account and have previously been taken to support diverse and sometimes conflicting N400 theories.
We also show that the model does not predict N400 effects in situations where such effects are not observed empirically (e.g. in response to syntactic irregularities), demonstrating the model's specificity. Furthermore, model comparison with a simple recurrent network model (SRN) trained to predict the next word based on the preceding context (which has also been proposed to account for the N400 component) shows that the SRN fails to capture the N400 data pattern in several instances where our model is in line with the empirical data. Thus, to date our model accounts for the widest range of different N400 effects reported in the literature.
With respect to training, the researcher acquired skills in neural network modeling enabling the described research.

Avances que van más allá del estado de la técnica e impacto potencial esperado (incluida la repercusión socioeconómica y las implicaciones sociales más amplias del proyecto hasta la fecha)

"The present work provides a computationally explicit and precise theoretical formulation of the N400’s functional basis. This theory of the N400 and thus the brain’s implicit representation of meaning can provide a solid foundation for future studies to further delineate the distinct roles of the N400 as well as other signals evoked during language processing, and can serve as the basis for examining the roles of experience and individual differences in deriving meaning from language. It also provides a computationally explicit account of a number of phenomena that have been taken to suggest that language comprehension does not always result in completely accurate representations of the linguistic input, but may sometimes give rise to representations that are plausibility-sensitive and just ""good enough"", as a result of the inherent uncertainty in real language processing situations.
The project thus provides the basis for extensive follow-up research that has resulted in continued funding and ongoing collaborations enabling the researcher to further develop and establish this line of research."

Simulating N400 amplitudes as change in an implicit and probabilistic representation of meaning