Building a computational basis for the brain response in the left ventral occipito-temporal cortex: Understanding the Sparse Familiarity Model of visual word recognition

Project Information

SFM4VOT

Grant agreement ID: 707932

DOI

10.3030/707932

Project closed

EC signature date 8 March 2016

Start date 1 April 2016

End date 15 July 2018

Funded under

• EXCELLENT SCIENCE - Marie Skłodowska-Curie Actions

Total cost

€ 171 460,80

EU contribution

€ 171 460,80

Coordinated by

JOHANN WOLFGANG GOETHE-UNIVERSITAET FRANKFURT AM MAIN
Germany

Periodic Reporting for period 1 - SFM4VOT (Building a computational basis for the brain response in the left ventral occipito-temporal cortex: Understanding the Sparse Familiarity Model of visual word recognition)

Reporting period: 2016-04-01 to 2018-03-31

Summary of the context and overall objectives of the project

Reading, at its core, reflects the extraction of meaning from a visual representation of the orthographic code. This process is achieved in a fast and automatic fashion by most but still, some individuals do not accomplish information extraction at a preferred high speed hampering daily life (i.e. from reading proper newspapers to manuals). The core objective of this proposal was to investigate and explicitly describe the cognitive processes that are characteristic for fast reading from a neuronal perspective using computational models. Such a computational description allows a fruitful scientific discourse by model comparison in basic research and, at best, informs treatment approaches in more applied settings.
The core cognitive process implemented in our computational model (originally the sparse familiarity model: SFM, now the lexical categorization model: LCM; Gagl, Richlan, Ludersdorfer, Sassenhagen, & Fiebach, 2016) is the categorization of letter strings in meaningful or meaningless. In work package 1 (WP1) the objective was to investigate how the lexical categorization process is influenced by learning and in WP2 the objective was the generalization of the lexical categorization process to other image categories like faces or objects, which are all known to be processed in the ventral part of the temporal cortex.

Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far

In WP1, I investigated the influence of learning on the lexical categorization process, in cooperation with a doctoral student (Susanne Eisenhauer) and my supervisor Christian Fiebach (Student assistant, Stefanie Wu), in two experiments (one MEG, one behavioral: Eisenhauer et al., 2018). Both studies involved an extensive training period of two to three days with an experimental data acquisition session after training. One experiment involved the measurement of brain activation with MEG (57 participants involved). In the second experiment, the behavioral performance was recorded (33 participants involved). In addition, both experiments included a priming task that allowed investigating the LCM process with and without contextual information. The one central finding in relation to the LCM was that training altered the brain activation in the left posterior part of the brain during reading.

In the second study of WP1, I (in cooperation with research assistant Klara Gregorova) realized two extensive intervention studies investigating if an LCM based training increases reading speed for second language learners (Gregorova & Gagl, in preparation). Here we implemented one experiment with an assessment of reading speed before training, three sessions of lexical categorization training (45 min), and an assessment of reading speed afterward (25 participants included). In the second experiment, we paired the lexical categorization training with a phonics training, currently the only effective training for dyslexia, in a randomized controlled trial (32 participants included in both interventions). Both studies showed transfer effects of the intervention to reading speed indicating the potential of the training.

In WP2, the objective was to evaluate if the core process of the LCM generalizes to other visual stimuli. Central to the implementation of the LCM for words is the estimation of a similarity index, now realized on the basis of the orthographic Levenshtein distance (Yarkoni, Balota, & Yap, 2008), which cannot be estimated for other visual stimuli. To bypass this issue, we originally planned to study generalization using a noise manipulation for words and other visual stimuli like faces. Instead, we decided to implement an image-based similarity index, with the potential for generalization, generating an explicit mechanistic explanation. For WP2, I (in cooperation with Jona Sassenhagen, Sophia Haan, Fabio Richlan, and Christian Fiebach), therefore, implemented a second computational model, the visual-orthographic prediction model (VOP), which allows realizing an orthographic similarity measure based on the images of words only (Gagl et al., submitted). In the study, we implemented and evaluated the VOP model with four behavioral studies (87 measured participants and open data from English and French), one fMRI (39 measured participants) and one EEG study (31 measured participants) both investigating brain activation. In addition, we conducted a first generalization investigation by showing that the VOP model, typically realized with computerized scripts, can be used to index the readability of handwritten scripts (48 participants).

Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far)

The central issue of the SFM4VOT project was building a computationally explicit description of orthographic processing in the left ventral occipito-temporal cortex based on neurocognitive theory and data. Here the possibility of realizing explicit model comparisons is a central progress compared to the classic verbal/descriptive models. In the two studies which describe the models (Gagl et al., 2016, submitted), the model comparison method was a central approach to compare classical and new ideas quantitatively not relying on linguistic descriptions. Critical here is that for both models the comparisons are based on a broad empirical basis. Such explicit investigations will allow a fruitful scientific discourse in future basic research.
Beyond this, the processes described in the models (lexical categorization and sensory information optimization) provide a better understanding of the visual word recognition process. Hence, all proposed processes generate new hypotheses for treatment and diagnostics of slow reading. This is especially important as current evidence from treatment approaches (Galuschka, Ise, Krick, & Schulte-Körne, 2014) suggests that only one treatment approach was found effective with only a small effect size. The implementation of a promising intervention on the basis of lexical categorization (Gregorova & Gagl, in preparation) is the first step towards a neurocognitively motivated training approach of orthographic processing. Currently, we focus on second language learners, a large and growing group of individuals, and plan to adopt the approach for young readers at the beginning of literacy acquisition. The wider impact of an efficient diagnostic and treatment program is that higher reading skills increase information processing skills of slow readers. This is critical for daily life decisions at work and elsewhere since these are based only on the available information which is limited when the individual capacities, one being the speed of reading, are low.


References
Eisenhauer, S., Fiebach, C. J., & Gagl, B. (2018). Dissociable prelexical and lexical contributions to visual word recognition and priming: Evidence from MEG and behavior. BioRxiv, 410795.
Gagl, B., Richlan, F., Ludersdorfer, P., Sassenhagen, J., & Fiebach, C. J. (2016). The lexical categorization model: A computational model of left ventral occipito-temporal cortex activation in visual word recognition. BioRxiv, 085332.
Gagl, B., Sassenhagen, J., Haan, S., Richlan, F., & Fiebach, C. J. (submitted). Visual word recognition relies on a sensory prediction error signal.
Galuschka, K., Ise, E., Krick, K., & Schulte-Körne, G. (2014). PLOS ONE, 9(2), e89900.
Gregorova, K., & Gagl, B. (in preparation). Lexical categorization training is successful in increasing reading speed of L2-German readers.
Yarkoni, T., Balota, D., & Yap, M. (2008). Psychonomic Bulletin & Review, 15(5), 971–979.

Schematic description of the lexical categoriation model