Final Report Summary - ILP (The Role of Intention in Language Processing)
The goal of this project was to enlarge our understanding of the cortical dynamics underpinning the conscious intention to engage in language processing. Concretely, we assessed the brain's spatiotemporal response to core mental operations involved in uttering (production) and reading (comprehension) words and how that response is affected and shaped by top-down goal-directed behavior, both within and across those two language modalities. At the practical level, the research approach followed to achieve those objectives can be divided in three complementary and incremental lines of investigation: (1) We explored how the conscious intention to perform a language act (global goal-directed linguistic intention) modulates the manner in which words become activated in the brain; (2) compared under conditions of such conscious intention the spatiotemporal brain dynamics elicited by words between producing and reading language (word's cerebral network under intention); and (3) how the cortical dynamics of that language network may be modulated by different, specific linguistic intentions (local goal-directed linguistic intentions). To address these issues we combined experimental designs from traditional psycholinguistics with two neuroscientific techniques: Electroencephalography (EEG) and (the derived) Event-Related brain Potentials (ERPs), a noninvasive neurophysiological technique which can plot the brain's electrical response to a stimulus and cognitive event with milliseconds precision (excellent temporal resolution) by means of surface electrodes placed on a participants' head (outer skull); And Magnetoencephalography (MEG), a noninvasive functional neuroimage technique which can map brain activity in time (good spatial and temporal resolution) by recording magnetic fields produced by electrical currents in the brain elicited in response to sensorial, motor or cognitive processing.
1) Global goal-directed linguistic intention:
Most models of language processing, psycholinguistic and neurobiological alike, assume that words become activated in the brain in an automatic fashion. That is, due to the excessive practice and repetition of language behaviour throughout mental life, input (e.g. written words, objects, others' speech) which is strongly linked to that behaviour will trigger word representations in the brain, regardless whether we intend to engage in language processing or not. This is a dynamic which is called spreading activation and which is assumed to underly both language production and comprehension. Not that spreading activation models of lexical access assume that the language system is fully encapsulated from goal-directed and attentional influences, but they do assume that those top-down modulations come into play reactively; that is, after some linguistic knowledge has been triggered in a feedforward manner by the input (e.g. think how difficult it is – if not impossible – to not read written words). Nonetheless, the amount of empirical research dedicated to uncovering when and how goal-directed behaviour may affect the mental operations of language is scarce and the aim of this part of the project was to contribute to this important research topic.
Elsewhere (the input to the current project), we have already showed that the above assumption of automaticity does not hold for language production. Strijkers and colleagues (2011) observed that the time-course of word access, as measured with ERPs, fluctuated in function of a speaker's intentions. When there was the conscious intention to speak (overt object naming) the brain rapidly showed traces of word activation (around 150 ms), but when no such intention was present (non-verbal object categorization) no word-related brain response was observed until 350 ms after stimulus onset. From these results the authors concluded that rapid access to word knowledge in the brain is not solely driven by automaticity, but dependent and facilitated by top-down biasing signals in function of a speaker's intention to engage in language production. The first aim in this part of the project was to explore whether a similar modulatory influence elicited by global goal-directed linguistic intention was functional in word reading as well. This question is relevant, since there might be good reasons to assume that reading is a more strongly automatized skill then word production: Reading, compared to object naming, is a more passive process where the input is already linguistic nature (written words).
To address this issue, we compared the electrophysiological pattern (ERP) elicited by a word's lexical frequency, a well-establish physiological index of word access, between a language-relevant and language-irrelevant task (Strijkers, Bertrand & Grainger, under review, JOCN). Participants were presented with meaningful written words in different font colors and had to either read them for comprehension (semantic categorization) or ignore the meaning of the written words and instead assess in which color they were presented (color categorization). The relevant experimental contrast with respect to the question posed concerned a direct comparison of the ERPs elicited by the same word frequency effect (the difference between low frequency and high frequency words) between the two tasks. We observed that ERP waveforms elicited by low frequency words started to diverge from those elicited by high frequency words as early as 120 ms after presentation and for a wide array of scalp electrodes in the semantic categorization task. In contrast, in the color categorization task, the ERP differences between the same low and high frequency words became visible after 250 ms and with a restricted topographic distribution. Compatible with the results reported for speech production, these data revealed that, even though written words do trigger automatic recognition processes in the brain (as documented by the presence of a word frequency effect in a language-irrelevant, purely visual task), a significantly earlier and qualitatively richer neural response to words critically depends on the top-down driven intention to engage actively and consciously in language processing (see also Baus, Strijkers & Costa, 2013, Front. Psychol.). Furthermore, the similar electrophysiological pattern of results between speaking and reading, suggests a similar top-down mechanism for both linguistic skills, at least when the brain is preparing (globally) an upcoming language act.
2) Word's cerebral network under intention:
Having established that the goal-directed intention to engage in language processing ensures rapid access to word knowledge, in this part of the project we wanted to investigate exactly which type of word knowledge becomes available so rapidly and compare this dynamic between reading and speaking. It is, therefore, currently debated in the field of language science whether linguistic knowledge is retrieved in an incremental hierarchical fashion or not. According to some researchers the semantic, lexical, phonological and phonetic/articulatory information of words are engendered by specialized neuronal systems which are triggered progressively in time. In contrast, others argue for a processing dynamic underpinning language where concepts, the lexical knowledge with which they are associated, the corresponding phonology and the articulatory plans are supported by wide-spread distributed neural circuits which emerge in a rapid near-simultaneous fashion. Here, in a series of MEG studies, we contrasted these different views by tracking the time-course of cortical area activations involved in lexico-semantic word processing versus those recruited for phonological and articulation programming.
In particular, for the language production experiment (Strijkers, Costa & Pulvermuller, under review, Cer. Cortex), anatomically constrained MEG activity was recorded while participants engaged in an overt object naming task. The critical contrast concerned a comparison between the spatiotemporal signature of the word frequency effect (index for lexico-semantic processing) with the spatiotemporal pattern elicited by articulatory motor programs which differ between speech sounds (index for phonological and articulatory processing) – such as phonemes primarily using the tongue (e.g. alveolar [d]; donkey) compared with phonemes especially drawing on lip movement (e.g. bilabial [m]; monkey). We observed a remarkably early modulation of brain activity in the mid temporal gyrus (MTG) and the left inferior frontal gyrus (LIFG) elicited by the lexical frequency of an object's name, along with a simultaneous dependency of local brain responses in the motor cortex and the superior temporal gyrus (STG) with respect to the particular articulatory movement (lip vs. tongue) required to utter different word-initial speech sounds. Put differently, within 200 ms of processing both brain regions associated with lexico-semantic processing (MTG and LIFG) and brain regions associated with phonology and articulation (STG and motor cortex) became activated. These results provide compelling evidence that in the course of speech planning, lexico-semantic and phonological/phonetic processes emerge rapidly together, drawing in parallel on a distributed fronto-temporal neural network.
At present, we are collecting comparable MEG data for reading (Strijkers & Pulvermuller, in prep.). In this study, participants are presented with the written equivalents of the picture names used in the production experiment and instructed to silently read the words. The same experimental contrasts are investigated in this study (lexical frequency versus first phoneme distinction). We predict to observe a similar spatiotemporal pattern elicited by these variables. That is, provided some potential minor differences in time-course and sources (which would be interesting by itself as well), we expect to observe early parallel activation for the lexico-semantic variable (modulating as in production the MTG and the LIFG) and the phoneme-related variable (modulating as in production the STG and possibly also the motor cortex). Such result would highlight that a similar rapid, fronto-temporal distributed cortical dynamic underpins the intentional activation of words in the brain when speaking or reading. Knowledge which offers an important contribution to integrated models of language production and comprehension (see also: Strijkers, Runnqvist, Costa & Holcomb, 2013, BBS; Strijkers, 2014, Lang. Cogn. Neurosci.).
3) Local goal-directed linguistic intention:
In this final part of the project, we wanted to explore: a) whether the functional role of proactive top-down modulations in language processing go beyond a general preparatory biasing (i.e. placing the brain in a 'language state'), and can also affect word retrieval in a more local, specific manner in function of particular linguistic goals; and b) how exactly this more local goal-directed control would influence the fronto-temporal network of intentional speaking and reading as identified in part 2. In a first series of experiments we investigated how top-down processing deals with extra-linguistic factors when engaged in language processing. For one, we demonstrated (with ERPs) that the specific intention to speak in a dominant (first) language versus a weaker (second) language affects the temporal dynamics of word activation within the first 200 ms of processing by proactively inhibiting words from the dominant language in order to facilitate second language production (Strijkers, Baus, Runnqvist, FitzPatrick & Costa, 2013, Brain Lang.). Secondly, in reading, we have shown that extra-linguistic information of a written word, such as its length, modulates the speed with which meaning is triggered in the brain (the N400 ERP response). That is, we observed that the brain relies on physical properties of the input (the length in this case) to predict in a top-down manner a possible set of word candidates, hereby facilitating word recognition in the cerebral cortex (Ktori, Strijkers & Grainger, in. prep.). Finally, we ran a series of object naming and word reading ERP experiments where we presented congruently with the target language-related distractor information. Crucially, we manipulated the lexical frequency of both the targets (language-relevant knowledge) and the distractors (language-irrelevant knowledge), in order to track whether the time-course of the lexical frequency effect would be different between the language input we intend to process and the language input we intend to ignore. While we are currently analyzing these data, (very) preliminary results indicate, as predicted, that the lexical frequency effect occurs earlier and stronger for the language-relevant input compared to the distractor input (Strijkers, Runnqvist, D'Hooge, Hartsuiker & Grainger, in prep.). Such finding would thus suggest that top-down intention does not just enhance the language network in a global fashion, but is capable of facilitating specific words of relevance to the present intention.
In a second series of experiments, currently running, we wanted to test whether intra-linguistic differences can modulate in a proactive (predictive) fashion a word's cerebral network under intention (Strijkers, Munding, Chanoine, Bardier & Alario, in prep.). To do so, we explored, making use of MEG, whether function words could serve as top-down input for upcoming nouns and verbs (e.g. in French: la table ('the table') for nouns, and il mange ('he eats') for verbs), both for word production and perception. Specifically, we predict that the speed and sources of the neural network underlying words will fluctuate in function of which grammatical word class is predicted by the function words, namely earlier activation of frontal ('action-related') sources of the network when a function word associated with verbs is presented and faster activation of occipito-temporal ('object-related') sources when a function word linked to nouns is presented. Such finding would provide explicit evidence that top-down prediction can pre-activate local cortical sources of the language network in function of distinct linguistic intentions.
In conclusion, in this project we have demonstrated that the brain's capacity to rapidly activate words when speaking or reading is instigated by the conscious intention to engage in language processing. We have shown that the type of word knowledge retrieved in such rapid top-down manner is not restricted to a single (low-level) linguistic operation and brain region, but instead recruits a widely distributed neuronal network including many (if not all) different types of linguistic knowledge of a word in parallel. And finally, it was demonstrated that the spatiotemporal dynamics of this language network can be shaped in function of specific extra-linguistic information, such as bilingualism, physical properties of the language input and attentional demands, and possibly (pending results) specific intra-linguistic information, such as grammatical class, as well. The data brought forward in this project has strongly advanced our understanding of how goal-directed intention can influence basic language processing in the brain, has promoted intra-disciplinary research (both within language sciences and across other domains of cognition), provides conclusions of high relevance for integrated psycholinguistic and neuroscientific theories of language production and language comprehension, and has offered a substantial contribution to the domain-general question about the neurobiological organization and dynamics underpinning human's most defining cognitive skill, namely our capacity for language.
1) Global goal-directed linguistic intention:
Most models of language processing, psycholinguistic and neurobiological alike, assume that words become activated in the brain in an automatic fashion. That is, due to the excessive practice and repetition of language behaviour throughout mental life, input (e.g. written words, objects, others' speech) which is strongly linked to that behaviour will trigger word representations in the brain, regardless whether we intend to engage in language processing or not. This is a dynamic which is called spreading activation and which is assumed to underly both language production and comprehension. Not that spreading activation models of lexical access assume that the language system is fully encapsulated from goal-directed and attentional influences, but they do assume that those top-down modulations come into play reactively; that is, after some linguistic knowledge has been triggered in a feedforward manner by the input (e.g. think how difficult it is – if not impossible – to not read written words). Nonetheless, the amount of empirical research dedicated to uncovering when and how goal-directed behaviour may affect the mental operations of language is scarce and the aim of this part of the project was to contribute to this important research topic.
Elsewhere (the input to the current project), we have already showed that the above assumption of automaticity does not hold for language production. Strijkers and colleagues (2011) observed that the time-course of word access, as measured with ERPs, fluctuated in function of a speaker's intentions. When there was the conscious intention to speak (overt object naming) the brain rapidly showed traces of word activation (around 150 ms), but when no such intention was present (non-verbal object categorization) no word-related brain response was observed until 350 ms after stimulus onset. From these results the authors concluded that rapid access to word knowledge in the brain is not solely driven by automaticity, but dependent and facilitated by top-down biasing signals in function of a speaker's intention to engage in language production. The first aim in this part of the project was to explore whether a similar modulatory influence elicited by global goal-directed linguistic intention was functional in word reading as well. This question is relevant, since there might be good reasons to assume that reading is a more strongly automatized skill then word production: Reading, compared to object naming, is a more passive process where the input is already linguistic nature (written words).
To address this issue, we compared the electrophysiological pattern (ERP) elicited by a word's lexical frequency, a well-establish physiological index of word access, between a language-relevant and language-irrelevant task (Strijkers, Bertrand & Grainger, under review, JOCN). Participants were presented with meaningful written words in different font colors and had to either read them for comprehension (semantic categorization) or ignore the meaning of the written words and instead assess in which color they were presented (color categorization). The relevant experimental contrast with respect to the question posed concerned a direct comparison of the ERPs elicited by the same word frequency effect (the difference between low frequency and high frequency words) between the two tasks. We observed that ERP waveforms elicited by low frequency words started to diverge from those elicited by high frequency words as early as 120 ms after presentation and for a wide array of scalp electrodes in the semantic categorization task. In contrast, in the color categorization task, the ERP differences between the same low and high frequency words became visible after 250 ms and with a restricted topographic distribution. Compatible with the results reported for speech production, these data revealed that, even though written words do trigger automatic recognition processes in the brain (as documented by the presence of a word frequency effect in a language-irrelevant, purely visual task), a significantly earlier and qualitatively richer neural response to words critically depends on the top-down driven intention to engage actively and consciously in language processing (see also Baus, Strijkers & Costa, 2013, Front. Psychol.). Furthermore, the similar electrophysiological pattern of results between speaking and reading, suggests a similar top-down mechanism for both linguistic skills, at least when the brain is preparing (globally) an upcoming language act.
2) Word's cerebral network under intention:
Having established that the goal-directed intention to engage in language processing ensures rapid access to word knowledge, in this part of the project we wanted to investigate exactly which type of word knowledge becomes available so rapidly and compare this dynamic between reading and speaking. It is, therefore, currently debated in the field of language science whether linguistic knowledge is retrieved in an incremental hierarchical fashion or not. According to some researchers the semantic, lexical, phonological and phonetic/articulatory information of words are engendered by specialized neuronal systems which are triggered progressively in time. In contrast, others argue for a processing dynamic underpinning language where concepts, the lexical knowledge with which they are associated, the corresponding phonology and the articulatory plans are supported by wide-spread distributed neural circuits which emerge in a rapid near-simultaneous fashion. Here, in a series of MEG studies, we contrasted these different views by tracking the time-course of cortical area activations involved in lexico-semantic word processing versus those recruited for phonological and articulation programming.
In particular, for the language production experiment (Strijkers, Costa & Pulvermuller, under review, Cer. Cortex), anatomically constrained MEG activity was recorded while participants engaged in an overt object naming task. The critical contrast concerned a comparison between the spatiotemporal signature of the word frequency effect (index for lexico-semantic processing) with the spatiotemporal pattern elicited by articulatory motor programs which differ between speech sounds (index for phonological and articulatory processing) – such as phonemes primarily using the tongue (e.g. alveolar [d]; donkey) compared with phonemes especially drawing on lip movement (e.g. bilabial [m]; monkey). We observed a remarkably early modulation of brain activity in the mid temporal gyrus (MTG) and the left inferior frontal gyrus (LIFG) elicited by the lexical frequency of an object's name, along with a simultaneous dependency of local brain responses in the motor cortex and the superior temporal gyrus (STG) with respect to the particular articulatory movement (lip vs. tongue) required to utter different word-initial speech sounds. Put differently, within 200 ms of processing both brain regions associated with lexico-semantic processing (MTG and LIFG) and brain regions associated with phonology and articulation (STG and motor cortex) became activated. These results provide compelling evidence that in the course of speech planning, lexico-semantic and phonological/phonetic processes emerge rapidly together, drawing in parallel on a distributed fronto-temporal neural network.
At present, we are collecting comparable MEG data for reading (Strijkers & Pulvermuller, in prep.). In this study, participants are presented with the written equivalents of the picture names used in the production experiment and instructed to silently read the words. The same experimental contrasts are investigated in this study (lexical frequency versus first phoneme distinction). We predict to observe a similar spatiotemporal pattern elicited by these variables. That is, provided some potential minor differences in time-course and sources (which would be interesting by itself as well), we expect to observe early parallel activation for the lexico-semantic variable (modulating as in production the MTG and the LIFG) and the phoneme-related variable (modulating as in production the STG and possibly also the motor cortex). Such result would highlight that a similar rapid, fronto-temporal distributed cortical dynamic underpins the intentional activation of words in the brain when speaking or reading. Knowledge which offers an important contribution to integrated models of language production and comprehension (see also: Strijkers, Runnqvist, Costa & Holcomb, 2013, BBS; Strijkers, 2014, Lang. Cogn. Neurosci.).
3) Local goal-directed linguistic intention:
In this final part of the project, we wanted to explore: a) whether the functional role of proactive top-down modulations in language processing go beyond a general preparatory biasing (i.e. placing the brain in a 'language state'), and can also affect word retrieval in a more local, specific manner in function of particular linguistic goals; and b) how exactly this more local goal-directed control would influence the fronto-temporal network of intentional speaking and reading as identified in part 2. In a first series of experiments we investigated how top-down processing deals with extra-linguistic factors when engaged in language processing. For one, we demonstrated (with ERPs) that the specific intention to speak in a dominant (first) language versus a weaker (second) language affects the temporal dynamics of word activation within the first 200 ms of processing by proactively inhibiting words from the dominant language in order to facilitate second language production (Strijkers, Baus, Runnqvist, FitzPatrick & Costa, 2013, Brain Lang.). Secondly, in reading, we have shown that extra-linguistic information of a written word, such as its length, modulates the speed with which meaning is triggered in the brain (the N400 ERP response). That is, we observed that the brain relies on physical properties of the input (the length in this case) to predict in a top-down manner a possible set of word candidates, hereby facilitating word recognition in the cerebral cortex (Ktori, Strijkers & Grainger, in. prep.). Finally, we ran a series of object naming and word reading ERP experiments where we presented congruently with the target language-related distractor information. Crucially, we manipulated the lexical frequency of both the targets (language-relevant knowledge) and the distractors (language-irrelevant knowledge), in order to track whether the time-course of the lexical frequency effect would be different between the language input we intend to process and the language input we intend to ignore. While we are currently analyzing these data, (very) preliminary results indicate, as predicted, that the lexical frequency effect occurs earlier and stronger for the language-relevant input compared to the distractor input (Strijkers, Runnqvist, D'Hooge, Hartsuiker & Grainger, in prep.). Such finding would thus suggest that top-down intention does not just enhance the language network in a global fashion, but is capable of facilitating specific words of relevance to the present intention.
In a second series of experiments, currently running, we wanted to test whether intra-linguistic differences can modulate in a proactive (predictive) fashion a word's cerebral network under intention (Strijkers, Munding, Chanoine, Bardier & Alario, in prep.). To do so, we explored, making use of MEG, whether function words could serve as top-down input for upcoming nouns and verbs (e.g. in French: la table ('the table') for nouns, and il mange ('he eats') for verbs), both for word production and perception. Specifically, we predict that the speed and sources of the neural network underlying words will fluctuate in function of which grammatical word class is predicted by the function words, namely earlier activation of frontal ('action-related') sources of the network when a function word associated with verbs is presented and faster activation of occipito-temporal ('object-related') sources when a function word linked to nouns is presented. Such finding would provide explicit evidence that top-down prediction can pre-activate local cortical sources of the language network in function of distinct linguistic intentions.
In conclusion, in this project we have demonstrated that the brain's capacity to rapidly activate words when speaking or reading is instigated by the conscious intention to engage in language processing. We have shown that the type of word knowledge retrieved in such rapid top-down manner is not restricted to a single (low-level) linguistic operation and brain region, but instead recruits a widely distributed neuronal network including many (if not all) different types of linguistic knowledge of a word in parallel. And finally, it was demonstrated that the spatiotemporal dynamics of this language network can be shaped in function of specific extra-linguistic information, such as bilingualism, physical properties of the language input and attentional demands, and possibly (pending results) specific intra-linguistic information, such as grammatical class, as well. The data brought forward in this project has strongly advanced our understanding of how goal-directed intention can influence basic language processing in the brain, has promoted intra-disciplinary research (both within language sciences and across other domains of cognition), provides conclusions of high relevance for integrated psycholinguistic and neuroscientific theories of language production and language comprehension, and has offered a substantial contribution to the domain-general question about the neurobiological organization and dynamics underpinning human's most defining cognitive skill, namely our capacity for language.