Skip to main content

The cerebral representation of sequences and roles : investigating the origins of human uniqueness.

Periodic Reporting for period 3 - NeuroSyntax (The cerebral representation of sequences and roles : investigating the origins of human uniqueness.)

Reporting period: 2019-10-01 to 2021-03-31

The aim of the NeuroSyntax project is to investigate the brain mechanisms of the humans’ remarkable capacities to grasp, memorize, and produce complex sequences and rules, as manifested for instance in language and mathematics.
Our research focuses on the hypothesis that, during its evolution, the human brain may have acquired a capacity to represent nested rules, based in part on the expansion of circuits involving the inferior frontal gyrus (encompassing “Broca’s area”). The idea is that recursive tree structures underlie natural language, but also artificial languages such as the language of mathematics. In fact, we propose that this human ability may be evident even in a much simpler context -- for instance, when listening to a sequence of two sounds A and B, such as “AABB-BBAA”, humans may use a simple “mental language” using repeats and “for-loops” in order to encode it as “two pairs, then the same in reverse order”. Other primates, such as macaque monkeys, may be unable to achieve such an abstract level of coding, and would therefore be stuck with lower-level representations of the items and their transitions.
To evaluate those hypotheses, we run experiments with artificial “mini-languages” in human behavior, fMRI, MEG and intracranial recordings, and we compare them with the networks for natural language and for mathematics in human adults. We also perform very similar experiments in monkeys and in humans, using fMRI as a shared methodology. Finally, in a theory section, we develop formal mathematical models and implementation models that attempt to characterize how the human brain implements languages.
The results should clarify the brain mechanisms of human language and abstraction abilities, and shed light on their ontogeny and phylogeny.
We have designed artificial languages that attempt to capture the human competence for (1) sequences of sounds A and B ; (2) sequences of spatial locations forming geometrical shapes such as a square ; and (3) continuous shapes such as a circle or a spiral. In each case, the proposed language makes specific predictions about the psychological complexity of various sequences or shapes : the idea is that, the shorter the mental program, the simpler it should be to memorize it or to detect outliers.

We have begun to test those ideas in human adults, using behavior first, then moving on to brain imaging measures using functional MRI and magneto-encephalography. In the case of the language of geometry, we already have obtained good evidence that humans do indeed « compress » spatial sequences in memory using such a « language of thought », that the postulated primitive operations of rule-based compression can be decoded from brain responses, and that even uneducated adults or children behave similarly.

A key question is whether a single « language of thought » can account for all human abilities to memorize sequences, whether they are in the visual or the auditory modality, or whether they use spatial cues (locations) or identity cues (e.g. musical notes). The results so far are mixed. On the one hand, at the behavioral level, we find good evidence for very similar principles of sequence compression in the visual and auditory modalities. On the other hand, at the brain-imaging level, we find multiple parallel brain networks and so far have not been able to identify a single brain region where we could decode an abstact representation of sequences independently of cue or modality. Furthermore, we systematically use human fMRI to compare the representation of artificial sequences with that of natural language and of mathematics. The results suggest that the « language of geometry » calls upon brain areas that are entirely differently from those of natural language -- and therefore lead us to the hypothesis that multiple brain areas have acquired superior abilities to represented nested rules and tree structures during human evolution.

We have begun the much more difficult task of obtaining similar behavioral and functional MRI data from non-human primates, using the artificial languages that we have designed. The results so far indicate that both macaque and humans can store the transitions between items, but that humans can do more and exhibit specific activations of Broca’s area when they compress sequences using a language with nested structures. Still, in a behavioral study in collaboration with Liping Wang, we have been surprised to discover that macaque monkeys could acquire much more abstract knowledge than we thought possible : after much training, they clearly understand the concept of « mirror-reversed sequence » (e.g. ABC-CBA), which forces us to revise previous hypotheses about the type of language that they can encode. Work is under way to test their brain responses to auditory sequences of sounds whose variable complexity is well predicted by our language.

Finally, an exciting part of this research is to develop mathematical models of how language may be encoded at the single-neuron level. In this respect, we have made significant progress in analyzing how a classical artificial neural network for language (called LSTM) dedicates units to syntactic structures. Single neurons appear to be dedicated to singular and to plural nouns, in a way that carries the information over intermediate syntactic phrases when needed for noun-verb agreement (e.g. « the car that passes the trucks IS red). This research both clarifies how artificial neural networks work, and makes very specific predictions that we can test in our human recordings of brain activity.
By the end of the project, we hope to obtain
- A clear mathematical theory of how humans encode the abstract structure of sequences
- A clear picture of which brain areas are involved, depending on whether the stimuli are auditory or visual, spatial or non-spatial, and linguistic or mathematical
- The first insights into how single neurons and neuronal circuits encode syntactic structures
- Comparative paradigms that allow us to perform the same exact experiments in humans and in monkeys, and to determine what brain organization for language(s), if any, is unique to the human brain.