Periodic Reporting for period 4 - NeuroSyntax (The cerebral representation of sequences and roles : investigating the origins of human uniqueness.)
Periodo di rendicontazione: 2021-04-01 al 2022-03-31
The aim of the NeuroSyntax project is to investigate the brain mechanisms that underlie humans' remarkable capacities to grasp, memorize, and produce complex sequences and rules, as manifested for instance in language and mathematics.
Our research focuses on the hypothesis that, during its evolution, the human brain may have acquired a capacity to represent nested rules, based in part on the expansion of circuits involving the inferior frontal gyrus (encompassing “Broca’s area”). The idea is that recursive tree structures underlie natural language, but also artificial languages such as the language of mathematics. In fact, we propose that this human ability may be evident even in a much simpler context -- for instance, when listening to a sequence of two sounds A and B, such as “AABB-BBAA”, humans may use a simple “mental language” using repeats and “for-loops” in order to encode it as “two pairs, then the same in reverse order”. Other primates, such as macaque monkeys, may be unable to achieve such an abstract level of coding, and would therefore be stuck with lower-level representations of the items and their transitions.
To evaluate those hypotheses, during the project we ran experiments with artificial “mini-languages” while measuring human behavior and also examining the underlying brain mechanisms with functional MRI, MEG and intracranial recordings. We compared them with the networks for natural language and for mathematics in human adults. We also performed very similar experiments in non-human primates. Finally, in a theory section, we developed formal mathematical models and implementation models that attempt to characterize how the human brain implements such languages.
The project resulted in a series of discoveries on how humans encode geometrical and temporal information, in the visual modality (for geometrical information and for written language) as well as in the auditory modality (for tone sequences and for spoken language). The results definitely establish that (1) humans use tree-like recursive structures, akin to "mental languages" to spontaneously encode a broad variety of regularities, including auditory sequences and geometrical shapes; and (2) whenever we were able to test it, monkeys and baboons failed to do so on the very same tests. The results have strong implications for our understanding of the origins of human singularity, as they suggest that our brain may have evolved a new mode of symbolic recursive representation, not available to other animals.
Our research focuses on the hypothesis that, during its evolution, the human brain may have acquired a capacity to represent nested rules, based in part on the expansion of circuits involving the inferior frontal gyrus (encompassing “Broca’s area”). The idea is that recursive tree structures underlie natural language, but also artificial languages such as the language of mathematics. In fact, we propose that this human ability may be evident even in a much simpler context -- for instance, when listening to a sequence of two sounds A and B, such as “AABB-BBAA”, humans may use a simple “mental language” using repeats and “for-loops” in order to encode it as “two pairs, then the same in reverse order”. Other primates, such as macaque monkeys, may be unable to achieve such an abstract level of coding, and would therefore be stuck with lower-level representations of the items and their transitions.
To evaluate those hypotheses, during the project we ran experiments with artificial “mini-languages” while measuring human behavior and also examining the underlying brain mechanisms with functional MRI, MEG and intracranial recordings. We compared them with the networks for natural language and for mathematics in human adults. We also performed very similar experiments in non-human primates. Finally, in a theory section, we developed formal mathematical models and implementation models that attempt to characterize how the human brain implements such languages.
The project resulted in a series of discoveries on how humans encode geometrical and temporal information, in the visual modality (for geometrical information and for written language) as well as in the auditory modality (for tone sequences and for spoken language). The results definitely establish that (1) humans use tree-like recursive structures, akin to "mental languages" to spontaneously encode a broad variety of regularities, including auditory sequences and geometrical shapes; and (2) whenever we were able to test it, monkeys and baboons failed to do so on the very same tests. The results have strong implications for our understanding of the origins of human singularity, as they suggest that our brain may have evolved a new mode of symbolic recursive representation, not available to other animals.
Our goal was to test the hypothesis that only humans can recombine mental symbols flexibly using a mental “language of thought”. We tested this idea by designing artificial mini-languages in which the stimuli are simple, non-verbal, and yet demonstrably involve nested structures; then checking whether monkeys and humans can acquire them.
Sequences of spatial locations in humans. Marie Amalric and Stanislas Dehaene designed a geometrical language that can describe, in a compact manner, spatial sequences of locations. Behavioral research proving that it explains human behavior in a spatial working memory task was published by Marie Amalric in PLOS Computational Biology in 2016, at the onset of this project. With Liping Wang, we published the fMRI version of this task in human adults during eye tracking of these sequences (Neuroimage, 2019). Fosca Al Roumi then acquired and analyzed a full set of human MEG data with 20 subjects during this geometrical task (Neuron, 2021), showing how the human brain “factorizes” a sequence into numerical and geometrical primitives, as predicted by our model.
We next tested whether monkeys possess a similar language. Liping Wang and Stanislas Dehaene started with a much simpler task of memorizing and repeating a spatial sequence of 2, 3 or 4 items, and showed that although macaque monkeys do so in a manner strikingly different from human adults and children, without any understanding of the geometry of the sequence (Zhang et al, 2022). Similar research was performed on baboons, in collaboration with Joel Fagot ( Scientific Reports, 2020).
Geometrical representation of static shapes. Mathias Sablé-Meyer extended the language of thought for geometry to the case of static geometrical shapes. We performed many behavioral experiments showing that humans, but not baboons, are sensitive to shape regularity, as determined by the presence of primitives of parallelism, right angle, equal length and symmetry (Sablé-Meyer et al., PNAS 2021). We also designed a more generic language that can account for all of the major shapes that are repeatedly found in all human cultures throughout the world (e.g. lines, circles, spirals, squares, and their combinations) and proved that, again, the language accounts for human but not baboon behavior (Sablé-Meyer et al., Cognitive Science, in revision).
Language for sound sequences. We designed and tested another mini-language for binary auditory sequences made of two sounds x and Y). The language captures why humans find some sequences regular. We first proved that human working memory for sequences is well captured by this language (Planton et al., PLOS Biology, 2021), then showed that the same is true of human MEG and fMRI signals. Maxime Maheu proved that humans indeed possess two distinct systems for discrete rules (unique to humans) and for statistical regularities (shared with other primates) (Maheu et al., Elife 2019, and Nature human behaviour, 2022).
Representation of natural language in humans. We analyzed the networks for natural language with a similar approach. Yair Lakretz investigated whether and how artificial neural networks implement linguistic structures (Lakretz et al., Cognition, 2021). To experimentally study the neural encoding of sentences, we established collaborations that gave us access to intracranial recordings in the human brain. In collaboration with Prof. Itzhak Fried from UCLA, we now have a database of human intracranial signals, including single neuron firing, during sentence processing, which we believe to be quite unique at the international level (Lakretz et al., in preparation). Other collaborations were launched with Prof. Nitin Tandon from UTHealth center in Houston, Prof. Florian Mormann at the University of Bonn Medical Center, and with Professors Fabrice Bartolomei and Christian Bénar at Aix-Marseille University. Intracranial signals were recorded while patients performed various versions of our previous sentence paradigm (Nelson, et. al, PNAS, 2017). The resultsconfirm that sentence processing in humans involves the construction of recursive tree structures, not just ordinal sequences, and that ramping brain signals are a marker of such construction.
Overall, we proved that (1) the human brain spontaneously compresses linguistic, auditory and geometrical information using recursive tree structures; and (2) monkeys and baboons fail to do so on the very same tests. The results have strong implications for our understanding of the origins of human singularity (Dehaene et al., Trends in Cognitive Science, 2022, in revision).
Sequences of spatial locations in humans. Marie Amalric and Stanislas Dehaene designed a geometrical language that can describe, in a compact manner, spatial sequences of locations. Behavioral research proving that it explains human behavior in a spatial working memory task was published by Marie Amalric in PLOS Computational Biology in 2016, at the onset of this project. With Liping Wang, we published the fMRI version of this task in human adults during eye tracking of these sequences (Neuroimage, 2019). Fosca Al Roumi then acquired and analyzed a full set of human MEG data with 20 subjects during this geometrical task (Neuron, 2021), showing how the human brain “factorizes” a sequence into numerical and geometrical primitives, as predicted by our model.
We next tested whether monkeys possess a similar language. Liping Wang and Stanislas Dehaene started with a much simpler task of memorizing and repeating a spatial sequence of 2, 3 or 4 items, and showed that although macaque monkeys do so in a manner strikingly different from human adults and children, without any understanding of the geometry of the sequence (Zhang et al, 2022). Similar research was performed on baboons, in collaboration with Joel Fagot ( Scientific Reports, 2020).
Geometrical representation of static shapes. Mathias Sablé-Meyer extended the language of thought for geometry to the case of static geometrical shapes. We performed many behavioral experiments showing that humans, but not baboons, are sensitive to shape regularity, as determined by the presence of primitives of parallelism, right angle, equal length and symmetry (Sablé-Meyer et al., PNAS 2021). We also designed a more generic language that can account for all of the major shapes that are repeatedly found in all human cultures throughout the world (e.g. lines, circles, spirals, squares, and their combinations) and proved that, again, the language accounts for human but not baboon behavior (Sablé-Meyer et al., Cognitive Science, in revision).
Language for sound sequences. We designed and tested another mini-language for binary auditory sequences made of two sounds x and Y). The language captures why humans find some sequences regular. We first proved that human working memory for sequences is well captured by this language (Planton et al., PLOS Biology, 2021), then showed that the same is true of human MEG and fMRI signals. Maxime Maheu proved that humans indeed possess two distinct systems for discrete rules (unique to humans) and for statistical regularities (shared with other primates) (Maheu et al., Elife 2019, and Nature human behaviour, 2022).
Representation of natural language in humans. We analyzed the networks for natural language with a similar approach. Yair Lakretz investigated whether and how artificial neural networks implement linguistic structures (Lakretz et al., Cognition, 2021). To experimentally study the neural encoding of sentences, we established collaborations that gave us access to intracranial recordings in the human brain. In collaboration with Prof. Itzhak Fried from UCLA, we now have a database of human intracranial signals, including single neuron firing, during sentence processing, which we believe to be quite unique at the international level (Lakretz et al., in preparation). Other collaborations were launched with Prof. Nitin Tandon from UTHealth center in Houston, Prof. Florian Mormann at the University of Bonn Medical Center, and with Professors Fabrice Bartolomei and Christian Bénar at Aix-Marseille University. Intracranial signals were recorded while patients performed various versions of our previous sentence paradigm (Nelson, et. al, PNAS, 2017). The resultsconfirm that sentence processing in humans involves the construction of recursive tree structures, not just ordinal sequences, and that ramping brain signals are a marker of such construction.
Overall, we proved that (1) the human brain spontaneously compresses linguistic, auditory and geometrical information using recursive tree structures; and (2) monkeys and baboons fail to do so on the very same tests. The results have strong implications for our understanding of the origins of human singularity (Dehaene et al., Trends in Cognitive Science, 2022, in revision).