Using deep neural networks to understand functional specialization in the human visual cortex

Informations projet

DEEPFUNC

N° de convention de subvention: 101117441

DOI

10.3030/101117441

Date de signature de la CE 18 Decembre 2023

Date de début 1 Janvier 2024

Date de fin 31 Decembre 2028

Financé au titre de

• European Research Council (ERC)

Coût total

€ 1 494 750,00

Contribution de l’UE

€ 1 494 750,00

Coordonné par

JUSTUS-LIEBIG-UNIVERSITAET GIESSEN
Germany

Periodic Reporting for period 1 - DEEPFUNC (Using deep neural networks to understand functional specialization in the human visual cortex)

Période du rapport: 2024-01-01 au 2026-06-30

Résumé du contexte et des objectifs généraux du projet

Understanding why the human visual cortex is organized into functionally specialized regions is one of the central open questions in cognitive neuroscience. While decades of research have documented selective responses for faces, bodies, and scenes, we still lack a principled account of why certain visual categories recruit specialized neural circuitry while others do not. Existing approaches have largely relied on contrast-based neuroimaging and descriptive accounts of selectivity, leaving unresolved whether functional specialization reflects computational constraints, developmental input statistics, or intrinsic neural biases. DEEPFUNC addresses this gap by introducing a unified computational–neuroscientific framework to explain the origins of functional specialization. Focusing on the human ventral visual pathway, the project combines large-scale natural visual input, hypothesis- and data-driven analyses, deep neural networks, and behavioral and neural data. This integrative approach allows us to move beyond descriptive mapping toward mechanistic explanation. The project is structured around three core objectives. First (WP1 – What), we will precisely characterize which properties of visual categories give rise to functional specialization. Second (WP2 – How), we will determine how natural visual input statistics and experience shape the development of specialization. Third (WP3 – Why), we will identify the computational and feature-level principles that determine where and why specialized representations emerge in human visual cortex. By systematically linking deep neural networks with human neuroimaging and behavior, DEEPFUNC establishes a testable computational account of functional specialization. This framework enables us to evaluate competing hypotheses, generate falsifiable predictions, and uncover principles that generalize beyond vision. The expected impact of DEEPFUNC is threefold. Scientifically, it advances the field from descriptive characterization to mechanistic explanation of functional specialization. Methodologically, it provides scalable tools for analyzing and comparing biological and artificial neural systems. Conceptually, it informs the design of intelligent machines by identifying when and why specialized processing architectures are advantageous. More broadly, clarifying how and why the brain organizes information into specialized systems contributes to a deeper understanding of the architecture of the human mind and brain.

Travail effectué depuis le début du projet jusqu’à la fin de la période considérée dans le rapport et principaux résultats atteints jusqu’à présent

During the current reporting period, substantial scientific progress has been achieved across all three work packages. In WP1, we combined large-scale brain imaging data with computational modeling to investigate how the human visual system organizes information. Rather than viewing category-selective brain regions as single-purpose modules, we showed that these regions are composed of multiple interpretable representational dimensions. Building on this insight, we developed a new quantitative method to measure category selectivity across hundreds of object categories in both human visual cortex and artificial neural networks. This large-scale analysis confirmed well-known selectivities (such as for faces and scenes) and revealed additional selective patterns across object space. Importantly, our results indicate that category selectivity is better described as a graded phenomenon rather than a simple all-or-none property. We are currently investigating which visual features and computational principles give rise to these selectivity gradients. In parallel, we are synthesizing artificial visual categories that systematically vary in their degree of specialization in order to experimentally test how graded selectivity influences learning and neural processing. In WP2, we examine how visual experience shapes specialization across development. We began analyzing large-scale egocentric video datasets capturing visual input during childhood and adulthood. We developed tools to quantify how often different visual categories appear and where they occur in the visual field. In addition, we created a method to identify images that most clearly reveal differences between computational models trained on different types of visual experience. Ongoing work uses these tools to test how developmental input shapes visual representations. In WP3, we investigate why specialized processing emerges where it does in the brain. We systematically examined how functional specialization can be reliably measured in artificial neural networks and how different measurement approaches relate to causal interventions. We are now extending this work to track how specialized representations emerge during training and to test how modifying visual features and input statistics influences cortical organization. Together, these activities are advancing the project from descriptive mapping toward understanding why and how functional specialization emerges in the human brain.

Progrès au-delà de l’état des connaissances et impact potentiel prévu (y compris l’impact socio-économique et les conséquences sociétales plus larges du projet jusqu’à présent)

The results of DEEPFUNC advance the field of cognitive neuroscience in several ways. First, the large-scale analysis of category selectivity across hundreds of object categories, combined with data-driven voxel decomposition, challenges the traditional view of category-selective brain regions as discrete modules. By demonstrating that specialization is multidimensional and graded rather than binary, DEEPFUNC advances our understanding of functional organization in the human visual cortex. Second, the direct comparison between human brain representations and artificial neural networks revealed systematic divergences in how selectivity emerges. Identifying this dissociation provides concrete guidance for improving computational architectures and evaluation strategies in artificial intelligence, thereby strengthening the neuroscience–AI feedback loop. Third, DEEPFUNC delivers publicly available methodological infrastructure that supports reproducible research at the interface of neuroscience and AI. The open-source tools developed within the project standardize dataset management, causal analysis, and representational comparison, thereby lowering barriers for interdisciplinary research and strengthening methodological rigor across the field. In the longer term, the project has broader impact potential. Understanding how and why functional specialization emerges can inform the design of artificial systems that balance specialized and general-purpose processing. The framework is also relevant for developmental and clinical research, where altered specialization patterns are implicated in neurodevelopmental and perceptual disorders. Overall, DEEPFUNC contributes a computational and testable account of functional specialization, new large-scale analytical tools, and open infrastructure that advance both neuroscience and artificial intelligence research.