Adaptive functions of visual systems

Información del proyecto

AdaptiveVision

Identificador del acuerdo de subvención: 101045003

DOI

10.3030/101045003

Fecha de la firma de la CE 15 Septiembre 2022

Fecha de inicio 1 Abril 2023

Fecha de finalización 31 Marzo 2028

Financiado con arreglo a

• European Research Council (ERC)

Coste total

€ 1 999 613,00

Aportación de la UE

€ 1 999 613,00

Coordinado por

JOHANNES GUTENBERG-UNIVERSITAT MAINZ
Germany

Periodic Reporting for period 1 - AdaptiveVision (Adaptive functions of visual systems)

Período documentado: 2023-04-01 hasta 2025-09-30

Resumen del contexto y de los objetivos generales del proyecto

The processing of visual information allows humans, animals, and computer-vision based machines to navigate the world. All visual systems face common challenges when the world rapidly changes. Such changes are often generated by an animal’s own movement. Self-motion for example causes fast changes in illumination and generates global motion patterns on the eye, due to the movement of the world relative to the observer. Diverse visual systems face these common challenges but must also deal with important differences. First, animals experience different environments. Second, animals show different types of behavior, such as walking or flying, and behavior will alter the visual cues that the animal encounters. The goal of AdaptiveVision is to first understand common principles of visual system function, and to then work out how diverse visual systems adapt to specific environmental and behavioral constraints. To achieve this, AdaptiveVision will study two essential visual computations, the robust estimation of contrast in dynamically changing environments, and the encoding of global motion cues generated by self-motion.

Trabajo realizado desde el comienzo del proyecto hasta el final del período abarcado por el informe y los principales resultados hasta la fecha

We have indentified how flies stably process contrast n visual circuitry downstream of photoreceptors such that it remains unaffected by rapid changes in luminance. Using in vivo 2-photon imaging of neural responses in a living fly who was exposed to visual stimuli, we showed that neurons Tm9 and Tm1 in the visual OFF pathway are the first to exhibit luminance-invariant responses, unlike Tm2 and Tm4. We worked out the biophysical mechanisms and demonstrated that an inhibitory, glutamatergic mechanism is necessary this rapid luminance gain control in Tm9, but not Tm1. We combined experimental and theoretical approaches to show that local spatial pooling of luminance is required for luminance-invariant visual processing. Thus, the fly visual system has implemented a mechanisms by which contrast signals are corrected by a luminance signal that explores locally around the point at which contrast must be computed. This work is now published in Gür et al. 2024, Nature Communications.
We have also done extensive work on the recently released full adult fly brain connectome, and are currently exploring connectome datasets to understand the basis for global motion processing. The basis for our current work is published in Cornean et al. 2024, Nature Communications, and we have contributed to the release of the FlyWire connectome, as part of the FlyWire consortium. This is published in Schlegel et al. 2024 Nature, and Dorkenwald et al. 2024 Nature.

Avances que van más allá del estado de la técnica e impacto potencial esperado (incluida la repercusión socioeconómica y las implicaciones sociales más amplias del proyecto hasta la fecha)

This project deals with unsolved problems computing the most fundamental feature in vision - contrast and motion cues – in challenging conditions. These can be rapid changes in background luminance or the presence of global motion patterns caused by the object's own movement. These are challenges that no only flies face, but all visually guided animals including humans, but also all devices relying on machine vision, or artifical neural networks that compute information from dynamic visual scenes.
In an infamous example in the development of autonomously-driving cars, the human safety driver of the vehicle died because the car’s camera-based vision system could not identify a suddenly appearing bright truck against a bright background.

Although the self-driving car industry has found a solution to this problem by including additional radar- or lidar-based technology, it is striking that our eyes are suited to accommodate these apparently challenging conditions that are common to natural scenes. Our eyes (and the eyes of other animals) can do what computer-vision-based devices fail to do and perform a luminance-invariant computation of contrast at rapid time scales, allowing us to identify both clouds in the sky and trees in the dark forest, when facing a scenic view. Here we aim to learn from the eyes of animals about how contrast and global motion cues can be computed stably, especially when we are facing challenging, rapidly changing conditions.

summary-figure-sr1-01.png