Periodic Reporting for period 4 - CODE4Vision (Computational Dissection of Effective Circuitry and Encoding in the Retina for Normal and Restored Vision)
Reporting period: 2021-12-01 to 2022-11-30
When light enters the eye, it falls onto a thin layer of neuronal tissue at the back of the eyeball – the retina. Here, light is taken up by photoreceptors and converted into electrical signals. These are then processed by an interconnected network of neurons and finally transformed into sequences of electrical pulses (so-called “action potentials” or “spikes”) in the output neurons of the retina, the retinal ganglion cells. The ganglion cells send this electrical activity to various downstream areas in the brain. Thus, all visual information of the outside world received by the brain must be encoded in the patterns of spikes that occur in the ganglion cells. How this activity represents visual information and how the neural network of the retina achieves this encoding of visual signals are among the fundamental questions about our sense of vision.
Moreover, understanding the representation of visual information in the spike patterns of retinal ganglion cells is also of concrete importance for the development of vision restoration therapies for patients suffering from loss of photoreceptors. About a million people worldwide are affected by inherited degeneration of photoreceptors, a condition known as retinitis pigmentosa, which results in a progressive loss of vision, ultimately leading to blindness. No cure is currently available to prevent or halt the photoreceptor degeneration. Thus, much effort is devoted towards developing artificial ways of restoring vision. This may happen through retinal prostheses that electrically activate surviving nerve cells in the retina via an inserted electronic chip or through restoring light sensitivity via inserted light-sensitive proteins. In both cases, best functional restoration of vision is to be expected when the artificial stimulation can recreate the natural activity patterns as closely as possible. In order to find appropriate stimulation strategies, mathematical models that describe how the retina translates natural as well as artificial stimulation into activity patterns will be a valuable tool.
In the CODE4Vision project, we therefore aim at understanding visual signal processing in the retinal network under natural visual stimulation as well as under artificial stimulation through inserted light-sensitive proteins. To do so, we record the spikes of retinal ganglion cells in isolated retinal tissue with miniature electrodes while stimulating the retina with various light patterns. A key step for analyzing how the retina processes these light patterns is to identify the collection of inputs that a given ganglion cell receives from other neurons in the retinal network. The individual inputs are not readily observable, and we therefore develop mathematical techniques, based on methods from computational statistics and machine learning, in order to infer, for example, how many excitatory neurons send their signals to a given ganglion cell and where these input neurons are located. Based on the identified input layout, we can then investigate the signal transmission between the input neurons and the ganglion cells and build mathematical models that capture how the retina responds to natural as well as artificial stimulation.
Moreover, understanding the representation of visual information in the spike patterns of retinal ganglion cells is also of concrete importance for the development of vision restoration therapies for patients suffering from loss of photoreceptors. About a million people worldwide are affected by inherited degeneration of photoreceptors, a condition known as retinitis pigmentosa, which results in a progressive loss of vision, ultimately leading to blindness. No cure is currently available to prevent or halt the photoreceptor degeneration. Thus, much effort is devoted towards developing artificial ways of restoring vision. This may happen through retinal prostheses that electrically activate surviving nerve cells in the retina via an inserted electronic chip or through restoring light sensitivity via inserted light-sensitive proteins. In both cases, best functional restoration of vision is to be expected when the artificial stimulation can recreate the natural activity patterns as closely as possible. In order to find appropriate stimulation strategies, mathematical models that describe how the retina translates natural as well as artificial stimulation into activity patterns will be a valuable tool.
In the CODE4Vision project, we therefore aim at understanding visual signal processing in the retinal network under natural visual stimulation as well as under artificial stimulation through inserted light-sensitive proteins. To do so, we record the spikes of retinal ganglion cells in isolated retinal tissue with miniature electrodes while stimulating the retina with various light patterns. A key step for analyzing how the retina processes these light patterns is to identify the collection of inputs that a given ganglion cell receives from other neurons in the retinal network. The individual inputs are not readily observable, and we therefore develop mathematical techniques, based on methods from computational statistics and machine learning, in order to infer, for example, how many excitatory neurons send their signals to a given ganglion cell and where these input neurons are located. Based on the identified input layout, we can then investigate the signal transmission between the input neurons and the ganglion cells and build mathematical models that capture how the retina responds to natural as well as artificial stimulation.
In order to record spikes from large populations of retinal ganglion cells, we have established a new recording method in the lab where we use semiconductor chips with more than 4,000 recording sites. We have applied this to simultaneously obtain data from several hundred ganglion cells in retinal tissue obtained from mice or salamanders. We have also developed a novel computational method to identify the layout of excitatory inputs that feed into the recorded ganglion cells.
Based on this information about input signals into ganglion cells, we have furthermore studied how signals are transformed when a ganglion cell pools visual signals under natural stimulation. Particular questions here have been how ganglion cells combine signals over space or over different chromatic channels and how they adapt when visual contrast changes in part of a scene, such as when objects shift around or when the illumination conditions change. We have found that some specific cells combine spatial or chromatic signals linearly, others perform complex computational operations. With respect to adaptation, certain cells tend to represent primarily the object with strongest contrast in their field of view, whereas others remain sensitive to additional objects, thus diversifying the information that is transmitted by different types of ganglion cells to the rest of the brain. When considering stimuli with simulated eye movements, we have observed that it is not the responses of individual ganglion cells that are informative about the motion direction, but that the information rather lies in the relative activity among multiple ganglion cells.
Finally, we have investigated how artificially inserted light-sensitive proteins can be used to recreate natural activity in retinas with degenerated photoreceptors. We have compared activity of ganglion cells under natural and artificial stimulation and found that certain cell-specific response properties, like the temporal extent of evoked activity, are retained under artificial stimulation, whereas others, like the timing of response onsets and the sensitivity to complex stimuli, are altered and require transformations of the stimulus to become more natural.
The work has led to a number of high-profile publications, including in the journals Neuron, Nature Communications, and Trends in Neurosciences. In addition, we have made several electrophysiological datasets and analysis software packages publicly available and presented the results at international conferences as well as at public outreach events.
Based on this information about input signals into ganglion cells, we have furthermore studied how signals are transformed when a ganglion cell pools visual signals under natural stimulation. Particular questions here have been how ganglion cells combine signals over space or over different chromatic channels and how they adapt when visual contrast changes in part of a scene, such as when objects shift around or when the illumination conditions change. We have found that some specific cells combine spatial or chromatic signals linearly, others perform complex computational operations. With respect to adaptation, certain cells tend to represent primarily the object with strongest contrast in their field of view, whereas others remain sensitive to additional objects, thus diversifying the information that is transmitted by different types of ganglion cells to the rest of the brain. When considering stimuli with simulated eye movements, we have observed that it is not the responses of individual ganglion cells that are informative about the motion direction, but that the information rather lies in the relative activity among multiple ganglion cells.
Finally, we have investigated how artificially inserted light-sensitive proteins can be used to recreate natural activity in retinas with degenerated photoreceptors. We have compared activity of ganglion cells under natural and artificial stimulation and found that certain cell-specific response properties, like the temporal extent of evoked activity, are retained under artificial stimulation, whereas others, like the timing of response onsets and the sensitivity to complex stimuli, are altered and require transformations of the stimulus to become more natural.
The work has led to a number of high-profile publications, including in the journals Neuron, Nature Communications, and Trends in Neurosciences. In addition, we have made several electrophysiological datasets and analysis software packages publicly available and presented the results at international conferences as well as at public outreach events.
A breakthrough methodological development of the project has been the discovery how the layout of presynaptic excitatory bipolar cells can be inferred from the spiking activity of individual recorded retinal ganglion cells. This allows new insights into the connectivity patterns inside the retina as well as new types of model descriptions for how visual stimulation is translated into patterns of spikes by the retina.
In addition, the project has led to several unexpected physiological findings about signal processing in the retina under natural stimulation. In particular, we have observed that retinal bipolar cells are not simple, linear encoders of visual contrast, but can perform nonlinear computations. Furthermore, we have demonstrated that visual motion corresponding to eye movements makes the motion encoding of individual direction-selective ganglion cells ambiguous, but considering a population of cells can disambiguate the code. We have also shown that eye movements can induce correlated activity between ganglion cells in a way that violates the classical theory of efficient coding in the retina, but opens the possibility for a population-based encoding of specific visual features.
In addition, the project has led to several unexpected physiological findings about signal processing in the retina under natural stimulation. In particular, we have observed that retinal bipolar cells are not simple, linear encoders of visual contrast, but can perform nonlinear computations. Furthermore, we have demonstrated that visual motion corresponding to eye movements makes the motion encoding of individual direction-selective ganglion cells ambiguous, but considering a population of cells can disambiguate the code. We have also shown that eye movements can induce correlated activity between ganglion cells in a way that violates the classical theory of efficient coding in the retina, but opens the possibility for a population-based encoding of specific visual features.