Periodic Reporting for period 3 - BACKUP (Unveiling the relationship between brain connectivity and function by integrated photonics)
Période du rapport: 2021-11-01 au 2023-04-30
BACKUP addresses the fundamental question of which is the role of neuron activity and plasticity in information elaboration and storage in the brain. Within an interdisciplinary team, BACKUP develops a hybrid neuromorphic computing platform. Integrated photonic circuits are interfaced to both electronic circuits and neuronal circuits (in vitro experiments) to emulate brain functions and develop schemes able to supplement (backup) neuronal functions. The photonic network is based on reconfigurable matrices of nonlinear nodes formed by microring resonators, which enter a nonlinear regime by positive optical feedback. These networks resemble human brain. BACKUP will push this analogy further by interfacing the photonic network with neurons making hybrid network. By using optogenetics, we control the synaptic strengthening and the neuron activity. Deep learning algorithms model the biological network functionality, initially within a separate artificial network and, then, in an integrated hybrid artificial-biological network.
We aim at:
Developing a photonic integrated reservoir-computing network (RCN);
Developing dynamic memories in photonic integrated circuits using RCN;
Developing hybrid interfaces between a neuronal network and a photonic integrated circuit;
Developing a hybrid electronic, photonic and biological network that computes jointly;
Addressing neuronal network activity by photonic RCN to simulate in vitro memory storage and retrieval;
Elaborating the signal from RCN and neuronal circuits in order to cope with plastic changes in pathological brain conditions such as amnesia and epilepsy.
The long-term vision is that hybrid neuromorphic photonic networks will:
clarify the way brain thinks
compute beyond von Neumann
control and supplement specific neuronal functions
We aim at:
Developing a photonic integrated reservoir-computing network (RCN);
Developing dynamic memories in photonic integrated circuits using RCN;
Developing hybrid interfaces between a neuronal network and a photonic integrated circuit;
Developing a hybrid electronic, photonic and biological network that computes jointly;
Addressing neuronal network activity by photonic RCN to simulate in vitro memory storage and retrieval;
Elaborating the signal from RCN and neuronal circuits in order to cope with plastic changes in pathological brain conditions such as amnesia and epilepsy.
The long-term vision is that hybrid neuromorphic photonic networks will:
clarify the way brain thinks
compute beyond von Neumann
control and supplement specific neuronal functions
During the first half of the project we developed the experimental platform and we performed a series of experiments with the aim to confirm the starting hypothesis of the project, i.e. it is possible to make a biological and an artificial network compute together. We proceeded along two parallel tracks: A) development of photonic neural networks and b) development of biological neuronal networks. In the second half of the project we aim to merge these two tracks to enable joint computation.
Specifically, we have developed for track A:
1. an experimental setup to test the photonic neural networks under dynamical operation (up to 40 Gbps) (see figure setup)
2. a comprehensive modelling and simulation framework for single and array of nonlinear microring resonators. Model and simulation have been experimentally verified on silicon based microring resonator. The simulation program is freely accessible ( https://gitlab.com/erc-backup/precise).
3. a reservoir computing network based on nonlinear microring resonator and time multiplexing able to perform both digital as well as analog operations
4. a silicon photonics complex perceptron based on integrated time delay lines which shows both memory and nonlinearity. High speed operation (10 Gbps) have been demonstrated. Image of the chip is shown in figure chip and neurons
5. a concept, design and chip to all-optically mitigate nonlinearities in optical fiber with a neural network (a patent has been filed). Validation of the design is pending due to delay in chip manufacturing.
6. complex electronic circuits with nonlinear elements to simulate the complex dynamics of array of photonic microring resonators
On the other side for track B, we have developed:
1. a comprehensive experimental setup to perform optogenetics biological experiments. This comprehends a biological laboratory to prepare the neuronal culture, a super-resolution spinning disk confocal optical microscope interfaced to a multielectrode array (MEA) system and a digital light projector system. Here, neuronal cultures are prepared (see figure chip and neurons) and measured under patterned light excitation.
2. optogenic tools to stimulate by light and to consequently activate only specific neurons in the neuronal network. To this end we have infected neurons with Chr and monitored neuron activity with Ca imaging or MEA recording after patterned light excitation. Clear evidences of neuronal network formation have been collected.
3. experiments that demonstrates that astrocytes gain control of synaptic strengthening and memory consolidation upon neuronal instruction. Taking advantage of super resolution microscopy coupled to spinning disk we visualize synapses enwrapped by astrocytes and we explore proBDNF trafficking that can contribute to memory formation and consolidation.
4. a photonic chip to point by point excite single neurons in the neuronal culture. The chip has been packaged with optical fibers in such a way that can be operated under the microscope (see image array in the microscope). Neuronal culture are plated on the chip in a sterile condition in order to perform experiments at different neuron ages. Experiments are undergoing.
We have organized an international workshop on reservoir computing (https://event.unitn.it/erc/) a meeting on the ethical implications of the research on biological and artificial neuron networks (https://r1.unitn.it/back-up/Focus%20on%20Ethics%20Biolaw%20Journal.pdf) and a summer school on neuromorphic photonics (https://event.unitn.it/nmp2021/). A number of ESR (both PhD students and post-docs) is trained within the project.
Specifically, we have developed for track A:
1. an experimental setup to test the photonic neural networks under dynamical operation (up to 40 Gbps) (see figure setup)
2. a comprehensive modelling and simulation framework for single and array of nonlinear microring resonators. Model and simulation have been experimentally verified on silicon based microring resonator. The simulation program is freely accessible ( https://gitlab.com/erc-backup/precise).
3. a reservoir computing network based on nonlinear microring resonator and time multiplexing able to perform both digital as well as analog operations
4. a silicon photonics complex perceptron based on integrated time delay lines which shows both memory and nonlinearity. High speed operation (10 Gbps) have been demonstrated. Image of the chip is shown in figure chip and neurons
5. a concept, design and chip to all-optically mitigate nonlinearities in optical fiber with a neural network (a patent has been filed). Validation of the design is pending due to delay in chip manufacturing.
6. complex electronic circuits with nonlinear elements to simulate the complex dynamics of array of photonic microring resonators
On the other side for track B, we have developed:
1. a comprehensive experimental setup to perform optogenetics biological experiments. This comprehends a biological laboratory to prepare the neuronal culture, a super-resolution spinning disk confocal optical microscope interfaced to a multielectrode array (MEA) system and a digital light projector system. Here, neuronal cultures are prepared (see figure chip and neurons) and measured under patterned light excitation.
2. optogenic tools to stimulate by light and to consequently activate only specific neurons in the neuronal network. To this end we have infected neurons with Chr and monitored neuron activity with Ca imaging or MEA recording after patterned light excitation. Clear evidences of neuronal network formation have been collected.
3. experiments that demonstrates that astrocytes gain control of synaptic strengthening and memory consolidation upon neuronal instruction. Taking advantage of super resolution microscopy coupled to spinning disk we visualize synapses enwrapped by astrocytes and we explore proBDNF trafficking that can contribute to memory formation and consolidation.
4. a photonic chip to point by point excite single neurons in the neuronal culture. The chip has been packaged with optical fibers in such a way that can be operated under the microscope (see image array in the microscope). Neuronal culture are plated on the chip in a sterile condition in order to perform experiments at different neuron ages. Experiments are undergoing.
We have organized an international workshop on reservoir computing (https://event.unitn.it/erc/) a meeting on the ethical implications of the research on biological and artificial neuron networks (https://r1.unitn.it/back-up/Focus%20on%20Ethics%20Biolaw%20Journal.pdf) and a summer school on neuromorphic photonics (https://event.unitn.it/nmp2021/). A number of ESR (both PhD students and post-docs) is trained within the project.
BACKUP represents the first attempt to merge photonics and neuronal network in a single coherent device. Therefore in the first part of the project we concentrated on three different aspects. First, we setup and finalize the techniques and procedures to produce neuronal networks that can be addressed by patterned light. Second, we develop a modeling of the photonic chip in order to design and simulate photonic neural networks. Third, we setup and finalize the design and the validation of photonic neural networks on chip. The activity in the second half of the project will be thus concentrated on making the two platforms working together. To this aim we will develop procedure to write optically and read electronically engrams in the neuronal culture, we will develop high complexity photonic neural network, we will interface these photonic neural networks with the neuronal culture and, finally, we will address in-vitro neurological disorder as epilepsy and amnesia.
On the biological side, we were able to setup a complex experimental apparatus where both optical imaging and electrophysiological recordings are integrated with a light patterned excitation source. This last is based on a custom-built digital light projector for top excitation or on a integrated photonic chip where scatters allow to excite directly the top lying neuorons. With this setup we demonstrate the formation of network of neurons elucidating the role of the astrocytes in the formation of engrams.
On the biological side, we were able to setup a complex experimental apparatus where both optical imaging and electrophysiological recordings are integrated with a light patterned excitation source. This last is based on a custom-built digital light projector for top excitation or on a integrated photonic chip where scatters allow to excite directly the top lying neuorons. With this setup we demonstrate the formation of network of neurons elucidating the role of the astrocytes in the formation of engrams.