Periodic Reporting for period 2 - NeuraViPeR (Neural Active Visual Prosthetics for Restoring Function)
Okres sprawozdawczy: 2022-03-01 do 2023-08-31
In the project, we are developing innovative approaches for stimulation with high-electrode-count interfacing with the visual cortex. The work includes the creation of thin flexible probes that cause minimal tissue damage; new electrode coatings that will be stable even with long-term repeated electrical stimulation; and novel microchip methods for combining online channeling of the stimulation currents to many thousands of electrodes. It also combines stimulation with the monitoring of neuronal activity in higher cortical areas. We are also developing new deep learning algorithms that transform the camera footage into stimulation patterns for the cortex and that use feedback on recorded brain states and eye tracking to improve perception in a closed-loop approach. The software algorithms will be translated onto low-latency, power-efficient neuromorphic deep learning hardware, to create a neuroprosthesis system that is robust, and portable.
We have designed, simulated and fabricated the first version of the recording and stimulation CMOS ASIC comprised of 8 stimulation units, 128 output stages and 64 recording channels. The stimulation units can flexibility be programmed in terms of pulse amplitude, frequency, duration and polarity. The recording channels provide low noise and low power amplification, filtering and digitization of multi-unit neural activity.
We have completed the design of the convolutional neural network hardware accelerator, VPDNN, that supports the required features of the phosphene stimulation neural network. Its performance (latency, power) was compared to implementation on other embedded devices. We also applied weight pruning and bit quantization of parameters to the stimulation network corresponding to the bit precision supported by VPDNN. The system works with inputs from a camera and the network output was transmitted successfully to recording and stimulation CMOS ASIC.
For validating the NeuraViPeR probe technology, mice were implanted with a flexible polyimide probes in the primary visual cortex (area V1). Microstimulation through the electrodes successfully evoked a behavioral response in mice trained on a go/no-go stimulation detection task. We were able to monitor the stimulation threshold over up to 13 months revealing a stable perceptual threshold. Currently we are focusing on the ex-vivo evaluation of probe biocompatibility.
On the software side we developed a Reinforcement Learning (RL) based network to optimize phosphene generation taking into account task-specific constraints. This framework is used to train novel deep networks for stimulus generation in an end-to-end fashion to optimize phosphene patterns and solve multi-stage navigation-based tasks in virtual environments. Algorithms were developed which maintain performance in the presence of perturbations in dynamical systems. An in-silico framework to train a prosthesis controller via RL was designed to restore function in an impaired neural system using dynamical systems and recurrent neural networks (RNN) to formulate the components of a brain-machine interaction. An end-to-end framework including a biologically plausible phosphene simulator was used to learn safe stimulation parameters (amplitude, pulse width and frequency) for phosphene generation optimized for naturalistic vision settings via real-life navigation videos.
We have conducted simultaneous recordings and stimulation experiments over a 6-month period in the primary visual cortex of three blind subjects. Our efforts have led to significant advancements in our protocols for selecting candidates for a cortical visual prosthesis. Throughout this process, we meticulously monitored the evolution of electrode impedance measurements, ensuring the stability and reliability of our electrodes over time. With each blind volunteer, we implemented protocols to evaluate both technical and functional aspects of using intracortical electrical stimulation as an interface for a visual neuroprosthesis.
These results have a significant scientific impact in the field and support the case for using penetrating microelectrodes. They also highlight a number of important unanswered questions that have to be solved before a cortical visual neuroprosthesis can be considered a viable clinical therapy or option. We expect that our results until the end of the project will allow us to advance the development of a technology that can enhance safety in navigation and provide greater confidence for individuals with profound blindness in many environments.