Periodic Reporting for period 3 - Natural BionicS (Natural Integration of Bionic Limbs via Spinal Interfacing)
Okres sprawozdawczy: 2022-06-01 do 2023-11-30
Missing a limb leads to dramatic impairments in the capacity to move and interact with the environment and to a substantial worsening in quality of life. This deficiency is also associated with a large portion of the sensory-motor cortex facing neural deafness. Missing or damaged limbs can be substituted by robotic limbs, connected to humans with neural interfacing.
The project NaturalBionicS aims at creating bio-connectors (compacted in a bio-hub) to access the spinal cord circuitries of patients with amputations so that bionic limbs can be controlled and sensed by the patient in a similar way as natural limbs. The core approach consists of utilising biological pathways of encoding and decoding neural information. This corresponds to the use of biological structures still present after the amputation and through which it is easier to communicate with the nervous system of the patient.
At this stage of the project, the progress towards the final goals has been substantial in each of the three main disciplines – neurosurgery, neural interfacing, robotics – as well as in the synergies between disciplines. The surgical research has demonstrated in pre-clinical models that it is possible to transfer multiple nerves previously controlling the missing limb in muscle tissue above the amputation. The muscle tissue is reinnervated and becomes electrically active so that the nerve activity can be measured from it. In this way, the reinnervated muscle becomes a “screen” of the activity of the nerves that were controlling the missing limb. With respect to other approaches, in which different nerves are transferred in different muscles, we demonstrated the possibility to hyper-reinnervate a single muscle, so that the muscle becomes a compact “neural image” of the control of the missing limb by the patient. Moreover, we demonstrated surgically that skin sensors can be reinnervated by the sensory nerves previously sensing the missing limb. For this purpose, a small skin flap is surgically located on top of the reinnervated muscle and the sensory fibers previously sensing the missing limb are transferred to this newly transplanted skin. In this way, mechanical or electrical elicitation of the reinnervated skin will be sensed as sensation coming from the missing limb.
The two reinnervation procedures (motor and sensory) establish the biological structures that allow us to extract information (motor) and to provide information (sensory) from and to the nervous system. We have then interfaced these structures with novel implantable arrays of electrodes to establish a bidirectional interface (from and to the nervous system). In doing so, we have demonstrated that the activity recorded from the reinnervated muscles with the novel implants can be decoded to infer the motor intent of the patient. Moreover, the activity of the multiple nerves can be separated into the individual sources (that is, into the individual nerve activity underlying specific motor tasks) by processing methods based on machine learning and artificial intelligence. This achievement (separation of nerve activity from the hyper-reinnervated muscle) confirms one of the highest-risk hypotheses we made at the beginning of the project.
Finally, the decoded neural activity has been mapped into robotic movements. A novel soft robotic hand controlled by commands that can be synergistically combined has been actuated by the neural information extracted from the bio hub. Interestingly, we were able to show that it is possible to match the neural activity corresponding to specific hand motions to the commands provided to the motors of the robotic hand. This matching has been possible by mapping the neural representation of a motor task to the kinematic posture of the hand. This allowed us to progress towards the very ambitious goal of matching the sensory-motor image of the missing limb emerging from the neural interfacing to the soft robotic arms/legs that will embed kinematic synergies and tactile-proprioceptive functions, intimately matched with the neural sensory-motor synergies extracted from the bio-hub.
The project has therefore achieved a full proof-of-concept of all its main foundational assumptions. The next phase will be dedicated to the clinical integration of the developed technologies for a full real-life demonstration of the achieved scientific results.
The project NaturalBionicS aims at creating bio-connectors (compacted in a bio-hub) to access the spinal cord circuitries of patients with amputations so that bionic limbs can be controlled and sensed by the patient in a similar way as natural limbs. The core approach consists of utilising biological pathways of encoding and decoding neural information. This corresponds to the use of biological structures still present after the amputation and through which it is easier to communicate with the nervous system of the patient.
At this stage of the project, the progress towards the final goals has been substantial in each of the three main disciplines – neurosurgery, neural interfacing, robotics – as well as in the synergies between disciplines. The surgical research has demonstrated in pre-clinical models that it is possible to transfer multiple nerves previously controlling the missing limb in muscle tissue above the amputation. The muscle tissue is reinnervated and becomes electrically active so that the nerve activity can be measured from it. In this way, the reinnervated muscle becomes a “screen” of the activity of the nerves that were controlling the missing limb. With respect to other approaches, in which different nerves are transferred in different muscles, we demonstrated the possibility to hyper-reinnervate a single muscle, so that the muscle becomes a compact “neural image” of the control of the missing limb by the patient. Moreover, we demonstrated surgically that skin sensors can be reinnervated by the sensory nerves previously sensing the missing limb. For this purpose, a small skin flap is surgically located on top of the reinnervated muscle and the sensory fibers previously sensing the missing limb are transferred to this newly transplanted skin. In this way, mechanical or electrical elicitation of the reinnervated skin will be sensed as sensation coming from the missing limb.
The two reinnervation procedures (motor and sensory) establish the biological structures that allow us to extract information (motor) and to provide information (sensory) from and to the nervous system. We have then interfaced these structures with novel implantable arrays of electrodes to establish a bidirectional interface (from and to the nervous system). In doing so, we have demonstrated that the activity recorded from the reinnervated muscles with the novel implants can be decoded to infer the motor intent of the patient. Moreover, the activity of the multiple nerves can be separated into the individual sources (that is, into the individual nerve activity underlying specific motor tasks) by processing methods based on machine learning and artificial intelligence. This achievement (separation of nerve activity from the hyper-reinnervated muscle) confirms one of the highest-risk hypotheses we made at the beginning of the project.
Finally, the decoded neural activity has been mapped into robotic movements. A novel soft robotic hand controlled by commands that can be synergistically combined has been actuated by the neural information extracted from the bio hub. Interestingly, we were able to show that it is possible to match the neural activity corresponding to specific hand motions to the commands provided to the motors of the robotic hand. This matching has been possible by mapping the neural representation of a motor task to the kinematic posture of the hand. This allowed us to progress towards the very ambitious goal of matching the sensory-motor image of the missing limb emerging from the neural interfacing to the soft robotic arms/legs that will embed kinematic synergies and tactile-proprioceptive functions, intimately matched with the neural sensory-motor synergies extracted from the bio-hub.
The project has therefore achieved a full proof-of-concept of all its main foundational assumptions. The next phase will be dedicated to the clinical integration of the developed technologies for a full real-life demonstration of the achieved scientific results.
The workplan has been followed closely and the activities are on time with respect to the plan (see Annex). The work progresses in the different areas are briefly summarised in the following while details are provided in the scientific publications:
- Surgery. An animal model of multiple nerve transfers into a single muscle has been developed and validated with extensive electrophysiological measurements. Multi-channel implanted EMG systems have been used in this animal model to decode the activity of motor neurons from the multiple innervating nerves (see progress in Interfacing below). Moreover, an animal model of skin graft and sensory reinnervation has also been developed and tested.
- Interfacing. A simulator of intramuscular electrical activity has been designed to generate synthetic signals modelling motor neuron activity as generated by multiple innervating nerves. Thin-film electrodes consisting of linear arrays of 40 electrodes have been designed and tested in the animal models described above. A two-dimensional array grid electrode has been designed and manufactured by the company Cortec for animal use. Real-time decomposition methods for high-density EMG have been developed, tested, and validated.
- Robotics. The main progress has been on the design, manufacturing and test of a soft robotic hand actuated by two kinematic synergies, which constitutes an important step forward in the development of limbs with natural interaction with the environment and requiring advanced control signals. Progresses have also been done in the design of robotic wrists and elbows and in variable stiffness actuators. Soft foot prostheses have also been finalised and preliminary tested in patients.
- Integration. A clinical bionic board has been established at the beginning of the project to initiate the integration of the activities and patient recruitment. The board consists of the project PIs and senior members of the three research teams. The clinical bionic board has met regularly at the Medical University of Vienna from the beginning of the project until March 2020, when personal meetings had to be interrupted due to the pandemic. Nonetheless, meetings continued to be held regularly in a remote setting. Interaction of the three teams has been extensive and has already determined the publication of 4 joint articles between PIs. Measures on patients by the three teams with integration of the technologies developed so far have been performed since the beginning of the project.
In addition to the progresses described above, all teams have further exploited the developed technologies in additional application areas. For example, the robotic hands have been applied in pioneering studies on stroke rehabilitation, the EMG decoding systems in the study of neonatal development and in spinal cord injury patients, and the surgical nerve transfer methods have been translated into a new paradigm for treating spasticity.
- Surgery. An animal model of multiple nerve transfers into a single muscle has been developed and validated with extensive electrophysiological measurements. Multi-channel implanted EMG systems have been used in this animal model to decode the activity of motor neurons from the multiple innervating nerves (see progress in Interfacing below). Moreover, an animal model of skin graft and sensory reinnervation has also been developed and tested.
- Interfacing. A simulator of intramuscular electrical activity has been designed to generate synthetic signals modelling motor neuron activity as generated by multiple innervating nerves. Thin-film electrodes consisting of linear arrays of 40 electrodes have been designed and tested in the animal models described above. A two-dimensional array grid electrode has been designed and manufactured by the company Cortec for animal use. Real-time decomposition methods for high-density EMG have been developed, tested, and validated.
- Robotics. The main progress has been on the design, manufacturing and test of a soft robotic hand actuated by two kinematic synergies, which constitutes an important step forward in the development of limbs with natural interaction with the environment and requiring advanced control signals. Progresses have also been done in the design of robotic wrists and elbows and in variable stiffness actuators. Soft foot prostheses have also been finalised and preliminary tested in patients.
- Integration. A clinical bionic board has been established at the beginning of the project to initiate the integration of the activities and patient recruitment. The board consists of the project PIs and senior members of the three research teams. The clinical bionic board has met regularly at the Medical University of Vienna from the beginning of the project until March 2020, when personal meetings had to be interrupted due to the pandemic. Nonetheless, meetings continued to be held regularly in a remote setting. Interaction of the three teams has been extensive and has already determined the publication of 4 joint articles between PIs. Measures on patients by the three teams with integration of the technologies developed so far have been performed since the beginning of the project.
In addition to the progresses described above, all teams have further exploited the developed technologies in additional application areas. For example, the robotic hands have been applied in pioneering studies on stroke rehabilitation, the EMG decoding systems in the study of neonatal development and in spinal cord injury patients, and the surgical nerve transfer methods have been translated into a new paradigm for treating spasticity.
In the first 18 months of the project, important scientific achievements have already been reached. This is documented in detail in the list of publications. The quality of the research and the substantial advance of the state of the art is clearly testified by the quality of the published research (which include papers in the New England Journal of Medicine, Nature Biomedical Engineering, Science Robotics, Science Advances, as well as core engineering Journals, such as Transactions of the IEEE). The research highlights are: the development of breakthrough methods for multiple nerve transfer into a single muscle to establish a high information transfer from the spinal cord to the target muscle (here used as bioscreen); the design of AI-based decoding algorithms that process muscle signals and extract the activity of the innervating nerve fibres; and the design of multi-synergy soft robotics. These developments have already been published in top Journals in the respective fields. It is expected that the next period of the project will see a much greater integration of the individual advances into highly innovative systems that combines the breakthroughs in the three disciplines.