Final Report Summary - GRASP CONTROL & BMI (Grasp-Related Neuronal Activity in Monkey and Human and Its Applicability in BMI)
The complexity and precision of human hand movements are obvious to all, but the neurological control of skilled hand function is not well understood. The loss of this control has devastating consequences for human daily life, and restoring it, and thereby bringing relief to the patient’s condition should be an important aim of biomedical science.
The major objective of this proposal was to make an interdisciplinary investigation of both the neuronal activity controlling skilled hand movements and how this activity could be used in a Brain-Machine Interface (BMI) to control hand postures suitable for grasp of different everyday objects. BMIs translate neuronal activity of the brain into commands driving a machine and, thereby, bypass interrupted signalling pathways (e.g. spinal cord injury) or replace the effector (e.g. amputation or accident affecting the hand). Their applications are mainly seen in the field of rehabilitation and medical care for paralyzed patients to restore social interaction or movement capabilities. For example, spinal injury patients rank the loss of skilled grasp as one of the most debilitating features of their injury (Anderson, 2004). Thus, grasp BMIs are of great importance and highly desirable, but not yet readily available.
Within this project, we investigated the reliability and stability of grasp-related neuronal activity recorded intra-cortically in the motor areas; assessed how accurately different grasp types can be decoded using spiking activity and local field potentials; investigated the spatial organization of grasp-related neuronal activity in the motor cortex and studied whether grasp-specific properties of neuronal signals change with changes in the context of grasp.
Our premise was that design of a useful grasp BMI could benefit directly from a better understanding of the neurophysiological basis of skilled grasp (e.g. How are grasping movements controlled?) and that there would be mutual benefit for neurophysiological research from BMI (e.g. What information can be extracted from different brain areas?).
As originally proposed for this project, we have recorded intracortical single-unit activity (SUA) and local field potentials (LFP) through multiple electrodes in macaque primary motor cortex (M1), ventral pre-motor cortex (PMv) and boundary areas between M1 and PMv.
Signals in these areas were acquired using acute recordings (single electrodes: Eckhorn microdrives, Thomas Recording; multiple electrodes in the vertical dimension: U-probe, Plexon) and chronic recordings (multi-electrode arrays: floating micro arrays, Microprobes).
We have collected data from three macaques (acute recordings: three hemispheres; chronic recordings: two hemispheres) and humans while they viewed, reached, grasped, displaced, held and released different objects, before returning their hand to a start position.
Two different experimental setups were used for this task: one allowing investigation of six different grasps and another to investigate three different grasps. The latter setup allowed us to record neuronal activity while the subjects themselves executed reach-to-grasp movements and while they observed the same actions performed by a human experimenter, i.e. during both action execution and observation.
These setups also enabled us to record two sessions in which we investigated the changes in neuronal activity introduced when one of the familiar objects was replaced by an object (grasp) that the subject had not seen or manipulated before.
In order to carry out these investigations a number of preparatory procedures and tests needed to be carried out. These included MRI scans, surgical implantation of headpieces and of electrodes in the pyramidal tract in order to identify pyramidal tract neurons via antidromic stimulation and collision. We also implanted electrodes to record the activity of up to 16 hand and arm muscles (EMG). EMG signals have been transmitted either via a direct, wired connection or more recently with a wireless, telemetric implant.
During the experiments themselves, in addition to the recordings, we carried out functional mapping using intracortical microstimulation (ICMS) and identified mirror neurons using standard behavioural tests.
In addition to the objectives and tasks in the original proposal, we have investigated the LFP during both movement execution and movement observation (as a way of assessing the activity of the mirror neuron system at the population level), used reversible inactivation (Muscimol) to investigate how the network is affected on the level of LFPs and performed transcranial magnetic stimulation during reach-to-grasp movements. We also investigated the effects of spiking activity on the analysis of LFPs both in simulations using real data and in-vivo by injecting currents into the active brain.
Moreover, we have been performing extensive analyses of the recorded data and have been summarising the results in paper, manuscripts, talks and conference abstracts.
The work and data acquired within this project led to six collaborative projects (Newcastle University, University College London, Aix-Marseille University) of which four are ongoing.
We found that the type of grasp can be inferred with high accuracy from intracortical neuronal activity. Activity in M1 allows for the highest decoding performance, followed by that of PMv and the area between M1 and PMv. Compared to M1 alone, combining signals from M1 and PMv increases decoding performance but not substantially. Although the correlation of LFP modulations recorded simultaneously at different depths (U-probe) is very high for neighbouring electrodes, it can be negative for most distant electrodes (superficial vs. deep layer). The strength of modulation and correlation across electrodes depends on both the object grasped and depth in the cortex. Within M1 there seems to be a region (close to the central sulcus) whose neuronal activity allows for highest decoding accuracy. This region is deeper (2-4 mm) than regions reachable with the widely used Utah array (~1 mm deep), which may have implications for both human and monkey studies using these arrays. Decoding performance is higher for recording sites at which ICMS effects at hand/arm muscles were detected. Spiking activity of pyramidal tract neurons and the LFPs recorded in their vicinity allowed for higher decoding performance than the same signals recorded from and next to unidentified neurons.
The impedance of chronically implanted electrodes has decreased from initially around 1MΩ to around 0.7MΩ. Despite this fall, signal quality has remained high over a long period (>14 months). Spiking activity of single neurons could be detected from shortly after array implantation until the end of the recordings. The yield of well-identified spikes from single neurons was surprisingly low. LFP modulations could be detected at almost all recordings sites within the arrays.
Low-frequency LFPs in both PMv and M1 were modulated during both movement execution and observation, which corroborates the genuine neuronal origin of this signal component. No muscle activity was detected during observation; this is an essential control for checking that there are no covert movements, and establishing that the modulated activity can genuinely be ascribed to the mirror neuron system. While these LFPs modulated only during dynamic periods, beta modulated during both dynamic and static periods. Our results indicated some degree of non-overlap in the functional neuronal networks for execution and observation.
We revealed how spike amplitude, spike duration, firing rate and noise statistic influence the extent to which spikes contaminate the analysis of LFPs. Contamination varies with these parameters and can affect LFPs down to around 10 Hz; below this it is theoretically possible but unlikely. LFP frequencies up to the (high-) gamma band can remain unaffected but signals above must always be carefully analysed. We proposed a method to reveal modulations in spectrograms, which also allows the detection of spike contamination, and provide a systematic guide to assess spike contamination of intra-cortical LFPs.
In addition to the results reported above, we expect that our ongoing analysis will reveal further findings regarding the spatial distribution and long-term stability of grasp specificity, the neuronal representation of grasps, the function of low-frequency LFPs and how spiking activity influences LFPs in-vivo.
The findings of this basic research project have broadened our understanding of grasp control and provided a guide as to how grasp BMIs can be improved for better rehabilitation and medical care for paralyzed patients; with the aim to restore social interaction or movement capabilities. We have contributed answers to fundamental research questions regarding the function of LFPs in movement control, neuronal representation of grasp, LFPs within the concept of the mirror system and the effect of spiking activity on the analysis and interpretation of LFPs. These findings have already triggered further research and we hope and expect they will prove beneficial for helping paralyzed patients.
The major objective of this proposal was to make an interdisciplinary investigation of both the neuronal activity controlling skilled hand movements and how this activity could be used in a Brain-Machine Interface (BMI) to control hand postures suitable for grasp of different everyday objects. BMIs translate neuronal activity of the brain into commands driving a machine and, thereby, bypass interrupted signalling pathways (e.g. spinal cord injury) or replace the effector (e.g. amputation or accident affecting the hand). Their applications are mainly seen in the field of rehabilitation and medical care for paralyzed patients to restore social interaction or movement capabilities. For example, spinal injury patients rank the loss of skilled grasp as one of the most debilitating features of their injury (Anderson, 2004). Thus, grasp BMIs are of great importance and highly desirable, but not yet readily available.
Within this project, we investigated the reliability and stability of grasp-related neuronal activity recorded intra-cortically in the motor areas; assessed how accurately different grasp types can be decoded using spiking activity and local field potentials; investigated the spatial organization of grasp-related neuronal activity in the motor cortex and studied whether grasp-specific properties of neuronal signals change with changes in the context of grasp.
Our premise was that design of a useful grasp BMI could benefit directly from a better understanding of the neurophysiological basis of skilled grasp (e.g. How are grasping movements controlled?) and that there would be mutual benefit for neurophysiological research from BMI (e.g. What information can be extracted from different brain areas?).
As originally proposed for this project, we have recorded intracortical single-unit activity (SUA) and local field potentials (LFP) through multiple electrodes in macaque primary motor cortex (M1), ventral pre-motor cortex (PMv) and boundary areas between M1 and PMv.
Signals in these areas were acquired using acute recordings (single electrodes: Eckhorn microdrives, Thomas Recording; multiple electrodes in the vertical dimension: U-probe, Plexon) and chronic recordings (multi-electrode arrays: floating micro arrays, Microprobes).
We have collected data from three macaques (acute recordings: three hemispheres; chronic recordings: two hemispheres) and humans while they viewed, reached, grasped, displaced, held and released different objects, before returning their hand to a start position.
Two different experimental setups were used for this task: one allowing investigation of six different grasps and another to investigate three different grasps. The latter setup allowed us to record neuronal activity while the subjects themselves executed reach-to-grasp movements and while they observed the same actions performed by a human experimenter, i.e. during both action execution and observation.
These setups also enabled us to record two sessions in which we investigated the changes in neuronal activity introduced when one of the familiar objects was replaced by an object (grasp) that the subject had not seen or manipulated before.
In order to carry out these investigations a number of preparatory procedures and tests needed to be carried out. These included MRI scans, surgical implantation of headpieces and of electrodes in the pyramidal tract in order to identify pyramidal tract neurons via antidromic stimulation and collision. We also implanted electrodes to record the activity of up to 16 hand and arm muscles (EMG). EMG signals have been transmitted either via a direct, wired connection or more recently with a wireless, telemetric implant.
During the experiments themselves, in addition to the recordings, we carried out functional mapping using intracortical microstimulation (ICMS) and identified mirror neurons using standard behavioural tests.
In addition to the objectives and tasks in the original proposal, we have investigated the LFP during both movement execution and movement observation (as a way of assessing the activity of the mirror neuron system at the population level), used reversible inactivation (Muscimol) to investigate how the network is affected on the level of LFPs and performed transcranial magnetic stimulation during reach-to-grasp movements. We also investigated the effects of spiking activity on the analysis of LFPs both in simulations using real data and in-vivo by injecting currents into the active brain.
Moreover, we have been performing extensive analyses of the recorded data and have been summarising the results in paper, manuscripts, talks and conference abstracts.
The work and data acquired within this project led to six collaborative projects (Newcastle University, University College London, Aix-Marseille University) of which four are ongoing.
We found that the type of grasp can be inferred with high accuracy from intracortical neuronal activity. Activity in M1 allows for the highest decoding performance, followed by that of PMv and the area between M1 and PMv. Compared to M1 alone, combining signals from M1 and PMv increases decoding performance but not substantially. Although the correlation of LFP modulations recorded simultaneously at different depths (U-probe) is very high for neighbouring electrodes, it can be negative for most distant electrodes (superficial vs. deep layer). The strength of modulation and correlation across electrodes depends on both the object grasped and depth in the cortex. Within M1 there seems to be a region (close to the central sulcus) whose neuronal activity allows for highest decoding accuracy. This region is deeper (2-4 mm) than regions reachable with the widely used Utah array (~1 mm deep), which may have implications for both human and monkey studies using these arrays. Decoding performance is higher for recording sites at which ICMS effects at hand/arm muscles were detected. Spiking activity of pyramidal tract neurons and the LFPs recorded in their vicinity allowed for higher decoding performance than the same signals recorded from and next to unidentified neurons.
The impedance of chronically implanted electrodes has decreased from initially around 1MΩ to around 0.7MΩ. Despite this fall, signal quality has remained high over a long period (>14 months). Spiking activity of single neurons could be detected from shortly after array implantation until the end of the recordings. The yield of well-identified spikes from single neurons was surprisingly low. LFP modulations could be detected at almost all recordings sites within the arrays.
Low-frequency LFPs in both PMv and M1 were modulated during both movement execution and observation, which corroborates the genuine neuronal origin of this signal component. No muscle activity was detected during observation; this is an essential control for checking that there are no covert movements, and establishing that the modulated activity can genuinely be ascribed to the mirror neuron system. While these LFPs modulated only during dynamic periods, beta modulated during both dynamic and static periods. Our results indicated some degree of non-overlap in the functional neuronal networks for execution and observation.
We revealed how spike amplitude, spike duration, firing rate and noise statistic influence the extent to which spikes contaminate the analysis of LFPs. Contamination varies with these parameters and can affect LFPs down to around 10 Hz; below this it is theoretically possible but unlikely. LFP frequencies up to the (high-) gamma band can remain unaffected but signals above must always be carefully analysed. We proposed a method to reveal modulations in spectrograms, which also allows the detection of spike contamination, and provide a systematic guide to assess spike contamination of intra-cortical LFPs.
In addition to the results reported above, we expect that our ongoing analysis will reveal further findings regarding the spatial distribution and long-term stability of grasp specificity, the neuronal representation of grasps, the function of low-frequency LFPs and how spiking activity influences LFPs in-vivo.
The findings of this basic research project have broadened our understanding of grasp control and provided a guide as to how grasp BMIs can be improved for better rehabilitation and medical care for paralyzed patients; with the aim to restore social interaction or movement capabilities. We have contributed answers to fundamental research questions regarding the function of LFPs in movement control, neuronal representation of grasp, LFPs within the concept of the mirror system and the effect of spiking activity on the analysis and interpretation of LFPs. These findings have already triggered further research and we hope and expect they will prove beneficial for helping paralyzed patients.
