Periodic Reporting for period 1 - Neuroprosthesis-UI (Neuroprosthesis user interface based on residual motor skills and muscle activity in persons with upper limb disabilities)
Okres sprawozdawczy: 2020-07-01 do 2022-06-30
For the duration of this project, I have focused on advancing in the development of a hybrid neuroprosthesis for persons with upper limb disabilities due to neural deficiencies. This device is intended for the execution of daily living activities in the context of rehabilitation. In particular, I have developed a user interface to control such device , in spite of users disabilities, by exploring their residual movements and muscle activity.
Diseases or traumas that cause neural disabilities such as spinal cord injuries (SCI) or stroke can result in major impacts in one’s life. Upper limb disabilities usually lead to a lack of autonomy in executing basic tasks such as hygiene and feeding, as well as an overall low quality of life and the occurrence of secondary health problems, e.g. obesity, pressure ulcers and depression.
Technologies aiming to assist movement in such circumstances exist. Notably, robotic exoskeletons and functional electrical stimulation (FES) can elicit movement even in paralyzed limbs. However, each has significant drawbacks that limit their use in rehabilitation clinics and/or at patients’ homes. Robotic devices are usually big and heavy not only because of the mechanical structure, but also the motors and sometimes batteries. They are expensive, rarely portable and difficult to operate. FES systems are affordable, simple, light and portable. Nevertheless, electrical stimulation elicits rapid fatigue, which makes intensive therapy impossible and limits function assistance.
The approach in my project is a hybrid system that explores the advantages of both technologies all while trying to avoid the drawbacks . The proposed device assists in elbow and hand function, consisting in two modules. The elbow module has a break system that does not elicit movement, but is capable of holding the present joint position. This is intended to mitigate fatigue by deactivating the FES when the break is on. The hand module is fully actuated by FES. Both modules rely on FES for movement, eliminating the need for motors. This reduces overall weight and external energy requirements. The device is intended for the execution of various tasks, including activities of daily living (ADL), in the context of rehabilitation.
Among all scientific and technical challenges around the development of such a system, my project focuses on the user interface. The person wearing the device has an upper limb disability which limits their capability of manipulating any standard controls, such as hand buttons or joysticks. Nevertheless, this population usually retains some residual movements and muscle contraction capabilities, particularly around the shoulders. This work explores these residual voluntary actions as inputs and translates them into the device’s commands according to the decoded user intent.
This project addressed the above-mentioned challenges by using wearable sensors capable of measuring movement – inertial measuring units (IMUs) – and muscle activity – electromyography (EMG). These sensors were positioned on specific body parts on the user, depending on their personal skills. Signals acquired were processed and decoded into user intended by machine learning algorithms. All that was done with a user centered paradigm, including a multi-center user requirements questionnaire. Finally, a hybrid upper limb neuroprosthesis was designed and a first prototype was built.
The project’s objective is to develop a high-level user interface with which persons with upper limb disabilities can control neuroprostheses for ADLs and rehabilitation.
Because the target user has no hand function, the interface cannot rely on traditional strategies such as hand operated buttons or joysticks. Instead, I employed IMUs and EMG sensors to explore residual movements and muscle activity. Similar solutions have been employed in other contexts with limited success. This project aimed at developing such user interface to control a semi passive hybrid neuroprosthesis for ADL in rehabilitation for persons that suffered a stroke or SCI.
Diseases or traumas that cause neural disabilities such as spinal cord injuries (SCI) or stroke can result in major impacts in one’s life. Upper limb disabilities usually lead to a lack of autonomy in executing basic tasks such as hygiene and feeding, as well as an overall low quality of life and the occurrence of secondary health problems, e.g. obesity, pressure ulcers and depression.
Technologies aiming to assist movement in such circumstances exist. Notably, robotic exoskeletons and functional electrical stimulation (FES) can elicit movement even in paralyzed limbs. However, each has significant drawbacks that limit their use in rehabilitation clinics and/or at patients’ homes. Robotic devices are usually big and heavy not only because of the mechanical structure, but also the motors and sometimes batteries. They are expensive, rarely portable and difficult to operate. FES systems are affordable, simple, light and portable. Nevertheless, electrical stimulation elicits rapid fatigue, which makes intensive therapy impossible and limits function assistance.
The approach in my project is a hybrid system that explores the advantages of both technologies all while trying to avoid the drawbacks . The proposed device assists in elbow and hand function, consisting in two modules. The elbow module has a break system that does not elicit movement, but is capable of holding the present joint position. This is intended to mitigate fatigue by deactivating the FES when the break is on. The hand module is fully actuated by FES. Both modules rely on FES for movement, eliminating the need for motors. This reduces overall weight and external energy requirements. The device is intended for the execution of various tasks, including activities of daily living (ADL), in the context of rehabilitation.
Among all scientific and technical challenges around the development of such a system, my project focuses on the user interface. The person wearing the device has an upper limb disability which limits their capability of manipulating any standard controls, such as hand buttons or joysticks. Nevertheless, this population usually retains some residual movements and muscle contraction capabilities, particularly around the shoulders. This work explores these residual voluntary actions as inputs and translates them into the device’s commands according to the decoded user intent.
This project addressed the above-mentioned challenges by using wearable sensors capable of measuring movement – inertial measuring units (IMUs) – and muscle activity – electromyography (EMG). These sensors were positioned on specific body parts on the user, depending on their personal skills. Signals acquired were processed and decoded into user intended by machine learning algorithms. All that was done with a user centered paradigm, including a multi-center user requirements questionnaire. Finally, a hybrid upper limb neuroprosthesis was designed and a first prototype was built.
The project’s objective is to develop a high-level user interface with which persons with upper limb disabilities can control neuroprostheses for ADLs and rehabilitation.
Because the target user has no hand function, the interface cannot rely on traditional strategies such as hand operated buttons or joysticks. Instead, I employed IMUs and EMG sensors to explore residual movements and muscle activity. Similar solutions have been employed in other contexts with limited success. This project aimed at developing such user interface to control a semi passive hybrid neuroprosthesis for ADL in rehabilitation for persons that suffered a stroke or SCI.
Stroke patients and healthcare professionals with experience in rehabilitation were reached and invited to participate in a survey with the goal to identify hybrid neuroprostheses user requirements. This task involved the coordination of a total of 8 people, including students and clinicians from two hospitals in France and one university in Brazil. Healthcare professionals were reached by means of direct contact, social medias, and email. Patients were invited during their stay or appointments at the hospitals, or during visits for rehabilitation therapy.
Healthcare professionals could respond online by themselves, from a link that was sent to them. Patients were supervised during the application of the questionnaire by a healthcare professional or a member of my team that was trained to apply it. In each country where the survey took place (Brazil and France), only native speakers of the local language were assigned to apply the questionnaire in order to maximize the patient comfort.
Then, able body subjects were recruited for experimental sessions that would take between 30 minutes and 1 hour. In total, over 30 of such sessions took place, resulting in tens of GB of data.
This data was then used to develop different machine learning models to control future assistive devices.
In the clinical trial, two participants with tetraplegia successfully controlled a partially implanted neuroprosthesis using the system developed in this project.
The results of this work were published in journal papers and conferences.
Healthcare professionals could respond online by themselves, from a link that was sent to them. Patients were supervised during the application of the questionnaire by a healthcare professional or a member of my team that was trained to apply it. In each country where the survey took place (Brazil and France), only native speakers of the local language were assigned to apply the questionnaire in order to maximize the patient comfort.
Then, able body subjects were recruited for experimental sessions that would take between 30 minutes and 1 hour. In total, over 30 of such sessions took place, resulting in tens of GB of data.
This data was then used to develop different machine learning models to control future assistive devices.
In the clinical trial, two participants with tetraplegia successfully controlled a partially implanted neuroprosthesis using the system developed in this project.
The results of this work were published in journal papers and conferences.
The user requirement survey conducted during this project is novel for a hybrid device and provides invaluable insights to the development of such technologies. This has the potential to better guide the scientific and industrial efforts towards more accessible and successful hybrid neuroprostheses in the near future.
The biomechanics data collected in the able body protocol contain full upper body inertial data, as well as trapezius and platysma muscle activation data. All of this, collected in over 30 sessions, represents the most complete upper body biomechanics data set of subjects performing upper limb activities of daily living that I am aware of. As soon as this is published, it will provide valuable data for the scientific community to further explore it and develop different solutions.
The classification algorithms developed in this project present new evidences of the feasibility of exploring residual movement and muscle activity of neuroprostheses control, consolidating the concept. In addition, it explores different machine learning techniques, discussing some of their strong and weak points.
The overall results advance the scientific knowledge in the field, bringing realistic and useful hybrid neuroprostheses closer to reality and to its final users.
The biomechanics data collected in the able body protocol contain full upper body inertial data, as well as trapezius and platysma muscle activation data. All of this, collected in over 30 sessions, represents the most complete upper body biomechanics data set of subjects performing upper limb activities of daily living that I am aware of. As soon as this is published, it will provide valuable data for the scientific community to further explore it and develop different solutions.
The classification algorithms developed in this project present new evidences of the feasibility of exploring residual movement and muscle activity of neuroprostheses control, consolidating the concept. In addition, it explores different machine learning techniques, discussing some of their strong and weak points.
The overall results advance the scientific knowledge in the field, bringing realistic and useful hybrid neuroprostheses closer to reality and to its final users.