Periodic Reporting for period 3 - eNHANCE (eNHANCE - intention based enhancement of reaching and grasping in physically disabled people - personalized to maximize user performance)
Período documentado: 2017-08-01 hasta 2019-03-31
The overall objective of the eNHANCE project is to develop, demonstrate and initiate exploitation of new concepts to enhance and train upper extremity motor function during daily-life in people with physical disabilities.
To this end, we have developed a mechatronic arm, wrist and hand support, controlled by eye-tracking technology. The eye-tracking technology is supplemented with intention detection and personalized adaptive support. The system is modular: separate components can be used to create assistive devices for different patient groups such as DMD and CVA.
For DMD patients, the system can assist in performing daily life activities using the intention detection. It recognizes the intended movement such as picking up an object or pouring from a bottle, through the eyetracking. The eNHANCE system then guides you during the entire movement, from positioning your arm to closing your hand to grasp a bottle with the assistive glove and then turn your wrist to pour the contents.
For Stroke patients, the system supports them during their therapeutic sessions. The system has a motivational game to maximize the number of reaching efforts of the patient, as well as a visualization of the efforts of the patients for the physical therapist. The support level adapts automatically to optimize the patients’ own strength.
The use of the system could reduce the need for round the clock care and decrease the pressure on informal caregivers. It provides DMD patients with more freedom, increasing their independence and quality of life. For stroke patients, it could increase the added value of therapy, and optimize the care given by the physical therapists.
To this end, we have developed a mechatronic arm, wrist and hand support, controlled by eye-tracking technology. The eye-tracking technology is supplemented with intention detection and personalized adaptive support. The system is modular: separate components can be used to create assistive devices for different patient groups such as DMD and CVA.
For DMD patients, the system can assist in performing daily life activities using the intention detection. It recognizes the intended movement such as picking up an object or pouring from a bottle, through the eyetracking. The eNHANCE system then guides you during the entire movement, from positioning your arm to closing your hand to grasp a bottle with the assistive glove and then turn your wrist to pour the contents.
For Stroke patients, the system supports them during their therapeutic sessions. The system has a motivational game to maximize the number of reaching efforts of the patient, as well as a visualization of the efforts of the patients for the physical therapist. The support level adapts automatically to optimize the patients’ own strength.
The use of the system could reduce the need for round the clock care and decrease the pressure on informal caregivers. It provides DMD patients with more freedom, increasing their independence and quality of life. For stroke patients, it could increase the added value of therapy, and optimize the care given by the physical therapists.
A mechatronic arm, wrist and hand support were developed, controlled by eye-tracking technology. This system was supplemented with intention detection, adaptive personalized support, a motivational game, and a visualization tool for the Physical Therapist.
Functional and user requirements were elicited. These were used in the development of two different arm support. The first is supplemented with safety features and a human model to fit individual patients’ needs. The second, the ExoArm was improved with a separately developed eye-tracking software and can be mounted on a wheelchair.
The wrist support has been developed as a compact device for active pro-supination of the wrist. The device can exert high torques to overcome the forces required to pro-supinate the forearm of stroke patients and is suitable for both DMD and CVA patients
The hand support is the world’s first robotic glove controlled by eye-tracking. Brushless actuators to be implemented with the hand support were developed for an active soft exoskeleton.
The eye-tracking software that controls the mechanical system has a novel method for real-time, high accuracy and high precision 3D gaze point estimation. Combined with a semantic fovea that labels real-time object and scenes of an egocentric camera images leads to real-time object of interest and intention of actions recognition.
Human action grammars-based recognition of the intention of a task, based on a large experiment to gather raw data, and intention of low-level action, combines the gaze information, and the context of the interaction, allowing us to predict the intended course of action. The system has successfully been tested at ICL with 10 healthy and one disabled subject.
Additionally, a motivational game was developed specifically for the eNHANCE system, as well as a visualization tool that presents the performance of the system and the user to the Physical therapist.
The performance of the (sub)system and components were evaluated and validated during four user test phases with seven different system configurations. It highlighted the need for an individualized approach through modular technology in both hardware and its control.
A business and exploitation plan has been developed to ensure the exploitation of the concept after the end of the project.
Functional and user requirements were elicited. These were used in the development of two different arm support. The first is supplemented with safety features and a human model to fit individual patients’ needs. The second, the ExoArm was improved with a separately developed eye-tracking software and can be mounted on a wheelchair.
The wrist support has been developed as a compact device for active pro-supination of the wrist. The device can exert high torques to overcome the forces required to pro-supinate the forearm of stroke patients and is suitable for both DMD and CVA patients
The hand support is the world’s first robotic glove controlled by eye-tracking. Brushless actuators to be implemented with the hand support were developed for an active soft exoskeleton.
The eye-tracking software that controls the mechanical system has a novel method for real-time, high accuracy and high precision 3D gaze point estimation. Combined with a semantic fovea that labels real-time object and scenes of an egocentric camera images leads to real-time object of interest and intention of actions recognition.
Human action grammars-based recognition of the intention of a task, based on a large experiment to gather raw data, and intention of low-level action, combines the gaze information, and the context of the interaction, allowing us to predict the intended course of action. The system has successfully been tested at ICL with 10 healthy and one disabled subject.
Additionally, a motivational game was developed specifically for the eNHANCE system, as well as a visualization tool that presents the performance of the system and the user to the Physical therapist.
The performance of the (sub)system and components were evaluated and validated during four user test phases with seven different system configurations. It highlighted the need for an individualized approach through modular technology in both hardware and its control.
A business and exploitation plan has been developed to ensure the exploitation of the concept after the end of the project.
This is the first active and intuitive support system for arm, hand and wrist support that exists on the market controlled by eyetracking. Our system is modular and can be used in many different compositions, tailored to the individual patients’ needs.
The user requirement identification phase resulted in a framework for crucial themes to be addressed when designing Assistive Technology (AT) for the affected arm/hand. This framework added a number of themes from the literature that are usually overlooked, but identified as central topics in users’ views on the use and adoption of AT.
Two versions of the ENHANCE arm support were developed over the course of the project. Five controlled and actuated DOF’s and an innovative human model was implemented in the developed arm support. The second, the ExoArm has got an early version of eyetracking on a wheelchair mounted arm support. This goes far beyond the state of the art for arm and hand support.
The robotic glove is the only one that currently exists on the market that can be controlled with the eyes. So it the wrist support with active pro-supination within a small volume. The actuator for the hand support has integrated brushless motors where they are typically are not used for exoskeletons but have a number of benefits.
For the first time, we addressed transparency of a soft-robotic glove in similar ways as done with lower limbs rehab robotics, to assess whether the presence of the robot interferes with normal movement. This shows that although the glove is not completely transparent, its influence on normal movement is limited, while its added support doesn’t interfere with movement patterns. Yet, it does increase grip strength by over 15%.
A support level controller that can automatically adjust the amount of support to the need of the patient was developed. The algorithm, as well as the integrated monitoring tool that supports the decision making of Physical Therapists, are unique on the market.
Achieving gaze-contingent, action grammar-based interaction has led to a complex system that is simple to operate by naïve users; the successful use of the system by a disabled person is evidence to this and goes beyond the results obtained with more intrusive, and training intensive methods such as direct brain interfacing. The 3D gaze estimation method produces a more stable, and more accurate 3D estimation of gaze than existing methods. The modular architecture created for gaze-contingent human-robot interaction is unique and expandable. The human action grammars-based intention decoding results in more intuitive and comfortable human-robot interaction.
Testing and comparing user performance between different ways of controlling robotic arms and gloves to inform the design hasn’t been done before in rehab or assistive technology. Concluding, contrary to what is generally assumed, (subjective) user preferences don’t follow (objective) user performance. This is new and highly valuable information.
When the eNHANCE system is fully developed and funded, it can impact the lives of up to 50% of all stroke patients, as well as patients with reduced grip, spinal cord injuries, CP and ALS patients.
The user requirement identification phase resulted in a framework for crucial themes to be addressed when designing Assistive Technology (AT) for the affected arm/hand. This framework added a number of themes from the literature that are usually overlooked, but identified as central topics in users’ views on the use and adoption of AT.
Two versions of the ENHANCE arm support were developed over the course of the project. Five controlled and actuated DOF’s and an innovative human model was implemented in the developed arm support. The second, the ExoArm has got an early version of eyetracking on a wheelchair mounted arm support. This goes far beyond the state of the art for arm and hand support.
The robotic glove is the only one that currently exists on the market that can be controlled with the eyes. So it the wrist support with active pro-supination within a small volume. The actuator for the hand support has integrated brushless motors where they are typically are not used for exoskeletons but have a number of benefits.
For the first time, we addressed transparency of a soft-robotic glove in similar ways as done with lower limbs rehab robotics, to assess whether the presence of the robot interferes with normal movement. This shows that although the glove is not completely transparent, its influence on normal movement is limited, while its added support doesn’t interfere with movement patterns. Yet, it does increase grip strength by over 15%.
A support level controller that can automatically adjust the amount of support to the need of the patient was developed. The algorithm, as well as the integrated monitoring tool that supports the decision making of Physical Therapists, are unique on the market.
Achieving gaze-contingent, action grammar-based interaction has led to a complex system that is simple to operate by naïve users; the successful use of the system by a disabled person is evidence to this and goes beyond the results obtained with more intrusive, and training intensive methods such as direct brain interfacing. The 3D gaze estimation method produces a more stable, and more accurate 3D estimation of gaze than existing methods. The modular architecture created for gaze-contingent human-robot interaction is unique and expandable. The human action grammars-based intention decoding results in more intuitive and comfortable human-robot interaction.
Testing and comparing user performance between different ways of controlling robotic arms and gloves to inform the design hasn’t been done before in rehab or assistive technology. Concluding, contrary to what is generally assumed, (subjective) user preferences don’t follow (objective) user performance. This is new and highly valuable information.
When the eNHANCE system is fully developed and funded, it can impact the lives of up to 50% of all stroke patients, as well as patients with reduced grip, spinal cord injuries, CP and ALS patients.