Periodic Reporting for period 1 - HARIA (HUMAN-ROBOT SENSORIMOTOR AUGMENTATION - WEARABLE SENSORIMOTOR INTERFACES AND SUPERNUMERARY ROBOTIC LIMBS FOR HUMANS WITH UPPER-LIMB DISABILITIES)
Période du rapport: 2022-10-01 au 2023-09-30
HARIA re-defines the nature of physical human-robot interaction, laying the foundations of a new research field, i.e. human sensorimotor augmentation. The idea is to study how to integrate humans and AI-powered supernumerary robotic limbs to perform complex manipulation tasks with both biological and artificial limbs under the direct control of the human.
A fundamental challenge in HARIA will be to find the right trade-off between motion task parameters that are controlled by the user, and the level of robot autonomy. The enabling core technology of this interplay is the wearable sensorimotor interface that establishes a connection between the human sensorimotor system and the system of actuators and sensors of the robot, allowing for reciprocal awareness, trustworthiness, and mutual understanding. The sensorimotor interface captures signals from human body motion, or muscle activation, that are mapped onto commands for the robot limbs using a novel approach that exploits the redundancy of the human sensorimotor system. This will lead to the simultaneous control of natural and artificial limbs. Interfaces will also exploit combinations of somatosensory stimuli to convey haptic sensations related to the task at hand, further improving users’ control accuracy and level of engagement in the collaboration. HARIA sensorimotor augmentation will be applied to assist people with upper-limb disabilities, focusing on two target populations: chronic stroke and spinal cord injured patients.
The project is based on a user-centered design framework, which is implemented also thanks to competences in the Consortium pertaining to social sciences and humanities. The development of HARIA technologies will be user-centered and iterative, i.e. based on involving the user in all stages of the development process. Input will be gathered not only from target end-users, but also from other target groups, including manufacturers and installers of the devices, as well as potential caregivers and family members. User-centered design (UCD) is needed to make sure that end-users are not only able to use the assistive robotic devices but keen to use them.
So far, wearable extra limbs have been mainly used to augment healthy humans’ capabilities, while collaborative arms and grippers have been mainly applied to industrial assembly tasks. HARIA focuses on applying these technologies as assistive devices and targets a large set of end-users with upper-limb disabilities. HARIA systems will be integrated in three different scenarios involving activities of daily living to be performed by selected stroke survivors and spinal cord injured individuals. Thanks to the sensorimotor interfaces, users will be actively in charge of controlling the robot. This will represent a strong motivation for using HARIA devices, ensuring their long-term adoption. The societal mid-term impact on life after stroke and spinal cord injury will be evident in the recovered autonomy in the execution of daily activities, leading also to a positive consequence for caregivers and rehab facilities. A societal long-term impact is expected in domestic rehabilitation and homecare thanks to the possibility of exploiting remotely, at home, the HARIA technologies. In addition, the outcomes of the project will impact other domains of the service robotics field, defining novel paradigms for human-robot communication.
A fundamental challenge in HARIA will be to find the right trade-off between motion task parameters that are controlled by the user, and the level of robot autonomy. The enabling core technology of this interplay is the wearable sensorimotor interface that establishes a connection between the human sensorimotor system and the system of actuators and sensors of the robot, allowing for reciprocal awareness, trustworthiness, and mutual understanding. The sensorimotor interface captures signals from human body motion, or muscle activation, that are mapped onto commands for the robot limbs using a novel approach that exploits the redundancy of the human sensorimotor system. This will lead to the simultaneous control of natural and artificial limbs. Interfaces will also exploit combinations of somatosensory stimuli to convey haptic sensations related to the task at hand, further improving users’ control accuracy and level of engagement in the collaboration. HARIA sensorimotor augmentation will be applied to assist people with upper-limb disabilities, focusing on two target populations: chronic stroke and spinal cord injured patients.
The project is based on a user-centered design framework, which is implemented also thanks to competences in the Consortium pertaining to social sciences and humanities. The development of HARIA technologies will be user-centered and iterative, i.e. based on involving the user in all stages of the development process. Input will be gathered not only from target end-users, but also from other target groups, including manufacturers and installers of the devices, as well as potential caregivers and family members. User-centered design (UCD) is needed to make sure that end-users are not only able to use the assistive robotic devices but keen to use them.
So far, wearable extra limbs have been mainly used to augment healthy humans’ capabilities, while collaborative arms and grippers have been mainly applied to industrial assembly tasks. HARIA focuses on applying these technologies as assistive devices and targets a large set of end-users with upper-limb disabilities. HARIA systems will be integrated in three different scenarios involving activities of daily living to be performed by selected stroke survivors and spinal cord injured individuals. Thanks to the sensorimotor interfaces, users will be actively in charge of controlling the robot. This will represent a strong motivation for using HARIA devices, ensuring their long-term adoption. The societal mid-term impact on life after stroke and spinal cord injury will be evident in the recovered autonomy in the execution of daily activities, leading also to a positive consequence for caregivers and rehab facilities. A societal long-term impact is expected in domestic rehabilitation and homecare thanks to the possibility of exploiting remotely, at home, the HARIA technologies. In addition, the outcomes of the project will impact other domains of the service robotics field, defining novel paradigms for human-robot communication.
The scientific work of the first reporting period of the HARIA project has mostly focused on the design and development of initial prototypes of HARIA core technologies, including supernumerary fingers and collaborative grippers and sensing and actuation modules for wearable sensorimotor interfaces. In parallel, the methodologies for human-robot augmentation have started to be developed.
Preliminary work for the integration of hardware and software components has started, especially with respect to the development of the robot simulators.
An important part of the work of the Consortium in the first reporting period revolved around the implementation of the user-centered design approach at the basis of HARIA. Several interviews and workshops with different stakeholders have been organized in different countries (Sweden, Spain, Italy). The results of these activities were analyzed in depth and led to the definition of users’ needs and requirements, as well as to the identification of usage scenarios and Personas.
Preliminary work for the integration of hardware and software components has started, especially with respect to the development of the robot simulators.
An important part of the work of the Consortium in the first reporting period revolved around the implementation of the user-centered design approach at the basis of HARIA. Several interviews and workshops with different stakeholders have been organized in different countries (Sweden, Spain, Italy). The results of these activities were analyzed in depth and led to the definition of users’ needs and requirements, as well as to the identification of usage scenarios and Personas.
The progress made in all the HARIA objectives has brought relevant results from the technological and the methodological point of view.
The novel prototypes of collaborative grippers, for example, represent important innovations in the field of robotic manipulation in which there is still a lack of versatile and safe grippers to be used in collaboration with humans in unstructured environments. The proposed sensing and actuation modules for sensorimotor interfaces, including a proof-of-concept HD-EMG device, a laser pointer, and a versatile tactile interface have the potential to impact the field of human-robot interfaces and of biosignals readings, in which wearability and intuitiveness still represent important challenges.
From the point of view of robot planning and control, the newly proposed methods for affordance-based scene representation are showing effective and unforeseen applications of recent advancements in the field of artificial intelligence. From the neuroscience perspective, the null-space control strategies studied in the project are unveiling important insights on the human motor and muscular control patterns, showing the importance of considering subjective capabilities when choosing the null-space control law.
The Consortium started implementing a user-centered approach from the very beginning of the project, conducting interviews and workshops with several stakeholders (patients, caregivers, relatives, healthcare professionals) to assess the impact of disabilities in the daily life of patients and their relatives, as well as their expectations and needs, and their possible propension towards the use of robotic assistive technologies.
The early interaction with stakeholders has allowed the Consortium to identify key users' needs to ensure further uptake and success of augmentation technologies.
The novel prototypes of collaborative grippers, for example, represent important innovations in the field of robotic manipulation in which there is still a lack of versatile and safe grippers to be used in collaboration with humans in unstructured environments. The proposed sensing and actuation modules for sensorimotor interfaces, including a proof-of-concept HD-EMG device, a laser pointer, and a versatile tactile interface have the potential to impact the field of human-robot interfaces and of biosignals readings, in which wearability and intuitiveness still represent important challenges.
From the point of view of robot planning and control, the newly proposed methods for affordance-based scene representation are showing effective and unforeseen applications of recent advancements in the field of artificial intelligence. From the neuroscience perspective, the null-space control strategies studied in the project are unveiling important insights on the human motor and muscular control patterns, showing the importance of considering subjective capabilities when choosing the null-space control law.
The Consortium started implementing a user-centered approach from the very beginning of the project, conducting interviews and workshops with several stakeholders (patients, caregivers, relatives, healthcare professionals) to assess the impact of disabilities in the daily life of patients and their relatives, as well as their expectations and needs, and their possible propension towards the use of robotic assistive technologies.
The early interaction with stakeholders has allowed the Consortium to identify key users' needs to ensure further uptake and success of augmentation technologies.