Wearable Integrated Soft Haptic Display for Prosthetics

Project Information

WISH

Grant agreement ID: 101069179

DOI

10.3030/101069179

Project closed

EC signature date 28 June 2022

Start date 1 July 2022

End date 31 December 2023

Funded under

• European Research Council (ERC)

Total cost

No data

EU contribution

€ 150 000,00

Coordinated by

FONDAZIONE ISTITUTO ITALIANO DI TECNOLOGIA
Italy

Periodic Reporting for period 1 - WISH (Wearable Integrated Soft Haptic Display for Prosthetics)

Reporting period: 2022-07-01 to 2023-12-31

Summary of the context and overall objectives of the project

The sense of touch is the faculty that enables us to interact with and gather a wealth of information from the external world, such as objects' softness, dimensions, shape, texture, temperature, and more. Currently, 375,000 people resort to amputation each year, and unfortunately, this number is projected to double by 2050. Amputation involves not only the loss of a limb but also the loss of the ability to receive sensory information.
In recent decades, significant progress has been made in rebuilding the motor and control functions of robotic hands. However, little has been done to restore adequate sensory information.
The underlying concept of our project is to develop feedback devices capable of providing various stimuli, ranging from vibration to force, and even the combination of multiple sensory inputs. All of this will be seamlessly integrated into the prosthetic socket worn by the user.
We believe that incorporating feedback within the prosthesis can enhance user acceptance and bring about improvements in everyday life. Our goal is to go beyond the restoration of motor functions and address the critical aspect of sensory information loss associated with amputation. By doing so, we aim to positively impact the lives of individuals who have undergone amputations and contribute to a more inclusive and functional society.
The significance of our project lies in bridging the gap between current advancements in motor control and the often-neglected realm of sensory feedback in prosthetics. By enhancing the user experience and providing a more comprehensive solution, we anticipate our project will not only improve the quality of life for amputees but also set a new standard in prosthetic technology, ultimately influencing future developments in the field.

Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far

During the project, three main activities were pursued: continued investigation into force feedback using a liquid, exploration of integrating a vibrational device within the socket to provide information on initial contact and object texture, and the creation of a multi-modal device transmitting both force and sensory information, seamlessly integrated into the prosthetic socket.
For the force feedback investigation, a hydraulic circuit was created using a 12V peristaltic pump (0.1-60 rpm, 250-300 mA, by Grothen). The aim was to examine pressure levels achievable with a liquid and how the stimulus was perceived on the arm.
Regarding the integration of a vibrational device (Vibro-Inertial Bionic Enhancement System - VIBES) within the socket, it was incorporated into the SoftHand Pro (SHP). VIBES includes planar vibrotactile actuators, an electronic board, and Inertial Measurement Units (IMUs, MPU-9250) on the SHP's distal phalanx. The Haptuator Planar (HP) by TactileLabs, a lightweight, compact, and skin-friendly vibrotactile actuator, was used. The control strategy involved filtering and scaling acceleration signals recorded by IMUs in real-time.
Lastly, a dual feedback device, the Prosthetic Upper-Limb Sensory Enhancement (PULSE), was created. PULSE combines the haptic feedback capabilities of the Wearable Integrated Soft Haptic (WISH) device and VIBES, offering comprehensive information on force, initial contact, and texture cues.
For the force feedback component, the control strategy involved mapping the pressure stimulation to the residual current value of the SHP motor. This innovative approach allowed us to understand the force applied by the prosthetic hand during object grasping. The mapping process associated each hand value with a corresponding pressure value in the PULSE device. This information empowers users to discern the force exerted by the prosthetic hand, enhancing their interaction with the environment.
The control strategies for pressure stimulation and vibrational feedback in PULSE were akin to those described for VIBES, ensuring a cohesive and integrated sensory experience for users. The device also features switches on the prosthetic device, granting users the autonomy to activate or deactivate feedback components according to their preferences.
Both VIBES and PULSE underwent psychophysical characterization, assessing the efficacy of stimuli with both able-bodied subjects and a pilot subject. Objectives included evaluating the potential for higher pressure stimuli with a liquid, assessing user perception of vibrational stimuli within the prosthesis, and creating a multi-feedback system with force and initial contact cues. The aim was to enhance the embodiment of the prosthesis and provide a valuable aid in daily life, with the user retaining the discretion to activate or deactivate the feedback features.

Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far)

From the experiments conducted with both devices, the results have demonstrated that each device effectively delivers stimuli, and the conveyed information is clear and easily recognizable by the subjects. Throughout the experimental sessions, subjects using VIBES successfully distinguished various textures with high accuracy. According to the subjects' feedback, VIBES provided an intuitive experience. It is noteworthy that VIBES complements the SoftHand Pro framework, offering the unique advantage of dual haptic feedback for both initial contact and object texture.
Concerning the PULSE device, experimental results indicated improved subject performance when tasks were executed with feedback compared to those performed without, especially regarding the demand on mental workload. Notably, the presence of dual feedback was deemed non-intrusive.
In general, both devices serve as an initial and robust example of integrating feedback within a prosthetic socket, offering valuable assistance in enhancing the subject's sense of embodiment. It is crucial to emphasize that the tests conducted during this project serve as validation for a proof of concept, and further testing is imperative to explore the full potential of these devices.
The potential impacts of these findings extend to the realm of prosthetics, offering enhanced sensory experiences for amputees. The key needs for further uptake and success include additional research to refine and expand the capabilities of the devices, demonstration sessions to showcase their effectiveness, access to markets and finance for broader adoption, considerations for commercialization, and support for intellectual property rights (IPR) along with internationalization efforts. A supportive regulatory and standardization framework will be instrumental in ensuring widespread acceptance and utilization of these innovative prosthetic technologies.
In addition to the technical developments, we conducted a comprehensive market analysis and formulated an exploitation plan to facilitate the commercialization of our innovative devices, ensuring their broader accessibility and impact.