Periodic Reporting for period 1 - NIMA (NIMA: Non-invasive Interface for Movement Augmentation)
Periodo di rendicontazione: 2020-10-01 al 2021-09-30
If successful, NIMA will take human motor capacities to a new level and allow humans to perform tasks that are impossible to perform with the natural limbs alone. The reach of such movement augmentation is far, with applications ranging from work in industrial environments (e.g. overhead work in aircraft assembly), and surgical environments, to everyday applications where subjects can control their smartphone apps while simultaneously using their hands. For instance, laparoscopic surgeons could become able to perform surgical procedures with three hands allowing them to carry out tasks that require a skilled synchronisation of the manipulation of three surgical instruments that currently cannot be realised with minimally invasive access.
As an interdisciplinary team of experts in neuroscience, neurotechnology, human-machine interfaces, robotics, and ethics, we collaborate to accomplish the following objectives: (i) Push the borders of technology by creating a wearable supernumerary robotic limb (SRL), and non-invasive interfaces with multimodal sensory feedback to control multiple limbs or objects; (ii) Understand the cognitive and neural mechanisms underlying movement augmentation; (iii) Apply movement augmentation to extend a surgeon capabilities and autonomy, develop manipulation with a wearable supernumerary robotic arm and the two hands as well as a 3-hands computer interface; (iv) Evaluate the ethical and safety aspects of movement augmentation.
Substantial progress has been made in developing and evaluating body interfaces for movement augmentation where our results demonstrate that subjects can control three virtual hands. For the neural interface we investigated whether subjects can voluntarily control individual motor units from a single muscle independently as a basis to control multiple limbs simultaneously. Our findings suggest that motor units could not be controlled independently during initial recruitment but once motor units were active flexibility of control may increase. For the muscle interface, a pilot study on the wrist was carried out with preliminary results on muscle activation patterns that subjects can generate independent of wrist force.
A wearable sensory feedback suit able to provide vibrotactile and electrotactile stimulation with high versatility and large bandwidth has been developed and validated. We could confirm the feasibility of proprioceptive-guided real-time tracking of an SRL through our system.
A wearable SRL was developed based on a light commercial manipulator and tested. As a step towards application of movement augmentation to robot-assisted surgery, an ergonomic foot interface with four degrees-of-freedom has been used to control a soft endoscope.
The challenge of human motor augmentation raises numerous ethical and safety questions, both at the micro scale of individual (safety of users sharing their workspace with a SRL or being impacted by movement augmentation abilities) or at the macro scale of society and humanity (ethics of movement augmentation). Regarding workspace safety; a risk analysis was conducted with the NIMA SRL platforms and a set of hardware and software measures were proposed to ensure user safety. In parallel, ethical reflections on issues were developed and an online survey was administered to relevant stakeholders for gathering information on potential real-life interactions enabled by NIMA.
Already at this early stage of the project, the consortium has published several publications at high-ranked journals and conferences including joint publications from multiple partners such as a comprehensive review on movement augmentation and a submitted manuscript on ethics of augmentation. Furthermore, several members of the consortium have launched a lecture series and organised a symposium on movement augmentation at the IEEE conference on Neural Engineering 2021 increasing visibility of work carried out in NIMA. In summary, with its foundational work, we expect NIMA to provide building blocks of knowledge, skills, methods and technologies as well as safety data and ethical considerations that future work in movement augmentation will be able to build upon.