Skip to main content

Understanding neural coding of touch as enabling technology for prosthetics and robotics

Periodic Reporting for period 1 - NeuTouch (Understanding neural coding of touch as enabling technology for prosthetics and robotics)

Reporting period: 2019-03-01 to 2021-02-28

Touch is a gateway to explore the world and a mean to establish contact for interaction.It is a crucial skill for robots, to learn the properties of objects and how to use them, and to enable cooperative behaviour where robots and people act together to accomplish a task. It is even more crucial in the deployment of prosthetic devices, to give the user the sensation of contact and a richer set of information that can make the use of prostheses easier and more acceptable.
Biological sensory systems have developed to best capture the properties of surrounding objects and environment that are useful for acting in the world.When grasping a glass of water, our hand automatically adjusts the force used to stably hold the glass depending on its size, weight, slippery, softness.This is done by an efficient system that spares the slightest bit of information, to optimise power consumption. As such, artificial systems have much to learn from biology, to develop cheap solutions that can run in a very small device and at minimum energy cost. This is true in autonomous systems(robots)and in medical devices that run on the body of people,to minimize heat produced and maximize battery life.
NeuTouch aims at improving artificial tactile systems, by training a generation of researchers that study biological tactile systems, develop a technology that is based on the same principles,and use it to enable robots to help humans in daily tasks and artificial limbs to give the user the sensation of touch.
15 PhD students are part of an international and multi-disciplinary network of universities and research centres,expert in neuroscience, neuromorphic engineering, robotics,and prosthetics.The network is completed by companies where students will deploy and test their know-how.This is the key to give them the capability to bring innovation to the scientific community and to the industrial world, leading to the development of science that will have a direct impact in society by responding to the needs of people.
NeuTouch fellows work to understand the neural code in tactile biological perception, to develop technologies for artificial tactile sensing and feedback, and to integrate them in robots and prostheses.
Understanding which neural features encode tactile sensing and inform behavioural choices can lead to a better design of artificial spiking neural networks that form robust and readable representations of tactile information. To this aim, Miguel Casal developed a software toolbox based on the principles of information theory and machine learning to quantify how populations of neurons encode information. He developed a data analysis formalism to analyse spike trains simultaneously recorded in different areas of the brain. Victoria Lang, Miguel and Giulia Corniani analysed humans microneurography data and simulation of biological mechanoreceptors to understand their different and complementary role in encoding stimuli properties. Zahra Youssefi acquired electrophysiological data from rats in a vibrotactile discrimination task to understand how the touch system adapts to the changing statistics of the world. To complement the activity on neural encoding, Giulia C. studied the embodiment of the sense of touch, by estimating the innervation density of Aß tactile afferents on the whole human body.
Towards the use of touch in prosthetic devices, Outman Akoussi developed technologies for improving long term nerve stimulation. He developed a hybrid implant including extra- and intra-fascicular electrodes for combined stimulation or recording purposes. He developed a model to simulate the interaction between nerve and implant that led to the design of novel geometry that minimizes the stresses applied at the interface, improving implant longevity. His study highlighted implants should include coatings with low elastic modulus lower and developed a technology to coat soft elastomers with extremely soft hydrogels. Giulia Dominjianni developed a modular platform to assess sensory-motor learning of a third arm in an immersive virtual environment and started running preliminary experiments.
Towards the use of tactile sensing in robots, it is important to understand how behaviour is linked to perception. Simon Müller-Cleve has developed protocols to monitor human exploration strategies and interaction forces between hands and objects during blind object recognition. Towards generating memory of sequences and prediction capabilities needed to explore objects, Pablo Villacis developed a Deep Gaussian Processes framework to model tactile information integration over time and applied it to the estimation of the hand position over a circular surface. At the same time, different robotic platforms have been equipped with enabling technologies for tactile sensing. Simon implemented a skin-plugin for the iCub hand in the Gazebo simulator, Jakub Cmiral simulated force profiles observed when operating switches, knobs, and sliders. Luca Lach developed a grasp force controller with tactile feedback for parallel-jaw grippers and multi-fingered hands that reduces undesired object movements and prevents damage. Luca’s sensorised gripper for the robot TiAGO has been sent to production.
Towards the development of artificial tactile sensing technologies, Joao Neto and Luca De Pamphilis studied and improved techniques to print nanowires and nanotransistors. Ella Janotte and Michele Mastella developed low-power circuits based on mixed-mode sub-threshold CMOS for the readout of tactile transducers and for pre-processing using spiking neural networks for orientation and texture selectivity.
NeuTouch offered an international summer school on Touch for Prosthetics attended by NeuTouch and external students, postdocs and faculties. As part of NeuTouch’s editorial board, NeuTouch fellows learnt to use twitter and instagram for scientific content and to organise events such as the school and a symposium at the IEEE NER21 conference. They are learning local language and improving their argumentation skills in English.
NeuTouch contributes to the improvement of artificial touch by deepening our understanding of how biology extracts information from tactile stimuli and on how this information influences behaviour. This understanding is fundamental to deliver better tactile perception in artificial devices, improving their capabilities in supporting humans both by assistive and collaborative work from robots, and for better usability of prosthetic devices.
This knowledge has been used to develop and test circuits for smart, efficient, and miniaturised artificial tactile sensors for robots and prostheses.
Several simulation tools are available to analyse neural encoding, test tactile sensing-based grasp control and manipulation in robots, test tactile feedback in virtual environments, study the interaction of electrodes with neural tissue for improving implants for sensory feedback in prosthetic users.
These tools support a better understanding of the biological sense of touch and the improvement of the use of tactile sensing in robots and prosthetics. The latter led to the development of new electrodes for stimulation of neural tissue improving implant longevity.
The interdisciplinary nature of the project is fostering collaborations and nurturing a team of young researchers that are likely to bring breakthrough ideas and attitude, by creating synergistic links between different expertise areas.
NeuTouch Network
Project Logo
Tactile Innervation of the Human Body