Physical Cognition for Intelligent Control and Safe Human-Robot Interaction

Información del proyecto

Sestosenso

Identificador del acuerdo de subvención: 101070310

DOI

10.3030/101070310

Proyecto cerrado

Fecha de la firma de la CE 20 Junio 2022

Fecha de inicio 1 Octubre 2022

Fecha de finalización 30 Septiembre 2025

Financiado con arreglo a

• Digital, Industry and Space

Coste total

€ 4 157 056,25

Aportación de la UE

€ 4 157 054,50

Coordinado por

UNIVERSITA DEGLI STUDI DI GENOVA
Italy

Periodic Reporting for period 2 - Sestosenso (Physical Cognition for Intelligent Control and Safe Human-Robot Interaction)

Período documentado: 2024-04-01 hasta 2025-09-30

Resumen del contexto y de los objetivos generales del proyecto

Sestosenso developed hardware and software enabling robots to self-adapt to changing conditions and safely switch between autonomous and human-interactive modes in cooperative scenarios.

Guided by Industry 4.0 and 5.0 principles, the project addressed the demand for robots that operate safely with minimal infrastructure, reducing setup time and cost while enhancing shopfloor flexibility. It also explored human factors and operator acceptance of human-robot interaction (HRI) technologies.

Sestosenso advanced sensing technologies—from hardware to cognitive control—through networks of embedded proximity and tactile sensors across the robot body, creating unified proxy-tactile perception to support:

- Autonomous and human-driven robot operations.
- New task capabilities via joint proxy-tactile feedback.
- Improved safety in human-robot cooperation and collaboration.
- Robot proximity and tactile based self-localization and environment mapping.
- Scalable manufacturing of large-area distributed sensors.

Rooted in AI, data, and robotics innovation, Sestosenso focused on human-centered technologies for industrial and service applications, demonstrated through three use cases:

A. Car assembly: safe human-robot cooperative tasks in dynamic environments.
B. Logistics: large-object handling and packaging through whole-arm manipulation.
C. Grape harvesting: outdoor operation with unmodeled obstacles using sensorized exoskeletons.

Trabajo realizado desde el comienzo del proyecto hasta el final del período abarcado por el informe y los principales resultados hasta la fecha

During the project, the following scientific and technical (S/T) activities were carried out:

- Development of proxy-tactile sensors adaptable to various robot types and geometries (WP2).
- Design of large-area tactile sensor concepts for cost-effective and reliable components (WP2).
- Implementation of real-time proxy-tactile sensor network nodes using EtherCAT and real-time wireless systems (WP3).
- Development of multimodal middleware for real-time proxy-tactile data acquisition and robot control (WP4).
- Research and implementation of proxy-tactile–based control methods for robot self-awareness, gesture recognition, and task-oriented HRI (WP4/WP5/WP6).
- Definition, integration, and engineering of experimental use cases (WP7).
- Assessment of human factors and user acceptance of the technology (WP1/WP5/WP7).

Key project innovations:

- OptoSkin: patented, optical-based, cost-effective tactile sensing technology.
- ProxySkin: distributed large-area proximity and tactile sensor system.
- Cyskin+: fully embedded capacitive large-area tactile sensing technology.
- PeSkin / PeSkin+: printed electronic capacitive and multilayer tactile sensor technologies.
- Multimodal Middleware: real-time acquisition and integration framework for heterogeneous proxy-tactile sensors.

Main S&T achievements:

- Integration of proxy-tactile sensors into three industrial workstations (Use Cases 1–3).
- Studies on safe human-robot interaction and operator ergonomics.
- Application of geometric algebra for robotic manipulation and collision detection.
- Development of gesture recognition and proxy-tactile exploratory behaviors.
- Real-time wireless sensor networks for mobile platforms (e.g. exoskeletons).
- Integration and validation of proxy-tactile technologies across multiple robotic systems.

Avances que van más allá del estado de la técnica e impacto potencial esperado (incluida la repercusión socioeconómica y las implicaciones sociales más amplias del proyecto hasta la fecha)

All scientific and technical goals were achieved at various TRLs and successfully integrated into the three use cases. The project’s results have been published in high-quality peer-reviewed journals and conference proceedings, with additional publications forthcoming.

OptoSkin (patented) and ProxySkin have been identified as market-ready innovations for further development under the EIC Transition program, while CySkin+, PeSkin, PeSkin+, and the Multimodal Middleware are under review by the Innovation Radar initiative.

Sestosenso was demonstrated in automotive (car assembly), logistics (robotized packaging), and agriculture (assistive robotics). Its technologies are now being transferred to assistive robotics (TRL4 robot avatars). Emerging applications include shipyard and aerospace assembly, mobile logistics, and education and training (XR and animatronic avatars). Beyond these, Sestosenso supports privacy-compliant HRI scenarios where video or audio sensing is unsuitable.

The consortium—comprising universities, research institutes, and companies—prioritized R&D, knowledge transfer, and indirect industrial impact over immediate commercialization, fostering new R&D exploitation paths. Several partners are also exploring long-term opportunities for spin-offs and market deployment.

Details about the project, and the dissemination and exploitation activities can be found on the project's website: www.sestosenso.eu.

Periodic updates about publications and general project activities have been also published on

- X: twitter.com/sestosenso_eu
- Linkedin: https://www.linkedin.com/company/sestosenso-eu/

The Sestosenso Consortium

PeSkin demonstrator

Cooperation inside car chassis

Sestosenso logo

Sestosenso Team

OptoSkin poster at ICRA 2024

Proximity sensing inside car chassis

Sestosenso ecosystem. Proxy-tactile systems on robots and exoscheletons for collaborative tasks.

Preliminary integration of Sestosenso proxy-skin on a robot.

Sestosenso roll-up OptoSkin and ProxySkin

Optoskin Brochure

Graphical image of OptoSkin

Graphical image of OptoSkin on robot

OptoSkin Brochure for Innovation Radar