Physical Cognition for Intelligent Control and Safe Human-Robot Interaction

Projektinformationen

Sestosenso

ID Finanzhilfevereinbarung: 101070310

DOI

10.3030/101070310

Projekt abgeschlossen

EK-Unterschriftsdatum 20 Juni 2022

Startdatum 1 Oktober 2022

Enddatum 30 September 2025

Finanziert unter

• Digital, Industry and Space

Gesamtkosten

€ 4 157 056,25

EU-Beitrag

€ 4 157 054,50

Koordiniert durch

UNIVERSITA DEGLI STUDI DI GENOVA
Italy

Periodic Reporting for period 2 - Sestosenso (Physical Cognition for Intelligent Control and Safe Human-Robot Interaction)

Berichtszeitraum: 2024-04-01 bis 2025-09-30

Zusammenfassung vom Kontext und den Gesamtzielen des Projekts

Sestosenso developed hardware and software enabling robots to self-adapt to changing conditions and safely switch between autonomous and human-interactive modes in cooperative scenarios.

Guided by Industry 4.0 and 5.0 principles, the project addressed the demand for robots that operate safely with minimal infrastructure, reducing setup time and cost while enhancing shopfloor flexibility. It also explored human factors and operator acceptance of human-robot interaction (HRI) technologies.

Sestosenso advanced sensing technologies—from hardware to cognitive control—through networks of embedded proximity and tactile sensors across the robot body, creating unified proxy-tactile perception to support:

- Autonomous and human-driven robot operations.
- New task capabilities via joint proxy-tactile feedback.
- Improved safety in human-robot cooperation and collaboration.
- Robot proximity and tactile based self-localization and environment mapping.
- Scalable manufacturing of large-area distributed sensors.

Rooted in AI, data, and robotics innovation, Sestosenso focused on human-centered technologies for industrial and service applications, demonstrated through three use cases:

A. Car assembly: safe human-robot cooperative tasks in dynamic environments.
B. Logistics: large-object handling and packaging through whole-arm manipulation.
C. Grape harvesting: outdoor operation with unmodeled obstacles using sensorized exoskeletons.

Arbeit, die ab Beginn des Projekts bis zum Ende des durch den Bericht erfassten Berichtszeitraums geleistet wurde, und die wichtigsten bis dahin erzielten Ergebnisse

During the project, the following scientific and technical (S/T) activities were carried out:

- Development of proxy-tactile sensors adaptable to various robot types and geometries (WP2).
- Design of large-area tactile sensor concepts for cost-effective and reliable components (WP2).
- Implementation of real-time proxy-tactile sensor network nodes using EtherCAT and real-time wireless systems (WP3).
- Development of multimodal middleware for real-time proxy-tactile data acquisition and robot control (WP4).
- Research and implementation of proxy-tactile–based control methods for robot self-awareness, gesture recognition, and task-oriented HRI (WP4/WP5/WP6).
- Definition, integration, and engineering of experimental use cases (WP7).
- Assessment of human factors and user acceptance of the technology (WP1/WP5/WP7).

Key project innovations:

- OptoSkin: patented, optical-based, cost-effective tactile sensing technology.
- ProxySkin: distributed large-area proximity and tactile sensor system.
- Cyskin+: fully embedded capacitive large-area tactile sensing technology.
- PeSkin / PeSkin+: printed electronic capacitive and multilayer tactile sensor technologies.
- Multimodal Middleware: real-time acquisition and integration framework for heterogeneous proxy-tactile sensors.

Main S&T achievements:

- Integration of proxy-tactile sensors into three industrial workstations (Use Cases 1–3).
- Studies on safe human-robot interaction and operator ergonomics.
- Application of geometric algebra for robotic manipulation and collision detection.
- Development of gesture recognition and proxy-tactile exploratory behaviors.
- Real-time wireless sensor networks for mobile platforms (e.g. exoskeletons).
- Integration and validation of proxy-tactile technologies across multiple robotic systems.

Fortschritte, die über den aktuellen Stand der Technik hinausgehen und voraussichtliche potenzielle Auswirkungen (einschließlich der bis dato erzielten sozioökonomischen Auswirkungen und weiter gefassten gesellschaftlichen Auswirkungen des Projekts)

All scientific and technical goals were achieved at various TRLs and successfully integrated into the three use cases. The project’s results have been published in high-quality peer-reviewed journals and conference proceedings, with additional publications forthcoming.

OptoSkin (patented) and ProxySkin have been identified as market-ready innovations for further development under the EIC Transition program, while CySkin+, PeSkin, PeSkin+, and the Multimodal Middleware are under review by the Innovation Radar initiative.

Sestosenso was demonstrated in automotive (car assembly), logistics (robotized packaging), and agriculture (assistive robotics). Its technologies are now being transferred to assistive robotics (TRL4 robot avatars). Emerging applications include shipyard and aerospace assembly, mobile logistics, and education and training (XR and animatronic avatars). Beyond these, Sestosenso supports privacy-compliant HRI scenarios where video or audio sensing is unsuitable.

The consortium—comprising universities, research institutes, and companies—prioritized R&D, knowledge transfer, and indirect industrial impact over immediate commercialization, fostering new R&D exploitation paths. Several partners are also exploring long-term opportunities for spin-offs and market deployment.

Details about the project, and the dissemination and exploitation activities can be found on the project's website: www.sestosenso.eu.

Periodic updates about publications and general project activities have been also published on

- X: twitter.com/sestosenso_eu
- Linkedin: https://www.linkedin.com/company/sestosenso-eu/

The Sestosenso Consortium

PeSkin demonstrator

Cooperation inside car chassis

Sestosenso logo

Sestosenso Team

OptoSkin poster at ICRA 2024

Proximity sensing inside car chassis

Sestosenso ecosystem. Proxy-tactile systems on robots and exoscheletons for collaborative tasks.

Preliminary integration of Sestosenso proxy-skin on a robot.

Sestosenso roll-up OptoSkin and ProxySkin

Optoskin Brochure

Graphical image of OptoSkin

Graphical image of OptoSkin on robot

OptoSkin Brochure for Innovation Radar