Physical Cognition for Intelligent Control and Safe Human-Robot Interaction

Informazioni relative al progetto

Sestosenso

ID dell’accordo di sovvenzione: 101070310

DOI

10.3030/101070310

Progetto chiuso

Data della firma CE 20 Giugno 2022

Data di avvio 1 Ottobre 2022

Data di completamento 30 Settembre 2025

Finanziato da

• Digital, Industry and Space

Costo totale

€ 4 157 056,25

Contributo UE

€ 4 157 054,50

Coordinato da

UNIVERSITA DEGLI STUDI DI GENOVA
Italy

Periodic Reporting for period 2 - Sestosenso (Physical Cognition for Intelligent Control and Safe Human-Robot Interaction)

Periodo di rendicontazione: 2024-04-01 al 2025-09-30

Sintesi del contesto e degli obiettivi generali del progetto

Sestosenso developed hardware and software enabling robots to self-adapt to changing conditions and safely switch between autonomous and human-interactive modes in cooperative scenarios.

Guided by Industry 4.0 and 5.0 principles, the project addressed the demand for robots that operate safely with minimal infrastructure, reducing setup time and cost while enhancing shopfloor flexibility. It also explored human factors and operator acceptance of human-robot interaction (HRI) technologies.

Sestosenso advanced sensing technologies—from hardware to cognitive control—through networks of embedded proximity and tactile sensors across the robot body, creating unified proxy-tactile perception to support:

- Autonomous and human-driven robot operations.
- New task capabilities via joint proxy-tactile feedback.
- Improved safety in human-robot cooperation and collaboration.
- Robot proximity and tactile based self-localization and environment mapping.
- Scalable manufacturing of large-area distributed sensors.

Rooted in AI, data, and robotics innovation, Sestosenso focused on human-centered technologies for industrial and service applications, demonstrated through three use cases:

A. Car assembly: safe human-robot cooperative tasks in dynamic environments.
B. Logistics: large-object handling and packaging through whole-arm manipulation.
C. Grape harvesting: outdoor operation with unmodeled obstacles using sensorized exoskeletons.

Lavoro eseguito dall’inizio del progetto fino alla fine del periodo coperto dalla relazione e principali risultati finora ottenuti

During the project, the following scientific and technical (S/T) activities were carried out:

- Development of proxy-tactile sensors adaptable to various robot types and geometries (WP2).
- Design of large-area tactile sensor concepts for cost-effective and reliable components (WP2).
- Implementation of real-time proxy-tactile sensor network nodes using EtherCAT and real-time wireless systems (WP3).
- Development of multimodal middleware for real-time proxy-tactile data acquisition and robot control (WP4).
- Research and implementation of proxy-tactile–based control methods for robot self-awareness, gesture recognition, and task-oriented HRI (WP4/WP5/WP6).
- Definition, integration, and engineering of experimental use cases (WP7).
- Assessment of human factors and user acceptance of the technology (WP1/WP5/WP7).

Key project innovations:

- OptoSkin: patented, optical-based, cost-effective tactile sensing technology.
- ProxySkin: distributed large-area proximity and tactile sensor system.
- Cyskin+: fully embedded capacitive large-area tactile sensing technology.
- PeSkin / PeSkin+: printed electronic capacitive and multilayer tactile sensor technologies.
- Multimodal Middleware: real-time acquisition and integration framework for heterogeneous proxy-tactile sensors.

Main S&T achievements:

- Integration of proxy-tactile sensors into three industrial workstations (Use Cases 1–3).
- Studies on safe human-robot interaction and operator ergonomics.
- Application of geometric algebra for robotic manipulation and collision detection.
- Development of gesture recognition and proxy-tactile exploratory behaviors.
- Real-time wireless sensor networks for mobile platforms (e.g. exoskeletons).
- Integration and validation of proxy-tactile technologies across multiple robotic systems.

Progressi oltre lo stato dell’arte e potenziale impatto previsto (incluso l’impatto socioeconomico e le implicazioni sociali più ampie del progetto fino ad ora)

All scientific and technical goals were achieved at various TRLs and successfully integrated into the three use cases. The project’s results have been published in high-quality peer-reviewed journals and conference proceedings, with additional publications forthcoming.

OptoSkin (patented) and ProxySkin have been identified as market-ready innovations for further development under the EIC Transition program, while CySkin+, PeSkin, PeSkin+, and the Multimodal Middleware are under review by the Innovation Radar initiative.

Sestosenso was demonstrated in automotive (car assembly), logistics (robotized packaging), and agriculture (assistive robotics). Its technologies are now being transferred to assistive robotics (TRL4 robot avatars). Emerging applications include shipyard and aerospace assembly, mobile logistics, and education and training (XR and animatronic avatars). Beyond these, Sestosenso supports privacy-compliant HRI scenarios where video or audio sensing is unsuitable.

The consortium—comprising universities, research institutes, and companies—prioritized R&D, knowledge transfer, and indirect industrial impact over immediate commercialization, fostering new R&D exploitation paths. Several partners are also exploring long-term opportunities for spin-offs and market deployment.

Details about the project, and the dissemination and exploitation activities can be found on the project's website: www.sestosenso.eu.

Periodic updates about publications and general project activities have been also published on

- X: twitter.com/sestosenso_eu
- Linkedin: https://www.linkedin.com/company/sestosenso-eu/

The Sestosenso Consortium

PeSkin demonstrator

Cooperation inside car chassis

Sestosenso logo

Sestosenso Team

OptoSkin poster at ICRA 2024

Proximity sensing inside car chassis

Sestosenso ecosystem. Proxy-tactile systems on robots and exoscheletons for collaborative tasks.

Preliminary integration of Sestosenso proxy-skin on a robot.

Sestosenso roll-up OptoSkin and ProxySkin

Optoskin Brochure

Graphical image of OptoSkin

Graphical image of OptoSkin on robot

OptoSkin Brochure for Innovation Radar