Periodic Reporting for period 2 - Sestosenso (Physical Cognition for Intelligent Control and Safe Human-Robot Interaction)
Periodo di rendicontazione: 2024-04-01 al 2025-09-30
Sestosenso developed hardware and software enabling robots to self-adapt to changing conditions and safely switch between autonomous and human-interactive modes in cooperative scenarios.
Guided by Industry 4.0 and 5.0 principles, the project addressed the demand for robots that operate safely with minimal infrastructure, reducing setup time and cost while enhancing shopfloor flexibility. It also explored human factors and operator acceptance of human-robot interaction (HRI) technologies.
Sestosenso advanced sensing technologies—from hardware to cognitive control—through networks of embedded proximity and tactile sensors across the robot body, creating unified proxy-tactile perception to support:
- Autonomous and human-driven robot operations.
- New task capabilities via joint proxy-tactile feedback.
- Improved safety in human-robot cooperation and collaboration.
- Robot proximity and tactile based self-localization and environment mapping.
- Scalable manufacturing of large-area distributed sensors.
Rooted in AI, data, and robotics innovation, Sestosenso focused on human-centered technologies for industrial and service applications, demonstrated through three use cases:
A. Car assembly: safe human-robot cooperative tasks in dynamic environments.
B. Logistics: large-object handling and packaging through whole-arm manipulation.
C. Grape harvesting: outdoor operation with unmodeled obstacles using sensorized exoskeletons.
Guided by Industry 4.0 and 5.0 principles, the project addressed the demand for robots that operate safely with minimal infrastructure, reducing setup time and cost while enhancing shopfloor flexibility. It also explored human factors and operator acceptance of human-robot interaction (HRI) technologies.
Sestosenso advanced sensing technologies—from hardware to cognitive control—through networks of embedded proximity and tactile sensors across the robot body, creating unified proxy-tactile perception to support:
- Autonomous and human-driven robot operations.
- New task capabilities via joint proxy-tactile feedback.
- Improved safety in human-robot cooperation and collaboration.
- Robot proximity and tactile based self-localization and environment mapping.
- Scalable manufacturing of large-area distributed sensors.
Rooted in AI, data, and robotics innovation, Sestosenso focused on human-centered technologies for industrial and service applications, demonstrated through three use cases:
A. Car assembly: safe human-robot cooperative tasks in dynamic environments.
B. Logistics: large-object handling and packaging through whole-arm manipulation.
C. Grape harvesting: outdoor operation with unmodeled obstacles using sensorized exoskeletons.
During the project, the following scientific and technical (S/T) activities were carried out:
- Development of proxy-tactile sensors adaptable to various robot types and geometries (WP2).
- Design of large-area tactile sensor concepts for cost-effective and reliable components (WP2).
- Implementation of real-time proxy-tactile sensor network nodes using EtherCAT and real-time wireless systems (WP3).
- Development of multimodal middleware for real-time proxy-tactile data acquisition and robot control (WP4).
- Research and implementation of proxy-tactile–based control methods for robot self-awareness, gesture recognition, and task-oriented HRI (WP4/WP5/WP6).
- Definition, integration, and engineering of experimental use cases (WP7).
- Assessment of human factors and user acceptance of the technology (WP1/WP5/WP7).
Key project innovations:
- OptoSkin: patented, optical-based, cost-effective tactile sensing technology.
- ProxySkin: distributed large-area proximity and tactile sensor system.
- Cyskin+: fully embedded capacitive large-area tactile sensing technology.
- PeSkin / PeSkin+: printed electronic capacitive and multilayer tactile sensor technologies.
- Multimodal Middleware: real-time acquisition and integration framework for heterogeneous proxy-tactile sensors.
Main S&T achievements:
- Integration of proxy-tactile sensors into three industrial workstations (Use Cases 1–3).
- Studies on safe human-robot interaction and operator ergonomics.
- Application of geometric algebra for robotic manipulation and collision detection.
- Development of gesture recognition and proxy-tactile exploratory behaviors.
- Real-time wireless sensor networks for mobile platforms (e.g. exoskeletons).
- Integration and validation of proxy-tactile technologies across multiple robotic systems.
- Development of proxy-tactile sensors adaptable to various robot types and geometries (WP2).
- Design of large-area tactile sensor concepts for cost-effective and reliable components (WP2).
- Implementation of real-time proxy-tactile sensor network nodes using EtherCAT and real-time wireless systems (WP3).
- Development of multimodal middleware for real-time proxy-tactile data acquisition and robot control (WP4).
- Research and implementation of proxy-tactile–based control methods for robot self-awareness, gesture recognition, and task-oriented HRI (WP4/WP5/WP6).
- Definition, integration, and engineering of experimental use cases (WP7).
- Assessment of human factors and user acceptance of the technology (WP1/WP5/WP7).
Key project innovations:
- OptoSkin: patented, optical-based, cost-effective tactile sensing technology.
- ProxySkin: distributed large-area proximity and tactile sensor system.
- Cyskin+: fully embedded capacitive large-area tactile sensing technology.
- PeSkin / PeSkin+: printed electronic capacitive and multilayer tactile sensor technologies.
- Multimodal Middleware: real-time acquisition and integration framework for heterogeneous proxy-tactile sensors.
Main S&T achievements:
- Integration of proxy-tactile sensors into three industrial workstations (Use Cases 1–3).
- Studies on safe human-robot interaction and operator ergonomics.
- Application of geometric algebra for robotic manipulation and collision detection.
- Development of gesture recognition and proxy-tactile exploratory behaviors.
- Real-time wireless sensor networks for mobile platforms (e.g. exoskeletons).
- Integration and validation of proxy-tactile technologies across multiple robotic systems.
All scientific and technical goals were achieved at various TRLs and successfully integrated into the three use cases. The project’s results have been published in high-quality peer-reviewed journals and conference proceedings, with additional publications forthcoming.
OptoSkin (patented) and ProxySkin have been identified as market-ready innovations for further development under the EIC Transition program, while CySkin+, PeSkin, PeSkin+, and the Multimodal Middleware are under review by the Innovation Radar initiative.
Sestosenso was demonstrated in automotive (car assembly), logistics (robotized packaging), and agriculture (assistive robotics). Its technologies are now being transferred to assistive robotics (TRL4 robot avatars). Emerging applications include shipyard and aerospace assembly, mobile logistics, and education and training (XR and animatronic avatars). Beyond these, Sestosenso supports privacy-compliant HRI scenarios where video or audio sensing is unsuitable.
The consortium—comprising universities, research institutes, and companies—prioritized R&D, knowledge transfer, and indirect industrial impact over immediate commercialization, fostering new R&D exploitation paths. Several partners are also exploring long-term opportunities for spin-offs and market deployment.
Details about the project, and the dissemination and exploitation activities can be found on the project's website: www.sestosenso.eu.
Periodic updates about publications and general project activities have been also published on
- X: twitter.com/sestosenso_eu
- Linkedin: https://www.linkedin.com/company/sestosenso-eu/(si apre in una nuova finestra)
OptoSkin (patented) and ProxySkin have been identified as market-ready innovations for further development under the EIC Transition program, while CySkin+, PeSkin, PeSkin+, and the Multimodal Middleware are under review by the Innovation Radar initiative.
Sestosenso was demonstrated in automotive (car assembly), logistics (robotized packaging), and agriculture (assistive robotics). Its technologies are now being transferred to assistive robotics (TRL4 robot avatars). Emerging applications include shipyard and aerospace assembly, mobile logistics, and education and training (XR and animatronic avatars). Beyond these, Sestosenso supports privacy-compliant HRI scenarios where video or audio sensing is unsuitable.
The consortium—comprising universities, research institutes, and companies—prioritized R&D, knowledge transfer, and indirect industrial impact over immediate commercialization, fostering new R&D exploitation paths. Several partners are also exploring long-term opportunities for spin-offs and market deployment.
Details about the project, and the dissemination and exploitation activities can be found on the project's website: www.sestosenso.eu.
Periodic updates about publications and general project activities have been also published on
- X: twitter.com/sestosenso_eu
- Linkedin: https://www.linkedin.com/company/sestosenso-eu/(si apre in una nuova finestra)