Robotic tEchnologies for the Manipulation of cOmplex DeformablE Linear objects

Project Information

REMODEL

Grant agreement ID: 870133

DOI 10.3030/870133

Project closed

EC signature date 11 October 2019

Start date 1 November 2019

End date 31 October 2023

Funded under

• INDUSTRIAL LEADERSHIP - Leadership in enabling and industrial technologies - Advanced manufacturing and processing

Total cost € 5 920 412,50

EU contribution € 5 920 412,50

Coordinated by

ALMA MATER STUDIORUM - UNIVERSITA DI BOLOGNA
Italy

Deliverables

Documents, reports (13)

CAD platform interface

The CAD platform interface documentation and software will be released

Second assessment of system performance

This deliverable will report the system performance from the point of view of the impact Imp1 Imp2 and Imp3 measured through periodic measurements and surveys carried out at different project development stages

Stakeholder analysis and mapping

A report on the stakeholder analysis, dissemination and communication activities tailored to each group.

Intermediate exploitation plan

An updated plan on management procedures and channels for exploitation and protection of the project results

Interactive perception

A technical report or a publication on interactive perception in deformable linear objects handling

Robotic platforms

A description of the robotic platforms for the evaluation of the scenarios envisaged in the project

Force/tactile and pressure/proximity sensors

These deliverables include hardware software and technical documentation of the sensory system components

Planner architecture

A technical report describing both the architecture and the data exchanged between the different components.

Preliminary exploitation plan

Preliminary plan on management procedures and channels for exploitation and protection of the project results.

Final exploitation plan

An updated plan on management procedures and channels for exploitation and protection of the project results including business plans for the project SMEs and guidelines for the market expansion

First assessment of system performance

This deliverable will report the system performance from the point of view of the impact Imp1, Imp2 and Imp3 measured through periodic measurements and surveys carried out at different project development stages.

Risk assessments, safety requirements and measures

The evaluation of potential risks for the considered use cases will be detailed and the definition of the safety requirements will be provided in this deliverable. The safety measures to be introduced in the User Interface and in the robotic platform will be detailed.

Third assessment of system performance

This deliverable will report the system performance from the point of view of the impact Imp1 Imp2 and Imp3 measured through periodic measurements and surveys carried out at different project development stages

Demonstrators, pilots, prototypes (6)

Cable grasping

A technical document or a publication reporting the cable grasping strategy

Skill-based teaching by demonstration

A demonstrator of the skill-based teaching by demonstration will be released.

Wiring harness manipulation

A technical document or a publication reporting the wiring harness manipulation strategy.

Bimanual cable manipulation

A technical document or a publication reporting the bimanual cable manipulation strategy.

User Interface

The User Interface will be released together with its documentation and a report on validation tests.

Report on the robot safety tests

This document will report the results of the safety test performed on the robotic platform. This deliverable includes both the safety controller documentation and implementation in the robotic platform. An updated version of this deliverable will be submitted at month 48.

Open Research Data Pilot (3)

Initial data management plan

Initial plan on the management of the project data.

Final data management plan

Final plan on the management of the project data

Intermediate data management plan

Intermediate plan on the management of the project data.

Websites, patent fillings, videos etc. (3)

Dissemination and communication plan

Initial plan on the dissemination and communication activities (M6); First update (M24); second update (M42).

Project website

The REMODEL website will be set up. The structure and the contents are detailed in section 2.2.

Report on the dissemination activities and materials

Visual identity, promotion materials, engagement with stakeholders, media activity, technical dissemination.

Publications

Other (1)

Effective Deployment of CNNs for 3DoF Pose Estimation and Grasping in Industrial Settings

Author(s): De Gregorio, Daniele; Zanella, Riccardo; Palli, Gianluca; Di Stefano, Luigi
Published in: Issue 1, 2020
Publisher: IEEE

Peer reviewed articles (32)

An Approach based on Machine Vision for the identification and shape estima-tion of deformable linear objects

Author(s): Pablo Malvido Fresnillo, Saigopal Vasudevan, Wael M. Mohammed, Jose L. Martinez Lastra (TAU), Jose A. Perez Garcia (UVIGO).
Published in: Mechatronics, 2023, ISSN 0957-4158
Publisher: Pergamon Press Ltd.
DOI: 10.1016/j.mechatron-ics.2023.103085

Exploiting In-Hand Knowledge in Hybrid Joint-Cartesian Mapping for Anthropomorphic Robotic Hands

Author(s): Roberto Meattini; Davide Chiaravalli; Gianluca Palli; Claudio Melchiorri
Published in: IEEE Robotics and Automation Letters, Issue 3, 2021, Page(s) 5517 - 5524, ISSN 2377-3766
Publisher: IEEE
DOI: 10.1109/lra.2021.3078658

Optical Force/Tactile Sensors for Robotic Applications

Author(s): Marco Costanzo and Salvatore Pirozzi
Published in: IEEE Instrumentation & Measurement MagazineSpecial Issue on “Optical measurements”, 2021, ISSN 1094-6969
Publisher: Institute of Electrical and Electronics Engineers
DOI: 10.1109/mim.2021.9491003

Deformable Linear Objects 3D Shape Estimation and Tracking From Multiple 2D Views

Author(s): Alessio Caporali, Kevin Galassi, Gianluca Palli
Published in: IEEE Robotics and Automation Letters, 2023, ISSN 2377-3766
Publisher: IEEE
DOI: 10.1109/lra.2023.3273518

Proximity Sensor for Thin Wire Recognition and Manipulation

Author(s): Andrea Cirillo, Gianluca Laudante, Salvatore Pirozzi
Published in: This article belongs to the Special Issue State-of-Art in Sensors for Robotic Applications, 2021, ISSN 2075-1702
Publisher: MDPI
DOI: 10.3390/machines9090188

Effective Deployment of CNNs for 3DoF Pose Estimation and Grasping in Industrial Settings

Author(s): Daniele De Gregorio; Riccardo Zanella; Gianluca Palli; Luigi Di Stefano
Published in: 25th International Conference on Pattern Recognition, ICPR 2020, Issue 3, 2021, ISSN 1051-4651
Publisher: IEEE
DOI: 10.1109/icpr48806.2021.9411912

Robot Learning-Based Pipeline for Autonomous Reshaping of a Deformable Linear Object in Cluttered Backgrounds

Author(s): Riccardo Zanella; Gianluca Palli
Published in: IEEE Access, Vol 9, Pp 138296-138306 (2021), Issue 5, 2021, ISSN 2169-3536
Publisher: Institute of Electrical and Electronics Engineers Inc.
DOI: 10.1109/access.2021.3118209

Point Cloud Registration With Object-Centric Alignment

Author(s): Bare Luka Zagar; Ekim Yurtsever; Arne Peters; Alois C. Knoll
Published in: Crossref, Issue 21693536, 2022, ISSN 2169-3536
Publisher: Institute of Electrical and Electronics Engineers Inc.
DOI: 10.1109/access.2022.3191352

A Weakly Super-vised Semi-Automatic Image Labeling Approach for Deformable Linear Objects

Author(s): Alessio Caporali; Matteo Pantano; Lucas Janisch; Daniel Regulin; Gianluca Palli; Dongheui Lee
Published in: IEEE Robotics and Automation Letters, 2023, ISSN 2377-3766
Publisher: IEEE
DOI: 10.1109/lra.2023.3234799

Extrinsic Calibration of an Eye-In-Hand 2D LiDAR Sensor in Unstructured Environments Using ICP

Author(s): Arne Peters, Adam Schmidt, Alois C. Knoll
Published in: IEEE Robotics and Automation Letters, Issue 5/2, 2020, Page(s) 929-936, ISSN 2377-3766
Publisher: IEEE
DOI: 10.1109/lra.2020.2965878

Robot Programming by Demonstration: Trajectory Learning Enhanced by sEMG-Based User Hand Stiffness Estimation

Author(s): L Biagiotti, R Meattini, D Chiaravalli, G Palli, C Melchiorri
Published in: IEEE Transactions on Robotics, 2023, ISSN 1552-3098
Publisher: Institute of Electrical and Electronics Engineers
DOI: 10.1109/tro.2023.3258669

Simulative Evaluation of a Joint-Cartesian Hybrid Motion Mapping for Robot Hands Based on Spatial In-Hand Information

Author(s): R Meattini, D Chiaravalli, G Palli, C Melchiorri
Published in: Frontiers in Robotics and AI, 2022
Publisher: Frontiers
DOI: 10.3389/frobt.2022.878364

Tactile sensor data interpretation for estimation of wire features

Author(s): Andrea Cirillo, Gianluca Laudante, Salvatore Pirozzi
Published in: MDPI Electronics, 2021, ISSN 2079-9292
Publisher: MDPI
DOI: 10.3390/electronics10121458

Vision-Based Robotic Solution for Wire Insertion with an Assigned Label Orientation

Author(s): Pasquale Cirillo, Gianluca Laudante, Salvatore Pirozzi
Published in: IEEE Access, 2021, Page(s) 102278-102289, ISSN 2169-3536
Publisher: Institute of Electrical and Electronics Engineers Inc.
DOI: 10.1109/access.2021.3098472

Tell Me, What Do You See?—Interpretable Classification of Wiring Harness Branches with Deep Neural Networks

Author(s): Piotr Kicki; Michal Bednarek; Paweł Lembicz; Grzegorz Mierzwiak; Amadeusz Szymko; Marek Kraft; Krzysztof Walas
Published in: Sensors (Basel, Switzerland), Issue 14248220, 2021, ISSN 1424-8220
Publisher: Multidisciplinary Digital Publishing Institute (MDPI)
DOI: 10.3390/s21134327

sEMG-Based Human-in-the-Loop Control of Elbow Assistive Robots for Physical Tasks and Muscle Strength Training

Author(s): Roberto Meattini, Davide Chiaravalli, Gianluca Palli, Claudio Melchiorri
Published in: IEEE Robotics and Automation Letters, Issue 5/4, 2020, Page(s) 5795-5802, ISSN 2377-3766
Publisher: IEEE
DOI: 10.1109/lra.2020.3010741

CAD-Based Robot Programming Solution for Wire Harness Manufacturing in Aeronautic Sector

Author(s): Javier González Huarte, Maite Ortiz de Zarate, and Aitor Ibarguren
Published in: Robotics, 2023, ISSN 2218-6581
Publisher: MDPI
DOI: 10.3390/robotics12050130

RT-DLO: Real-Time Deformable Linear Objects Instance Segmentation

Author(s): Alessio Caporali; Kevin Galassi; Bare Luka Žagar; Riccardo Zanella; Gianluca Palli; Alois C Knoll
Published in: IEEE Transac-tions on Industrial Informatics, 2023, ISSN 1551-3203
Publisher: Institute of Electrical and Electronics Engineers
DOI: 10.1109/tii.2023.3245641

A Safe and Energy Efficient Robotic System for Industrial Automatic Tests on Domestic Appliances: Problem Statement and Proof of Concept

Author(s): Wendwosen Bellete Bedada, Rawan Kalawoun, Ismayil Ahmadli, Gianluca Palli
Published in: Procedia Manufacturing, Issue 51, 2020, Page(s) 454-461, ISSN 2351-9789
Publisher: Elsevier
DOI: 10.1016/j.promfg.2020.10.064

Extending the motion planning framework—MoveIt with advanced manipulation functions for industrial applications

Author(s): Pablo Malvido Fresnillo, Saigopal Vasudevan, Wael M. Mohammed, Jose L. Martinez Lastra,Jose A. Perez Garcia
Published in: Robotics and Com-puter-Integrated Manufacturing, 2023, ISSN 0736-5845
Publisher: Pergamon Press Ltd.
DOI: 10.1016/j.rcim.2023.102559

Ariadne+: Deep Learning-based Augmented Framework for the Instance Segmentation of Wires

Author(s): Alessio Caporali; Riccardo Zanella; Daniele De Gregorio; Gianluca Palli
Published in: IEEE Transactions on Industrial Informatics, 2022, ISSN 1551-3203
Publisher: Institute of Electrical and Electronics Engineers
DOI: 10.1109/tii.2022.3154477

New model-based manipulation technique for reshaping deformable linear objects

Author(s): Alaa Khalifa; Gianluca Palli
Published in: The International Journal of Advanced Manufacturing Technology, Issue 6, 2022, Page(s) 3575–3583
Publisher: Springer
DOI: 10.1007/s00170-021-08107-x

Combining Unsupervised Muscle Co-Contraction Estimation With Bio-Feedback Allows Augmented Kinesthetic Teaching

Author(s): Roberto Meattini; Davide Chiaravalli; Luigi Biagiotti; Gianluca Palli; Claudio Melchiorri
Published in: IEEE Robotics and Automation Letters, Issue 4, 2021, Page(s) 6180 - 6187, ISSN 2377-3766
Publisher: IEEE
DOI: 10.1109/lra.2021.3092269

Symplectic Integration for Multivariate Dynamic Spline-Based Model of Deformable Linear Objects

Author(s): Alaa Khalifa; Gianluca Palli
Published in: Journal of Computational and Nonlinear Dynamics, Issue 4, 2021
Publisher: ASME
DOI: 10.48550/arxiv.2108.08935

Beyond the Baseline: 3D Reconstruction of Tiny Objects With Si ngle Camera Ste reo R obot

Author(s): Daniele De Gregorio; Matteo Poggi; Pierluigi Zama Ramirez; Gianluca Palli; Stefano Mattoccia; Luigi Di Stefano
Published in: IEEE Access, Issue 4, 2021, ISSN 2169-3536
Publisher: Institute of Electrical and Electronics Engineers Inc.
DOI: 10.1109/access.2021.3108626

Tactile Sensors for Parallel Grippers: Design and Characterization.

Author(s): Andrea Cirillo; Marco Costanzo; Gianluca Laudante; Salvatore Pirozzi
Published in: Sensors, Vol 21, Iss 1915, p 1915 (2021), Issue 5, 2021, ISSN 1424-8220
Publisher: Multidisciplinary Digital Publishing Institute (MDPI)
DOI: 10.3390/s21051915

Tactile Sensors for Parallel Grippers: Design and Characterization

Author(s): Andrea Cirillo, Marco Costanzo, Gianluca Laudante, Salvatore Pirozzi
Published in: Sensors, Issue 21/5, 2021, Page(s) 1915, ISSN 1424-8220
Publisher: Multidisciplinary Digital Publishing Institute (MDPI)
DOI: 10.3390/s21051915

Stability and Convergence Analysis of 3D Feature-Based Visual Servoing

Author(s): Marco Costanzo; Giuseppe De Maria; Ciro Natale; Antonio Russo
Published in: IEEE Ro-botics and Automation Letters, Issue 23773766, 2022, ISSN 2377-3766
Publisher: IEEE
DOI: 10.1109/LRA.2022.3211154

sEMG-Based Minimally Supervised Regression Using Soft-DTW Neural Networks for Robot Hand Grasping Control

Author(s): Roberto Meattini; Alessandra Bernardini; Gianluca Palli; Claudio Melchiorri
Published in: IEEE Robotics and Automation Letters, 2022, ISSN 2377-3766
Publisher: IEEE
DOI: 10.1109/lra.2022.3193247

Human to Robot Hand Motion Mapping Methods: Review and Classification

Author(s): Roberto Meattini; Raúl Suárez; Gianluca Palli; Claudio Melchiorri
Published in: Crossref, Issue 3, 2022, ISSN 1552-3098
Publisher: Institute of Electrical and Electronics Engineers
DOI: 10.1109/tro.2022.3205510

FASTDLO: Fast Deformable Linear Objects Instance Segmentation

Author(s): Caporali, A., Galassi, K., Zanella, R., Palli, G.
Published in: IEEE Robotics and Automation Letters, 2022, ISSN 2377-3766
Publisher: IEEE
DOI: 10.1109/lra.2022.3189791

A Method for Understanding and Digitizing Manipulation Activities using Programming by Demonstration in Robotic Applications

Author(s): Pablo Malvido Fresnillo, Saigopal Vasudevan, Wael M. Mohammed, Jose L. Martinez Lastra (TAU), Jose A. Perez Garcia (UVIGO).
Published in: Robotics and Autonomous Systems, 2023, ISSN 0921-8890
Publisher: Elsevier BV
DOI: 10.1016/j.robot.2023.104556

Conference proceedings (23)

New Metrics for Industrial Depth Sensors Evaluation for Precise Robotic Applications

Author(s): Cop, Konrad P.; Peters, Arne; Žagar, Bare L; Hettegger, Daniel; Knoll, Alois C
Published in: 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2021
Publisher: IEEE
DOI: 10.1109/iros51168.2021.9636322

An approach for the bimanual manipulation of a deformable linear object using a dual-arm industrial robot : cable routing use case

Author(s): Pablo Malvido Fresnillo; Saigopal Vasudevan; Wael M. Mohammed
Published in: 5th IEEE International Conference on Industrial Cyber-Physical Systems (ICPS), 24 – 26 May 2022, Issue 1, 2022
Publisher: IEEE
DOI: 10.1109/icps51978.2022.9816981

A Low Cost Tactile Sensor for Large Surfaces Based on Deformable Skin with Embedded IMU

Author(s): Yuki Iwamoto, Roberto Meattini, Davide Chiaravalli, Gianluca Palli, Koji Shibuya, Claudio Melchiorri
Published in: 2020 IEEE Conference on Industrial Cyberphysical Systems (ICPS), 2020, Page(s) 501-506, ISBN 978-1-7281-6389-5
Publisher: IEEE
DOI: 10.1109/icps48405.2020.9274737

Robotic Wires Manipulation for Switchgear Cabling and Wiring Harness Manufacturing

Author(s): Kevin Galassi, Gianluca Palli
Published in: 2021 4th IEEE International Conference on Industrial Cyber-Physical Systems (ICPS), 2021
Publisher: IEEE
DOI: 10.1109/icps49255.2021.9468128

Validating DLO Models from Shape Observation

Author(s): Gianluca Palli, Salvatore Pirozzi
Published in: 2021
Publisher: IEEE/ASME
DOI: 10.1109/aim46487.2021.9517570

A Cyber-Physical System for Clothes Detection, Manipulation and Washing Machine Loading

Author(s): Alessio Caporali; Wendwosen Bellete Bedada; Gianluca Palli
Published in: 2021 4th IEEE International Conference on Industrial Cyber-Physical Systems (ICPS), Issue 4, 2021, ISBN 978-1-7281-6207-2
Publisher: IEEE
DOI: 10.1109/icps49255.2021.9468189

Combining Vision and Tactile Data for Cable Grasping

Author(s): Alessio Caporali; Kevin Galassi; Gianluca Laudante; Gianluca Palli; Salvatore Pirozzi
Published in: AIM, Issue 2, 2021
Publisher: IEEE/ASME
DOI: 10.1109/aim46487.2021.9517447

Auto-generated Wires Dataset for Semantic Segmentation with Domain-Independence

Author(s): Zanella, R., Caporali, A., Tadaka, K., De Gregorio, D., Palli, G.
Published in: 2021
Publisher: IEEE
DOI: 10.1109/icccr49711.2021.9349395

Graphical visualization of contact forces and hand movements during in-hand manipulation

Author(s): Raúl Suárez, Andrés Montaño
Published in: 27th Annual Conference of the IEEE Industrial Electronics Society, 2022
Publisher: IEEE
DOI: 10.1109/etfa52439.2022.9921664

Training an Under-actuated Gripper for Grasping Shallow Objects Using Reinforcement Learning

Author(s): Wael M. Mohammed, Miroslaw Nejman, Fernando Castano, Jose L. Martinez Lastra, Stanislaw Strzelczak, Alberto Villalonga
Published in: 2020 IEEE Conference on Industrial Cyberphysical Systems (ICPS), 2020, Page(s) 493-498, ISBN 978-1-7281-6389-5
Publisher: IEEE
DOI: 10.1109/icps48405.2020.9274727

Towards the automation of wire harness manufacturing: a robotic manipulator with sensorized fingers

Author(s): Andrea Govoni, Gianluca Laudante, Michele Mirto, Ciro Natale, Salvatore Pirozzi
Published in: 2023 9th International Conference on Control, Decision and Infor-mation Technolo-gies (CoDIT), 2023
Publisher: IEEE

3D DLO Shape Detection and Grasp Planning from Multiple 2D Views

Author(s): Alessio Caporali; Kevin Galassi; Gianluca Palli
Published in: AIM, Issue 10, 2021
Publisher: IEEE/ASME
DOI: 10.1109/aim46487.2021.9517655

Deformable Objects Grasping and Shape Detection with Tactile Fingers and Industrial Grippers

Author(s): Pablo Malvido Fresnillo; Saigopal Vasudevan; Wael M. Mohammed; Jose L. Martinez Lastra; Gianluca Laudante; Salvatore Pirozzi; Kevin Galassi; Gianluca Palli
Published in: ICPS, Issue 1, 2021
Publisher: IEEE
DOI: 10.1109/icps49255.2021.9468151

DLOFTBs – Fast Tracking of Deformable Linear Objects with B-splines

Author(s): Piotr Kicki, Amadeusz Szymko, Krzysztof Walas
Published in: IEEE International Conference on Robotics and Automation (ICRA), 2023
Publisher: IEEE
DOI: 10.1109/icra48891.2023.10160437

Real-Time Instance Segmentation of Pedestrians using Transfer Learning

Author(s): Bare Luka Žagar; Tobias Preintner; Alois C. Knoll; Ekim Yurtsever
Published in: 2022 27th International Conference on Automation and Computing (ICAC), 2022, ISBN 978-1-6654-9807-4
Publisher: IEEE
DOI: 10.1109/icac55051.2022.9911121

Towards a Twisted String Actuated Haptic Device: Experimental Testing of a 2-D Virtual Environment and Teleoperation Interface

Author(s): Linda Feenstra; Umberto Scarcia; Riccardo Zanella; Roberto Meattini; Davide Chiaravalli; Gianluca Palli; Claudio Melchiorri
Published in: 2021 20th International Conference on Advanced Robotics (ICAR), 2022
Publisher: IEEE
DOI: 10.1109/icar53236.2021.9659420

Cable Detection and Manipulation for DLO-in-Hole Assembly Tasks

Author(s): Alessio Caporali, Kevin Galassi, Gianluca Palli
Published in: 5th IEEE International Conference on Industrial Cyber-Physical Systems (ICPS), 24 – 26 May 2022, 2022
Publisher: IEEE
DOI: 10.1109/icps51978.2022.9817006

An Approach for Modeling Grasping Configuration Using Ontology-based Taxonomy

Author(s): Wael M. Mohammed, Pablo Malvido Fresnillo, Saigopal Vasudevan, Ziga Gosar, Jose L. Martinez Lastra
Published in: 2020 IEEE Conference on Industrial Cyberphysical Systems (ICPS), 2020, Page(s) 507-513, ISBN 978-1-7281-6389-5
Publisher: IEEE
DOI: 10.1109/icps48405.2020.9274760

Wire Grasping by Using Proximity and Tactile Sensors

Author(s): Cirillo Andrea; Laudante Gianluca; Pirozzi Salvatore
Published in: 5th IEEE International Conference on Industrial Cyber-Physical Systems (ICPS), 24 – 26 May 2022, Issue 2, 2022
Publisher: IEEE
DOI: 10.1109/ICPS51978.2022.9816936

Copy and Paste Augmentation for Deformable Wiring Harness Bags Segmentation

Author(s): Bare Luka Žagar, Alessio Caporali, Amadeusz Szymko, Piotr Kicki, Krzysztof Walas, Gianluca Palli, Alois C Knoll
Published in: 2023 IEEE/ASME International Conference on Advanced Intelligent Mechatronics (AIM), 2023
Publisher: IEEE
DOI: 10.1109/aim46323.2023.10196168

Pointcloud-based Identification of Optimal Grasping Poses for Cloth-like Deformable Objects

Author(s): Alessio Caporali, Gianluca Palli
Published in: 2020 25th IEEE International Conference on Emerging Technologies and Factory Automation (ETFA), 2020, Page(s) 581-586, ISBN 978-1-7281-8956-7
Publisher: IEEE
DOI: 10.1109/etfa46521.2020.9211879

Model-based Manipulation of Deformable Linear Objects by Multivariate Dynamic Splines

Author(s): Gianluca Palli
Published in: 2020 IEEE Conference on Industrial Cyberphysical Systems (ICPS), 2020, Page(s) 520-525, ISBN 978-1-7281-6389-5
Publisher: IEEE
DOI: 10.1109/icps48405.2020.9274730

FASTDLO: Towards Real-Time Perception of Deformable Linear Objects

Author(s): Caporali, Alessio; Palli, Gianluca; Galassi, Kevin
Published in: Fourth Italian Conference on Robotics and Intelligent Machines, Issue 1, 2022, ISBN 9788894580532
Publisher: Zenodo
DOI: 10.5281/zenodo.7531208