Reconstructing the Past: Artificial Intelligence and Robotics Meet Cultural Heritage

Project Information

RePAIR

Grant agreement ID: 964854

DOI

10.3030/964854

Project closed

EC signature date 21 December 2020

Start date 1 September 2021

End date 31 October 2025

Funded under

• EXCELLENT SCIENCE - Future and Emerging Technologies (FET)

Total cost

€ 3 522 010,00

EU contribution

€ 3 522 007,00

Coordinated by

UNIVERSITA CA' FOSCARI VENEZIA
Italy

CORDIS provides links to public deliverables and publications of HORIZON projects.

Links to deliverables and publications from FP7 projects, as well as links to some specific result types such as dataset and software, are dynamically retrieved from OpenAIRE .

Deliverables

Other (3)

Integration guidelines and plan 3

Describes the 3rd version of the guidelines and plans needed for a successful integration of the different components of the RePAIR system.

Integration guidelines and plan 1

Describes the 1st version of the guidelines and plans needed for a successful integration of the different components of the RePAIR system.

Integration guidelines and plan 2

Describes the 2nd version of the guidelines and plans needed for a successful integration of the different components of the RePAIR system.

Documents, reports (13)

Setup of integration and sharing repository

Describes the setup of the repositories to be created to facilitate interaction during the development phase

Fragment recognition and pose estimation

Describes the work done in WP2.2 where we shall develop matching algorithms to recognise and estimate the pose of the fragments, given possibly partial views and different sensor modalities.

Bi-manual motion planning

Describes the work done in WP5.2 whew we will develop efficient motion planning techniques for the bi-manual robotic system.

Acquisition and 3D modelling of fragments

Describes the work done in WP21 which will focus on deriving an accurate 3D model for each archaeological fragment together with a mapped texture so as to obtain appearance and geometrybased descriptors that will be used by the puzzlesolving and grasping modules

Hand pose selection for grasp planning using vision sensors

Describes the work done in WP51 which aims to learn by explorationimitation the parameters of the visual servoing controllers for softhands

Adaptable bi-manual manipulation platform

Describes the work done in WP4.2 which focuses on the development and control of a dedicated twin robot arm to execute the manipulation tasks of the fresco pieces.

Quality Management plan

Sets the quality standards to be pursued in the whole project

Exploitation plan (including IP provisions)

Describes the exploitation plan of the project including IP provisions

Manipulation platform requirements

Describes the work done in WP41 and outlines the desired specifications of the robot and vision systems based on the requirements of the manipulation tasks with the fresco pieces

Style priors and their formal description for appearance prediction

Describes the work done in WP31 where highlevel knowledge of fresco styles will be formalised in ways that allow its integration into the puzzle solving algorithm

View selection for visual perception tasks

Describes the work done in WP5.3 which will develop optimisation methods to select the arm pose that provides the most informative visual input for a successful grasp.

End-effectors modules

Describes the work done in WP 4.3 where we will leverage on soft robotic principles together with under-actuation to develop soft and adaptive end-effectors.

Strategic communication and dissemination plan

Describes the strategic communication and dissemination plan of the project

Open Research Data Pilot (1)

Data Management Plan

It provides indications as to the management of data in relation to the H2020 Open Research Data Pilot Data Management Plan might be updated during the project implementation

Websites, patent fillings, videos etc. (1)

Project website and social media (including YouTube channel)

Website of the project and social media platforms

Publications

Conference proceedings (48)

Semantic Motif Segmentation of Archaeological Fresco Fragments

Author(s): A. Enayati, L. Palmieri, S. Vascon, M. Pelillo, S. Aslan
Published in: Proceedings of the IEEE International Conference on Computer Vision - 4th Workshop on e-Heritage (ICCVW), 2023
Publisher: IEEE

Reassembling Broken Objects Using Breaking Curves

Author(s): Ali Alagrami, Luca Palmieri, Sinem Aslan, Marcello Pelillo, Sebastiano Vascon
Published in: Lecture Notes in Computer Science, Pattern Recognition, 2024, Page(s) 197-208
Publisher: Springer Nature Switzerland
DOI: 10.1007/978-3-031-78456-9_13

“Reassembling Broken Objects using Breaking Curves”.

Author(s): A. Alagrami, L. Palmieri, S. Aslan, M. Pelillo, S. Vascon,
Published in: Proceedings of the IEEE International Conference on Pattern Recognition, 2024
Publisher: IEEE

Viewpoint Push Planning for Mapping of Unknown Confined Spaces.

Author(s): N. Dengler, S. Pan, V. Kalagaturu, R. Menon, M. Dawood, M. Bennewitz.
Published in: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)., 2023
Publisher: IEEE

Map Space Belief Prediction for Manipulation-Enhanced Mapping.

Author(s): J.M.C Marques, N. Dengler, T. Zaenker, J. Mücke, S. Wang, M. Bennewitz, K. Hauser.
Published in: Robotics: Science and Systems (RSS), 2025
Publisher: RSS

3DSGrasp: 3D Shape-Completion for Robotic Grasp.

Author(s): Seyed S. Mohammadi, Nuno F. Duarte, Dimitrios Dimou, Yiming Wang, Matteo Taiana, Pietro Morerio, Atabak Dehban, Alexandre Bernardino, Alessio del Bue, and José Santos-Victor.
Published in: 2023 IEEE International Conference on Robotics and Automation (ICRA), pp. 3815-3822, 2023
Publisher: IEEE

Solving Jigsaw Puzzles in the Wild: Human-Guided Reconstruction of Cultural Heritage Fragments

Author(s): Omidreza Safaei, Sinem Aslan, Sebastiano Vascon, Luca Palmieri, Marina Khoroshiltseva, Marcello Pelillo
Published in: 2025 IEEE 35th International Workshop on Machine Learning for Signal Processing (MLSP), 2025, Page(s) 1-6
Publisher: IEEE
DOI: 10.1109/mlsp62443.2025.11204324

Diffassemble: A Unified Graph-Diffusion Model for 2D and 3D Reassembly.

Author(s): G. Scarpellini, S.Fiorini, F. Giuliari, P. Morerio, A .Del Bue.
Published in: 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)., 2024
Publisher: IEEE/CVF

Decentralized Collision-Aware Inverse Kinematics Solver for Heterogeneous Multi- Arm Sistem.

Author(s): N. Dengler, S. Pan, V. Kalagaturu, R. Menon, M. Dawood, M. Bennewitz, Dawnlk
Published in: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2023
Publisher: IEEE

Benchmarking shape completion methods for robotic grasping.

Author(s): João Balão, Atabak Dehban, Plinio Moreno, and José Santos-Victor.
Published in: 2022 IEEE International Conference on Development and Learning (ICDL), pp. 223-230, 2022
Publisher: IEEE

”Learning Goal-Directed Object Pushing in Cluttered Scenes with Location- Based Attention

Author(s): S. Marangoz, R. Menon, N. Dengler, M. Bennewitz
Published in: “ EEE-RAS International Conference on Humanoid Robots (Humanoids), 2023, 2023
Publisher: EEE-RAS

In-Hand Manipulation of Unseen Objects Through 3D Vision.

Author(s): Martim Pereira, Dimitrios Dimou, and Plinio Moreno.
Published in: Iberian Robotics Conference, pp. 163-174., 2022
Publisher: Springer International Publishing

Multi-phase relaxation labeling for square jigsaw puzzle solving.

Author(s): B. Vardi, M. Khoroshilteva, A. Torcinovich, M. Pelillo and O. Ben-Shahar.
Published in: Proceedings of the 18th International Joint Conference on Computer Vision, Imaging and Computer Graphics: Theory and Applications (VISAPP), 2023
Publisher: VISAPP

3DSGrasp: 3D Shape-Completion for Robotic Grasp

Author(s): Seyed S. Mohammadi, Nuno F. Duarte, Dimitrios Dimou, Yiming Wang, Matteo Taiana, Pietro Morerio, Atabak Dehban, Plinio Moreno, Alexandre Bernardino, Alessio Del Bue, José Santos-Victor
Published in: 2023 IEEE International Conference on Robotics and Automation (ICRA), 2024, Page(s) 3815-3822
Publisher: IEEE
DOI: 10.1109/icra48891.2023.10160350

GRASPLAT: Enabling dexterous grasping through novel view synthesis

Author(s): "Matteo Bortolon, Nuno Ferreira Duarte, Plinio Moreno, Fabio Poiesi, Jose Santos-Victor, and Alessio Del Bue. """
Published in: 2025 IEEE International Conference on Intelligent Robots and Systems (IROS), 2025
Publisher: IEEE

Reassembling broken objects using breaking curves.

Author(s): L. Palmieri. S. Aslan, A. Alagrami, M. Pelillo, and S. Vascon.
Published in: CVPR 2023 Workshop on 3D Vision and Robotics, 2023
Publisher: CVPR

“Nash Meets Wertheimer: Using Good Continuation in Jigsaw Puzzles”

Author(s): M. Khoroshiltseva, L. Palmieri, S. Aslan, S. Vascon, M. Pelillo,
Published in: Proceedings of the 17th Asian Conference on Computer Vision (ACCV), 2024
Publisher: ACCV

Viewpoint Push Planning for Mapping of UNknow Confined Spaces

Author(s): B. Kreis, R Menon, B.K. Adinarayan, j.de Heuvel, M.Bennewitz
Published in: International Conference on Intelligent Autonomous Systems (IAS- 18), 2023
Publisher: IEEE

6DGS: “6D Pose Estimation from a Single Image and a 3D Gaussian Splatting Model

Author(s): M. Bortolon, T. Tsesmelis, J. Stuart, F. Poiesi and A. Del Bue.”
Published in: Proceedings of the European Conference on Computer Vision (ECCV), 2024
Publisher: ECCV

Estimating Object Physical Properties from RGB-D Vision and Depth Robot Sensors Using Deep Learning.

Author(s): Ricardo Pedreiras Cardoso, and Plinio Moreno.
Published in: Gonçalves, N., Oliveira, H.P., Sánchez, J.A. (eds) Pattern Recognition and Image Analysis. Lecture Notes in Computer Science, vol 15937, 2025
Publisher: IbPRIA

3DSGrasp: 3D Shape-Completion for Robotic Grasp.

Author(s): S.S. Mohammadi, N. Ferreira Duarte, D. Dimou, Y. Wang, M. Taiana, P. Morerio, A. Dehban, P. Moreno, A. Bernardino, A. Del Bue and J. Santos-Victor.
Published in: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), 2023
Publisher: IEEE

The Dataset of Paintings from the Pompeii Archive: A task of the RePAIR Project

Author(s): M.C. Napolitano, S. Giudice, E. Gravina, G. Scapati, A.Zambrano, G. Zuchtriegel, S. Aslan, M. Khoroshiltseva, L. Palmieri, M. Pelillo, S. Vascon, O. Ben-Shahar, Y. Ohayon, O. Shahar, A. Traviglia, . Italy 6-8 July (2023);
Published in: Proceedings of AIRPA VI Pictura Parietum,, 2023
Publisher: AIRPA VI

IFFNeRF: “Initialisation Free and Fast 6DoF pose estimation from a single image and a NeRF model”.

Author(s): M. Bortolon, T. Tsesmelis, J. Stuart, F. Poiesi and A. Del Bue.
Published in: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA),, 2024
Publisher: ICRA

Force Feedback Control For Dexterous Robotic Hands Using Conditional Postural Synergies.

Author(s): "Dimitrios Dimou, José Santos-Victor, and Plinio Moreno."""
Published in: 2022 IEEE-RAS 21st International Conference on Humanoid Robots (Humanoids), pp. 674-679., 2022
Publisher: IEEE

Improving the Estimation of Object mass from images.

Author(s): João Andrade, and Plinio Moreno.
Published in: 2023 IEEE International Conference on Autonomous Robot Systems and Competitions (ICARSC), pp. 199-206., 2023
Publisher: IEEE

“Re-assembling the past: The RePAIR dataset and benchmark for real-world 2D and 3D puzzle solving.

Author(s): T. Tsesmelis, L. Palmieri, M. Khoroshiltseva, A. Islam, G. Elkin, O. Itzhak Shahar, G. Scarpellini, S. Fiorini, Y. Ohayon, N. Alali, S. Aslan, P. Morerio, S. Vascon, E. Gravina, M. C. Napolitano, G. Scarpati, G. Zuchtriegel, A. Spühler, M. E. Fuchs, S. James, O. Ben-Shahar, M. Pelillo, A. Del Bue. - TracData (2024);
Published in: Proceedings of the Thirty-Eighth Annual Conference on Neural Information” Processing Systems (NeurIPS), 2024
Publisher: NeurIPS

Pictorial and apictorial polygonal jigsaw puzzles from arbitrary number of crossing cuts.

Author(s): P. Harel, O.I. Shahar, and O. Ben-Shahar.
Published in: The International Journal of Computer Vision, 132, pp 3428–3462, 2024., 2024
Publisher: IEEE

Learning Goal-Directed Object Pushing in Cluttered Scenes with Location- Based Attention.

Author(s): N. Dengler, J. Del Aguila Ferrandis, J. Moura, S. Vijayakumar, M. Bennewitz.
Published in: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)., 2025
Publisher: IEEE

Learning Goal-Oriented Non-Prehensile Pushing in Cluttered Scenes

Author(s): Nils Dengler, David Großklaus, Maren Bennewitz
Published in: 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2024, Page(s) 1116-1122
Publisher: IEEE
DOI: 10.1109/iros47612.2022.9981873

The RePAIR Project: Datasets for archaeological and restoration studies in Pompeii.

Author(s): G. Zuchtriegel, M. A. Brunetto, E. Gravina, M. C. Napolitano, F. Ricciardi, A. Zambrano, M. Khoroshiltseva, L. Palmieri, M. Pelillo, S. Vascon, G. Elkin, O. I. Shahar, Y. Ohayon, N. Alali, and O. Ben-Shahar.
Published in: in the Proceedings of the Digital (DH),, 2025
Publisher: DH

Improving the Estimation of Object mass from images.

Author(s): João Andrade, and Plinio Moreno.
Published in: 2023 IEEE International Conference on Autonomous Robot Systems and Competitions (ICARSC), pp. 199-206., 2023
Publisher: IEEE

Relaxation labeling meets GANs: Solving jigsaw puzzles with missing borders

Author(s): M. Khoroshiltseva, A. Traviglia, M. Pelillo and S. Vascon
Published in: Proceedings of the 21st International Conference on Image Analysis and Processing, 2022
Publisher: Springer
DOI: 10.1007/978-3-031-06433-3_3

Bimanual grasping generation and evaluation for multifingered robotic hands.

Author(s): António Dinis and Plinio Moreno.
Published in: 2025 9th International Conference on Control, Automation and Diagnosis (ICCAD’25)., 2025
Publisher: IEEE

Multi-phase relaxation labeling for square jigsaw puzzle solving

Author(s): B. Vardi, M. Khoroshiltseva, A. Torcinovich, M. Pelillo, and O. Ben-Shahar
Published in: Proceedings of the 18th International Joint Conference on Computer Vision, Imaging, and Computer Graphics: Theory and Applications, 2023
Publisher: SCITEPRESS
DOI: 10.5220/0011622800003417

Model-Based Lookahead Reinforcement Learning for in-hand manipulation.

Author(s): Alexandre Lopes, Catarina Barata, and Plinio Moreno.
Published in: Accepted to publication on Eight Iberian Robotics Conference, ROBOT2025, 2025
Publisher: IEEE

DawnIK: Decentralized Collision-Aware Inverse Kinematics Solver for Heterogeneous Multi-Arm Systems.

Author(s): S. Marangoz, R. Menon, N. Dengler, M. Bennewitz.
Published in: IEEE-RAS International Conference on Humanoid Robots (Humanoids)., 2023
Publisher: IEEE-RAS

Robotic hand synergies for in-hand regrasping driven by object information.

Author(s): Dimitrios Dimou, José Santos-Victor, and Plinio Moreno.
Published in: Autonomous Robots 47, no. 4 (2023): 453-464, 2023
Publisher: IEEE

Measuring Uncertainty in Shape Completion to Improve Grasp Quality.

Author(s): "Nuno Ferreira Duarte, Seyed S. Mohammadi, Plinio Moreno, Alessio Del Bue, and Jose Santos-Victor. """
Published in: 2025 IEEE International Conference on Intelligent Robots and Systems (IROS)., 2025
Publisher: IEEE

Grasp pose sampling for precision grasp types with multi-fingered robotic hands.

Author(s): Dimitrios Dimou, José Santos-Victor, and Plinio Moreno.
Published in: 2022 IEEE-RAS 21st International Conference on Humanoid Robots (Humanoids), pp. 773-779., 2022
Publisher: IEEE

GANzzle: Reframing jigsaw puzzle solving as a retrieval task using a generative mental image

Author(s): D. Talon, A. Del Bue, S. James
Published in: Proceedings of IEEE International Conference on Image Processing, 2022
Publisher: IEEE
DOI: 10.1109/icip46576.2022.9897553

ReassembleNet: Learnable Keypoints and Diffusion for 2D Fresco Reconstruction.

Author(s): Islam, A., Fiorini, S., James, S., Morerio, P., Del Bue, A.D.
Published in: KeYypoints-and-Diffusion-2D-Islam- Fiorini/d56e6b457bbeaf9af66b585bd5cc5ee449b73ccc, 2025
Publisher: ArXiv

Compact Multi-Object Placement Using Adjacency-Aware Reinforcement Learning

Author(s): Benedikt Kreis, Nils Dengler, Jorge de Heuvel, Rohit Menon, Hamsa Perur, Maren Bennewitz
Published in: 2024 IEEE-RAS 23rd International Conference on Humanoid Robots (Humanoids), 2025, Page(s) 698-705
Publisher: IEEE
DOI: 10.1109/humanoids58906.2024.10769899

Task-oriented grasping for dexterous robots using postural synergies and reinforcement learning.

Author(s): Dimitris Dimou, José Santos Victor, and Plinio Moreno.
Published in: 2024 Eighth IEEE International Conference on Robotic Computing (IRC), pp. 65-71., 2024
Publisher: IEEE

Compact Multi-Object Placement Using Adjacency-Aware Reinforcement Learning.

Author(s): N. Dengler, J. Del Aguila Ferrandis, J. Moura, S. Vijayakumar, M. Bennewitz
Published in: arXiv preprint 2024, 2024
Publisher: arXiv preprint 2024

Trajectories in Hand Synergy Spaces for in-hand manipulation.

Author(s): Jorge Telo, Dimitrios Dimou, and Plinio Moreno.
Published in: 2024 IEEE International Conference on Autonomous Robot Systems and Competitions (ICARSC), pp. 213-218., 2024
Publisher: IEEE

Next Best View For Point-Cloud Model Acquisition: Bayesian Approximation and Uncertainty Analysis.

Author(s): Madalena Caldeira, and Plinio Moreno.
Published in: 2024 Eighth IEEE International Conference on Robotic Computing (IRC) pp. 91-96, 2024
Publisher: IEEE

Image-based grasping for soft-hands.

Author(s): Ana Rita Brás and Plinio Moreno.
Published in: 2025 9th International Conference on Control, Automation and Diagnosis (ICCAD’25)., 2025
Publisher: IEEE

Recognizing Artistic Style of Archaeological Image Fragments Using Deep Style Extrapolation

Author(s): Gur Elkin, Ofir Itzhak Shahar, Yaniv Ohayon, Nadav Alali, Ohad Ben-Shahar
Published in: Lecture Notes in Computer Science, Culture and Computing, 2025, Page(s) 115-131
Publisher: Springer Nature Switzerland
DOI: 10.1007/978-3-031-93160-4_8

Peer reviewed articles (1)

Automatizing 3D reconstruction pipelines for speeding-up cultural heritage digitization

Author(s): Gianluca Bison, Luca Palmieri, Sinem Aslan, Sebastiano Vascon, Marcello Pelillo
Published in: Multimedia Tools and Applications, Issue 84, 2025, Page(s) 39577-39597, ISSN 1573-7721
Publisher: Springer Science and Business Media LLC
DOI: 10.1007/s11042-025-20918-8