Multimodal Scanning of Cultural Heritage Assets for their multilayered digitization and preventive conservation via spatiotemporal 4D Reconstruction and 3D Printing

Project Information

Scan4Reco

Grant agreement ID: 665091

DOI

10.3030/665091

Project closed

EC signature date 21 May 2015

Start date 1 October 2015

End date 30 September 2018

Funded under

• SOCIETAL CHALLENGES - Europe In A Changing World - Inclusive, Innovative And Reflective Societies

Total cost

€ 3 762 762,00

EU contribution

€ 3 417 762,00

Coordinated by

ETHNIKO KENTRO EREVNAS KAI TECHNOLOGIKIS ANAPTYXIS
Greece

Periodic Reporting for period 2 - Scan4Reco (Multimodal Scanning of Cultural Heritage Assets for their multilayered digitization and preventive conservation via spatiotemporal 4D Reconstruction and 3D Printing)

Reporting period: 2016-10-01 to 2018-09-30

Summary of the context and overall objectives of the project

The European cultural heritage includes a rich variety of artifacts. Restoration procedures vary across the whole spectrum of materials & the age of cultural items making the problem of their conservation difficult to address. In this context, we will see the objectives of Scan4Reco along with the main conclusions that were reached:
Objective 1: A modular system has been developed consisting of: i) a motorized mechanical arm that allows safe, accurate and repeatable sensor probe positioning; ii) several complementary sensors iii) a rack mounted central processing pc controlling all devices and hosting the Scan4Reco integrated platform.
Objective 2: To realize this objective a comprehensive XSD schema has been implemented to hierarchically organize all metadata regarding a CH asset. Also a 3D printing pipeline was developed for joint color and translucency printing supporting printing of multilayered replicas.
Objective 3: Aside the digitization of CH assets through multi-view 3D reconstruction, techniques have been developed that provide detailed insights regarding the surface structure and material composition, in the form of appearance profiles, and rendering maps useful for analysis and visualization.
Objective 4: Reference samples of reference materials have been subjected to accelerated ageing. In this regard a material database has been developed documenting the status of each material at different ageing steps. Using this database specific ageing models were produced through deep learning architectures.
Objective 5: Within Scan4Reco a simulation engine has been developed deploying two complementary methods. The first one is a particle-based method that correlates degradation with particle concentration. The second method links simulation to physical measurements by using material specific ageing models.
Objective 6: For the realization of the DSS, experts’ knowledge has been organized in conservation ontologies. Diagnosis algorithms have been also implemented for the identification of degradations. Finally, a text synthesis module has been developed that utilizes the conservation ontologies for the automatic generation of conservation strategies.
Objective 7: This objective has been fulfilled with the successful completion of the Scan4Reco pilots. A wide range of CH items were examined with end-user feedback being positive whereas all KPIs have been met.
Objective 8: This objective has been met through the development of the Scan4Reco VR Museum that hosts all CH assets studied within the project.

Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far

The work performed can be summarized into the following points:
1. Analysis of current practices & state of the art (SoA)
Each partner has performed a detailed state-of-the-art analysis of the scientific areas he is involved in. Related deliverables are D1.2 and D2.1.
2. Assessment & analysis of user requirements
The assessment of end-user requirements was carried out through a questionnaire addressed to conservators, curators, archaeologists & art historians. The relevant deliverable is D2.3.
3. Definition of use case scenarios to be used in the validation processes
Partners decided on the kind of objects to be used in the system’s validation processes defining specifications like size, characteristics of the surface and artistic technique. Related deliverables are D2.2 D3.1 and D3.2.
4. Definition of system’s specifications and architecture
The technical characteristics of all the involved hardware were considered to extract the specifications of the Scan4Reco system. Taking into account all the specifications a detailed architecture has been described. Related deliverables are D2.2 D2.4 and D2.5.
5. Artificial ageing of material samples and reference data acquisition
To document the ageing process of materials, reference samples were artificially aged and measured for several types of egg-tempera pigments as well as bronze and silver alloys. All collected measurements were organized in a database. Related deliverables are D3.1 D3.2 D3.3 and D3.4.
6. Multi-sensorial and multi-spectral scanning
Within Scan4Reco significant technical work was carried out for the actual development of sensor probes and acquisition software for the de-noising and pre-processing of sensory signals. This work was essential for the integration of all sensors under the Scan4Reco platform. Related deliverables are D4.1 D4.2 D4.3 D4.4 and D5.7.
7. Spatiotemporal Simulation & Reconstruction
To achieve realistic rendering results, techniques have been developed to enhance 3D model representations with dynamic physical properties. Utilizing these techniques, complementary spatiotemporal methods were investigated. Related deliverables are D5.2 D5.3 D5.4 D5.5 and D5.6.
8. Decision Support System & System Integration
An achievement of Scan4Reco was the development of a DSS that integrates all individual technologies and uses a motorized mechanical arm for automated sensor positioning. Related deliverables are D5.1 D6.1 D6.2 D6.3 D6.4 and D6.6.
9. Scan4Reco applications, Trials, Evaluation & VR Museum
In Scan4Reco trials several CH assets were successfully examined with the participation of external conservation scientists. All the digitized CH assets are hosted in the Scan4Reco VR Museum. Related deliverables are D7.1 D7.2 D7.3 D7.4 and D6.5.
10. Definition of the project’s ethical plan & compliance with institutional ethical requirements
Scan4Reco follows Cultural-Heritage-related EU legislation, policies, guidelines and elaborates all available ethical formulations. Related submitted deliverable is D1.3.
11. Horizontal activities for the dissemination & exploitation of the project’s activities
Dissemination activities include the Scan4Reco website, a mailing list of end-users, a social media strategy, a ZENODO portal, a Wi-Ki portal, 52 accepted publications & 4 project presentations, collaboration with 3 EU projects and links to industry. Related deliverables are D8.1 and D8.2. For the exploitation of the Scan4Reco outcomes a detailed plan is elaborated in D8.3.

Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far)

The Scan4Reco platform as has been evaluated on the project pilots offers a novel solution that can support multiple conservation tasks bringing together interdisciplinary technologies while guarantying safety, portability and modularity. A significant advancement is the application of motion planning on integrated motion devices to automate sensor positioning with safety and to provide registered measurements.
A very important outcome of Scan4Reco is its publicly available material database. It contains extensive measurements on reference materials that were iteratively artificially aged and measured. This database was extensively used throughout the project for developing diagnostic algorithms and defining ageing models. Another achievement has been the conservation-oriented Decision Support System that integrates all Scan4Reco technologies for acquisition, diagnosis and simulation. The DSS, depending on the identified degradations on the CH asset, provides comprehensive conservation approaches.
Scan4Reco has also made a breakthrough in 3D printing where a method has been developed for joint colour and translucency printing, supporting multiple RGBA textures and multiple 3D shapes. Another contribution of Scan4Reco in engaging the public is the development of a VR museum where all CH assets that were digitized within the project are exhibited.

The Scan4Reco Logo