Periodic Reporting for period 2 - GRAVITATE (Geometric Reconstruction And noVel semantIc reunificaTion of culturAl heriTage objEcts)
Okres sprawozdawczy: 2016-06-01 do 2018-11-30
The vast majority of archaeological artefacts are discovered in a fragmentary state, and the poor state of preservation of these pieces further hampers the extent of the archaeological research possible. Moreover, related pieces of historical importance and interest may be dispersed across different collections making the study of their relationships or, in special cases, their reassembly difficult and even impossible.
Much effort and time is expended in trying to re-associate across collections pieces which share a common history, reunify pieces that were once part of the same artefact or reassemble these eroded and damaged fragments as accurately and completely as possible, whether for public display or historical research.
Recent technology developments in the area of 3D scanning and digital shape analysis and matching have shown that it is possible to scan recently fractured surfaces and reassemble them using visualisation tools and geometric matching. However, most of the applications which are encountered in the reconstruction of cultural heritage objects call for non-exact matching techniques, since surfaces are generally heavily abraded or damaged, and exact or near-exact matches are not possible. GRAVITATE aims to address this issue through a combination of geometrical and semantic approaches to discover similarity and likelihood of match. In addition to the development of improved matching algorithms more suited to this specific task, the project partners are working together to create a pipeline specifically targeted at the similarity assessments of 3D artefacts found in the real world, concurrently evaluating heterogeneous properties such as geometric aspects (e.g. curvature, size, roundness or mass distribution), photometric aspects (e.g. texture, colour distribution or surface patterning) and semantic annotation which includes contextual knowledge added by curators.
With the improved technical capability being developed by GRAVITATE, we will be able to go far beyond the manual or semi-manual matching techniques currently used. Moreover, by adding explicit semantic components to our understanding of the parts of artefacts we will enable researchers to discover matches and similarities that may not have been obvious in the past.
Much effort and time is expended in trying to re-associate across collections pieces which share a common history, reunify pieces that were once part of the same artefact or reassemble these eroded and damaged fragments as accurately and completely as possible, whether for public display or historical research.
Recent technology developments in the area of 3D scanning and digital shape analysis and matching have shown that it is possible to scan recently fractured surfaces and reassemble them using visualisation tools and geometric matching. However, most of the applications which are encountered in the reconstruction of cultural heritage objects call for non-exact matching techniques, since surfaces are generally heavily abraded or damaged, and exact or near-exact matches are not possible. GRAVITATE aims to address this issue through a combination of geometrical and semantic approaches to discover similarity and likelihood of match. In addition to the development of improved matching algorithms more suited to this specific task, the project partners are working together to create a pipeline specifically targeted at the similarity assessments of 3D artefacts found in the real world, concurrently evaluating heterogeneous properties such as geometric aspects (e.g. curvature, size, roundness or mass distribution), photometric aspects (e.g. texture, colour distribution or surface patterning) and semantic annotation which includes contextual knowledge added by curators.
With the improved technical capability being developed by GRAVITATE, we will be able to go far beyond the manual or semi-manual matching techniques currently used. Moreover, by adding explicit semantic components to our understanding of the parts of artefacts we will enable researchers to discover matches and similarities that may not have been obvious in the past.
A requirements capture exercise has been ongoing throughout the period. Four distinct classes of users for the planned re-association, reunification and reassembly features have been identified: researchers, whose generic aim is to solve some scientific questions; conservators, who aim to restore broken artefacts; curators, who look after and display archaeological collections; and illustrators, who work primarily with curators and visually interpret pieces for the public and for researchers. Through a questionnaire and through face to face interviews with experts, the requirements of these (often overlapping) users have been elicited. The discussions with users have been facilitated by the early development of some interactive user interface mock-ups which have provided a focus both for discussions with users and for planning the research and development activities of consortium members.
An extensive state of the art review has been conducted (GRAVITATE D3.1) encompassing both the geometric and semantic aspects of the work in the context of cultural heritage. The report reviews prior work on shape descriptors and the combination of multiple descriptors as a way to measure similarity; shape mating and matching approaches including the mathematics of contact and combinatorial methods in 2D jigsaw puzzles; semantic matching through graph matching of data enriched through natural language processing; and a review of vocabularies used in the cultural heritage domain. The report concludes with a discussion of work which brings together geometric and semantic data.
The project needs data to work on and a survey of the data related to the primary use case of fragmented terracotta funerary statues from Salamis in Cyprus has been conducted (GRAVITATE D6.1). 3D models of the Salamis fragments have been made available to the consortium and the relevant museum catalogue data from the Ashmolean, Fitzwilliam and Cyprus museums has been mapped to the CIDOC-CRM schema used by the British Museum and also made available to the consortium. A data management plan has been prepared, documenting how data will be preserved and shared.
These three strands of work form a baseline for decisions regarding the user interface, the architecture and the research work to be done. A second iteration of the user interface mock-up has been produced along with usage scenarios and some prototype implementations. Work has proceeded on defining the GRAVITATE platform architecture and preliminary research has been undertaken along with a roadmap defining the likely research directions and associated milestones.
Early on in the period, a logo and website were created as well as a project leaflet to be used at conferences. Every partner has undertaken dissemination activities and plans made for the future including a Eurographics workshop on Graphics and Cultural Heritage to be held in October 2016.
An extensive state of the art review has been conducted (GRAVITATE D3.1) encompassing both the geometric and semantic aspects of the work in the context of cultural heritage. The report reviews prior work on shape descriptors and the combination of multiple descriptors as a way to measure similarity; shape mating and matching approaches including the mathematics of contact and combinatorial methods in 2D jigsaw puzzles; semantic matching through graph matching of data enriched through natural language processing; and a review of vocabularies used in the cultural heritage domain. The report concludes with a discussion of work which brings together geometric and semantic data.
The project needs data to work on and a survey of the data related to the primary use case of fragmented terracotta funerary statues from Salamis in Cyprus has been conducted (GRAVITATE D6.1). 3D models of the Salamis fragments have been made available to the consortium and the relevant museum catalogue data from the Ashmolean, Fitzwilliam and Cyprus museums has been mapped to the CIDOC-CRM schema used by the British Museum and also made available to the consortium. A data management plan has been prepared, documenting how data will be preserved and shared.
These three strands of work form a baseline for decisions regarding the user interface, the architecture and the research work to be done. A second iteration of the user interface mock-up has been produced along with usage scenarios and some prototype implementations. Work has proceeded on defining the GRAVITATE platform architecture and preliminary research has been undertaken along with a roadmap defining the likely research directions and associated milestones.
Early on in the period, a logo and website were created as well as a project leaflet to be used at conferences. Every partner has undertaken dissemination activities and plans made for the future including a Eurographics workshop on Graphics and Cultural Heritage to be held in October 2016.
Both the GRAVITATE platform and associated research activities are at an early stage and so no progress beyond the state of the art can yet be claimed. Advances are expected in geometric characterisation, geometric matching, the representation of data related to fragmentary archaeological artefacts and in the fusion of semantic and geometric descriptors and similarity measures.
“Digitisation breathes new life into material from the past, and turns it into a formidable asset for the individual user and an important building block of the digital economy”. This quote from the well-known report of the Comité des Sages synthesizes perfectly the expected impacts of GRAVITATE. The project’s tools take existing digital assets and extends them: enriching the semantic metadata and extracting new features and measures from the 3D models. Further tools bring these datasets together to find hitherto unknown relationships between archaeological fragments and potential new insight into past cultures. The digital assets are both enhanced and brought into new relevance for both professionals and the public.
Even at this early stage, exciting exploitation possibilities have opened up in two seemingly unrelated domains: planetary science and forensic anthropology. Discussions with experts in these fields reveals a strong correlation between the GRAVITATE tools and the needs for meteorite reconstruction and the reassembly of human remains following mass disasters. These unexpected opportunities will be pursued alongside other impact areas.
“Digitisation breathes new life into material from the past, and turns it into a formidable asset for the individual user and an important building block of the digital economy”. This quote from the well-known report of the Comité des Sages synthesizes perfectly the expected impacts of GRAVITATE. The project’s tools take existing digital assets and extends them: enriching the semantic metadata and extracting new features and measures from the 3D models. Further tools bring these datasets together to find hitherto unknown relationships between archaeological fragments and potential new insight into past cultures. The digital assets are both enhanced and brought into new relevance for both professionals and the public.
Even at this early stage, exciting exploitation possibilities have opened up in two seemingly unrelated domains: planetary science and forensic anthropology. Discussions with experts in these fields reveals a strong correlation between the GRAVITATE tools and the needs for meteorite reconstruction and the reassembly of human remains following mass disasters. These unexpected opportunities will be pursued alongside other impact areas.