Periodic Reporting for period 1 - Transparent3D (Intuitive, rapid and efficient in-situ measurement of complex transparent 3D objects with a focus on cultural heritage artifacts)
Berichtszeitraum: 2021-03-15 bis 2023-03-14
The issue being addressed is the need to be able to rapidly digitize the mass amounts of sensitive glass cultural heritage objects available in Europe and prevent them from being destroyed and eternally lost via material decay, acts of terrorism and natural disasters.
This is useful for society because cultural heritage objects are a fundamental part of the cultural identity of each country and if they are lost without the ability of being recreated, a part of a nation’s identity is lost with them. To date, glass and transparent objects are the most overlooked and tricky to digitize as no one has been able to demonstrate a system with the capability of doing so practically, in order to enable mass digitization of such objects and use of the technology by non-experts.
The objective of Transparent 3D is to create an easy-to-use, non-contact measurement hardware and software package which dramatically reduces the cost of measuring complex transparent objects by greatly accelerating and streamlining the measurement procedure. The technique used will be based on the principle of non-contact optical measurement. The hardware and software package will employ a combination of techniques in order to optimize and simplify the complete 3D reconstruction process of complex transparent cultural heritage objects.
The impact of the platform created and its competitive advantages over existing techniques will be demonstrated in the field with case studies conducted at three cultural heritage institutions in Greece, Italy and France.
The conclusions of the action was to successfully develop and prototype a method for 3D scanning tranparent objects using an single RGB camera. It was used to rapidly digitize hollow glass cultural heritage items, and was also shown to apply to contemporary transparent glass and plastic items. On the specific objects tested, we demonstrated an accuracy of 1mm - 2mm on average, which is considered to be adequate for cultural heritage digitization and preservation applications.
A patent is being applied for after which we will continue to develop the technique to improve the accuracy and also develop and commercialize the techique even further after the end of the action in collaboration with the host institution.
This is useful for society because cultural heritage objects are a fundamental part of the cultural identity of each country and if they are lost without the ability of being recreated, a part of a nation’s identity is lost with them. To date, glass and transparent objects are the most overlooked and tricky to digitize as no one has been able to demonstrate a system with the capability of doing so practically, in order to enable mass digitization of such objects and use of the technology by non-experts.
The objective of Transparent 3D is to create an easy-to-use, non-contact measurement hardware and software package which dramatically reduces the cost of measuring complex transparent objects by greatly accelerating and streamlining the measurement procedure. The technique used will be based on the principle of non-contact optical measurement. The hardware and software package will employ a combination of techniques in order to optimize and simplify the complete 3D reconstruction process of complex transparent cultural heritage objects.
The impact of the platform created and its competitive advantages over existing techniques will be demonstrated in the field with case studies conducted at three cultural heritage institutions in Greece, Italy and France.
The conclusions of the action was to successfully develop and prototype a method for 3D scanning tranparent objects using an single RGB camera. It was used to rapidly digitize hollow glass cultural heritage items, and was also shown to apply to contemporary transparent glass and plastic items. On the specific objects tested, we demonstrated an accuracy of 1mm - 2mm on average, which is considered to be adequate for cultural heritage digitization and preservation applications.
A patent is being applied for after which we will continue to develop the technique to improve the accuracy and also develop and commercialize the techique even further after the end of the action in collaboration with the host institution.
The project started on 15/3/2021.
Investigation of use of high-incidence angle structured light.
- Technique was abandoned due to complexity and very small amount of measurement area
Investigation of use of visual hull as method to extract external shape for glass objects:
Investigation of use of polarized cameras for reconstruction of glass objects.
- Created apparatus using polarized camera and rotation table under a protective canvas to prevent reflections.
Methods of registering the rotational views for the polarization setup include:
- Tried use of elevated platform with markers stuck on the side
- Tried use of printed markers on the table
- Ended up calibrating externals once via checkerboard and just rotating camera in world space as the rotation table stepper motor was accurate enough
Investigated types of cultural heritage objects, and their history:
- Collaboration with archaeologist in FORTH Institute of Mediterranean studies.
- Investigated the use of photogrammetry on textured glass objects and
Exploitation and Dissemination Result: Produced publication which was sent to EUROMED 2021 conference and presented online due to COVID restrictions.
Investigated the use of inverse rendering to reconstruct transparent glass objects:
- Collected cultural heritage objects from the 1970's
- Attempted use of back illuminated images to reconstruct object assuming they were "smoke clouds" in Mitsuba 3
- Was able to get accuracies on the external shape of the items with an accuracy of 0.2mm but it failed on all internal shape extraction.
Exploitation and Dissemination Result: Published paper at EUROMED 2022 international conference in Limassol, Cyprus.
Invented method of reconstructing both internal and external shape of contemporay transparent objects using single RGB camera for data collection.
- Performed patent search from within FORTH and identified that technique has commercialization potential.
Exploitation and Dissemination Result: Visited Control 2023 in Stuttgart Germany to verify the commercialization novelty of the technique.
Exploitation and Dissemination Result: In the process of applying for an international PCT patent with FORTH to protect this technology.
Exploitation and Dissemination Result: Two academic papers prepared on plastic and glass objects using this technique are being prepared.
Investigation of use of high-incidence angle structured light.
- Technique was abandoned due to complexity and very small amount of measurement area
Investigation of use of visual hull as method to extract external shape for glass objects:
Investigation of use of polarized cameras for reconstruction of glass objects.
- Created apparatus using polarized camera and rotation table under a protective canvas to prevent reflections.
Methods of registering the rotational views for the polarization setup include:
- Tried use of elevated platform with markers stuck on the side
- Tried use of printed markers on the table
- Ended up calibrating externals once via checkerboard and just rotating camera in world space as the rotation table stepper motor was accurate enough
Investigated types of cultural heritage objects, and their history:
- Collaboration with archaeologist in FORTH Institute of Mediterranean studies.
- Investigated the use of photogrammetry on textured glass objects and
Exploitation and Dissemination Result: Produced publication which was sent to EUROMED 2021 conference and presented online due to COVID restrictions.
Investigated the use of inverse rendering to reconstruct transparent glass objects:
- Collected cultural heritage objects from the 1970's
- Attempted use of back illuminated images to reconstruct object assuming they were "smoke clouds" in Mitsuba 3
- Was able to get accuracies on the external shape of the items with an accuracy of 0.2mm but it failed on all internal shape extraction.
Exploitation and Dissemination Result: Published paper at EUROMED 2022 international conference in Limassol, Cyprus.
Invented method of reconstructing both internal and external shape of contemporay transparent objects using single RGB camera for data collection.
- Performed patent search from within FORTH and identified that technique has commercialization potential.
Exploitation and Dissemination Result: Visited Control 2023 in Stuttgart Germany to verify the commercialization novelty of the technique.
Exploitation and Dissemination Result: In the process of applying for an international PCT patent with FORTH to protect this technology.
Exploitation and Dissemination Result: Two academic papers prepared on plastic and glass objects using this technique are being prepared.
The final progress beyond the state of the art, has been to demonstrate for the first time, the easy 3D reconstruction of transaprent objects without the use of any spray overcoating, with cheap and affordable RGB cameras for data collection that anyone can use. The process is very simple and data acquisition is under 2 minutes per item.
A patent is being submitted which is expected to lead to the commercialization of the technology and provide an easy-to-use 3D scanning system which can be leveraged by curators to mass-digitize sensitive glass objects of cultural heritage which are at risk of being lost due to decay, terrorist attack and natural disasters.
The creation of a practical, easy, cheap and fast digitization process for preserving Europe’s glass cultural heritage will eventually facilitate the achievement of a holistic online and offline digital repository for documentation, preservation and touristic purposes. The societal impact of facilitating this digital repository will be the deeper knowledge and understanding of European's for their own cultural heritage as well as that of neighboring (and potentially culturally and historically similar countries), therefore promoting friendship, tolerance, international collaboration and contributing towards the ultimate goal of promoting further integration and solidarity between European nations.
In addition to the primary objectives of the project we have found that our technology can also operate on transparent plastic objects. Therefore the creation of new markets for automated quality control of industrial transparent objects will be created, which will add value to European industrial products and improve the bottom line of Europe's GDP.
A patent is being submitted which is expected to lead to the commercialization of the technology and provide an easy-to-use 3D scanning system which can be leveraged by curators to mass-digitize sensitive glass objects of cultural heritage which are at risk of being lost due to decay, terrorist attack and natural disasters.
The creation of a practical, easy, cheap and fast digitization process for preserving Europe’s glass cultural heritage will eventually facilitate the achievement of a holistic online and offline digital repository for documentation, preservation and touristic purposes. The societal impact of facilitating this digital repository will be the deeper knowledge and understanding of European's for their own cultural heritage as well as that of neighboring (and potentially culturally and historically similar countries), therefore promoting friendship, tolerance, international collaboration and contributing towards the ultimate goal of promoting further integration and solidarity between European nations.
In addition to the primary objectives of the project we have found that our technology can also operate on transparent plastic objects. Therefore the creation of new markets for automated quality control of industrial transparent objects will be created, which will add value to European industrial products and improve the bottom line of Europe's GDP.