Periodic Reporting for period 1 - PERCEIVE (Perceptive Enhanced Realities of Colored collEctions through AI and Virtual Experiences)
Okres sprawozdawczy: 2023-02-01 do 2024-04-30
PERCEIVE aims at improving the perception, preservation, curation, exhibition, understanding and access of colored Cultural Heritage collections. It aims at developing a reliable way to Study, Reconstruct, Render, Remap, Exhibit and Re-appropriate them. These collections are, in fact, a priority because of their high fragility that requires shared methods to preserve and exhibit them (i.e. textiles start fading in only 10 years, while only small traces of the original polychromy on classical statues is left today), because of the complexity of their study (especially with the attempts of reconstructing their original appearance), and because of the importance of their communication to future generations and of the shaping of European common identity around the concepts of “care” and diversity/variety” (i.e. the different shades of skin colors that were common in the Greek and Roman times).
PERCEIVE aims at advancing the digital capability of scientists and cultural institutions, through a service-based AI architecture and toolkit; and by developing a new design theory for onsite and remote VR/AR/MR experiences, based on “Care” “Accessibility” and “Authenticity” concepts, with and for the creative industries. Our expected outcomes include services and tools.
PERCEIVE has set five different objectives:
1. Make heritage science results accessible for immediate use and further development to Creative Industry sectors
2. Improve museums curatorial approaches to the study of colored collection through digital shared methodology and to the use of digital practices to exhibit them
3. Improve color reconstruction and prediction methodologies and workflow
4. Widen the access and explore the use of colored collections and digital artworks, involving visitors and citizens, improving their understanding and increasing their engagement with experiences that could be perceived as “authentic”.
5. EU Citizens re-appropriation of Colored Collections, increasing awareness of their fragility and importance, through the development of “sense of care”, based on the sense of wonder, emotion and embodiment.
PERCEIVE aims at advancing the digital capability of scientists and cultural institutions, through a service-based AI architecture and toolkit; and by developing a new design theory for onsite and remote VR/AR/MR experiences, based on “Care” “Accessibility” and “Authenticity” concepts, with and for the creative industries. Our expected outcomes include services and tools.
PERCEIVE has set five different objectives:
1. Make heritage science results accessible for immediate use and further development to Creative Industry sectors
2. Improve museums curatorial approaches to the study of colored collection through digital shared methodology and to the use of digital practices to exhibit them
3. Improve color reconstruction and prediction methodologies and workflow
4. Widen the access and explore the use of colored collections and digital artworks, involving visitors and citizens, improving their understanding and increasing their engagement with experiences that could be perceived as “authentic”.
5. EU Citizens re-appropriation of Colored Collections, increasing awareness of their fragility and importance, through the development of “sense of care”, based on the sense of wonder, emotion and embodiment.
The activities performed and main achievements obtained between M1 and M15 include: the submission of 12 deliverables; definition of a scientific and theoretical base for the entire project; Development of the Concept for the Open Museum; set-up of five Working Groups; Set-up the common SharePoint; Development of a NextCloud Federated Cloud; Started an extensive Data acquisition campaign; Designed the PERCEIVE colour knowledge repository; Defined the Requirements for WP4 and WP5; Started to develop AI and IBR core, platform, tools and services; Carried out Co-Design activities focused on the development of the prototypes; Created a visual identity and logo; Developed the PERCEIVE website and Newslettet (www.perceive-horizon.eu); Co-organised a scientific conference (https://www.gch2023.eu/)(odnośnik otworzy się w nowym oknie); Prepared an open access book on the PERCEIVE theory and technologies (‘Chromatic Visions’); Initiated networking activities with the Digital Cultural Heritage Cluster network
Expected advancements beyond the State of the Art, in the different fields relevant in PERCEIVE, include:
[Domain: Color Imaging] Color is critical in any digitization, preservation or restoration project in CH; PERCEIVE aims to develop a comprehensive appearance model, key to producing better images that facilitate research and widen access to cultural assets; a comprehensive image quality model capable of dealing with a wide scope of CH assets imagery and individual characteristics and needs, allowing the design of emerging imaging systems and algorithms that address multiple perceptual attributes.
[Domain: Color Analysis and Color Reconstructions and Prediction] Color reconstruction of residual polychromies is based on the analysis of remaining traces of pigments; PERCEIVE aims to adopt ML / AI technologies beyond the State of the Art: to enable unlimited combinations of several simultaneous hypotheses, offering multiple possible solutions to experts, promoting expert dialogue to validate successive phases / versions of the reconstruction.
[Domain: Color reconstruction and color changes prediction in paintings and works on paper] Darkening, fading and yellowing often accompanied by flaking, crumbling and chalking of the paint, affect thousands of iconic paintings and works on paper, with dramatic threats for their understanding, preservation, and management. PERCEIVE aims to work beyond the State of the Art through extensive research aimed at understanding the overall color change pathway and factors triggering such a process, supported by ML / AI technologies, enabling the definition of evidence-based reconstructive hypotheses
[Domain: Color reconstruction of textiles] Collections of historical textiles suffer color degradation thus limiting the access of the public; Expectations beyond the State of the Art include the development of a methodology to speed up the process of analysis, simulation and prediction of color change, based on the acquisition of the original pieces, the creation of mockups and their acquisition in different conditions and the reconstruction of the original color with IBR and AI technologies.
[Domain: Color reconstruction and animation of photographs] Old photos could rapidly deteriorate colors, especially in cases such as the Authocromes. Expectations beyond the State of the Art include the identification and inclusion of new AI architecture that generalizes colour reconstruction of the photographs.
[Domain: AI and Machine Learning methods to color and lighting reconstruction] Artificial Intelligence and Machine Learning can give cultural institutions, scientists and creative industries the advantage of automating the variety of manual processes. The project will include the a) development of shared methods and tools chains through a service-based approach, b) adoption of AI and Image Based approaches also to complex 3d collections, c) iteration including external training datasets and development of shared data sources.
[Domain: Exhibition Design and Virtual Experiences]: Museums are increasingly innovating in order to be attractive to their large public. Expectations include the development of a concept-based design to the creation of digital applications for the citizens; the creation of prototypes designed around authenticity and sense of care; the development of new Hybrid solutions.
[Domain: Color Imaging] Color is critical in any digitization, preservation or restoration project in CH; PERCEIVE aims to develop a comprehensive appearance model, key to producing better images that facilitate research and widen access to cultural assets; a comprehensive image quality model capable of dealing with a wide scope of CH assets imagery and individual characteristics and needs, allowing the design of emerging imaging systems and algorithms that address multiple perceptual attributes.
[Domain: Color Analysis and Color Reconstructions and Prediction] Color reconstruction of residual polychromies is based on the analysis of remaining traces of pigments; PERCEIVE aims to adopt ML / AI technologies beyond the State of the Art: to enable unlimited combinations of several simultaneous hypotheses, offering multiple possible solutions to experts, promoting expert dialogue to validate successive phases / versions of the reconstruction.
[Domain: Color reconstruction and color changes prediction in paintings and works on paper] Darkening, fading and yellowing often accompanied by flaking, crumbling and chalking of the paint, affect thousands of iconic paintings and works on paper, with dramatic threats for their understanding, preservation, and management. PERCEIVE aims to work beyond the State of the Art through extensive research aimed at understanding the overall color change pathway and factors triggering such a process, supported by ML / AI technologies, enabling the definition of evidence-based reconstructive hypotheses
[Domain: Color reconstruction of textiles] Collections of historical textiles suffer color degradation thus limiting the access of the public; Expectations beyond the State of the Art include the development of a methodology to speed up the process of analysis, simulation and prediction of color change, based on the acquisition of the original pieces, the creation of mockups and their acquisition in different conditions and the reconstruction of the original color with IBR and AI technologies.
[Domain: Color reconstruction and animation of photographs] Old photos could rapidly deteriorate colors, especially in cases such as the Authocromes. Expectations beyond the State of the Art include the identification and inclusion of new AI architecture that generalizes colour reconstruction of the photographs.
[Domain: AI and Machine Learning methods to color and lighting reconstruction] Artificial Intelligence and Machine Learning can give cultural institutions, scientists and creative industries the advantage of automating the variety of manual processes. The project will include the a) development of shared methods and tools chains through a service-based approach, b) adoption of AI and Image Based approaches also to complex 3d collections, c) iteration including external training datasets and development of shared data sources.
[Domain: Exhibition Design and Virtual Experiences]: Museums are increasingly innovating in order to be attractive to their large public. Expectations include the development of a concept-based design to the creation of digital applications for the citizens; the creation of prototypes designed around authenticity and sense of care; the development of new Hybrid solutions.