Periodic Reporting for period 1 - MEMORISE (Virtualisation and Multimodal Exploration of Heritage on Nazi Persecution)
Reporting period: 2022-10-01 to 2023-09-30
- An infrastructure that assists 50 consortial and associated memorial sites and archives in processing 80,000+ HNP content items to make them persistently accessible to the general public.
-A suite of digital tools for presenting, narrating and engaging with HNP in new ways.
-Semi-automated, AI-based services in the form of 1, a data processing pipeline to process Dutch, English and German HNP materials, and 2, an AI engine to recommend individualised content to general public users.
-An intersectional toolkit – including tools and best-practice documentation for data processing, knowledge graph creation, curation, 3D modelling, narration and interactive visual exploration – to accelerate the use of digital tools in cultural heritage and adjacent cultural industries.
-An educational concept that evaluates the benefits and limitations of user interfaces for learning on and engaging with HNP in on-site and online settings, with a particular focus on the necessity of virtual solutions in times of a pandemic.
- A comprehensive evaluation methodology that stretches across the entire project duration to assess the real needs in handling HNP to reach a global audience.
-The digitised collections (metadata and historical documents) of the partners within MEMORISE, and in the future institutions outside of the project, will be not only connected with each other, but also made available for virtualization, education, research, museum use and communication through a rich annotation and knowledge extraction processes aided by AI powered NLP methods and curated by human experts that will substantially improve efficiency and effectiveness of the process. The approach will be easily scalable to related cultural heritage domains.
-MEMORISE will advance the implementation and handling of 3D models by deploying and extending best practices in a novel pipeline model, focusing on sustainable interoperability within the context of the semantic web and sensitive heritage. The specific novelty will be the context-specific integration of knowledge extracted from source materials into spatiotemporal representations of HNP memorial sites. For that purpose, geospatial and temporal metadata will be annotated and connected to a geospatial ontology, which will be tailored for individual memorial sites and thus provide a model to adopt by other historical sites.
-MEMORISE will propose a novel synthetic agent (AI engine) grounded on neuroscientific principles to under- stand the users’ cognitive state and assist in the exploration of historical datasets. This agent will be based on a cognitive architecture theory that implements its perception, cognition and action capabilities. The synthetic agent will be able to dynamically build an HNP user model based on multimodal signal interpretation to assess their cognitive states. It will learn from the interactions with the current user and use information from previous experiences, forming a memory unit that the agent will use to refine the content proposal and create an individual experience.
-MEMORISE will deploy a HNP platform to assist researchers for the first time getting an overview of existing materials. Geo-temporal visualizations will arrange those materials in an appropriate historical context, and a novel map-based network will make the Nazi transportation infrastructure visible. We will develop a comparative reader to allow the parallel reading of Nazi persecution experiences, and we will further explore new ways of storytelling enabled for the first time by AI-based services tailored to the users’ own off-site and on-site experiences. The presentation will include transnational data to highlight global relations across HNP resources.
-MEMORISE for the first time offers a holistic integration of digital heritage culture, considering ethical, educational and social media guidelines and a comprehensive multiphase implementation testing. It will be the fir in-depth study on using digital technologies for engaging with HNP and difficult pasts. It will involve many users to evaluate the educational capabilities of our solutions.