Periodic Reporting for period 2 - SHIFT (MetamorphoSis of cultural Heritage Into augmented hypermedia assets For enhanced accessibiliTy and inclusion)
Período documentado: 2023-10-01 hasta 2025-09-30
The SHIFT project leverage on the advances of Artificial Intelligence (AI) and Machine Learning (ML) technologies to improve cultural heritage access to European citizens experiencing sensory impairment. The project aims enrich the experience of interacting with cultural assets through visual, auditory, and sensory modalities, such as using haptics. Additionally, the SHIFT project also will develop tools and methodologies to enable contemporary description of cultural assets through language evolution models.
The main Project Objectives:
o Enable cultural heritage institutions to successfully adopt digital strategies to promote public engagement
o Offer unique experiences to the visitors through a hybrid of visual-auditory-haptics sensors for experiencing cultural heritage
o Enable digital interactions with cultural assets using haptic technology
o Represent cultural heritage to contemporary times through the development of language models
o Adopt accessibility by design principles and tools to promote compliance with International standards
o Implement tools for creating ownership chain for protecting copyrights of digital media
The following technologies and platforms will be used:
• AI algorithms implemented for enabling visual accessibility of cultural heritage content for people experiencing partial visual impairment
• Language model development to map historical references to contemporary language usage.
• Design and implementation of haptic interfaces for enriching access to cultural heritage assets
• AI algorithms to translate text into audio format for achieving economies of scale in emotional audio transcription
The main Project Objectives:
o Enable cultural heritage institutions to successfully adopt digital strategies to promote public engagement
o Offer unique experiences to the visitors through a hybrid of visual-auditory-haptics sensors for experiencing cultural heritage
o Enable digital interactions with cultural assets using haptic technology
o Represent cultural heritage to contemporary times through the development of language models
o Adopt accessibility by design principles and tools to promote compliance with International standards
o Implement tools for creating ownership chain for protecting copyrights of digital media
The following technologies and platforms will be used:
• AI algorithms implemented for enabling visual accessibility of cultural heritage content for people experiencing partial visual impairment
• Language model development to map historical references to contemporary language usage.
• Design and implementation of haptic interfaces for enriching access to cultural heritage assets
• AI algorithms to translate text into audio format for achieving economies of scale in emotional audio transcription
The main activities performed in the second reporting period (M13 – M36) were the following:
O1. Enable cultural heritage institutions to successfully adopt digital strategies to promote public engagement.
As described in the first report, this objective was achieved through task 1.1 which identified the basic objectives and missions pursued by cultural heritage institutions in general and by the participating partners in particular, for example, by implementing inclusive measures in a given spatial and climatic scenario. Based on these requirements, the development and implementation of the tools planned within the SHIFT project was built.
O2. Offer unique experiences to the visitors through a hybrid of visual-auditory-haptics sensors for experiencing cultural heritage.
During the second reporting period, the project advanced the use of AI-generated content (AIC) as a core enabler for offering unique, multi-sensory cultural heritage experiences that combine visual, auditory, and haptic modalities.
O3. Enable digital interactions with cultural assets using haptic technology.
O3 is addressed within WP3, through the development of the SHIFT Haptics Tool. The primary aim of this objective was to enrich 3D digital twins of CH assets with realistic haptic feedback, enabling tangible and meaningful interaction within virtual environments. To that end, the project developed the SHIFT Haptics Tool and integrated it within the XR Accessibility Framework, providing task-relevant, multimodal interaction with digital artefacts.
O4. Represent cultural heritage to contemporary times through the development of language models.
As part of our ongoing mission to bring CH assets into contemporary relevance, we have actively pursued specific methods leading to tangible outcomes. Our strategy involved the comprehensive application of various Large Language Models (LLMs), including innovative technologies such as LLaMA, LLaMA2, GPT, BERT and fine-tuned variations such as Vicuna.
O5. Adopt accessibility by design principles and tools to promote compliance with international standards.
Objective O5 has been addressed during the reporting period, through the systematic adoption of accessibility-by-design principles throughout the development lifecycle of the SHIFT platform and its associated tools.
O6. Implement tools for creating ownership chain for protecting copyrights of digital media.
During the second reporting period, the project implemented a set of interoperable tools that create and maintain a verifiable ownership chain for digital media, thereby protecting copyrights for both original and AI-generated cultural assets. The integrated pipeline now covers the full lifecycle from asset ingestion to derivative creation and rights tracking.
O7. Contribution to international standards to exchange metadata models with cultural institutions and copyright protection of content ownership
The activities related to standardization were focused on the analysis of existing standards, the standards used within the project implementation and contribution to different standards organizations. The research and scientific results of SHIFT are compatible with many existing and emerging standards, and the standardization work is based on the relevant experience and know-how of the Consortium partners and their connections with the standardization bodies.
O8: Dissemination and communication strategies for wider-scale adoption of SHIFT results across cultural and creative industries
In M13-M36 of the project consortium intensified its communication and dissemination efforts. Specifically by:
Reviewing and updateing the Communication and Dissemination plan and producing two more target-audience specific plans:
• for the general public (focusing on non-users)
• targetting cultural heritage professionals
D6.1 was resubmitted and approved.
O1. Enable cultural heritage institutions to successfully adopt digital strategies to promote public engagement.
As described in the first report, this objective was achieved through task 1.1 which identified the basic objectives and missions pursued by cultural heritage institutions in general and by the participating partners in particular, for example, by implementing inclusive measures in a given spatial and climatic scenario. Based on these requirements, the development and implementation of the tools planned within the SHIFT project was built.
O2. Offer unique experiences to the visitors through a hybrid of visual-auditory-haptics sensors for experiencing cultural heritage.
During the second reporting period, the project advanced the use of AI-generated content (AIC) as a core enabler for offering unique, multi-sensory cultural heritage experiences that combine visual, auditory, and haptic modalities.
O3. Enable digital interactions with cultural assets using haptic technology.
O3 is addressed within WP3, through the development of the SHIFT Haptics Tool. The primary aim of this objective was to enrich 3D digital twins of CH assets with realistic haptic feedback, enabling tangible and meaningful interaction within virtual environments. To that end, the project developed the SHIFT Haptics Tool and integrated it within the XR Accessibility Framework, providing task-relevant, multimodal interaction with digital artefacts.
O4. Represent cultural heritage to contemporary times through the development of language models.
As part of our ongoing mission to bring CH assets into contemporary relevance, we have actively pursued specific methods leading to tangible outcomes. Our strategy involved the comprehensive application of various Large Language Models (LLMs), including innovative technologies such as LLaMA, LLaMA2, GPT, BERT and fine-tuned variations such as Vicuna.
O5. Adopt accessibility by design principles and tools to promote compliance with international standards.
Objective O5 has been addressed during the reporting period, through the systematic adoption of accessibility-by-design principles throughout the development lifecycle of the SHIFT platform and its associated tools.
O6. Implement tools for creating ownership chain for protecting copyrights of digital media.
During the second reporting period, the project implemented a set of interoperable tools that create and maintain a verifiable ownership chain for digital media, thereby protecting copyrights for both original and AI-generated cultural assets. The integrated pipeline now covers the full lifecycle from asset ingestion to derivative creation and rights tracking.
O7. Contribution to international standards to exchange metadata models with cultural institutions and copyright protection of content ownership
The activities related to standardization were focused on the analysis of existing standards, the standards used within the project implementation and contribution to different standards organizations. The research and scientific results of SHIFT are compatible with many existing and emerging standards, and the standardization work is based on the relevant experience and know-how of the Consortium partners and their connections with the standardization bodies.
O8: Dissemination and communication strategies for wider-scale adoption of SHIFT results across cultural and creative industries
In M13-M36 of the project consortium intensified its communication and dissemination efforts. Specifically by:
Reviewing and updateing the Communication and Dissemination plan and producing two more target-audience specific plans:
• for the general public (focusing on non-users)
• targetting cultural heritage professionals
D6.1 was resubmitted and approved.
The SHIFT project will bring benefits to society by supporting the mainstreaming of new digital tools for creative and commercial collaborations, as well as cutting-edge solutions.
The expected impacts are summarized below:
- Developing, Integrating and strengthening the use of digital technologies to protect, conserve, restore cultural heritage and the arts in complementarity with other research methods.
- Facilitating and expanding access to cultural assets through digital technologies and tools and in parallel or as an alternative to physical access to cultural heritage.
- Supporting AI benchmarking, including time analysis and other digital means to improve innovation and knowledge sharing in the cultural and creative sectors.
- Increasing the competitiveness of the cultural and creative industries on the domestic and international market and providing opportunities for the creation of new and sustainable jobs.
- Exploring the role of digital tools such as 3D/4D simulations, virtual and augmented reality technologies in engaging with cultural heritage during and after the COVID-19 crisis.
- Using digital tools and other outputs to provide effective and comprehensive solutions to the real needs of access, protection and preservation of cultural heritage, including the "born digital".
The expected impacts are summarized below:
- Developing, Integrating and strengthening the use of digital technologies to protect, conserve, restore cultural heritage and the arts in complementarity with other research methods.
- Facilitating and expanding access to cultural assets through digital technologies and tools and in parallel or as an alternative to physical access to cultural heritage.
- Supporting AI benchmarking, including time analysis and other digital means to improve innovation and knowledge sharing in the cultural and creative sectors.
- Increasing the competitiveness of the cultural and creative industries on the domestic and international market and providing opportunities for the creation of new and sustainable jobs.
- Exploring the role of digital tools such as 3D/4D simulations, virtual and augmented reality technologies in engaging with cultural heritage during and after the COVID-19 crisis.
- Using digital tools and other outputs to provide effective and comprehensive solutions to the real needs of access, protection and preservation of cultural heritage, including the "born digital".