Over the first reporting period, the team has embraced the TRANSMIXR human centric approach. Through multidisciplinary efforts, significant progress has been made with respect to user and technical requirements definition. The project has interacted with 360 stakeholders via requirements and design workshops, surveys, interviews as well as soliciting feedback on proof of concepts demonstrations both internal and external to the project. Furthermore, novel production workflows have been identified, resulting in guidelines for volumetric video capturing, the use of AI techniques for creating 3D content and the use of social XR pipelines to support enhanced communication and collaboration when creating experiences.
The TRANSMIXR multimodal media content ingestion pipeline is completed. The multimodal analysis components that support 360° and volumetric video formats as well as multimodal summarisation components are well progressed. These tools can help with a deeper understanding of multimodal media content that can be used to facilitate the creation of complex narratives. With respect to the development of creation toolsets for the new production workflows, again significant progress has been made in the first reporting period. The toolset includes: three volumetric and one motion capture systems with unique characteristics; AI-based tools for the creation of 3D static and dynamic content; and a template-based XR experience creation system that supports non-technical CCS professionals to adapt XR experiences for end-users. An open source volumetric video pipeline, has been developed for facilitating the communication between customers (e.g. cultural heritage institution) and producers of XR experiences. Finally, extensive work has been dedicated to the monitoring and extension of formats, standards, and metrics based on the needs of the TRANSMIXR use cases. In terms of end user experiences, to ensure consistency across TRANSMIXR experiences, a suite of assets and design guidelines have been created. Moreover, a number of user experience implementations for the MVPs of WP5, including two versions of a novel distributed control room for broadcasters, a template for simplifying the production process of curators of cultural heritage institutions, and the initial concept of a new performance based on a non-player character supporting interaction and conversation (using an LLM). Finally a range of user studies (including the creation of datasets) with TRANSMIXR technologies and related content formats (Social XR, point cloud, AR, VR) was undertaken. To date and per use case, MVPs that include co-creation workflows & immersive media experiences have been realised through dedicated pilot and evaluation teams.
