Community Research and Development Information Service - CORDIS

  • European Commission
  • Projects and Results
  • Periodic Reporting for period 1 - ImmersiaTV (Immersive Experiences around TV, an integrated toolset for the production and distribution of immersive and interactive content across devices.)

ImmersiaTV Report Summary

Project ID: 688619
Funded under: H2020-EU.2.1.1.

Periodic Reporting for period 1 - ImmersiaTV (Immersive Experiences around TV, an integrated toolset for the production and distribution of immersive and interactive content across devices.)

Reporting period: 2016-01-01 to 2017-03-31

Summary of the context and overall objectives of the project

The majority of TV consumers now watch TV programs in a multi-display environment. Second screens –most often smartphones - are generally used to check information not directly related to the events in the TV content being watched. The arrival of virtual reality displays to the living room is an opportunity to deliver innovative engaging experiences, despite it increases the need for consistent experiences across displays. Thus, ImmersiaTV is focusing its work to meet the following objectives:
OBJ1. Create a new cinematographic language combining immersive paradigms with traditional storytelling techniques.
OBJ2. Extend the production pipeline to create omnidirectional content for a multi-platform environment.
OBJ3. Re-design the distribution chain to address the specific technical challenges that omnidirectional content imposes in terms of capture, compression, distribution, reception, and rendering.
OBJ4. Maximize the quality of the end-user experience, across devices, and within the technical limitations of existing production structures, distribution facilities and reception devices to create an optimal immersive experience.
OBJ5. Maximize the impact of the ImmersiaTV solutions within the ecosystem of content creators, broadcasters, and consumers.

Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far

The work performed in ImmersiaTV has been divided in 5 Work packages:
Work Package 1 mostly focuses on management and administrative tasks to ensure a proper progress of the project.
Work Package 2, Requirements, format and creation of Immersive experiences, is centered on the user (professional and residential) requirements and content creation. During the first period, WP2 gathered requirements through different user centered design activities and settled the common knowledge ground for the consortium. In addition to that, content conceptualization and initial production tests have been also included in the activities of this Work Package with three iterations, 2 of them almost fully executed in the period 1 time frame.
Work Package 3, Immersive Broadcast Platform, is centered in the design and implementation of the ImmersiaTV platform. In a first iteration the objective has been to create a multi-stream multi-display on-demand streaming service involving several synchronized video streams, both omnidirectional and not, that could be edited by professional users and visualized by end-users in groups. In Capture (T3.2 - Capture), Videostitch Studio solution was improved in terms of stitching speed, stabilization and calibration. In production tools (T3.3 - Production Tools), we designed and implemented a plugin for Adobe Premiere Pro that allow the user to edit and render multi-screen synchronized content with omnidirectional and directive footage. In content processing (T3.4 - Encoding and Decoding), studies were made towards the optimization of the encoding process for period 2 pilots. In distribution and reception (T3.5 - Delivery and Reception), a content publication and transcoding server adapted to the project specificities was developed and iteratively improved. In visualization and interaction (T3.6 - Interaction and Display), different interactive players/clients were developed and iteratively improved. In addition to this implementation tasks, quality metrics were designed (T3.7 Quality of Experience) and all components have been integrated in different public and controlled demos as part of the piloting activities. By the end of the first period the second iteration has started, focusing in the adaptation of the existing platform and tools to a live production environment.
Work Package 4, Demonstration Pilots, is centered in demonstrating the results of the project through pilots, evaluating the results with the involved groups of users and providing new sets of requirements. To achieve this, a living lab research approach is applied to ensure a user-centered design approach and iterative development and evaluation of the different pilot phases. During the whole period, an important number of demonstrations have been made (in cooperation with WP5) and some of them have incorporated formal evaluation methods.
Work Package 5, Dissemination, exploitation and communication, is a transversal work package centered on providing visibility to the work performed in the project, analysing the market and developing the exploitation plans of the project results. During the first period, besides the maintenance of typical communication tools (web, twitter, FB), ImmersiaTV has achieved a good impact among the ICT-Media community (presence at IBC 2016, best demo prize at NEM Summit 2016) and exploitation plans have been thoroughly discussed.

Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far)

The work carried out has achieved significant progress beyond the state of the art, across the entire capture, editing, delivery and playout of multi-device video-based synchronized content.

Regarding capture, VideoStitch improved their stitching solution to maximize its ease of use (e.g. camera presets). Lightbox and VRT have explored novel ways of shooting based on the combination of traditional and omnidirectional cameras, and now starting to consider the constraints of live production for pilot 2.
Producing content for multi-device synchronous consumption has also required the joint development (i2CAT, PSNC) of custom innovative production tools, integrated with industrial solutions (Premiere Pro), and a custom metadata format to allow specifying interaction events that occur synchronized across devices. The ImmersiaTV plugin allows defining the inserts that are placed within an omnidirectional video and which tracks are rendered in each device: TV, tablet or HMD. Next steps will deploy a solution for Live Multi-platform editing based on Cinegy’s solutions.
Regarding delivery, offline content delivery required the implementation and deployment of a converter to DASH, a web app for content publication and their integration with a delivery server. EPFL leads our efforts to introduce alternatives to the equirectangular projections for video encoding, such as cubemap projections, which have been thoroughly analyzed in terms of impact in compression efficiency, visual quality, complexity, and integration with conventional workflows.
We also developed together the end-user solution necessary to play multi-platform content. Libraries for synchronized playout, both inter-stream and inter-device, have been developed, with alternative solutions based either on PTP or on DVBCSS standards. Such libraries have been integrated both within a native solution for windows and android, and as a web-based app (tested on Chrome and Firefox). Current work is focused towards improving the resolution of the videos being played, and will be followed by the implementation of inter-device interaction mechanisms. All this work is assessed by IMEC with end-users and professionals whether our solutions match their initial requirements.

Overall we have and will continue to demonstrate significant impact across the entire content production chain, from capture to delivery, and we will continue our efforts in finding ways to make the ImmersiaTV solutions available to a wider audience with public presentations of our solutions in large-impact forums (IBC, NAB, imec ITF), and exploration of opportunities for industrial adoption of the solutions implemented.

Related information

Follow us on: RSS Facebook Twitter YouTube Managed by the EU Publications Office Top