Periodic Reporting for period 2 - HDR4EU (Enabling End-to-End HDR Ecosystem)
Okres sprawozdawczy: 2018-07-01 do 2020-06-30
The main goal of HDR4EU is to position EU companies in the creative sector as world leaders in the emerging HDR format by producing, piloting and demonstrating a set of professional tools, techniques and guidelines allowing for an HDR ecosystem to emerge, yielding a notably superior viewer experience in terms of image quality and content personalization. HDR4EU addresses all open challenges by having the following specific and measurable objectives that directly map onto work packages and tasks:
• Producing guidelines on how to shoot and post-produce HDR footage in a way that maximizes the expressive capabilities of the HDR medium while considering the limitations of visual perception with respect to HDR images, based on exemplary content created and demonstrated in the project;
• Piloting and demonstrating high quality real-time tools for conversion among HDR and SDR formats and among standard and wide colour gamuts, which are essential for HDR capture, monitoring, post- production, distribution and display, for cinema, TV, videogames and AR/VR;
• Producing and demonstrating colour management and grading tools so that the mastering process has a simple workflow that can deliver consistent images across the whole spectrum of possible display devices and environments;
• Creating a first market implementation of live (immersive) HDR content generation tools for broadcasting, and a web implementation for broadband internet;
• Demonstrating and validating a new product: an HDR projection system for movie theatres, since traditional cinema projector technology is not capable of providing images with a dynamic range that is actually high;
• Producing and demonstrating automated and user-assisted personalization tools, allowing content to be adapted to the viewer’s personal characteristics and individual preferences, as well as to the viewing environment.
• Producing guidelines on how to shoot and post-produce HDR footage in a way that maximizes the expressive capabilities of the HDR medium while considering the limitations of visual perception with respect to HDR images, based on exemplary content created and demonstrated in the project;
• Piloting and demonstrating high quality real-time tools for conversion among HDR and SDR formats and among standard and wide colour gamuts, which are essential for HDR capture, monitoring, post- production, distribution and display, for cinema, TV, videogames and AR/VR;
• Producing and demonstrating colour management and grading tools so that the mastering process has a simple workflow that can deliver consistent images across the whole spectrum of possible display devices and environments;
• Creating a first market implementation of live (immersive) HDR content generation tools for broadcasting, and a web implementation for broadband internet;
• Demonstrating and validating a new product: an HDR projection system for movie theatres, since traditional cinema projector technology is not capable of providing images with a dynamic range that is actually high;
• Producing and demonstrating automated and user-assisted personalization tools, allowing content to be adapted to the viewer’s personal characteristics and individual preferences, as well as to the viewing environment.
WP1 Management: HDR4EU has completed its first 12 months being able to successfully achieve the objectives that had been set for the first year of the project.
WP2 Capture: Arri conducted a survey among partners concerning the technical requirements of test content and has shot or sourced 26 of the 32 requested items. Together Arri and Brainstorm also conducted an enhanced HDR shoot for the purpose of evaluating the HDR virtual studio tools developed for this project. Smoke & Mirrors are in the process of completing the necessary facility upgrades required to carry out a state of the art HDR post production work flow.
WP3 Generation for multiplatform delivery: In response to the problems associated with colour grading HDR content using traditional digital methods, Filmlight has developed new tools for colour grading and management based on a novel colour appearance model (TCAM). The model is also being used as a foundation for the cross conversion and environmental customization tools which are currently under development. Working prototypes for all of these components have been developed so that further testing can be carried out. Live demonstrations and seminars are being conducted to encourage adoption of these new tools. UPF is contributing to the expansion of the HDR tool set by developing tone mapping and inverse tone mapping algorithms. In the past year, algorithm prototypes have been developed and psychophysical tests have been conducted pertaining to the optimization of their parameters.
WP4 Real Time Generation: Brainstorm's eStudio tool has been updated with HDR capture and play out functionality. This updated version of the software is being shared with select customers for the purpose of validating the colour conversion methods. UPF is currently in the process of developing photo-realistic rendering and HDR environmental light extraction tools for the web-based 3D graphics application WebGLStudio. An interactive web demonstrator has been created to show users a proof of concept version of these tools while the main render pipeline is upgraded to support full functionality.
WP5 Visualization: Barco has made progress towards optimized real-time processing of HDR images for the phase and amplitude modulators by implementing an improved light steering algorithm. Along with providing image quality advantages over the conventional algorithm, this new method increases light efficiency up to 40%. With this improvement a proof of concept prototype has been built and used in a series of industry demonstrations, discussions, experiments and consumer studies. At UPF, methods are under development for the conversion of mastered SDR content to HDR and vice versa. Prototype versions of the algorithms have been developed and psychophysical experiments for fine tuning the parameters are being prepared.
WP6 Product Validation, Testing and Evaluation: After verifying that their current HDR set up and work flow reflects the state of the art, colourists at S&M have performed a test grading process using material from WP2 as well as from other sources. SDR and HDR masters were graded from test material, and the entire process was evaluated on a qualitative basis as colour grading work flow efficiency is inherently difficult to quantify.
WP7 Market Impact and Acceptance: Project results have begun to be disseminated to the research and consumer communities in the form of live demonstrations conducted by Filmlight and Barco at major industry events. The two companies are also engaged in a number of initiatives to explore HDR opportunities and remain influential in discussions concerning the future of HDR. Filmlight has been continuously monitoring the market for updates on the state of the art HDR solutions which are relevant to the work being done in HDR4EU and their findings have been reported to project partners. Infrastructure has also been launched to facilitate further dissemination of project results including the project website and data management plan.
WP2 Capture: Arri conducted a survey among partners concerning the technical requirements of test content and has shot or sourced 26 of the 32 requested items. Together Arri and Brainstorm also conducted an enhanced HDR shoot for the purpose of evaluating the HDR virtual studio tools developed for this project. Smoke & Mirrors are in the process of completing the necessary facility upgrades required to carry out a state of the art HDR post production work flow.
WP3 Generation for multiplatform delivery: In response to the problems associated with colour grading HDR content using traditional digital methods, Filmlight has developed new tools for colour grading and management based on a novel colour appearance model (TCAM). The model is also being used as a foundation for the cross conversion and environmental customization tools which are currently under development. Working prototypes for all of these components have been developed so that further testing can be carried out. Live demonstrations and seminars are being conducted to encourage adoption of these new tools. UPF is contributing to the expansion of the HDR tool set by developing tone mapping and inverse tone mapping algorithms. In the past year, algorithm prototypes have been developed and psychophysical tests have been conducted pertaining to the optimization of their parameters.
WP4 Real Time Generation: Brainstorm's eStudio tool has been updated with HDR capture and play out functionality. This updated version of the software is being shared with select customers for the purpose of validating the colour conversion methods. UPF is currently in the process of developing photo-realistic rendering and HDR environmental light extraction tools for the web-based 3D graphics application WebGLStudio. An interactive web demonstrator has been created to show users a proof of concept version of these tools while the main render pipeline is upgraded to support full functionality.
WP5 Visualization: Barco has made progress towards optimized real-time processing of HDR images for the phase and amplitude modulators by implementing an improved light steering algorithm. Along with providing image quality advantages over the conventional algorithm, this new method increases light efficiency up to 40%. With this improvement a proof of concept prototype has been built and used in a series of industry demonstrations, discussions, experiments and consumer studies. At UPF, methods are under development for the conversion of mastered SDR content to HDR and vice versa. Prototype versions of the algorithms have been developed and psychophysical experiments for fine tuning the parameters are being prepared.
WP6 Product Validation, Testing and Evaluation: After verifying that their current HDR set up and work flow reflects the state of the art, colourists at S&M have performed a test grading process using material from WP2 as well as from other sources. SDR and HDR masters were graded from test material, and the entire process was evaluated on a qualitative basis as colour grading work flow efficiency is inherently difficult to quantify.
WP7 Market Impact and Acceptance: Project results have begun to be disseminated to the research and consumer communities in the form of live demonstrations conducted by Filmlight and Barco at major industry events. The two companies are also engaged in a number of initiatives to explore HDR opportunities and remain influential in discussions concerning the future of HDR. Filmlight has been continuously monitoring the market for updates on the state of the art HDR solutions which are relevant to the work being done in HDR4EU and their findings have been reported to project partners. Infrastructure has also been launched to facilitate further dissemination of project results including the project website and data management plan.
The impacts of HDR4EU, while quite substantial or even remarkable for each player involved (content creators, hardware makers, service providers, etc.) could not be achieved if they were to be addressed independently by each partner. Isolated efforts, as the ones that have been carried out so far, can only have a limited impact. However, HDR4EU, by facilitating the emergence of an HDR ecosystem, will have an impact much larger than the sum of its parts: content creators will have guidelines to produce HDR content with a stunning, fundamentally improved appearance, which then can be adequately and smoothly postproduced with a simple HDR workflow before being properly displayed on affordable, energy-efficient HDR cinema projectors in optimal image quality conditions or on home or portable displays that personalise the content appearance for an optimal viewer experience. This will increase the demand for more HDR content, hence creating a virtuous cycle that is beneficial for all involved and supports and strengthens the HDR ecosystem. The importance of this ecosystem in the European context cannot be overstated: the European film industry comprises over 75,000 companies, employing more than 370,000 people, with a revenue of some EUR 60 Billion in 201073, while the cultural and industrial significance of cinema are recognized by a number of EU protection laws and film-support schemes (EUR 2.1 Billion in 2009). The whole media sector where HDR4EU will impact is significantly larger.