A major actitivity was to train the ESRs scientifically, in communication and dissemination, in exploitation and entrepreneurship. Scientific training has taken place in the form of summer schools, first secondments and local training. Four week-long training events have been completed, including scientific training, transferable skills, entrepreneurship and teambuilding.
Activities and selected results within the 5 steps of the full signal path are reported below:
1) Acquisition: A reconfigurable multi-camera, multi-view capture was established and an algorithm was developed for creating high-quality visual experiences based on the captured data. Based on these, a first high-resolution high dynamic range light field data set was captured and made available to the community.
2) Coding and Processing: The quality of light field images has been studied and an analysis of the advantage of Epipolar Plane Image representations was presented. A new technique for compression of the resulting large image data sets has been devised. A novel semantic aware Tone Mapping Operator for HDR images has also been developed.
3) Display: For faithful reconstruction of the 5D plenoptic function by hyper-realistic displays (H-RD), from the point of view of visual perception of light-field imaging, significant progress has been achieved on sampling of the plenoptic function. Combined with the work on perceptual models, a number of novel experimental (in-lab) displays have been developed (see below.)
4) Perceptual Models: A novel technique taking advantage of binocular fusion to boost perceived local contrast and visual quality of images has been devised. It is also used with the novel developed High-Dynamic-Range Multi-Focal Stereo (HDR-MF-S) display with an end-to-end imaging and rendering system. Research is conducted to improve the colour experience for individual observers, e.g. contributions have been made to develop a multi-primary HDR display with perceptual control.
5) Image Quality: To develop a novel surface quality metric for controlling object appearances, a CNN based metric was devised for detecting visible artifacts in 2D images, which can also be extended to light fields. Research on immersion was conducted, and discussed in light of implications on immersive audio-visual (AV) experiences. This work comprised defining immersion, followed up by subjective tests on audio-visual data, to establish test methodology and conducting subjective testing of immersion.
Besides the many significant scientific papers and conference presentations, important means of dissemination has been in form of high quality data sets and selected software. The RealVision project, the ideas and results have been presented at international trade shows.
The direct exploitation is primarily in form of collaboration with RealVision, industrial beneficiaries, but more indirectly also the strong industrial associated partners.
