Final Report Summary - INTERFERE (Sparse Signal Coding for Interference-based Imaging Modalities)
In recent years, the domain of plenoptic imaging – for which holography is a key enabling modality – has known significant scientific and industrial evolutions. In this context, the ERC INTERFERE project stands on a matured vision of a future end-to-end holographic television system. Summarized, such systems are envisioned to use heterogeneous signal sources, originating from light field sensors and camera arrays, point cloud scanners, time-of-flight (TOF) cameras, holographic sensing devices (for miniature scenes), that are subsequently converted to a generic, sparse, holographic representation format, utilizing computer generated holography (CGH) techniques.
From this holographic ‘information container’, a continuum of views can be extracted that serves as input for e.g. holographic head mounted display (HMD) devices or interactive holographic visualization tables. How to efficiently generate and represent this holographic information is a key question that is being addressed by INTERFERE. Research in the context as well as the associated dissemination actions have been steered along this vision. INTERFERE delivered novel technologies and methodologies at microscopic and macroscopic scales. The research addressed the problem as a whole from capture, representation, transmission and final display of complex amplitude holograms.
CGH algorithms have been designed that handle efficiently occlusions, allow for advanced illumination models and holograms of realistic scenes. Moreover, the project focused on solutions that decrease the computation complexity by for example deploying sparse mappings of diffraction patterns. In addition, GPU implementations have been designed that support short and far distance light wave propagation, generation of large holograms and parallelization over multiple GPUs.
Various space-frequency transforms such as wavelets, directional packet-based wavelet transforms, wave atoms and short-time Fourier transforms have been evaluated on holographic data. Also a novel modulo wavelet transform based on a fast lifting scheme processes phase data to allow for fast phase unwrapping in 4D profilometry. Particularly a new unitary transform models the diffraction between arbitrary (non-planar) surfaces: these transforms are reversible and therefore generalize Fresnelets to non-planar target spaces and hence are allowing for more efficient representations of deep holographic scenes. The project resulted in an codec architecture to compress dynamic holograms. This architecture is designed based upon a motion compensation system that accurately compensates the motion between frames by manipulating the hologram in space-frequency domain. The intraframes and prediction errors are encoded utilizing a short-time Fourier transform based coding.
Holograms are recordings of constructive and destructive light interference patterns and the signal is very different from natural imagery. Therefore, adapted quality assessment metrics and procedures had to be designed. A generalized objective quality metric based on sparse coding that is suitable for both classic and holographic image content was proposed. A novel prediction accuracy analysis uses the separation ratio for each partial quality estimator, which allows for accurately evaluating the response of quality metrics given certain stimuli. Subjective quality assessment procedures for holographic data which are based on alternative rendering modalities such as regular 2D displays and/or autostereoscopic light field displays allow for effective testing of the structural integrity of the information contained by the holograms.
Many results of INTERFERE are also directly applicable at the microscopic scale. For example, a theoretically optical design of the distance from sample to the detector, in terms of assumed sparsity in the image reconstruction domain: hence, this design methodology provides guidelines for configuring a compressed sensing setup for holographic capturing. A state-of-the-art holographic image codec has been implemented, based on packet and directional wavelet transforms. It provides excellent rate distortion performance for microscopic holographic content.
From this holographic ‘information container’, a continuum of views can be extracted that serves as input for e.g. holographic head mounted display (HMD) devices or interactive holographic visualization tables. How to efficiently generate and represent this holographic information is a key question that is being addressed by INTERFERE. Research in the context as well as the associated dissemination actions have been steered along this vision. INTERFERE delivered novel technologies and methodologies at microscopic and macroscopic scales. The research addressed the problem as a whole from capture, representation, transmission and final display of complex amplitude holograms.
CGH algorithms have been designed that handle efficiently occlusions, allow for advanced illumination models and holograms of realistic scenes. Moreover, the project focused on solutions that decrease the computation complexity by for example deploying sparse mappings of diffraction patterns. In addition, GPU implementations have been designed that support short and far distance light wave propagation, generation of large holograms and parallelization over multiple GPUs.
Various space-frequency transforms such as wavelets, directional packet-based wavelet transforms, wave atoms and short-time Fourier transforms have been evaluated on holographic data. Also a novel modulo wavelet transform based on a fast lifting scheme processes phase data to allow for fast phase unwrapping in 4D profilometry. Particularly a new unitary transform models the diffraction between arbitrary (non-planar) surfaces: these transforms are reversible and therefore generalize Fresnelets to non-planar target spaces and hence are allowing for more efficient representations of deep holographic scenes. The project resulted in an codec architecture to compress dynamic holograms. This architecture is designed based upon a motion compensation system that accurately compensates the motion between frames by manipulating the hologram in space-frequency domain. The intraframes and prediction errors are encoded utilizing a short-time Fourier transform based coding.
Holograms are recordings of constructive and destructive light interference patterns and the signal is very different from natural imagery. Therefore, adapted quality assessment metrics and procedures had to be designed. A generalized objective quality metric based on sparse coding that is suitable for both classic and holographic image content was proposed. A novel prediction accuracy analysis uses the separation ratio for each partial quality estimator, which allows for accurately evaluating the response of quality metrics given certain stimuli. Subjective quality assessment procedures for holographic data which are based on alternative rendering modalities such as regular 2D displays and/or autostereoscopic light field displays allow for effective testing of the structural integrity of the information contained by the holograms.
Many results of INTERFERE are also directly applicable at the microscopic scale. For example, a theoretically optical design of the distance from sample to the detector, in terms of assumed sparsity in the image reconstruction domain: hence, this design methodology provides guidelines for configuring a compressed sensing setup for holographic capturing. A state-of-the-art holographic image codec has been implemented, based on packet and directional wavelet transforms. It provides excellent rate distortion performance for microscopic holographic content.