- On the theoretical side, we have introduced a general framework
that unifies pseudodifferential inpainting and interpolation with
radial basis functions. We have also established a rigorous existence
theory for edge-enhancing anisotropic diffusion which offers
state-of-the-art performance for general imagery. Moreover, we have
laid the theoretical foundations of inpainting-based codecs in terms
of sparsification and quantisation scale-spaces.
- Many alternative approaches for sparse inpainting have been studied
that are either novel or have not been introduced into inpainting-based
codecs so far. This includes pseudodifferential inpainting, Shepard
interpolation, inpainting with smoothed particle hydrodynamics,
Euler's elastica, diffusion-shock inpainting, and sparse exemplar-based
inpainting. The latter allowed to reconstruct texture in a quality
that exceeds previous approaches by a large margin.
- We have developed various extensions and adaptations of inpainting-based
codecs. This includes dedicated models for piecewise smooth images, motion
fields, and audio signals. Also novel features beyond grey or colour values
have been introduced, e.g. local averages. These variants, which can offer
superior performance in individual applications, illustrate the generality
and flexibility of the inpainting-based compression paradigm.
- Improved data selection strategies have been developed that are more
efficient and allow reconstructions of higher quality. This includes
densification strategies which use error map dithering and do not need
many inpaintings, as well as deep neural network approaches that do not
require any inpaintings at all. Most importantly, we have established
a general framework for the simultaneous incorporation of different
feature types for inpainting-based image representations.
- To encode these optimised data in a compact way, we have performed a
systematic evaluation of coding strategies for sparse inpainting masks.
This allowed us to find and adapt the most powerful ones.
- Various algorithmic accelerations have been implemented that lead
to fast data inpaintings. We have shown how one can speed up
exemplar-based inpainting with space-filling curves, and we have
demonstated the advantages of finite element methods that exploit
adaptive triangulations. An approach that applies the discrete cosine
transform within block decompositions allowed us to perform real-time
video decoding in Full HD resolution already with a purely CPU-based
implementation. Last but not least, exploring the parallelisation
capabilities of domain decomposition methods on a GPU, we were able
to inpaint almost 40 colour images in 4K resolution in one second.
This constitutes the envisioned 4K real-time demonstrator and
concludes the INCOVID project.
These achievements have been presented in invited keynote talks at
leading conferences in the fields of data compression (DCC 2018), applied
mathematics (GAMM 2018), and imaging science (SIAM IS 2022), ensuring
a wide dissemination within these scientific communities.
