Final Report Summary - PROVISION (PeRceptually Optimised VIdeo compreSION.)
http://www.provision-itn.eu
The PROVISION ITN consisted of 5 Full Partners: Fraunhofer HHI (DE), University of Bristol (UK), BBC (UK), RWTH (DE) and University of Nantes (FR). Also, 6 Associated Partners – YouTube (US), Netflix (US), Technicolor (FR), VITEC (FR), Queen Marry University (UK) and TU Berlin (DE) participated in PROVISION, which offered secondments and technical guidance to the project fellows. The project took on the challenge to deliver higher volumes of more immersive content over congested band-limited channels. It delivers world-leading technical innovation through a unique highly integrated training programme, producing skilled staff, efficient and validated solutions and impact through standardisation. Only through interdisciplinary collaboration across academia and industry, can the required transformational research be achieved. The projects objectives were to: 1.) provide the highest levels of technical and transferable skills training to the researcher team; 2.) conduct training in the context of an international research collaboration between some of the leading universities, research institutes, companies and end-user enterprises in Europe; 3.) complement the PROVISION consortium with international experts as associated partners, exposing the young scientists to cultural and organisational variations in research; 4.) enhance the PROVISION consortium by internationally respected visiting scientists from the public and the private sector to guarantee optimal conditions for all its researchers (ESR and ER).
To achieve the project goals and objectives, 18 fellows participated in the project. The fellows achieved significant technological progress within their research teams, as well as during the individual secondments. Here, scientific advancement in the area of perceptual video coding has been achieved through the fellows’ individual work, which was structured along 5 scientific work packages. The technological highlights include: Identification of shortcoming in current video codecs and measurement tool contribution to international standardisation in WP1, optical-flow-based motion modelling for irregular textures and 3D structural reconstruction for prediction in WP2, three-cluster texture model with adapted coding, new content synthesis methods through template matching and subsampling in WP3, visual attention models for 2D and 360° video, development of perceptually optimized intra coding, new visual disruption quality metric and non-reference VQA models in WP4, as well as successful core validation through oral/poster presentations in international events, demonstrations, subjective assessment, feedback by international experts and joint topic-based working groups in WP5. The novelty of the scientific work has been proven through a number of scientific publications, including more than 50 papers at high-level international conferences, such as ICIP, ICME, ICASSP, MMSP, PCS, EUSIPCO, QoMEX and international journals, such as Electronic Imaging. Several publications were honoured with Best Paper Awards. Furthermore, the fellows have provided more than 15 contributions to standard and are co-inventor of 4 patents. In addition to these activities, a novel type of activity, Grand Challenges, also took place. Here, PROVISION submitted two proposals in 2017 and PROVISION was offered to organize both. Each fellow significantly contributed to the scientific community in image processing and video coding, with the most advanced research topic within PROVISION given below:
Name Most Significant Achievement
Dan Grois Test set, framework and core experiment definition
Angeliki Katsenou RD-curve prediction from uncompressed content
Sebastian Schwarz Improved reference picture list sorting and signaling
Gayathri Venugopal Fast template matching for intra prediction
Ivan Župančić Variable bitrate rate control
Santiago de Luxán Hernández Multiple transform residual coding
Mariana Afonso Low-complexity spatio-temporal adaptation for video compression
Thomas Ntasios Video texture analysis and classification
Miltiadis Alexios Papadopoulos Perceptual RDO augmentation
Andre Seixas Dias JND-optimization and intra prediction for video coding
Uday Thakur Fine detail reconstruction for B pictures
Olena Chubach Motion-based texture synthesis
Hossein Bakhshi Golestani Motion compensation using estimated 3D-Models
Karam Naser Short-term perceptual fidelity metric for dynamic texture coding
Yashas Rai Objective prediction of visual disruption of coding artifacts
Suiyi Ling Perceptual model of temporal structural artifacts
Jesús Gutiérrez Image feature characterization of lightfield content
Ahmed Aldahdooh Video chain optimization using content characteristics
This could also be achieved through PROVISION training activities as a central component in the career development of its researchers, enabling them to enhance existing and develop new skills, both related to technical and complementary topics. This included training on different video coding architectures, challenges of commercializing video technology, overview, perspectives and principles in standardisation, as well as on vitae career development, pitching ideas and presentation skills, video making sessions, technical project management, patents and IPR in video compression, writing technical research papers and PhD planning and organization, start-ups and business plans, CV production and interview techniques. Fellows have been engaged in these PROVISION training events. All Fellows gained experience in presenting their work during work package updates and engaged in discussions on progress and planning. Fellows finally gained significant presentation experience through the PROVISION Public Workshop (Berlin) and Summer School on Video Compression and Processing (SVCP) (Kerkrade).
Through their scientific achievements and presentations to the scientific community at respective conferences and standardization meetings, as well as the wider audience at more general events and exhibitions, the fellows highly successfully promoted PROVISION technology beyond the international community in image processing. After the end of their PROVISION engagement, all fellows successfully secured their future technical careers with continuing employments at the full partner’s premises, at new employments at technical companies in the area of video compression or at an own start-up.
The PROVISION ITN consisted of 5 Full Partners: Fraunhofer HHI (DE), University of Bristol (UK), BBC (UK), RWTH (DE) and University of Nantes (FR). Also, 6 Associated Partners – YouTube (US), Netflix (US), Technicolor (FR), VITEC (FR), Queen Marry University (UK) and TU Berlin (DE) participated in PROVISION, which offered secondments and technical guidance to the project fellows. The project took on the challenge to deliver higher volumes of more immersive content over congested band-limited channels. It delivers world-leading technical innovation through a unique highly integrated training programme, producing skilled staff, efficient and validated solutions and impact through standardisation. Only through interdisciplinary collaboration across academia and industry, can the required transformational research be achieved. The projects objectives were to: 1.) provide the highest levels of technical and transferable skills training to the researcher team; 2.) conduct training in the context of an international research collaboration between some of the leading universities, research institutes, companies and end-user enterprises in Europe; 3.) complement the PROVISION consortium with international experts as associated partners, exposing the young scientists to cultural and organisational variations in research; 4.) enhance the PROVISION consortium by internationally respected visiting scientists from the public and the private sector to guarantee optimal conditions for all its researchers (ESR and ER).
To achieve the project goals and objectives, 18 fellows participated in the project. The fellows achieved significant technological progress within their research teams, as well as during the individual secondments. Here, scientific advancement in the area of perceptual video coding has been achieved through the fellows’ individual work, which was structured along 5 scientific work packages. The technological highlights include: Identification of shortcoming in current video codecs and measurement tool contribution to international standardisation in WP1, optical-flow-based motion modelling for irregular textures and 3D structural reconstruction for prediction in WP2, three-cluster texture model with adapted coding, new content synthesis methods through template matching and subsampling in WP3, visual attention models for 2D and 360° video, development of perceptually optimized intra coding, new visual disruption quality metric and non-reference VQA models in WP4, as well as successful core validation through oral/poster presentations in international events, demonstrations, subjective assessment, feedback by international experts and joint topic-based working groups in WP5. The novelty of the scientific work has been proven through a number of scientific publications, including more than 50 papers at high-level international conferences, such as ICIP, ICME, ICASSP, MMSP, PCS, EUSIPCO, QoMEX and international journals, such as Electronic Imaging. Several publications were honoured with Best Paper Awards. Furthermore, the fellows have provided more than 15 contributions to standard and are co-inventor of 4 patents. In addition to these activities, a novel type of activity, Grand Challenges, also took place. Here, PROVISION submitted two proposals in 2017 and PROVISION was offered to organize both. Each fellow significantly contributed to the scientific community in image processing and video coding, with the most advanced research topic within PROVISION given below:
Name Most Significant Achievement
Dan Grois Test set, framework and core experiment definition
Angeliki Katsenou RD-curve prediction from uncompressed content
Sebastian Schwarz Improved reference picture list sorting and signaling
Gayathri Venugopal Fast template matching for intra prediction
Ivan Župančić Variable bitrate rate control
Santiago de Luxán Hernández Multiple transform residual coding
Mariana Afonso Low-complexity spatio-temporal adaptation for video compression
Thomas Ntasios Video texture analysis and classification
Miltiadis Alexios Papadopoulos Perceptual RDO augmentation
Andre Seixas Dias JND-optimization and intra prediction for video coding
Uday Thakur Fine detail reconstruction for B pictures
Olena Chubach Motion-based texture synthesis
Hossein Bakhshi Golestani Motion compensation using estimated 3D-Models
Karam Naser Short-term perceptual fidelity metric for dynamic texture coding
Yashas Rai Objective prediction of visual disruption of coding artifacts
Suiyi Ling Perceptual model of temporal structural artifacts
Jesús Gutiérrez Image feature characterization of lightfield content
Ahmed Aldahdooh Video chain optimization using content characteristics
This could also be achieved through PROVISION training activities as a central component in the career development of its researchers, enabling them to enhance existing and develop new skills, both related to technical and complementary topics. This included training on different video coding architectures, challenges of commercializing video technology, overview, perspectives and principles in standardisation, as well as on vitae career development, pitching ideas and presentation skills, video making sessions, technical project management, patents and IPR in video compression, writing technical research papers and PhD planning and organization, start-ups and business plans, CV production and interview techniques. Fellows have been engaged in these PROVISION training events. All Fellows gained experience in presenting their work during work package updates and engaged in discussions on progress and planning. Fellows finally gained significant presentation experience through the PROVISION Public Workshop (Berlin) and Summer School on Video Compression and Processing (SVCP) (Kerkrade).
Through their scientific achievements and presentations to the scientific community at respective conferences and standardization meetings, as well as the wider audience at more general events and exhibitions, the fellows highly successfully promoted PROVISION technology beyond the international community in image processing. After the end of their PROVISION engagement, all fellows successfully secured their future technical careers with continuing employments at the full partner’s premises, at new employments at technical companies in the area of video compression or at an own start-up.