Skip to main content

Personalised Content Creation for the Deaf Community in a Connected Digital Single Market

Deliverables

Integrated Pilot 1 system and evaluation report

This deliverable will document the CONTENT4ALL Pilot 1 integrated modules. All information on the pilot and its parameters, structures, formats, system requirements, and operational conditions will be described in detail. The report will also provide CONTENT4ALL evaluation based on technical, functional and business criteria, derived from the demonstrations

Initial requirements and specifications for pilot demonstrators

This report will mainly specify the application requirements, development environments and specifications for the application that will be used for the demonstration scenarios. Key scenarios identified will be included with use cases and evaluation tools for the application. High level object oriented design diagrams along with actors and entities involved will be presented.

Pilot 1 demonstrator architecture, integration plan, and evaluation methodologies

A plan for executing the T5.5 Pilot 1 large scale demonstration of the remote sign-interpretation scenario will be presented, including a full description of the system architecture, integration plan and trials to be carried out. Evaluation methodologies and metrics will also be included, along with a rationale for choosing those methodologies.

Integrated Pilot 2 system and evaluation report

This deliverable will document the CONTENT4ALL Pilot 2 integrated modules. Software and documentation will be delivered with instructions on installing, updating, management, configuration, troubleshooting and integration. All information on the pilot and its parameters, structures, formats, system requirements, and operational conditions will be described in detail. The report will also provide CONTENT4ALL evaluation based on technical, functional and business criteria, derived from the demonstrations.

Pilot 2 demonstrator architecture, integration plan, and evaluation methodologies

A plan for executing the T5.6 large scale demonstration of the remote sign-interpretation scenario will be presented, including a full description of the system architecture, integration plan and trials to be carried out. Evaluation methodologies and metrics will also be included, along with a rationale for choosing those methodologies.

Initial reference system architecture

An initial architecture will be proposed, including detailed block diagrams, interface descriptions, operating scenarios, and functionality. The proposed architecture will provide a framework to carry out tasks in remaining work packages. Key technologies to be developed will be highlighted and a high level design approach/success measures will be described.

Final Report on 3D model capture and rendering

D3.1 will be updated based on the final status of the prototype algorithms, developed in WP3, WP4 and WP5. Performance results will be available from the prototype algorithms.

Initial Report on 3D model capture and rendering

This deliverable will provide information and a description of the WP3 achievements on a yearly basis. It will describe the novel hybrid human body model representation, algorithms for tracking, warping and rendering as well as output samples and evaluation results

Data Management Plan

This document will form the basis for collecting, managing and curating the data generated in CONTENT4ALL to be released as open research data in D5.7. The data management plan will specifically outline how the data is to be used and the legal aspects of sharing, exploiting and commericalizing products based on the shared information.

Final reference system architecture report

D2.1 will be updated based on the final status of the prototype algorithms, developed in WP3, WP4 and WP5. Compared to D2.1, performance results will be available from the prototype algorithms. This will provide precise and final definitions of the interfaces between components and will enable the optimisation of the architecture and the elimination of potential performance bottlenecks.

Final requirements and specifications for pilot demonstrators

D2.2 will be updated to provide the final requirements and specifications for demonstration and evaluation. Inputs, outputs, communication channels, test cases, operational environment information will be included. Results from initial versions of the application and outcomes from WP5 with respect to scalable application development will be included.

Project Web-site

An interactive website will be established (www.content4all.com). The website (including FTP services for consortium members) will describe the aims of the project, describe the partners, and will list the publications and standards contributions. It will also allow public deliverables to be downloaded. Links to social networking will also be included in the website. A secure area will be set up for confidential documentation for consortium members. This website will be disseminated in all major events to provide a unified view of the project (T6.2).

Bi-annual project newsletters

Bi-annual project newsletters will provide an insight to the project activities. The newsletter will be created as a multi-media document, including promotional materials which will be created for different audiences - technical audiences and the general public. The latter can be reached through CONTENT4ALL related information in TV programmes, e.g. BBC Click in the UK. The Newsletter will also provide “position statements” from experts in different disciplines related to CONTENT4ALL (T6.2).

Project Leaflet

A leaflet will be produced to publicise the key objectives of the project and will be circulated in all major related events/conferences/forums etc. The leaflet will be maintained as a “live” document throughout the whole project, and updated based on project achievements (T6.2).

Sign-language translation dataset

This deliverable will be a dataset of visual sign language samples with associated metadata and labels that will be used for motion graph creations as well as avatar animation. The dataset will be released as open research data to the public.

Open research data for sign-translation

This deliverable will consist of domain-specific open research data generated from the large scale data collection in T5.5. The data collected in Pilot 1 will be reduced to a structured, annotated dataset for T5.6; hence, this reduced dataset will also be included as part of the dataset. Specific documentation of the data collection, analysis and evaluation approached will be included for reference and adoption by other researchers in the future.

Searching for OpenAIRE data...

Publications

Evaluation of Modifications to CPPPSNR in 360° Video Quality Assessment

Author(s): Jayasingam Adhuran, Gosala Kulupana, Chathura Galkandage, Anil Fernando
Published in: 2019 27th European Signal Processing Conference (EUSIPCO), 2019, Page(s) 1-5, ISBN 978-9-0827-9703-9
Publisher: IEEE
DOI: 10.23919/eusipco.2019.8903145

Sign Language Transformers: Joint End-to-End Sign Language Recognition and Translation

Author(s): Necati Cihan Camgoz, Oscar Koller, Simon Hadfield, Richard Bowden
Published in: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020, Page(s) 10020-10030, ISBN 978-1-7281-7168-5
Publisher: IEEE
DOI: 10.1109/cvpr42600.2020.01004

Affine Intra-prediction for Versatile Video Coding

Author(s): Jayasingam Adhuran, Anil Fernando, Gosala Kulupana, Saverio Blasi
Published in: 2020 28th European Signal Processing Conference (EUSIPCO), 2021, Page(s) 545-549, ISBN 978-9-0827-9705-3
Publisher: IEEE
DOI: 10.23919/eusipco47968.2020.9287579

Virtual Frames as Long-Term Reference Frames for HEVC Inter-Prediction

Author(s): Buddhiprabha Erabadda, Thanuja Mallikarachchi, Gosala Kulupana, Anil Fernando
Published in: 2020 IEEE International Conference on Consumer Electronics (ICCE), 2020, Page(s) 1-2, ISBN 978-1-7281-5186-1
Publisher: IEEE
DOI: 10.1109/icce46568.2020.9043054

Neural Face Models for Example-Based Visual Speech Synthesis

Author(s): Wolfgang Paier, Anna Hilsmann, Peter Eisert
Published in: European Conference on Visual Media Production, 2020, Page(s) 1-10, ISBN 9781450381987
Publisher: ACM
DOI: 10.1145/3429341.3429356

Optimal Distortion Minimization for 360 ° Video Compression with VVC

Author(s): Jayasingam Adhuran, Gosala Kulupana, Chathura Galkandage, Anil Fernando
Published in: 2020 IEEE International Conference on Consumer Electronics (ICCE), 2020, Page(s) 1-3, ISBN 978-1-7281-5186-1
Publisher: IEEE
DOI: 10.1109/icce46568.2020.9043034

Progressive Non-rigid Registration of Temporal Mesh Sequences

Author(s): Wieland Morgenstern, Anna Hilsmann, Peter Eisert
Published in: European Conference on Visual Media Production, 2019, Page(s) 1-10, ISBN 9781450370035
Publisher: ACM
DOI: 10.1145/3359998.3369411

Adversarial Training for Multi-Channel Sign Language Production

Author(s): Saunders, Ben; Camgoz, Necati Cihan; Bowden, Richard
Published in: The 31st British Machine Vision Virtual Conference, 9, 2020
Publisher: The 31st British Machine Vision Virtual Conference

Content Adaptive Fast CU Size Selection for HEVC Intra-Prediction

Author(s): Buddhiprabha Erabadda, Thanuja Mallikarachchi, Gosala Kulupana, Anil Fernando
Published in: 2019 IEEE International Conference on Consumer Electronics (ICCE), 2019, Page(s) 1-2, ISBN 978-1-5386-7910-4
Publisher: IEEE
DOI: 10.1109/icce.2019.8662119

Bit allocation and encoding parameter selection for rate-controlled error resilient HEVC video encoding

Author(s): Gosala Kulupana, Dumidu S. Talagala, Anil Fernando, Hemantha Kodikara Arachchi
Published in: 2018 IEEE International Conference on Consumer Electronics (ICCE), 2018, Page(s) 1-4, ISBN 978-1-5386-3025-9
Publisher: IEEE
DOI: 10.1109/icce.2018.8326287

Machine Learning Approaches for Intra-Prediction in HEVC

Author(s): Buddhiprabha Erabadda, Thanuja Mallikarachchi, Gosala Kulupana, Anil Fernando
Published in: 2018 IEEE 7th Global Conference on Consumer Electronics (GCCE), 2018, Page(s) 206-209, ISBN 978-1-5386-6309-7
Publisher: IEEE
DOI: 10.1109/gcce.2018.8574648

Decoding complexity-aware, rate, distortion optimized HEVC video encoding

Author(s): Thanuja Mallikarachchi, Dumidu S. Talagala, Hemantha Kodikara Arachchi, Anil Fernando
Published in: 2018 IEEE International Conference on Consumer Electronics (ICCE), 2018, Page(s) 1-4, ISBN 978-1-5386-3025-9
Publisher: IEEE
DOI: 10.1109/icce.2018.8326110

Neural Sign Language Translation

Author(s): Necati Cihan Camgoz, Simon Hadfield, Oscar Koller, Hermann Ney, Richard Bowden
Published in: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018, Page(s) 7784-7793, ISBN 978-1-5386-6420-9
Publisher: IEEE
DOI: 10.1109/cvpr.2018.00812

An adaptive video streaming framework for Scalable HEVC (SHVC) standard

Author(s): Sarat Rakngan, Thanuja Mallikarachchi, Anil Fernando
Published in: 2019 IEEE International Conference on Consumer Electronics (ICCE), 2019, Page(s) 1-2, ISBN 978-1-5386-7910-4
Publisher: IEEE
DOI: 10.1109/icce.2019.8662075

QoE Modelling of High Dynamic Range Video

Author(s): Carl C. Udora, Junaid Mir, Chatura Galkandage, Anil Fernando
Published in: 2019 IEEE International Conference on Consumer Electronics (ICCE), 2019, Page(s) 1-2, ISBN 978-1-5386-7910-4
Publisher: IEEE
DOI: 10.1109/icce.2019.8662122

Sign Language Production using Neural Machine Translation and Generative Adversarial Networks

Author(s): Stephanie Stoll and Necati Cihan Camgoz and Simon Hadfield and Richard Bowden
Published in: Proceedings of the British Conference on Machine Vision (BMVC), 2018
Publisher: BMVA Press

Efficient VVC Intra Coding for 360° Video with Residual Weighting and Adaptive Quantization

Author(s): Jayasingam Adhuran, Chathura Galkandage, Gosala Kulupana, Anil Fernando
Published in: 2020 IEEE International Conference on Consumer Electronics (ICCE), 2020, Page(s) 1-5, ISBN 978-1-7281-5186-1
Publisher: IEEE
DOI: 10.1109/icce46568.2020.9043002

Multi-channel Transformers for Multi-articulatory Sign Language Translation

Author(s): Camgoz, Necati Cihan; Koller, Oscar; Hadfield, Simon; Bowden, Richard
Published in: 8, 2020
Publisher: 16th European Conference on Computer Vision (ECCV), ACVR Workshop

Weakly-Supervised 3D Pose Estimation from a Single Image using Multi-View Consistency

Author(s): Rochette, Guillaume; Russell, Chris; Bowden, Richard
Published in: 30th British Machine Vision Conference, 1, 2019
Publisher: 30th British Machine Vision Conference

NestedVAE: Isolating Common Factors via Weak Supervision

Author(s): Matthew J. Vowels, Necati Cihan Camgoz, Richard Bowden
Published in: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020, Page(s) 9199-9209, ISBN 978-1-7281-7168-5
Publisher: IEEE
DOI: 10.1109/cvpr42600.2020.00922

Fast CU Size Decisions for HEVC Inter-Prediction Using Support Vector Machines

Author(s): Buddhiprabha Erabadda, Thanuja Mallikarachchi, Gosala Kulupana, Anil Fernando
Published in: 2019 27th European Signal Processing Conference (EUSIPCO), 2019, Page(s) 1-5, ISBN 978-9-0827-9703-9
Publisher: IEEE
DOI: 10.23919/eusipco.2019.8903081

iCUS: Intelligent CU Size Selection for HEVC Inter Prediction

Author(s): Buddhiprabha Erabadda, Thanuja Mallikarachchi, Gosala Kulupana, Anil Fernando
Published in: IEEE Access, 8, 2020, Page(s) 141143-141158, ISSN 2169-3536
Publisher: Institute of Electrical and Electronics Engineers Inc.
DOI: 10.1109/access.2020.3013804

Interactive facial animation with deep neural networks

Author(s): Wolfgang Paier, Anna Hilsmann, Peter Eisert
Published in: IET Computer Vision, 14/6, 2020, Page(s) 359-369, ISSN 1751-9632
Publisher: Institution of Engineering and Technology
DOI: 10.1049/iet-cvi.2019.0790

Quality of Experience (QoE)-Aware Fast Coding Unit Size Selection for HEVC Intra-prediction

Author(s): Buddhiprabha Erabadda, Thanuja Mallikarachchi, Chaminda Hewage, Anil Fernando
Published in: Future Internet, 11/8, 2019, Page(s) 175, ISSN 1999-5903
Publisher: Future Internet
DOI: 10.3390/fi11080175

Parameter-based Affine Intra Prediction of Screen Content in Versatile Video Coding

Author(s): Jayasingam Adhuran, Gosala Kulupana, Saverio Blasi, Anil Fernando
Published in: IEEE Transactions on Circuits and Systems for Video Technology, 2020, Page(s) 1-1, ISSN 1051-8215
Publisher: Institute of Electrical and Electronics Engineers
DOI: 10.1109/tcsvt.2020.3038159

Multiple Quantization Parameter Optimization in Versatile Video Coding for 360° Videos

Author(s): Jayasingam Adhuran, Gosala Kulupana, Chathura Galkandage, Anil Fernando
Published in: IEEE Transactions on Consumer Electronics, 66/3, 2020, Page(s) 213-222, ISSN 0098-3063
Publisher: Institute of Electrical and Electronics Engineers
DOI: 10.1109/tce.2020.3001231

Decoding-Complexity-Aware HEVC Encoding Using a Complexity–Rate–Distortion Model

Author(s): Thanuja Mallikarachchi, Dumidu S. Talagala, Hemantha Kodikara Arachchi, Anil Fernando
Published in: IEEE Transactions on Consumer Electronics, 64/1, 2018, Page(s) 35-43, ISSN 0098-3063
Publisher: Institute of Electrical and Electronics Engineers
DOI: 10.1109/tce.2018.2810479

Text2Sign: Towards Sign Language Production Using Neural Machine Translation and Generative Adversarial Networks

Author(s): Stephanie Stoll, Necati Cihan Camgoz, Simon Hadfield, Richard Bowden
Published in: International Journal of Computer Vision, 128/4, 2020, Page(s) 891-908, ISSN 0920-5691
Publisher: Kluwer Academic Publishers
DOI: 10.1007/s11263-019-01281-2

Markerless Multiview Motion Capture with 3D Shape Model Adaptation

Author(s): P. Fechteler, A. Hilsmann, P. Eisert
Published in: Computer Graphics Forum, 2019, ISSN 0167-7055
Publisher: Blackwell Publishing Inc.
DOI: 10.1111/cgf.13608

End User Video Quality Prediction and Coding Parameters Selection at the Encoder for Robust HEVC Video Transmission

Author(s): Gosala Kulupana, Dumidu S. Talagala, Hemantha Kodikara Arachchi, Anil Fernando
Published in: IEEE Transactions on Circuits and Systems for Video Technology, 2018, Page(s) 1-1, ISSN 1051-8215
Publisher: Institute of Electrical and Electronics Engineers
DOI: 10.1109/tcsvt.2018.2879956

Going beyond free viewpoint: creating animatable volumetric video of human performances

Author(s): Anna Hilsmann, Philipp Fechteler, Wieland Morgenstern, Wolfgang Paier, Ingo Feldmann, Oliver Schreer, Peter Eisert
Published in: IET Computer Vision, 14/6, 2020, Page(s) 350-358, ISSN 1751-9632
Publisher: Institution of Engineering and Technology
DOI: 10.1049/iet-cvi.2019.0786

Weakly Supervised Learning with Multi-Stream CNN-LSTM-HMMs to Discover Sequential Parallelism in Sign Language Videos

Author(s): Oscar Koller, Necati Cihan Camgoz, Hermann Ney, Richard Bowden
Published in: IEEE Transactions on Pattern Analysis and Machine Intelligence, 42/9, 2020, Page(s) 2306-2320, ISSN 0162-8828
Publisher: Institute of Electrical and Electronics Engineers
DOI: 10.1109/tpami.2019.2911077

Development of a Multilingual Questionnaire for the Deaf Community – Guidelines and Challenges

Author(s): Astrid Oehme, Vaishnavi Upadrasta, Philipp Kotsch
Published in: HCI International 2020 – Late Breaking Papers: Universal Access and Inclusive Design - 22nd HCI International Conference, HCII 2020, Copenhagen, Denmark, July 19–24, 2020, Proceedings, 12426, 2020, Page(s) 103-113, ISBN 978-3-030-60148-5
Publisher: Springer International Publishing
DOI: 10.1007/978-3-030-60149-2_9

User-Centred Design for Accessibility in Media Content – Sign Language and Virtual Signer

Author(s): Vaishnavi Upadrasta, Astrid Oehme, Sandra Böhm
Published in: HCI International 2020 – Late Breaking Papers: Universal Access and Inclusive Design - 22nd HCI International Conference, HCII 2020, Copenhagen, Denmark, July 19–24, 2020, Proceedings, 12426, 2020, Page(s) 126-143, ISBN 978-3-030-60148-5
Publisher: Springer International Publishing
DOI: 10.1007/978-3-030-60149-2_11

Progressive Transformers for End-to-End Sign Language Production

Author(s): Ben Saunders, Necati Cihan Camgoz, Richard Bowden
Published in: Computer Vision – ECCV 2020 - 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XI, 12356, 2020, Page(s) 687-705, ISBN 978-3-030-58620-1
Publisher: Springer International Publishing
DOI: 10.1007/978-3-030-58621-8_40

Everybody Sign Now: Translating Spoken Language to Photo Realistic Sign Language Video

Author(s): Saunders, Ben; Camgoz, Necati Cihan; Bowden, Richard
Published in: 3, 2020
Publisher: Self