Multi-modal Context Modelling for Machine Translation

Publications

Peer reviewed articles (3)

Unraveling the Contribution of Image Captioning and Neural Machine Translation for Multimodal Machine Translation

Author(s): Chiraag Lala, Pranava Madhyastha, Josiah Wang, Lucia Specia
Published in: The Prague Bulletin of Mathematical Linguistics, Issue 108/1, 2017, Page(s) 197–208, ISSN 1804-0462
DOI: 10.1515/pralin-2017-0020

The role of image representations in vision to language tasks

Author(s): PRANAVA MADHYASTHA, JOSIAH WANG, LUCIA SPECIA
Published in: Natural Language Engineering, Issue 24/03, 2018, Page(s) 415-439, ISSN 1351-3249
DOI: 10.1017/s1351324918000116

Assessing multilingual multimodal image description: Studies of native speaker preferences and translator choices

Author(s): STELLA FRANK, DESMOND ELLIOTT, LUCIA SPECIA
Published in: Natural Language Engineering, Issue 24/03, 2018, Page(s) 393-413, ISSN 1351-3249
DOI: 10.1017/s1351324918000074

Conference proceedings (15)

Object Counts! Bringing Explicit Detections Back into Image Captioning

Author(s): Wang, Josiah; Madhyastha, Pranava; Specia, Lucia
Published in: Proceedings of 2018 Conference of the North American Chapter of the Association for Computational Linguistics, Issue 2, 2018

Findings of the Second Shared Task on Multimodal Machine Translation and Multilingual Image Description

Author(s): Elliott, Desmond; Frank, Stella; Barrault, Loïc; Bougares, Fethi; Specia, Lucia
Published in: Proceedings of the Second Conference on Machine Translation, Issue 3, 2017, Page(s) 215--233

Defoiling Foiled Image Captions

Author(s): Madhyastha, Pranava; Wang, Josiah; Specia, Lucia
Published in: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics (NAACL 2018), Issue 1, 2018

End-to-end Image Captioning Exploits Multimodal Distributional Similarity

Author(s): Madhyastha, Pranava; Wang, Josiah; Specia, Lucia
Published in: Proceedings of the Bristish Machine Vision Conference (BMVC), Issue 1, 2018

Sheffield MultiMT: Using Object Posterior Predictions for Multimodal Machine Translation

Author(s): Pranava Swaroop Madhyastha, Josiah Wang, and Lucia Specia
Published in: Second Conference on Machine Translation, 2017, Page(s) 470–476

Sheffield submissions for wmt18 multimodal translation shared task

Author(s): C. Lala, P. Swaroop Madhyastha, Carolina Scarton, L. Specia
Published in: Conference on Machine Translation, 2018

How2: A Large-scale Dataset for Multimodal Language Understanding

Author(s): R. Sanabria, O. Caglayan, S. Palaskar, D. Elliott, L. Barrault, L. Specia, F. Metze
Published in: NeurIPS Workshop on Visually Grounded Interaction and Language (ViGIL), 2018

Probing the Tole of Images in Multimodal Machine Translation

Author(s): O. Caglayan, L. Barrault, P. Madhyastha, L. Specia
Published in: NAACL, 2019

Findings of the Third Shared Task on Multimodal Machine Translation

Author(s): L. Barrault, F. Bougares, L. Specia, C. Lala, D. Elliott, S. Frank
Published in: Conference on Machine Translation, 2018

Semi-supervised Adaptation of RNNLMs by Fine-tuning with Domain-specific Auxiliary Features

Author(s): Salil Deena, Raymond W. M. Ng, Pranava Madhyastha, Lucia Specia, and Thomas Hain
Published in: INTERSPEECH, 2017, Page(s) 2715–2719

Multi30K: Multilingual English-German Image Descriptions

Author(s): Desmond Elliott, Stella Frank, Khalil Sima'an, and Lucia Specia
Published in: 5th Workshop on Vision and Language, 2016, Page(s) 70–74

Multimodal Lexical Translation

Author(s): Chiraag Lala and Lucia Specia
Published in: Eleventh International Conference on Language Resources and Evaluation, 2018, Page(s) 3810–3817

Exploring the use of acoustic embeddings in neural machine translation

Author(s): Salil Deena, Raymond Ng, Pranava Madhyastha, Lucia Specia, and Thomas Hain
Published in: IEEE ASRU Workshop, 2017

SHEF-Multimodal: Grounding Machine Translation on Images

Author(s): Kashif Shah, Josiah Wang and Lucia Specia
Published in: Conference on Machine Translation, 2016

A Shared Task on Multimodal Machine Translation and Crosslingual Image Description

Author(s): Lucia Specia, Stella Frank, Khalil Sima'an, and Desmond Elliott
Published in: First Conference on Machine Translation, 2016, Page(s) 540–550

Datasets

Datasets via OpenAIRE (5)

Cross-lingual Visual Pre-training for Multimodal Machine Translation

Author(s): Ozan Caglayan; Menekse Kuyu; Mustafa Sercan Amac; Pranava Madhyastha; Erkut Erdem; Aykut Erdem; Lucia Specia

MultiSubs: A Large-scale Multimodal and Multilingual Dataset

Author(s): Wang, Josiah
Published in: Zenodo

Cross-lingual Visual Pre-training for Multimodal Machine Translation

Author(s): Ozan Caglayan; Menekse Kuyu; Mustafa Sercan Amac; Pranava Madhyastha; Erkut Erdem; Aykut Erdem; Lucia Specia

MultiSubs: A Large-scale Multimodal and Multilingual Dataset

Author(s): Wang, Josiah
Published in: Zenodo

BERTGen: Multi-task Generation through BERT

Author(s): Faidon Mitzalis; Ozan Caglayan; Pranava Madhyastha; Lucia Specia