Disagreements and Language Interpretation

Informations projet

DALI

N° de convention de subvention: 695662

DOI

10.3030/695662

Projet clôturé

Date de signature de la CE 25 Août 2016

Date de début 1 Septembre 2016

Date de fin 31 Janvier 2022

Financé au titre de

• EXCELLENT SCIENCE - European Research Council (ERC)

Coût total

€ 2 499 471,00

Contribution de l’UE

€ 2 499 471,00

Coordonné par

QUEEN MARY UNIVERSITY OF LONDON
United Kingdom

CORDIS fournit des liens vers les livrables publics et les publications des projets HORIZON.

Les liens vers les livrables et les publications des projets du 7e PC, ainsi que les liens vers certains types de résultats spécifiques tels que les jeux de données et les logiciels, sont récupérés dynamiquement sur OpenAIRE .

Publications

Conference proceedings (36)

Anaphoric zero pronoun identification: A multilingual approach

Auteurs: Abdulrahman Aloraini, Massimo Poesio
Publié dans: Proceedings of the Third Workshop on Computational Models of Reference, Anaphora and Coreference, 2020, Page(s) 22–32
Éditeur: Association for Computational Linguistics

Coreference Resolution for the Biomedical Domain: A Survey

Auteurs: Pengcheng Lu, Massimo Poesio
Publié dans: Proceedings of the Fourth Workshop on Computational Models of Reference, Anaphora and Coreference (CRAC), 2021, Page(s) 12–23
Éditeur: Association for Computational Linguistics
DOI: 10.18653/v1/2021.crac-1.2

Beyond black & white: Leveraging annotator disagreement via soft-label multi-task learning

Auteurs: Tommaso Fornaciari, Alexandra Uma, Silviu Paun, Barbara Plank, Dirk Hovy, Massimo Poesio
Publié dans: Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2021, Page(s) 2591–2597
Éditeur: Association for Computational Linguistics
DOI: 10.18653/v1/2021.naacl-main.204

The CODI-CRAC 2021 Shared Task on Anaphora, Bridging, and Discourse Deixis in Dialogue

Auteurs: Sopan Khosla, Juntao Yu, Ramesh Manuvinakurike, Vincent Ng, Massimo Poesio, Michael Strube, Carolyn Rosé
Publié dans: Proceedings of the CODI-CRAC 2021 Shared Task on Anaphora, Bridging, and Discourse Deixis in Dialogue, 2021, Page(s) 1–15
Éditeur: Association for Computational Linguistics
DOI: 10.18653/v1/2021.codi-sharedtask.1

Stay together: A system for single and split-antecedent anaphora resolution

Auteurs: Juntao Yu, Nafise Sadat Moosavi, Silviu Paun, Massimo Poesio
Publié dans: Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2021, Page(s) 4174–4184
Éditeur: Association for Computational Linguistics
DOI: 10.18653/v1/2021.naacl-main.329

Making text annotation fun with a clicker game

Auteurs: Chris Madge, Richard Bartle, Jon Chamberlain, Udo Kruschwitz, Massimo Poesio
Publié dans: Proceedings of the 14th International Conference on the Foundations of Digital Games, 2019, Page(s) 1-6, ISBN 9781450372176
Éditeur: ACM
DOI: 10.1145/3337722.3341869

Multitask Learning-Based Neural Bridging Reference Resolution

Auteurs: Juntao Yu, Massimo Poesio
Publié dans: Proceedings of the 28th International Conference on Computational Linguistics (COLING), 2020, Page(s) 3534–3546
Éditeur: International Committee on Computational Linguistics and Association for Computational Linguistics
DOI: 10.18653/v1/2020.coling-main.315

Crowdsourcing and Aggregating Nested Markable Annotations

Auteurs: Chris Madge, Juntao Yu, Jon Chamberlain, Udo Kruschwitz, Silviu Paun, Massimo Poesio
Publié dans: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019, Page(s) 797-807
Éditeur: Association for Computational Linguistics
DOI: 10.18653/v1/p19-1077

Cipher: A Prototype Game-with-a-Purpose for Detecting Errors in Text

Auteurs: Liang Xu, Jon Chamberlain
Publié dans: Workshop on Games and Natural Language Processing (Games and NLP 2020), 2020, Page(s) 17–25
Éditeur: European Language Resources Association

Classification of Low-Agreement Pronouns Through Collaborative Dialogue: A Proof of Concept

Auteurs: Janosch Haber, Massimo Poesio
Publié dans: Proceedings of the 24th Workshop on the Semantics and Pragmatics of Dialogue - Full Papers, 2020
Éditeur: SEMDIAL

Assessing polyseme sense similarity through co-predication acceptability and contextualised embedding distance

Auteurs: Janosch Haber, Massimo Poesio
Publié dans: Proceedings of the Ninth Joint Conference on Lexical and Computational Semantics, 2020, Page(s) 114–124
Éditeur: Association for Computational Linguistics

https://aclanthology.org/2020.nlp4call-1.3/

Auteurs: Etiene da Cruz Dalcol, Massimo Poesio
Publié dans: Proceedings of the 9th Workshop on NLP for Computer Assisted Language Learning, 2020, Page(s) 21–36
Éditeur: LiU Electronic Press

Patterns of Polysemy and Homonymy in Contextualised Language Models

Auteurs: Janosch Haber, Massimo Poesio
Publié dans: Findings of the Association for Computational Linguistics: EMNLP 2021, 2021, Page(s) 2663–2676
Éditeur: Association for Computational Linguistics
DOI: 10.18653/v1/2021.findings-emnlp.226

Wormingo - a 'true gamification' approach to anaphoric annotation

Auteurs: Doruk Kicikoglu, Richard Bartle, Jon Chamberlain, Massimo Poesio
Publié dans: Proceedings of the 14th International Conference on the Foundations of Digital Games, 2019, Page(s) 1-7, ISBN 9781450372176
Éditeur: ACM
DOI: 10.1145/3337722.3341868

Named Entity Recognition as Dependency Parsing

Auteurs: Juntao Yu, Bernd Bohnet, Massimo Poesio
Publié dans: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 2020, Page(s) 6470-6476
Éditeur: Association for Computational Linguistics
DOI: 10.18653/v1/2020.acl-main.577

Progression in a Language Annotation Game with a Purpose

Auteurs: Chris Madge, Juntao Yu, Jon Chamberlain, Udo Kruschwitz, Silviu Paun, Massimo Poesio
Publié dans: HCOMP-19: Proceedings of the AAAI Human Computation Conference, 2019
Éditeur: AAAI

Aggregation Driven Progression for GWAPs

Auteurs: Doruk Kicikoglu,Richard Bartle, Silviu Paun, Jon Chamberlain, Massimo Poesio
Publié dans: Proceedings of the LREC 2020 Workshop Games and Natural Language Processing, 2020, Page(s) 79-84
Éditeur: European Language Resources Association (ELRA)

Incremental Game Mechanics Applied to Text Annotation

Auteurs: Christopher Madge, Richard Bartle, Jon Chamberlain, Udo Kruschwitz, Massimo Poesio
Publié dans: Proceedings of the Annual Symposium on Computer-Human Interaction in Play, 2019, Page(s) 545-558, ISBN 9781450366885
Éditeur: ACM
DOI: 10.1145/3311350.3347184

Neural coreference resolution for Arabic

Auteurs: Abdulrahman Aloraini, Juntao Yu, Massimo Poesio
Publié dans: Proceedings of the Third Workshop on Computational Models of Reference, Anaphora and Coreference, 2020, Page(s) 99–110
Éditeur: Association for Computational Linguistics

Using Automatically Extracted Minimum Spans to Disentangle Coreference Evaluation from Boundary Detection

Auteurs: Nafise Sadat Moosavi, Leo Born, Massimo Poesio, Michael Strube
Publié dans: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019, Page(s) 4168-4178
Éditeur: Association for Computational Linguistics
DOI: 10.18653/v1/p19-1408

Word Sense Distance in Human Similarity Judgements and Contextualised Word Embeddings

Auteurs: Janosch Haber, Massimo Poesio
Publié dans: Proceedings of the Probability and Meaning Conference (PaM 2020), 2020, Page(s) 128–145
Éditeur: Association for Computational Linguistics

SemEval-2021 Task 12: Learning with Disagreements

Auteurs: Alexandra Uma, Tommaso Fornaciari, Anca Dumitrache, Tristan Miller, Jon Chamberlain, Barbara Plank, Edwin Simpson, Massimo Poesio
Publié dans: Proceedings of the 15th International Workshop on Semantic Evaluation (SemEval-2021), 2021, Page(s) 338–347
Éditeur: Association for Computational Linguistics
DOI: 10.18653/v1/2021.semeval-1.41

A Cluster Ranking Model for Full Anaphora Resolution

Auteurs: Juntao Yu, Alexandra Uma, Massimo Poesio
Publié dans: Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), 2020, Page(s) 11-20
Éditeur: European Language Resources Association (ELRA)

Testing TileAttack with Three Key Audiences

Auteurs: Chris Madge, Massimo Poesio, Udo Kruschwitz, Jon Chamberlain
Publié dans: Proceedings of the 2018 LREC Workshop on Games and Gamification for Natural Language Processing (Games4NLP), 2018, ISBN 979-10-95546-10-8
Éditeur: European Language Resources Association (ELRA)

The Design Of A Clicker Game for Text Labelling

Auteurs: Chris Madge, Richard Bartle, Jon Chamberlain, Udo Kruschwitz, Massimo Poesio
Publié dans: 2019 IEEE Conference on Games (CoG), 2019, Page(s) 1-4, ISBN 978-1-7281-1884-0
Éditeur: IEEE
DOI: 10.1109/cig.2019.8848068

Experiment-Driven Development of a GWAP for Marking Segments in Text

Auteurs: Chris Madge, Jon Chamberlain, Udo Kruschwitz, Massimo Poesio
Publié dans: Extended Abstracts Publication of the Annual Symposium on Computer-Human Interaction in Play - CHI PLAY '17 Extended Abstracts, 2017, Page(s) 397-404, ISBN 9781450351119
Éditeur: ACM Press
DOI: 10.1145/3130859.3131332

Metrics of games-with-a-purpose for NLP applications

Auteurs: Jon Chamberlain, Richard Bartle, Udo Kruschwitz, Chris Madge, Massimo Poesio
Publié dans: Proceedings of the GAMES4NLP 2017 Symposium, 2017
Éditeur: University of Essex

Testing game mechanics in games with a purpose for NLP applications

Auteurs: Chris Madge, Richard Bartle, Jon Chamberlain, Udo Kruschwitz, Massimo Poesio
Publié dans: Proceedings of the GAMES4NLP Symposium, 2017
Éditeur: University of Essex

Anaphora Resolution with the ARRAU Corpus

Auteurs: Massimo Poesio, Yulia Grishina, Varada Kolhatkar, Nafise Moosavi, Ina Roesiger, Adam Roussel, Fabian Simonjetz, Alexandra Uma, Olga Uryupina, Juntao Yu, Heike Zinsmeister
Publié dans: Proceedings of the First Workshop on Computational Models of Reference, Anaphora and Coreference, 2018, Page(s) 11-22
Éditeur: Association for Computational Linguistics
DOI: 10.18653/v1/w18-0702

A Probabilistic Annotation Model for Crowdsourcing Coreference

Auteurs: Silviu Paun, Jon Chamberlain, Udo Kruschwitz, Juntao Yu, Massimo Poesio
Publié dans: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, 2018, Page(s) 1926-1937
Éditeur: Association for Computational Linguistics
DOI: 10.18653/v1/d18-1218

We need to consider disagreement in evaluation

Auteurs: Valerio Basile, Michael Fell, Tommaso Fornaciari, Dirk Hovy, Silviu Paun, Barbara Plank, Massimo Poesio, Alexandra Uma
Publié dans: Proceedings of the 1st Workshop on Benchmarking: Past, Present and Future, 2021, Page(s) 15–21
Éditeur: Association for Computational Linguistics
DOI: 10.18653/v1/2021.bppf-1.3

Speaking Outside the Box: Exploring the Benefits of Unconstrained Input in Crowdsourcing and Citizen Science Platforms

Auteurs: Jon Chamberlain, Udo Kruschwitz and Massimo Poesio
Publié dans: Proceedings of the LREC 2020 Workshop on `Citizen Linguistics in Language Resource Development', 2020, Page(s) 26-34
Éditeur: European Language Resources Association

Neural Mention Detection

Auteurs: Juntao Yu, Bernd Bohnet, Massimo Poesio
Publié dans: Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), 2020, Page(s) 1-10
Éditeur: European Language Resources Association (ELRA)

A case for soft loss functions

Auteurs: Alexandra Uma, Tommaso Fornaciari, Dirk Hovy, Silviu Paun, Barbara Plank, Massimo Poesio
Publié dans: Proceedings of the Eighth AAAI Conference on Human Computation and Crowdsourcing, 2020, Page(s) 173-177
Éditeur: AAAI

A Crowdsourced Corpus of Multiple Judgments and Disagreement on Anaphoric Interpretation

Auteurs: Massimo Poesio, Jon Chamberlain, Silviu Paun, Juntao Yu, Alexandra Uma, Udo Kruschwitz
Publié dans: Proceedings of the 2019 Conference of the North, 2019, Page(s) 1778-1789
Éditeur: Association for Computational Linguistics
DOI: 10.18653/v1/n19-1176

Free the plural: Unrestricted split-antecedent anaphora resolution

Auteurs: Juntao Yu, Nafise Sadat Moosavi, Silviu Paun, Massimo Poesio
Publié dans: Proceedings of the 28th International Conference on Computational Linguistics, 2020, Page(s) 6113–6125
Éditeur: International Committee on Computational Linguistics and Association for Computational Linguistics
DOI: 10.18653/v1/2020.coling-main.538

Monographic books (3)

Proceedings of the Fourth Workshop on Computational Models of Reference, Anaphora and Coreference

Auteurs: Maciej Ogrodniczuk, Sameer Pradhan, Massimo Poesio, Yulia Grishina, Vincent Ng
Publié dans: 2021
Éditeur: Association for Computational Linguistics

Proceedings of the First Workshop on Computational Models of Reference, Anaphora and Coreference

Auteurs: Massimo Poesio, Vincent Ng, Maciej Ogrodniczuk
Publié dans: 2018
Éditeur: Association for Computational Linguistics
DOI: 10.18653/v1/w18-07

Statistical Methods for Annotation Analysis

Auteurs: Silviu Paun, Ron Artstein, Massimo Poesio
Publié dans: Synthesis Lectures on Human Language Technologies, 2022
Éditeur: Morgan Claypool
DOI: 10.2200/s01131ed1v01y202109hlt054

Thesis and dissertations (2)

Making the Most of Crowd Information: Learning and Evaluation in AI tasks with Disagreements

Auteurs: Alexandra Uma
Publié dans: 2021
Éditeur: Queen Mary University of London

Gamifying Language Resource Acquisition

Auteurs: Christopher Madge
Publié dans: 2019
Éditeur: Queen Mary University of London

Peer reviewed articles (5)

Comparing Bayesian Models of Annotation

Auteurs: Silviu Paun, Bob Carpenter, Jon Chamberlain, Dirk Hovy, Udo Kruschwitz, Massimo Poesio
Publié dans: Transactions of the Association for Computational Linguistics, 2018, Page(s) 571-585, ISSN 2307-387X
Éditeur: The MIT Press

Optimising crowdsourcing efficiency: Amplifying human computation with validation

Auteurs: Jon Chamberlain, Udo Kruschwitz, Massimo Poesio
Publié dans: it - Information Technology, Numéro 60/1, 2018, Page(s) 41-49, ISSN 2196-7032
Éditeur: De Gruyter
DOI: 10.1515/itit-2017-0020

Annotating a broad range of anaphoric phenomena, in a variety of genres: the ARRAU Corpus

Auteurs: Olga Uryupina, Ron Artstein, Antonella Bristot, Federica Cavicchio, Francesca Delogu, Kepa J. Rodriguez, Massimo Poesio
Publié dans: Natural Language Engineering, Numéro 26/1, 2020, Page(s) 95-128, ISSN 1351-3249
Éditeur: Cambridge University Press
DOI: 10.1017/s1351324919000056

Scaling and Disagreements: Bias, Noise and Ambiguity

Auteurs: Alexandra Uma, Dina Almanea, Massimo Poesio
Publié dans: Frontiers in Artificial Intelligence, 2022, ISSN 2624-8212
Éditeur: Frontiers

Learning from Disagreement: A Survey

Auteurs: Alexandra N Uma, Tommaso Fornaciari, Dirk Hovy, Silviu Paun, Barbara Plank, Massimo Poesio
Publié dans: Journal of Artificial Intelligence Research, 2021, Page(s) 1385-1470, ISSN 1076-9757
Éditeur: Morgan Kaufmann Publishers, Inc.
DOI: 10.1613/jair.1.12752