Disagreements and Language Interpretation

Anaphoric zero pronoun identification: A multilingual approach

Author(s): Abdulrahman Aloraini, Massimo Poesio
Published in: Proceedings of the Third Workshop on Computational Models of Reference, Anaphora and Coreference, 2020, Page(s) 22–32
Publisher: Association for Computational Linguistics

Coreference Resolution for the Biomedical Domain: A Survey

Author(s): Pengcheng Lu, Massimo Poesio
Published in: Proceedings of the Fourth Workshop on Computational Models of Reference, Anaphora and Coreference (CRAC), 2021, Page(s) 12–23
Publisher: Association for Computational Linguistics
DOI: 10.18653/v1/2021.crac-1.2

Beyond black & white: Leveraging annotator disagreement via soft-label multi-task learning

Author(s): Tommaso Fornaciari, Alexandra Uma, Silviu Paun, Barbara Plank, Dirk Hovy, Massimo Poesio
Published in: Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2021, Page(s) 2591–2597
Publisher: Association for Computational Linguistics
DOI: 10.18653/v1/2021.naacl-main.204

The CODI-CRAC 2021 Shared Task on Anaphora, Bridging, and Discourse Deixis in Dialogue

Author(s): Sopan Khosla, Juntao Yu, Ramesh Manuvinakurike, Vincent Ng, Massimo Poesio, Michael Strube, Carolyn Rosé
Published in: Proceedings of the CODI-CRAC 2021 Shared Task on Anaphora, Bridging, and Discourse Deixis in Dialogue, 2021, Page(s) 1–15
Publisher: Association for Computational Linguistics
DOI: 10.18653/v1/2021.codi-sharedtask.1

Stay together: A system for single and split-antecedent anaphora resolution

Author(s): Juntao Yu, Nafise Sadat Moosavi, Silviu Paun, Massimo Poesio
Published in: Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2021, Page(s) 4174–4184
Publisher: Association for Computational Linguistics
DOI: 10.18653/v1/2021.naacl-main.329

Making text annotation fun with a clicker game

Author(s): Chris Madge, Richard Bartle, Jon Chamberlain, Udo Kruschwitz, Massimo Poesio
Published in: Proceedings of the 14th International Conference on the Foundations of Digital Games, 2019, Page(s) 1-6, ISBN 9781450372176
Publisher: ACM
DOI: 10.1145/3337722.3341869

Multitask Learning-Based Neural Bridging Reference Resolution

Author(s): Juntao Yu, Massimo Poesio
Published in: Proceedings of the 28th International Conference on Computational Linguistics (COLING), 2020, Page(s) 3534–3546
Publisher: International Committee on Computational Linguistics and Association for Computational Linguistics
DOI: 10.18653/v1/2020.coling-main.315

Crowdsourcing and Aggregating Nested Markable Annotations

Author(s): Chris Madge, Juntao Yu, Jon Chamberlain, Udo Kruschwitz, Silviu Paun, Massimo Poesio
Published in: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019, Page(s) 797-807
Publisher: Association for Computational Linguistics
DOI: 10.18653/v1/p19-1077

Cipher: A Prototype Game-with-a-Purpose for Detecting Errors in Text

Author(s): Liang Xu, Jon Chamberlain
Published in: Workshop on Games and Natural Language Processing (Games and NLP 2020), 2020, Page(s) 17–25
Publisher: European Language Resources Association

Classification of Low-Agreement Pronouns Through Collaborative Dialogue: A Proof of Concept

Author(s): Janosch Haber, Massimo Poesio
Published in: Proceedings of the 24th Workshop on the Semantics and Pragmatics of Dialogue - Full Papers, 2020
Publisher: SEMDIAL

Assessing polyseme sense similarity through co-predication acceptability and contextualised embedding distance

Author(s): Janosch Haber, Massimo Poesio
Published in: Proceedings of the Ninth Joint Conference on Lexical and Computational Semantics, 2020, Page(s) 114–124
Publisher: Association for Computational Linguistics

https://aclanthology.org/2020.nlp4call-1.3/

Author(s): Etiene da Cruz Dalcol, Massimo Poesio
Published in: Proceedings of the 9th Workshop on NLP for Computer Assisted Language Learning, 2020, Page(s) 21–36
Publisher: LiU Electronic Press

Patterns of Polysemy and Homonymy in Contextualised Language Models

Author(s): Janosch Haber, Massimo Poesio
Published in: Findings of the Association for Computational Linguistics: EMNLP 2021, 2021, Page(s) 2663–2676
Publisher: Association for Computational Linguistics
DOI: 10.18653/v1/2021.findings-emnlp.226

Wormingo - a 'true gamification' approach to anaphoric annotation

Author(s): Doruk Kicikoglu, Richard Bartle, Jon Chamberlain, Massimo Poesio
Published in: Proceedings of the 14th International Conference on the Foundations of Digital Games, 2019, Page(s) 1-7, ISBN 9781450372176
Publisher: ACM
DOI: 10.1145/3337722.3341868

Named Entity Recognition as Dependency Parsing

Author(s): Juntao Yu, Bernd Bohnet, Massimo Poesio
Published in: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 2020, Page(s) 6470-6476
Publisher: Association for Computational Linguistics
DOI: 10.18653/v1/2020.acl-main.577

Progression in a Language Annotation Game with a Purpose

Author(s): Chris Madge, Juntao Yu, Jon Chamberlain, Udo Kruschwitz, Silviu Paun, Massimo Poesio
Published in: HCOMP-19: Proceedings of the AAAI Human Computation Conference, 2019
Publisher: AAAI

Aggregation Driven Progression for GWAPs

Author(s): Doruk Kicikoglu,Richard Bartle, Silviu Paun, Jon Chamberlain, Massimo Poesio
Published in: Proceedings of the LREC 2020 Workshop Games and Natural Language Processing, 2020, Page(s) 79-84
Publisher: European Language Resources Association (ELRA)

Incremental Game Mechanics Applied to Text Annotation

Author(s): Christopher Madge, Richard Bartle, Jon Chamberlain, Udo Kruschwitz, Massimo Poesio
Published in: Proceedings of the Annual Symposium on Computer-Human Interaction in Play, 2019, Page(s) 545-558, ISBN 9781450366885
Publisher: ACM
DOI: 10.1145/3311350.3347184

Neural coreference resolution for Arabic

Author(s): Abdulrahman Aloraini, Juntao Yu, Massimo Poesio
Published in: Proceedings of the Third Workshop on Computational Models of Reference, Anaphora and Coreference, 2020, Page(s) 99–110
Publisher: Association for Computational Linguistics

Using Automatically Extracted Minimum Spans to Disentangle Coreference Evaluation from Boundary Detection

Author(s): Nafise Sadat Moosavi, Leo Born, Massimo Poesio, Michael Strube
Published in: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019, Page(s) 4168-4178
Publisher: Association for Computational Linguistics
DOI: 10.18653/v1/p19-1408

Word Sense Distance in Human Similarity Judgements and Contextualised Word Embeddings

Author(s): Janosch Haber, Massimo Poesio
Published in: Proceedings of the Probability and Meaning Conference (PaM 2020), 2020, Page(s) 128–145
Publisher: Association for Computational Linguistics

SemEval-2021 Task 12: Learning with Disagreements

Author(s): Alexandra Uma, Tommaso Fornaciari, Anca Dumitrache, Tristan Miller, Jon Chamberlain, Barbara Plank, Edwin Simpson, Massimo Poesio
Published in: Proceedings of the 15th International Workshop on Semantic Evaluation (SemEval-2021), 2021, Page(s) 338–347
Publisher: Association for Computational Linguistics
DOI: 10.18653/v1/2021.semeval-1.41

A Cluster Ranking Model for Full Anaphora Resolution

Author(s): Juntao Yu, Alexandra Uma, Massimo Poesio
Published in: Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), 2020, Page(s) 11-20
Publisher: European Language Resources Association (ELRA)

Testing TileAttack with Three Key Audiences

Author(s): Chris Madge, Massimo Poesio, Udo Kruschwitz, Jon Chamberlain
Published in: Proceedings of the 2018 LREC Workshop on Games and Gamification for Natural Language Processing (Games4NLP), 2018, ISBN 979-10-95546-10-8
Publisher: European Language Resources Association (ELRA)

The Design Of A Clicker Game for Text Labelling

Author(s): Chris Madge, Richard Bartle, Jon Chamberlain, Udo Kruschwitz, Massimo Poesio
Published in: 2019 IEEE Conference on Games (CoG), 2019, Page(s) 1-4, ISBN 978-1-7281-1884-0
Publisher: IEEE
DOI: 10.1109/cig.2019.8848068

Experiment-Driven Development of a GWAP for Marking Segments in Text

Author(s): Chris Madge, Jon Chamberlain, Udo Kruschwitz, Massimo Poesio
Published in: Extended Abstracts Publication of the Annual Symposium on Computer-Human Interaction in Play - CHI PLAY '17 Extended Abstracts, 2017, Page(s) 397-404, ISBN 9781450351119
Publisher: ACM Press
DOI: 10.1145/3130859.3131332

Metrics of games-with-a-purpose for NLP applications

Author(s): Jon Chamberlain, Richard Bartle, Udo Kruschwitz, Chris Madge, Massimo Poesio
Published in: Proceedings of the GAMES4NLP 2017 Symposium, 2017
Publisher: University of Essex

Testing game mechanics in games with a purpose for NLP applications

Author(s): Chris Madge, Richard Bartle, Jon Chamberlain, Udo Kruschwitz, Massimo Poesio
Published in: Proceedings of the GAMES4NLP Symposium, 2017
Publisher: University of Essex

Anaphora Resolution with the ARRAU Corpus

Author(s): Massimo Poesio, Yulia Grishina, Varada Kolhatkar, Nafise Moosavi, Ina Roesiger, Adam Roussel, Fabian Simonjetz, Alexandra Uma, Olga Uryupina, Juntao Yu, Heike Zinsmeister
Published in: Proceedings of the First Workshop on Computational Models of Reference, Anaphora and Coreference, 2018, Page(s) 11-22
Publisher: Association for Computational Linguistics
DOI: 10.18653/v1/w18-0702

A Probabilistic Annotation Model for Crowdsourcing Coreference

Author(s): Silviu Paun, Jon Chamberlain, Udo Kruschwitz, Juntao Yu, Massimo Poesio
Published in: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, 2018, Page(s) 1926-1937
Publisher: Association for Computational Linguistics
DOI: 10.18653/v1/d18-1218

We need to consider disagreement in evaluation

Author(s): Valerio Basile, Michael Fell, Tommaso Fornaciari, Dirk Hovy, Silviu Paun, Barbara Plank, Massimo Poesio, Alexandra Uma
Published in: Proceedings of the 1st Workshop on Benchmarking: Past, Present and Future, 2021, Page(s) 15–21
Publisher: Association for Computational Linguistics
DOI: 10.18653/v1/2021.bppf-1.3

Speaking Outside the Box: Exploring the Benefits of Unconstrained Input in Crowdsourcing and Citizen Science Platforms

Author(s): Jon Chamberlain, Udo Kruschwitz and Massimo Poesio
Published in: Proceedings of the LREC 2020 Workshop on `Citizen Linguistics in Language Resource Development', 2020, Page(s) 26-34
Publisher: European Language Resources Association

Neural Mention Detection

Author(s): Juntao Yu, Bernd Bohnet, Massimo Poesio
Published in: Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), 2020, Page(s) 1-10
Publisher: European Language Resources Association (ELRA)

A case for soft loss functions

Author(s): Alexandra Uma, Tommaso Fornaciari, Dirk Hovy, Silviu Paun, Barbara Plank, Massimo Poesio
Published in: Proceedings of the Eighth AAAI Conference on Human Computation and Crowdsourcing, 2020, Page(s) 173-177
Publisher: AAAI

A Crowdsourced Corpus of Multiple Judgments and Disagreement on Anaphoric Interpretation

Author(s): Massimo Poesio, Jon Chamberlain, Silviu Paun, Juntao Yu, Alexandra Uma, Udo Kruschwitz
Published in: Proceedings of the 2019 Conference of the North, 2019, Page(s) 1778-1789
Publisher: Association for Computational Linguistics
DOI: 10.18653/v1/n19-1176

Free the plural: Unrestricted split-antecedent anaphora resolution

Author(s): Juntao Yu, Nafise Sadat Moosavi, Silviu Paun, Massimo Poesio
Published in: Proceedings of the 28th International Conference on Computational Linguistics, 2020, Page(s) 6113–6125
Publisher: International Committee on Computational Linguistics and Association for Computational Linguistics
DOI: 10.18653/v1/2020.coling-main.538

Proceedings of the Fourth Workshop on Computational Models of Reference, Anaphora and Coreference

Author(s): Maciej Ogrodniczuk, Sameer Pradhan, Massimo Poesio, Yulia Grishina, Vincent Ng
Published in: 2021
Publisher: Association for Computational Linguistics

Proceedings of the First Workshop on Computational Models of Reference, Anaphora and Coreference

Author(s): Massimo Poesio, Vincent Ng, Maciej Ogrodniczuk
Published in: 2018
Publisher: Association for Computational Linguistics
DOI: 10.18653/v1/w18-07

Statistical Methods for Annotation Analysis

Author(s): Silviu Paun, Ron Artstein, Massimo Poesio
Published in: Synthesis Lectures on Human Language Technologies, 2022
Publisher: Morgan Claypool
DOI: 10.2200/s01131ed1v01y202109hlt054

Making the Most of Crowd Information: Learning and Evaluation in AI tasks with Disagreements

Author(s): Alexandra Uma
Published in: 2021
Publisher: Queen Mary University of London

Gamifying Language Resource Acquisition

Author(s): Christopher Madge
Published in: 2019
Publisher: Queen Mary University of London

Comparing Bayesian Models of Annotation

Author(s): Silviu Paun, Bob Carpenter, Jon Chamberlain, Dirk Hovy, Udo Kruschwitz, Massimo Poesio
Published in: Transactions of the Association for Computational Linguistics, 2018, Page(s) 571-585, ISSN 2307-387X
Publisher: The MIT Press

Optimising crowdsourcing efficiency: Amplifying human computation with validation

Author(s): Jon Chamberlain, Udo Kruschwitz, Massimo Poesio
Published in: it - Information Technology, Issue 60/1, 2018, Page(s) 41-49, ISSN 2196-7032
Publisher: De Gruyter
DOI: 10.1515/itit-2017-0020

Annotating a broad range of anaphoric phenomena, in a variety of genres: the ARRAU Corpus

Author(s): Olga Uryupina, Ron Artstein, Antonella Bristot, Federica Cavicchio, Francesca Delogu, Kepa J. Rodriguez, Massimo Poesio
Published in: Natural Language Engineering, Issue 26/1, 2020, Page(s) 95-128, ISSN 1351-3249
Publisher: Cambridge University Press
DOI: 10.1017/s1351324919000056

Scaling and Disagreements: Bias, Noise and Ambiguity

Author(s): Alexandra Uma, Dina Almanea, Massimo Poesio
Published in: Frontiers in Artificial Intelligence, 2022, ISSN 2624-8212
Publisher: Frontiers

Learning from Disagreement: A Survey

Author(s): Alexandra N Uma, Tommaso Fornaciari, Dirk Hovy, Silviu Paun, Barbara Plank, Massimo Poesio
Published in: Journal of Artificial Intelligence Research, 2021, Page(s) 1385-1470, ISSN 1076-9757
Publisher: Morgan Kaufmann Publishers, Inc.
DOI: 10.1613/jair.1.12752

Publications

Share this page Share this page on social networks

Download Download the content of the page