Distributed dynamic REpresentations for diAlogue Management

Informations projet

DREAM

N° de convention de subvention: 819455

DOI

10.3030/819455

Projet clôturé

Date de signature de la CE 12 Février 2019

Date de début 1 Septembre 2019

Date de fin 31 Mai 2025

Financé au titre de

• EXCELLENT SCIENCE - European Research Council (ERC)

Coût total

€ 2 000 000,00

Contribution de l’UE

€ 2 000 000,00

Coordonné par

UNIVERSITEIT VAN AMSTERDAM
Netherlands

CORDIS fournit des liens vers les livrables publics et les publications des projets HORIZON.

Les liens vers les livrables et les publications des projets du 7e PC, ainsi que les liens vers certains types de résultats spécifiques tels que les jeux de données et les logiciels, sont récupérés dynamiquement sur OpenAIRE .

Livrables

Open Research Data Pilot (1)

Data Management Plan

The Data Management Plan will specify: - what kind of data will be generated and collected, - where and how it will be stored, and - who will have access to it and under what licence.

Publications

Conference proceedings (40)

Linguistic and Gestural Coordination: Do Learners Converge in Collaborative Dialogue?

Auteurs: Arabella Sinclair, Bertrand Schneider
Publié dans: Proceedings of the 14th International Conference on Educational Data Mining, 2021, ISBN 978-1-7336736-2-4
Éditeur: EDM

Describing Images Fast and Slow: Quantifying and Predicting the Variation in Human Signals during Visuo-Linguistic Processes

Auteurs: Ece Takmaz, Sandro Pezzelle, Raquel Fernández
Publié dans: Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), 2024
Éditeur: Association for Computational Linguistics

Detecting Age-Related Linguistic Patterns in Dialogue: Toward Adaptive Conversational Systems

Auteurs: Lennert Jansen, Arabella Sinclair, Margot J. Goot, Raquel Fernández, Sandro Pezzelle
Publié dans: Proceedings of the Eighth Italian Conference on Computational Linguistics, 2021
Éditeur: Italian Association for Computational Linguistics

Be Different to Be Better! A Benchmark to Leverage the Complementarity of Language and Vision

Auteurs: Sandro Pezzelle, Claudio Greco, Greta Gandolfi, Eleonora Gualdoni, Raffaella Bernardi
Publié dans: Findings of the Association for Computational Linguistics: EMNLP 2020, 2020, Page(s) 2751-2767
Éditeur: Association for Computational Linguistics
DOI: 10.18653/v1/2020.findings-emnlp.248

Towards Pragmatic Production Strategies for Natural Language Generation Tasks

Auteurs: Mario Giulianelli
Publié dans: Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, 2023, Page(s) 7978-7984
Éditeur: Association for Computational Linguistics
DOI: 10.18653/v1/2022.emnlp-main.544

Stop Measuring Calibration When Humans Disagree

Auteurs: Joris Baan, Wilker Aziz, Barbara Plank, Raquel Fernandez
Publié dans: Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, 2023, Page(s) 1892-1915
Éditeur: Association for Computational Linguistics
DOI: 10.18653/v1/2022.emnlp-main.124

GROOViST: A Metric for Grounding Objects in Visual Storytelling

Auteurs: Aditya Surikuchi, Sandro Pezzelle, Raquel Fernández
Publié dans: Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, 2023, Page(s) 3331-3339
Éditeur: Association for Computational Linguistics
DOI: 10.18653/v1/2023.emnlp-main.202

I see what you mean: Co-Speech Gestures for Reference Resolution in Multimodal Dialogue

Auteurs: Esam Ghaleb, Bulat Khaertdinov, Asli Ozyurek, Raquel Fernández
Publié dans: Findings of the Association for Computational Linguistics: ACL 2025, 2025
Éditeur: Association for Computational Linguistics

Analysing Human Strategies of Information Transmission as a Function of Discourse Context

Auteurs: Mario Giulianelli, Raquel Fernández
Publié dans: Proceedings of the 25th Conference on Computational Natural Language Learning, 2021
Éditeur: Association for Computational Linguistics
DOI: 10.18653/v1/2021.conll-1.50

Experiential Semantic Information and Brain Alignment: Are Multimodal Models Better than Language Models?

Auteurs: Anna Bavaresco, Raquel Fernández
Publié dans: Proceedings of the 29th Conference on Computational Natural Language Learning (CoNLL), 2025
Éditeur: Association for Computational Linguistics

Do Not Fire the Linguist: Grammatical Profiles Help Language Models Detect Semantic Change

Auteurs: Mario Giulianelli, Andrey Kutuzov, Lidia Pivovarova
Publié dans: Proceedings of the 3rd Workshop on Computational Approaches to Historical Language Change, 2022, Page(s) 54-67
Éditeur: Association for Computational Linguistics
DOI: 10.18653/v1/2022.lchange-1.6

Attribution and Alignment: Effects of Local Context Repetition on Utterance Production and Comprehension in Dialogue

Auteurs: Aron Molnar, Jaap Jumelet, Mario Giulianelli, Arabella Sinclair
Publié dans: Proceedings of the 27th Conference on Computational Natural Language Learning (CoNLL), 2023, Page(s) 254-273
Éditeur: Association for Computational Linguistics
DOI: 10.18653/v1/2023.conll-1.18

LLMs instead of Human Judges? A Large Scale Empirical Study across 20 NLP Evaluation Tasks

Auteurs: Anna Bavaresco, Raffaella Bernardi, Leonardo Bertolazzi, Desmond Elliott, Raquel Fernández, Albert Gatt, Esam Ghaleb, Mario Giulianelli, Michael Hanna, Alexander Koller, Andre Martins, Philipp Mondorf, Vera Neplenbroek, Sandro Pezzelle, Barbara Plank, David Schlangen, Alessandro Suglia, Aditya K Surikuchi, Ece Takmaz, Alberto Testoni
Publié dans: Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), 2025
Éditeur: Association for Computational Linguistics

Grounded and Ungrounded Referring Expressions in Human Dialogues: Language Mirrors Different Grounding Conditions

Auteurs: Eleonora Gualdoni, Raffaella Bernardi, Sandro Pezzelle, Raquel Fernández
Publié dans: Proceedings of the 7th Italian Conference on Computational Linguistics, 2020
Éditeur: CEUR Workshop Proceedings

AnaLog: Testing Analytical and Deductive Logic Learnability in Language Models

Auteurs: Samuel Ryb, Mario Giulianelli, Arabella Sinclair, Raquel Fernández
Publié dans: Proceedings of the 11th Joint Conference on Lexical and Computational Semantics, 2022, Page(s) 55-68
Éditeur: Association for Computational Linguistics
DOI: 10.18653/v1/2022.starsem-1.5

Analysing cross-speaker convergence through the lens of automatically detected shared linguistic constructions

Auteurs: Ghaleb, E.; Rasenberg, M.; Pouw, W.; Toni, I.; Holler, J.; Özyürek, A.; Fernandez, R.
Publié dans: Proceedings of the 46th Annual Meeting of the Cognitive Science Society (CogSci 2024), 2024
Éditeur: Cognitive Science Society

Refer, Reuse, Reduce: Generating Subsequent References in Visual and Conversational Contexts

Auteurs: Ece Takmaz, Mario Giulianelli, Sandro Pezzelle, Arabella Sinclair, Raquel Fernández
Publié dans: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2020, Page(s) 4350-4368
Éditeur: Association for Computational Linguistics
DOI: 10.18653/v1/2020.emnlp-main.353

Information Value: Measuring Utterance Predictability as Distance from Plausible Alternatives

Auteurs: Mario Giulianelli, Sarenne Wallbridge, Raquel Fernández
Publié dans: Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, 2023, Page(s) 5633-5653
Éditeur: Association for Computational Linguistics
DOI: 10.18653/v1/2023.emnlp-main.343

Is Information Density Uniform in Task-Oriented Dialogues?

Auteurs: Mario Giulianelli, Arabella Sinclair, Raquel Fernández
Publié dans: Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, 2021
Éditeur: Association for Computational Linguistics
DOI: 10.18653/v1/2021.emnlp-main.652

Speaking the Language of Your Listener: Audience-Aware Adaptation via Plug-and-Play Theory of Mind

Auteurs: Ece Takmaz, Nicolo’ Brandizzi, Mario Giulianelli, Sandro Pezzelle, Raquel Fernandez
Publié dans: Findings of the Association for Computational Linguistics: ACL 2023, 2023, Page(s) 4198-4217
Éditeur: Association for Computational Linguistics
DOI: 10.18653/v1/2023.findings-acl.258

Analysing Cross-Speaker Convergence in Face-to-Face Dialogue through the Lens of Automatically Detected Shared Linguistic Constructions

Auteurs: Ghaleb, Esam; Rasenberg, Marlou; Pouw, Wim; Toni, Ivan; Holler, Judith; Ozyurek, Asli; Fernandez, Raquel
Publié dans: Proceedings of the Annual Meeting of the Cognitive Science Society, 2024
Éditeur: Cognitive Science Society

Probing Cross-Modal Representations in Multi-Step Relational Reasoning

Auteurs: Iuliia Parfenova, Desmond Elliott, Raquel Fernández, Sandro Pezzelle
Publié dans: Proceedings of the 6th Workshop on Representation Learning for NLP (RepL4NLP-2021), 2021
Éditeur: Association for Computational Linguistics
DOI: 10.18653/v1/2021.repl4nlp-1.16

Grammatical Profiling for Semantic Change Detection

Auteurs: Mario Giulianelli, Andrey Kutuzov, Lidia Pivovarova
Publié dans: 2021
Éditeur: Association for Computational Linguistics
DOI: 10.18653/v1/2021.conll-1.33

Learning Co-Speech Gesture Representations in Dialogue through Contrastive Learning: An Intrinsic Evaluation

Auteurs: Esam Ghaleb, Bulat Khaertdinov, Marlou Rasenberg, Wim Pouw, Judith Holler, Aslı Özyürek, & Raquel Fernández
Publié dans: ICMI '24: Proceedings of the 26th International Conference on Multimodal Interaction, 2024
Éditeur: Association for Computing Machinery

Construction Coordination in First and Second Language Acquisition

Auteurs: Sinclair, A.J.; Fernández, R.
Publié dans: Proceedings of the 25th Workshop on the Semantics and Pragmatics of Dialogue - Full Papers, 2021
Éditeur: SEMDIAL

Don’t Buy it! Reassessing the Ad Understanding Abilities of Contrastive Multimodal Models

Auteurs: Anna Bavaresco, Alberto Testoni, Raquel Fernández
Publié dans: Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), 2024
Éditeur: Association for Computational Linguistics

Learning Co-Speech Gesture Representations in Dialogue through Contrastive Learning: An Intrinsic Evaluation

Auteurs: Esam Ghaleb, Bulat Khaertdinov, Wim Pouw, Marlou Rasenberg, Judith Holler, Asli Ozyurek, Raquel Fernandez
Publié dans: International Conference on Multimodel Interaction, 2025, Page(s) 274-283
Éditeur: ACM
DOI: 10.1145/3678957.3685707

The BLA Benchmark: Investigating Basic Language Abilities of Pre-Trained Multimodal Models

Auteurs: Xinyi Chen, Raquel Fernández, Sandro Pezzelle
Publié dans: Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, 2023, Page(s) 5817-5830
Éditeur: Association for Computational Linguistics
DOI: 10.18653/v1/2023.emnlp-main.356

Cross-Lingual Consistency of Factual Knowledge in Multilingual Language Models

Auteurs: Jirui Qi, Raquel Fernández, Arianna Bisazza
Publié dans: Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, 2023, Page(s) 10650-10666
Éditeur: Association for Computational Linguistics
DOI: 10.18653/v1/2023.emnlp-main.658

Asking the Right Question at the Right Time: Human and Model Uncertainty Guidance to Ask Clarification Questions

Auteurs: Alberto Testoni, Raquel Fernández
Publié dans: Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), 2024
Éditeur: Association for Computational Linguistics

Interpretable Word Sense Representations via Definition Generation: The Case of Semantic Change Analysis

Auteurs: Mario Giulianelli, Iris Luden, Raquel Fernandez, Andrey Kutuzov
Publié dans: Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2023, Page(s) 3130-3148
Éditeur: Association for Computational Linguistics
DOI: 10.18653/v1/2023.acl-long.176

Team DMG at CMCL 2022 Shared Task: Transformer Adapters for the Multi- and Cross-Lingual Prediction of Human Reading Behavior

Auteurs: Ece Takmaz
Publié dans: Proceedings of the Workshop on Cognitive Modeling and Computational Linguistics, 2022, Page(s) 136-144
Éditeur: Association for Computational Linguistics
DOI: 10.18653/v1/2022.cmcl-1.16

Construction Repetition Reduces Information Rate in Dialogue

Auteurs: Mario Giulianelli, Arabella Sinclair, Raquel Fernández
Publié dans: Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), 2024, Page(s) 665-682
Éditeur: Association for Computational Linguistics
DOI: 10.18653/v1/2022.aacl-main.51

Controllable Text Generation for All Ages: Evaluating a Plug-and-Play Approach to Age-Adapted Dialogue

Auteurs: Lennert Jansen, Štěpán Lars Laichter, Arabella Sinclair, Margot van der Goot, Raquel Fernandez, Sandro Pezzelle
Publié dans: Proceedings of the 2nd Workshop on Natural Language Generation, Evaluation, and Metrics (GEM), 2023, Page(s) 172-188
Éditeur: Association for Computational Linguistics
DOI: 10.18653/v1/2022.gem-1.14

Analysing Lexical Semantic Change with Contextualised Word Representations

Auteurs: Mario Giulianelli, Marco Del Tredici, Raquel Fernández
Publié dans: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 2020, Page(s) 3960-3973
Éditeur: Association for Computational Linguistics
DOI: 10.18653/v1/2020.acl-main.365

Less Descriptive yet Discriminative: Quantifying the Properties of Multimodal Referring Utterances via CLIP

Auteurs: Ece Takmaz, Sandro Pezzelle, Raquel Fernández
Publié dans: Proceedings of the Workshop on Cognitive Modeling and Computational Linguistics, 2022, Page(s) 36-42
Éditeur: Association for Computational Linguistics
DOI: 10.18653/v1/2022.cmcl-1.4

What Comes Next? Evaluating Uncertainty in Neural Text Generators Against Human Production Variability

Auteurs: Mario Giulianelli, Joris Baan, Wilker Aziz, Raquel Fernández, Barbara Plank
Publié dans: Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, 2023, Page(s) 14349-14371
Éditeur: Association for Computational Linguistics
DOI: 10.18653/v1/2023.emnlp-main.887

Naming, Describing, and Quantifying Visual Objects in Humans and LLMs

Auteurs: Alberto Testoni, Juell Sprott, Sandro Pezzelle
Publié dans: Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), 2024
Éditeur: Association for Computational Linguistics

Construction coordination in first and second language acquisition

Auteurs: Arabella Sinclair, Raquel Fernández
Publié dans: Proceedings of the 25th Workshop on the Semantics and Pragmatics of Dialogue, 2021
Éditeur: SemDial

Generating Image Descriptions via Sequential Cross-Modal Alignment Guided by Human Gaze

Auteurs: Ece Takmaz, Sandro Pezzelle, Lisa Beinborn, Raquel Fernández
Publié dans: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2020, Page(s) 4664-4677
Éditeur: Association for Computational Linguistics
DOI: 10.18653/v1/2020.emnlp-main.377

Peer reviewed articles (8)

Structural Persistence in Language Models: Priming as a Window into Abstract Language Representations

Auteurs: Arabella Sinclair, Jaap Jumelet, Willem Zuidema, Raquel Fernández
Publié dans: Transactions of the Association for Computational Linguistics, Numéro 10, 2022, Page(s) 1031-1050, ISSN 2307-387X
Éditeur: MIT Press
DOI: 10.1162/tacl_a_00504

Linguistic issues behind visual question answering

Auteurs: Raffaella Bernardi, Sandro Pezzelle
Publié dans: Language and Linguistics Compass, 2021, ISSN 1749-818X
Éditeur: Blackwell
DOI: 10.1111/lnc3.12417

Alignment of code switching varies with proficiency in second language learning dialogue

Auteurs: Arabella J. Sinclair, Raquel Fernández
Publié dans: System, Numéro 113, 2024, Page(s) 102952, ISSN 0346-251X
Éditeur: Pergamon Press Ltd.
DOI: 10.1016/j.system.2022.102952

The Efficiency of Question‐Asking Strategies in a Real‐World Visual Search Task

Auteurs: Alberto Testoni, Raffaella Bernardi, Azzurra Ruggeri
Publié dans: Cognitive Science, Numéro 47, 2024, ISSN 0364-0213
Éditeur: Lawrence Erlbaum Associates Inc.
DOI: 10.1111/cogs.13396

Semantic Adaptation to the Interpretation of Gradable Adjectives via Active Linguistic Interaction

Auteurs: Sandro Pezzelle, Raquel Fernández
Publié dans: Cognitive Science, Numéro 47, 2023, ISSN 0364-0213
Éditeur: Lawrence Erlbaum Associates Inc.
DOI: 10.1111/cogs.13248

A taxonomy and review of generalization research in NLP

Auteurs: Hupkes, Dieuwke; Giulianelli, Mario; Dankers, Verna; Artetxe, Mikel; Elazar, Yanai; Pimentel, Tiago; Christodoulopoulos, Christos; Lasri, Karim; Saphra, Naomi; Sinclair, Arabella; Ulmer, Dennis Thomas; Schottmann, Florian; Batsuren, Khuyagbaatar; Sun, Kaiser; Sinha, Koustouv; Khalatbari, Leila; Ryskina, Maria; Cotterell, Ryan; Jin, Zhijing
Publié dans: Nature Machine Intelligence, 2023, Page(s) 1161–1174, ISSN 2522-5839
Éditeur: Springer Nature

Beyond Perplexity: Examining Temporal Generalization in Large Language Models via Definition Generation

Auteurs: Iris Luden, Mario Giulianelli, Raquel Fernández
Publié dans: Computational Linguistics in the Netherlands Journal, 2024, ISSN 2211-4009
Éditeur: CLIN/OJS

Word Representation Learning in Multimodal Pre-Trained Transformers: An Intrinsic Evaluation

Auteurs: Sandro Pezzelle, Ece Takmaz, Raquel Fernández
Publié dans: Transactions of the Association for Computational Linguistics, 2021, ISSN 2307-387X
Éditeur: Association for Computational Linguistics
DOI: 10.1162/tacl_a_00443

Book chapters (1)

Grounded and Ungrounded Referring Expressions in Human Dialogues: Language Mirrors Different Grounding Conditions

Auteurs: Eleonora Gualdoni, Raffaella Bernardi, Raquel Fernández, Sandro Pezzelle
Publié dans: Proceedings of the Seventh Italian Conference on Computational Linguistics CLiC-it 2020, 2025, Page(s) 223-229
Éditeur: Accademia University Press
DOI: 10.4000/books.aaccademia.8600

Thesis and dissertations (2)

Neural models of language use

Auteurs: Giulianelli, M.
Publié dans: 2023
Éditeur: University of Amsterdam

Visual and linguistic processes in deep neural networks

Auteurs: Takmaz, E.K.
Publié dans: 2024
Éditeur: University of Amsterdam

Other (3)

Modelling Multimodal Integration in Human Concept Processing with Vision-Language Models.

Auteurs: Anna Bavaresco, Marianne Heer Kloots, Sandro Pezzelle, & Raquel Fernández
Publié dans: 2025
Éditeur: Arxiv

Incremental Alternative Sampling as a Lens into the Temporal and Representational Resolution of Linguistic Prediction.

Auteurs: Mario Giulianelli, Sarenne Wallbridge, Ryan Cotterell, & Raquel Fernández
Publié dans: 2025
Éditeur: PsyArXiv

RACQUET: Unveiling the Dangers of Overlooked Referential Ambiguity in Visual LLMs.

Auteurs: Alberto Testoni, Barbara Plank, & Raquel Fernández
Publié dans: 2025
Éditeur: Arxiv