Distributed dynamic REpresentations for diAlogue Management

Informazioni relative al progetto

DREAM

ID dell’accordo di sovvenzione: 819455

DOI

10.3030/819455

Progetto chiuso

Data della firma CE 12 Febbraio 2019

Data di avvio 1 Settembre 2019

Data di completamento 31 Maggio 2025

Finanziato da

• EXCELLENT SCIENCE - European Research Council (ERC)

Costo totale

€ 2 000 000,00

Contributo UE

€ 2 000 000,00

Coordinato da

UNIVERSITEIT VAN AMSTERDAM
Netherlands

CORDIS fornisce collegamenti ai risultati finali pubblici e alle pubblicazioni dei progetti ORIZZONTE.

I link ai risultati e alle pubblicazioni dei progetti del 7° PQ, così come i link ad alcuni tipi di risultati specifici come dataset e software, sono recuperati dinamicamente da .OpenAIRE .

Risultati finali

Open Research Data Pilot (1)

Data Management Plan

The Data Management Plan will specify: - what kind of data will be generated and collected, - where and how it will be stored, and - who will have access to it and under what licence.

Pubblicazioni

Conference proceedings (40)

Linguistic and Gestural Coordination: Do Learners Converge in Collaborative Dialogue?

Autori: Arabella Sinclair, Bertrand Schneider
Pubblicato in: Proceedings of the 14th International Conference on Educational Data Mining, 2021, ISBN 978-1-7336736-2-4
Editore: EDM

Describing Images Fast and Slow: Quantifying and Predicting the Variation in Human Signals during Visuo-Linguistic Processes

Autori: Ece Takmaz, Sandro Pezzelle, Raquel Fernández
Pubblicato in: Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), 2024
Editore: Association for Computational Linguistics

Detecting Age-Related Linguistic Patterns in Dialogue: Toward Adaptive Conversational Systems

Autori: Lennert Jansen, Arabella Sinclair, Margot J. Goot, Raquel Fernández, Sandro Pezzelle
Pubblicato in: Proceedings of the Eighth Italian Conference on Computational Linguistics, 2021
Editore: Italian Association for Computational Linguistics

Be Different to Be Better! A Benchmark to Leverage the Complementarity of Language and Vision

Autori: Sandro Pezzelle, Claudio Greco, Greta Gandolfi, Eleonora Gualdoni, Raffaella Bernardi
Pubblicato in: Findings of the Association for Computational Linguistics: EMNLP 2020, 2020, Pagina/e 2751-2767
Editore: Association for Computational Linguistics
DOI: 10.18653/v1/2020.findings-emnlp.248

Towards Pragmatic Production Strategies for Natural Language Generation Tasks

Autori: Mario Giulianelli
Pubblicato in: Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, 2023, Pagina/e 7978-7984
Editore: Association for Computational Linguistics
DOI: 10.18653/v1/2022.emnlp-main.544

Stop Measuring Calibration When Humans Disagree

Autori: Joris Baan, Wilker Aziz, Barbara Plank, Raquel Fernandez
Pubblicato in: Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, 2023, Pagina/e 1892-1915
Editore: Association for Computational Linguistics
DOI: 10.18653/v1/2022.emnlp-main.124

GROOViST: A Metric for Grounding Objects in Visual Storytelling

Autori: Aditya Surikuchi, Sandro Pezzelle, Raquel Fernández
Pubblicato in: Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, 2023, Pagina/e 3331-3339
Editore: Association for Computational Linguistics
DOI: 10.18653/v1/2023.emnlp-main.202

I see what you mean: Co-Speech Gestures for Reference Resolution in Multimodal Dialogue

Autori: Esam Ghaleb, Bulat Khaertdinov, Asli Ozyurek, Raquel Fernández
Pubblicato in: Findings of the Association for Computational Linguistics: ACL 2025, 2025
Editore: Association for Computational Linguistics

Analysing Human Strategies of Information Transmission as a Function of Discourse Context

Autori: Mario Giulianelli, Raquel Fernández
Pubblicato in: Proceedings of the 25th Conference on Computational Natural Language Learning, 2021
Editore: Association for Computational Linguistics
DOI: 10.18653/v1/2021.conll-1.50

Experiential Semantic Information and Brain Alignment: Are Multimodal Models Better than Language Models?

Autori: Anna Bavaresco, Raquel Fernández
Pubblicato in: Proceedings of the 29th Conference on Computational Natural Language Learning (CoNLL), 2025
Editore: Association for Computational Linguistics

Do Not Fire the Linguist: Grammatical Profiles Help Language Models Detect Semantic Change

Autori: Mario Giulianelli, Andrey Kutuzov, Lidia Pivovarova
Pubblicato in: Proceedings of the 3rd Workshop on Computational Approaches to Historical Language Change, 2022, Pagina/e 54-67
Editore: Association for Computational Linguistics
DOI: 10.18653/v1/2022.lchange-1.6

Attribution and Alignment: Effects of Local Context Repetition on Utterance Production and Comprehension in Dialogue

Autori: Aron Molnar, Jaap Jumelet, Mario Giulianelli, Arabella Sinclair
Pubblicato in: Proceedings of the 27th Conference on Computational Natural Language Learning (CoNLL), 2023, Pagina/e 254-273
Editore: Association for Computational Linguistics
DOI: 10.18653/v1/2023.conll-1.18

LLMs instead of Human Judges? A Large Scale Empirical Study across 20 NLP Evaluation Tasks

Autori: Anna Bavaresco, Raffaella Bernardi, Leonardo Bertolazzi, Desmond Elliott, Raquel Fernández, Albert Gatt, Esam Ghaleb, Mario Giulianelli, Michael Hanna, Alexander Koller, Andre Martins, Philipp Mondorf, Vera Neplenbroek, Sandro Pezzelle, Barbara Plank, David Schlangen, Alessandro Suglia, Aditya K Surikuchi, Ece Takmaz, Alberto Testoni
Pubblicato in: Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), 2025
Editore: Association for Computational Linguistics

Grounded and Ungrounded Referring Expressions in Human Dialogues: Language Mirrors Different Grounding Conditions

Autori: Eleonora Gualdoni, Raffaella Bernardi, Sandro Pezzelle, Raquel Fernández
Pubblicato in: Proceedings of the 7th Italian Conference on Computational Linguistics, 2020
Editore: CEUR Workshop Proceedings

AnaLog: Testing Analytical and Deductive Logic Learnability in Language Models

Autori: Samuel Ryb, Mario Giulianelli, Arabella Sinclair, Raquel Fernández
Pubblicato in: Proceedings of the 11th Joint Conference on Lexical and Computational Semantics, 2022, Pagina/e 55-68
Editore: Association for Computational Linguistics
DOI: 10.18653/v1/2022.starsem-1.5

Analysing cross-speaker convergence through the lens of automatically detected shared linguistic constructions

Autori: Ghaleb, E.; Rasenberg, M.; Pouw, W.; Toni, I.; Holler, J.; Özyürek, A.; Fernandez, R.
Pubblicato in: Proceedings of the 46th Annual Meeting of the Cognitive Science Society (CogSci 2024), 2024
Editore: Cognitive Science Society

Refer, Reuse, Reduce: Generating Subsequent References in Visual and Conversational Contexts

Autori: Ece Takmaz, Mario Giulianelli, Sandro Pezzelle, Arabella Sinclair, Raquel Fernández
Pubblicato in: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2020, Pagina/e 4350-4368
Editore: Association for Computational Linguistics
DOI: 10.18653/v1/2020.emnlp-main.353

Information Value: Measuring Utterance Predictability as Distance from Plausible Alternatives

Autori: Mario Giulianelli, Sarenne Wallbridge, Raquel Fernández
Pubblicato in: Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, 2023, Pagina/e 5633-5653
Editore: Association for Computational Linguistics
DOI: 10.18653/v1/2023.emnlp-main.343

Is Information Density Uniform in Task-Oriented Dialogues?

Autori: Mario Giulianelli, Arabella Sinclair, Raquel Fernández
Pubblicato in: Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, 2021
Editore: Association for Computational Linguistics
DOI: 10.18653/v1/2021.emnlp-main.652

Speaking the Language of Your Listener: Audience-Aware Adaptation via Plug-and-Play Theory of Mind

Autori: Ece Takmaz, Nicolo’ Brandizzi, Mario Giulianelli, Sandro Pezzelle, Raquel Fernandez
Pubblicato in: Findings of the Association for Computational Linguistics: ACL 2023, 2023, Pagina/e 4198-4217
Editore: Association for Computational Linguistics
DOI: 10.18653/v1/2023.findings-acl.258

Analysing Cross-Speaker Convergence in Face-to-Face Dialogue through the Lens of Automatically Detected Shared Linguistic Constructions

Autori: Ghaleb, Esam; Rasenberg, Marlou; Pouw, Wim; Toni, Ivan; Holler, Judith; Ozyurek, Asli; Fernandez, Raquel
Pubblicato in: Proceedings of the Annual Meeting of the Cognitive Science Society, 2024
Editore: Cognitive Science Society

Probing Cross-Modal Representations in Multi-Step Relational Reasoning

Autori: Iuliia Parfenova, Desmond Elliott, Raquel Fernández, Sandro Pezzelle
Pubblicato in: Proceedings of the 6th Workshop on Representation Learning for NLP (RepL4NLP-2021), 2021
Editore: Association for Computational Linguistics
DOI: 10.18653/v1/2021.repl4nlp-1.16

Grammatical Profiling for Semantic Change Detection

Autori: Mario Giulianelli, Andrey Kutuzov, Lidia Pivovarova
Pubblicato in: 2021
Editore: Association for Computational Linguistics
DOI: 10.18653/v1/2021.conll-1.33

Learning Co-Speech Gesture Representations in Dialogue through Contrastive Learning: An Intrinsic Evaluation

Autori: Esam Ghaleb, Bulat Khaertdinov, Marlou Rasenberg, Wim Pouw, Judith Holler, Aslı Özyürek, & Raquel Fernández
Pubblicato in: ICMI '24: Proceedings of the 26th International Conference on Multimodal Interaction, 2024
Editore: Association for Computing Machinery

Construction Coordination in First and Second Language Acquisition

Autori: Sinclair, A.J.; Fernández, R.
Pubblicato in: Proceedings of the 25th Workshop on the Semantics and Pragmatics of Dialogue - Full Papers, 2021
Editore: SEMDIAL

Don’t Buy it! Reassessing the Ad Understanding Abilities of Contrastive Multimodal Models

Autori: Anna Bavaresco, Alberto Testoni, Raquel Fernández
Pubblicato in: Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), 2024
Editore: Association for Computational Linguistics

Learning Co-Speech Gesture Representations in Dialogue through Contrastive Learning: An Intrinsic Evaluation

Autori: Esam Ghaleb, Bulat Khaertdinov, Wim Pouw, Marlou Rasenberg, Judith Holler, Asli Ozyurek, Raquel Fernandez
Pubblicato in: International Conference on Multimodel Interaction, 2025, Pagina/e 274-283
Editore: ACM
DOI: 10.1145/3678957.3685707

The BLA Benchmark: Investigating Basic Language Abilities of Pre-Trained Multimodal Models

Autori: Xinyi Chen, Raquel Fernández, Sandro Pezzelle
Pubblicato in: Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, 2023, Pagina/e 5817-5830
Editore: Association for Computational Linguistics
DOI: 10.18653/v1/2023.emnlp-main.356

Cross-Lingual Consistency of Factual Knowledge in Multilingual Language Models

Autori: Jirui Qi, Raquel Fernández, Arianna Bisazza
Pubblicato in: Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, 2023, Pagina/e 10650-10666
Editore: Association for Computational Linguistics
DOI: 10.18653/v1/2023.emnlp-main.658

Asking the Right Question at the Right Time: Human and Model Uncertainty Guidance to Ask Clarification Questions

Autori: Alberto Testoni, Raquel Fernández
Pubblicato in: Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), 2024
Editore: Association for Computational Linguistics

Interpretable Word Sense Representations via Definition Generation: The Case of Semantic Change Analysis

Autori: Mario Giulianelli, Iris Luden, Raquel Fernandez, Andrey Kutuzov
Pubblicato in: Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2023, Pagina/e 3130-3148
Editore: Association for Computational Linguistics
DOI: 10.18653/v1/2023.acl-long.176

Team DMG at CMCL 2022 Shared Task: Transformer Adapters for the Multi- and Cross-Lingual Prediction of Human Reading Behavior

Autori: Ece Takmaz
Pubblicato in: Proceedings of the Workshop on Cognitive Modeling and Computational Linguistics, 2022, Pagina/e 136-144
Editore: Association for Computational Linguistics
DOI: 10.18653/v1/2022.cmcl-1.16

Construction Repetition Reduces Information Rate in Dialogue

Autori: Mario Giulianelli, Arabella Sinclair, Raquel Fernández
Pubblicato in: Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), 2024, Pagina/e 665-682
Editore: Association for Computational Linguistics
DOI: 10.18653/v1/2022.aacl-main.51

Controllable Text Generation for All Ages: Evaluating a Plug-and-Play Approach to Age-Adapted Dialogue

Autori: Lennert Jansen, Štěpán Lars Laichter, Arabella Sinclair, Margot van der Goot, Raquel Fernandez, Sandro Pezzelle
Pubblicato in: Proceedings of the 2nd Workshop on Natural Language Generation, Evaluation, and Metrics (GEM), 2023, Pagina/e 172-188
Editore: Association for Computational Linguistics
DOI: 10.18653/v1/2022.gem-1.14

Analysing Lexical Semantic Change with Contextualised Word Representations

Autori: Mario Giulianelli, Marco Del Tredici, Raquel Fernández
Pubblicato in: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 2020, Pagina/e 3960-3973
Editore: Association for Computational Linguistics
DOI: 10.18653/v1/2020.acl-main.365

Less Descriptive yet Discriminative: Quantifying the Properties of Multimodal Referring Utterances via CLIP

Autori: Ece Takmaz, Sandro Pezzelle, Raquel Fernández
Pubblicato in: Proceedings of the Workshop on Cognitive Modeling and Computational Linguistics, 2022, Pagina/e 36-42
Editore: Association for Computational Linguistics
DOI: 10.18653/v1/2022.cmcl-1.4

What Comes Next? Evaluating Uncertainty in Neural Text Generators Against Human Production Variability

Autori: Mario Giulianelli, Joris Baan, Wilker Aziz, Raquel Fernández, Barbara Plank
Pubblicato in: Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, 2023, Pagina/e 14349-14371
Editore: Association for Computational Linguistics
DOI: 10.18653/v1/2023.emnlp-main.887

Naming, Describing, and Quantifying Visual Objects in Humans and LLMs

Autori: Alberto Testoni, Juell Sprott, Sandro Pezzelle
Pubblicato in: Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), 2024
Editore: Association for Computational Linguistics

Construction coordination in first and second language acquisition

Autori: Arabella Sinclair, Raquel Fernández
Pubblicato in: Proceedings of the 25th Workshop on the Semantics and Pragmatics of Dialogue, 2021
Editore: SemDial

Generating Image Descriptions via Sequential Cross-Modal Alignment Guided by Human Gaze

Autori: Ece Takmaz, Sandro Pezzelle, Lisa Beinborn, Raquel Fernández
Pubblicato in: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2020, Pagina/e 4664-4677
Editore: Association for Computational Linguistics
DOI: 10.18653/v1/2020.emnlp-main.377

Peer reviewed articles (8)

Structural Persistence in Language Models: Priming as a Window into Abstract Language Representations

Autori: Arabella Sinclair, Jaap Jumelet, Willem Zuidema, Raquel Fernández
Pubblicato in: Transactions of the Association for Computational Linguistics, Numero 10, 2022, Pagina/e 1031-1050, ISSN 2307-387X
Editore: MIT Press
DOI: 10.1162/tacl_a_00504

Linguistic issues behind visual question answering

Autori: Raffaella Bernardi, Sandro Pezzelle
Pubblicato in: Language and Linguistics Compass, 2021, ISSN 1749-818X
Editore: Blackwell
DOI: 10.1111/lnc3.12417

Alignment of code switching varies with proficiency in second language learning dialogue

Autori: Arabella J. Sinclair, Raquel Fernández
Pubblicato in: System, Numero 113, 2024, Pagina/e 102952, ISSN 0346-251X
Editore: Pergamon Press Ltd.
DOI: 10.1016/j.system.2022.102952

The Efficiency of Question‐Asking Strategies in a Real‐World Visual Search Task

Autori: Alberto Testoni, Raffaella Bernardi, Azzurra Ruggeri
Pubblicato in: Cognitive Science, Numero 47, 2024, ISSN 0364-0213
Editore: Lawrence Erlbaum Associates Inc.
DOI: 10.1111/cogs.13396

Semantic Adaptation to the Interpretation of Gradable Adjectives via Active Linguistic Interaction

Autori: Sandro Pezzelle, Raquel Fernández
Pubblicato in: Cognitive Science, Numero 47, 2023, ISSN 0364-0213
Editore: Lawrence Erlbaum Associates Inc.
DOI: 10.1111/cogs.13248

A taxonomy and review of generalization research in NLP

Autori: Hupkes, Dieuwke; Giulianelli, Mario; Dankers, Verna; Artetxe, Mikel; Elazar, Yanai; Pimentel, Tiago; Christodoulopoulos, Christos; Lasri, Karim; Saphra, Naomi; Sinclair, Arabella; Ulmer, Dennis Thomas; Schottmann, Florian; Batsuren, Khuyagbaatar; Sun, Kaiser; Sinha, Koustouv; Khalatbari, Leila; Ryskina, Maria; Cotterell, Ryan; Jin, Zhijing
Pubblicato in: Nature Machine Intelligence, 2023, Pagina/e 1161–1174, ISSN 2522-5839
Editore: Springer Nature

Beyond Perplexity: Examining Temporal Generalization in Large Language Models via Definition Generation

Autori: Iris Luden, Mario Giulianelli, Raquel Fernández
Pubblicato in: Computational Linguistics in the Netherlands Journal, 2024, ISSN 2211-4009
Editore: CLIN/OJS

Word Representation Learning in Multimodal Pre-Trained Transformers: An Intrinsic Evaluation

Autori: Sandro Pezzelle, Ece Takmaz, Raquel Fernández
Pubblicato in: Transactions of the Association for Computational Linguistics, 2021, ISSN 2307-387X
Editore: Association for Computational Linguistics
DOI: 10.1162/tacl_a_00443

Book chapters (1)

Grounded and Ungrounded Referring Expressions in Human Dialogues: Language Mirrors Different Grounding Conditions

Autori: Eleonora Gualdoni, Raffaella Bernardi, Raquel Fernández, Sandro Pezzelle
Pubblicato in: Proceedings of the Seventh Italian Conference on Computational Linguistics CLiC-it 2020, 2025, Pagina/e 223-229
Editore: Accademia University Press
DOI: 10.4000/books.aaccademia.8600

Thesis and dissertations (2)

Neural models of language use

Autori: Giulianelli, M.
Pubblicato in: 2023
Editore: University of Amsterdam

Visual and linguistic processes in deep neural networks

Autori: Takmaz, E.K.
Pubblicato in: 2024
Editore: University of Amsterdam

Other (3)

Modelling Multimodal Integration in Human Concept Processing with Vision-Language Models.

Autori: Anna Bavaresco, Marianne Heer Kloots, Sandro Pezzelle, & Raquel Fernández
Pubblicato in: 2025
Editore: Arxiv

Incremental Alternative Sampling as a Lens into the Temporal and Representational Resolution of Linguistic Prediction.

Autori: Mario Giulianelli, Sarenne Wallbridge, Ryan Cotterell, & Raquel Fernández
Pubblicato in: 2025
Editore: PsyArXiv

RACQUET: Unveiling the Dangers of Overlooked Referential Ambiguity in Visual LLMs.

Autori: Alberto Testoni, Barbara Plank, & Raquel Fernández
Pubblicato in: 2025
Editore: Arxiv