My Personal AI Mediator for Virtual MEETtings BetWEEN People

Projektinformationen

Meetween

ID Finanzhilfevereinbarung: 101135798

DOI

10.3030/101135798

EK-Unterschriftsdatum 3 November 2023

Startdatum 1 Januar 2024

Enddatum 31 Dezember 2027

Finanziert unter

• Digital, Industry and Space

Gesamtkosten

€ 7 239 725,00

EU-Beitrag

€ 7 239 725,00

Koordiniert durch

TRANSLATED SRL
Italy

CORDIS bietet Links zu öffentlichen Ergebnissen und Veröffentlichungen von HORIZONT-Projekten.

Links zu Ergebnissen und Veröffentlichungen von RP7-Projekten sowie Links zu einigen Typen spezifischer Ergebnisse wie Datensätzen und Software werden dynamisch von OpenAIRE abgerufen.

Leistungen

Other (5)

Downstream task trained models - V1

This deliverable reports the efforts to adapt the best models to satisfy production constraints, such as low latency, low flickering, and low computational costs, and develop APIs to serve different applications and needs.

Speech LMM open release - V1

SpeechLMM is released three times during the project and each release is documented in a version of this deliverable.

Model cards - V1

All partners will author a deliverable containing model cards.

Speech LMM training journal - V1

This deliverable documents the “making of” SpeechLMM and is used for dissemination to the scientific and technology community in the form of the SpeechLMM training journal.

Human factors and security scenarios - V1

This deliverable provides human factors and a security report describing vulnerability and criticality scenarios.

Data sets, microdata, etc (1)

Mumospee open release - V1

This deliverable documents the creation of one of the largest open multimodal datasets, Mumospee, released over three generations, in coordination with the corresponding SpeechLMM generation activities.

Veröffentlichungen

Conference proceedings (21)

Factorized-VITS: Decoupling Prosody and Text in End-to-End Speech Synthesis without External or Secondary Aligner

Autoren: Yining Liu, Alexander Waibel
Veröffentlicht in: ICASSP 2025 - 2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2025
Herausgeber: IEEE
DOI: 10.1109/ICASSP49660.2025.10890003

SimulSeamless: FBK at IWSLT 2024 Simultaneous Speech Translation

Autoren: Sara Papi, Marco Gaido, Matteo Negri, Luisa Bentivogli
Veröffentlicht in: Proceedings of the 21st International Conference on Spoken Language Translation (IWSLT 2024), 2024
Herausgeber: Association for Computational Linguistics
DOI: 10.18653/V1/2024.IWSLT-1.11

NUTSHELL: A Dataset for Abstract Generation from Scientific Talks

Autoren: Maike Züfle, Sara Papi, Beatrice Savoldi, Marco Gaido, Luisa Bentivogli, Jan Niehues
Veröffentlicht in: Proceedings of the 22nd International Conference on Spoken Language Translation (IWSLT 2025), 2025
Herausgeber: Association for Computational Linguistics
DOI: 10.18653/V1/2025.IWSLT-1.2

FBK@IWSLT Test Suites Task: Gender Bias evaluation with MuST-SHE

Autoren: Beatrice Savoldi, Marco Gaido, Matteo Negri, Luisa Bentivogli
Veröffentlicht in: Proceedings of the 21st International Conference on Spoken Language Translation (IWSLT 2024), 2024
Herausgeber: Association for Computational Linguistics
DOI: 10.18653/V1/2024.IWSLT-1.10

MOSEL: 950,000 Hours of Speech Data for Open-Source Speech Foundation Model Training on EU Languages

Autoren: Marco Gaido, Sara Papi, Luisa Bentivogli, Alessio Brutti, Mauro Cettolo, Roberto Gretter, Marco Matassoni, Mohamed Nabih, Matteo Negri
Veröffentlicht in: Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, 2024
Herausgeber: Association for Computational Linguistics
DOI: 10.18653/V1/2024.EMNLP-MAIN.771

StreamAtt: Direct Streaming Speech-to-Text Translation with Attention-based Audio History Selection

Autoren: Sara Papi, Marco Gaido, Matteo Negri, Luisa Bentivogli
Veröffentlicht in: Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2024
Herausgeber: Association for Computational Linguistics
DOI: 10.18653/V1/2024.ACL-LONG.202

Quality Estimation with $k$-nearest Neighbors and Automatic Evaluation for Model-specific Quality Estimation

Autoren: Dinh, Tu Anh; Palzer, Tobias; Niehues, Jan
Veröffentlicht in: 2024
Herausgeber: EAMT 2024
DOI: 10.5445/IR/1000174743

GFG - Gender-Fair Generation: A CALAMITA Challenge

Autoren: Simona Frenda, Andrea Piergentili, Beatrice Savoldi, Marco Madeddu, Martina Rosola, Silvia Casola, Chiara Ferrando, Viviana Patti, Matteo Negri, Luisa Bentivogli
Veröffentlicht in: Proceedings of the 10th Italian Conference on Computational Linguistics (CLiC-it 2024), 2024
Herausgeber: CEUR Workshop Proceedings

FINDINGS OF THE IWSLT 2024 EVALUATION CAMPAIGN

Autoren: Ibrahim Said Ahmad, Antonios Anastasopoulos, Ondřej Bojar, Claudia Borg, Marine Carpuat, Roldano Cattoni, Mauro Cettolo, William Chen, Qianqian Dong, Marcello Federico, Barry Haddow, Dávid Javorský, Mateusz Krubiński, Tsz Kim Lam, Xutai Ma, Prashant Mathur, Evgeny Matusov, Chandresh Maurya, John McCrae, Kenton Murray, Satoshi Nakamura, Matteo Negri, Jan Niehues, Xing Niu, Atul Kr. Ojha, John Ortega, Sara Papi, Peter Polák, Adam Pospíšil, Pavel Pecina, Elizabeth Salesky, Nivedita Sethiya, Balaram Sarkar, Jiatong Shi, Claytone Sikasote, Matthias Sperber, Sebastian Stüker, Katsuhito Sudoh, Brian Thompson, Alex Waibel, Shinji Watanabe, Patrick Wilken, Petr Zemánek, Rodolfo Zevallos
Veröffentlicht in: Proceedings of the 21st International Conference on Spoken Language Translation (IWSLT 2024), 2024
Herausgeber: Association for Computational Linguistics
DOI: 10.18653/V1/2024.IWSLT-1.1

Speech Translation with Speech Foundation Models and Large Language Models: What is There and What is Missing?

Autoren: Marco Gaido, Sara Papi, Matteo Negri, Luisa Bentivogli
Veröffentlicht in: Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2024
Herausgeber: Association for Computational Linguistics
DOI: 10.18653/V1/2024.ACL-LONG.789

Decoupled Vocabulary Learning Enables Zero-Shot Translation from Unseen Languages

Autoren: Mullov, Carlos; Pham, Quan; Waibel, Alexander
Veröffentlicht in: Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2024
Herausgeber: ACL
DOI: 10.5445/IR/1000174872

Optimizing Rare Word Accuracy in Direct Speech Translation with a Retrieval-and-Demonstration Approach

Autoren: Li, Siqi; Liu, Danni; Niehues, Jan
Veröffentlicht in: Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, 2024
Herausgeber: EMNLP
DOI: 10.48550/ARXIV.2409.09009

MAGNET - MAchines GeNErating Translations: A CALAMITA Challenge

Autoren: Mauro Cettolo, Andrea Piergentili, Sara Papi, Marco Gaido, Matteo Negri, Luisa Bentivogli
Veröffentlicht in: Proceedings of the 10th Italian Conference on Computational Linguistics (CLiC-it 2024), 2024
Herausgeber: CEUR Workshop Proceedings

An LLM-as-a-judge Approach for Scalable Gender-Neutral Translation Evaluation

Autoren: Andrea Piergentili, Beatrice Savoldi, Matteo Negri, Luisa Bentivogli
Veröffentlicht in: Proceedings of the 3rd Workshop on Gender-Inclusive Translation Technologies (GITT 2025), 2025
Herausgeber: European Association for Machine Translation

From Speech to Summary: A Comprehensive Survey of Speech Summarization

Autoren: Retkowski, Fabian; Züfle, Maike; Sudmann, Andreas; Pfau, Dinah; Niehues, Jan; Waibel, Alexander
Veröffentlicht in: 2025
Herausgeber: EMNLP 2025 Main
DOI: 10.5445/IR/1000180972

Speech Foundation Models and Crowdsourcing for Efficient, High-Quality Data Collection

Autoren: Beomseok Lee, Marco Gaido, Ioan Calapodescu, Laurent Besacier, Matteo Negri
Veröffentlicht in: Proceedings of the 31st International Conference on Computational Linguistics, 2025
Herausgeber: Association for Computational Linguistics

Prepending or Cross-Attention for Speech-to-Text? An Empirical Comparison

Autoren: Tsz Kin Lam, Marco Gaido, Sara Papi, Luisa Bentivogli, Barry Haddow
Veröffentlicht in: Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), 2025
Herausgeber: Association for Computational Linguistics
DOI: 10.18653/V1/2025.NAACL-LONG.153

Cross-lingual Evaluation of Multilingual Text Generation

Autoren: Shamil Chollampatt, Minh Quang Pham, Sathish Reddy Indurthi, Marco Turchi
Veröffentlicht in: Proceedings of the 31st International Conference on Computational Linguistics, 2025
Herausgeber: Association for Computational Linguistics

What the Harm? Quantifying the Tangible Impact of Gender Bias in Machine Translation with a Human-centered Study

Autoren: Beatrice Savoldi, Sara Papi, Matteo Negri, Ana Guerberof-Arenas, Luisa Bentivogli
Veröffentlicht in: Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, 2024
Herausgeber: Association for Computational Linguistics
DOI: 10.18653/V1/2024.EMNLP-MAIN.1002

Explainability for Speech Models: On the Challenges of Acoustic Feature Selection

Autoren: Dennis Fucci, Beatrice Savoldi, Marco Gaido, Matteo Negri, Mauro Cettolo, Luisa Bentivogli
Veröffentlicht in: Proceedings of the 10th Italian Conference on Computational Linguistics (CLiC-it 2024), 2024
Herausgeber: CEUR Workshop Proceedings

Findings of the Quality Estimation Shared Task at WMT 2024: Are LLMs Closing the Gap in QE?

Autoren: Chrysoula Zerva, Frederic Blain, José G. C. De Souza, Diptesh Kanojia, Sourabh Deoghare, Nuno M. Guerreiro, Giuseppe Attanasio, Ricardo Rei, Constantin Orasan, Matteo Negri, Marco Turchi, Rajen Chatterjee, Pushpak Bhattacharyya, Markus Freitag, André Martins
Veröffentlicht in: Proceedings of the Ninth Conference on Machine Translation, 2024
Herausgeber: Association for Computational Linguistics
DOI: 10.18653/V1/2024.WMT-1.3

Peer reviewed articles (2)

How “Real” is Your Real-Time Simultaneous Speech-to-Text Translation System?

Autoren: Sara Papi, Peter Polák, Dominik Macháček, Ondřej Bojar
Veröffentlicht in: Transactions of the Association for Computational Linguistics, Ausgabe 13, 2025, ISSN 2307-387X
Herausgeber: MIT Press
DOI: 10.1162/TACL_A_00740

A decade of gender bias in machine translation

Autoren: Beatrice Savoldi, Jasmijn Bastings, Luisa Bentivogli, Eva Vanmassenhove
Veröffentlicht in: Patterns, Ausgabe 6, 2025, ISSN 2666-3899
Herausgeber: Elsevier BV
DOI: 10.1016/J.PATTER.2025.101257

Other (1)

Facial Attribute Based Text Guided Face Anonymization

Autoren: Muştu, Mustafa İzzet; Ekenel, Hazım Kemal
Veröffentlicht in: 2025
Herausgeber: arXiv
DOI: 10.48550/ARXIV.2505.21002