Democratize Trustworthy and Efficient Large Language Model Technology for Europe

Informations projet

TrustLLM

N° de convention de subvention: 101135671

DOI

10.3030/101135671

Date de signature de la CE 20 Octobre 2023

Date de début 1 Novembre 2023

Date de fin 31 Octobre 2026

Financé au titre de

• Digital, Industry and Space

Coût total

€ 6 929 702,50

Contribution de l’UE

€ 6 929 701,00

Coordonné par

LINKOPINGS UNIVERSITET
Sweden

CORDIS fournit des liens vers les livrables publics et les publications des projets HORIZON.

Les liens vers les livrables et les publications des projets du 7e PC, ainsi que les liens vers certains types de résultats spécifiques tels que les jeux de données et les logiciels, sont récupérés dynamiquement sur OpenAIRE .

Livrables

Other (6)

Language-specific adapters for multilingual LLMs

Pre-trained adapters for Germanic languages, to be used with existing LLMs and the TrustLLM models.

Data formatting pipeline

Pipeline for formatting data as a preparation for LLM training.

Initial training code

First version of parallel training code for training LLMs on European HPC systems

Multi-dimensional evaluation metric for text generation

An evaluation process usable in an online environment for generated texts testing reliability, accuracy, fluency, etc.

Methods for factual correctness based on retriever modelling

Report and software framework for LLM with factual correctness (Version 1) based on retriever-based modelling.

Quality filtering and deduplication pipeline

Pipeline for doing quality filtering and deduplication of data as a preparation for LLM training.

Data sets, microdata, etc (4)

Alignment data

Multilingual datasets for instruction fine-tuning.

Bias Dataset

An evaluation dataset quantifying the models’ potential biases toward minority groups

Benchmarking Platform

Open-source software package allowing for automatic benchmarking using the evaluation datasets developed in the project.

Germanic Language Modelling Evaluation Dataset

An evaluation dataset quantifying the models’ general Germanic linguistic capabilities.

Documents, reports (5)

Communication and dissemination toolkit

Printed and digital material for C&D, for example flyers, posters, social media posts and videos

Project Handbook

Outline of planned management procedures, tools, project roles and responsibilities.

IPR Management Plan

Detailed IPR management plan

Strategic plan for communication and dissemination

Initial plan for strategic communication and dissemination, to be yearly updated (internally)

Design Five Use Cases

A report detailing the design for each of the use cases

Data Management Plan (2)

Data Management Plan, V2

Detailed data management plan, including the plans for open source and open access publishing, V2

Data Management Plan

Detailed data management plan, including the plans for open source and open access publishing

Publications

Conference proceedings (12)

How Reliable Are Automatic Evaluation Methods for Instruction-Tuned LLMs?

Auteurs: Ehsan Doostmohammadi, Oskar Holmström, Marco Kuhlmann
Publié dans: Findings of the Association for Computational Linguistics: EMNLP 2024, 2024
Éditeur: Association for Computational Linguistics
DOI: 10.18653/V1/2024.FINDINGS-EMNLP.367

FoQA: A Faroese Question-Answering Dataset

Auteurs: Annika Simonsen, Dan Saattrup Nielsen, Hafsteinn Einarsson
Publié dans: 2025
Éditeur: Proceedings of the Third Workshop on Resources and Representations for Under-Resourced Languages and Domains (RESOURCEFUL-2025)

Tokenizer Choice For LLM Training: Negligible or Crucial?

Auteurs: Mehdi Ali, Michael Fromm, Klaudia Thellmann, Richard Rutmann, Max Lübbering, Johannes Leveling, Katrin Klug, Jan Ebert, Niclas Doll, Jasper Buschhoff, Charvi Jain, Alexander Weber, Lena Jurkschat, Hammam Abdelwahab, Chelsea John, Pedro Ortiz Suarez, Malte
Publié dans: Findings of the Association for Computational Linguistics: NAACL 2024, 2024
Éditeur: Association for Computational Linguistics
DOI: 10.18653/V1/2024.FINDINGS-NAACL.247

Investigating Multilingual Instruction-Tuning: Do Polyglot Models Demand for Multilingual Instructions?

Auteurs: Alexander Arno Weber, Klaudia Thellmann, Jan Ebert, Nicolas Flores-Herr, Jens Lehmann, Michael Fromm, Mehdi Ali
Publié dans: Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, 2024
Éditeur: Association for Computational Linguistics
DOI: 10.18653/V1/2024.EMNLP-MAIN.1159

From text to knowledge graph: Comparing relation extraction methods in a practical context

Auteurs: Bakker, R. M., & Di Scala, D. L.
Publié dans: First International Workshop on Generative Neuro-Symbolic AI, co-located with ESWC, Numéro Vol. 4, p. 7, 2024
Éditeur: CEUR-WS

Memory and Bandwidth are All You Need for Fully Sharded Data Parallel

Auteurs: Jiangtao Wang, Jan Ebert, Oleg Filatov, Stefan Kesselheim
Publié dans: ICML'24 Workshop on Advancing Neural Network Training (WANT), 2025
Éditeur: arXiV
DOI: 10.48550/ARXIV.2504.03655

How to Tune a Multilingual Encoder Model for Germanic Languages: A Study of PEFT, Full Fine-Tuning, and Language Adapters

Auteurs: Romina Oji, Jenny Kunz
Publié dans: Proceedings of the Joint 25th Nordic Conference on Computational Linguistics and 11th Baltic Conference on Human Language Technologies (NoDaLiDa/Baltic-HLT 2025), 2025
Éditeur: University of Tartu Library

Do Multilingual Large Language Models Mitigate Stereotype Bias?

Auteurs: Shangrui Nie, Michael Fromm, Charles Welch, Rebekka Görge, Akbar Karimi, Joan Plepi, Nazia Mowmita, Nicolas Flores-Herr, Mehdi Ali, Lucie Flek
Publié dans: 2024
Éditeur: Association for Computational Linguistics
DOI: 10.18653/V1/2024.C3NLP-1.6

Train More Parameters But Mind Their Placement: Insights into Language Adaptation with PEFT

Auteurs: Jenny Kunz
Publié dans: Proceedings of the Joint 25th Nordic Conference on Computational Linguistics and 11th Baltic Conference on Human Language Technologies (NoDaLiDa/Baltic-HLT 2025), 2025
Éditeur: University of Tartu Library

The Impact of Language Adapters in Cross-Lingual Transfer for NLU

Auteurs: Jenny Kunz, Oskar Holmström
Publié dans: Proceedings of the 1st Workshop on Modular and Open Multilingual NLP (MOOMIN 2024), 2024
Éditeur: Association for Computational Linguistics

Encoder vs Decoder: Comparative Analysis of Encoder and Decoder Language Models on Multilingual NLU Tasks

Auteurs: Dan Saattrup Nielsen, Kenneth Enevoldsen, Peter Schneider-Kamp
Publié dans: 2025
Éditeur: Proceedings of the Joint 25th Nordic Conference on Computational Linguistics and 11th Baltic Conference on Human Language Technologies (NoDaLiDa/Baltic-HLT 2025)

Ontology Learning from Text: an Analysis on LLM Performance

Auteurs: Bakker, R. M., Di Scala, D. L., & de Boer, M. H. T.
Publié dans: Proceedings of the 3rd NLP4KGC International Workshop on Natural Language Processing for Knowledge Graph Creation, colocated with Semantics, Numéro pp. 17-19, 2024
Éditeur: CEUR-WS