Towards globally accessible language technology and its alignment to cultural contexts

Projektbeschreibung

Fortschritte in Technologie zur mehrsprachigen Verarbeitung natürlicher Sprache

Große Sprachmodelle wie ChatGPT, die in verschiedenen Sektoren weit verbreitet sind, stützen sich auf Technologien zur Verarbeitung natürlicher Sprache, um flüssige Antworten aus natürlichsprachlichen Anweisungen zu generieren. Sowohl große Sprachmodelle als auch andere Anwendungen auf der Grundlage der Verarbeitung natürlicher Sprache sind jedoch aufgrund der großen Mengen an linguistischen Daten, die für ihr Training benötigt werden, weitgehend auf einige wenige Sprachen beschränkt. Das Ziel des ERC-finanzierten Projekts CulturAL besteht darin, diese Barriere zu überwinden, wobei eine innovative Methodik für den sprachübergreifenden Transfer großer Sprachmodelle entwickelt wird, die deren Anpassung an eine Vielzahl von Sprachen, Dialekten und kulturellen Kontexten gestattet.

Ziel

My research project focuses on natural language processing (NLP), an area of artificial intelligence concerned with automatic interpretation and generation of human language. NLP is well-known for its widely-used applications, such as machine translation (MT), text mining, question answering or dialogue systems. Much of this technology is now powered by large language models (LLMs), such as ChatGPT, which take a natural language instruction or question as input, and generate a fluent response. Due to their impressive performance in a range of tasks and their ease of use, in less than a year they received a wide adoption in many societal contexts (education, journalism, healthcare and others), and became the central paradigm in the field of NLP.

The development of LLMs, however, requires access to a vast amount of data and resources in a given language, as well as considerable computational infrastructure. As a result, these models are in practice limited to a handful of widely-spoken languages, leaving over 6,000 of the world’s languages and dialects without access to language technology. Furthermore, research on LLM alignment, which aims to ensure the safety of their use, has been almost exclusively directed toward the English-speaking world. Taken together, these problems lead to a major inequity in today’s language technology and artificial intelligence more broadly.

Taking a step towards a more inclusive and equitable language technology, this project will develop a novel methodology for cross-lingual transfer of LLMs to a wide-range of (low-resource, understudied) languages and dialects, and their alignment to diverse cultural contexts. The project will, therefore, advance multilingual NLP technology, extending its reach to populations currently underserved by NLP and making it safe for them to use.