The EU-funded HPLT project applies high-performance computing (HPC) to scale and advance language technologies. Taking advantage of recent advances in machine learning and astonishing storage capacities, it creates and processes huge language data sets and produces language and translation models in numerous languages. The resulting models will be tested from various angles to ensure smooth integration, high accuracy, and regulatory compliance concerning privacy, unwanted biases and ethical issues. The models and data sets will be a game changer in the language service market in the EU and beyond. The resulting models will be open, free and available from established language repositories for anyone interested in pursuing research or innovation projects. The project, coordinated by the Charles University in Prague (CUNI), gathers partners from 5 different universities, 2 HPC centers and a private NLP company from around Europe. So far, we have collected about 1.85 PB of data from the Internet Archive and Common Crawl, processed, cleaned and released them, built encoder-only language models for 75 languages, and a few generative, decoder-only models (Finnish, Norwegian and a few other languages). In addition, translation models have been trained and released for 18 low-resource languages paired with English. Almost 10 million GPU and 10 million CPU hours have been secured and used on various HPC facilities throughout Europe. All data and models have been released through the HPLT website and HuggingFace platform.
