A one-stop shop for European language technologies
Language technologies (LT), which process or generate natural language in written or spoken form, such as automated text translation or dictation tools, could support the European Digital Single Market (DSM). However, many remain language-specific. “The European LT community is fragmented and, with no incentives to work with less used languages, many only concentrate on English and perhaps Spanish, French and German. There clearly needs to be a bridge between LT and their commercial and academic developers,” says coordinator of the ELG project Georg Rehm from the German Research Center for Artificial Intelligence, the project host. This EU-supported project has developed the European Language Grid, an umbrella cloud-based LT platform which brings together various tools, services, data sets and resources, alongside the development community. “ELG is a marketplace where interested parties can find the language technology tool that best fits their need,” Rehm adds. Currently the platform includes about 10 000 data sets and other resources, alongside over 3 600 tools for use on the platform and more than 1 700 European LT organisations.
The micro-services approach
Design of the ELG platform was inspired by micro-services – small, single-purpose applications that drive much of modern software engineering. Individual LT tools are packaged into virtual containers for ingestion into the platform as micro-services, to then become available via the ELG website and cloud APIs. This approach offers scalability, so with increased user demand the platform can grow organically by adding more hardware resources. When resources are added, descriptive metadata is applied – such as language/s supported, developer etc. – so that it can be found by user searches.
Building momentum
The first working prototype, available 10 months after the project’s start, underwent internal testing before onboarding of members. “User feedback has been phenomenal, with many acknowledging the gap we fill and several talking about the platform’s benefits at our annual conference, META-FORUM,” notes Rehm. The project also offered financial support to third parties, totalling around EUR 2 million, for projects to create new tools for ELG or make innovative use of those already available on the ELG platform. Fifteen projects were selected and the results made available on the platform.
Embedded far and wide
ELG directly addresses a key recommendation of the European Parliament resolution ‘Language equality in the digital age’ which calls for a “European language technology platform”. The project also helps counter digital language extinction by representing those languages spoken less. To help monitor this, the ELG platform hosts sister project ELE’s interactive dashboard which visualises levels of technology support received by each language. “Already fully functional, the ELG cloud platform has a lot of commercial interest, especially from smaller commercial European LT developers, in branching out into new geographical regions and markets,” explains Rehm. Currently ELG’s initiative is embedded in several new research projects. For example, ELE uses it for determining Digital Language Equality scores based on available services and resources in ELG, per language. In OpenGPT-X, funded by the German Federal Ministry for Economic Affairs and Climate Action, the team is modifying the platform to make it compatible with the emerging Gaia-X ecosystem. The ELG consortium have also put together a book which includes chapters on the platform’s technology, services and resources, alongside information about the 15 funded pilots. “The biggest challenge will now be to transform ELG into a ‘living’ platform actively used by the whole European LT community,” Rehm concludes. Towards this end, the team are establishing a not-for-profit legal entity to maintain and extend the ELG platform.
Keywords
ELG, languages, cloud, digital single market, micro-services