We built several monolingual (Slovene,Estonian) and multilingual (Croatian-Slovene-English, Finnish-Estonian-English, Lithuanian-Latvian-English) large pretrained contextual language models, created several novel evaluation benchmarks and showed improved performance of our models in monolingual processing and cross-lingual transfer. We developed a new approach of background knowledge injection into deep neural networks and adapted the techniques for explanation of classifier decisions and developed a novel technique to prevent dieselgate-like attacks on explanation techniques.
Next, we developed methods for named entity recognition and linking and event detection and tested them on well-known evaluation benchmarks (e.g. excellent results in HIPE competition at CLEF 2020, SlavNER 2021 shared task, TREC 2021 Incident Streams track). For keyword extraction, we proposed several novel methods including a multilingual unsupervised system RaKUn, monolingual supervised TNT-KID (also implemented in production by Ekspress Meedia), and a new cross-lingual method.
One focus was cross-lingual user-generated content analysis. We developed a set of new methods for author profiling, sentiment and opinion detection. In the most relevant applied task for news media, comment filtering, we developed cross-lingual methods that equal the accuracy of monolingual classifiers, but with much less target-language training data; and versions that improve performance by incorporating knowledge of topic. The Croatian media company 24sata is now using one of our comment moderation systems in production.
In news analysis research, we developed methods for interesting news retrieval, topic modelling, and a novel AutoBOT autoML approach for various classification tasks. We competed in several shared tasks including TREC 2021 background linking (1st place) and SemEval 2022 multilingual news article similarity. We also designed a novel document representation learning method based on knowledge-graphs and used it for fake news detection. We also developed a cross-lingual news sentiment detection method, and scalable methods for semantic change detection and viewpoint analysis. In addition, we created novel extractive, abstractive and visual summarisation systems.
Finally, we developed multilingual natural language generation (NLG) technology for automated journalism and tested it on EuroStat and Covid-19 datasets in six languages. We developed techniques to dynamically decide the order in which information is presented to the reader in automatically generated news texts, and several methods for headline generation.
We released a number of novel news articles and comments datasets and pretrained models and tested and integrated selected tools for keyword extraction and comment moderation to media partners’ production settings.
The tools are made available through the EMBEDDIA media assistant (EMA) platform that consists of:
- a live online EMBEDDIA demonstrator showcasing keyword extraction, comment filtering and news generation;
- dockerized components for easy installation and use of a selected range of the main tools;
- the EMBEDDIA Tools Explorer giving easy searchable access to all code and dockers;
- the TEXTA toolkit giving interactive user access to data exploration, investigative journalism and classification tools.
We disseminated the project results via the EMBEDDIA webpage (>40,000 unique visitors), Twitter account (@embeddiaproject) (>1,800 followers) and Facebook page. The EMBEDDIA media assistant had more than 270 users, and our code repository has more than 90 public items. The results of the project were published in more than 35 journal and more than 100 conference papers.
