The Multilingual Flexible Neuro-Symbolic Language Generation (M-FleNS) project aims to create new Natural Language Generation (NLG) tools and resources for the automatic creation of textual descriptions from abstract structured representations such as Knowledge Graphs, in order to address some existing issues related to, among other aspects, the accuracy of the generated contents, data and energy efficiency, bias management, and adaptability to low-resource languages. The attached figure "mflens_input-output" shows on the left a sample M-FleNS input Knowledge Graph, which consists of four different properties (Engine, Transmission, Assembly and BodyStyle) associated to the entity Audi_A1, and on the right a valid output text.
At the time the proposal was drafted (mid-2021), state-of-the-art systems for data-to-text generation were neural machine learning methods (e.g LSTMs) and fine-tuned, or even off-the-shelf, small-sized language models (e.g. T5). These systems needed significant amounts of resources (data, energy, compute), and struggled with accuracy, biases, low-resource settings or out-of-domain data. In late 2022, three months after the M-FleNS project started, very large instruction-tuned language models were made available, and the landscape of NLG changed drastically: these models were able to produce human-like texts for a number of languages in a zero-shot setting, making them widely adopted, despite their tremendous resource greed (both at learning and execution time). The sudden emergence of very large language models (VLLMs) had two main consequences for the project:
- The challenge of improving the quality of data-to-text systems became much smaller, so we dedicated more effort to the aspects on which VLLMs are still falling short, namely energy-efficiency and very low resource setting (as it is the case for Irish).
- VLLMs are now extremely popular, but are black boxes, and knowing how to evaluate the quality of the texts they produce is more crucial than ever. Creating resources and methods for human evaluation of text quality naturally became a focal point of the project.
The main scientific objectives of the project are the following:
1- Improve and extend the existing FORGe rule-based NLG system, which is very energy-efficient although it generally lacks fluency; the system should be made as language-independent as possible, and produce outputs in English, Irish and French.
2- Combine rule-based and (deep-)learning techniques for improving the fluency of the rule-based system while keeping the resource requirements low.
3- Make available a range of automatic and human evaluation methods and resources for assessing the quality of the texts produced by any type of NLG system.
