Within HimL, we have made progress beyond state-of-the-art in five areas, which we describe below.
Data and Adaptation: We have released a standard data set for training of medically-oriented
MT systems - the UFAL medical corpus (
https://ufal.mff.cuni.cz/ufal_medical_corpus(opens in new window)).
This covers 8 European language pairs, including the
HimL languages. Our main technique for adaptation has been fine-tuning the NMT systems
with in-domain data, where we extend the in-domain data with synthetic data created by selecting
from large mono-lingual crawls and back-translating. We also showed how to mine translations
from monolingual data, improving published techniques to work with medical terms and rare terms.
Semantics: Analysing the outputs of NMT shows that some problems that were apparent in earlier systems
re now extremely rare. However NMT still exhibits problems where
it omits important information, or adds incorrect information to the output. To counter this we use
a technique called "reconstruction", where the source should be reconstructable from the output.
We have also shown how to improve NMT using high quality dictionaries, and how to incorporate semantic
and syntactic information from external tools.
Morphology: NMT often produces more fluent, morphologically correct output, and indeed our analysis showed
that the placement and inflection of German verbs was much improved. However a consequence of the
move to NMT was that it was necessary to limit the vocabulary size using subwords. In HimL, we
developed a linguistically motivated way of segmenting German which improved over the standard, non-linguistic
segmentation method. We also showed how to improve NMT (in low-resource settings) by separately generating
the lemma and morphological tag of each word, and then synthesising.
To deploy our lab-based models, we wrapped our systems in a simple API which could be used by both NHS 24 and
Cochrane. The translation wrapper handles mark-up, and was re-engineered when we shifted to NMT in Year 3.
Our evaluation work has included ranking (generalised A-B tests), post-editing performance measurement and
user studies and surveys. We have also developed automatic and human assisted semantic metrics to
help us in tuning our systems towards fidelity. The user studies and surveys have looked at users' attitudes
towards MT and their assessment of its usefulness. We find that users want human-level
quality in translation (which is best achieved by post-editing) but they acknowledge that MT can be useful
if this is not possible.
In terms of scientfic impact, HimL has resulted in 55 scientific papers, listed on our website. We have
also contributed to the WMT17 biomedical shared translation task, making available a standard training set
as well as HimL test sets for all the HimL languages. HimL has contributed to the development
of two different open-source NMT systems (Nematus and HimL) which are gaining market-share amongst both
academic and non-academic users.
Outside the academic community, HimL can have an impact on the public health domain and beyond in other
public services. There are many cases where multilingual content is required and the improvements we have
seen in translation can make the production of this content (by post-editing) more economical and the
use of unedited MT (in cases where quality can be compromised) more feasible.
For Cochrane, the reviews and plain language summaries are a critical resource for healthcare consumers. A large
amount of Cochrane content has been translated into Spanish and French, and this has had a profound
effect on access to the Cochrane portal, indicating an unmet need. Cochrane do not have resources to reach all non-English speakers
with full human translation, so MT can help to bridge the gap.
