The WIDE project took on the challenge to develop a computational model of how language users understand and produce words. In psycholinguistics, the knowledge and skills that underly the comprehension and production of words is referred to as the ``mental lexicon''. This term reflects that traditionally, a speaker's word knowledge is conceptualized along the lines of a dictionary that lists words together with their meanings. However, earlier modeling work in psychology that made use of multi-layer artificial neural networks (the precursors of current deep learning models), argued that words are `calculated on the fly' rather than retrieved. The model developed in the WIDE project, the Discriminative Lexicon Model (DLM) also does not retrieve words from a mental dictionary, but calculates words' meanings from their forms, and calculates words' forms from their meanings. The DLM differs from the earlier models from psychology in that the architecture of its networks is substantially simplified. Words' forms and meanings are represented by high-dimensional numeric real-valued vectors, which are connected without any hidden layers. The resulting `wide' networks are mathematically equivalent to multivariate multiple regression models in statistics.
One important challenge for the WIDE project has been to clarify whether the simple and mathematically transparent linear networks of the DLM are up to the task of predicting meanings from forms, and forms from meanings. Application of the DLM to a series of typologically unrelated languages, including Estonian, Finnish, English, German, Russian, Navajo, Pame, Korean, Kinyarwanda, and Mandarin Chinese, revealed that the model performs very well on training data, and generalizes well to unseen data provided that (1) the morphology of the language is relatively regular and (2) that sufficient training data are available. As expected, generalization degrades for irregular morphology.
These results lead to the novel insight that the high-dimensional spaces of form and meaning are aligned to a much greater extent than previously thought. Thanks to the isomorphy between these spaces, the simple linear mappings of the DLM are able to generate accurate predictions not only for words forms and meanings, but also for unprimed and primed reaction times and spoken word durations.
Although the DLM does not work with form units for morphemes, it does implement semantic operations for inflectional and derivational morphology, using vector addition in semantic space. The DLM implements the conceptualization of, for instance, a Finnish partitive plural noun by adding the semantic vectors of the lexeme, the noun plural, and the partitive case. To this sum, an interaction term for number by case has to be added, given that an analysis of empirical Finnish word embeddings indicates that in Finnish, the meaning of the plural varies systematically with case. Figure 1 illustrates that in English, the meaning of noun plurals is also conditional, but instead of depending on case, plural meaning arises with the semantic class of the noun: each dot in this figure represents the change from singular to plural in a high-dimensional semantic space that was projected onto two two dimensions using t-distributed stochastic neighbor embedding. Selected semantic classes are highlighted using color coding. Importantly, when modeling with empirical embeddings for inflected words, the embeddings of the constituent semantic primitives and interactions are imputed, so that the model is able to conceptualize and produce novel inflected words.
The insights obtained with the WIDE project will be applied in a new project funded by the ERC, SUBLIMINAL. This project aims to improve smartphone apps for learning Mandarin as a second language, using the DLM model (in trial-to-trial learning mode) in combination with enhanced feedback on the phonetics and semantics of Mandarin words.
