Periodic Reporting for period 1 - LOVe (Linking Objects to Vectors in distributional semantics: A framework to anchor corpus-based meaning representations to the external world)
Periodo di rendicontazione: 2015-06-01 al 2017-05-31
The project has three main objectives:
1) To explore the representation of entity names (such as ""J. K. Rowling"") in distributional semantics.
2) To connect referential expressions (""the mug"", ""the book I bought"") to objects depicted in images.
3) To develop a semantic framework that combines information about entities and concepts in a meaningful way.
We find that it is possible for current distributional models to learn to refer directly from data, and that distributional representations usefully represent the meaning of entity names.
The project advances our scientific understanding of language, a defining trait of the human species, and makes significant progress towards building machines we can talk to, with the ensuing impact on our everyday lives.
"
Our second goal was to link referential expressions to the external world, where this time the external world is operationalized in terms of visual information (objects represented in images). We showed that our computational model is able to pick the image that corresponded to a referential expression (""the cat""), and to spot cases in which the referential expression is not adequate (for instance, asking for ""the cat"" when there are several different cats); we also showed that our model can learn the meaning of quantifiers like ""all"" and ""some"" (capturing the difference between ""all circles are black"" and ""some circles are black"") directly from images containing objects that correspond to these different expressions; finally, we showed that our model can combine visual information and linguistically-conveyed information: If yesterday I told you that I bought a particular mug, you can use this information when today I ask you to pass me the mug that I bought and there are several mugs to choose from.
Our third goal was to develop a semantic framework that encompasses conceptual and referential aspects of meaning. We made progress in this direction, with: 1) The description of a dual, conceptual and referential route in composition and its formalization; 2) the discussion of the limitations of distributional models for phenomena beyond the sentence level, pointing to specific directions in which the field needs to move; 3) the summarization and appraisal of the state of the art in the field of Formal Distributional Semantics, in a Special Issue in the top journal in the field, Computational Linguistics.
As for dissemination, as planned, I disseminated the project results via social networks and participating in ESSLLI 2016 (ESSLLI is the most renowned summer school in the field). I also disseminated the research results to the scientific community, with an exceptional publication record: Nine publications plus two accepted articles, six of which in the top venues in Computational Linguistics: One in ACL (top ranked venue), two in EMNLP (2nd ranked venue), two in Computational Linguistics (top journal in the field), and one in EACL (7th ranked venue). I also published an article in a high-ranked multidisciplinary science journal (PLoS ONE). Further dissemination was carried out through talks, most notably as the keynote speaker of three international workshops.
All the data gathered within the project, as well as all articles, are open and accessible to the scientific community and the whole of society.
"