Natural Language Processing (NLP) is the field of computer science and
artificial intelligence (AI) dedicated to making computers capable of
analysing and generating text and speech, allowing users to access
information from texts that, unaided, they would not have time to
read. The problem of unhindered access to information by its citizens
is a problem of vital importance to a democratic and dynamic society.
Its provision requires components such as parsers and language models,
named-entity recognisers, and knowledge-graphs. More recently, Large
Language Models such as GPT have revolutionized NLP.
Large Language Models (LLMs) work by representing units of text such
as words, sentences, or characters in terms contexts of other such
units in which they occur in vast amounts of text as "embeddings" of a
few hundred dimensions. Such models can be trained as sequential
models, the ability that underlies their fluency when used to generate
text, and that also underlies their tendency to "hallucinate",
producing plausible but factually incorrect text. They can be further
trained by fine-tuning for specific NLP task.
However, LLMs cost millions of dollars, needing vast computational
resources and text data to train. They are hard to update without
retraining from scratch, and slow in response. They work in a quite
different way to people. There is a need to investigating less data-
and compute-intensive methods. Since children learn language on the
basis of far less data, there is continuing interest in modeling
something more like human language understanding.
Unfortunately, we have very little idea of how to represent the
meanings associated with human understanding in a form that makes it
as obvious to the machine as it is to the human that text about one
company buying another answers to a question about their owning it.
Such "common-sense" implications are too obvious to human readers to
ever need to be explicitly stated in text. This makes it unlikely
that LLMs, merely by representing the contexts of use of relations
like "buy" and "own", actually represent the meaning of those words in
this sense.
The main emphasis of the project was to investigate an alternative
approach, using "machine-reading" with wide-coverage parsers to
extract relations between typed named entities from large amounts of
unlabeled text describing the same events from multiple sources, and
looking for evidence of entailment between those relations, based on
overlap between the sets of named-entity tuples that ground the
relations in the text. (For example, the companies in the "buy"
relation are included among those in the "own" relation, but not vice
versa.) On this basis, we construct a large entailment graph,
providing a semantics for question-answering (QA).
A further objective was to investigate the potential for LLMs to do
the same job,
Further objectives were to develop the Combinatory Categorial Grammar (CCG) parsers used to build the
graphs and in QA, and to develop CCG as an explanatory theory of grammar of use
to linguists and in the study of cognition more broadly.
