Skip to main content
Go to the home page of the European Commission (opens in new window)
English en
CORDIS - EU research results
CORDIS

Robust Generative Information Retrieval

Project description

Building trustworthy generative information retrieval

Traditional information retrieval relies on indexing and searching. A new generative information retrieval approach encodes entire document collections directly into a model. This retrieval is then formalised into a sequence-to-sequence learning problem, which allows for the optimisation of broader goals like fairness, diversity and long-term impact. Funded by the ERC, the UNITE project aims to build trustworthy generative retrieval systems. It will combine generative models with reinforcement learning. It will also ensure accuracy, reliability and resilience, while exploring explainability, reproducibility and safety. Applications will include news search, recommendation systems and climate-related information retrieval. UNITE promises to reshape how information retrieval is conceived, evaluated and optimised for societal benefit.

Objective

Generative information retrieval is an emerging paradigm that replaces the traditional index-then-retrieve pipeline by encoding all information about a document collection in the parameters of a model. The retrieval task is then formalized as a sequence-to-sequence learning problem, making it possible to optimize the system end-to-end. This enables optimization towards a broad range of goals, not just short-term utility ones but also broader long-term objectives, such as fairness and diversity.

Trustworthiness is a prerequisite for the development, deployment, and use of AI-based systems. With UNITE I propose a technical agenda to understand how we can build warranted trust in generative information retrieval while reaping the benefits of the potential this paradigm promises for optimizing for goals beyond short-term utility.

My methodological innovations will be based on advancing the foundations of generative information retrieval and a synthesis of generative information retrieval with reinforcement learning, capturing the sequential and interactive nature of retrieval, thus offering a principled way to deal with long-term goals. These advances will be pursued along three lines where generative information retrieval needs to uphold verifiable guarantees: accuracy, including well-defined and explained contexts of usage; reliability, including exhibiting parity with respect to sensitive attributes; and resilience to distributional shifts and adversarial examples. I will also study ways to probe generative information retrieval methods to aid explainability, reproducibility, and safety. We will demonstrate the utility of our new methodologies on tasks of great societal value: news search and recommendation, and information retrieval for climate impact.

While adventurous, UNITE has great algorithmic significance. It may lead to a fundamental re-assessment of how the field conceptualizes, evaluates and optimizes the success of information retrieval methods.

Fields of science (EuroSciVoc)

This project has not yet been classified with EuroSciVoc.
Be the first one to suggest relevant scientific fields and help us improve our classification service

Keywords

Programme(s)

Topic(s)

Funding Scheme

HORIZON-ERC - HORIZON ERC Grants

See all projects funded under this funding scheme

Call for proposal

(opens in new window) ERC-2024-ADG

See all projects funded under this call

Host institution

UNIVERSITEIT VAN AMSTERDAM
Net EU contribution
€ 2 868 930,00
Total cost
No data

Beneficiaries (1)