A Framework for Auditing Recommendation Engines under the DSA

Descripción del proyecto

Marco para la aplicación del Reglamento de Servicios Digitales

La reciente aparición de los grandes modelo de lenguaje (LLM) y la inteligencia artificial, junto con su creciente papel en las redes sociales y los motores de búsqueda, subraya la necesidad de supervisar y garantizar su funcionamiento seguro. La aplicación del Reglamento de Servicios Digitales (RSD) de la Unión Europea es un motor clave en este esfuerzo. El proyecto FARE_DSA, financiado por el Consejo Europeo de Investigación, tiene como objetivo desarrollar un marco para auditar exhaustivamente las motores de búsqueda y los LLM basados en chat, proporcionando una herramienta de prueba de concepto que apoyará y fomentará la aplicación del artículo 40 del RSD. El proyecto investigará el papel y la influencia de los algoritmos de los LLM y los motores de búsqueda en los procesos electorales y proporcionará orientaciones prácticas para superar los retos que plantea la aplicación del RSD.

Objetivo

This proposal presents a novel pipeline framework designed to systematically audit Search Engines (SEs) and chat-based Large Language Models (LLMs), serving as a proof-of-concept tool to support the implementation of Article 40 of the EU's Digital Services Act (DSA). The project addresses two critical goals: 1) to investigate how proprietary algorithms in SEs and LLMs influence electoral processes, and 2) to provide practical guidance to overcome complex DSA implementation challenges.

SEs, such as Google, and LLMs, such as ChatGPT, have become primary channels for accessing information. Yet, both rely on opaque and fast-evolving algorithms that are difficult to audit. This raises significant concerns regarding the fairness, transparency, and neutrality of information they offer, particularly in contexts critical to democracy, such as elections: given their widespread use, even minor algorithmic biases can have significant impacts, influencing voter perceptions and electoral outcomes.

The proposed framework leverages resources developed in two ERC-funded projects, including an advanced bot-based auditing tool. Previous studies conducted during high-stakes events, such as the 2024 EU Parliamentary elections, show systematic biases in search results and LLM responses and it is crucial to better understand the origin and implications of these observations. With FARE_DSA we want to probe this systemic bias by combining our independent auditing tool with the first-ever instances of granted access to platform data, through Article 40.

In collaboration with different stakeholders, we will address key technical and procedural hurdles and deliver actionable insight on DSA implementation. Overall, this proposal seeks to establish a rigorous, reproducible auditing methodology capable of informing policy decisions, enhancing transparency, and bolstering democratic resilience through improved regulation and oversight of algorithm-driven platforms.