Causal Inference for Exploration and Learning

Objetivo

Inspired by the fact that incorporation of causality in learning is what predominantly sets human judgment apart from machines, this proposal advocates the integration of causal reasoning into sequential decision-making frameworks, such as bandits and reinforcement learning. Historically, the two fields of machine learning (and its subfield of sequential decision-making) and causality have developed separately. In addition to historical reasons, most machine learning algorithms are causally agnostic because causal inference is inherently a hard problem.

The key objective of the proposed work is to enhance modern machine learning systems—especially in sequential decision-making settings such as multi-armed bandits, reinforcement learning (RL), and large language models (LLMs)—by systematically incorporating tools and principles from causal inference. The proposed enhancement will be achieved through the development of a novel framework for robust, flexible, and scalable causal inference that uses both interventional and counterfactual reasoning as opposed to traditional approaches that merely used observational data.

The project will span theoretical endeavors as well as the design of provably good practical algorithms in the sense that they are simultaneously efficient in terms of computational requirements, as well as the number of data samples needed to perform causal identification and explorations in complex applications that arise in sequential decision-making in uncertain environments.

If successful, our findings will have significant broader impact on the society through a wide range of applications, such as clinical trials, assessing policy decisions that arise in daily life, in business, economics, and scientific domains, all the way to generative AI applications such as training large language models.