In the first phase of the DEUCE project, we made a lot of progress in understanding, learning, and analyzing systems that exhibit uncertainty.
Within the first key challenge, handling complex and high-dimensional environments, we provided several results that tackle systems of high-dimensional representations, such as pixel representations or multi-agent settings. We highlight one of the first works that performs safe reinforcement learning from pixels, published at ICLR 2023, and a novel approach to handle multi-agent systems that operate under partial information and uncertainty with many agents, published at AAAI 2024.
For the second key challenge, we were able to provide novel results on the abstraction and verification of dynamical systems. Intuitively, high-dimensional, continuous, state and action spaces are abstracted into discrete spaces. The main result is that we provide guarantees that link the original system and the abstraction. To highlight two results, we published one paper at the AAAI 2024 conference dealing with various types of epistemic and aleatoric uncertainty and a comprehensive overview in the Journal of AI research. Moreover, we provided the first approach to safe and shielded reinforcement learning under partial observability, published at AAAI 2023.
For the third challenge, we provided a string of results that help the verification and learning of uncertain systems. As a highlight, we defined novel semantics for a complicated model: robust partially observable Markov decision processes. Our semantics help to understand uncertainty in a game-based fashion, published at IJCAI 2024. This work is already proving to have significant implications for the rest of the project and the research area in general. Moreover, we coined the term of 'active measuring' in reinforcement learning. Our techniques are able to actively exploit a system's potentially very expensive sensor, if the uncertainty is too high to make safe and correct decisions. One of the results in this stream has been published at AAAI 2024. Finally, we delved into safe and reliable offline reinforcement learning to account for the inherent uncertainty that occurs in offline settings with limited data. The first result handling partial observability is published at AAAI 2023.
