Skip to main content
Go to the home page of the European Commission (opens in new window)
English en
CORDIS - EU research results
CORDIS

Reliable Data-Driven Decision Making in Cyber-Physical Systems

Project description

Taking reinforcement learning to a whole new level

Reinforcement learning (RL) is the training of machine learning models to make a sequence of decisions, with great potential to help increase automation. The EU-funded RADDICS project seeks to overcome key challenges for reliable deployment of RL methods in high-stakes applications. Specifically, it investigates the use of probabilistic models, such as Gaussian processes and deep Bayesian models, to enable safe exploration. To this end, it combines confidence estimates from these models with techniques from robust control theory and formal verification. The research builds on recent breakthrough results on safe Bayesian optimisation, and is demonstrated on three real-world cyber-physical systems applications.

Objective

This ERC project pushes the boundary of reliable data-driven decision making in cyber-physical systems (CPS), by bridging reinforcement learning (RL), nonparametric estimation and robust optimization. RL is a powerful abstraction of decision making under uncertainty and has witnessed dramatic recent breakthroughs. Most of these successes have been in games such as Go - well specified, closed environments that - given enough computing power - can be extensively simulated and explored. In real-world CPS, however, accurate simulations are rarely available, and exploration in these applications is a highly dangerous proposition.

We strive to rethink Reinforcement Learning from the perspective of reliability and robustness required by real-world applications. We build on our recent breakthrough result on safe Bayesian optimization (SAFE-OPT): The approach allows - for the first time - to identify provably near-optimal policies in episodic RL tasks, while guaranteeing under some regularity assumptions that with high probability no unsafe states are visited - even if the set of safe parameter values is a priori unknown.

While extremely promising, this result has several fundamental limitations, which we seek to overcome in this ERC project. To this end we will (1) go beyond low-dimensional Gaussian process models and towards much richer deep Bayesian models; (2) go beyond episodic tasks, by explicitly reasoning about the dynamics and employing ideas from robust control theory and (3) tackle bootstrapping of safe initial policies by bridging simulations and real-world experiments via multi-fidelity Bayesian optimization, and by pursuing safe active imitation learning.

Our research is motivated by three real-world CPS applications, which we pursue in interdisciplinary collaboration: Safe exploration of and with robotic platforms; tuning the energy efficiency of photovoltaic powerplants and safely optimizing the performance of a Free Electron Laser.

Fields of science (EuroSciVoc)

Keywords

Programme(s)

Topic(s)

Funding Scheme

ERC-COG - Consolidator Grant

See all projects funded under this funding scheme

Call for proposal

(opens in new window) ERC-2018-COG

See all projects funded under this call

Host institution

EIDGENOESSISCHE TECHNISCHE HOCHSCHULE ZUERICH
Net EU contribution
€ 1 996 500,00
Address
Raemistrasse 101
8092 Zuerich
Switzerland
Region
Schweiz/Suisse/Svizzera Zürich Zürich
Activity type
Higher or Secondary Education Establishments
Links
Total cost
€ 1 996 500,00

Beneficiaries (1)