Skip to main content

"Scaling Up Reinforcement Learning: Structure Learning, Skill Acquisition, and Reward Shaping"

Objective

"Learning how to act optimally in high-dimensional stochastic dynamic environments is a fundamental problem in many areas of engineering and computer science. The basic setup is that of an agent who interacts with an environment trying to maximize some long term payoff while having access to observations of the state of the environment. A standard approach to solving this problem is the Reinforcement Learning (RL) paradigm in which an agent is trying to improve its policy by interacting with the environment or, more generally, by using different sources of information such as traces from an expert and interacting with a simulator. In spite of several success stories of the RL paradigm, a unified methodology for scaling-up RL has not emerged to date. The goal of this research proposal is to create a methodology for learning and acting in high-dimensional stochastic dynamic environments that would scale up to real-world applications well and that will be useful across domains and engineering disciplines.
We focus on three key aspects of learning and optimization in high dimensional stochastic dynamic environments that are interrelated and essential to scaling up RL. First, we consider the problem of structure learning. This is the problem of how to identify the key features and underlying structures in the environment that are most useful for optimization and learning. Second, we consider the problem of learning, defining, and optimizing skills. Skills are sub-policies whose goal is more focused than solving the whole optimization problem and can hence be more easily learned and optimized. Third, we consider changing the natural reward of the system to obtain desirable properties of the solution such as robustness, adversity to risk and smoothness of the control policy. In order to validate our approach we study two challenging real-world domains: a jet fighter flight simulator and a smart-grid short term control problem."

Call for proposal

ERC-2012-StG_20111012
See other projects for this call

Funding Scheme

ERC-SG - ERC Starting Grant

Host institution

TECHNION - ISRAEL INSTITUTE OF TECHNOLOGY
Address
Senate Building Technion City
32000 Haifa
Israel
Activity type
Higher or Secondary Education Establishments
EU contribution
€ 1 500 000
Principal investigator
Shie Mannor (Prof.)
Administrative Contact
Mark Davison (Mr.)

Beneficiaries (1)

TECHNION - ISRAEL INSTITUTE OF TECHNOLOGY
Israel
EU contribution
€ 1 500 000
Address
Senate Building Technion City
32000 Haifa
Activity type
Higher or Secondary Education Establishments
Principal investigator
Shie Mannor (Prof.)
Administrative Contact
Mark Davison (Mr.)