Data-Efficient Scalable Reinforcement Learning for Practical Robotic Environments

Projektinformationen

DESlRE

ID Finanzhilfevereinbarung: 798321

DOI

10.3030/798321

Projekt abgeschlossen

EK-Unterschriftsdatum 19 März 2018

Startdatum 1 April 2018

Enddatum 31 März 2020

Finanziert unter

• EXCELLENT SCIENCE - Marie Skłodowska-Curie Actions

Gesamtkosten

€ 159 460,80

EU-Beitrag

€ 159 460,80

Koordiniert durch

MAX-PLANCK-GESELLSCHAFT ZUR FORDERUNG DER WISSENSCHAFTEN EV
Germany

Periodic Reporting for period 1 - DESlRE (Data-Efficient Scalable Reinforcement Learning for Practical Robotic Environments)

Berichtszeitraum: 2018-04-01 bis 2020-03-31

Zusammenfassung vom Kontext und den Gesamtzielen des Projekts

The initial aim of the project was to develop algorithms suitable for challenging control tasks. Current algorithms that perform well in simulation typically transfers suboptimally to real test or robots. For example, one of the hot topics in research is termed- sim-to-real transfer, which aims to transfer amazing feats that algorithms can achieve in computer simulations to test time performance. If we can understand how to perform control in a highly stochastic environment, many problems in social decision-making can be solved. The overall objectives are to advance our understanding of the difficulty of such an application of algorithms and developing new ones.

Arbeit, die ab Beginn des Projekts bis zum Ende des durch den Bericht erfassten Berichtszeitraums geleistet wurde, und die wichtigsten bis dahin erzielten Ergebnisse

The fellow has developed various new optimization and control algorithms that are designed to handle the hybrid and stochastic environment. Our theory connects to the deep theory of robust optimization and robust control, as well as the robustness of machine learning algorithms such as kernel method. One of the most significant research outcomes was the newly proposed framework of kernel distributionally robust optimization algorithm. This elegant framework is a combination of principled robust optimization theory and kernel machine learning.

Fortschritte, die über den aktuellen Stand der Technik hinausgehen und voraussichtliche potenzielle Auswirkungen (einschließlich der bis dato erzielten sozioökonomischen Auswirkungen und weiter gefassten gesellschaftlichen Auswirkungen des Projekts)

Fellow's work presents new insights combining the theory of robust convex optimization and RKHS. It shows the theory of kernel methods can be used to make robust decisions for general decision-making problems. The work adds an interesting piece to both DRO and kernel method literature.

From a practical perspective, we have proposed easy-to-implement algorithms. As we discussed in recent works, one strength of our methods is its wide applicability. Many of today's learning tasks suffer from manifestations of distributional ambiguity. We believe practitioners from industry and business that wish to gain robustness in their learning or decision-making tasks can apply our kernel distributionally robust optimization algorithms.

We aim to design optimization and control algorithm that can hedge against distribution shift