Periodic Reporting for period 2 - CLariNet (A novel control paradigm for large-scale hybrid networks)
Okres sprawozdawczy: 2023-04-01 do 2024-09-30
I will develop efficient on-line control methods for large-scale Networks with Hybrid Dynamics (NHDs) in the presence of uncertainties, where hybrid dynamics refers to a combination of continuous dynamics, mode switches, and/or topology changes. This topic is one of the core fundamental open problems in the field of systems and control. It is also important from a societal point of view as today’s society depends heavily on the reliable and efficient operation of road, railway, electricity, gas, and water networks, all of which are examples of large-scale NHDs.
Control of large-scale NHDs is a very complex problem due to the large size of the networks, the presence of disturbances, and the hybrid dynamics, while a limited computation time is available. State-of-the-art control methods are not suited for large-scale NHDs as they either suffer from computational tractability issues or impose additional restrictions, resulting in a significantly reduced performance. To address this problem, I will create a new on-line control paradigm for large-scale NHDs based on an innovative integration of multi-agent optimization-based and learning-based control, allowing to unite the optimality of optimization-based control with the on-line tractability of learning-based control. I will bridge the gap between optimization-based and learning-based control for NHDs through the use of multi-scale multi-resolution piecewise affine models, explicit consideration of the graph structure of the network, the unique knowledge and experience I have in both optimization-based control and learning-based decision making, and an interdisciplinary integration of approaches from systems and control, computer science, and optimization.
This will result in systematic, very reliable, highly scalable, high-performance on-line control methods for large-scale NHDs. I will demonstrate their feasibility, benefits, and impact for green multi-modal transportation networks and smart multi-energy networks.
Control of large-scale NHDs is a very complex problem due to the large size of the networks, the presence of disturbances, and the hybrid dynamics, while a limited computation time is available. State-of-the-art control methods are not suited for large-scale NHDs as they either suffer from computational tractability issues or impose additional restrictions, resulting in a significantly reduced performance. To address this problem, I will create a new on-line control paradigm for large-scale NHDs based on an innovative integration of multi-agent optimization-based and learning-based control, allowing to unite the optimality of optimization-based control with the on-line tractability of learning-based control. I will bridge the gap between optimization-based and learning-based control for NHDs through the use of multi-scale multi-resolution piecewise affine models, explicit consideration of the graph structure of the network, the unique knowledge and experience I have in both optimization-based control and learning-based decision making, and an interdisciplinary integration of approaches from systems and control, computer science, and optimization.
This will result in systematic, very reliable, highly scalable, high-performance on-line control methods for large-scale NHDs. I will demonstrate their feasibility, benefits, and impact for green multi-modal transportation networks and smart multi-energy networks.
During the reporting period (covering Oct. 1, 2021 to March 31, 2024) the work on the core research lines RL1-6 as well as the application research line RL8 has started, all according the planning.
The first group of 3 PhD students and the postdoc started in Fall 2021, a second group of 3 PhD students has started in Fall 2022 and together with the PI worked on RL1-6 and RL8.
In the mean time the first achievements have been realized including the development of new methods for learning-based control for PWA systems with constraints (RL1, published in Automatica), the development of new network metrics and generalized partitioning algorithms for large-scale networked systems (RL2), an efficient control method for systems with real-valued and discrete dynamics based on reinforcement/supervised learning for discrete-actions and online optimization using real-valued linear programming (RL3), an multi-agent reinforcement learning approach for large-scale systems using distributed MPC as a function approximator (RL4), a scenario reduction algorithm endowed with performance and feasibility guarantees for uncertain linear systems (RL5), a theoretical performance bound for uncertain linear systems comparing an MPC control that uses an estimated model and the ideal infinite-horizon optimal controller
with knowledge of the true system (RL6, published in IEEE Control Systems Letters). Moreover, as regards the applications (RL8) in the field of intelligent transportation systems several methods for integrated learning-based and optimization-based control have been developed and demonstrated in simulation (RL8, published in Control Engineering Practice). In the field of smart energy systems, we have created a publicly available benchmark (including software codes and a dataset) to test distributed control techniques on the electricity network of the European economic area, called European Economic Area Electricity Network Benchmark (EEA-ENB).
The first group of 3 PhD students and the postdoc started in Fall 2021, a second group of 3 PhD students has started in Fall 2022 and together with the PI worked on RL1-6 and RL8.
In the mean time the first achievements have been realized including the development of new methods for learning-based control for PWA systems with constraints (RL1, published in Automatica), the development of new network metrics and generalized partitioning algorithms for large-scale networked systems (RL2), an efficient control method for systems with real-valued and discrete dynamics based on reinforcement/supervised learning for discrete-actions and online optimization using real-valued linear programming (RL3), an multi-agent reinforcement learning approach for large-scale systems using distributed MPC as a function approximator (RL4), a scenario reduction algorithm endowed with performance and feasibility guarantees for uncertain linear systems (RL5), a theoretical performance bound for uncertain linear systems comparing an MPC control that uses an estimated model and the ideal infinite-horizon optimal controller
with knowledge of the true system (RL6, published in IEEE Control Systems Letters). Moreover, as regards the applications (RL8) in the field of intelligent transportation systems several methods for integrated learning-based and optimization-based control have been developed and demonstrated in simulation (RL8, published in Control Engineering Practice). In the field of smart energy systems, we have created a publicly available benchmark (including software codes and a dataset) to test distributed control techniques on the electricity network of the European economic area, called European Economic Area Electricity Network Benchmark (EEA-ENB).
There is a large body of results for distributed and multi-agent control for systems with continuous variables as well as a large body of results for learning-based control and MPC for small-scale hybrid systems. What is missing is a general theory together with efficient computational methods for control of large-scale NHDs. Moreover, up to now the optimization-based control and learning-based control community have conceptually largely been separated, while both optimization-based control and learning-based control each have unique advantages that are essential for effective control of large-scale NHDs. My breakthrough idea is to bridge this gap for NHDs by using PWA models. Moreover, I propose to use a model-based multi-agent control approach with multiple control agents that coordinate their actions using a smart and novel integration of optimization-based and learning-based control approaches. This will result in a new control paradigm for large-scale NHDs, which I call multi-agent Integrated Optimization- and Learning-based (IOL) control.
My team and I will tackle three major challenging research questions that have to be addressed to obtain systematic, efficient, reliable, safe, and scalable multi-agent IOL control methods for large-scale NHDs:
C1: How to deal with the complexity of the NHD control problem, and how to obtain a balanced trade-off between tractability and performance?
C2: How to effectively integrate optimization-based and learning-based control methods for NHDs in such a way that the advantages of both methods are preserved?
C3: How to obtain coordination among the IOL control agents in such a way that all the control agents together contribute to the efficient, cost-effective, and reliable operation of the entire system?
My team and I will tackle three major challenging research questions that have to be addressed to obtain systematic, efficient, reliable, safe, and scalable multi-agent IOL control methods for large-scale NHDs:
C1: How to deal with the complexity of the NHD control problem, and how to obtain a balanced trade-off between tractability and performance?
C2: How to effectively integrate optimization-based and learning-based control methods for NHDs in such a way that the advantages of both methods are preserved?
C3: How to obtain coordination among the IOL control agents in such a way that all the control agents together contribute to the efficient, cost-effective, and reliable operation of the entire system?