Periodic Reporting for period 4 - CORNEA (Controlling evolutionary dynamics of networked autonomous agents)
Período documentado: 2022-11-01 hasta 2023-04-30
Large-scale technological, biological, economic, and social complex systems act as complex networks of interacting autonomous agents. Large numbers of interacting agents making self-interested decisions can result in highly complex, sometimes surprising, and often suboptimal, collective behaviors. Empowered by recent breakthroughs in data-driven cognitive learning technologies, networked agents collectively give rise to evolutionary dynamics that cannot be easily modelled, analysed and/or controlled using current systems and control theory. Consequently, there is an urgent need to develop new theoretical foundations to tackle the emerging challenging control problems associated with evolutionary dynamics for networked autonomous agents.
The wide spectrum of the research theme from theory to experimentation and the multi-disciplinary nature of the research approach lead to exciting advances in control science and engineering bringing scientific, technological and economic benefits. In engineering, the improved control of man-made autonomous robotic systems leads to new applications of environmental sampling and monitoring tasks. The control of large-scale, distributed engineering networks such as traffic networks and smart energy grids, once cast as a large number of coupled agents with autonomous decision-making capabilities, can also benefit from the control algorithms developed in this project. The overall objective of this project is to develop a rigorous theory for the control of evolutionary dynamics so that interacting autonomous agents can be guided to solve group tasks through the pursuit of individual goals in an evolutionary dynamical process. The theory is then tested, validated and improved against experimental results using autonomous robots.
The wide spectrum of the research theme from theory to experimentation and the multi-disciplinary nature of the research approach lead to exciting advances in control science and engineering bringing scientific, technological and economic benefits. In engineering, the improved control of man-made autonomous robotic systems leads to new applications of environmental sampling and monitoring tasks. The control of large-scale, distributed engineering networks such as traffic networks and smart energy grids, once cast as a large number of coupled agents with autonomous decision-making capabilities, can also benefit from the control algorithms developed in this project. The overall objective of this project is to develop a rigorous theory for the control of evolutionary dynamics so that interacting autonomous agents can be guided to solve group tasks through the pursuit of individual goals in an evolutionary dynamical process. The theory is then tested, validated and improved against experimental results using autonomous robots.
We have studied carefully the relationship between the agent-level strategy-updating and the population-level stability. The highlight is that we have proposed a completely new network dynamics model that has two layers, one for opinion dynamics and the other for collective decision-making.
We have also further developed the concept of accessibility and controllability for nonlinear complex networked systems. For example, we examined how structural properties can be utilized for nonlinear balanced equations, and make a new insightful connection between observability and privacy. With this new understanding of dynamics of agents in networks, we naturally introduce control input into the systems. We found that incentives are effective in guiding the behavior of large networks of agents. We have shown how to design incentives for binary decision networks and in the follow-up works, we will look into other types of decision-making dynamics. Although stochastic stability has been a classic topic studied in the past decades, we have shown that we can relax the technical requirement for being monotonic and obtained new Lyapunov criteria. The Lyapunov approach is particularly suitable for studying networks, when the dynamics at the nodes can be modeled by oscillators; we have published several papers on oscillator networks.
Because games are a central part in the decision-making models that we study, we have looked into various Nash equilibrium seeking algorithms in games. We have also studied how one or a few game players might be able to manipulate the outcome of the games, which has led to exciting new manipulation strategies. The game play can be further embedded into a changing environment, and thus the game play and the environment become a closed-loop system, whose stable equilibria or oscillatory trajectories may be directly interpreted by looking into the phase portraits of these dynamical systems. It is even more exciting to combine learning algorithms with game plays, and examine how historical data on game plays can help to choose the current decision-making strategies. We have shown that indeed learning can help players to be more cooperative, which discloses great potential to use learning to resolve challenging social dilemmas.
We have also further developed the concept of accessibility and controllability for nonlinear complex networked systems. For example, we examined how structural properties can be utilized for nonlinear balanced equations, and make a new insightful connection between observability and privacy. With this new understanding of dynamics of agents in networks, we naturally introduce control input into the systems. We found that incentives are effective in guiding the behavior of large networks of agents. We have shown how to design incentives for binary decision networks and in the follow-up works, we will look into other types of decision-making dynamics. Although stochastic stability has been a classic topic studied in the past decades, we have shown that we can relax the technical requirement for being monotonic and obtained new Lyapunov criteria. The Lyapunov approach is particularly suitable for studying networks, when the dynamics at the nodes can be modeled by oscillators; we have published several papers on oscillator networks.
Because games are a central part in the decision-making models that we study, we have looked into various Nash equilibrium seeking algorithms in games. We have also studied how one or a few game players might be able to manipulate the outcome of the games, which has led to exciting new manipulation strategies. The game play can be further embedded into a changing environment, and thus the game play and the environment become a closed-loop system, whose stable equilibria or oscillatory trajectories may be directly interpreted by looking into the phase portraits of these dynamical systems. It is even more exciting to combine learning algorithms with game plays, and examine how historical data on game plays can help to choose the current decision-making strategies. We have shown that indeed learning can help players to be more cooperative, which discloses great potential to use learning to resolve challenging social dilemmas.
We have worked on a class of new game-play strategies, called zero-determinant strategies. In particular, we have focused on multi-player games. Many of today’s most pressing societal concerns require decisions which take into account a distant and uncertain future. Recent developments in strategic decision-making suggest that individuals, or a small group of individuals, can unilaterally influence the collective outcome of complex social dilemmas. However, these results do not account for the extent to which decisions are moderated by uncertainty in the probability or timing of future outcomes that characterize the valuation of a (distant) uncertain future. We have developed a general framework that captures interactions among uncertainty, the resulting time-inconsistent discounting, and their consequences for decision-making processes. In deterministic limits, existing theories can be recovered. More importantly, new insights have been obtained into the possibilities for strategic influence when the valuation of the future is uncertain. We have shown that in order to unilaterally promote and sustain cooperation in social dilemmas, decisions of generous and extortionate strategies should be adjusted to the level of uncertainty. In particular, generous payoff relations cannot be enforced during periods of greater risk (which we term the “generosity gap”), unless the strategic enforcer orients their strategy towards a more distant future by consistently choosing “selfless” cooperative decisions; likewise, the possibilities for extortion are directly limited by the level of uncertainty. Our results have implications for policies that aim at solving societal concerns with consequences for a distant future and provides a theoretical starting point for investigating how collaborative decision-making can help solve long-standing societal dilemmas.
We have identified a surprising relationship between observability and privacy. As a quantitative criterion for privacy of “mechanisms” in the form of data-generating processes, the concept of differential privacy was first proposed in computer science and has later been applied to linear dynamical systems. However, differential privacy has not been studied in depth together with other properties of dynamical systems, and it has not been fully utilized for controller design. We have clarified that a classical concept in systems and control, input observability (sometimes referred to as left invertibility) has a strong connection with differential privacy. In particular, we have shown that the Gaussian mechanism can be made highly differentially private by adding small noise if the corresponding system is less input observable. In addition, enabled by our new insight into privacy, we have developed a method to design dynamic controllers for the classic tracking control problem while addressing privacy concerns. We call the obtained controller through our design method the privacy preserving controller.
We have identified a surprising relationship between observability and privacy. As a quantitative criterion for privacy of “mechanisms” in the form of data-generating processes, the concept of differential privacy was first proposed in computer science and has later been applied to linear dynamical systems. However, differential privacy has not been studied in depth together with other properties of dynamical systems, and it has not been fully utilized for controller design. We have clarified that a classical concept in systems and control, input observability (sometimes referred to as left invertibility) has a strong connection with differential privacy. In particular, we have shown that the Gaussian mechanism can be made highly differentially private by adding small noise if the corresponding system is less input observable. In addition, enabled by our new insight into privacy, we have developed a method to design dynamic controllers for the classic tracking control problem while addressing privacy concerns. We call the obtained controller through our design method the privacy preserving controller.