Final Report Summary - PURE-MAS (Planning under uncertainty for real-world multiagent systems)
The number of intelligent distributed systems that we encounter in daily life increases. A prime example is the upgrading of the electricity grid to a smart grid infrastructure. In a smart grid, each consumer or producer might have a computational entity that schedules the consumption as well as the production of energy. In what way such an agent should plan these activities depends on many factors, such as the inhabitants' pattern of energy use and preferences, and the current and predicted price of electricity.
How to plan efficiently and flexibly in such an inherently dynamic and uncertain context, given that the agent can interact with many other households, is still an open question, and one that is fundamental in the field of Artificial Intelligence (AI). In the PURe-MaS project, we have addressed this problem of automated planning under uncertainty for real-world multiagent systems.
Intelligent decision making in real-world scenarios requires an agent to plan ahead, while taking into account its limitations in sensing and actuation. Our research has been based on a canonical model for these types of planning problems, the Decentralized Partially Observable Markov Decision Process (Dec-POMDP), and on related models, collectively known as Multiagent Sequential Decision Making (MSDM). The project has resulted in several algorithmic advances in Dec-POMDP planners, as well as exploring MSDM methodology in robotic, smart-grid and road maintenance applications.
A key challenge for multiagent planning under uncertainty has traditionally been the issue of scalability, both in terms of planning horizon as well as the number of agents involved in a planning problem. Two of our advances address these important challenges. First, we have proposed one of the fastest optimal Dec-POMDP planners, which can compute optimal plans to planning horizons that were not attained before on many benchmark problems. Second, by exploiting graphical structure present in many problem domains, we have been able to scale up approximate Dec-POMDP planning to unprecedented team sizes.
Besides scalability, several other issues regarding real-world applicability of MSDM have been addressed in the PURe-MaS project. A current limitation of many MSDM methods is that they are offline, and compute carefully tuned joint plans that anticipate any contingency present in the model. In order to pave the way for online planning, we have explored two types of event-based representations. Events capture relevant changes in the environment, and as such provide a higher-level abstraction, that could be exploited for online planning more easily. In addition, we have proposed several algorithms for optimizing the use of communication between agents, taking advantage of the fact that many real-world agents such as robots have the possibility to communicate with each other.
Typically, in multiagent planning under uncertainty agents are assumed to be fully cooperative. In many domains agents are self-interested, however, which we tackled by combining game theory with planning. In the context of coordinating the maintenance activities of multiple contractors (each of which is self-interested), we used techniques from dynamic mechanism design to align the incentives of the different contractors. The resulting planning methodology forms the basis for a serious game, in which human players can learn that cooperation among competitors can be beneficial. At a longer term, these techniques can be used by road authorities to draw up better maintenance contracts.
Our theoretical and algorithmic advances have all been tested in simulation, and some on actual real-world multiagent systems. In particular, we demonstrated how a small team of robots can successfully track a mobile target using POMDP task auctions. The scenario combined uncertainty in acting, sensing and communication, which renders it challenging for classical planning solutions.
Finally, the project explored the application of optimization to smart grids, in an award-winning work on individual and collaborative district optimizations including multiple energy carriers. The work shows the results obtained from the optimization of a district of 200 households with micro-generation units and electric vehicles. Individual optimization is compared with collaborative optimization performed by an aggregator, and the optimization framework is developed for multiple energy carriers: electricity, heat and gas.
Contact details: Matthijs Spaan,
How to plan efficiently and flexibly in such an inherently dynamic and uncertain context, given that the agent can interact with many other households, is still an open question, and one that is fundamental in the field of Artificial Intelligence (AI). In the PURe-MaS project, we have addressed this problem of automated planning under uncertainty for real-world multiagent systems.
Intelligent decision making in real-world scenarios requires an agent to plan ahead, while taking into account its limitations in sensing and actuation. Our research has been based on a canonical model for these types of planning problems, the Decentralized Partially Observable Markov Decision Process (Dec-POMDP), and on related models, collectively known as Multiagent Sequential Decision Making (MSDM). The project has resulted in several algorithmic advances in Dec-POMDP planners, as well as exploring MSDM methodology in robotic, smart-grid and road maintenance applications.
A key challenge for multiagent planning under uncertainty has traditionally been the issue of scalability, both in terms of planning horizon as well as the number of agents involved in a planning problem. Two of our advances address these important challenges. First, we have proposed one of the fastest optimal Dec-POMDP planners, which can compute optimal plans to planning horizons that were not attained before on many benchmark problems. Second, by exploiting graphical structure present in many problem domains, we have been able to scale up approximate Dec-POMDP planning to unprecedented team sizes.
Besides scalability, several other issues regarding real-world applicability of MSDM have been addressed in the PURe-MaS project. A current limitation of many MSDM methods is that they are offline, and compute carefully tuned joint plans that anticipate any contingency present in the model. In order to pave the way for online planning, we have explored two types of event-based representations. Events capture relevant changes in the environment, and as such provide a higher-level abstraction, that could be exploited for online planning more easily. In addition, we have proposed several algorithms for optimizing the use of communication between agents, taking advantage of the fact that many real-world agents such as robots have the possibility to communicate with each other.
Typically, in multiagent planning under uncertainty agents are assumed to be fully cooperative. In many domains agents are self-interested, however, which we tackled by combining game theory with planning. In the context of coordinating the maintenance activities of multiple contractors (each of which is self-interested), we used techniques from dynamic mechanism design to align the incentives of the different contractors. The resulting planning methodology forms the basis for a serious game, in which human players can learn that cooperation among competitors can be beneficial. At a longer term, these techniques can be used by road authorities to draw up better maintenance contracts.
Our theoretical and algorithmic advances have all been tested in simulation, and some on actual real-world multiagent systems. In particular, we demonstrated how a small team of robots can successfully track a mobile target using POMDP task auctions. The scenario combined uncertainty in acting, sensing and communication, which renders it challenging for classical planning solutions.
Finally, the project explored the application of optimization to smart grids, in an award-winning work on individual and collaborative district optimizations including multiple energy carriers. The work shows the results obtained from the optimization of a district of 200 households with micro-generation units and electric vehicles. Individual optimization is compared with collaborative optimization performed by an aggregator, and the optimization framework is developed for multiple energy carriers: electricity, heat and gas.
Contact details: Matthijs Spaan,