## Periodic Reporting for period 1 - OPT4SMART (Distributed Optimization Methods for Smart Cyber-Physical Networks)

**Reporting period:**2015-10-01

**to**2017-03-31

## Summary of the context and overall objectives of the project

Smart communicating devices with their sensing, computing and control capabilities promise to make our society more intelligent, energy-efficient, safe and secure. To achieve these goals the main challenge is to exploit this powerful, but unstructured computational power. The most natural, and widely implemented, centralized computation model is not applicable to this extremely complex system in which processors are spatially distributed and communicate through asynchronous and unreliable communication. Thus, a novel peer-to-peer distributed computational model is appearing as a new opportunity in which a problem is solved cooperatively by peers, rather than by a unique provider that knows and owns all data. Optimization is a mathematical framework involving several research areas ranging from Engineering, Computer Science and Mathematics to Economics and Social Science. In particular, estimation, learning decision and control problems arising in modern scenarios from these areas need to face this computation revolution. A novel peer-to-peer distributed optimization setting is emerging in which processors with local memory, computation and communication capabilities cooperatively solve large-scale, structured optimization problems, without relying on any central coordinator.

The OPT4SMART project has a twofold ambitious goal: (i) to set-up a novel, comprehensive methodological framework to solve distributed optimization problems in peer-to-peer networks, and (ii) to provide numerical methods and toolboxes, based on this framework, to solve distributed estimation, learning, decision and control problems in complex cyber-physical networks. We aim at pursuing this main twofold objective by developing interdisciplinary methodologies arising from a synergic combination of optimization, control-systems, and graph theories. Specifically, the project addresses four main open challenges, whose solution represents a breakthrough for the treatment of distributed optimization over networks. Cyber-physical networks are intrinsically characterized by devices communicating with different time-scales, appearing and disappearing from the network, and by possibly unreliable communication channels. Thus, distributed optimization algorithms have to be designed by explicitly dealing with Asynchronous and unreliable communication. Most of the attention in the distributed optimization literature has been devoted to real-valued convex problems. In this project we will consider more general classes of optimization problems as non-convex, mixed-integer and combinatorial problems, which arise in several estimation, learning, decision and control applications. This century is without doubt the age of massive data. Scientists and analysts agree that the today’s real challenge is to get full value from this amount of available information. Optimization is clearly a significant part of this process so that very-large scale and big data optimization problems need to be solved. Finally, many problems arising in cyber-physical networks have to deal with a dynamic scenario, in which data streams change with time and a real-time computation is needed. Thus, dynamic optimization problems need to be solved by taking into account the above issues regarding communication, set-up and problem-size.

The OPT4SMART project has a twofold ambitious goal: (i) to set-up a novel, comprehensive methodological framework to solve distributed optimization problems in peer-to-peer networks, and (ii) to provide numerical methods and toolboxes, based on this framework, to solve distributed estimation, learning, decision and control problems in complex cyber-physical networks. We aim at pursuing this main twofold objective by developing interdisciplinary methodologies arising from a synergic combination of optimization, control-systems, and graph theories. Specifically, the project addresses four main open challenges, whose solution represents a breakthrough for the treatment of distributed optimization over networks. Cyber-physical networks are intrinsically characterized by devices communicating with different time-scales, appearing and disappearing from the network, and by possibly unreliable communication channels. Thus, distributed optimization algorithms have to be designed by explicitly dealing with Asynchronous and unreliable communication. Most of the attention in the distributed optimization literature has been devoted to real-valued convex problems. In this project we will consider more general classes of optimization problems as non-convex, mixed-integer and combinatorial problems, which arise in several estimation, learning, decision and control applications. This century is without doubt the age of massive data. Scientists and analysts agree that the today’s real challenge is to get full value from this amount of available information. Optimization is clearly a significant part of this process so that very-large scale and big data optimization problems need to be solved. Finally, many problems arising in cyber-physical networks have to deal with a dynamic scenario, in which data streams change with time and a real-time computation is needed. Thus, dynamic optimization problems need to be solved by taking into account the above issues regarding communication, set-up and problem-size.

## Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far

In the first reporting period the activities of OPT4SMART have focused, as planned, on the development of distributed optimization methods taking into account communication limitations (mainly asynchronous protocols), general set-ups beyond classical real-value convex problems, and large-scale and big-data problems according to WP1, and the investigation of estimation and learning problems involving distributed optimization as well as preliminary activities on decision and control (WP2).

As for the activities of WP1 on the development of a research framework for distributed optimization, we can group the most relevant contributions into four main parts.

A first set of contributions deals with the design of distributed algorithms for networks with asynchronous communication, thus going beyond existing methods typically tailored for synchronous networks (often with fixed graphs). We have considered a class of “cost-coupled” optimization problems in which the cost function is the sum of terms all depending on a common decision variable.

Distributed dual-decomposition based algorithms have been proposed for fairly general problems including both composite cost functions and local constraints, and, in particular, for networks with asynchronous communications. Specifically, in the asynchronous version each node performs the computation only when triggered by its local timer or by a message from a neighbourhood, so that no common, central clock is needed. Moreover, for the proposed algorithms no common parameter needs to be set centrally; each node has its own step-size that can be set locally. A key distinctive feature of the algorithm analysis is the combination of duality theory, randomized coordinate-descent methods, and properties of the proximal operator when applied to conjugate functions.

We are also investigating primal methods in collaboration with groups of the University of Padova and Lulea University of Technology. Specifically, the activity is focusing on the development of Newton-based distributed algorithms for asynchronous networks in which packet losses are explicitly taken into account. The proposed approach consists of combining an existing Newton-Raphson consensus approach with a robust (push-sum) consensus scheme proposed in the literature for average computation over networks with lossy communication. Under mild conditions on the (asynchronous) node updates, link failures, connectivity of the communication graph, step-size and smoothness of the cost function, we show that the resulting distributed optimization algorithm is locally exponentially stable with respect to the global solution. The proof is based on time-scale separation and Lyapunov theory. Interestingly, although the algorithm was explicitly proposed for convex problems, being a Newton-based method, it may also be used for non-convex problems in a neighbourhood of a local minimum.

A second set of contributions regards (cost-coupled) big-data, possibly non-convex, problems in which the dimension of the decision variable is “huge” and possibly depending on the network size, so that local computations cannot be performed on the whole decision variable. In a first batch of works, a special class of partitioned big-data problems has been considered in which each local cost function is coupled only with variables of the agent neighbours. Although special, we have shown how this class of problems models interesting estimation and decision problems. Asynchronous dual-decomposition based algorithms have been customized and extended in order to solve (convex) partitioned big-data problems. Then a class of primal algorithms has been proposed for non-convex problems. These algorithms are based on successive convexifications of the local cost functions through information exchange with neighbouring nodes. The convergence analysis relies on tools from coordinate descent methods for (centralized) big-data optimization. All the algorithms proposed for par

As for the activities of WP1 on the development of a research framework for distributed optimization, we can group the most relevant contributions into four main parts.

A first set of contributions deals with the design of distributed algorithms for networks with asynchronous communication, thus going beyond existing methods typically tailored for synchronous networks (often with fixed graphs). We have considered a class of “cost-coupled” optimization problems in which the cost function is the sum of terms all depending on a common decision variable.

Distributed dual-decomposition based algorithms have been proposed for fairly general problems including both composite cost functions and local constraints, and, in particular, for networks with asynchronous communications. Specifically, in the asynchronous version each node performs the computation only when triggered by its local timer or by a message from a neighbourhood, so that no common, central clock is needed. Moreover, for the proposed algorithms no common parameter needs to be set centrally; each node has its own step-size that can be set locally. A key distinctive feature of the algorithm analysis is the combination of duality theory, randomized coordinate-descent methods, and properties of the proximal operator when applied to conjugate functions.

We are also investigating primal methods in collaboration with groups of the University of Padova and Lulea University of Technology. Specifically, the activity is focusing on the development of Newton-based distributed algorithms for asynchronous networks in which packet losses are explicitly taken into account. The proposed approach consists of combining an existing Newton-Raphson consensus approach with a robust (push-sum) consensus scheme proposed in the literature for average computation over networks with lossy communication. Under mild conditions on the (asynchronous) node updates, link failures, connectivity of the communication graph, step-size and smoothness of the cost function, we show that the resulting distributed optimization algorithm is locally exponentially stable with respect to the global solution. The proof is based on time-scale separation and Lyapunov theory. Interestingly, although the algorithm was explicitly proposed for convex problems, being a Newton-based method, it may also be used for non-convex problems in a neighbourhood of a local minimum.

A second set of contributions regards (cost-coupled) big-data, possibly non-convex, problems in which the dimension of the decision variable is “huge” and possibly depending on the network size, so that local computations cannot be performed on the whole decision variable. In a first batch of works, a special class of partitioned big-data problems has been considered in which each local cost function is coupled only with variables of the agent neighbours. Although special, we have shown how this class of problems models interesting estimation and decision problems. Asynchronous dual-decomposition based algorithms have been customized and extended in order to solve (convex) partitioned big-data problems. Then a class of primal algorithms has been proposed for non-convex problems. These algorithms are based on successive convexifications of the local cost functions through information exchange with neighbouring nodes. The convergence analysis relies on tools from coordinate descent methods for (centralized) big-data optimization. All the algorithms proposed for par

## Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far)

Although we are at less than one third of the project schedule, significant scientific contributions have already been given to the state of the art. In particular, OPT4SMART has proposed solutions to address the four main challenges identified in the proposal and reported in the previous sections.

Asynchronous distributed optimization algorithms have been proposed that go beyond, or extend, algorithms available in the literature (that were suited for synchronous communications). Moreover, some of the proposed algorithms also work under unreliable communication (including for example packet losses or link failures). These methods, although at the initial stage of the project, have already contributed to significantly improve the applicability of existing methodologies to real scenarios. Indeed, handling asynchronous communication is a fundamental requirement when dealing with concrete application scenarios in unstructured cyber-physical network systems.

More general problems than the (smooth) real-valued convex problems available in the literature have been addressed. In order to address the challenges coming from applications in modern cyber-physical networks, big data problems have been addressed as well as non-smooth, non-convex and combinatorial problems. In some of the works these two challenges have been addressed simultaneously, that is distributed methods for non-convex big-data optimization problems and for some classes of combinatorial big-data problems have been proposed. It is worth mentioning that proposing and addressing a distributed big-data optimization set-up is already an important contribution per se. Indeed, while big-data optimization has received significant attention in recent years in the area of parallel optimization, almost no contributions were available in the distributed literature. The interest in the parallel optimization literature, as well as important scenarios identified in the project, are clear indications of the relevance of this framework and, thus, of the proposed methodologies.

From the point of view of concrete application scenarios in estimation, learning, decision and control, we are pursuing the ambitious goal of providing both general theoretical frameworks and numerical tools to address and solve problems of interest in modern cyber-physical network systems. Important results have been already obtained.

We have started to set-up a general probabilistic estimation and learning framework, based on a Bayesian approach, which allows agents in a network to estimate, learn, or classify local unknown quantities by properly fusing all the network information. As opposed to the available estimation and learning literature concentrating on the achievement of consensus on a common estimate of unknown parameters, in the framework we are developing, estimating common parameters is only a block of a more general and hierarchical scheme in which agents are able to estimate or learn local unknowns by borrowing strength from the ensemble. This general framework has been already successfully developed to traffic estimation problems, but we are also extending it to more general (interaction-based) learning problems that are relevant in several cyber-physical and social network contexts. Moreover, we have started to propose a set of numerical methods and toolboxes, relying on distributed optimization, to address resource allocation problems in cooperative control scenarios as, e.g., in the context of smart grids and robotic networks.

Overall, this set of methodologies and tools can have, potentially, a strong impact both at a scientific level and in terms of concrete social or industrial software solutions. Indeed, at a scientific level, a new revolution in the optimization area is in act after the one on parallel optimization, namely the construction of a novel distributed methodological framework. The OPT4SMART project is significantly contributing to this revolution by proposing distributed me

Asynchronous distributed optimization algorithms have been proposed that go beyond, or extend, algorithms available in the literature (that were suited for synchronous communications). Moreover, some of the proposed algorithms also work under unreliable communication (including for example packet losses or link failures). These methods, although at the initial stage of the project, have already contributed to significantly improve the applicability of existing methodologies to real scenarios. Indeed, handling asynchronous communication is a fundamental requirement when dealing with concrete application scenarios in unstructured cyber-physical network systems.

More general problems than the (smooth) real-valued convex problems available in the literature have been addressed. In order to address the challenges coming from applications in modern cyber-physical networks, big data problems have been addressed as well as non-smooth, non-convex and combinatorial problems. In some of the works these two challenges have been addressed simultaneously, that is distributed methods for non-convex big-data optimization problems and for some classes of combinatorial big-data problems have been proposed. It is worth mentioning that proposing and addressing a distributed big-data optimization set-up is already an important contribution per se. Indeed, while big-data optimization has received significant attention in recent years in the area of parallel optimization, almost no contributions were available in the distributed literature. The interest in the parallel optimization literature, as well as important scenarios identified in the project, are clear indications of the relevance of this framework and, thus, of the proposed methodologies.

From the point of view of concrete application scenarios in estimation, learning, decision and control, we are pursuing the ambitious goal of providing both general theoretical frameworks and numerical tools to address and solve problems of interest in modern cyber-physical network systems. Important results have been already obtained.

We have started to set-up a general probabilistic estimation and learning framework, based on a Bayesian approach, which allows agents in a network to estimate, learn, or classify local unknown quantities by properly fusing all the network information. As opposed to the available estimation and learning literature concentrating on the achievement of consensus on a common estimate of unknown parameters, in the framework we are developing, estimating common parameters is only a block of a more general and hierarchical scheme in which agents are able to estimate or learn local unknowns by borrowing strength from the ensemble. This general framework has been already successfully developed to traffic estimation problems, but we are also extending it to more general (interaction-based) learning problems that are relevant in several cyber-physical and social network contexts. Moreover, we have started to propose a set of numerical methods and toolboxes, relying on distributed optimization, to address resource allocation problems in cooperative control scenarios as, e.g., in the context of smart grids and robotic networks.

Overall, this set of methodologies and tools can have, potentially, a strong impact both at a scientific level and in terms of concrete social or industrial software solutions. Indeed, at a scientific level, a new revolution in the optimization area is in act after the one on parallel optimization, namely the construction of a novel distributed methodological framework. The OPT4SMART project is significantly contributing to this revolution by proposing distributed me