Periodic Reporting for period 4 - SO-ReCoDi (Spectral and Optimization Techniques for Robust Recovery, Combinatorial Constructions, and Distributed Algorithms)
Okres sprawozdawczy: 2024-03-01 do 2025-02-28
This project seeks new methodologies to model and solve problems of a fundamental nature in unsupervised machine learning and distributed computing, and to build new bridges between pure mathematics and computer science. The project locates itself in the area of theoretical computer science, and its objectives are new insights, new algorithms and new analytical methods, backed by rigorous mathematical proofs.
More specifically, the project develops new ways to apply mathematical methods from linear algebra and from convex optimization to three families of application domains:
1) Combinatorial problems arising in unsupervised machine learning in which one wants to discover structure in seemingly unstructured data, even in the presence of outliers and noise. Here it is the "robustness" of the algorithm to outliers (incorrect data points) which is most challenging and interesting.
2) The computational construction of sparse approximations to networks and of other objects that have certain "pseudorandom" properties. This has been a fertile area of collaboration between pure mathematics and computer science, and, by the study of problems of this nature, powerful methods from pure mathematics have been transferred to computer science (and then spread out to achieve a major impact in other areas of computer science) and vice versa.
3) The analysis of probabilistic processes in networks, motivated by distributed computing, computational social sciences, and network modeling of biological processes. Even in systems with very simple interactions, the global evolution of the system can be rich and complex, leading to the implementation of powerful algorithms by simple, local processes: here the project studies novel modeling and analytical methods.
An innovation of the project is to treat these three very different application domains in a unified way, with the unification provided by the fact that similar methods from linear algebra and from convex optimization apply to all three. One of the aims of the project is to use insights from each domain to develop innovative methods in the others.
The outcomes of the project include new algorithms for all domains and new mathematical techniques for the analysis of discrete dynamical systems. Together, these results help demonstrate the power and interconnectedness of these algorithmic paradigms and the underlying mathematical tool.
More specifically, the project develops new ways to apply mathematical methods from linear algebra and from convex optimization to three families of application domains:
1) Combinatorial problems arising in unsupervised machine learning in which one wants to discover structure in seemingly unstructured data, even in the presence of outliers and noise. Here it is the "robustness" of the algorithm to outliers (incorrect data points) which is most challenging and interesting.
2) The computational construction of sparse approximations to networks and of other objects that have certain "pseudorandom" properties. This has been a fertile area of collaboration between pure mathematics and computer science, and, by the study of problems of this nature, powerful methods from pure mathematics have been transferred to computer science (and then spread out to achieve a major impact in other areas of computer science) and vice versa.
3) The analysis of probabilistic processes in networks, motivated by distributed computing, computational social sciences, and network modeling of biological processes. Even in systems with very simple interactions, the global evolution of the system can be rich and complex, leading to the implementation of powerful algorithms by simple, local processes: here the project studies novel modeling and analytical methods.
An innovation of the project is to treat these three very different application domains in a unified way, with the unification provided by the fact that similar methods from linear algebra and from convex optimization apply to all three. One of the aims of the project is to use insights from each domain to develop innovative methods in the others.
The outcomes of the project include new algorithms for all domains and new mathematical techniques for the analysis of discrete dynamical systems. Together, these results help demonstrate the power and interconnectedness of these algorithmic paradigms and the underlying mathematical tool.
We discuss progress along the three main aims of the project.
1) Convex optimization and spectral techniques for discrete problems:
The main direction of research concerns the power of “spectral” techniques, i.e. those based on linear algebra and convex optimization, as captured by the techniques of linear and semidefinite programming. While both spectral algorithms and convex optimization are widely used in practice, each offers its own advantages, with spectral algorithms being simpler and more widely used, whereas convex optimization is more robust to outliers.
The first set of results concerns the power of linear programming and convex optimization. The PI and collaborators showed that these techniques are somewhat equivalent for certain recovery problems in networks, leading to works at SODA 2020 and FOCS 2020. Group member Chris Jones showed that a certain inference problem on networks cannot be solved by a broad class of semidefinite programming algorithms, which was published in STOC 2023.
The second set of results develops new algorithms for optimization problems using these techniques. The PI and collaborators developed robust algorithms for clustering problems on networks, a result which was published at AISTATS 2022. The PI and several group members studied the problem of optimizing random polynomials. This basic question is of a fundamental nature and led to several new results, published at CCC 2023, SODA 2024, and ICALP 2025. Group member Lucas Pesenti developed a convex optimization algorithm for a basic combinatorial problem known as discrepancy minimization which appeared in SODA 2023.
2) Constructions of sparse approximations
The proposal conjectured the existence of certain new types of sparse approximations of networks and of higher-dimensional analogs of networks called hypergraphs, along with a hypothetical approach to such conjectures. Most of the conjectures were proved by the PI and collaborators, presented at the FOCS 2019 conference.
The PI and collaborators also compared two previously studied definitions of approximation for sparse approximations of networks, known respectively as "cut sparsifiers" and "spectral sparsifiers". They rigorously proved that one notion provides a worse trade-off between approximation and sparsity than the other. These results were presented at SODA 2022.
3) Distributed algorithms
The PI and collaborators analyzed a completely decentralized process that sparsifies a dense network, creating a network in which every node has bounded connections, while preserving good connectivity. The result is that a simple protocol, inspired by real peer-to-peer networking protocols, achieves remarkable performance. This result was presented at SODA 2020.
The PI and collaborators then focused on the question of: is there a methodology analogous to spectral methods that can be applied to networks that change over time, to understand information diffusion? Some preliminary results, in which the authors analyze broadcast processes in dynamic networks with node churn, appeared in ICDCS 2021.
The PI and group members studied how distributed computation algorithms are impacted by effects such as noisy communication and stubborn agents and the graphical structure of the interactions between the individual agents. In detail, the group focuses on information spreading algorithms, studying the runtime of the “minority dynamics”, a highly chaotic dynamical system requiring a challenging analysis. This work appeared in SODA 2024. Other related works were published by group members in IJCAI 2023, DISC 2024, and the Theoretical Computer Science journal.
The group members also focus on the study of distributed models with restricted computation and communication, inspired by biological applications. In particular, this line of research led to several publications by group members published in PODC 2023, DISC 2024, PODC 2024, and Information Processing Letters 2025.
1) Convex optimization and spectral techniques for discrete problems:
The main direction of research concerns the power of “spectral” techniques, i.e. those based on linear algebra and convex optimization, as captured by the techniques of linear and semidefinite programming. While both spectral algorithms and convex optimization are widely used in practice, each offers its own advantages, with spectral algorithms being simpler and more widely used, whereas convex optimization is more robust to outliers.
The first set of results concerns the power of linear programming and convex optimization. The PI and collaborators showed that these techniques are somewhat equivalent for certain recovery problems in networks, leading to works at SODA 2020 and FOCS 2020. Group member Chris Jones showed that a certain inference problem on networks cannot be solved by a broad class of semidefinite programming algorithms, which was published in STOC 2023.
The second set of results develops new algorithms for optimization problems using these techniques. The PI and collaborators developed robust algorithms for clustering problems on networks, a result which was published at AISTATS 2022. The PI and several group members studied the problem of optimizing random polynomials. This basic question is of a fundamental nature and led to several new results, published at CCC 2023, SODA 2024, and ICALP 2025. Group member Lucas Pesenti developed a convex optimization algorithm for a basic combinatorial problem known as discrepancy minimization which appeared in SODA 2023.
2) Constructions of sparse approximations
The proposal conjectured the existence of certain new types of sparse approximations of networks and of higher-dimensional analogs of networks called hypergraphs, along with a hypothetical approach to such conjectures. Most of the conjectures were proved by the PI and collaborators, presented at the FOCS 2019 conference.
The PI and collaborators also compared two previously studied definitions of approximation for sparse approximations of networks, known respectively as "cut sparsifiers" and "spectral sparsifiers". They rigorously proved that one notion provides a worse trade-off between approximation and sparsity than the other. These results were presented at SODA 2022.
3) Distributed algorithms
The PI and collaborators analyzed a completely decentralized process that sparsifies a dense network, creating a network in which every node has bounded connections, while preserving good connectivity. The result is that a simple protocol, inspired by real peer-to-peer networking protocols, achieves remarkable performance. This result was presented at SODA 2020.
The PI and collaborators then focused on the question of: is there a methodology analogous to spectral methods that can be applied to networks that change over time, to understand information diffusion? Some preliminary results, in which the authors analyze broadcast processes in dynamic networks with node churn, appeared in ICDCS 2021.
The PI and group members studied how distributed computation algorithms are impacted by effects such as noisy communication and stubborn agents and the graphical structure of the interactions between the individual agents. In detail, the group focuses on information spreading algorithms, studying the runtime of the “minority dynamics”, a highly chaotic dynamical system requiring a challenging analysis. This work appeared in SODA 2024. Other related works were published by group members in IJCAI 2023, DISC 2024, and the Theoretical Computer Science journal.
The group members also focus on the study of distributed models with restricted computation and communication, inspired by biological applications. In particular, this line of research led to several publications by group members published in PODC 2023, DISC 2024, PODC 2024, and Information Processing Letters 2025.
One project completed during the tenure of the project remains under peer review. Along with visiting professor Salil Vadhan and his student Jake Ruotolo, the group studied algorithms for graph partitioning. One outcome was a new algorithm for the “cut improvement problem”, a subroutine which is widely used in practice.
Group members Isabella Ziccardi and Robin Vacus studied algorithms for the Leader Election, a fundamental problem in distributed systems. They develop an algorithm for Leader Election that can be implemented in various distributed models with many constraints, achieving high efficiency in terms of memory usage and the amount of information required by individual agents. The result was published in PODC 2025.
Group members Isabella Ziccardi and Robin Vacus studied algorithms for the Leader Election, a fundamental problem in distributed systems. They develop an algorithm for Leader Election that can be implemented in various distributed models with many constraints, achieving high efficiency in terms of memory usage and the amount of information required by individual agents. The result was published in PODC 2025.