Periodic Reporting for period 1 - DistOpt-BydWorstCase (Distributed Optimization Beyond Worst-Case Topologies) Reporting period: 2020-09-01 to 2022-02-28 Summary of the context and overall objectives of the project Modern computer systems are increasingly decentralized and massively distributed computations play a vital role in the systems of the future. This project aims to advance the foundational aspects of distributed computing, particularly MessagePassing algorithms for optimization problems. The strong shift towards distributed systems also lead to a flurry of works on such algorithms -- many optimization problems now have distributed algorithms with optimal worst-case performance guarantees. Generally, these guarantees cannot be improved because they match unconditional impossibility results, which prove severe limitations on the performance of distributed algorithms in some (pathological) network topologies. Real world networks, however, are never worst-case and do not share the limiting bottleneck characteristics of these pathological topologies. In fact, there is no known barrier for ultra-fast polylogarithmic-round distributed algorithms on any network of interest. This leaves an exponential gap between current worst-case-optimal algorithms and what is likely possible in many, if not all, real-world settings.Motivated by this, this project pursues a program for a general toolbox and theory for MessagePassing optimization algorithms that go beyond worst-case topologies. The main and guiding high-risk high-gain goal is the development of universally optimal distributed algorithms, which are competitive with the best algorithm on any given topology. This would constitute the strongest possible form of algorithmically adjusting to non-worst-case topologies and contribute to and strengthen the (theoretical) foundation to the modern and future distributed systems society relies upon. The project follows a detailed program with many concrete and smaller stepping stones towards this ambitious breakthrough objective. Many of the novel questions stemming from this program and proposal are highly interdisciplinary, crossing boundaries between information theory, distributed computing, topological graph theory, and other parts of theoretical computer science, and are fundamental in their own right. Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far Work on this project has begun immediately with the starting date of the award in 2020. The PI has spent the vast majority of their effort on this project. COVID-19 made in-person collaborations and the recruitment of personnel to this project challenging but the project managed to bring a finishing PhD student to ETHZ for seven months and officially recruit a postdoc to this project. This postdoc worked and significantly contributed to this ERC action as an unpaid collaborator long before being funded through this project. Their official start date on this project is January 2022. Collaborations on research for this ERC action (but not paid through this ERC) have been formed with multiple PhD students at ETHZ, and several postdocs and professors at ETHZ, EU-institutions, as well as US institutions. One ETHZ MSc thesis on this ERC action was started in 2021 and completed in 2022.Significant research results beyond the state of the art were obtained as a result of this effort and published in highly-competitive peer-reviewed venues (see below). The PI and other team members also widely disseminated the research results obtained and the general research direction set forth by this project in various talks at conferences, workshops, and invited talks at universities and industrial research labs. This includes keynotes by the PI at ALGO 2021 and the International Conference on Distributed Computing (DISC 2021). Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far) The project let to the first universally optimal distributed algorithms in the known topology setting and to break-through results in hop-constrained oblivious routing which, as explained in Section B.5 (page 95 of the grant agreement) are tightly linked to efficient distributed constructions of low-congestion shortcuts. New and powerful tree-embeddings for hop-constrained distances needed to be developed to construct these oblivious routings. These (partial) tree-embeddings have led to significant further results in hop-constrained versions of classical online and approximation algorithms for network design. These significant results beyond the state of art have let to many publications at the top venues in the area (STOC, FOCS, SODA, PODC) including five papers directly stemming from this project accepted to STOC in 2021&22.Progress on this project has been so successful that it is expected that even the most ambitious goals outlined in the proposal might be achievable until the end of the project. Further efforts and results building on these new foundations as well as work on implications and applications of the newly developed tools are already in progress.
