Periodic Reporting for period 3 - ScaleOpt (Scaling Methods for Discrete and Continuous Optimization)
Periodo di rendicontazione: 2021-01-01 al 2022-06-30
The project pertains to the theory of algorithms and optimization, the discipline of designing and analyzing computational tools and understanding their efficiency. Linear programming (LP) is probably the most influential optimization model with an immense impact in practice: in supply chain management, chip design, airline scheduling, and public transportation, just to name a few areas. The pursuit of more efficient LP algorithms has been a driving force in the development of optimization theory, establishing beautiful connections to discrete mathematics, geometry, and analysis.
Whereas fast and efficient LP algorithms already exist, it remains an intriguing open question to find a strongly polynomial algorithm for LP, namely, an algorithm where the number of arithmetic operations only depends on the number of variables and constraints, but not on the numerical bit-complexity of the problem description. Currently known “weakly” polynomial algorithms heavily rely on such bit-complexity information. In contrast, a strongly polynomial algorithm would only use the combinatorial structure of the problem, and would be robust against numerical instability. Finding a strongly polynomial algorithm for LP was listed by Fields medalist Smale as one of the most important mathematical challenges for the twenty-first century.
The main objective of the project is to expand the domain of linear and convex programs that are solvable in strongly polynomial time. We aim to identify and map the key geometric and algebraic conditions that can influence the running time of LP algorithms, and to explore and develop computational hardness arguments in this context.
The research combines tools from discrete and continuous optimization: interactions between these fields have led to ground-breaking developments in recent years. Our main technical approach is to further develop discrete and continuous scaling paradigms, advancing new algorithmic tools that can be also used to tackle other problems in optimization.
Whereas fast and efficient LP algorithms already exist, it remains an intriguing open question to find a strongly polynomial algorithm for LP, namely, an algorithm where the number of arithmetic operations only depends on the number of variables and constraints, but not on the numerical bit-complexity of the problem description. Currently known “weakly” polynomial algorithms heavily rely on such bit-complexity information. In contrast, a strongly polynomial algorithm would only use the combinatorial structure of the problem, and would be robust against numerical instability. Finding a strongly polynomial algorithm for LP was listed by Fields medalist Smale as one of the most important mathematical challenges for the twenty-first century.
The main objective of the project is to expand the domain of linear and convex programs that are solvable in strongly polynomial time. We aim to identify and map the key geometric and algebraic conditions that can influence the running time of LP algorithms, and to explore and develop computational hardness arguments in this context.
The research combines tools from discrete and continuous optimization: interactions between these fields have led to ground-breaking developments in recent years. Our main technical approach is to further develop discrete and continuous scaling paradigms, advancing new algorithmic tools that can be also used to tackle other problems in optimization.
We have already attained important project objectives and have made significant progress towards others. Publication highlights include three papers accepted at the flagship conferences in theoretical computer science, the Symposium on Theory of Computing and the Symposium on Foundations of Computer Science, and a paper published in the Journal of the ACM (JACM). We now highlight four areas of achievements.
1. Generalized flow problems: A key objective of the project is to develop a strongly polynomial algorithm for the minimum-cost generalized flow problem, a basic combinatorial optimization model where no such algorithm is known. Our prior work with Olver developed the first such algorithm for the corresponding flow maximization problem; the improved and strengthened variant of the algorithm was recently published in the JACM. The dual problem amounts to solving two variable per inequality linear systems (TVPI). Our recent work with Koh and Natura developed a novel strongly polynomial approach for this problem, applying the discrete Newton algorithm. This method, together with new insights from interior point methods as discussed next, provides a viable approach for the general minimum-cost flow problem.
2. Advancing combinatorial interior point methods: In two recent papers with Dadush, Huiberts, and Natura, we have significantly broadened the class of linear programs solvable in strongly polynomial time, and improved the complexity of such algorithms. In the first paper, we resolved a longstanding open question on finding a scaling invariant version of the `layered-least-squares’ interior point method. This class of interior point methods was introduced by Vavasis and Ye in 1996. The significance in the context of strongly polynomial computability is that the running time only depends on a certain condition number associated with the constraint matrix, but not on the right-hand side and cost vectors. The dependence is on the Stewart–Todd (ST) condition number that captures a certain geometric property of the associated linear subspace.
The running time dependence of our new algorithm is on the best possible value of the ST condition number attainable by column scaling that can be arbitrarily better than the original value of the measure. This appears to be the best dependence one can hope for in this class. Our main tool is a new, combinatorial characterization of the ST measure in terms of circuit imbalances, by using tools from matroid optimization. Our second paper devises a `black-box’ algorithm with the same condition number dependence and enables to turn the state-of-art approximate LP solvers to fast algorithms with running time only dependent on the optimized ST measure.
3. Strongly polynomial algorithms for convex programs: One cannot hope for strongly polynomial algorithms for convex programming in general; such algorithms can be possible only for restricted classes of rational convex programs, and only few examples are known. A particularly interesting family of rational convex programs arise in market equilibrium computation. A main open question in the research proposal was to find a strongly polynomial algorithm for linear Arrow–Debreu exchange markets, a remaining classical model where no such algorithm was known. We succeeded to resolve this question; a key new methodology entails approximating certain LPs by LPs of a simpler form; this technique might turn out to be useful in other areas.
4. Connections with tropical geometry: Tropical geometry originally arose in the context of algebraic geometry, and has profound connections to various branches of mathematics. A breakthrough result by Allamigeon et al. showed the impossibility of path following interior point methods being strongly polynomial for general LP using techniques from tropical geometry. Tropical geometry appears to be a key tool to understand the possible hardness limits of strongly polynomial computability.
In a number of papers with Loho, Celaya, Yuen, Schymura, Smith, and Sanyal, we develop tropical counterparts of concepts and results in polyhedral combinatorics. These include the notion of signed tropical convexity, pivoting algorithms, lattice point counting, volumes, and the tropical Carathéodory theorem.
1. Generalized flow problems: A key objective of the project is to develop a strongly polynomial algorithm for the minimum-cost generalized flow problem, a basic combinatorial optimization model where no such algorithm is known. Our prior work with Olver developed the first such algorithm for the corresponding flow maximization problem; the improved and strengthened variant of the algorithm was recently published in the JACM. The dual problem amounts to solving two variable per inequality linear systems (TVPI). Our recent work with Koh and Natura developed a novel strongly polynomial approach for this problem, applying the discrete Newton algorithm. This method, together with new insights from interior point methods as discussed next, provides a viable approach for the general minimum-cost flow problem.
2. Advancing combinatorial interior point methods: In two recent papers with Dadush, Huiberts, and Natura, we have significantly broadened the class of linear programs solvable in strongly polynomial time, and improved the complexity of such algorithms. In the first paper, we resolved a longstanding open question on finding a scaling invariant version of the `layered-least-squares’ interior point method. This class of interior point methods was introduced by Vavasis and Ye in 1996. The significance in the context of strongly polynomial computability is that the running time only depends on a certain condition number associated with the constraint matrix, but not on the right-hand side and cost vectors. The dependence is on the Stewart–Todd (ST) condition number that captures a certain geometric property of the associated linear subspace.
The running time dependence of our new algorithm is on the best possible value of the ST condition number attainable by column scaling that can be arbitrarily better than the original value of the measure. This appears to be the best dependence one can hope for in this class. Our main tool is a new, combinatorial characterization of the ST measure in terms of circuit imbalances, by using tools from matroid optimization. Our second paper devises a `black-box’ algorithm with the same condition number dependence and enables to turn the state-of-art approximate LP solvers to fast algorithms with running time only dependent on the optimized ST measure.
3. Strongly polynomial algorithms for convex programs: One cannot hope for strongly polynomial algorithms for convex programming in general; such algorithms can be possible only for restricted classes of rational convex programs, and only few examples are known. A particularly interesting family of rational convex programs arise in market equilibrium computation. A main open question in the research proposal was to find a strongly polynomial algorithm for linear Arrow–Debreu exchange markets, a remaining classical model where no such algorithm was known. We succeeded to resolve this question; a key new methodology entails approximating certain LPs by LPs of a simpler form; this technique might turn out to be useful in other areas.
4. Connections with tropical geometry: Tropical geometry originally arose in the context of algebraic geometry, and has profound connections to various branches of mathematics. A breakthrough result by Allamigeon et al. showed the impossibility of path following interior point methods being strongly polynomial for general LP using techniques from tropical geometry. Tropical geometry appears to be a key tool to understand the possible hardness limits of strongly polynomial computability.
In a number of papers with Loho, Celaya, Yuen, Schymura, Smith, and Sanyal, we develop tropical counterparts of concepts and results in polyhedral combinatorics. These include the notion of signed tropical convexity, pivoting algorithms, lattice point counting, volumes, and the tropical Carathéodory theorem.
The results attained thus far, along with ongoing work have the promise to attain the following main objectives by the end of the project. In terms of combinatorial LP classes, we aim to find strongly polynomial algorithms for minimum-cost generalized flows, as well as for classes of undiscounted Markov Decision Processes.
In another line of ongoing work, we aim to harness recent advancements in fast approximate solvers to improve on the best state of art strongly polynomial running times on important classes of problems, such as flows and matchings.
In terms of convex programming, we expect to devise an algorithm for separable convex quadratic programming with linear constraints with running time depending only on the size of the problem and the optimized value of the ST-condition measure. This requires the combination of multiple novel techniques developed in the project, and goes well beyond the expectations laid out in the research plan.
Another ongoing research direction addresses the Nash Social Welfare problem, a natural problem of fair division arising in mathematical economics. By combining tools from combinatorial optimization and discrete convex analysis, we expect to extend the current constant factor approximability results to a significantly broader class of valuation functions.
In tropical geometry, our vision is to establish formal hardness reductions between polynomial time solvability of tropical LPs and strongly polynomial solvability between classes of standard LPs: such a result could lead to a new type of hardness theory and could have far reaching consequences.
In another line of ongoing work, we aim to harness recent advancements in fast approximate solvers to improve on the best state of art strongly polynomial running times on important classes of problems, such as flows and matchings.
In terms of convex programming, we expect to devise an algorithm for separable convex quadratic programming with linear constraints with running time depending only on the size of the problem and the optimized value of the ST-condition measure. This requires the combination of multiple novel techniques developed in the project, and goes well beyond the expectations laid out in the research plan.
Another ongoing research direction addresses the Nash Social Welfare problem, a natural problem of fair division arising in mathematical economics. By combining tools from combinatorial optimization and discrete convex analysis, we expect to extend the current constant factor approximability results to a significantly broader class of valuation functions.
In tropical geometry, our vision is to establish formal hardness reductions between polynomial time solvability of tropical LPs and strongly polynomial solvability between classes of standard LPs: such a result could lead to a new type of hardness theory and could have far reaching consequences.