Final Report Summary - LBCAD (Lower bounds for combinatorial algorithms and dynamic problems)
The project led to multiple new scientific discoveries. Two highlights that stand out are a constant-factor approximation algorithm for computing edit distance running in sub-quadratic time, and the notion of catalytic computation. Both of these discoveries changed substantially our understanding of how far we can push lower bound and algorithmic techniques. Furthermore, the project resulted in several new data structure lower bounds, defined a combinatorial model for a certain class of algorithms for Boolean Matrix Multiplication and proved lower bounds for it. In addition to that the project provided a host of additional foundational and algorithmic results ranging from new approaches to long-standing open problems on Boolean functions to better understanding of the structure of graphs with applications for algorithms. Underpinning these results are new lower bound and algorithmic techniques that were developed.
We briefly describe two of our results. Edit distance is a measure of similarity of strings. It has numerous applications in various fields such as computational biology, pattern recognition, text processing and information retrieval. One of the results of the project is an algorithm for constant-factor approximation of edit distance that runs in truly sub-quadratic time. This constant-factor approximation algorithms represent a major breakthrough. Based on recent celebrated results in fine-grained complexity it is widely believed that there is no truly sub-quadratic algorithm for computing the edit distance exactly. Researchers were pessimistic also about constructing any constant factor approximation algorithm for the edit distance running in truly sub-quadratic time. The best algorithms to date gave poly-logarithmic approximation in truly sub-quadratic time, and it was known that certain class of algorithmic techniques cannot yield constant factor approximation. We give a truly sub-quadratic algorithm for constant factor approximation of edit distance. The result was presented at the Annual IEEE Symposium on Foundations of Computer Science (FOCS'18), the top conference in theoretical computer science, where it received the Best paper award.
Catalytic computation is an algorithmic technique in which we use memory occupied by data unrelated to the problem, e.g. someone else's data, to perform useful computation without destroying the data. This allows an algorithm to use much less of its own working memory than one would usually expect. For example, we can solve reachability in directed graphs in polynomial time using only O(log n) bits of our own working memory while using polynomial size extra memory regardless of what the extra memory contains (be it, e.g. incompressible or encrypted data). Upon finishing, the algorithm restores the original content of the extra memory. The best known polynomial time algorithms for the same problem that do not use the extra memory require almost linear working space. At this point we do not know the full potential of the concept and the algorithmic techniques but it is already clear that this substantially changes our fundamental understanding of space needed for computation.
We briefly describe two of our results. Edit distance is a measure of similarity of strings. It has numerous applications in various fields such as computational biology, pattern recognition, text processing and information retrieval. One of the results of the project is an algorithm for constant-factor approximation of edit distance that runs in truly sub-quadratic time. This constant-factor approximation algorithms represent a major breakthrough. Based on recent celebrated results in fine-grained complexity it is widely believed that there is no truly sub-quadratic algorithm for computing the edit distance exactly. Researchers were pessimistic also about constructing any constant factor approximation algorithm for the edit distance running in truly sub-quadratic time. The best algorithms to date gave poly-logarithmic approximation in truly sub-quadratic time, and it was known that certain class of algorithmic techniques cannot yield constant factor approximation. We give a truly sub-quadratic algorithm for constant factor approximation of edit distance. The result was presented at the Annual IEEE Symposium on Foundations of Computer Science (FOCS'18), the top conference in theoretical computer science, where it received the Best paper award.
Catalytic computation is an algorithmic technique in which we use memory occupied by data unrelated to the problem, e.g. someone else's data, to perform useful computation without destroying the data. This allows an algorithm to use much less of its own working memory than one would usually expect. For example, we can solve reachability in directed graphs in polynomial time using only O(log n) bits of our own working memory while using polynomial size extra memory regardless of what the extra memory contains (be it, e.g. incompressible or encrypted data). Upon finishing, the algorithm restores the original content of the extra memory. The best known polynomial time algorithms for the same problem that do not use the extra memory require almost linear working space. At this point we do not know the full potential of the concept and the algorithmic techniques but it is already clear that this substantially changes our fundamental understanding of space needed for computation.