Periodic Reporting for period 2 - NonnegativeRank (Geometry of Nonnegative Rank)
Reporting period: 2018-09-01 to 2019-08-31
This project is centered on nonnegative rank - a notion that modifies the definition of matrix rank. Our goal is to do basic research aimed at better understanding the geometry of matrices and tensors of nonnegative rank at most r. Nonnegative rank appears in various applications such as statistics, machine learning, audio processing, image compression and document analysis. Understanding the geometry of nonnegative rank is fundamental in its own right and important for the theory and algorithms behind the applications. The overall objectives of this project are investigation of zero patterns in nonnegative matrix factorizations, understanding the boundaries and semialgebraic descriptions of the set of matrices and tensors of nonnegative rank r for small r, and implications of this theory to applications in statistics and other fields. We concluded that tools from (algebraic) geometry, rigidity theory and optimization are essential for studying nonnegative rank and related notions, and that especially the study of uniqueness and identifiability aspects benefits enormously from geometric tools.
In collaboration with Robert Krone, we use ideas from rigidity theory to study uniqueness of nonnegative matrix factorizations in the case when nonnegative rank of a matrix is equal to its rank. We characterize infinitesimally rigid nonnegative factorizations, prove that a nonnegative factorization is infinitesimally rigid if and only if it is locally rigid and a certain matrix achieves its maximal possible Kruskal rank, and show that locally rigid nonnegative factorizations can be extended to globally rigid nonnegative factorizations. We also explore connections between rigidity of nonnegative factorizations and boundaries of the set of matrices of fixed nonnegative rank. Finally we extend these results from nonnegative factorizations to completely positive factorizations. These results appear in a preprint on arXiv and are submitted for publication.
One direction in the study of nonnegative rank is motivated by linear relaxations for hard combinatorial optimization problems. Nonnegative rank of the slack matrix of the feasible region of the relaxation measures the complexity of the linear relaxation and hence provides lower bounds for the complexity of the original optimization problem. In joint article with Per Austrin and Petteri Kaski, we study another restricted model of computation which is given by tensor networks for evaluating multilinear maps. We show that this model of computation captures the best algorithms for several problems, such as matrix multiplication, discrete Fourier transform, (3t)-clique counting and computing the permanent of a matrix. For counting homomorphisms of a general pattern graph P into a host graph on n vertices we obtain an upper bound that essentially matches the bound for counting cliques and yields small improvements over previous algorithms for many choices of P. There results are published in Proceedings of 10th Innovations in Theoretical Computer Science Conference.
In joint article with Elizabeth Allman, Hector Banos Cervantes, Robin Evans, Serkan Hosten, Daniel Lemke, John Rhodes and Piotr Zwiernik, we study binary tensors of nonnegative rank at most two and three. We give the boundary stratification of binary tensors of nonnegative rank at most two. For small tensors, we show that this stratification can be used for exact computation of maximum likelihood estimates (MLEs). This method guarantees finding the MLE whereas the EM algorithm does not. We show how the EM fixed point ideal provides an alternative method for obtaining the boundary decomposition and for computing MLEs. These results are published in Journal of Algebraic Statistics.
In the joint paper with Anastasiya Belyaeva, Lawrence J. Sun and Caroline Uhler, we study the problem of reconstructing the 3D organization of the genome from such whole-genome contact frequencies. We prove that the 3D organization of the DNA is not identifiable from pairwise distance measurements derived from Hi-C for diploid organisms. In fact, there are infinitely many solutions even in the noise-free setting. We then discuss various additional biologically relevant constraints and prove identifiability under these conditions. Finally, we provide SDP formulations for computing the 3D embedding of the DNA with these additional constraints and show that we can recover the true 3D embedding with high accuracy also from both noiseless and noisy measurements. These formulations minimize the trace of a Gram matrix as an approximation of rank minimization and finally use eigendecomposition to get a rank three approximation of the Gram matrix. These results are in the last stages of preparation and will be submitted in the near future.
In joint article with Carlos Amendola and Dimitra Kosta, we study the maximum likelihood estimation problem for several classes of toric Fano models. We start by exploring the maximum likelihood degree for all 2-dimensional Gorenstein toric Fano varieties. We then explore the reasons for the ML degree drop using A-discriminants and intersection theory. Finally, we show that toric Fano varieties associated to 3-valent phylogenetic trees have ML degree one and provide a formula for the maximum likelihood estimate. We prove it as a corollary to a more general result about the multiplicativity of ML degrees of codimension zero toric fiber products, and it also follows from a connection to a recent result about staged trees. These results appear in a preprint on arXiv and are submitted for publication.
These results have been presented at international seminars, workshops and conferences.
One direction in the study of nonnegative rank is motivated by linear relaxations for hard combinatorial optimization problems. Nonnegative rank of the slack matrix of the feasible region of the relaxation measures the complexity of the linear relaxation and hence provides lower bounds for the complexity of the original optimization problem. In joint article with Per Austrin and Petteri Kaski, we study another restricted model of computation which is given by tensor networks for evaluating multilinear maps. We show that this model of computation captures the best algorithms for several problems, such as matrix multiplication, discrete Fourier transform, (3t)-clique counting and computing the permanent of a matrix. For counting homomorphisms of a general pattern graph P into a host graph on n vertices we obtain an upper bound that essentially matches the bound for counting cliques and yields small improvements over previous algorithms for many choices of P. There results are published in Proceedings of 10th Innovations in Theoretical Computer Science Conference.
In joint article with Elizabeth Allman, Hector Banos Cervantes, Robin Evans, Serkan Hosten, Daniel Lemke, John Rhodes and Piotr Zwiernik, we study binary tensors of nonnegative rank at most two and three. We give the boundary stratification of binary tensors of nonnegative rank at most two. For small tensors, we show that this stratification can be used for exact computation of maximum likelihood estimates (MLEs). This method guarantees finding the MLE whereas the EM algorithm does not. We show how the EM fixed point ideal provides an alternative method for obtaining the boundary decomposition and for computing MLEs. These results are published in Journal of Algebraic Statistics.
In the joint paper with Anastasiya Belyaeva, Lawrence J. Sun and Caroline Uhler, we study the problem of reconstructing the 3D organization of the genome from such whole-genome contact frequencies. We prove that the 3D organization of the DNA is not identifiable from pairwise distance measurements derived from Hi-C for diploid organisms. In fact, there are infinitely many solutions even in the noise-free setting. We then discuss various additional biologically relevant constraints and prove identifiability under these conditions. Finally, we provide SDP formulations for computing the 3D embedding of the DNA with these additional constraints and show that we can recover the true 3D embedding with high accuracy also from both noiseless and noisy measurements. These formulations minimize the trace of a Gram matrix as an approximation of rank minimization and finally use eigendecomposition to get a rank three approximation of the Gram matrix. These results are in the last stages of preparation and will be submitted in the near future.
In joint article with Carlos Amendola and Dimitra Kosta, we study the maximum likelihood estimation problem for several classes of toric Fano models. We start by exploring the maximum likelihood degree for all 2-dimensional Gorenstein toric Fano varieties. We then explore the reasons for the ML degree drop using A-discriminants and intersection theory. Finally, we show that toric Fano varieties associated to 3-valent phylogenetic trees have ML degree one and provide a formula for the maximum likelihood estimate. We prove it as a corollary to a more general result about the multiplicativity of ML degrees of codimension zero toric fiber products, and it also follows from a connection to a recent result about staged trees. These results appear in a preprint on arXiv and are submitted for publication.
These results have been presented at international seminars, workshops and conferences.
Nonnegative matrix factorizations are often encountered in data mining applications where they are used to explain datasets by a small number of parts. For many of these applications it is desirable that there exists a unique nonnegative matrix factorization up to trivial modifications given by scalings and permutations. The uniqueness of nonnegative matrix factorizations has been widely studied in the literature. Rigidity theory provides a novel approach for studying uniqueness of nonnegative factorizations that allows to establish new results on uniqueness and boundaries. Complete boundary stratification of small tensors of nonnegative rank at most two demonstrates how this information can be used to find maximum likelihood estimates with guarantee.
It is very difficult to prove lower bounds on complexity for canonical NP-complete problems and models of computation provide an alternative for proving lower bounds in restrictive settings. Two well-known examples are linear and semidefinite models of computation that have nonnegative and positive semidefinite rank as the measures of complexity. Tensor networks provide another model of computation that is general enough to capture best algorithms for several problems.
As the spatial organization of the DNA plays an important role for gene regulation, DNA replication, and genomic integrity, we hope that our theoretical results will have an impact on 3D genome reconstruction of diploid organisms from whole-genome contact frequencies by biologists.
The multiplicative behavior of maximum likelihood estimation on codimension-0 toric fiber products generalizes similar results for undirected graphical models and staged trees.
It is very difficult to prove lower bounds on complexity for canonical NP-complete problems and models of computation provide an alternative for proving lower bounds in restrictive settings. Two well-known examples are linear and semidefinite models of computation that have nonnegative and positive semidefinite rank as the measures of complexity. Tensor networks provide another model of computation that is general enough to capture best algorithms for several problems.
As the spatial organization of the DNA plays an important role for gene regulation, DNA replication, and genomic integrity, we hope that our theoretical results will have an impact on 3D genome reconstruction of diploid organisms from whole-genome contact frequencies by biologists.
The multiplicative behavior of maximum likelihood estimation on codimension-0 toric fiber products generalizes similar results for undirected graphical models and staged trees.