Periodic Reporting for period 1 - HIGH-HOPeS (Higher-Order Hodge Laplacians for Processing of multi-way Signals)
Reporting period: 2023-01-01 to 2025-06-30
Network analysis has revolutionized our understanding of complex systems, and graph-based methods have emerged as powerful tools to process signals on non-Euclidean domains via graph signal processing and graph neural networks. The graph Laplacian and related matrices are pivotal to such analyses: i) the Laplacian serves as algebraic descriptor of the relationships between nodes; moreover, it is key for the analysis of network structure, for local operations such as averaging over connected nodes, and for network dynamics like diffusion and consensus; ii) Laplacian eigenvectors are natural basis-functions for data on graphs and endowed with meaningful variability notions for graph signals, akin to Fourier analysis in Euclidean domains. However, graphs are ill-equipped to encode multi-way and higher-order relations that are becoming increasingly important to comprehend complex datasets and systems in many applications, e.g. to understand group-dynamics in social systems, multi-gene interactions in genetic data, or multi- way drug interactions.
The goal of this project is to develop methods that can utilize such higher-order relations, going from mathematical models to efficient algorithms and software. Specifically, we will focus on ideas from algebraic topology and discrete calculus, according to which the graph Laplacian can be seen as part of a hierarchy of Hodge-Laplacians that emerge from treating graphs as instances of more general cell complexes that systematically encode couplings between node-tuples of any size. Our ambition is to i) provide more informative ways to represent and analyze the structure of complex systems, paying special attention to computational efficiency; ii) translate the success of graph-based signal processing to data on general topological spaces defined by cell complexes; and iii) by generalizing from graphs to neural networks on complexes, gain deeper theoretical insights on the principles of graph neural networks as special case.
The goal of this project is to develop methods that can utilize such higher-order relations, going from mathematical models to efficient algorithms and software. Specifically, we will focus on ideas from algebraic topology and discrete calculus, according to which the graph Laplacian can be seen as part of a hierarchy of Hodge-Laplacians that emerge from treating graphs as instances of more general cell complexes that systematically encode couplings between node-tuples of any size. Our ambition is to i) provide more informative ways to represent and analyze the structure of complex systems, paying special attention to computational efficiency; ii) translate the success of graph-based signal processing to data on general topological spaces defined by cell complexes; and iii) by generalizing from graphs to neural networks on complexes, gain deeper theoretical insights on the principles of graph neural networks as special case.
We explored several lines or research to achieve the above outlined goals.
We extended ideas from spectral clustering and topological data analysis for the analysis of high-dimensional point cloud data. Graph-based (spectral) clustering of point cloud data typically constructs a proximity graph from the observed data points and then analyses the spectral properties of the resulting graph Laplacian to group points.
Instead, we explored the use of first constructing a (simplicial) complex from the data --- similar to how it is done in topological data analysis --- but instead of considering the (persistent) homology of these complexes, we exploit the spectral properties of the associated Hodge Laplacians to related global topological features to individual (localized) points. This strategy enables us to obtain a richer set of clusters that respect further topological properties of the data (compared to "simple" spectral clustering).
Furthermore we developed several new methods to process signals on graphs and cell complexes and understand current methods for processing signals on graphs. For instance, we showed how over-smoothing can be provably mitigated in graph neural networks and provided a mathematical analysis of the convergence of the training of graph neural networks. We also provided random-walk based architectures to learn from simplicial complexes, developed techniques for outlier-detection of trajectories defined over discrete and discretized surfaces and considered the problem of inference of cell-complexes based on the observation of flows on the edges.
Moreover, we outlined a blueprint for topological deep-learning methods and an associated research agenda together with international collaborators, which provides many avenues for the development of new methods. In conjunction, we contributed to the development of topoX, a suite of python software packages for topological deep learning.
We extended ideas from spectral clustering and topological data analysis for the analysis of high-dimensional point cloud data. Graph-based (spectral) clustering of point cloud data typically constructs a proximity graph from the observed data points and then analyses the spectral properties of the resulting graph Laplacian to group points.
Instead, we explored the use of first constructing a (simplicial) complex from the data --- similar to how it is done in topological data analysis --- but instead of considering the (persistent) homology of these complexes, we exploit the spectral properties of the associated Hodge Laplacians to related global topological features to individual (localized) points. This strategy enables us to obtain a richer set of clusters that respect further topological properties of the data (compared to "simple" spectral clustering).
Furthermore we developed several new methods to process signals on graphs and cell complexes and understand current methods for processing signals on graphs. For instance, we showed how over-smoothing can be provably mitigated in graph neural networks and provided a mathematical analysis of the convergence of the training of graph neural networks. We also provided random-walk based architectures to learn from simplicial complexes, developed techniques for outlier-detection of trajectories defined over discrete and discretized surfaces and considered the problem of inference of cell-complexes based on the observation of flows on the edges.
Moreover, we outlined a blueprint for topological deep-learning methods and an associated research agenda together with international collaborators, which provides many avenues for the development of new methods. In conjunction, we contributed to the development of topoX, a suite of python software packages for topological deep learning.
At this stage (two years after the beginning of the project) we believe that further research is needed to consolidate the here established progress.
However, many of our works exceed the state-of-the-art and provide a fertile ground for further innovations. As a specific example, in one of our works we propose a method to determine the minimum order of a hypergraph necessary to accurately approximate the dynamics running on this hypergraph. Specifically, we first develop a mathematical framework that allows us to determine this order exactly when the type of dynamics is known. We then use these ideas in conjunction with a hypergraph neural network to directly learn the dynamics itself and the resulting order of the hypergraph from both synthetic and real datasets consisting of observed system trajectories. This approach may be used to infer higher-order network interactions from time-series, e.g. for biological systems.
However, many of our works exceed the state-of-the-art and provide a fertile ground for further innovations. As a specific example, in one of our works we propose a method to determine the minimum order of a hypergraph necessary to accurately approximate the dynamics running on this hypergraph. Specifically, we first develop a mathematical framework that allows us to determine this order exactly when the type of dynamics is known. We then use these ideas in conjunction with a hypergraph neural network to directly learn the dynamics itself and the resulting order of the hypergraph from both synthetic and real datasets consisting of observed system trajectories. This approach may be used to infer higher-order network interactions from time-series, e.g. for biological systems.