Periodic Reporting for period 1 - inEXASCALE (Analyzing and Exploiting Inexactness in Exascale Matrix Computations)
Reporting period: 2023-03-01 to 2025-08-31
The project inEXASCALE aims to change the way people think about designing and analyzing algorithms in the exascale era. On supercomputers that exist today, achieving even close to the peak performance is incredibly difficult if not impossible for many applications. Techniques designed to improve performance - making computations less expensive by reorganizing an algorithm, making intentional approximations, and using lower precision - all introduce what we can generally call "inexactness". The question is, with all this inexactness involved, does the algorithm still get close enough to the right answer? The effects of these sources of inexactness have been studied separately, but never together in a holistic way. By studying the combination of different sources of inexactness, we will reveal not only the limitations of these techniques, but also reveal new opportunities for developing algorithms that are both fast and provably accurate.
The project focuses on the rigorous mathematical analysis of computations subject to multiple sources of error. The project has had a number of successes thus far, including the finite precision analysis of mixed precision randomized algorithms for matrix computations, the analysis of the loss of orthogonality in synchronization-reducing orthogonalization methods, analyzing the effect of inexact course-level solves in multigrid methods, the finite precision analysis of inexact preconditioning methods for solving linear systems and least-squares problems, and the analysis of mixed precision computations and storage for hierarchical low-rank matrix representations. These rigorous mathematical analyses have lead to the development of a number of new adaptive algorithms, which are capable of selecting the best precision(s) to use in each part of a computation based on the particular input data.
The research has led to the development of new analyses and algorithms of inexact methods for matrix computations designed for supercomputers that go beyond the state-of-the-art. For example, our new low-synchronization orthogonalization routine has led to speedups of 4x compare to the state-of-the-art methods for large-scale problems in distributed memory environments. In general, our new adaptive algorithms have the potential to provide significant performance improvements for matrix computations while maintaining a user-specified accuracy.
To ensure further uptake, we will continue to implement our algorithms using both high-level numerical frameworks and parallel computing frameworks and make our codes publicly available.
To ensure further uptake, we will continue to implement our algorithms using both high-level numerical frameworks and parallel computing frameworks and make our codes publicly available.