SLING adopts a systematic approach of combining ideas and findings from neighboring fields to synthesize and generate innovative solutions. In machine learning, there is often a divide et impera approach where statistical and computational aspects are treated separately and by different scientific communities. Instead, SLING takes a holistic approach and fully leverages an interdisciplinary perspective.
For example, while considering memory compression for machine learning, we elaborated a connection between sketching approaches common in computer science and Galerkin methods developed in numerical analysis, studied using tools from approximation and interpolation theory. We showed that regularization theory for inverse problems, specifically regularization by projection, provides a unified framework when combined with concentration of measures results.
Another example is recognizing that iterative computations can be viewed as defining dynamical systems that adaptively navigate the trade-off between accuracy and efficiency, leading to massive reductions in compute time. Inexact optimization techniques, developed to handle approximate numerical computations, need to be adjusted to address irreducible stochastic errors, which can be studied using probabilistic tools. Once again, regularization theory, including its stochastic extensions, offers a natural framework for analyzing and deriving novel compressed algorithms.
SLING establishes a connection between theory and practice by testing the solutions developed in two diverse domains: high-energy physics and robotics. Our results demonstrate various scenarios where the compressed algorithms designed in SLING are accurate and simultaneously yielding substantial improvements in computational efficiency, often reducing compute time from hours to seconds. Notably, they also alleviate the need for cumbersome computational infrastructure, enabling the utilization of basic workstations.
