The effort in the DAPP project is focused on data-centric programming models, both general-purpose and for specific algorithms. Our main contribution is an Intermediate Representation (IR) that can be generated from different high-level languages and frameworks (Python, MATLAB, TensorFlow). The IR is graph-based and is organized around data movement, specifying dataflow and control-flow explicitly as part of the program. The Stateful DataFlow multiGraph (SDFG) combines fine-grained data dependencies with high-level control flow, and is both expressive and amenable to high-level and reusable program transformations, such as tiling, vectorization, and double buffering. These transformations are then applied to the SDFG in an interactive process (guided by a performance engineer), using extensible pattern matching and graph rewriting. SDFGs successfully map to CPUs, GPUs, and FPGAs, on a wide variety of application motifs: from fundamental computational kernels, through polyhedral applications, to graph analytics. The representation is both expressive and performant, allowing domain scientists to develop applications that can be tuned to approach peak hardware performance without modifying the original scientific code.
To provide a complete picture of the importance of data locality and data-centric programming, we summarized, together with other leading research groups, the collective knowledge on the topic. The review paper, titled “Trends in Data Locality Abstractions for HPC Systems”, was published at IEEE TPDS. For high-performance programming of FPGAs, we wrote a review and in-depth analysis of the techniques that are used to optimize HPC applications for data movement on dataflow architectures, consolidating the insights and state-of-the-art in High-Level Synthesis (HLS) for FPGAs.
While general-purpose programming interfaces and IRs can accelerate a wide variety of regular applications, irregular applications, such as graph algorithms, pose additional challenges that we investigated. In particular, we constructed graph representations that incur reduced data movement, such as the Log(Graph) succinct representation; representations that are amenable to efficient processing (e.g. using vectorization) for CPUs and GPUs with SlimSell; and efficient representations for streaming graph algorithms on FPGAs, namely a substream-centric representation for the Maximum Weighted Matching algorithm.
Optimizing communication performance is imperative for large-scale computing, as overheads limit the strong scalability of parallel applications. Today’s network cards also contain rather powerful processors optimized for data movement. As part of DAPP, we developed sPIN, a portable programming model to offload simple packet processing functions to the network card. The model provides both the simplicity of accelerator languages such as CUDA, and the flexibility of directly controlling the network card to optimize collective communication operations and system services by bypassing the CPU.
Lastly, we leveraged Machine Learning techniques and Deep Learning to statically analyze and comprehend code semantics. In particular, we propose Neural Code Comprehension, a novel processing pipeline to learn code semantics robustly, based on a novel embedding space that we call inst2vec. The pipeline analyzes the dataflow of an application (using an IR), and apply it to a variety of program analysis tasks, including algorithm classification, hardware mapping (i.e. whether a program will run faster on a CPU or a GPU), and thread workload coarsening factors, where we set a new state-of-the-art in accuracy for two out of the three tasks.
