Exascale Programming Models for Heterogeneous Systems

Información del proyecto

EPiGRAM-HS

Identificador del acuerdo de subvención: 801039

DOI

10.3030/801039

Proyecto cerrado

Fecha de la firma de la CE 18 Abril 2018

Fecha de inicio 1 Septiembre 2018

Fecha de finalización 31 Agosto 2021

Financiado con arreglo a

• EXCELLENT SCIENCE - Future and Emerging Technologies (FET)

Coste total

€ 3 998 741,25

Aportación de la UE

€ 3 998 741,25

Coordinado por

KUNGLIGA TEKNISKA HOEGSKOLAN
Sweden

CORDIS proporciona enlaces a los documentos públicos y las publicaciones de los proyectos de los programas marco HORIZONTE.

Los enlaces a los documentos y las publicaciones de los proyectos del Séptimo Programa Marco, así como los enlaces a algunos tipos de resultados específicos, como conjuntos de datos y «software», se obtienen dinámicamente de OpenAIRE .

Resultado final

Documents, reports (24)

Report on Current Landscape for Novel Network Hardware and Programming Models

This report will survey existing available network hardware and software, focusing on a gap analysis between hardware capabilities and programming models and communication library implementations.

Report on dissemination, training, exploitation and standardization activities.

This deliverable presents the overall EPiGRAMHS dissemination training exploitation activities and interaction with standardization bodies and collaboration with other European exascale projects and initiatives in the three years of the project

Final design specification and prototype im- plementation report of MPI and GPI extensions for heterogeneous systems with distributed GPUs and FPGAs

This report will present the design for MPI RMA operations for distributed GPUs MPI planned collectives on heterogeneous systems and GPI for distributed FPGAs

Best practices document for porting applications to large-scale heterogeneous systems.

This report collects experiences about application porting to large-scale heterogeneous systems, describing the difficulties and how these were addressed, and provides best-practices.

Report on final implementation of APIs and runtime system for data placement, migration and access on diverse memories

This deliverable describes the implementation of APIs and runtime system for data movement on complex memories and report performance results

Report on dissemination, training, exploitation and standardization activities, and updated plan.

This deliverable presents EPiGRAM-HS dissemination, training, exploitation activities, and interaction with standardization bodies and collaboration with other European exascale projects and initiatives in the first year of the project. This deliverable will include an update of the dissemination and exploitation plan.

Design document of GPI-space extension for distributed FPGAs and DSL for deep-learning applications

This deliverable presents the initial design of GPI-space extension for distributed FPGAs and DSL for deep-learning applications.

Experiences and best practices on programming emerging transport technologies for data movement.

This deliverable collects the experience with new and emerging transport technologies for data movement presenting potential challenges in using them and describing how to address them

Report on current and emerging transport technologies for data movement.

This deliverable provides an overview of the current and emerging transport technologies that are relevant to the EPiGRAM-HS software components.

Report on experiences in using MPI and GASPI on systems with low-power microprocessors.

This report reports gained using MPI and GASPI using heterogeneous systems with lowpower microprocessors identifying difficulties and proposing possible extensions to MPI and GASPI

Report on EPiGRAM-HS website including vision, plan for updates and website monitoring strategies.

Project website will be established to show the work of the project. It will focus on giving a clear message of the project’s achievements and act as a repository of information on EPiGRAM-HS.

Report on state of the art of novel compute elements and gap analysis in MPI and GASPI.

This report surveys new emerging low-power microprocessor, focusing on a gap analysis between hardware capabilities and MPI and GASPI programming systems.

Update on Current Landscape for Novel Network Hardware and Programming Models

This report will cover additional hardware and software technologies that have become available during the lifetime of the project including a summary of the impact of the outputs generated by this work package

Final design specification and prototype implementation report of APIs and runtime system for data placement, migration and access on diverse memories.

This deliverable reports about first implementation of APIs and runtime system and eventually changes to the design documents to overcome implementation issues

Report on dissemination, training, exploitation and standardization activities, and updated plan 2.

This deliverable presents EPiGRAMHS dissemination training exploitation activities and interaction with standardization bodies and collaboration with other European exascale projects and initiatives This deliverable will include an update of the dissemination and exploitation plan

Final design specification and prototype implementation report of GPI-Space extension for distributed FPGAS and DSL for deep-learning applications

This deliverable presents the initial design of GPI-space extension for distributed FPGAs and DSL for deep-learning applications after initial feedback from applications. In addition, it describes the prototype implementation of the GPU-Space extension and of the DSL.

Integration of EPiGRAM-HS programming environment in applications

This report presents the parallel performance of all the EPiGRAMHS software components in applications analyzing scalability and improved achieved with respect with the implementation at the beginning of the project

Report on initial porting of applications to large-scale heterogeneous systems

This deliverable reports about the initial effort of porting of application to distributed systems with GPUs and FPGAs. It describes the porting strategy and a testing plan.

Plan for dissemination, training, exploitation and standardization.

This deliverable presents the initial dissemination plan including target audiences and activities, training and exploitation plan. In addition, it will present EPiGRAM-HS interaction with the MPI Forum and GASPI Forum.

Report on application requirements and roadmap.

This deliverable describes the application requirements for the development of the EPiGRAM-HS programming environment, and it identifies the steps for the application developments for the application porting to large-scale heterogeneous systems. In addition, in this deliverable we select the applications to validate each EPiGRAM-HS programming environment component.

Initial design document of MPI and GPI extensions for heterogeneous systems with distributed GPUs and FPGAs

This report will present the design for MPI RMA operations for distributed GPUs, MPI planned collectives on heterogeneous systems and GPI for distributed FPGAs

Report on final implementation of MPI and GPI extensions for heterogeneous systems with distributed GPUs and FPGAs

This report will present the design for MPI RMA operations for distributed GPUs MPI planned collectives onheterogeneous systems and GPI for distributed FPGAs

Initial design of memory abstraction device for diverse memories.

This deliverable describes the initial design of EPiGRAM-HS memory abstraction device, comprising APIs for simplified and optimized data movement and a runtime system for automatic data placement on diverse memories.

Report on final implementation of GPI-Space extension for distributed FPGAs and DSL for deep-learning applications

This report describes the final implementation of GPISSpace extension for FPGAs and a DSL for deeplearning applications

Publicaciones

Peer reviewed articles (2)

The Old and the New: Can Physics-Informed Deep-Learning Replace Traditional Linear Solvers?

Autores: Stefano Markidis 
Publicado en: Frontiers in Big Data, Vol 4 (2021), Edición 5, 2021, ISSN 2624-909X
Editor: Frontiers in Big Data
DOI: 10.3389/fdata.2021.669097

MPI collective communication through a single set of interfaces: A case for orthogonality  

Autores: Jesper Larsson Träff; Sascha Hunold; Guillaume Mercier; Daniel J. Holmes
Publicado en: Parallel Computing, Edición 2, 2021, ISSN 0167-8191
Editor: Elsevier BV
DOI: 10.1016/j.parco.2021.102826

Conference proceedings (26)

RFaaS: RDMA-Enabled FaaS Platform for Serverless High-Performance Computing

Autores: Copik, Marcin; Taranov, Konstantin; Calotoiu, Alexandru; Hoefler, Torsten
Publicado en: Edición 10, 2022
Editor: USENIX Annual Technical Conference

Data Movement Is All You Need: A Case Study on Optimizing Transformers

Autores: Ivanov, Andrei; Dryden, Nikoli; Ben-Nun, Tal; Li, Shigang; Hoefler, Torsten
Publicado en: Proceedings of Machine Learning and Systems, Edición 11, 2021
Editor: SPCL

Communication and Timing Issues with MPI Virtualization.

Autores: Alexandr Nigay; Lukas Mosimann; Timo Schneider; Torsten Hoefler
Publicado en: EuroMPI, Edición 10, 2020
Editor: ACM
DOI: 10.1145/3416315.3416317

Flare: flexible in-network allreduce

Autores: Daniele De Sensi, Salvatore Di Girolamo, Saleh Ashkboos, Shigang Li, Torsten Hoefler
Publicado en: 2021
Editor: ACM
DOI: 10.5281/zenodo.4836022

A Deep Learning-Based Particle-in-Cell Method for Plasma Simulations

Autores: Xavier Aguilar; Stefano Markidis
Publicado en: 2021 IEEE International Conference on Cluster Computing (CLUSTER), 2021
Editor: IEEE
DOI: 10.1109/cluster48925.2021.00103

Chimera: Efficiently Training Large-Scale Neural Networks with Bidirectional Pipelines

Autores: Shigang Li, Torsten Hoefler
Publicado en: SC '21: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, 2021
Editor: ACM
DOI: 10.1145/3458817.3476145

A RISC-V in-network accelerator for flexible high-performance low-power packet processing

Autores: Salvatore Di Girolamo; Andreas Kurth; Alexandru Calotoiu; Thomas Benz; Timo Schneider; Jakub Beranek; Luca Benini; Torsten Hoefler
Publicado en: 2021 ACM/IEEE 48th Annual International Symposium on Computer Architecture (ISCA), Edición 9, 2021
Editor: IEEE
DOI: 10.1109/isca52012.2021.00079

On the parallel I/O optimality of linear algebra kernels: near-optimal matrix factorizations

Autores: G. Kwasniewski, M. Kabi ́c, T. Ben-Nun, A. Nikolaos Ziogas, J. Eirik Saethre, A. Gaillard, T. Schneider, M. Besta, A. Kozhevnikov, J. Vande- Vondele, T. Hoefler
Publicado en: 2021
Editor: ACM
DOI: 10.1145/3458817.3476167

FBLAS: Streaming Linear Algebra on FPGA

Autores: De Matteis, Tiziano; Licht, Johannes de Fine; Hoefler, Torsten
Publicado en: SC20: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, Edición 9, 2020
Editor: IEEE
DOI: 10.1109/sc41405.2020.00063

Learning representations in Bayesian Confidence Propagation neural networks

Autores: Naresh Balaji Ravichandran, Anders Lansner, Pawel Herman
Publicado en: 2020
Editor: IEEE
DOI: 10.1109/ijcnn48605.2020.9207061

Why is MPI (perceived to be) so complex?: Part 1—Does strong

Autores: Daniel J. Holmes, Anthony Skjellum, Derek Schafer
Publicado en: EuroMPI, 2020
Editor: ACM
DOI: 10.1145/3416315.3416318

StreamBrain: An HPC Framework for Brain-like Neural Networks on CPUs, GPUs and FPGAs

Autores: Artur Podobas, Martin Svedin, Steven W. D. Chien, Ivy B. Peng, Naresh Balaji Ravichandran, Pawel Herman, Anders Lansner, Stefano Markidis
Publicado en: 2021
Editor: ACM
DOI: 10.1145/3468044.3468052

Benchmarking the Nvidia GPU Lineage: From Early K80 to Modern A100 with Asynchronous Memory Transfers

Autores: Martin Svedin, Steven W. D. Chien, Gibson Chikafa, Niclas Jansson, Artur Podobas
Publicado en: 2021
Editor: ACM
DOI: 10.1145/3468044.3468053

Mamba: Portable Array-based Abstractions for Heterogeneous High-Performance Systems

Autores: Dykes, T., Foyer, C., Richardson, H., Svedin, M., Podobas, A., Jansson, N., Markidis, S., Tate, A., McIntosh-Smith, S.
Publicado en: 2021
Editor: IEEE
DOI: 10.1109/p3hpc54578.2021.00005

Spectral Element Simulations on the NEC SX-Aurora TSUBASA

Autores: Niclas Jansson
Publicado en: 2021
Editor: ACM
DOI: 10.1145/3432261.3432265

Characterizing Deep-Learning I/O Workloads in TensorFlow

Autores: Steven W. D. Chien, Stefano Markidis, Chaitanya Prasad Sishtla, Luis Santos, Pawel Herman, Sai Narasimhamurthy, Erwin Laure
Publicado en: 2018 IEEE/ACM 3rd International Workshop on Parallel Data Storage & Data Intensive Scalable Computing Systems (PDSW-DISCS), 2018, Página(s) 54-63, ISBN 978-1-7281-0192-7
Editor: IEEE
DOI: 10.1109/PDSW-DISCS.2018.00011

TensorFlow Doing HPC

Autores: Steven W. D. Chien, Stefano Markidis, Vyacheslav Olshevsky, Yaroslav Bulatov, Erwin Laure, Jeffrey Vetter
Publicado en: 2019 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW), 2019, Página(s) 509-518, ISBN 978-1-7281-3510-6
Editor: IEEE
DOI: 10.1109/IPDPSW.2019.00092

Streaming message interface - high-performance distributed memory programming on reconfigurable hardware

Autores: Tiziano De Matteis, Johannes de Fine Licht, Jakub Beránek, Torsten Hoefler
Publicado en: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, 2019, Página(s) 1-33, ISBN 9781-450362290
Editor: ACM
DOI: 10.1145/3295500.3356201

Exposition, clarification, and expansion of MPI semantic terms and conventions - is a nonblocking MPI function permitted to block?

Autores: Purushotham V. Bangalore, Rolf Rabenseifner, Daniel J. Holmes, Julien Jaeger, Guillaume Mercier, Claudia Blaas-Schenner, Anthony Skjellum
Publicado en: Proceedings of the 26th European MPI Users' Group Meeting on - EuroMPI '19, 2019, Página(s) 1-10, ISBN 9781-450371759
Editor: ACM Press
DOI: 10.1145/3343211.3343213

Performance Evaluation of Advanced Features in CUDA Unified Memory

Autores: Steven Chien, Ivy Peng, Stefano Markidis
Publicado en: 2019 IEEE/ACM Workshop on Memory Centric High Performance Computing (MCHPC), 2019, Página(s) 50-57, ISBN 978-1-7281-6007-8
Editor: IEEE
DOI: 10.1109/mchpc49590.2019.00014

MPI Sessions: Evaluation of an Implementation in Open MPI

Autores: Nathan Hjelm, Howard Pritchard, Samuel K. Gutierrez, Daniel J. Holmes, Ralph Castain, Anthony Skjellum
Publicado en: 2019 IEEE International Conference on Cluster Computing (CLUSTER), 2019, Página(s) 1-11, ISBN 978-1-7281-4734-5
Editor: IEEE
DOI: 10.1109/cluster.2019.8891002

sputniPIC: An Implicit Particle-in-Cell Code for Multi-GPU Systems

Autores: Steven W. D. Chien, Jonas Nylund, Gabriel Bengtsson, Ivy B. Peng, Artur Podobas, Stefano Markidis
Publicado en: 2020
Editor: IEEE
DOI: 10.1109/sbac-pad49847.2020.00030

Higgs Boson Classification: Brain-inspired BCPNN Learning with StreamBrain

Autores: Svedin, Martin; Podobas, Artur; Chien, Steven W. D.; Markidis, Stefano
Publicado en: 2021 IEEE International Conference on Cluster Computing (CLUSTER), Edición 6, 2021
Editor: IEEEE
DOI: 10.1109/cluster48925.2021.00105

Automatic Particle Trajectory Classification in Plasma Simulations

Autores: Markidis, Stefano; Peng, Ivy; Podobas, Artur; Jongsuebchoke, Itthinat; Bengtsson, Gabriel; Herman, Pawel
Publicado en: Crossref, Edición 14, 2020
Editor: IEEE
DOI: 10.1109/mlhpcai4s51975.2020.00014

Collectives and Communicators: A Case for Orthogonality: (Or: How to get rid of MPI neighbor and enhance Cartesian colletives)

Autores: Jesper Larsson Tr ̈aff, Sascha Hunold, Guillaume Mercier, Daniel J. Holmes
Publicado en: EuroMPI/USA '20: 27th European MPI Users' Group Meeting, 2020
Editor: ACM
DOI: 10.1145/3416315.3416319

Semi-supervised learning with Bayesian Confidence Propagation Neural Network

Autores: Naresh Balaji Ravichandran, Anders Lansner, Pawel Herman
Publicado en: ESANN 2021 proceedings, European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning, 2021
Editor: European Symposium on Artificial Neural Networks
DOI: 10.14428/esann/2021.es2021-156

Other (1)

A Data-Centric Optimization Framework for Machine Learning

Autores: Oliver Rausch, Tal Ben-Nun, Nikoli Dryden, Andrei Ivanov, Shigang Li, Torsten Hoefler
Publicado en: 2021
Editor: Arxiv

Book chapters (3)

Brain-Like Approaches to Unsupervised Learning of Hidden Representations - A Comparative Study

Autores: Naresh Balaji Ravichandran, Anders Lansner, Pawel Herman
Publicado en: Artificial Neural Networks and Machine Learning – ICANN 2021, 2021
Editor: Springer
DOI: 10.1007/978-3-030-86383-8_13

Posit NPB: Assessing the Precision Improvement in HPC Scientific Applications

Autores: Steven W. D. Chien, Ivy B. Peng, Stefano Markidis
Publicado en: Parallel Processing and Applied Mathematics - 13th International Conference, PPAM 2019, Bialystok, Poland, September 8–11, 2019, Revised Selected Papers, Part I, Edición 12043, 2020, Página(s) 301-310, ISBN 978-3-030-43228-7
Editor: Springer International Publishing
DOI: 10.1007/978-3-030-43229-4_26

Multi-GPU Acceleration of the iPIC3D Implicit Particle-in-Cell Code

Autores: Chaitanya Prasad Sishtla, Steven W. D. Chien, Vyacheslav Olshevsky, Erwin Laure, Stefano Markidis
Publicado en: Computational Science – ICCS 2019 - 19th International Conference, Faro, Portugal, June 12–14, 2019, Proceedings, Part V, Edición 11540, 2019, Página(s) 612-618, ISBN 978-3-030-22749-4
Editor: Springer International Publishing
DOI: 10.1007/978-3-030-22750-0_58