Skip to main content

Adaptive multi-tier intelligent data manager for Exascale

Objectif

The growing need to process extremely large data sets is one of the main drivers for building exascale HPC systems today. However, the flat storage hierarchies found in classic HPC architectures no longer satisfy the performance requirements of data-processing applications. Uncoordinated file access in combination with limited bandwidth make the centralised back-end parallel file system a serious bottleneck. At the same time, emerging multi-tier storage hierarchies come with the potential to remove this barrier. But maximising performance still requires careful control to avoid congestion and balance computational with storage performance. Unfortunately, appropriate interfaces and policies for managing such an enhanced I/O stack are still lacking.

The main objective of the ADMIRE project is to establish this control by creating an active I/O stack that dynamically adjusts computation and storage requirements through intelligent global coordination, malleability of computation and I/O, and the scheduling of storage resources along all levels of the storage hierarchy. To achieve this, we will develop a software-defined framework based on the principles of scalable monitoring and control, separated control and data paths, and the orchestration of key system components and applications through embedded control points.

Our software-only solution will allow the throughput of HPC systems and the performance of individual applications to be substantially increased – and consequently energy consumption to be decreased – by taking advantage of fast and power-efficient node-local storage tiers using novel, European ad-hoc storage systems and in-transit/in-situ processing facilities. Furthermore, our enhanced I/O stack will offer quality-of-service (QoS) and resilience. An integrated and operational prototype will be validated with several use cases from various domains, including climate/weather, life sciences, physics, remote sensing, and deep learning.

Appel à propositions

H2020-JTI-EuroHPC-2019-1
Voir d’autres projets de cet appel

Régime de financement

EuroHPC-RIA - EuroHPC-RIA

Coordinateur

UNIVERSIDAD CARLOS III DE MADRID
Adresse
Calle Madrid 126
28903 Getafe (Madrid)
Espagne
Type d’activité
Higher or Secondary Education Establishments
Contribution de l’UE
€ 383 937,50

Participants (13)

JOHANNES GUTENBERG-UNIVERSITAT MAINZ
Allemagne
Contribution de l’UE
€ 340 562,50
Adresse
Saarstrasse 21
55122 Mainz
Type d’activité
Higher or Secondary Education Establishments
BARCELONA SUPERCOMPUTING CENTER-CENTRO NACIONAL DE SUPERCOMPUTACION
Espagne
Contribution de l’UE
€ 304 375
Adresse
Calle Jordi Girona 31
08034 Barcelona
Type d’activité
Research Organisations
TECHNISCHE UNIVERSITAT DARMSTADT
Allemagne
Contribution de l’UE
€ 432 905
Adresse
Karolinenplatz 5
64289 Darmstadt
Type d’activité
Higher or Secondary Education Establishments
DATADIRECT NETWORKS FRANCE
France
Contribution de l’UE
€ 425 000
Adresse
10-12 Rue Andras Beck
92360 Meudon La Foret
Type d’activité
Private for-profit entities (excluding Higher or Secondary Education Establishments)
INSTITUT NATIONAL DE RECHERCHE EN INFORMATIQUE ET AUTOMATIQUE
France
Contribution de l’UE
€ 345 000
Adresse
Domaine De Voluceau Rocquencourt
78153 Le Chesnay Cedex
Type d’activité
Research Organisations
PARATOOLS SAS
France
Contribution de l’UE
€ 318 425
Adresse
2 Rue De La Piquetterie
91680 Bruyeres Le Chatel
Type d’activité
Private for-profit entities (excluding Higher or Secondary Education Establishments)
FORSCHUNGSZENTRUM JULICH GMBH
Allemagne
Contribution de l’UE
€ 300 886,88
Adresse
Wilhelm Johnen Strasse
52428 Julich
Type d’activité
Research Organisations
CONSORZIO INTERUNIVERSITARIO NAZIONALE PER L'INFORMATICA
Italie
Contribution de l’UE
€ 298 750
Adresse
Via Ariosto 25
00185 Roma
Type d’activité
Higher or Secondary Education Establishments
CINECA CONSORZIO INTERUNIVERSITARIO
Italie
Contribution de l’UE
€ 116 250
Adresse
Via Magnanelli 6/3
40033 Casalecchio Di Reno Bo
Type d’activité
Research Organisations
E4 COMPUTER ENGINEERING SPA
Italie
Contribution de l’UE
€ 153 863,75
Adresse
Via Martiri Della Liberta 66
42019 Scandiano
Type d’activité
Private for-profit entities (excluding Higher or Secondary Education Establishments)
INSTYTUT CHEMII BIOORGANICZNEJ POLSKIEJ AKADEMII NAUK
Pologne
Contribution de l’UE
€ 275 250
Adresse
Noskowskiego 12-14
61 704 Poznan
Type d’activité
Research Organisations
KUNGLIGA TEKNISKA HOEGSKOLAN
Suède
Contribution de l’UE
€ 156 437,50
Adresse
Brinellvagen 8
100 44 Stockholm
Type d’activité
Higher or Secondary Education Establishments
MAX-PLANCK-GESELLSCHAFT ZUR FORDERUNG DER WISSENSCHAFTEN EV
Allemagne
Contribution de l’UE
€ 130 000
Adresse
Hofgartenstrasse 8
80539 Muenchen
Type d’activité
Research Organisations