Descrizione del progetto
Un sistema di memoria avanzato potrebbe portare il calcolo su scala exa più vicino alla realtà
Il progetto IO-SEA, finanziato dall’UE, prevede di fornire una nuova gestione dei dati e una piattaforma di archiviazione per il calcolo su scala exa basata su archivi di oggetti, gestione gerarchica dell’archivio e fornitura on-demand di servizi di archiviazione. La piattaforma farà un uso efficiente dei livelli di archiviazione, tra cui il protocollo NVMe (Non-Volatile Memory Express) e le memorie ad accesso casuale non volatili (NVRAM), fino alle tecnologie basate su nastro. L’uso di nodi di dati effimeri e di accessi ai dati dovrebbe permettere agli utenti di utilizzare in modo flessibile il sistema usando vari paradigmi di accesso ai dati ben noti. Queste risorse effimere elimineranno il problema di trattare le risorse di archiviazione e i componenti del sistema come statici. I nuovi metodi di gestione dei dati sviluppati potrebbero essere utilizzati per applicazioni data-intensive in piattaforme su scala exa.
Obiettivo
IO-SEA aims to provide a novel data management and storage platform for exascale computing based on hierarchical storage management (HSM) and on-demand provisioning of storage services. The platform will efficiently make use of storage tiers spanning NVMe and NVRAM at the top all the way down to tape-based technologies. System requirements are driven by data intensive use-cases, in a very strict co-design approach. The concept of ephemeral data nodes and data accessors is introduced that allow users to flexibly operate the system, using various well-known data access paradigms, such as POSIX namespaces, S3/Swift Interfaces, MPI-IO and other data formats and protocols. These ephemeral resources eliminate the problem of treating storage resources as static and unchanging system components – which is not a tenable proposition for data intensive exascale environments. The methods and techniques are applicable to exascale class data intensive applications and workflows that need to be deployed in highly heterogeneous computing environments.
Critical aspects of intelligent data placement are considered for extreme volumes of data. This ensures that the right resources among the storage tiers are used and accessed by data nodes as close as possible to compute nodes – optimising performance, cost, and energy at extreme scale. Advanced IO instrumentation and monitoring features will be developed to that effect leveraging the latest advancements in AI and machine learning to systematically analyse the telemetry records to make smart decisions on data placement. These ideas coupled with in-storage-computation remove unnecessary data movements within the system.
The IO-SEA project (EuroHPC-2019-1 topic b) has connections to the DEEP-SEA (topic d) and RED-SEA (topic c) project. It leverages technologies developed by the SAGE, SAGE2 and NextGEN-IO projects, and strengthens the TLR of the developed products and technologies.
Campo scientifico
Parole chiave
Programma(i)
Meccanismo di finanziamento
RIA - Research and Innovation actionCoordinatore
75015 PARIS 15
Francia