Project description
Revolutionising data storage in extreme-scale computing and AI
The high-performance computing landscape is evolving owing to the massive amounts of data generated by scientific instruments, sensors, and simulations. To efficiently store, process, and analyse these data, existing storage technologies must be adapted to achieve higher efficiencies. The EU-funded Sage2 project, building upon the SAGE platform, aims to validate a next-generation storage system addressing the new use case requirements in extreme-scale computing and AI/deep learning. This advanced storage infrastructure will enhance scientific throughput, scalability, time and energy efficiency, and overall productivity for developers and users.
Objective
The landscape for High Performance Computing is changing with the proliferation of enormous volumes of data created by scientific instruments and sensors, in addition to data from simulations. This data needs to be stored, processed and analysed, and existing storage system technologies in the realm of extreme computing need to be adapted to achieve reasonable efficiency in achieving higher scientific throughput. We started on the journey to address this problem with the SAGE project. The HPC use cases and the technology ecosystem is now further evolving and there are new requirements and innovations that are brought to the forefront. It is extremely critical to address them today without “reinventing the wheel” leveraging existing initiatives and know-how to build the pieces of the Exascale puzzle as quickly and efficiently as we can.
The SAGE paradigm already provides a basic framework to address the extreme scale data aspects of High Performance Computing on the path to Exascale. Sage2 (Percipient StorAGe for Exascale Data Centric Computing 2) intends to validate a next generation storage system building on top of the already existing SAGE platform to address new use case requirements in the areas of extreme scale computing scientific workflows and AI/deep learning leveraging the latest developments in storage infrastructure software and storage technology ecosystem.
Sage2 aims to provide significantly enhanced scientific throughput, improved scalability, and, time & energy to solution for the use cases at scale. Sage2 will also dramatically increase the productivity of developers and users of these systems.
This proposal is aligned to FETHPC-02 - 2017:part (c). Sage2 provides a highly performant and resilient, QoS capable multi tiered storage system, with data layouts across the tiers managed by the Mero Object Store, which is capable of handling in-transit/in-situ processing of data within the storage system, accessible through the Clovis API.
Fields of science
- social scienceseconomics and businesseconomicsproduction economicsproductivity
- engineering and technologyelectrical engineering, electronic engineering, information engineeringelectronic engineeringsensors
- natural sciencesbiological sciencesecologyecosystems
- natural sciencescomputer and information sciencesartificial intelligencemachine learningdeep learning
- engineering and technologyelectrical engineering, electronic engineering, information engineeringelectronic engineeringcomputer hardwaresupercomputers
Programme(s)
Funding Scheme
RIA - Research and Innovation actionCoordinator
PO9 1SA Havant
United Kingdom