Evolve is a European project bringing together HPC, AI and Big Data experts. HPC, AI and BigData share a common point: the need for performance at scale.
However, they differentiate by many other aspects: nature of the data processed, workflow, software stack. This challenges the idea that a generic purpose processor
or storage architecture will meet their requirements.Therefore Evolve is building an heterogeneous supercomputing platform jointly with the software components to
leverage transparently this heterogeneity at the advantage of the applications.Evolve platform supports traditional CPU, with GPU and state-of-the-art FPGA.
Storage-wise, the platform is accelerated by 8 DDN's IME units and a Lustre parallel file system. More importantly, Evolve is a customer-focused project, where partners
provide 9 pilots applications to validate the approach. This crucial applications effort allows Evolve to optimize and adjust in an agile way during project’s lifetime.
With carefully designed middleware, API and connectors, Evolve is already able to accelerate Spark applications in a fully transparent manner, to containerize parallel
file systems in Docker and allow Zeppelin notebooks to access the supercomputer. Already one of the Pilot application with complex workflow including MPI parallel
code and AI can be orchestrated on the platform taking advantage of the diversity of the computing resources. Much is still on-going: resource management and
monitoring, data protecting, articulation between Kubernetes and Slurm or even deeper storage hierarchies.
Several external partners have already been engaged to test and validate the Evolve system and help us to mature the solution. The future will be diverse!
