The project achieved a prototype that applies its software solution to composable workflows useful in Renewable Energy Industry and in the Manufacturing Industry for applications in designing energy efficient products. The workflow definition has been kept at a generic level with a cross-industry purpose in order to broaden applicability.
All modules of the prototype (API, identity/data/workflow management, application containerization, decision module, energy monitoring and optimization, user interface…) are functional (though could be hardened and extended for a production platform) and the whole platform relies on DevOps tools managing the different services through a Kubernetes cluster.
Concerning the main elements of the prototype:
1. Workflows: two have been selected to reflect classical HPC and AI use cases: a CFD workflow with OpenFoam, and an object recognition in videos with Tensorflow. Applications have been containerized with Singularity and integrated with the workflow manager (NextFlow).
2. Web interface: a simple web interface has been implemented and allows a user to execute the use cases
3. Energy: EAR has been extended to have a Lite version which does not require root privileges to be installed; a new reporting plugin with HEROES requirements has been added; additional metrics have been added.
4. Workflow Placement: a new Decision Module plugin has been added to OKA, it leverages machine learning and multi-criteria decision algorithms to propose placement options for the workflows. OKA analytical and predictive capabilities have been enhanced to support the use cases and integrate with EAR for energy metrics.
5. Identity, data and workflow management: multiple services have been implemented to provide an abstraction to the target clusters. Through a single entry-point (REST APIs), users can manage data in dedicated object stores managed by HEROES per user/organization. Workflow submission is simplified through the selection of available workflow templates, integration with the Decision Module, and integration of an updated version of Nextflow. Identity management is delegated to a centralized Keycloak, and access to remote clusters is done in a secure way despite sharing accounts on the clusters (thanks to containerization and data management with the HEROES-FS).