Significant progress was made in parameterizing ML tasks. This includes developing a taxonomy from AI publications and expert brainstorming sessions, creating an abstract structure for task modeling, and incorporating knowledge embedding techniques to parameterize tasks into a semantic space. This work enables the description of ML tasks as graphs of connected nodes, enhancing flexibility and efficiency in task modeling. Additionally, a comprehensive knowledge graph database using RDF and Turtle syntax has been created to store these relationships, facilitating efficient problem solving and knowledge recycling. To help build the knowledge graph, we are also using an LLM in combination with public model hosting sites (e.g. Huggingface). The result will be checked and edited by human experts. Last in providing viable ML solutions, we are using knowledge graph based retrieval augmented generation (Graph-RAG) to avoid hallucination, and provide multiple solution candidates.
Efforts have also been made to identify all the cross-layer optimizations suitable for improving the energy efficiency of the deep neural network (DNN) accelerators. The large design space is explored by the NAS (Neural Architecture Search) gu0ided by optimization objectives targeting both application and hardware requirements.
Within the main achievements, we can present a flexible HLS hardware library of custom hardware architectures, which can facilitate various DNN topologies. Note that first we presented custom hardware architectures for standard One-dimensional Convolutional Neural Networks (1D-CNNs), depth-wise separable 1D-CNNs, and various other DNN layers and components suitable for unidimensional signal processing.
Regarding PIM solutions, we also conducted research to identify the specific ML tasks or DNN layers that can be offloaded to UPMEM PIM to improve energy efficiency, and identified the implementation challenges of UPMEM PIM for these workloads. We have now established an automated flow to integrate new and/or configurable instructions into our current DPU processors to easily extend and adapt the DPU design for DNN acceleration. We have also implemented a solution on FPGA to increase the computing capacity of UPMEM’s PIM DRAMs by moving some operations (typically MAC operations) from the DPU to the SAs. This implementation is now fully operational.
We also addressed the resource costs associated with the training phase of machine learning (ML) model development. We introduce three novel methodologies that significantly enhance the training efficiency of ML models by optimizing the number of training examples required, minimizing the necessity for labeled data, and reducing memory consumption. These optimizations subsequently impact computational demand, energy usage, and the carbon footprint of training ML models.
In the field of HCI, we have conducted qualitative studies to better understand the awareness of ML and human-computer interaction (HCI) experts on their impact on sustainability, as well as their existing workflows. We also presented a framework that structures the different intersections between sustainability with ML and HCI and describes the resulting research areas based on recent work.
Finally, we have presented a design of the SustainML framework that integrates the results of the project. We have also carried out the development of the SustainML library that will be used by the consortium partners to integrate the different modules of the framework. In addition, we have created a first proof of concept of the front-end of the framework, so that both the project partners and the early adopters of the SustainML framework can test and validate it.
