Periodic Reporting for period 1 - Green.Dat.AI (Energy-efficient AI-ready Data Spaces)
Période du rapport: 2023-01-01 au 2024-06-30
GREEN.DAT.AI aims to channel AI’s potential towards Europe's sustainability goals by developing novel energy-efficient large-scale data analytics services, ready for use in industrial AI-based systems while reducing environmental impact of data management processes.
The project seeks to reduce the carbon footprint of AI technologies by creating an AI-ready Data Space (DS) infrastructure that supports energy-efficient AI techniques, addressing critical business and societal challenges.
Key strategies include designing AI algorithms with lower energy consumption, optimising hardware and software to improve energy efficiency, and employing techniques such as hardware acceleration, model optimisation, and data compression. Scalability is crucial, ensuring that AI services can adapt to demand without significant increases in energy consumption, while maintaining or surpassing performance standards.
The project's vision includes developing smart AI-powered applications that transition computation from data centers to edge devices, thus reducing data flow and enhancing privacy and reaction times. Federated Learning (FL) mechanisms will enable data sharing with minimal data transfer, supported by a novel Toolbox of reusable energy-efficient AI services.
A benchmarking and evaluation framework will be developed to measure and compare energy efficiency of various AI services, addressing the need for accurate energy consumption models and energy-aware algorithms. These services will be assessed across 4 industries—Smart Energy, Smart Agriculture, Smart Mobility, and Smart Banking—through 6 distinct pilots leveraging data spaces.
DS will serve as a digital enabler for collaborative data-driven services, ensuring data sovereignty, transparency, and fairness, thus facilitating seamless data exchange, incorporating robust security mechanisms and supporting interoperability, data quality, and privacy. By combining DS with AI services, GREEN.DAT.AI will unlock new potential for AI applications, enhancing predictive capabilities, increasing efficiency, and automating tasks.
The project seeks to reduce the carbon footprint of AI technologies by creating an AI-ready Data Space (DS) infrastructure that supports energy-efficient AI techniques, addressing critical business and societal challenges.
Key strategies include designing AI algorithms with lower energy consumption, optimising hardware and software to improve energy efficiency, and employing techniques such as hardware acceleration, model optimisation, and data compression. Scalability is crucial, ensuring that AI services can adapt to demand without significant increases in energy consumption, while maintaining or surpassing performance standards.
The project's vision includes developing smart AI-powered applications that transition computation from data centers to edge devices, thus reducing data flow and enhancing privacy and reaction times. Federated Learning (FL) mechanisms will enable data sharing with minimal data transfer, supported by a novel Toolbox of reusable energy-efficient AI services.
A benchmarking and evaluation framework will be developed to measure and compare energy efficiency of various AI services, addressing the need for accurate energy consumption models and energy-aware algorithms. These services will be assessed across 4 industries—Smart Energy, Smart Agriculture, Smart Mobility, and Smart Banking—through 6 distinct pilots leveraging data spaces.
DS will serve as a digital enabler for collaborative data-driven services, ensuring data sovereignty, transparency, and fairness, thus facilitating seamless data exchange, incorporating robust security mechanisms and supporting interoperability, data quality, and privacy. By combining DS with AI services, GREEN.DAT.AI will unlock new potential for AI applications, enhancing predictive capabilities, increasing efficiency, and automating tasks.
GREEN.DAT.AI aims to harness AI and DS to support the green transition by developing a data management framework for energy-efficient AI tools. This infrastructure enables efficient data processing, analysis, and sharing compatible with AI workflows. During the first reporting period, the project partners have:
• Developed a Reference Architecture and data governance framework, aligned with GAIA-X and IDSA guidelines, supporting secure, interoperable, and data-sovereign exchanges.
• Delivered a technical analysis for each UC, including key performance indicators (KPIs), expected impacts, and specific analytics algorithms to be implemented.
• Established a testing framework to assess AI algorithms performance, including energy efficiency.
• Set up a BDA infrastructure with two environments for data ingestion, harmonisation, processing and storage (service-related environment and data space-related environment).
• Designed and integrated the GREEN.DAT.AI DS with the essential operational components.
• Developed an Eclipse Dataspace Components (EDC) connector for DS access, data search and contracting exchange between data providers and consumers.
• Deployed tools for data management orchestration including: a Visualisation workbench, Streamhandler platform, Workflow Management Engine (WME) and security wrappers.
• Developed 18 energy-efficient large-scale data analytics services to address the technical requirements of the UCs.
• Created detailed workflows for each UC specifying services and datasets to be used.
• Delivered description of UC objectives, including business goals, sustainability, data management needs and AI targets.
• Developed a Reference Architecture and data governance framework, aligned with GAIA-X and IDSA guidelines, supporting secure, interoperable, and data-sovereign exchanges.
• Delivered a technical analysis for each UC, including key performance indicators (KPIs), expected impacts, and specific analytics algorithms to be implemented.
• Established a testing framework to assess AI algorithms performance, including energy efficiency.
• Set up a BDA infrastructure with two environments for data ingestion, harmonisation, processing and storage (service-related environment and data space-related environment).
• Designed and integrated the GREEN.DAT.AI DS with the essential operational components.
• Developed an Eclipse Dataspace Components (EDC) connector for DS access, data search and contracting exchange between data providers and consumers.
• Deployed tools for data management orchestration including: a Visualisation workbench, Streamhandler platform, Workflow Management Engine (WME) and security wrappers.
• Developed 18 energy-efficient large-scale data analytics services to address the technical requirements of the UCs.
• Created detailed workflows for each UC specifying services and datasets to be used.
• Delivered description of UC objectives, including business goals, sustainability, data management needs and AI targets.
GREEN.DAT.AI will drive innovation in data management solutions by developing secure, energy-efficient tools that enhance data usability, discoverability, provenance, synthetic generation, quality management, interoperability, metadata management, along with security, privacy, and integrity.
Indicative advancements beyond the state of the art include:
1. Sustainability – Reducing Environmental Impact
- Development of energy aware analytics services to minimise energy footprint, whilst being adaptable to different user needs.
- Implementing solutions to reduce learning data amount, learning time and improve energy demand accuracy.
2. Digital infrastructure and AI tools
- Creation of a Big Data Infrastructure, on top of which a toolkit of AI-enabled energy efficient algorithms will be applied to the UCs.
- Use of FL mechanisms, which use distributed data for inferring information based on decentralised collaborative modelling.
- Shift computation from cloud to personal devices to reduce data flow and potential leakage risks.
- Enable edge processing to eliminate transmission costs and achieve faster inference.
- Combination of services with the platform for new data quality management capabilities (data cleaning, validation, enrichment, co-creation, identification of bias).
3. Business and Operations
- Developing technologies and data infrastructures that can be shared in various industry sectors.
- Customise data-driven FLS services to external stakeholders needs and use of technology through open-source licenses.
- Combining and upscaling services and DS to stimulate their use in a wide range of different industry problems
4. Social Innovation
- Move computing from data centres to edge devices, making AI more accessible.
- Empowering citizens and stakeholders engagement through co-creation workshops to elicit their requirements, and user acceptance tests.
Indicative advancements beyond the state of the art include:
1. Sustainability – Reducing Environmental Impact
- Development of energy aware analytics services to minimise energy footprint, whilst being adaptable to different user needs.
- Implementing solutions to reduce learning data amount, learning time and improve energy demand accuracy.
2. Digital infrastructure and AI tools
- Creation of a Big Data Infrastructure, on top of which a toolkit of AI-enabled energy efficient algorithms will be applied to the UCs.
- Use of FL mechanisms, which use distributed data for inferring information based on decentralised collaborative modelling.
- Shift computation from cloud to personal devices to reduce data flow and potential leakage risks.
- Enable edge processing to eliminate transmission costs and achieve faster inference.
- Combination of services with the platform for new data quality management capabilities (data cleaning, validation, enrichment, co-creation, identification of bias).
3. Business and Operations
- Developing technologies and data infrastructures that can be shared in various industry sectors.
- Customise data-driven FLS services to external stakeholders needs and use of technology through open-source licenses.
- Combining and upscaling services and DS to stimulate their use in a wide range of different industry problems
4. Social Innovation
- Move computing from data centres to edge devices, making AI more accessible.
- Empowering citizens and stakeholders engagement through co-creation workshops to elicit their requirements, and user acceptance tests.