Periodic Reporting for period 2 - BD4NRG (BD4NRG: Big Data for Next Generation Energy)
Período documentado: 2022-07-01 hasta 2023-12-31
Rising decentralization of the energy system is unveiling an enormous opportunity for energy stakeholders to leverage on big data & AI technologies to improve decision making and prepare the ground for scalable and efficient consumer-centric energy systems, bringing active consumers centre stage. There are however barriers hampering the exploitation of this potential, such as lack of understanding from utilities side on the added value from the availability of increasing data amounts, insufficient culture of data sharing among B2B operators, and utilities with active energy consumers/prosumers, and lack of standardized big data architectures for smart grids and regulatory frameworks not enabling data sharing.
Overall BD4NRG aims at evolving, upscaling and demonstrating an innovative energy-tailored Big Data Analytics Toolbox (BD4NRG Toolbox), significantly contributing to achieve a technoeconomic optimal management of the Electric Power and Energy Systems value chain.
BD4NRG will
i) deliver a Smart Energy Reference Architecture, aligning BDVA SRIA, IDSA and FIWARE architectures, SAREF standard and extend COSMAG specification enabling B2B multi-party data exchange, while providing interoperability of leading-edge big data technologies with smart grid standards and operational frameworks
ii) deploy TRL 5-6 technology enablers, such as scalable sovereignty-preserving hybrid DLT/off-chain data governance, big data elastic pipeline orchestration, IoT/edge AI-based federated learning and multi-resource sharing tokenized marketplace, loosely integrate and deploy them within the BD4NRG framework
iii) deliver a TRL8 open modular big data analytics toolbox as front-end for one-stop-shop analytics services development by orchestrating legacy and/or third party assets (data, computing resources, models, algorithms)
iv) validate its framework, by delivering predictive and prescriptive AI-based big data analytics on 12 large scale pilots, deployed by different energy stakeholders (TSOs, DSOs, aggregators, storage/renewable assets operators, energy communities, ESCOs, municipalities, financial institutions), fully covering the energy value chain
v) setup a data-driven ecosystem in which new energy data providers will be federated, attract SMEs for novel energy services provisioning through cascading funding and validate a hybrid energy/industry value chain supporting B2B joint digital platforms
Overall BD4NRG aims at evolving, upscaling and demonstrating an innovative energy-tailored Big Data Analytics Toolbox (BD4NRG Toolbox), significantly contributing to achieve a technoeconomic optimal management of the Electric Power and Energy Systems value chain.
BD4NRG will
i) deliver a Smart Energy Reference Architecture, aligning BDVA SRIA, IDSA and FIWARE architectures, SAREF standard and extend COSMAG specification enabling B2B multi-party data exchange, while providing interoperability of leading-edge big data technologies with smart grid standards and operational frameworks
ii) deploy TRL 5-6 technology enablers, such as scalable sovereignty-preserving hybrid DLT/off-chain data governance, big data elastic pipeline orchestration, IoT/edge AI-based federated learning and multi-resource sharing tokenized marketplace, loosely integrate and deploy them within the BD4NRG framework
iii) deliver a TRL8 open modular big data analytics toolbox as front-end for one-stop-shop analytics services development by orchestrating legacy and/or third party assets (data, computing resources, models, algorithms)
iv) validate its framework, by delivering predictive and prescriptive AI-based big data analytics on 12 large scale pilots, deployed by different energy stakeholders (TSOs, DSOs, aggregators, storage/renewable assets operators, energy communities, ESCOs, municipalities, financial institutions), fully covering the energy value chain
v) setup a data-driven ecosystem in which new energy data providers will be federated, attract SMEs for novel energy services provisioning through cascading funding and validate a hybrid energy/industry value chain supporting B2B joint digital platforms
By incorporating the context of 12 pilot demonstrators, 10 main beneficiaries were identified with 39 pilot actors, 17 use cases and 69 user requirements. WP2 focused on these aspects leading to the BD4NRG reference architecture (D2.6) which leveraged European energy data exchange reference architectures, within the Data Management Working Group framework of DG Energy BRIDGE Cluster.
The definition of a Data Governance Layer, and establishment of a unified system architecture are the main outcomes of WP3, which analysed the BD4NRG Governance Layer and IDSA conceptual architectures integration. In the 2nd reporting period WP3 activities intensified and closer collaboration between partners was established. The components developed for the 1st Technology release were further enhanced, for the 2nd and 3rd technology releases respectively, reaching their final version at the end of the 3rd development cycle.
WP4 activities focused on approaches to adopt an edge/fog/cloud architecture to process energy big data. Analytics services have been developed, such as classification, regression, forecasting and optimization problems, to be used under pilot demonstrators’ context. A first version of these analytics is reported (D4.6) while the main achievements in the second reporting included Federated catalogue and Vocabulary Hub and Cross-stakeholder Transfer learning (TL). Visual Analytics services based on Apache Superset, BD4NRG Analytics Toolbox, based on Eclipse Graphene and integrated with BD4NRG Marketplace (WP5) and with Identity Manager (WP3) are also deployed. Analytics services/models in Marketplace can be automatically onboarded in Toolbox.
WP5 activities are related to BD4NRG Marketplace for digital assets monetisation and included the definitions of the main functionalities to be implemented through the Application/Business scenario process. A final release of the marketplace is available, establishing linkage to the other BD4NRG components.
The pilot demonstrators successfully tested and validated BD4NRG tools and solutions. In D6.1 the Pilot Specific KPIs were identified to validate the Use Cases’ impact, while the Pilot Deployment Plan proposed the methodology to be followed in the Evaluation Cycles. The Evaluation Cycles have been reported (D6.2 D6.3 and D6.4) with the first 2 providing the necessary feedback on the corresponding Technology Releases, while the final one focused on the M&V plan and providing an overall assessment of BD4NRG pilot demonstrators. In the dedicated pilot WPs (WP7 for BD-4-DER pilots, WP8 for BD-4-NET and WP9 for BD-4-ENEF pilots) the work was guided through WP6 overall pilot activities, and resulted in 12 Pilot Documentation Deliverables, covering the activities carried out within the demonstrators and depicting their full picture and their results. The overall impact assessment, adoption guidelines and evaluation of BD4NRG tools and solutions towards the pilot demonstrators has been generated, providing an overview of BD4NRG and presenting its most significant outcomes in respect to its demonstrators (D6.4).
Dissemination activities (WP11) dealt with project branding and online presence, communication material creation, standardization activities support, related programs/initiatives synergies and the exploitation and business plan identification. BD4NRG was very active in dissemination activities, having collaborated in numerous events with other initiatives/projects (outside the EC funded projects ecosystem too). WP1 underwent through an extensive market analysis with a view to optimize the strategic market positioning of the main project outcomes, and made available a variety of potential validated business models. In that respect FSTP through Open Call was a very useful exercise to test and validate start-up oriented models generated by BD4NRG.
The definition of a Data Governance Layer, and establishment of a unified system architecture are the main outcomes of WP3, which analysed the BD4NRG Governance Layer and IDSA conceptual architectures integration. In the 2nd reporting period WP3 activities intensified and closer collaboration between partners was established. The components developed for the 1st Technology release were further enhanced, for the 2nd and 3rd technology releases respectively, reaching their final version at the end of the 3rd development cycle.
WP4 activities focused on approaches to adopt an edge/fog/cloud architecture to process energy big data. Analytics services have been developed, such as classification, regression, forecasting and optimization problems, to be used under pilot demonstrators’ context. A first version of these analytics is reported (D4.6) while the main achievements in the second reporting included Federated catalogue and Vocabulary Hub and Cross-stakeholder Transfer learning (TL). Visual Analytics services based on Apache Superset, BD4NRG Analytics Toolbox, based on Eclipse Graphene and integrated with BD4NRG Marketplace (WP5) and with Identity Manager (WP3) are also deployed. Analytics services/models in Marketplace can be automatically onboarded in Toolbox.
WP5 activities are related to BD4NRG Marketplace for digital assets monetisation and included the definitions of the main functionalities to be implemented through the Application/Business scenario process. A final release of the marketplace is available, establishing linkage to the other BD4NRG components.
The pilot demonstrators successfully tested and validated BD4NRG tools and solutions. In D6.1 the Pilot Specific KPIs were identified to validate the Use Cases’ impact, while the Pilot Deployment Plan proposed the methodology to be followed in the Evaluation Cycles. The Evaluation Cycles have been reported (D6.2 D6.3 and D6.4) with the first 2 providing the necessary feedback on the corresponding Technology Releases, while the final one focused on the M&V plan and providing an overall assessment of BD4NRG pilot demonstrators. In the dedicated pilot WPs (WP7 for BD-4-DER pilots, WP8 for BD-4-NET and WP9 for BD-4-ENEF pilots) the work was guided through WP6 overall pilot activities, and resulted in 12 Pilot Documentation Deliverables, covering the activities carried out within the demonstrators and depicting their full picture and their results. The overall impact assessment, adoption guidelines and evaluation of BD4NRG tools and solutions towards the pilot demonstrators has been generated, providing an overview of BD4NRG and presenting its most significant outcomes in respect to its demonstrators (D6.4).
Dissemination activities (WP11) dealt with project branding and online presence, communication material creation, standardization activities support, related programs/initiatives synergies and the exploitation and business plan identification. BD4NRG was very active in dissemination activities, having collaborated in numerous events with other initiatives/projects (outside the EC funded projects ecosystem too). WP1 underwent through an extensive market analysis with a view to optimize the strategic market positioning of the main project outcomes, and made available a variety of potential validated business models. In that respect FSTP through Open Call was a very useful exercise to test and validate start-up oriented models generated by BD4NRG.
Main progress of BD4NRG project consisted of:
i) the BD4NRG-GOVERNANCE encompassing modules related to data collection, handling, semantic annotation, integration, distributed storage.
ii) the BD4NRG-PROCESSING/ANALYTICS including ML/DL models and providing a set of analytics tools as a service (SaaS/PaaS/IaaS models). BD4NRG defined 3 main categories of cross-stakeholder energy and non-energy services that developed and tested in Large-Scale Pilots context. The Toolbox is able to handle heterogeneous data types from multiple domains and sources. Collected data can be processed to create different business cases considering the interaction of different stakeholders and scales emerged by the LSPs.
iii) the BD4NRG-MARKETPLACE, delivering a participatory Smart grid blockchain-based enabler, in which B2B/B2C players may share and exchange tokenized heterogeneous assets, ranging from energy data to energy consuming/generating assets and services.
BD4NRG supported the creation and exploitation of innovative services, by utilizing a wide variety of data, for safe and effective operation of energy networks and renewable generation assets.
i) the BD4NRG-GOVERNANCE encompassing modules related to data collection, handling, semantic annotation, integration, distributed storage.
ii) the BD4NRG-PROCESSING/ANALYTICS including ML/DL models and providing a set of analytics tools as a service (SaaS/PaaS/IaaS models). BD4NRG defined 3 main categories of cross-stakeholder energy and non-energy services that developed and tested in Large-Scale Pilots context. The Toolbox is able to handle heterogeneous data types from multiple domains and sources. Collected data can be processed to create different business cases considering the interaction of different stakeholders and scales emerged by the LSPs.
iii) the BD4NRG-MARKETPLACE, delivering a participatory Smart grid blockchain-based enabler, in which B2B/B2C players may share and exchange tokenized heterogeneous assets, ranging from energy data to energy consuming/generating assets and services.
BD4NRG supported the creation and exploitation of innovative services, by utilizing a wide variety of data, for safe and effective operation of energy networks and renewable generation assets.