Periodic Reporting for period 2 - BD4OPEM (Big Data for OPen innovation Energy Marketplace)
Reporting period: 2021-07-01 to 2022-06-30
BD4OPEM is developing an open innovation marketplace through an analytic toolbox that integrates solutions based on artificial intelligence to improve the monitoring, operation, maintenance and planning of electrical distribution grids.
Energy power systems face important challenges to cope with the requirements and needs of an ever-increasing number of distributed generation and consumption devices in an interconnected world. The appropriate monitoring, acquisition and processing of the generated data can boost innovative tools and services for enhancing electrical distribution grids and customer’s electricity usage, while contributing to the required power system decarbonisation.
The overall objectives of BD4OPEM are to design, develop and deploy a marketplace for large-scale multi-party data exchanges, management, governance and real-time processing of energy-related data in smart electrical distribution grids. This will enable a reliable operation of these grids providing innovative energy services through an open, modular data analytics toolbox, and deploying a generic format for data in the data lake.
This platform will be conceived as the interface between the data providers (those stakeholders that hold and own big amounts of data) and the data analysts (those stakeholders that develop the tools that are able to collect, analyse, and elaborate the data to offer energy services to the same data providers, as well as other ESCOs, environmental authorities, prosumers, etc.). Data analysts will develop solutions that serve the DSO's and other stakeholders for a better management of their networks. This project will extract more value from the available data providing new big data solutions for the operation, planning and maintenance of highly complex networks.
Energy power systems face important challenges to cope with the requirements and needs of an ever-increasing number of distributed generation and consumption devices in an interconnected world. The appropriate monitoring, acquisition and processing of the generated data can boost innovative tools and services for enhancing electrical distribution grids and customer’s electricity usage, while contributing to the required power system decarbonisation.
The overall objectives of BD4OPEM are to design, develop and deploy a marketplace for large-scale multi-party data exchanges, management, governance and real-time processing of energy-related data in smart electrical distribution grids. This will enable a reliable operation of these grids providing innovative energy services through an open, modular data analytics toolbox, and deploying a generic format for data in the data lake.
This platform will be conceived as the interface between the data providers (those stakeholders that hold and own big amounts of data) and the data analysts (those stakeholders that develop the tools that are able to collect, analyse, and elaborate the data to offer energy services to the same data providers, as well as other ESCOs, environmental authorities, prosumers, etc.). Data analysts will develop solutions that serve the DSO's and other stakeholders for a better management of their networks. This project will extract more value from the available data providing new big data solutions for the operation, planning and maintenance of highly complex networks.
The technical progress of the project has focused on designing the BD4OPEM architecture, with the principles of scalability and interoperability, on ensuring the appropriate data integration and management, on developing the BD4OPEM Marketplace and on defining the requirements of the Big Data services.
The advances of this first period have permitted the identification and analysis of business and technical use-cases, the collection and definition of standards and protocols to be implemented and the definition of the architecture of BD4OPEM platform. BD4OPEM architecture has been built based on SGAM reference architecture (with the required identification and analysis of business and technical use-cases) and on the “4+1” View Model (logical, implementation, process, deployment and scenarios views). This architecture will be adapted to the specificities of the five pilot sites, which offer a significant amount of data from diverse sources, covering different attributes (e.g. voltage, active power), domains (energy consumption generation, failures, weather, topology), grid assets (MV/LV grid transformer, Smart Meters installed at customer premises) and also related to data dimension (volume, velocity, variety, value, veracity). This data heterogeneity is typically the first barrier to analyse and exploit data, so there is a need for harmonization of the data.
A seamless and scalable data access is being ensured from all the pilot data providers to the BD4OPEM Marketplace and to the various data end-users, for the realisation of advanced analytics services. The analysis of the involved data sources and data flows (including the identification of data formats, data granularity and communication frequency) is being used as basis for implementation of the unified interfaces for data acquisition across heterogeneous sources.
The Marketplace being created is enabling the integration of third-party services and applications, including tools for employing Business Intelligence on top of the Big Data analytics deployed for energy-related data. The solution is enabling the share of energy domain data, by adopting the an Open Innovation Marketplace, permitting the “matchmaking” among providers of data, providers of services and interested users, as well as the ability to request and create new services. The interaction between solution (or service) providers with the Data Analytics Engine is being ensured so as to facilitate the development of the BD4OPEM services, which have now the scope and requirements fully identified.
The advances of this first period have permitted the identification and analysis of business and technical use-cases, the collection and definition of standards and protocols to be implemented and the definition of the architecture of BD4OPEM platform. BD4OPEM architecture has been built based on SGAM reference architecture (with the required identification and analysis of business and technical use-cases) and on the “4+1” View Model (logical, implementation, process, deployment and scenarios views). This architecture will be adapted to the specificities of the five pilot sites, which offer a significant amount of data from diverse sources, covering different attributes (e.g. voltage, active power), domains (energy consumption generation, failures, weather, topology), grid assets (MV/LV grid transformer, Smart Meters installed at customer premises) and also related to data dimension (volume, velocity, variety, value, veracity). This data heterogeneity is typically the first barrier to analyse and exploit data, so there is a need for harmonization of the data.
A seamless and scalable data access is being ensured from all the pilot data providers to the BD4OPEM Marketplace and to the various data end-users, for the realisation of advanced analytics services. The analysis of the involved data sources and data flows (including the identification of data formats, data granularity and communication frequency) is being used as basis for implementation of the unified interfaces for data acquisition across heterogeneous sources.
The Marketplace being created is enabling the integration of third-party services and applications, including tools for employing Business Intelligence on top of the Big Data analytics deployed for energy-related data. The solution is enabling the share of energy domain data, by adopting the an Open Innovation Marketplace, permitting the “matchmaking” among providers of data, providers of services and interested users, as well as the ability to request and create new services. The interaction between solution (or service) providers with the Data Analytics Engine is being ensured so as to facilitate the development of the BD4OPEM services, which have now the scope and requirements fully identified.
The progress in information and communication technologies are contributing to a smart grids deployment in which the role of data is key. In order to take advantage of this volume and variety of data dealing also with the velocity at which it is being generated, big data analytic tools are needed.
The application of techniques like statistics, machine learning, data mining, pattern recognition, forecasting and reinforcement learning allow to extract valuable information of the big data in energy systems *. BD4OPEM Analytic Toolbox is based on this insight to go beyond the state of the art through the development of an Analytic Toolbox and through the enhancement of distribution grid data driven services including but not limited to topology identification, observability, predictive maintenance, grid simulations, impact of PV, EV and new loads, no technical losses, fraud detection, aggregator flexibility calculation, energy management systems and grid reinforcement planning.
This will result in the creation of an Open Innovation Marketplace, based on a cloud based Analytic Toolbox which ensures secures data flows from data providers to solution providers, always compliant with GDPR requirements, so that asset management is enhanced, consumer participation in energy balancing is promoted and new data-driven business models are created through innovative energy services. The project will demonstrate these features at 5 pilot sites (Spain, Turkey, Slovenia, Belgium and Denmark) with distributed energy generation.
- Replicability: Data will be collected from legacy systems and stored in a data lake. This structure enables new and existing players to link their platforms to the lake. The platform will ensure replicability and scalability, fully compatible and open to everyone.
- Socio-economics: BD4OPEM will create growth possibilities for the project members, for the energy sector in general, the European IT market and the European Sustainable Innovation Ecosystem. It will result in the creation of highly qualified jobs in the ICT and energy sectors. Also, it will reduce the technology gap between countries. The project will facilitate a technology convergence and promote innovative big data solutions for energy in countries where there is less data and technology available.
- Environment: BD4OPEM encourages a more efficient use of energy resources and the penetration of renewable energy, leading to a reduction in greenhouse gas emissions and promoting a more effective and smart usage of energy through flexibility and storage.
- Market Transformation: BD4OPEM is a clear example of interaction between different stakeholders. Exploitation and dissemination tasks will facilitate the extension of this model to the European Sustainable Innovation Ecosystem
- Policy: The topics addressed within BD4OPEM are consistent with European and international standards, policies and initiatives, aiming to develop the next generation technologies. Furthermore, the project is dedicated to advancement and enrichment of these efforts.
The application of techniques like statistics, machine learning, data mining, pattern recognition, forecasting and reinforcement learning allow to extract valuable information of the big data in energy systems *. BD4OPEM Analytic Toolbox is based on this insight to go beyond the state of the art through the development of an Analytic Toolbox and through the enhancement of distribution grid data driven services including but not limited to topology identification, observability, predictive maintenance, grid simulations, impact of PV, EV and new loads, no technical losses, fraud detection, aggregator flexibility calculation, energy management systems and grid reinforcement planning.
This will result in the creation of an Open Innovation Marketplace, based on a cloud based Analytic Toolbox which ensures secures data flows from data providers to solution providers, always compliant with GDPR requirements, so that asset management is enhanced, consumer participation in energy balancing is promoted and new data-driven business models are created through innovative energy services. The project will demonstrate these features at 5 pilot sites (Spain, Turkey, Slovenia, Belgium and Denmark) with distributed energy generation.
- Replicability: Data will be collected from legacy systems and stored in a data lake. This structure enables new and existing players to link their platforms to the lake. The platform will ensure replicability and scalability, fully compatible and open to everyone.
- Socio-economics: BD4OPEM will create growth possibilities for the project members, for the energy sector in general, the European IT market and the European Sustainable Innovation Ecosystem. It will result in the creation of highly qualified jobs in the ICT and energy sectors. Also, it will reduce the technology gap between countries. The project will facilitate a technology convergence and promote innovative big data solutions for energy in countries where there is less data and technology available.
- Environment: BD4OPEM encourages a more efficient use of energy resources and the penetration of renewable energy, leading to a reduction in greenhouse gas emissions and promoting a more effective and smart usage of energy through flexibility and storage.
- Market Transformation: BD4OPEM is a clear example of interaction between different stakeholders. Exploitation and dissemination tasks will facilitate the extension of this model to the European Sustainable Innovation Ecosystem
- Policy: The topics addressed within BD4OPEM are consistent with European and international standards, policies and initiatives, aiming to develop the next generation technologies. Furthermore, the project is dedicated to advancement and enrichment of these efforts.