Periodic Reporting for period 2 - Smart4RES (Next Generation Modelling and Forecasting of Variable Renewable Generation for Large-scale Integration in Energy Systems and Markets)
Reporting period: 2021-05-01 to 2022-04-30
Forecasting renewable energy production is a mature discipline, but forecasting errors are still significant, in particular in challenging weather conditions. These errors impact the competitiveness of RES in electricity markets and impede a large penetration of RES into power systems.
This is why the overarching objective of Smart4RES is to develop and test next generation tools that enable an increase of at least 15 % in RES forecasting performance and enhanced value in applications by considering the entire modelling and value chain of RES forecasting.
A first objective of the project consists in defining requirements for forecasting technologies to enable near 100% RES penetration by 2030 and beyond. Based on these requirements, Smart4RES proposes RES-dedicated weather forecasting and new RES production forecasting tools, using various sources of data and developing high-resolution approaches.
Streamlining the optimal extraction of value from RES data and forecasts is crucial to enhance the value of RES forecasting and associated services. Smart4RES proposes new forecasting products, data markets and new business models in order to remunerate agents who contribute to an increase in RES forecasting quality and value in applications.
Finally, Smart4RES addresses the value of RES forecasting in power system applications such as trading in electricity markets and provision of system services to TSOs and DSOs. Data-driven solutions aim at simplifying the model chain from data to decision. Decision-aid tools are designed to enable a large penetration of RES production, combined with storage. Models will be validated in living labs and the value of forecasting will be assessed in cost benefit analysis.
This is why the overarching objective of Smart4RES is to develop and test next generation tools that enable an increase of at least 15 % in RES forecasting performance and enhanced value in applications by considering the entire modelling and value chain of RES forecasting.
A first objective of the project consists in defining requirements for forecasting technologies to enable near 100% RES penetration by 2030 and beyond. Based on these requirements, Smart4RES proposes RES-dedicated weather forecasting and new RES production forecasting tools, using various sources of data and developing high-resolution approaches.
Streamlining the optimal extraction of value from RES data and forecasts is crucial to enhance the value of RES forecasting and associated services. Smart4RES proposes new forecasting products, data markets and new business models in order to remunerate agents who contribute to an increase in RES forecasting quality and value in applications.
Finally, Smart4RES addresses the value of RES forecasting in power system applications such as trading in electricity markets and provision of system services to TSOs and DSOs. Data-driven solutions aim at simplifying the model chain from data to decision. Decision-aid tools are designed to enable a large penetration of RES production, combined with storage. Models will be validated in living labs and the value of forecasting will be assessed in cost benefit analysis.
Forecasting requirements for Smart4RES developments are described in 11 Use Cases and 8 specific KPIs, presented in Deliverable D1.1. New forecasting products covering expected developments in power systems have been identified and presented in Deliverable D1.2.
The collection of datasets presented in Deliverables D1.3 and D1.4 supports Smart4RES models development and validation.
New weather forecasting solutions adapted to the context of RES forecasting have been produced. Improved Numerical Weather Predictions (NWP) have been generated at high resolution and the physical modelling of solar irradiance has been improved. The wealth of information contained in ensembles is transformed into useful forecasting products for RES applications, such as Pseudo-Deterministic forecasts and seamless forecasts that ensure continuity over the forecasting lead-times. Local predictions at sub-minute temporal resolution are produced via different developments such as the integration of a network of all-sky-imagers, the combination of multiple data sources and the simulation of turbulent weather processes through Large Eddy Simulation. These forecasting products are presented in Deliverables D2.1 D2.2 and D2.3. The RMSE of wind speed and solar irradiance forecasts is improved in the order of 10%.
Multi-source data approaches have been proposed to improve short-term RES forecasting. Modern statistical and machine learning models efficiently exploit the information contained in new data sources including high-resolution weather data produced by Smart4RES, presented in Deliverables D3.1 and D3.2. It could be demonstrated that all approaches yield a significant improvement of RES power forecasting skill in alignment with the Forecasting KPIs of the project, mostly reaching or exceeding the project KPI target values of 9-12% (solar) and 7-9% (wind) RMSE improvement for the up to 30-min ahead forecast.
The value of data streams from distributed resources is increased by privacy-preserving collaborative and distributed learning approaches that set a new standard within the field of renewable energy forecasting, improving the global forecasting performance. This is transformed into new revenue streams for RES-related data providers thanks to the cutting-edge proposal of a data market for energy applications, relying on several methodological and application-related developments.
innovative modelling and decision-aid tools to support the integration of renewable energy in power systems and electricity markets have been developed. These tools develop advanced predictive machine-learning and optimization approaches performing dispatch of synchronous inertia, multi-criteria decision-aid tool for grid management. Prescriptive analytics combine forecasting and optimization to deploy explainable trading decisions. Distributionally robust optimization hedges trading strategies against high uncertainties in both RES production and market prices, that are notoriously difficult to predict accurately.
The collection of datasets presented in Deliverables D1.3 and D1.4 supports Smart4RES models development and validation.
New weather forecasting solutions adapted to the context of RES forecasting have been produced. Improved Numerical Weather Predictions (NWP) have been generated at high resolution and the physical modelling of solar irradiance has been improved. The wealth of information contained in ensembles is transformed into useful forecasting products for RES applications, such as Pseudo-Deterministic forecasts and seamless forecasts that ensure continuity over the forecasting lead-times. Local predictions at sub-minute temporal resolution are produced via different developments such as the integration of a network of all-sky-imagers, the combination of multiple data sources and the simulation of turbulent weather processes through Large Eddy Simulation. These forecasting products are presented in Deliverables D2.1 D2.2 and D2.3. The RMSE of wind speed and solar irradiance forecasts is improved in the order of 10%.
Multi-source data approaches have been proposed to improve short-term RES forecasting. Modern statistical and machine learning models efficiently exploit the information contained in new data sources including high-resolution weather data produced by Smart4RES, presented in Deliverables D3.1 and D3.2. It could be demonstrated that all approaches yield a significant improvement of RES power forecasting skill in alignment with the Forecasting KPIs of the project, mostly reaching or exceeding the project KPI target values of 9-12% (solar) and 7-9% (wind) RMSE improvement for the up to 30-min ahead forecast.
The value of data streams from distributed resources is increased by privacy-preserving collaborative and distributed learning approaches that set a new standard within the field of renewable energy forecasting, improving the global forecasting performance. This is transformed into new revenue streams for RES-related data providers thanks to the cutting-edge proposal of a data market for energy applications, relying on several methodological and application-related developments.
innovative modelling and decision-aid tools to support the integration of renewable energy in power systems and electricity markets have been developed. These tools develop advanced predictive machine-learning and optimization approaches performing dispatch of synchronous inertia, multi-criteria decision-aid tool for grid management. Prescriptive analytics combine forecasting and optimization to deploy explainable trading decisions. Distributionally robust optimization hedges trading strategies against high uncertainties in both RES production and market prices, that are notoriously difficult to predict accurately.
Weather forecasting innovations:
- Pseudo-deterministic forecasts were constructed to exploit the high-resolution ensemble weather predictions and better predict infra-hour variability of wind speed in a user-friendly way for RES applications.
- Large-Eddy Simulations (LES) provide realistic high-resolution weather variability on case studies with complex terrain.
- High-resolution regional forecasts of solar irradiance were derived from an array of all-sky imagers covering a large region in northern Germany.
RES production forecasting innovations:
- Integration of advanced weather forecasts into RES power forecasting models,
- RES production forecasting at high temporal resolution via machine learning solutions,
- Seamless approach for RES forecasting.
Innovations in optimal extraction of value from data:
- Development of a cutting-edge method for privacy-preserving collaborative forecasting based on distributed optimization and peer-to-peer information exchange
- Development of algorithmic solutions for a data marketplace of RES applications, where agents are remunerated based on the value of the information they share and define boundaries on the prices they are willing to buy or sell.
Innovations in decision-aid tools for large-scale penetration of renewable energy in power systems and electricity markets:
- Machine-learning based approach for fast security assessment in isolated power systems, reducing the modelling effort.
- Data-driven predictive management of distribution grids integrating prediction of the network sensitivity to constraints and hierarchical forecasting of load/RES
- Optimization of multiple products under joint operation of RES & storage, including ancillary services for isolated power systems.
- Prescriptive analytics method that simplifies the decision-making model chain and explains the impact of input data on decision cost, e.g. trading of RES
- Formalization of the population effect of RES producers, where a large penetration of renewable energy impacts electricity prices
- Pseudo-deterministic forecasts were constructed to exploit the high-resolution ensemble weather predictions and better predict infra-hour variability of wind speed in a user-friendly way for RES applications.
- Large-Eddy Simulations (LES) provide realistic high-resolution weather variability on case studies with complex terrain.
- High-resolution regional forecasts of solar irradiance were derived from an array of all-sky imagers covering a large region in northern Germany.
RES production forecasting innovations:
- Integration of advanced weather forecasts into RES power forecasting models,
- RES production forecasting at high temporal resolution via machine learning solutions,
- Seamless approach for RES forecasting.
Innovations in optimal extraction of value from data:
- Development of a cutting-edge method for privacy-preserving collaborative forecasting based on distributed optimization and peer-to-peer information exchange
- Development of algorithmic solutions for a data marketplace of RES applications, where agents are remunerated based on the value of the information they share and define boundaries on the prices they are willing to buy or sell.
Innovations in decision-aid tools for large-scale penetration of renewable energy in power systems and electricity markets:
- Machine-learning based approach for fast security assessment in isolated power systems, reducing the modelling effort.
- Data-driven predictive management of distribution grids integrating prediction of the network sensitivity to constraints and hierarchical forecasting of load/RES
- Optimization of multiple products under joint operation of RES & storage, including ancillary services for isolated power systems.
- Prescriptive analytics method that simplifies the decision-making model chain and explains the impact of input data on decision cost, e.g. trading of RES
- Formalization of the population effect of RES producers, where a large penetration of renewable energy impacts electricity prices