Periodic Reporting for period 1 - ARCAFF (Active Region Classification and Flare Forecasting)
Reporting period: 2022-12-01 to 2023-11-30
Solar flares and coronal mass ejections (CMEs) are the most powerful eruptive events in our solar system. Studying these phenomena can provide a unique opportunity to better understand fundamental processes on the Sun, and critically to better forecast these key space weather drivers. This research is gaining attention because of the potential impacts space weather events can have on our technologies on Earth and near-Earth space and beyond. Solar eruptions can disrupt a range of vital technologies and infrastructure, including power grids, radio communication, navigation systems, and spacecraft instrumentation.
There is an increasing need to improve the accuracy of space weather forecasts as society becomes ever more dependent on technology and space exploration moves further away from Earth with missions such as ESA Heracles, NASA Artemis, and SpaceX Interplanetary Transport System. One of the most important questions in space weather forecasting is given a set of solar observations can we predict if a flare is imminent?
To answer this question, it's essential to connect surface and atmospheric solar features (called solar signatures) to solar flares. By studying these links we can improve upon existing forecasts and create new types of forecasts, the main focus of ARCAFF. Studying these links will also provide physical insight into the processes at play addressing key scientific questions. Solar flares and their origins have been studied for decades, but finding the relationships between solar signatures and flares is challenging, analysing the large amounts of data with traditional methods has been a significant challenge.
There is an urgent need to improve space weather forecasts. Current forecasts, using statistical models or event machine learning, have limitations. The ARCAFF project aims to go beyond current capabilities using deep learning and large amounts of data to model the relationships between solar signatures and flares.
ARCAFF seeks to develop an advanced flare forecasting system using deep neural networks (DNNs). The objectives include classifying active regions, localising and classifying active regions, and predicting flares in real-time. As the project progresses, it aims to release datasets, trained DNNs, and publications. These objectives are:
Objective 1 - Active region classifications using magnetogram cutouts
Objective 2 - Active region localisation and classification using full disk magnetograms
Objective 3 - Point-in-time flare prediction using full disk magnetograms
Objective 4 - Point-in-time flare prediction using full disk multimodal observations
Objective 5 - Time series flare prediction based on time series of full disk multimodal observation.
The project's complexity increases with each objective, but it aims to add value at each stage. The trained DNNs will be deployed as live services on SolarMonitor.org. ARCAFF uses modern computing technology and infrastructure to efficiently analyse vast amounts of solar observations. The project is ambitious but measured, focusing on innovation while managing risks, ultimately advancing space weather forecasting and solar flare prediction.
There is an increasing need to improve the accuracy of space weather forecasts as society becomes ever more dependent on technology and space exploration moves further away from Earth with missions such as ESA Heracles, NASA Artemis, and SpaceX Interplanetary Transport System. One of the most important questions in space weather forecasting is given a set of solar observations can we predict if a flare is imminent?
To answer this question, it's essential to connect surface and atmospheric solar features (called solar signatures) to solar flares. By studying these links we can improve upon existing forecasts and create new types of forecasts, the main focus of ARCAFF. Studying these links will also provide physical insight into the processes at play addressing key scientific questions. Solar flares and their origins have been studied for decades, but finding the relationships between solar signatures and flares is challenging, analysing the large amounts of data with traditional methods has been a significant challenge.
There is an urgent need to improve space weather forecasts. Current forecasts, using statistical models or event machine learning, have limitations. The ARCAFF project aims to go beyond current capabilities using deep learning and large amounts of data to model the relationships between solar signatures and flares.
ARCAFF seeks to develop an advanced flare forecasting system using deep neural networks (DNNs). The objectives include classifying active regions, localising and classifying active regions, and predicting flares in real-time. As the project progresses, it aims to release datasets, trained DNNs, and publications. These objectives are:
Objective 1 - Active region classifications using magnetogram cutouts
Objective 2 - Active region localisation and classification using full disk magnetograms
Objective 3 - Point-in-time flare prediction using full disk magnetograms
Objective 4 - Point-in-time flare prediction using full disk multimodal observations
Objective 5 - Time series flare prediction based on time series of full disk multimodal observation.
The project's complexity increases with each objective, but it aims to add value at each stage. The trained DNNs will be deployed as live services on SolarMonitor.org. ARCAFF uses modern computing technology and infrastructure to efficiently analyse vast amounts of solar observations. The project is ambitious but measured, focusing on innovation while managing risks, ultimately advancing space weather forecasting and solar flare prediction.
The majority of the work performed thus far has been focused on Objective 1 - Active region Classifications using Magnetogram Cutouts and Objective 2 - Active Region Detection and Classification using full disk magnetograms. Specifically for Objective 1, the team downloaded and processed active region classification data from NOAA SWPC between 1996-01-01 and 2022-12-31, totaling approximately 10,000 files (45MB). They also processed corresponding full disk magnetograms from SOHO/MDI (~4900 files, 6.9 GB) and SDO/HMI (~4600 files, 61 GB), including rotation, scaling, and bad data flagging. Active region cutouts were created, resulting in datasets of magnetogram cutouts and tables with classifications, which were published on GitHub/Zenodo. Similarly for objective 2, DIAS and the team obtained active region bounding boxes from the SHARP database (28,000 files, 559 MB). They linked bounding boxes and active region classifications using a public database, creating a dataset consisting of full disk magnetograms and tables with classifications and bounding boxes. The code and datasets were published on GitHub/Zenodo. In both cases a review of applicable DL architectures/models was performed. Additionally mock ups of the final APIs were created and integrated into the PITHA e-Science Center in anticipation of the completed DL models. The API codes were also containerised to support their deployment in cloud environments.
Objective 3 - Point-in-time flare prediction using full disk magnetograms and Objective 4 - Point-in-time flare prediction using full disk multimodal observations required the association of solar observation to solar flare. The team obtained flare event data from the HEK (SWPC ~45,000, SSW Latest events 24,000). They started associating different flare catalogues and active regions. In both cases a review of applicable DL architectures/models was performed.
Additionally for Objective 4 the download of EUV data corresponding to previously downloaded magnetograms from the previous objective was started.
Objective 3 - Point-in-time flare prediction using full disk magnetograms and Objective 4 - Point-in-time flare prediction using full disk multimodal observations required the association of solar observation to solar flare. The team obtained flare event data from the HEK (SWPC ~45,000, SSW Latest events 24,000). They started associating different flare catalogues and active regions. In both cases a review of applicable DL architectures/models was performed.
Additionally for Objective 4 the download of EUV data corresponding to previously downloaded magnetograms from the previous objective was started.