Periodic Reporting for period 2 - iMIRACLI (innovative MachIne leaRning to constrain Aerosol-cloud CLimate Impacts (iMIRACLI))
Reporting period: 2022-01-01 to 2024-06-30
Climate change is one of the most urgent problems facing mankind. Yet, the uncertainty of non-greenhouse gas perturbations (radiative forcing) associated with air pollution and its effect on clouds (aerosol-cloud interactions) limits our understanding of climate sensitivity. Progress has been hampered by the difficulty of disentangling aerosol effects on clouds and climate from their co-variability with confounding factors. Additional challenges are posed by limitations in remote sensing, low signal-to-noise ratios, and computational challenges like the scale and heterogeneity of datasets. Innovative techniques developed by the AI and machine learning community show huge potential but have not yet found their way into climate sciences – and climate scientists are currently not trained to capitalise on these advances.
The iMIRACLI ITN built on the hypothesis that merging machine learning and climate science will provide a breakthrough in the exploration of existing datasets, and advance our understanding of aerosol-cloud forcing and climate sensitivity. Its innovative training plan matched Early Career Researchers (ESRs) with supervisors from climate and data sciences as well as a non-academic advisor and secondment, and provides them with state-of-the-art data and climate science training. Partners from the non-academic sector provide training in a commercial and non-academic research settings.
The overall objective of iMIRACLI was to train and shape a new generation of climate data scientists with a solid foundation in climate sciences and competence in the latest machine learning techniques; ideally trained for employment in the academic and commercial sectors.
This innovative approach aimed to answer our top-level science question: Can we develop and expand machine learning solutions to the analysis of the exploding amounts of climate data, to deliver a breakthrough in climate research, by tracing and quantifying the impact of aerosol perturbations from the microscale to the imprints on large-scale climate?
We addressed this top-level objective through a combination of science questions (SQs) emerging from climate and data sciences.
Significant advances have been made in all areas, addressing our top-level objective and all related science questions, which were disseminated through a large number of conference presentations and publications and also presented at an international iMIRACLI workshop on machine learning for climate science at Oxford in June 2024, which also concluded the action.
The iMIRACLI ITN built on the hypothesis that merging machine learning and climate science will provide a breakthrough in the exploration of existing datasets, and advance our understanding of aerosol-cloud forcing and climate sensitivity. Its innovative training plan matched Early Career Researchers (ESRs) with supervisors from climate and data sciences as well as a non-academic advisor and secondment, and provides them with state-of-the-art data and climate science training. Partners from the non-academic sector provide training in a commercial and non-academic research settings.
The overall objective of iMIRACLI was to train and shape a new generation of climate data scientists with a solid foundation in climate sciences and competence in the latest machine learning techniques; ideally trained for employment in the academic and commercial sectors.
This innovative approach aimed to answer our top-level science question: Can we develop and expand machine learning solutions to the analysis of the exploding amounts of climate data, to deliver a breakthrough in climate research, by tracing and quantifying the impact of aerosol perturbations from the microscale to the imprints on large-scale climate?
We addressed this top-level objective through a combination of science questions (SQs) emerging from climate and data sciences.
Significant advances have been made in all areas, addressing our top-level objective and all related science questions, which were disseminated through a large number of conference presentations and publications and also presented at an international iMIRACLI workshop on machine learning for climate science at Oxford in June 2024, which also concluded the action.
The key to creating a new generation of climate data scientists was the establishment of a successful training programme.
While the first iMIRACLI in-person summer school at Oxford had to be cancelled due to the covid pandemic, it was successfully replaced with an online programme, as well as a hackathon co-organised with the Climate Informatics conference. The second summer school in Valencia had to be cancelled but was successfully replaced by an online hackathon and followed by an extra in-person spring school ahead of the 2022 EGU conference in Vienna. After the lift of covid-19 restrictions, our in-person training programme was extended to compensate for initially limited in-person interactions with further three major events: the iMIRACLI summer school 2022 in Stockholm, the iMIRACLI summer school 2023 in Patras (jointly with the EC project FORCeS to provide networking and career development opportunities) and the International iMIRACLI workshop on machine learning for climate science at Oxford from 24-28 June 2024, which also marked the conclusion of the action.
In terms of our science objective, significant advancements have been made, addressing all science questions:
SQ1 Process-level detection: we showed e.g. that limitations in previous satellite-based statistical detection and quantification of aerosol-cloud interactions can be overcome by working in radiance space rather than with retrievals as well as that there is a distinct impact of aerosols on cirrus that can be disentangled from other influencing factors using machine learning.
SQ2 Process-level attribution: extensive work was devoted to the causal attribution of process level cloud changes to aerosol perturbations. Use of opportunistic experiments, such as pollution tracks of ships, as instrumental variables allowed us to detect and quantify the cloud response to known aerosol perturbations.
SQ3 Climate change detection and attribution, e.g. using causal networks to identify and constrain the role of aerosols for daily temperature range over Europe. Causal networks were also applied to disentangle aerosol forcing responses. New network-based constraint methodologies were developed to constrain climate sensitivity.
SQ4 Learning feature representations from heterogeneous data sources: we developed e.g. new methods to detect autoconversion rates (the process of changing cloud droplets to rain droplets) from satellites, using high-resolution atmospheric model output as training dataset.
SQ5 Physically-constrained spatiotemporal modelling: we developed new methods for downscaling, addressing the challenges in refining low-resolution climate data, an essential aspect of spatio-temporal climate modelling and developed a Bayesian machine learning model (FaIRGP), to emulate surface temperatures as well as novel invertible neural network approaches for aerosol optical depth retrievals with embedded uncertainty quantification.
SQ6 Causal inference: significant methodological developments on causal discovery and inference were made, including novel techniques for causal discovery in the presence of multiple time scales, as well as the discovery of latent variables for climate time series.
iMIRACLI has delivered a significant number (>47 to date) of scientific publications (journal articles and conference proceedings) and its results were extensively disseminated through conference and workshop presentations (>80). In addition, the www.imiracli.eu webpage and our the iMIRACLI Twitter/X account @iMIRACLI_ITN were widely used for dissemination. In addition, iMIRACLI was the seed for the new UN ITU discovery series on AI for climate science, bringing together the international community and educating stakeholders and public alike.
While the first iMIRACLI in-person summer school at Oxford had to be cancelled due to the covid pandemic, it was successfully replaced with an online programme, as well as a hackathon co-organised with the Climate Informatics conference. The second summer school in Valencia had to be cancelled but was successfully replaced by an online hackathon and followed by an extra in-person spring school ahead of the 2022 EGU conference in Vienna. After the lift of covid-19 restrictions, our in-person training programme was extended to compensate for initially limited in-person interactions with further three major events: the iMIRACLI summer school 2022 in Stockholm, the iMIRACLI summer school 2023 in Patras (jointly with the EC project FORCeS to provide networking and career development opportunities) and the International iMIRACLI workshop on machine learning for climate science at Oxford from 24-28 June 2024, which also marked the conclusion of the action.
In terms of our science objective, significant advancements have been made, addressing all science questions:
SQ1 Process-level detection: we showed e.g. that limitations in previous satellite-based statistical detection and quantification of aerosol-cloud interactions can be overcome by working in radiance space rather than with retrievals as well as that there is a distinct impact of aerosols on cirrus that can be disentangled from other influencing factors using machine learning.
SQ2 Process-level attribution: extensive work was devoted to the causal attribution of process level cloud changes to aerosol perturbations. Use of opportunistic experiments, such as pollution tracks of ships, as instrumental variables allowed us to detect and quantify the cloud response to known aerosol perturbations.
SQ3 Climate change detection and attribution, e.g. using causal networks to identify and constrain the role of aerosols for daily temperature range over Europe. Causal networks were also applied to disentangle aerosol forcing responses. New network-based constraint methodologies were developed to constrain climate sensitivity.
SQ4 Learning feature representations from heterogeneous data sources: we developed e.g. new methods to detect autoconversion rates (the process of changing cloud droplets to rain droplets) from satellites, using high-resolution atmospheric model output as training dataset.
SQ5 Physically-constrained spatiotemporal modelling: we developed new methods for downscaling, addressing the challenges in refining low-resolution climate data, an essential aspect of spatio-temporal climate modelling and developed a Bayesian machine learning model (FaIRGP), to emulate surface temperatures as well as novel invertible neural network approaches for aerosol optical depth retrievals with embedded uncertainty quantification.
SQ6 Causal inference: significant methodological developments on causal discovery and inference were made, including novel techniques for causal discovery in the presence of multiple time scales, as well as the discovery of latent variables for climate time series.
iMIRACLI has delivered a significant number (>47 to date) of scientific publications (journal articles and conference proceedings) and its results were extensively disseminated through conference and workshop presentations (>80). In addition, the www.imiracli.eu webpage and our the iMIRACLI Twitter/X account @iMIRACLI_ITN were widely used for dissemination. In addition, iMIRACLI was the seed for the new UN ITU discovery series on AI for climate science, bringing together the international community and educating stakeholders and public alike.
iMIRACLI was a frontier research training and research programme that started at a time when AI for climate sciences was in its infancy. It made a significant contribution to advance the state of the art in multiple ways:
- It delivered on its promise to train and shape a new generation of climate data scientists with a solid foundation in climate sciences and competence, with an excellent cohort now entering the job market.
- It advanced the use of AI for climate science in multiple ways but in particular also on the observational side.
- It delivered significant progress in our understanding of aerosol-cloud interactions, one of the largest uncertainties in anthropogenic climate change.
As a frontier training and research project, the socio-economic impact and wider societal implications are primarily achieved through training an outstanding cohort of ESRs to fill a crucial training need in AI for climate science as well as through the dissemination of scientific results to policymakers, including through the Intergovernmental Panel on Climate Change (IPCC) assessment reports.
- It delivered on its promise to train and shape a new generation of climate data scientists with a solid foundation in climate sciences and competence, with an excellent cohort now entering the job market.
- It advanced the use of AI for climate science in multiple ways but in particular also on the observational side.
- It delivered significant progress in our understanding of aerosol-cloud interactions, one of the largest uncertainties in anthropogenic climate change.
As a frontier training and research project, the socio-economic impact and wider societal implications are primarily achieved through training an outstanding cohort of ESRs to fill a crucial training need in AI for climate science as well as through the dissemination of scientific results to policymakers, including through the Intergovernmental Panel on Climate Change (IPCC) assessment reports.