Taming the Uncertainty Monster: Lessons from Astrochemistry

Project Information

TUMLA

Grant agreement ID: 101026214

DOI

10.3030/101026214

Project closed

EC signature date 29 April 2021

Start date 1 September 2021

End date 31 August 2023

Funded under

• EXCELLENT SCIENCE - Marie Skłodowska-Curie Actions

Total cost

€ 184 707,84

EU contribution

€ 184 707,84

Coordinated by

UNIVERSITE DE RENNES
France

Periodic Reporting for period 1 - TUMLA (Taming the Uncertainty Monster: Lessons from Astrochemistry)

Reporting period: 2021-09-01 to 2023-08-31

Summary of the context and overall objectives of the project

TUMLA studies the methods developed by astrochemists to assess the reliability of astrochemical models, aiming at computing the abundance of molecules in the interstellar medium (ISM) as a function of time, despite the fact that the models and the observations against which they must be validated are highly uncertain. These uncertainties, the multiplicity of tasks that models must fill, their different phases of development—from young models that need improvement to mature ones, ready to get confronted to observations—, the strong interdisciplinarity of the field make astrochemistry unique in terms of the challenges that astrochemists must face in evaluating the accuracy of their models. Such an exceptional epistemic situation represents a huge opportunity for philosophers both to refine their understanding of model evaluation and to contribute to developing new methods for model evaluation. Philosophers have not addressed the epistemic challenges generated by uncertainties in science other than climate science. The aim of TUMLA is to fill this gap, such as to contribute to an overall philosophical account of model evaluation based on the contemporary practice of scientists, and to provide concepts facilitating the communication of uncertainties to a general audience.

Hence the three main objectives defined for TUMLA:

1. A critical analysis of sensitivity analysis in astrochemistry
Astrochemists have developed innovative tools to manage uncertainties that have no counterpart in computational astrophysics, such as the identification of key chemical reactions—those that have the most important impact on the model accuracy—, through sensitivity analysis (SA). The fellow will analyze the conceptual foundations of this methodology and critically assess its application to each stage of the development of the models .

2. Interdisciplinary facilitation
The fellow’s aim is to use her background in philosophy, her interdisciplinary training in astrophysics and cosmology, as well as her experience of field work in astronomy, to facilitate and enhance an effective interdisciplinary collaboration between the experimentalists, modelers and theoreticians working in astrochemistry at the IPR.

3. A philosophical account of model evaluation that respects the diversity of roles played by models, adapts the evaluation accordingly and makes justice to the sophistication of the astrochemists’ methods.

Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far

WP1: TUMLA has allowed to provide a philosophical analysis of SA and of its essential role in guiding scientists navigating "in the dark", i.e. with very few constraints and observations available to compare to their model's outputs. This work fostered a collaboration with Dr M. Van de Sande, who decided to apply SA to astrochemical models of AGB stars. It will result in two co-authored publications in top-tier journals. Their publication has been delayed, as calculations could not be launched without taking into account the latest update of the chemical database UMIDST, released in May 2023.

WP2: The paper entitled "Extending the loop: iterative development of models in context of high uncertainties", submitted this summer, presents an account of model evaluation in context of high uncertainty, with a specific focus on the iterative processes that underly the refinement of young models. This account has been presented in an interdisciplinary symposium at the Philosophy of Science Association Biennal Meeting 2022.

WP3: 300 molecules have been detected in the ISM since 1937, with an average of 6 per year after 2005 and 74 during the last two years only. Spectroscopic observations cannot be interpreted without theoretical data such as collisional rate coefficients Yet, those have only been calculated for around 50 systems so far. Deciding which systems theoreticians should prioritize has thus become the most pressing challenge of the field. Our interdisciplinary facilitation approach has thus been devoted to propose strategies for systematizing the choice of systems, based on their potential as interdisciplinary targets, notably through a new validation method (the Snap Hook Method) accepted for publication in Philosophy of Science and the exploratory tool VARADEX to test the sensitivity of astrophysical conditions to collisional data.

WP4: Management, Dissemination and Communication of the project results (MDC)

Results were disseminated through 13 invited talks and three peer-reviewed conferences. Two international workshops have been held, one in Rennes in July 2022 (https://tumla.sciencesconf.org)) the other at the University of Western Ontario in September 2022 ( https://www.rotman.uwo.ca/event/philosophical-perspectives-on-astrochemistry/). We also presented our project to a broad audience through the official project website, the COLLEXISM team website, interviews and podcasts, and a conference given at Les Mardis de l'Espace des Sciences ( https://www.youtube.com/watch?v=Or6wxCUhy0Y).

Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far)

TUMLA has provided a full account of model development, starting with an incomplete template slowly refined through iterative methods such as sensitivity analysis, as well as different criteria allowing to embed this account in the adequacy-for-purpose view of model evaluation. Model evaluation has been a hot topic of debate during the last two decades. But tackling the issue of models that are not mature enough to be compared to observations; as well as that of their evaluation -how to decide whether model is becoming more accurate when it is not clear what "accurate" means ?- had yet to be done. We are hoping that the account provided will help to promote exploratory tools, especially to funding agencies usually privileging the predictive perspective, and to better communicate how science works in context of high uncertainties to a broad audience.

As mentioned above, the most pressing challenge today for the field is how to choose the systems of astrophysical interest upon which theoreticians should focus, given the current lack of theoretical data compared to observational needs. A. Godard-Palluet and myself have conceived an innovative "Snap Hook" method to address this challenge, that permits to indirectly validate and evaluate the accuracy of computational calculations even in the absence of experimental or observational data. The goal of the paper "Navigating in the dark", accepted for publication in Philosophy of Science, is to give to this method, that seems to be implicitly relied upon in astrochemistry, a proper formulation, in order for philosophers of science to enter the debate, and to highlight its undeniable potential in terms of interdisciplinary facilitation and knowledge transmission.

Dr. B. Desrousseaux, A. Godard-Palluet and myself have also worked on building exploratory tools allowing to test the sensitivity of astrophysical conditions to collisional rate coefficients for a given system, in order to decide whether this sensitivity justifies the computational time and cost involved. The idea consists in applying a random perturbation either to a low-accuracy set of collisional rate coefficients when available, or to those of an isotopologue, to test how astrophysical conditions responds to this change. A first version of the interface VARADEX has been presented during the 2023 workshop in Corsica, but has not yet be made public.

Group picture during the workshop held in Corsica, 2023.

Group picture of the TUMLA workshop, July 2022.

Logo of the TUMLA project

Poster of the TUMLA workshop, July 2022.