Periodic Reporting for period 2 - REALDARK (REAL-time discovery strategies for DARK matter and dark sector signals at the ATLAS detector with Run-3 LHC data)
Reporting period: 2023-05-01 to 2024-10-31
In recent years, CERN scientists made a groundbreaking discovery at the Large Hadron Collider (LHC), uncovering a particle known as the Higgs boson. This finding confirmed a theory called the Standard Model, which describes the fundamental particles and forces that make up our universe. However, while the Standard Model explains most of the ordinary matter we’re made of, it falls short in explaining a mysterious substance called Dark Matter.
Dark Matter is an invisible and elusive form of matter that makes up a significant portion of the universe's mass. We can't see it directly, but we know it's there because of its gravitational effects on galaxies and other cosmic structures. Despite its importance, the Standard Model doesn't account for Dark Matter, leaving scientists, and humanity, puzzled.
At the Large Hadron Collider, we could be producing dark matter and related particles from collisions of ordinary matter, which occur 40 millions of times per second. This is an example of a challenge that both physics and astronomy are facing: the amount of data being collected is growing in size and complexity, but our ability to store, use, and analyse this data isn't growing accordingly. This is an issue that is faced by both research and society given the current “data deluge”. The consequence of this mismatch between data volumes and resources for our quest for dark matter is that we are blind to a number of viable dark matter hypotheses when using traditional data-taking and data processing techniques.
The overall objective of this project is to devise, implement and take data using innovative techniques at the LHC. We use real-time analysis (analysing as much data as possible, as soon as it’s recorded by an experiment) and machine learning methods to compress the data.
Within this project, we deploy tools and techniques following Open Science / FAIR (Findable, Accessible, Interoperable and Reproducible) principles for data and software, adding a focus on the energetic sustainability of software.
Dark Matter is an invisible and elusive form of matter that makes up a significant portion of the universe's mass. We can't see it directly, but we know it's there because of its gravitational effects on galaxies and other cosmic structures. Despite its importance, the Standard Model doesn't account for Dark Matter, leaving scientists, and humanity, puzzled.
At the Large Hadron Collider, we could be producing dark matter and related particles from collisions of ordinary matter, which occur 40 millions of times per second. This is an example of a challenge that both physics and astronomy are facing: the amount of data being collected is growing in size and complexity, but our ability to store, use, and analyse this data isn't growing accordingly. This is an issue that is faced by both research and society given the current “data deluge”. The consequence of this mismatch between data volumes and resources for our quest for dark matter is that we are blind to a number of viable dark matter hypotheses when using traditional data-taking and data processing techniques.
The overall objective of this project is to devise, implement and take data using innovative techniques at the LHC. We use real-time analysis (analysing as much data as possible, as soon as it’s recorded by an experiment) and machine learning methods to compress the data.
Within this project, we deploy tools and techniques following Open Science / FAIR (Findable, Accessible, Interoperable and Reproducible) principles for data and software, adding a focus on the energetic sustainability of software.
In the period from the beginning of the project to the end of the period covered by the report, we want to highlight the following work and results achieved so far:
(a) We deployed all software needed to take data in real-time using two new techniques that reduce the data volume by keeping a performance comparable to traditional techniques, and increase the amount of data that can be taken and would otherwise have been discarded. We have used this software to take data since the LHC start-up in Summer 2022. These techniques are described in [https://arxiv.org/abs/2401.06630 published in JINST]. We are currently analysing this data for signs of dark matter-related particles.
(b) We are leading the work on the calibration of the data we have recorded and contribute to the evaluation of its performance as data is taken, and a PDRA in the project co-coordinates the overall ATLAS experiment work on this kind of calibration.
(c) We have published two peer-reviewed papers within the ATLAS collaboration on searches for new particles that would be evidence of a dark matter model that has not yet been fully probed, called dark QCD [https://arxiv.org/abs/2311.03944 published in JHEP, ATLAS briefing at https://atlas.cern/Updates/Briefing/Dark-Jets] [https://arxiv.org/abs/2305.18037 published in PLB, ATLAS briefing at https://atlas.cern/Updates/Briefing/Semi-Visible-Jets](opens in new window). No evidence for these new particles has been found, so we have set constraints on the parameters of different models that will inform future searches.
(d) We are working on understanding and harmonising benchmarks for dark matter and dark sector searches, at collider and in the context of the global dark matter search across multiple experiments. A PDRA in the project has organised and joined workshops with theory and cross-experiment experts for preliminary discussions. The peer-reviewed papers and whitepapers we have edited and contributed to can be found at [https://arxiv.org/abs/2203.12035 https://arxiv.org/abs/2206.03456(opens in new window) https://arxiv.org/abs/2209.13128 https://arxiv.org/abs/2211.07027(opens in new window) https://arxiv.org/abs/2210.01770](opens in new window). In this context, we also facilitate cross-talk and information finding through the iDMEu platform (www.idmeu.org included in proceedings from the TAUP conference at https://arxiv.org/abs/2312.14192(opens in new window)) that we developed within this project.
(e) We have developed an Open-Source software package to compress data using Machine Learning, called Baler, that can be found on GitHub [https://github.com/baler-collaboration/baler] and [https://doi.org/10.5281/zenodo.10723669]. This is still a prototype and we have not reached the ultimate performance we aim for, but it has already shown promising results in particle physics and a number of fields.
(f) We work on Open Science tools and software sustainability. We delivered a working prototype of the Dark Matter Science Project for FAIR and reproducible data analysis from different experiments searching for dark matter at [https://eoscfuture.eu/data/dark-matter/]. In terms of software sustainability, we coordinate and take part in various activities about FAIR and sustainable data, software and machine learning tools. We have used initial studies by undergraduates students within this grant to join more systematic efforts to pursue these topics in parallel.
(e) The postdocs and students in the group are presenting their work at national meetings and international conferences, as well as organising workshops. Relevant proceedings and workshop reports can be found on [https://www.epj-conferences.org/articles/epjconf/abs/2024/05/epjconf_chep2024_02030/epjconf_chep2024_02030.html https://arxiv.org/abs/2311.16330](opens in new window)
(a) We deployed all software needed to take data in real-time using two new techniques that reduce the data volume by keeping a performance comparable to traditional techniques, and increase the amount of data that can be taken and would otherwise have been discarded. We have used this software to take data since the LHC start-up in Summer 2022. These techniques are described in [https://arxiv.org/abs/2401.06630 published in JINST]. We are currently analysing this data for signs of dark matter-related particles.
(b) We are leading the work on the calibration of the data we have recorded and contribute to the evaluation of its performance as data is taken, and a PDRA in the project co-coordinates the overall ATLAS experiment work on this kind of calibration.
(c) We have published two peer-reviewed papers within the ATLAS collaboration on searches for new particles that would be evidence of a dark matter model that has not yet been fully probed, called dark QCD [https://arxiv.org/abs/2311.03944 published in JHEP, ATLAS briefing at https://atlas.cern/Updates/Briefing/Dark-Jets] [https://arxiv.org/abs/2305.18037 published in PLB, ATLAS briefing at https://atlas.cern/Updates/Briefing/Semi-Visible-Jets](opens in new window). No evidence for these new particles has been found, so we have set constraints on the parameters of different models that will inform future searches.
(d) We are working on understanding and harmonising benchmarks for dark matter and dark sector searches, at collider and in the context of the global dark matter search across multiple experiments. A PDRA in the project has organised and joined workshops with theory and cross-experiment experts for preliminary discussions. The peer-reviewed papers and whitepapers we have edited and contributed to can be found at [https://arxiv.org/abs/2203.12035 https://arxiv.org/abs/2206.03456(opens in new window) https://arxiv.org/abs/2209.13128 https://arxiv.org/abs/2211.07027(opens in new window) https://arxiv.org/abs/2210.01770](opens in new window). In this context, we also facilitate cross-talk and information finding through the iDMEu platform (www.idmeu.org included in proceedings from the TAUP conference at https://arxiv.org/abs/2312.14192(opens in new window)) that we developed within this project.
(e) We have developed an Open-Source software package to compress data using Machine Learning, called Baler, that can be found on GitHub [https://github.com/baler-collaboration/baler] and [https://doi.org/10.5281/zenodo.10723669]. This is still a prototype and we have not reached the ultimate performance we aim for, but it has already shown promising results in particle physics and a number of fields.
(f) We work on Open Science tools and software sustainability. We delivered a working prototype of the Dark Matter Science Project for FAIR and reproducible data analysis from different experiments searching for dark matter at [https://eoscfuture.eu/data/dark-matter/]. In terms of software sustainability, we coordinate and take part in various activities about FAIR and sustainable data, software and machine learning tools. We have used initial studies by undergraduates students within this grant to join more systematic efforts to pursue these topics in parallel.
(e) The postdocs and students in the group are presenting their work at national meetings and international conferences, as well as organising workshops. Relevant proceedings and workshop reports can be found on [https://www.epj-conferences.org/articles/epjconf/abs/2024/05/epjconf_chep2024_02030/epjconf_chep2024_02030.html https://arxiv.org/abs/2311.16330](opens in new window)
Overall, this project represents a step forward in the quest to unravel the mysteries of Dark Matter using new data-taking techniques and tools that can also be extended to other fields. With cutting-edge software development for real-time analysis, expertise in dark matter at collider experiment, and by working with collaborators to obtain complementary theoretical and experimental insights, we are part of a worldwide effort towards a deeper understanding of one of the universe's greatest enigmas.
The deployment of new ways of collecting data in the ATLAS experiment represents progress beyond the state of the art. For the first time, ATLAS can perform full physics analyses using high-level quantities for various types of physics objects recorded in real-time. This expansion allows for a much broader range of studies, particularly improving our ability to search for dark matter. Additionally, the second new technique we have deployed allows access to selected raw data, and enables probing more complex new physics models where dark matter is only one of the many new particles that are part of a so-called dark sector.
These improvements, made in collaboration with other LHC experiments, will also be crucial for handling the increasing amount of data expected in the next data-taking period of the LHC, where data rates will further increase.
We are currently analysing the data recorded with these new techniques as it comes in, iterating with the theory community to understand the best benchmarks to represent wide classes of models, and ensuring that the data analysis software we write is FAIR and sustainable (in collaboration with computer scientists at the University of Manchester and beyond) and implemented on the Virtual Research Environment platform so that others can reproduce our results. Until the end of the project, we expect to complete the ongoing physics analyses and publish both the results of these searches as well as papers about their interpretation and contextualisation with respect to other experiments and future developments in the field.
Another exciting development from this first period is the success of the machine learning data compression tool that we developed, which shows great promise not only for particle physics but for other fields as well. While more work is planned before the end of the project to move from prototype to production, the early results are very encouraging and suggest this tool could lead to further progress beyond the state of the art.
The deployment of new ways of collecting data in the ATLAS experiment represents progress beyond the state of the art. For the first time, ATLAS can perform full physics analyses using high-level quantities for various types of physics objects recorded in real-time. This expansion allows for a much broader range of studies, particularly improving our ability to search for dark matter. Additionally, the second new technique we have deployed allows access to selected raw data, and enables probing more complex new physics models where dark matter is only one of the many new particles that are part of a so-called dark sector.
These improvements, made in collaboration with other LHC experiments, will also be crucial for handling the increasing amount of data expected in the next data-taking period of the LHC, where data rates will further increase.
We are currently analysing the data recorded with these new techniques as it comes in, iterating with the theory community to understand the best benchmarks to represent wide classes of models, and ensuring that the data analysis software we write is FAIR and sustainable (in collaboration with computer scientists at the University of Manchester and beyond) and implemented on the Virtual Research Environment platform so that others can reproduce our results. Until the end of the project, we expect to complete the ongoing physics analyses and publish both the results of these searches as well as papers about their interpretation and contextualisation with respect to other experiments and future developments in the field.
Another exciting development from this first period is the success of the machine learning data compression tool that we developed, which shows great promise not only for particle physics but for other fields as well. While more work is planned before the end of the project to move from prototype to production, the early results are very encouraging and suggest this tool could lead to further progress beyond the state of the art.