Periodic Reporting for period 2 - SMARTHEP (SMARTHEP: Synergies between Machine leArning, Real Time analysis and Hybrid architectures for efficient Event Processing and decision making)
Reporting period: 2023-10-01 to 2025-09-30
The analysis of data plays a huge role in decision-making for many industries and continual advances means that in order to stay competitive, many organisations rely on using data to make more informed decisions about their customers, competitors, products and services. Organisations leveraging data available to them can be the difference between them keeping up with their competitors and remaining relevant to their customers.
The volume of data available to research and industry is increasing at an exponential rate. The increase in data collection is not always matched by comparable data storage, utilisation and analysis capabilities. This means that most data produced is either discarded, not recorded or recorded and stored without being analysed.
High Energy Physics (HEP) experiments have the ability to produce hundreds of gigabytes of data per second. Current resources and the time taken to make decisions about the data are not scaled to adequately process and utilise this data. In order to make the most of the data in a cost-effective way, data-taking and data-analysis needs to become more efficient. The training of a new generation of researchers to work towards Real-Time Analysis is part of the solution needed to deliver this paradigm shift.
Synergies between Machine learning, Real-Time analysis and Hybrid architectures for efficient Event Processing and decision making (SMARTHEP) is a European Training Network (ETN) with the aim of training a new generation of Early Stage Researchers (ESRs) to use real-time decision-making effectively leading to data-collection and analysis becoming synonymous.
SMARTHEP brings together scientists from the four major collaborations which have been driving the development of Real-Time analysis (RTA) and key specialists from computer science and industry. By solving concrete problems as a community, SMARTHEP will bring forward a more widespread use of RTA techniques, enabling future HEP discoveries and generating large-scale impact to industry.In addition ESRs will contribute to European growth exploiting their hands-on work to produce concrete commercial deliverables in fields that can most profit from RTA, such as transport, manufacturing, and finance.
The volume of data available to research and industry is increasing at an exponential rate. The increase in data collection is not always matched by comparable data storage, utilisation and analysis capabilities. This means that most data produced is either discarded, not recorded or recorded and stored without being analysed.
High Energy Physics (HEP) experiments have the ability to produce hundreds of gigabytes of data per second. Current resources and the time taken to make decisions about the data are not scaled to adequately process and utilise this data. In order to make the most of the data in a cost-effective way, data-taking and data-analysis needs to become more efficient. The training of a new generation of researchers to work towards Real-Time Analysis is part of the solution needed to deliver this paradigm shift.
Synergies between Machine learning, Real-Time analysis and Hybrid architectures for efficient Event Processing and decision making (SMARTHEP) is a European Training Network (ETN) with the aim of training a new generation of Early Stage Researchers (ESRs) to use real-time decision-making effectively leading to data-collection and analysis becoming synonymous.
SMARTHEP brings together scientists from the four major collaborations which have been driving the development of Real-Time analysis (RTA) and key specialists from computer science and industry. By solving concrete problems as a community, SMARTHEP will bring forward a more widespread use of RTA techniques, enabling future HEP discoveries and generating large-scale impact to industry.In addition ESRs will contribute to European growth exploiting their hands-on work to produce concrete commercial deliverables in fields that can most profit from RTA, such as transport, manufacturing, and finance.
The SMARTHEP project comprises 12 Early Stage Researchers (ESRs) from 10 countries who began their PhDs in October 2022, selected from 199 applicants representing 55 nationalities. Working across the four main Large Hadron Collider (LHC) experiments (ALICE, ATLAS, CMS, and LHCb), their focus is on real-time analysis systems known as "trigger systems." These systems perform first-pass analyses on massive data delivered by the LHC in milliseconds, selecting data for further analysis and ultimately contributing to fundamental particle measurements and new particle searches.
The project tackled a fundamental challenge of our data-rich age: how to make fast, intelligent decisions when faced with enormous streams of information. SMARTHEP's goal is to accelerate decision-making using machine learning and hybrid computing architectures, while ensuring high-quality data despite the constrained environment in which first-pass analysis is performed. Since these goals are common to industry and science, cross-talk between academia and industry is integral to the PhD projects. To reach these goals, SMARTHEP brought together universities, research institutions and industry partners, organising specialised schools covering machine learning, scientific writing, data visualisation and science communication. These were complemented by hands-on secondments at IBM, Verizon Connect and CERN, giving researchers direct experience of how cutting-edge techniques translate between academic research and commercial applications.
The researchers made substantial contributions to the trigger systems at CERN, which must filter 40 million particle collision events per second down to a manageable few thousand. Their work improved how these systems select interesting physics events in real time, directly enabling more efficient data collection. Several researchers developed novel machine learning algorithms—including graph neural networks and transformer models—that operate within extremely tight time constraints while maintaining high accuracy.
Beyond fundamental physics, ESRs developed bird's eye view perception systems for autonomous driving requiring no expensive sensor-based training data, created open-source tools for detecting financial fraud using realistic synthetic transaction data, and built traffic prediction models. These applications demonstrate how techniques from particle physics can address real-world challenges in transport safety and financial security.
The network produced over 20 open-access publications, including three major review papers on LHC triggers, machine learning for real-time analysis, and hybrid computing architectures, mostly in peer-reviewed journals and presented at major international conferences. Five researchers have completed their PhDs, with the remainder on track to finish by early 2027. Graduates have secured positions ranging from postdoctoral research to applied scientist roles in finance and technology, reflecting the programme's success in preparing versatile professionals for both academia and industry.
The project tackled a fundamental challenge of our data-rich age: how to make fast, intelligent decisions when faced with enormous streams of information. SMARTHEP's goal is to accelerate decision-making using machine learning and hybrid computing architectures, while ensuring high-quality data despite the constrained environment in which first-pass analysis is performed. Since these goals are common to industry and science, cross-talk between academia and industry is integral to the PhD projects. To reach these goals, SMARTHEP brought together universities, research institutions and industry partners, organising specialised schools covering machine learning, scientific writing, data visualisation and science communication. These were complemented by hands-on secondments at IBM, Verizon Connect and CERN, giving researchers direct experience of how cutting-edge techniques translate between academic research and commercial applications.
The researchers made substantial contributions to the trigger systems at CERN, which must filter 40 million particle collision events per second down to a manageable few thousand. Their work improved how these systems select interesting physics events in real time, directly enabling more efficient data collection. Several researchers developed novel machine learning algorithms—including graph neural networks and transformer models—that operate within extremely tight time constraints while maintaining high accuracy.
Beyond fundamental physics, ESRs developed bird's eye view perception systems for autonomous driving requiring no expensive sensor-based training data, created open-source tools for detecting financial fraud using realistic synthetic transaction data, and built traffic prediction models. These applications demonstrate how techniques from particle physics can address real-world challenges in transport safety and financial security.
The network produced over 20 open-access publications, including three major review papers on LHC triggers, machine learning for real-time analysis, and hybrid computing architectures, mostly in peer-reviewed journals and presented at major international conferences. Five researchers have completed their PhDs, with the remainder on track to finish by early 2027. Graduates have secured positions ranging from postdoctoral research to applied scientist roles in finance and technology, reflecting the programme's success in preparing versatile professionals for both academia and industry.
SMARTHEP pushed the boundaries of what is possible in real-time data processing by combining machine learning with specialised computing hardware.
The project deployed sophisticated neural networks designed to handle the complex, feature-rich data structures found in particle detectors on graphics processing units (GPUs) and field-programmable gate arrays (FPGAs).
These advances allow intelligent decision-making at speeds and scales previously unachievable, with algorithms reaching orders-of-magnitude improvements with respect to conventional approaches.
The societal impact of SMARTHEP extends well beyond particle physics. In autonomous driving and fleet safety, the project developed RendBEV, a self-supervised method enabling vehicles to understand their surroundings from dashboard cameras without requiring expensive laser-based sensors for training, potentially reducing the cost of safer driver assistance systems. In finance, researchers created open-source frameworks for generating realistic synthetic banking data, allowing fraud detection algorithms to be developed without compromising customer privacy.
The project has established lasting connections through strategic partnerships with NextGen Triggers ar CERN, the European Coalition for AI in Fundamental Sciences, and the EPIGRAPHY network, ensuring that training materials and technical expertise will influence future research programmes. An industry engagement event at CERN in May 2025 was attended by over thirty companies, creating opportunities for technology transfer between particle physics and sectors including manufacturing, transport and finance.
Looking ahead, the techniques developed are positioned to benefit the upcoming High-Luminosity upgrade of the Large Hadron Collider. More broadly, the project has demonstrated that the extreme data challenges of particle physics produce solutions applicable wherever fast, accurate decisions must be made from complex data streams from detecting anomalies in industrial processes to improving road safety through real-time video analysis.
The project deployed sophisticated neural networks designed to handle the complex, feature-rich data structures found in particle detectors on graphics processing units (GPUs) and field-programmable gate arrays (FPGAs).
These advances allow intelligent decision-making at speeds and scales previously unachievable, with algorithms reaching orders-of-magnitude improvements with respect to conventional approaches.
The societal impact of SMARTHEP extends well beyond particle physics. In autonomous driving and fleet safety, the project developed RendBEV, a self-supervised method enabling vehicles to understand their surroundings from dashboard cameras without requiring expensive laser-based sensors for training, potentially reducing the cost of safer driver assistance systems. In finance, researchers created open-source frameworks for generating realistic synthetic banking data, allowing fraud detection algorithms to be developed without compromising customer privacy.
The project has established lasting connections through strategic partnerships with NextGen Triggers ar CERN, the European Coalition for AI in Fundamental Sciences, and the EPIGRAPHY network, ensuring that training materials and technical expertise will influence future research programmes. An industry engagement event at CERN in May 2025 was attended by over thirty companies, creating opportunities for technology transfer between particle physics and sectors including manufacturing, transport and finance.
Looking ahead, the techniques developed are positioned to benefit the upcoming High-Luminosity upgrade of the Large Hadron Collider. More broadly, the project has demonstrated that the extreme data challenges of particle physics produce solutions applicable wherever fast, accurate decisions must be made from complex data streams from detecting anomalies in industrial processes to improving road safety through real-time video analysis.