Periodic Reporting for period 1 - COMPACT (Understanding gravity using a COMprehensive search for fast-spinning Pulsars And CompacT binaries)
Periodo di rendicontazione: 2023-05-01 al 2025-10-31
COMPACT is a tailor-made pulsar search programme that uses the MeerKAT and Effelsberg telescopes to discover two exotic classes of pulsars,
- Class I: Pulsars that are in highly relativistic binary orbits around other neutron stars (NSs), white dwarves (WDs) or black holes (BH) with orbital periods of a few hours or less, and
- Class II: pulsars with extremely fast spin periods of the order of a millisecond or less.
Such discoveries will provide insights into a multitude of fundamental physics and astrophysics, ranging from constraints on neutron star interiors, to performing better, and at times unique tests of gravity. These discoveries are also natural gravitational wave emitters; relativistic binary pulsar systems discovered with COMPACT would be observable with future space-based gravitational-wave detectors, such as LISA. Depending on potential asymmetries in the shape of neutron stars, ultrafast rotators discovered with COMPACT may already be detected with current ground-based gravitational-wave detectors, like LIGO and Virgo. Multi-messenger observations of this kind would be ideal test-beds for our understanding of gravity and matter under extreme conditions.
- Class I: Pulsars that are in highly relativistic binary orbits around other neutron stars (NSs), white dwarves (WDs) or black holes (BH) with orbital periods of a few hours or less, and
- Class II: pulsars with extremely fast spin periods of the order of a millisecond or less.
Such discoveries will provide insights into a multitude of fundamental physics and astrophysics, ranging from constraints on neutron star interiors, to performing better, and at times unique tests of gravity. These discoveries are also natural gravitational wave emitters; relativistic binary pulsar systems discovered with COMPACT would be observable with future space-based gravitational-wave detectors, such as LISA. Depending on potential asymmetries in the shape of neutron stars, ultrafast rotators discovered with COMPACT may already be detected with current ground-based gravitational-wave detectors, like LIGO and Virgo. Multi-messenger observations of this kind would be ideal test-beds for our understanding of gravity and matter under extreme conditions.
COMPACT formally launched on 1 May 2023. The implementation of COMPACT is split into different working groups. The first 18 months were slotted firstly to technical work on purchasing and installing the various computing systems needed for recording and processing data and in parallel, also to write the required software to process these PB scale data acquisition in a reasonable time. The duration of 19-60 months were slotted for the science working groups involved in discovery and follow up of the two classes of COMPACT targets mentioned above. These work packages also include novel methodologies that go beyond the state of the art of current pipelines.
1. A pulsar injector: We built a software that is capable of injecting fake pulsars. This tool can probabilistically inject any flavor of pulsar signal into real astrophysical data. The probabilistic injection allows the tool to inject at any arbitrarily low signal-to-noise ratio (SNR), a key feature missing from many existing pulsar injectors. This tool has an extensive range of customisable features to control the nature of the injected pulsar. Intrinsic features include pulse period, higher order period derivatives, duty-cycle, custom pulse-profile shapes, pulse microstructure and spectral index. Binary motion can be tuned via a linear acceleration or a full five Keplerian orbital model. Customisable observational effects include on-sky position, inter-stellar medium (ISM) scattering and scintillation, intra-channel dispersion measure (DM) smearing, and solar system barycentering. With this software as a base, we are developing an injection pipeline to understand the actual sensitivity of our searches to different parameters
2. A novel scheme for finding relativistic binaries: In the COMPACT searches, in addition to the standard acceleration searches, we are performing full 3 & 5-Keplerian parameter searches. These require an enormous number of search trials, which, combined with the large number of beams being produced, it becomes computationally infeasible to run all of these searches. We have developed a novel algorithm called COMPASS, designed to quickly find potential binary pulsar candidates that can reduce the parameter space for the Keplerian parameter searches. This algorithm uses template matching of channel-pair cross-correlations to simultaneously probe any desired DM space for periodic, dispersed signals. By testing and developing this algorithm using the pulsar injector, the current version is able to detect pulsars with S/N < 20 in binary orbits outside the sensitivity range of acceleration/jerk searches. Using the first COMPACT observation of the globular cluster NGC1851, the algorithm was successfully able to blindly find real pulsars. Future plans include a pruning/optimization addition to the algorithm, allowing for efficient searching of highly accelerated binary pulsar signals and the development of a new tool for determining the unknown DM of globular clusters with no pulsars.
3. A robust candidate classifier: Due to the high volume of pulsar candidates, manual inspection is impractical, making automated classification essential. This project addresses limitations in existing classifiers by developing a universal feature extraction tool and a deep learning classifier, using the largest and most diverse pulsar dataset, including tests on injected signals. The feature extractor processes various pulsar data formats and standardizes them for faster, consistent analysis. Before we build the ultimate classifier, we initially try out several so-called “classical” machine learning (ML) architectures that work on a reduced representation of the data using basic 1D features like statistical moments. We use this as a baseline performance that any deep learning architecture will need to surpass.
Most of the following are technical achievements, akin to the planning of the project.
1. Obtaining baseband data from MeerKAT: We can record data streaming at 1.6 Tb/s realtime onto our system, which is the only system in the world across any telescope that is capable of recording at this rate.
2. With our limited human resources, we have now built a way to run pulsar searches (which can be extended to any general computation) across a network of high performance clusters with minimal human intervention. This is publicly available on our github repository. This and all other software developed for the project is public and can be found here: https://github.com/erc-compact(si apre in una nuova finestra).
3. We are currently one of only 2 groups in the world that can perform targeted Keplerian searches for relativistic orbits. We also are the only group with data completely sensitive to ultra-fast rotating pulsars.
1. A pulsar injector: We built a software that is capable of injecting fake pulsars. This tool can probabilistically inject any flavor of pulsar signal into real astrophysical data. The probabilistic injection allows the tool to inject at any arbitrarily low signal-to-noise ratio (SNR), a key feature missing from many existing pulsar injectors. This tool has an extensive range of customisable features to control the nature of the injected pulsar. Intrinsic features include pulse period, higher order period derivatives, duty-cycle, custom pulse-profile shapes, pulse microstructure and spectral index. Binary motion can be tuned via a linear acceleration or a full five Keplerian orbital model. Customisable observational effects include on-sky position, inter-stellar medium (ISM) scattering and scintillation, intra-channel dispersion measure (DM) smearing, and solar system barycentering. With this software as a base, we are developing an injection pipeline to understand the actual sensitivity of our searches to different parameters
2. A novel scheme for finding relativistic binaries: In the COMPACT searches, in addition to the standard acceleration searches, we are performing full 3 & 5-Keplerian parameter searches. These require an enormous number of search trials, which, combined with the large number of beams being produced, it becomes computationally infeasible to run all of these searches. We have developed a novel algorithm called COMPASS, designed to quickly find potential binary pulsar candidates that can reduce the parameter space for the Keplerian parameter searches. This algorithm uses template matching of channel-pair cross-correlations to simultaneously probe any desired DM space for periodic, dispersed signals. By testing and developing this algorithm using the pulsar injector, the current version is able to detect pulsars with S/N < 20 in binary orbits outside the sensitivity range of acceleration/jerk searches. Using the first COMPACT observation of the globular cluster NGC1851, the algorithm was successfully able to blindly find real pulsars. Future plans include a pruning/optimization addition to the algorithm, allowing for efficient searching of highly accelerated binary pulsar signals and the development of a new tool for determining the unknown DM of globular clusters with no pulsars.
3. A robust candidate classifier: Due to the high volume of pulsar candidates, manual inspection is impractical, making automated classification essential. This project addresses limitations in existing classifiers by developing a universal feature extraction tool and a deep learning classifier, using the largest and most diverse pulsar dataset, including tests on injected signals. The feature extractor processes various pulsar data formats and standardizes them for faster, consistent analysis. Before we build the ultimate classifier, we initially try out several so-called “classical” machine learning (ML) architectures that work on a reduced representation of the data using basic 1D features like statistical moments. We use this as a baseline performance that any deep learning architecture will need to surpass.
Most of the following are technical achievements, akin to the planning of the project.
1. Obtaining baseband data from MeerKAT: We can record data streaming at 1.6 Tb/s realtime onto our system, which is the only system in the world across any telescope that is capable of recording at this rate.
2. With our limited human resources, we have now built a way to run pulsar searches (which can be extended to any general computation) across a network of high performance clusters with minimal human intervention. This is publicly available on our github repository. This and all other software developed for the project is public and can be found here: https://github.com/erc-compact(si apre in una nuova finestra).
3. We are currently one of only 2 groups in the world that can perform targeted Keplerian searches for relativistic orbits. We also are the only group with data completely sensitive to ultra-fast rotating pulsars.
1. We have developed, for the first time, a robust way to inject low signal to noise fake pulsars into real astrophysical data. This will be extremely useful to understand the effectiveness of present software. Also, this can serve a quick way to fully test any changes to our search pipeline or any novel methodology that is introduced into our searches, without the fear of them not working as expected.
2. Our new methodology to search for relativistic binaries radically changes the way such searches are done. It is now evidently clear that at the very least, for at least some part of the parameter space, this will serve as a way to search 10x faster than existing search techniques. This will eventually be published as a general software that can be used by the wider scientific community.
2. Our new methodology to search for relativistic binaries radically changes the way such searches are done. It is now evidently clear that at the very least, for at least some part of the parameter space, this will serve as a way to search 10x faster than existing search techniques. This will eventually be published as a general software that can be used by the wider scientific community.