Periodic Reporting for period 2 - SUNDIAL (SUrvey Network for Deep Imaging Analysis and Learning)
Reporting period: 2019-04-01 to 2021-09-30
The objective of SUNDIAL is to train researchers to address the most prominent CI topics related to the analysis of Big Data and their application to galaxy evolution studies. These are:
(1) Automatic detection of faint low surface brightness features (dwarf galaxies, merger remnants, intracluster light) in deep astronomical surveys, and interpreting them astrophysically in terms of galaxy formation and evolution.
(2) Automated object recognition in Big Data sets: (a) the unsupervised identification of groups of objects with similar (clustering), properties and (b) the supervised assignment of objects into pre-defined target classes (classification). The addition of prior information from astrophysics will be crucial in both cases.
(3) Simulations of galaxy interaction, their characterisation and visualisation. The simulations serve to identify the critical characterisation, necessary to optimally identify how observations can be described. Such comparisons will lead to a better parametrisation and understanding of galaxy cluster evolution.
At the end of the network, we can see that our concept has worked. By putting together a team of astronomers and computer scientists, we have managed to create a set of tools that help to advance galaxy evolution science, published them in computer science journals, applied them to datasets in astronomy, and again published those results in astronomical journals. While being useful for our own science, the tools are only now being picked up by the community, implying that it is too early to see their full impact. The same holds for applications of these tools in other areas. Very important is that this collaboration has led to an efficient group of scientists, which is continuing to work together on making the tools better and applying them to more areas.
(1) Automatic detection of faint low surface brightness features (dwarf galaxies, merger remnants, intracluster light) in deep astronomical surveys, and interpreting them astrophysically in terms of galaxy formation and evolution.
(2) Automated object recognition in Big Data sets: (a) the unsupervised identification of groups of objects with similar (clustering), properties and (b) the supervised assignment of objects into pre-defined target classes (classification). The addition of prior information from astrophysics will be crucial in both cases.
(3) Simulations of galaxy interaction, their characterisation and visualisation. The simulations serve to identify the critical characterisation, necessary to optimally identify how observations can be described. Such comparisons will lead to a better parametrisation and understanding of galaxy cluster evolution.
At the end of the network, we can see that our concept has worked. By putting together a team of astronomers and computer scientists, we have managed to create a set of tools that help to advance galaxy evolution science, published them in computer science journals, applied them to datasets in astronomy, and again published those results in astronomical journals. While being useful for our own science, the tools are only now being picked up by the community, implying that it is too early to see their full impact. The same holds for applications of these tools in other areas. Very important is that this collaboration has led to an efficient group of scientists, which is continuing to work together on making the tools better and applying them to more areas.
In WP2 we have improved a detection algorithm previously developed by our group, called MTObjects. tool has been compared extensively with other faint object detection methods, and turns out to give a superior performance in most aspects. While the single-band MTObjects has been the most effective, we have also worked on a multi-band version, using graph-representation. This latter version Is very promising, although it is still very calculation-intensive, and therefore cannot handle large images at the moment.
Our comparison of various faint detection methods shows that MTObjects is the best method to use to detect faint galaxies in deep data. It is reliable, fast, and objective. A number of applications has been done with MTO.
In WP3 we have applied ML techniques to work on methods for the classification of galaxies. We are able to produce state-of-the-art results, but are limited by the human bias in morphological classification schemes. In the field of photometric redshifts, we used ML methods to prove that a large part of the erroneous prediction can be related to previously undetected errors in the spectroscopic information (usually assumed in the literature to be "error free"). As an application, we used FDS (the Fornax Deep Survey) data to perform multicomponent structural decompositions of 586 galaxies in the Fornax cluster We also have calculated “non-parametric” measures of morphology (concentration, asymmetry, smoothness, Gini, M20) in order to characterise galaxies systematically independent of the complexity of their structures, and studied the presence of a nucleus. A second application of classification uses ML tools to determine a new sample of UCD galaxies in the Fornax Cluster. Up to now such objects were only found in the very center of galaxy clusters.
In WP4 we have run simulations of dwarf galaxies falling into clusters, including gas, stars and dark matter. The simulations serve to identify the critical characterisation, necessary to optimally identify how observations can be described. Such comparisons will lead to a better parametrisation and understanding of galaxy cluster evolution. To describe these simulations, we have developed models which are able to automatically detect dense (possibly lower-dimensional) structures embedded in a substantial noisy background. As a testbed for comparison with real observations, we use the jellyfish galaxy NGC 1427A, a galaxy with large amounts of gas forming many stars at present. The models describing the simulations produce manifolds, that can be of different dimensionalities and the methodology does not assume their number is known in advance. The method gives us potentially unprecedented possibilities to quantitatively compare simulations with each other and with observations (with full flexibility in defining the observation space). More refined calibration of simulations enabled by this methodology can help us to better understand the physics of dwarf galaxies falling into clusters.
Our comparison of various faint detection methods shows that MTObjects is the best method to use to detect faint galaxies in deep data. It is reliable, fast, and objective. A number of applications has been done with MTO.
In WP3 we have applied ML techniques to work on methods for the classification of galaxies. We are able to produce state-of-the-art results, but are limited by the human bias in morphological classification schemes. In the field of photometric redshifts, we used ML methods to prove that a large part of the erroneous prediction can be related to previously undetected errors in the spectroscopic information (usually assumed in the literature to be "error free"). As an application, we used FDS (the Fornax Deep Survey) data to perform multicomponent structural decompositions of 586 galaxies in the Fornax cluster We also have calculated “non-parametric” measures of morphology (concentration, asymmetry, smoothness, Gini, M20) in order to characterise galaxies systematically independent of the complexity of their structures, and studied the presence of a nucleus. A second application of classification uses ML tools to determine a new sample of UCD galaxies in the Fornax Cluster. Up to now such objects were only found in the very center of galaxy clusters.
In WP4 we have run simulations of dwarf galaxies falling into clusters, including gas, stars and dark matter. The simulations serve to identify the critical characterisation, necessary to optimally identify how observations can be described. Such comparisons will lead to a better parametrisation and understanding of galaxy cluster evolution. To describe these simulations, we have developed models which are able to automatically detect dense (possibly lower-dimensional) structures embedded in a substantial noisy background. As a testbed for comparison with real observations, we use the jellyfish galaxy NGC 1427A, a galaxy with large amounts of gas forming many stars at present. The models describing the simulations produce manifolds, that can be of different dimensionalities and the methodology does not assume their number is known in advance. The method gives us potentially unprecedented possibilities to quantitatively compare simulations with each other and with observations (with full flexibility in defining the observation space). More refined calibration of simulations enabled by this methodology can help us to better understand the physics of dwarf galaxies falling into clusters.
SUNDIAL has produced several results beyond the state of the art. Here the most promising ones for the future are given:
- The development of the MTO source finder
- Application of ML techniques to several problems in galaxy classification
- Development of a filament finder applicable on a wide range of datasets.
The work of SUNDIAL has had considerable impact already, due to the publications that have appeared until now, the activities that were attended by people from outside the SUNDIAL collaboration, the BLOG on the website, the contacts with industry, and the interactions of the public through the zooniverse site. Most impact has come from our source finder algorithm MTObjects, which is very competitive for the upcoming missions EUCLID and LSST.
However, much more impact is expected in the future. This is, because the tools that we developed are not very well known yet, but are excellent, and can be applied for many problems in astrophysics. For example, MTObjects will soon have a multi-color option, optimized for surveys with three or more photometric passbands. Since many astronomical surveys have 3 or 4 bands, this can have a lot of use. Also, our manifold finding methods can find filamentary structures in a variety of datasets, among which the Cosmic Web, Gaia, etc.
Since these applications are not limited to astronomy only, we also expect applications from other fields, like, e.g. medical imaging. Although, up to now, we are not in the process of commercializing any software, the previously named algorithms are possible candidates that at some point could be used.
Most of the ESR’s have graduated or will graduate the next months. They all found new jobs at other institutions or companies where they will use the knowledge they gathered during the project period.
During the project period the ESR’s completed their research as mentioned above. Beside this very important task they also participated in a variety of dissemination activities for society.
- The development of the MTO source finder
- Application of ML techniques to several problems in galaxy classification
- Development of a filament finder applicable on a wide range of datasets.
The work of SUNDIAL has had considerable impact already, due to the publications that have appeared until now, the activities that were attended by people from outside the SUNDIAL collaboration, the BLOG on the website, the contacts with industry, and the interactions of the public through the zooniverse site. Most impact has come from our source finder algorithm MTObjects, which is very competitive for the upcoming missions EUCLID and LSST.
However, much more impact is expected in the future. This is, because the tools that we developed are not very well known yet, but are excellent, and can be applied for many problems in astrophysics. For example, MTObjects will soon have a multi-color option, optimized for surveys with three or more photometric passbands. Since many astronomical surveys have 3 or 4 bands, this can have a lot of use. Also, our manifold finding methods can find filamentary structures in a variety of datasets, among which the Cosmic Web, Gaia, etc.
Since these applications are not limited to astronomy only, we also expect applications from other fields, like, e.g. medical imaging. Although, up to now, we are not in the process of commercializing any software, the previously named algorithms are possible candidates that at some point could be used.
Most of the ESR’s have graduated or will graduate the next months. They all found new jobs at other institutions or companies where they will use the knowledge they gathered during the project period.
During the project period the ESR’s completed their research as mentioned above. Beside this very important task they also participated in a variety of dissemination activities for society.