Periodic Reporting for period 1 - EXPLORE (Innovative Scientific Data Exploration and Exploitation Applications for Space Sciences)
Période du rapport: 2020-11-01 au 2022-04-30
The EXPLORE project aims to facilitate efficient and user-friendly exploitation of data from astrophysics and planetary space missions and supporting ground-based surveys.
EXPLORE is developing six novel Scientific Data Applications (SDAs) that relate to three areas of space science chosen for their timeliness, complementary data structures, and appeal to a broad scientific audience. Two of EXPLORE’s topics relate to galactic science (G-Tomo and G-Arch), two to stellar characterisation (S-Phot and S-Disco) and two to lunar observation (L-Explo and L-Hex).
The SDAs draw primarily on datasets from recent lunar missions and ESA’s cornerstone mission, Gaia. The tools deploy Machine Learning and advanced visual analytics to maximise scientific return-on-investment and to support scientific discovery by revealing previously unexploited aspects of the data.
G-Arch enables users to characterise large samples of stellar spectra from Gaia, to excavate the history of our galaxy through its chemical evolution. G-Tomo allows users to extract information on the absorption of light by interstellar dust grains and visualise the 3D distribution of dust in the Milky Way, supporting investigations of the relationship between interstellar clouds and populations of stars, and the effects of dust on observations of astrophysical objects.
S-Phot enables users to combine photometric and parallax data from Gaia and other relevant databases to reveal properties of stars in the Milky Way, such as their temperature, luminosity or evidence of circumstellar dust. S-Disco applies Machine Learning and Deep Learning to discover anomalies within large stellar datasets (e.g. from Gaia), such as unusual stars or unexpected populations or patterns.
L-Explo identifies subtle differences and anomalies within lunar orbital data to enhance geological mapping of the Moon. L-Hex supports human and robotic exploration of the Moon by providing access to historic and recent lunar data.
Each SDA uses the same software framework to ease development, integration and deployment on science platforms. The SDAs are tested and demonstrated on the dedicated, cloud-based analysis and exploitation platform (https://explore-platform.eu) and will be integrated and deployed on science platforms e.g. ESA Datalabs.
The SDAs will be released as open-source software, in accordance with Findable, Accessible, Interoperable and Reusable (FAIR) principles. Communication, dissemination and exploitation activities aim not only to develop a wide user base for the SDAs, but to stimulate space science communities to engage in further development and improvement of the tools. Ultimately, EXPLORE’s goal is to provide a flexible infrastructure for researchers to create and offer their own services on science platforms, with tools that are close to the input data and open to the community for direct, on-demand exploitation.
EXPLORE is developing six novel Scientific Data Applications (SDAs) that relate to three areas of space science chosen for their timeliness, complementary data structures, and appeal to a broad scientific audience. Two of EXPLORE’s topics relate to galactic science (G-Tomo and G-Arch), two to stellar characterisation (S-Phot and S-Disco) and two to lunar observation (L-Explo and L-Hex).
The SDAs draw primarily on datasets from recent lunar missions and ESA’s cornerstone mission, Gaia. The tools deploy Machine Learning and advanced visual analytics to maximise scientific return-on-investment and to support scientific discovery by revealing previously unexploited aspects of the data.
G-Arch enables users to characterise large samples of stellar spectra from Gaia, to excavate the history of our galaxy through its chemical evolution. G-Tomo allows users to extract information on the absorption of light by interstellar dust grains and visualise the 3D distribution of dust in the Milky Way, supporting investigations of the relationship between interstellar clouds and populations of stars, and the effects of dust on observations of astrophysical objects.
S-Phot enables users to combine photometric and parallax data from Gaia and other relevant databases to reveal properties of stars in the Milky Way, such as their temperature, luminosity or evidence of circumstellar dust. S-Disco applies Machine Learning and Deep Learning to discover anomalies within large stellar datasets (e.g. from Gaia), such as unusual stars or unexpected populations or patterns.
L-Explo identifies subtle differences and anomalies within lunar orbital data to enhance geological mapping of the Moon. L-Hex supports human and robotic exploration of the Moon by providing access to historic and recent lunar data.
Each SDA uses the same software framework to ease development, integration and deployment on science platforms. The SDAs are tested and demonstrated on the dedicated, cloud-based analysis and exploitation platform (https://explore-platform.eu) and will be integrated and deployed on science platforms e.g. ESA Datalabs.
The SDAs will be released as open-source software, in accordance with Findable, Accessible, Interoperable and Reusable (FAIR) principles. Communication, dissemination and exploitation activities aim not only to develop a wide user base for the SDAs, but to stimulate space science communities to engage in further development and improvement of the tools. Ultimately, EXPLORE’s goal is to provide a flexible infrastructure for researchers to create and offer their own services on science platforms, with tools that are close to the input data and open to the community for direct, on-demand exploitation.
At the midpoint of the project, prototypes of all six SDAs have been developed. Beta-testing and access to selected early adopters is supported through the EXPLORE platform.
Upcoming user-workshops, held online and at related conferences, will gather feedback to guide development of the final project outputs. The SDAs are being presented at major international conferences, including the European Lunar Symposium, European Astronomical Society Annual Meeting and Artificial Intelligence for Science and Operations in Astronomy (SciOps) Meeting.
G-Tomo and G-Arch prototypes have been tested on synthetic Gaia data, in preparation for the Gaia third data release (DR3) on 13 June 2022. The G-Tomo prototype has provided access to two 3D maps of interstellar dust in the Milky Way based on data from the early DR3 (eDR3) and 2MASS (Lallement et al 2022), and a follow-up work (Vergely et al 2022) intercalibrates data from complementary dust catalogues to improve 3D reconstruction and offer new maps at different spatial scales.
Three papers relating to G-Arch have been prepared for publication to coincide with Gaia DR3. These include an analysis of stellar atmospheric parameters and chemical abundances, combined with precise information on position and radial movement, for about 5.5 million stars (Recio-Blanco et al. 2022). The methodology for processing the Gaia Radial Velocity Spectrometer (RVS) data in DR3, described in a second paper (Recio-Blanco et al 2022), is shared by the EXPLORE G-Arch SDA. G-Arch has also been used to support analysis of spatial and kinematical properties of absorption features known as diffuse interstellar bands present in Gaia RVS spectra (Schultheis et al 2022). Further publications relate to the development of a galactic chemical evolution model.
Two scientific projects with the stellar SDAs are in progress. The S-Phot prototype facilitates the location and retrieval of measurements of the brightness of stars from archives obtained from different telescopes and instruments, enabling accurate characterisation of stars and stellar groups in the Milky Way. The use of the S-Phot prototype to obtain stellar luminosities and temperatures for nearby mass-losing stars is described in a paper in preparation.
S-Disco enables researchers to ask creative questions and to facilitate discovery by identifying unusual stars (outliers) in a specific region of the galaxy, searching for locations in the galaxy that have a particular concentration or lack of outliers, or investigating properties of stars e.g. luminosity, colour or metallicity. The S-Disco prototype enables the exploration of spectral similarity and positions of about 75,000 stars, using high quality data derived from the APOGEE survey and Gaia mission.
The Space Browser on the EXPLORE platform allows registered users to query planetary data collections through an interactive map browser. Users can constrain the number of results, view data products, browse images, add data as layers to the planetary surface basemap, and store products in the workspace for use with SDAs, such as L-Explo and L-Hex. The lunar SDAs are being tested by planetary mapping community groups, and the first paper is under review.
Upcoming user-workshops, held online and at related conferences, will gather feedback to guide development of the final project outputs. The SDAs are being presented at major international conferences, including the European Lunar Symposium, European Astronomical Society Annual Meeting and Artificial Intelligence for Science and Operations in Astronomy (SciOps) Meeting.
G-Tomo and G-Arch prototypes have been tested on synthetic Gaia data, in preparation for the Gaia third data release (DR3) on 13 June 2022. The G-Tomo prototype has provided access to two 3D maps of interstellar dust in the Milky Way based on data from the early DR3 (eDR3) and 2MASS (Lallement et al 2022), and a follow-up work (Vergely et al 2022) intercalibrates data from complementary dust catalogues to improve 3D reconstruction and offer new maps at different spatial scales.
Three papers relating to G-Arch have been prepared for publication to coincide with Gaia DR3. These include an analysis of stellar atmospheric parameters and chemical abundances, combined with precise information on position and radial movement, for about 5.5 million stars (Recio-Blanco et al. 2022). The methodology for processing the Gaia Radial Velocity Spectrometer (RVS) data in DR3, described in a second paper (Recio-Blanco et al 2022), is shared by the EXPLORE G-Arch SDA. G-Arch has also been used to support analysis of spatial and kinematical properties of absorption features known as diffuse interstellar bands present in Gaia RVS spectra (Schultheis et al 2022). Further publications relate to the development of a galactic chemical evolution model.
Two scientific projects with the stellar SDAs are in progress. The S-Phot prototype facilitates the location and retrieval of measurements of the brightness of stars from archives obtained from different telescopes and instruments, enabling accurate characterisation of stars and stellar groups in the Milky Way. The use of the S-Phot prototype to obtain stellar luminosities and temperatures for nearby mass-losing stars is described in a paper in preparation.
S-Disco enables researchers to ask creative questions and to facilitate discovery by identifying unusual stars (outliers) in a specific region of the galaxy, searching for locations in the galaxy that have a particular concentration or lack of outliers, or investigating properties of stars e.g. luminosity, colour or metallicity. The S-Disco prototype enables the exploration of spectral similarity and positions of about 75,000 stars, using high quality data derived from the APOGEE survey and Gaia mission.
The Space Browser on the EXPLORE platform allows registered users to query planetary data collections through an interactive map browser. Users can constrain the number of results, view data products, browse images, add data as layers to the planetary surface basemap, and store products in the workspace for use with SDAs, such as L-Explo and L-Hex. The lunar SDAs are being tested by planetary mapping community groups, and the first paper is under review.
The scientific impact of the EXPLORE SDAs is already starting to emerge, e.g. scientific analysis implemented via G-Arch shows that the relationships between the chemistry and structure of the Milky Way reveal a complex galaxy with an eventful evolution. Upcoming publications and conference presentations relating to the SDAs are expected to result in media coverage.
The EXPLORE platform’s Space Browser, SDA dashboard and administrator dashboard have each constituted major software developments. Work on EXPLORE’s SDAs has resulted in several new features and extensions to Visualizer, an open-source research framework that enables users to visualise, analyse and interact with data in multiple ways through their web-browser. Users can now collaboratively design a dashboard to explore data together in real-time, e.g. by mutually selecting interesting data fields, creating visualisations and annotating selected areas of interest.
EXPLORE has been presented as an exemplar of academic-industrial collaboration to policy makers and the academic community. Through the EXPLORE Data Challenges, the project is reaching out to other fields of research, including AI, data analytics and geological mapping. A Junior Challenge will engage educational partners serving communities in deprived areas.
The EXPLORE platform’s Space Browser, SDA dashboard and administrator dashboard have each constituted major software developments. Work on EXPLORE’s SDAs has resulted in several new features and extensions to Visualizer, an open-source research framework that enables users to visualise, analyse and interact with data in multiple ways through their web-browser. Users can now collaboratively design a dashboard to explore data together in real-time, e.g. by mutually selecting interesting data fields, creating visualisations and annotating selected areas of interest.
EXPLORE has been presented as an exemplar of academic-industrial collaboration to policy makers and the academic community. Through the EXPLORE Data Challenges, the project is reaching out to other fields of research, including AI, data analytics and geological mapping. A Junior Challenge will engage educational partners serving communities in deprived areas.