Periodic Reporting for period 1 - CCVS (Copernicus CAL/VAL Solution)
Période du rapport: 2020-12-01 au 2022-11-30
The objective of the Copernicus Cal/Val Solution (CCVS) is to define a holistic solution for all Copernicus Sentinel missions (either operational or planned) to overcome current limitations of Calibration and Validation (Cal/Val) activities. Operational Cal/Val is required to ensure the quality of and build confidence in Copernicus data. However, these activities are currently limited by the following considerations:
• The requirements and objectives need to be revisited to consider new usage of Copernicus products, inter-operability requirements, and to anticipate the needs of future Copernicus missions
• Current Cal/Val activities are constrained by programmatic and budgetary requirements and do not necessarily follow scientific priorities
• Cal/Val activities depend on the operational availability of high-quality Fiducial Reference Measurements (FRM) which are today mostly provided by external entities without strong commitment to the Copernicus program
• Synergies within Copernicus and with other national and international programs are not systematically explored.
The impact of these limitations are threefold:
* The validation of Copernicus data products is not fully operational. Some variables are validated, partially, intermittently, or not all. As a result, some data products cannot be used in full confidence for issues relevant to science and society.
* The allocation of resources can be suboptimal because of a lack of coordination and prioritization.
To address these limitations CCVS proposes:
• An updated specification of Cal/Val requirements for the Sentinel missions, taking into account inter-operability needs
• An overview of existing Calibration and Validation sources and means
• A gap analysis identifying missing elements and required developments in terms of technologies and instrumentation, Cal/Val methods, instrumented sites and dissemination service.
• A comprehensive Copernicus Cal/Val Solution to organize the long-term provision of FRM for Sentinel missions
• A roadmap documenting how the Cal/Val Solution can be implemented, highlighting responsibility, cost and schedule aspects. This plan is elaborated in concertation with all stakeholders through four Working Groups gathering European Space Agencies, Copernicus Services, measurement networks and International partners.
• The requirements and objectives need to be revisited to consider new usage of Copernicus products, inter-operability requirements, and to anticipate the needs of future Copernicus missions
• Current Cal/Val activities are constrained by programmatic and budgetary requirements and do not necessarily follow scientific priorities
• Cal/Val activities depend on the operational availability of high-quality Fiducial Reference Measurements (FRM) which are today mostly provided by external entities without strong commitment to the Copernicus program
• Synergies within Copernicus and with other national and international programs are not systematically explored.
The impact of these limitations are threefold:
* The validation of Copernicus data products is not fully operational. Some variables are validated, partially, intermittently, or not all. As a result, some data products cannot be used in full confidence for issues relevant to science and society.
* The allocation of resources can be suboptimal because of a lack of coordination and prioritization.
To address these limitations CCVS proposes:
• An updated specification of Cal/Val requirements for the Sentinel missions, taking into account inter-operability needs
• An overview of existing Calibration and Validation sources and means
• A gap analysis identifying missing elements and required developments in terms of technologies and instrumentation, Cal/Val methods, instrumented sites and dissemination service.
• A comprehensive Copernicus Cal/Val Solution to organize the long-term provision of FRM for Sentinel missions
• A roadmap documenting how the Cal/Val Solution can be implemented, highlighting responsibility, cost and schedule aspects. This plan is elaborated in concertation with all stakeholders through four Working Groups gathering European Space Agencies, Copernicus Services, measurement networks and International partners.
A first phase of the project was devoted to the analysis of the situation. It ended as planned at T0+6 and led to the first project public workshop which presented the analysis and collected feedback from on the community and stakeholders on this analysis.
A first outcome of this phase is the publication of an updated list of cal/val requirements for all Sentinel missions (optical, altimetry, radar and atmospheric composition missions) presented in a unified way. A second set of documents presents a survey of existing cal/val data sources, including sensor pre-flight calibration, on-board calibration devices, vicarious methods on natural scenes, comparisons with other sensors and models, and finally with in-situ measurements from networks and field campaigns. The survey also addressed data distribution aspects. The document include an extensive survey of in-situ measurement networks relevant for all variables measured by the Sentinel missions, with an assessment of their maturity level and operational readiness. The assessment relied on published information as well as interactions with many experts (through questionnaires and interviews).
Morevover, a three-day virtual project workshop was organized and gathered more than 140 participants. The workshop allowed interactions between scientists and engineers from different domains to present their assessment of the status of the calibration and validation of the Copernicus program. Important topics to be addressed by the CCVS project were discussed: which measurements can be labelled "Fiducial Reference Measurements" ? How to ensure a sustainable funding for measurement networks ? Is there a need for a better coordination between measurement networks and cal/val experts ? What effort should be put on the maintenance of reference data archives for validation ? A document summarizing the main outcomes of the workshop was produced.
The second phase of the project was dedicated to the elaboration of the solution itself. This solution was elaborated thanks to two complementary approaches:
* in a "bottom-up" approach, the different avenues to improve existing Cal/Val methods have been identified
* a "top-down" approach was derived from a maturity assessment of the Cal/Val status for each Sentinel core product, and identify where priority actions should be applied.
The activity was also supported by several working groups. Dedicated working group meetings were held with the Copernicus Services and the Space Agencies.
A second workshop was organized to interact with in-situ measurement laboratories and networks. Discussions were held with international partners through CEOS WGCV meetings.
Four different types of recommendations were issued:
* R&D activities were proposed on Cal/Val method, ground-based instrumentation an on-board calibration technologies,
* Recommendations for operational activities to be implemented or supported,
* Organization and coordination activities involving several stakeholders.
The list of recommendations was then structured into an actionable programme with an assessment of the criticality and required effort.
Potential pilots and funding sources for each activity were identified, and a tentative implementation timeframe was provided.
In addition, a quantitative assessment of the propagation of uncertainties to Level 3 was performed for some products.
A first outcome of this phase is the publication of an updated list of cal/val requirements for all Sentinel missions (optical, altimetry, radar and atmospheric composition missions) presented in a unified way. A second set of documents presents a survey of existing cal/val data sources, including sensor pre-flight calibration, on-board calibration devices, vicarious methods on natural scenes, comparisons with other sensors and models, and finally with in-situ measurements from networks and field campaigns. The survey also addressed data distribution aspects. The document include an extensive survey of in-situ measurement networks relevant for all variables measured by the Sentinel missions, with an assessment of their maturity level and operational readiness. The assessment relied on published information as well as interactions with many experts (through questionnaires and interviews).
Morevover, a three-day virtual project workshop was organized and gathered more than 140 participants. The workshop allowed interactions between scientists and engineers from different domains to present their assessment of the status of the calibration and validation of the Copernicus program. Important topics to be addressed by the CCVS project were discussed: which measurements can be labelled "Fiducial Reference Measurements" ? How to ensure a sustainable funding for measurement networks ? Is there a need for a better coordination between measurement networks and cal/val experts ? What effort should be put on the maintenance of reference data archives for validation ? A document summarizing the main outcomes of the workshop was produced.
The second phase of the project was dedicated to the elaboration of the solution itself. This solution was elaborated thanks to two complementary approaches:
* in a "bottom-up" approach, the different avenues to improve existing Cal/Val methods have been identified
* a "top-down" approach was derived from a maturity assessment of the Cal/Val status for each Sentinel core product, and identify where priority actions should be applied.
The activity was also supported by several working groups. Dedicated working group meetings were held with the Copernicus Services and the Space Agencies.
A second workshop was organized to interact with in-situ measurement laboratories and networks. Discussions were held with international partners through CEOS WGCV meetings.
Four different types of recommendations were issued:
* R&D activities were proposed on Cal/Val method, ground-based instrumentation an on-board calibration technologies,
* Recommendations for operational activities to be implemented or supported,
* Organization and coordination activities involving several stakeholders.
The list of recommendations was then structured into an actionable programme with an assessment of the criticality and required effort.
Potential pilots and funding sources for each activity were identified, and a tentative implementation timeframe was provided.
In addition, a quantitative assessment of the propagation of uncertainties to Level 3 was performed for some products.
In the frame of task 1, a set of Cal/Val requirements for each domain has been established, as well as a survey of existing Cal/Val reference sources.
Although these requirements were mostly available from other sources, their collection into an harmonized set of documents is expected to be a valuable reference for Cal/Val experts and data users.
The Cal/Val maturity assessment performed in the frame of Task 3 is the first such attempt covering all Sentinel data products.
The proposed Copernicus Cal/Val Solution is the main output of the project. This solution is expected to improve the overal quality of the data provided by the Copernicus programme, but also to improve the overall efficiency and leverage synergies with other programmes.
Many of the recommendations are completely original and were elaborated in the frame of the project.
This is the case for the proposition to implement a formal FRM labelling process, or to investigate metrology aspects of the validation of discrete (categorial) variables.
The study on Level 3 products highlighted the need for specific validation datasets with comparable spatial and temporal coverage, which opens new research perspectives.
Finally, the project has had a positive impact on improving collaboration between different communities, in particular between in-situ measurement networks and Cal/Val experts.
A case in point concerns the perspective of using ICOS vegetation measurements for the validation of Copernicus data products.
Although these requirements were mostly available from other sources, their collection into an harmonized set of documents is expected to be a valuable reference for Cal/Val experts and data users.
The Cal/Val maturity assessment performed in the frame of Task 3 is the first such attempt covering all Sentinel data products.
The proposed Copernicus Cal/Val Solution is the main output of the project. This solution is expected to improve the overal quality of the data provided by the Copernicus programme, but also to improve the overall efficiency and leverage synergies with other programmes.
Many of the recommendations are completely original and were elaborated in the frame of the project.
This is the case for the proposition to implement a formal FRM labelling process, or to investigate metrology aspects of the validation of discrete (categorial) variables.
The study on Level 3 products highlighted the need for specific validation datasets with comparable spatial and temporal coverage, which opens new research perspectives.
Finally, the project has had a positive impact on improving collaboration between different communities, in particular between in-situ measurement networks and Cal/Val experts.
A case in point concerns the perspective of using ICOS vegetation measurements for the validation of Copernicus data products.