Periodic Reporting for period 1 - NOSTROMO (Next-Generation Open-Source Tools for ATM Performance Modelling and Optimisation)
Reporting period: 2020-06-01 to 2021-05-31
The development of methodologies to evaluate the impact of new ATM concepts and technologies on high-level, system wide Key Performance Indicators (KPIs) has been a long-time objective of the ATM research community. Low-level validation activities based on fast-time simulation, human-in-the-loop (HITL) simulation, shadow-mode trials and live trials provide accurate estimates of the performance of a certain solution in a given operational environment; however, implementing such validation approaches for different combinations of solutions at a network-wide scale is infeasible, or at least prohibitive in terms of both cost and time. It is therefore necessary to resort to performance models that consolidate the results of low-level validation experiments conducted for different solutions at a local level and estimate the integrated impact of such solutions at network level.
In this context, the goal of NOSTROMO is to develop, demonstrate and evaluate an innovative modelling approach for the rigorous and comprehensive assessment of the performance impact of future ATM concepts and solutions at ECAC network level. This approach will bring together the ability of bottom-up microscopic models to capture emergent behaviour and interdependencies between different solutions with the level of tractability and interpretability required to effectively support decision-making. The specific objectives of the project are the following:
1. Develop a methodology for the construction of ATM performance metamodels that approximate the behaviour of computationally expensive simulation models so as to allow a systematic and efficient exploration of the model input-output space and a robust handling of the uncertainty associated with the model predictions, by exploiting recent advances in the field of active learning.
2. Implement and validate the proposed metamodeling methodology by developing metamodels of two state-of-the-art microsimulation tools (namely Mercury and FLITAN) able to reproduce ATM performance at ECAC level.
3. Develop a set of visualisation and visual analytics tools that facilitate the analysis, interpretation and communication of the results of the new performance metamodels.
4. Demonstrate and evaluate the maturity of the NOSTROMO approach and the capabilities of the newly developed toolset through a set of case studies addressing the performance assessment of SESAR Solutions at ECAC level. The case studies shall cover a variety of ATM phases, solutions and KPAs/KPIs sufficiently heterogeneous to allow a comprehensive benchmarking against the performance modelling methodologies currently in use, with the aim to analyse the added value and the limitations of the NOSTROMO approach and evaluate the appropriateness of its transition to SESAR IR and its potential to improve of the European Operational Concept Validation Methodology (E-OCVM).
NOSTROMO project will be developed in an incremental approach towards the objectives, by evaluating and refining the proposed methodology in an iterative manner in the light of the results obtained in its specific applications.
In particular, the following aspects have been addressed:
• An initial baseline of the NOSTROMO methodology has been defined that will be updated along project lifecycle.
• Considering the objective of this iteration, a full implementation of metamodels has not been performed yet, but the initial requirements for the metamodels development and the preliminary architecture including integration aspects have been defined.
• Data repository has been also deployed.
• The high-level requirements of the visualisation tools have been identified and some preliminary mock-ups have been developed.
• No performance analysis questions have been answered during this first step and only ad hoc experiments have been performed (test cases).
• A preliminary selection of case studies expected to be carried out in the following iterations has been performed.
As result of this first iteration, it can be stated that the methodology proposed by NOSTROMO to build metamodels is technically feasible although some improvements should be considering in the following iterations.
To achieve these goals, two additional iterations are planned:
• The second iteration will be the first attempt to implement and evaluate the case studies identified in the previous phase to demonstrate the applicability of the proposed methodology to assess the impact of isolated operational concepts on performance considering not only its technical feasibility but also operational aspects. At the beginning of this iteration, the metamodeling methodology will be refined according to the results of the “case zero”. During the second iteration, a dashboard equipped with a set of interactive visualisation tools will be developed. The dashboard will allow the user to analyse the outputs of the metamodels and explore trade-offs between KPAs/KPIs, with the ultimate purpose of supporting different types of decision-making process related to performance management.
• The third final iteration will be focused on the refinement of methodology and associated architecture. An improvement of the selected case studies and the final update of the visualisation dashboard will be also performed taking into account the results from the second iteration. In particular, the case studies will be updated to address not only the isolated operational concepts but also considering the interrelations between different solutions. At the end of this iteration, guidelines, and recommendations on the key aspects for future development and integration of the NOSTROMO methods and tools into the E-OCVM framework and SESAR Industrial Research Programme will be provided.