A Master ATM European Validation Plan

Project Information

MAEVA

Grant agreement ID: 11397

Start date 5 March 2004

End date 30 June 2004

Funded under

• Programme for research technological development and demonstration on "Competitive and sustainable growth 1998-2002"

Total cost € 2 800 000,00

EU contribution € 2 800 000,00

Coordinated by

INGENIERÍA DE SISTEMAS PARA LA DEFENSA DE ESPANA
Spain

Deliverables

Validation Guidelines Handbook (VGH)

The Validation Guidelines Handbook provides guidelines for conducting validation exercises in order to create the desired uniform framework. The objective of the VGH is not to be prescriptive, but more to assist its users. Based on an understanding of the ATM problems to be solved, it is possible to define a set of validation exercises. The validation exercises should also be based on an understanding of the proposed ATM concept and take into account EUROCONTROL’s ATM 2000+ Strategy for developing the ATM system. Furthermore, the VGH was a living document, subject to changes on the basis of feedback obtained from the validation-related projects. In this regard, the VGH was updated within MAEVA to address and include best and the most widely used practices applied in the monitored validation exercises. In this handbook, MAEVA proposes a five-step process for conducting validation exercises: Step 1: Define validation aims, objectives and hypotheses. Step 2: Prepare the validation plan and exercise runs. Step 3: Execute the exercise runs and take measurements. Step 4: Analyse results. Step 5: Develop and disseminate conclusions. Each of these steps is described in detail in the handbook. The purpose, description, process diagram, inputs, outputs and responsibilities are specified for each step (Part I of the VGH), and for each activity and sub-activity within each step (Part II of the VGH). The scope of the validation exercise framework, and therefore of the VGH, is primarily focused on the ATM validation exercises that will be conducted within the 5th FP, mainly fast-time and real-time simulations and shadow mode and operational trials. Notwithstanding this, the validation exercise framework has been kept as general as possible, to enable it to be applied to validation exercises conducted outside conduct the 5th FP.

Validation Master Plan (VMP)

This document is one of the main products of the first phase (Definition phase) of MAEVA. The document establishes a validation framework for analysing projects involving validation of the air traffic management (ATM) system monitored. This framework includes a reference model, a validation methodology, the operational concept developed by the TORCH project and the operational improvements established by Eurocontrol for the European ATM Programme (EATMP). The TORCH Operational Concept and the Operational Improvements are presented briefly. The MAEVA vision of the overall validation process is presented. This vision is based on the mapping of validation programmes and exercises over time, and then establishing relationships with the TORCH Operational Concept. The overall MAEVA overview of the validation process is presented in a series of tables that show the status of each of the monitored validation projects with regard to series of validation-related aspects. These aspects include the TORCH elements addressed, interactions with other projects, the ATM and validation scope and the project results. On the basis of all the information, this document identifies potential gaps and potential overlaps in addressing the elements of the TORCH Operational Concept, as broken down into operational improvements. Finally, the document presents conclusions reached from the analysis conducted of the individual projects.

Exploitable results

Master plan for air traffic control

The validation of air traffic management (ATM) systems is an on-going process which is carried out at the same time as the system is being developed. The detailed definition of validation programmes, methodologies, tests and trials for these systems is a quite complex task. The major objective of the Master ATM European validation plan (MAEVA) project was to provide the European Commission (EC) with an overall validation strategy for the future pre- European ATM Programme (EATMP) for its Fifth Framework Programme validation projects. MAEVA, which was sponsored by the Directorate General for Transport and Energy, was also expected to provide decision-making support in managing all Fifth Framework Programme validation related projects and coordinating with Eurocontrol and Member States within an overall European validation initiative. Results and conclusions are part of the MAEVA outcome which comprises: - The documentation (deliverables) produced summarising the monitoring, assessment and results obtained during the MAEVA lifetime. The most important deliverables were the validation guidelines handbook (VGH) and the validation master plan (VMP). - The MAEVA final report is also a major result of MAEVA. These deliverables are public and will be the main reference for the users of MAEVA: a. The VGH provides guidelines for conducting validation exercises based on an uniform validation framework. The objective of the VGH is to assist the users to perform their validation activities. This document was updated within MAEVA to address and include best and most widely used practices applied in the validation exercises being monitored. b. The VMP establishes a validation framework for analysing projects involving validation of the ATM system. The overall MAEVA overview of the validation processes is presented in a series of tables that show the status of each of the monitored validation projects with regard to series of validation-related aspects. On the basis of all the information, this document identifies potential gaps and potential overlaps and conclusions in addressing the elements of the TORCH operational concept. c. The MAEVA final report summarises the findings, recommendations and conclusions obtained after performing the assessment of the validation processes conducted by the monitored projects. Broad dissemination of the MAEVA results through the ATM community: three dissemination forums were organised during the project. These forums were held in Barcelona and in Palma de Mallorca (Spain). Findings on safety The main safety methodology used by the projects working in the four focus themes was ED-78A. Despite this there is limited expertise in this methodology in the ATM community and no single project has followed the entire process to its conclusion. In the four focus areas, safety was addressed to different degrees, ranging from a high level of detail in ASAS and A-SMGCS to more limited coverage for 4D trajectory management and CDM. Some projects performed extensive work on safety, which should to be beneficial to other projects and the overall safety process. The feedback or exposure of this work was quite limited however, and there is difficulty in accessing these results. Some of the safety results presented by NUP 2 on ASAS spacing applications imply significant certification issues for the airborne side that could make installation of the system uneconomic. Security was not examined by any of the projects monitored. This can be considered a safety issue. Findings on cost benefit analysis Limited work on cost benefit analysis was performed in the four focus areas and it was difficult for MAEVA to obtain documentation on the work that was completed. Earlier projects have not moved to operations owing to the unwillingness of stakeholders to invest in new technologies and systems. Cost benefit analysis was identified as an important factor in providing stakeholders with the information that they need in order to support the preparation of business cases for these investments. The metrics most often used in the CBAs that were performed are delay, predictability, flexibility, efficiency, access, capacity and cost of service. Capacity has historically been defined as throughput in TMA and en-route, but airport throughput is becoming increasingly important. The most significant problem in investing in ATM systems is the requirement for a number of stakeholders to invest in the technology. Consequently, coordination is required between airlines, ANSPs, airports (ideally all) and potentially other parties in order for the full benefits of these systems to accrue. An important difficulty in performing the CBAs was the unwillingness of stakeholders, in particular airlines, to provide data to support the analysis owing to commercial sensitivity. The work on CBA for ASAS was relatively detailed, but still not complete. Similarly, 4D trajectory management was addressed but there remain areas in which further work is needed. Very little CBA work has been performed for A-SMGCS and CDM, possibly due to the difficulty in quantifying the benefits of safety and data sharing in financial terms. Looking at the problem of CBA from the highest level it can be seen that there is significant interaction, interdependence and commonality between the various systems that are being considered. The implication of this is that an investment that may not make commercial sense viewed separately may in fact be justified when the system-wide costs and benefits are considered. The change in the economic and business environment over the past years has been and will be significant. These changes include the launch of the low-cost airlines in the 1980's, the impact of 11th September 2001 and libely structural changes in the ANSPs with the Single Sky legislation. All of these can be expected to have an impact on the requirements for business cases and consequently on the information required from CBAs. Findings on transition issues The taxonomy for analysing transition issues developed by MAEVA appears to be a good starting position for addressing this important area of R&D for ATM systems. Very little work was done directly by the projects on transition issues. This should not be taken as a criticism of projects as this area was not within the technical scope of the projects. It was possible to find some information on transition issues through more detailed analysis of the project deliverables. The status of transition issues can be summarised as: - The work performed on institutional issues focused on the certification of aircraft systems, primarily in A-SMGCS, ASAS and CDM. Some work within the projects considered the institutional aspects of the Single Sky. This work is not complete. - Technological issues were addressed primarily through the production of prototypes. - Human resources issues addressed by the monitored projects were training issues and user acceptability. - Section 8.2 presents the work that considered the CBA aspects of financial issues. - None of the monitored projects studied environmental issues. - Interoperability was not addressed by the projects.