Final Report Summary - I-VISION (Immersive Semantics-based Virtual Environments for the Design and Validation of Human-centred Aircraft Cockpits)
Executive Summary:
The aerospace industry aims at reducing product development times and costs but faces a major obstacle; the need to build several physical prototypes for verifying various factors during design. Human factors considerations in the design process of aeronautic products play a crucial role for the reliability and resilience of the systems involved, from an operational and error tolerant point of view. For system design purposes there is significant utility in applying human task and cognitive workload analysis, however in existing systems the analysis and the analysed artefacts are decoupled and implemented as separate entities. Such separation leads to high manual effort for integration, while missing chances for automation and thus cost-reduction. Semantic representation of scene content and application entities is necessary for several kinds of intelligent virtual engineering tasks. Existing systems fail to implement a closed loop between semantics and 3D geometries and generally suffer from scalability and real-time performance issues. i-VISION progressed the current status of cognitive-human analysis of operations in aircraft cockpits using VR technologies, by advancing the methodologies with requirements from modern operating conditions. The i-VISION project’s outcome was designed and implemented as a platform with three distinct and complementary scientific and technological objectives, briefly described as follows:
Human-Cockpit Operations Analysis Module: This module is a web based application that accommodates advanced human factor methods (Hierarchical Task Analysis, Error Analysis, Time Analysis, Self-Assessment) for analysing the human procedures and tasks during various phases and operating conditions in a virtual reality (VR) based aircraft cockpit.
Semantic Virtual Cockpit Module: Semantic technologies are used in the i-VISION project so as to enable engineers and human factor experts to assess a virtual aircraft cockpit in a timely and cost-effective way. This module connects the Virtual Cockpit Design Environment Module and the Human-Cockpit Operations Analysis Module by integrating data and observing workflows, and allows for complex querying.
Virtual cockpit design environments: The Virtual Cockpit Design Environment serves as a reusable and low-cost simulation test bed for experimenting with various configurations allowing the human–centred assessment of future cockpit architectures. It enables the user to participate in complex simulations, tracks and records his actions which in turn act as a basis for human factors analysis.
Project Context and Objectives:
The i-VISION tool makes use of human factors background knowledge, semantic and immersive virtual reality technologies in order to support the human factors design and validation activities in aircraft cockpits, during the early phases of the product life-cycle.
The overall work that covered a 36 months period of industrial driven requirements, research, development and realization of final demonstrators, involved three stages towards the achievement of the project’s objectives. In the first phase, all the necessary input, mainly from industrial partners, were collected for developing the requirements in terms of technology and application. These requirements were analysed in order to develop the specifications of the technological modules and every component of the tool was designed. Two use cases were defined in an early stage so as to enhance the applicability of the developments in real life scenarios and validate the tool in the final stages of i-VISION.
During the second phase, implementation of the individual i-VISION components was performed, based on the detailed design that proceeded, during phase one. Second phase is the phase where the main project development took place, such Human-Cockpit Operations Analysis Module, Semantic Virtual Cockpit Module, Virtual Cockpit Design Environment and the semantic interpreter (SISSI). Finally the integration of the modules in a single tool was performed, acting as the industrial demonstrator on which the validation activities eventually took place.
In the third phase the industrial partners along with the support of research and application partners detailed the pilot cases and scenarios, previously defined, for validating the i-VISION prototypes. The validation of the tool took place where each requirement was checked and documented in an individual table. In parallel to the previous work, dissemination and exploitation activities were performed together with project management activities.
The project’s achievements were realized through the following six work packages:
Work Package 1: Requirements Analysis and Specifications
Work package 1 had as main objective the specifications’ definition for the i-VISION developments to follow in work packages 2 and 3. Industrial partners provided requirements based on their needs for a usable integrated tool that constituted the base for the specifications definition taking into account the academic and technological partners’ experience and knowledge in the applicable methods for structured human factors assessment methods, semantics and virtual reality technologies. The specifications were structure and validated throughout the four work tasks that comprise this Work Package. Realistic scenarios and use-cases were created to underpin the development of robust requirements and ensure a mutual understanding of the tools to be developed and integrated in the following work packages. Finally, a major objective of this work package was to define formats and interfaces that would allow a seamless transition of methodical toolsets into the virtual reality platform.
Work Package 2: Human-Cockpit Operations Analysis Module
The objective of this work package was the development of the Human-Cockpit Operations Analysis Module (HCOAM). The work was broken down into three tasks, starting with the design of the software module as a web application, paying attention to creating a user-friendly, familiar interface, with a feel as close to the conventional methods. The next step was the development of an ontology for the human tasks and procedure model and finally the development of the software prototype to be used together with the semantic module and the integrated virtual environment.
Work Package 3: Semantic Virtual Cockpit Module
The objective of this work package was the development of the Semantic Virtual Cockpit Module. The work was broken down into three tasks, starting with the design of the software module then continuing with the development of the semantic knowledge manager and the development of the software prototype to be used together with the semantic module and the integrated virtual environment. WP3 relies on input from WP1 regarding the interface definitions and WP2 regarding task modelling.
Work Package 4: Virtual Cockpit Design Environment
This work package’s goal was the development of the Integrated Virtual Cockpit Environment. This was based on the HIM VR Platform, developed by OPTIS, where an add-on was developed for i-VISION purposes. The final Virtual Cockpit Design environment aimed at integrating all the technological components of i-VISION. This work package covers the creation of the 3D cockpit model, the adaptation of the VR platform, and the integration and interconnection of all research and development results coming from the others partners in view to provide a fully integrated virtual cockpit environment.
Work Package 5: Industrial Use-cases and Validation
In the context of this work package the industrial pilot scenarios were aimed to be defined and executed as a basis for the validation of the developed technologies. Usability tests with operators to assess the developed concepts were planned to be performed during these tests. Since the main objective of this work package was to validate the integration results of WP4 according to the requirements of WP1, a validation plan was to be defined as a basis to set-up the necessary equipment in order to assess the developed technologies under realistic conditions in an industrial environment.
Work Package 6: Dissemination and Exploitation
The objective of this work package was to prepare and support the exploitation and dissemination of the results of the i-VISION project. That included activities for the effective and sustainable dissemination of knowledge among and beyond the members of the consortium (beyond the life time of the project). Also, activities to promote the i-VISION project and European RTD efforts and make research results available to a larger scientific community, providing visibility to the results and attracting interest. Furthermore coordination of knowledge management and other innovation-related activities and engaging with policy makers, industrial stakeholders, SMEs, and other interested actors in a constant interaction and consultation mode with the project, as well as with their EU counterparts. More activities included promoting the exploitation of the results, providing for the adequate and effective protection of know-ledge created in the project, having due regard to the legitimate interests of the contractors concerned and reporting about these activities. Finally accelerating knowledge transfer and dissemination of know-how for future projects.
Work Package 7: Project Management
This WP was meant to cover the overall legal, contractual, financial and administrative management of the project and to establish a reliable contact with the EC throughout the project’s duration. The overall coordination and control of the international collaborative work was done by LMS as the coordinating partner, together with the management board. Moreover, this work package would ensure that the work is carried out in a time and cost effective manner and supervise the preparation and the overall quality of deliverables.
Project Results:
Work Package 1: Requirements Analysis and Specifications
The first task of the work package was Task 1.1: Product Requirements Analysis and Specifications. This task was responsible for the requirements elicitation of aerospace industry regarding the function and operation of the i-VISION tool the results of which are reported in D1.1.1. Based on these requirements, the specifications for the technology developments were built. On D1.1.2 an early cockpit model was described in terms of geometric components, on which the semantic developments were based. The model was designed to be easily scalable down to a smaller aircraft or up to an airliner. Task 1.2: Human Factors Requirements Analysis and Specifications focused on human factors requirements both for the technology and methods used in the Human-Cockpit Operations Analysis module, but also for the usability of the VR environments that were used in WP4. The requirements set by the industrial partners during this task provided the basis for the functionality of the Human-Cockpit Operations Analysis Module. The specifications provided description on the procedures and conditions taking place in the cockpit that are reported in D1.2.1. D1.2.2 describes an early model of the human procedures to be analysed that provided input for WP2 in order to drive the Human Tasks/Procedures Model task. In Task 1.3: Use-cases and Scenarios the use-cases and scenarios for i-VISION technologies were defined. These scenarios formed the basis for driving the technology development while laying the ground for the detailed definition in Task 5.1 where the pilot demonstrators were used for validating the developments. These scenarios and use-cases were used to generate a common understanding of the different scientific domains gathered in the i-VISION project and were reported in D1.3.1. Placing the use-case generation in such an early stage of the project enabled the consortium to agree on a final vision of the project results. The first use-case was human-centred cockpit design which included semi-automated work analysis, immersive human factor analysis and evaluation of design changes through virtual flight simulation. The second use-case of applications related to recurrent training of professional pilots, more specifically individual low-cost training with virtual co-pilot, remote collaborative training and analysis and supervision of trainees dealing with unexpected events. In order to have a smooth integration process towards the implementation of the pilot demonstrators, in addition to securing the robustness of the modules and environment cooperation, Task 1.4: Formats and Interfaces provided the framework in which the developments would take place. To facilitate the development process, an account of two things was needed: first, how to model the corresponding data in a way such that data handled by different modules could be aligned easily. The proposed method to achieve this was by resource-oriented modelling. Second, how to communicate between the components of the i-VISION software system in a lightweight fashion so as to ease the integration of the i-VISION tool’s components. The request-response message exchange pattern had proven successful for loosely coupled data exchange on the Web and RDF was presented as a flexible data model. In addition, in D1.4.1 high-level ontologies that can be used to represent virtual cockpits and hierarchical tasks were described, and the corresponding data serialisation and transport protocols were reported so as to ensure a homogeneous design of all components of i-VISION.
Work Package 2: Human-Cockpit Operations Analysis Module
In Task 2.1: Design of Human-Cockpit Operations Analysis Module the final down selection of Human Factors methods for use in i-VISION was performed and the method development processes that have been undertaken to enable the implementation of the methods in a virtual environment were documented in D2.1.1. The down selection was conducted in the context of mapping methods to the Human Factors requirements and mapping methods to the i-VISION use cases and scenarios. The results of Task 2.2: Human Tasks/Procedures Model were reported in D2.2.1and involved the detail of the Human-Task Procedures Model that underpins the Human Factors analysis methods in the i-VISION tool. The predominant method which acted as a baseline for the model was the Hierarchical Task Analysis (HTA). The deliverable summarised the method and provided a reference HTA based on an ILS approach and auto-landing task. That reference HTA was used as a template for the continuous development of the i-VISION tool and the subsequent evaluation and validation of the HF analyses with the tool. Work package 2 closed with Task 2.3: Development of Human-Cockpit Operations Analysis Module where the software module was progressed in two stages, the first prototype, described in D2.3.1 and the final prototype, described in D2.3.2. Human-Cockpit Operations analysis module was developed as a web application and provided to the stakeholders via a graphical user interface. In D2.3.1 the components of the module were described in a more technical level and a validation of the prototype’s usability in different stages of its development was included, in order to evaluate and justify the design decisions made through the development cycle, and finally set guidelines towards evolving into the final version of the HCOA module. The development of the first prototype implemented the design of Task 2.1 and was delivered in month 24 in order to conduct preliminary validations before continuing with the refinement of the final prototype to ensure the robustness of the software and the satisfaction of the industrial requirements.
The final prototype, described in D2.3.2 can work with captured human task and procedure data from the simulation (virtual reality environment) or manual input from the human factors expert. For some functionalities of the tool data is handled by the semantic module. The output it produces is analyses results which are displayed graphically in the interface of the module, and optionally highlighted in the virtual environment.
Work Package 3: Semantic Virtual Cockpit Module
Task 3.1: Design of Semantic Virtual Cockpit Module was about the design of the Semantic Virtual Cockpit Module. Reported in D3.1.1 were the constraints, requirements and necessary architecture for accessing, managing and manipulating semantic descriptions of scene graphs and the human task model design. In this deliverable there was also included the semantic modelling of different views on the elements of a cockpit. The interfaces for operating the semantic data management component were designed in detail, based on interface definitions from T1.4. An important aspect of the semantic module is integration and querying of semantic descriptions. In Task 3.2: Semantic Knowledge Management, the part of the Semantic Virtual Cockpit module that is responsible for managing the semantic data in the integrated environment was developed. D3.2.1 focused on how to access and combine the 3D geometric data and semantic data (high-level scene descriptions and human task descriptions) in a unified way, in order to facilitate analysis and simulation of interconnected components and tasks. In addition, the intelligent querying functions necessary for manipulating and analysing semantic data based on the input and functionality needed for the rest of the modules were also presented. The Knowledge Management Prototype contained the description of the data that are exchanged and distributed across the system starting from the development of the VR concepts that were used to detect the user interactions inside the Virtual Environment, formation of Semantic Queries and an initial mapping between the Linked Data - fu engine and a VR Scenegraph that was experimentally used.
Task 3.3: Development of Semantic Virtual Cockpit Module produced the first prototype of the module that hosted the semantic technologies developed for i-VISION. Two major components of the Semantic Virtual Cockpit module are described for this prototype; one is the VR Connector, a semantic stateful Linked Data interface to the stream of events in the Robot Operating System (ROS) network, which connects the components of the VR platform in the i-VISION tool, X-Plane and HIM. Second, a query processing and reasoning component which can evaluate queries while taking into account inferences, based on the data accessible from the VR platform. The semantic engine supports active components, i.e. is able to call functions in the VR platform based on inferences drawn. The first version of the prototype was delivered in month 24 in order to conduct preliminary validations before continuing with the refinement of the final prototype. In the final prototype, described in D3.3.2 all the four different steps required to access, integrate, query, and store data can be encoded in a high-level specification based on rules. Linked Data-Fu is an interpreter for such a specification, able to perform data retrieval, data integration via reasoning over retrieved schemata, and query evaluation. It allows for cohesive and seamless data processing that fulfils the performance requirements of a VR environment. It has been adapted to the i-VISION project’s specific needs, namely an HTTP interface for posting queries and programs directly has been developed as required in the final version of the Human-Cockpit Operations Analysis Module. Moreover, we developed a rule-based specification that interprets HTAs from the Human-Cockpit Operations Analysis Module such that we can track HTAs performed by pilots in the i-VISION tool. Further interpretations of HTAs allow for integrated querying of both the virtual aircraft and HTAs at the run-time of the i-VISION tool.
Work Package 4: Virtual Cockpit Design Environment
In Task 4.1: Design of Virtual Cockpit Design Environment the environment was designed to act as the means for information visualization and provisioning of the knowledge and analysis results in a highly interactive manner. D4.1.1 described the OPTIS HIM VR platform and as the Virtual Cockpit Design Environment refers to the architectural design of the i-VISION tool it also described the architecture of the design environment by detailing the functionality and the properties of its sub-components. This environment was designed to work with the Human-Cockpit Operations Analysis and Semantic modules in order to design virtual cockpits taking into account the use cases and scenarios described in deliverable D1.31. The design environment was carefully designed to be modular, easily updated and act as support for the evaluation of future cockpit designs. Also the flight simulation capabilities that were described, present a new solution in vast and innovative pilot training.
The next step was Task 4.2: Virtual Cockpit Model where the focus of the work was the description of the virtual cockpit in detail, providing screenshots with geometry and functional details. Also, cockpit elements correlation with the existing scenarios were made and finally cockpit modularity, versatility and testing oriented design was presented. An ontology of the virtual cockpit model was developed to be managed by the semantic knowledge manager. The model, presented in D.2.1 was used as a basis for the further development of the i-VISION tool. With this task as a bases, the prototype of Virtual Cockpit Design Environment that integrated the first versions of all the technological components, known as modules of the i-VISION tool was materialized during Task 4.3: Development of Virtual Cockpit Design Environment. This first prototype that was described in D4.3.1 took part in the preliminary validation activities of Task 4.4 in order to gather comments from the industrial partners and improve the functionality through refinement. It enabled a pilot to fly a virtual aircraft in virtual reality and with his actions during simulation being tracked. This, although it was the minimal environment that could demonstrate the i-VISION tool, it was the first step towards virtual cockpit evaluation with virtual reality and a good opportunity to observe how the tool would behave in harsh conditions of lighting, mobility and user expertise in the domain. Hardware and software concerns were taken into account and the final version of the virtual cockpit design environment prototype described in D4.3.2 integrated the final versions of all software modules. Major improvements of the final prototype included performance improvements for the 3D simulation, more realistic interactions with cockpit elements, integration of a virtual human for pilot or co-pilot simulation and support of semantic queries and physical human factors. The integrated prototype was developed in Task 4.4: Integration of Virtual Cockpit Design Environment and hosted all the technologies developed in i-VISION creating the final i-VISION tool.
D4.4.1 reports how developments from other work packages, namely the Human-Cockpit Operations Analysis Module, the Semantic Virtual Cockpit Module, the Virtual Reality Module, the Semantic Interpreter and the Flight Simulation Module, have been adjusted and interconnected. The resulting prototype acted as the final demonstrator of the project where the validation activities took place in work package 5.
Work Package 5: Industrial Use-cases and Validation
Task 5.1: Definition of Pilot Demonstrators prepared the validation of i-VISION prototypes against requirements from WP1. Deliverable 5.1.1 was comprised of refined scenarios, examples of supporting meta-data and a preliminary validation plan regarding i-VISION technologies. It described the amendments made to scenarios introduced previously and presented the common flying task of interest as well as the flight model chosen for the pilot demonstrator. It also gave examples of meta-data required for the industrial pilot demonstrator and showed several semantic mappings, serialized in JSON-LD (LD standing for Linked Data), a web development friendly format based on a data model equivalent to RDF. The preliminary validation plan was sketched based on the on-going progress and i-VISION requirements. Based on the definition of use-cases of this task, Task 5.2: Installation and Set-up of Demonstrators several real-life set-ups were built up to be able validate the different developments and modules of the i-VISION project. Several key components were provided by the partners to allow an integrated approach of the various use-cases. The first stage was described in D5.2.1 where a preliminary demonstrator was setup that provided a solid experimentation ground for the hardware set-ups that finally constituted the i-VISION tool, including some software concerns implied by the set-ups. The virtual reality hardware and some experimentations already made with it were presented. In stage two the final and mature demonstrators were setup in order to carry out the validation phase, through the execution and evaluation of the i-VISION scenarios. The sites that the pilot demonstrators were installed are the three industrial premises of Airbus G, located at Ottobrunn, Germany, Airbus F located in Suresnes, France and OPTIS located in Sophia Antipolis and Toulouse, France. In addition, two academic locations were also involved, especially during the development phase, those are the KIT at Karlsruhe, Germany and LMS at Patras, Greece. At the aforementioned sites, the prototypes developed in WP2, WP3 and WP4 were installed, based on a plan that defined which use-cases will be executed and what integrated prototypes were needed for the validation in the context of WP1 and WP6. Details about the demonstrator’s installation like requirements and instructions can be found in D5.2.2.
The purpose of Task 5.3: Validation was to assure the overall quality of the developed systems within i-VISION. Therefore within this task the various developments were intensively tested in respect to the different scenarios/use-cases, the software requirements and the overall system requirements as defined in Tasks 1.1 1.2 1.3 and 5.1. D5.3.1 presents a preliminary validation report where the different components developed in WP2, WP3 and WP4 were tested against the requirements defined in WP1. In this the validation parameters were gathered and the validation tests and activities that had been carried out in LMS Greece and at the industrial partners’ premises in Germany and France were described. The validation variables that were defined in this report served as the basis for the final validation of the i-VISION tool and were aligned in assuring the overall quality of the software developed. The validation efforts and results within the final validation phase of the i-VISION project were summarized in D5.3.2. The task of the document was to validate every requirement, consisting in total of 46 items from which 40 are mandatory while 6 are marked as nice-to-have. Each requirement was checked and documented in an individual table. The conclusion 39 out of 46 requirements are fully satisfied, while only three nice-to-have requirements were not validated.
Work Package 6: Dissemination and Exploitation
Promoting the use and dissemination of research results in the European industrial and the universal scientific communities was a key objective of the i-VISION project. The European Aeronautics Science Network (EASN) was in charge of Task 6.1: Dissemination where several dissemination activities were performed and detailed in D6.1.1 for the first period, in D6.1.2 until the end of the project. The dissemination activities included internal dissemination of results and knowledge generated within the i-VISION project by all partners to the relevant departments of their organisations and external dissemination beyond the consortium by the participating entities through publications of i-VISION related information in scientific and simple-language to either technical and/or non-technical networks. Also scientific publications in highly ranked journals, international and national conferences as well as workshops, the publication of project related information in the i-VISION public website which was designed for providing information about the project’s objectives, progress and results for different target groups and levels of dissemination, collaboration with other FP7 projects and display of the i- VISION simulator and videos in major aeronautics exhibitions (e.g. AERODAYS 2015, Farnborough 2016).
Due to the novelty research realized within i-VISION, the necessary measures were required to be applied towards protecting the legitimate interests of the involved parties with respect to the background introduced to the project and the foreground developed. To this end, special focus was placed on ensuring appropriate knowledge management and protection (Consortium Agreement). Finally, it is worth mentioning that the i-VISION partners will continue to disseminate the project’s results even after its completion.
The second task of this work package is Task 6.2: Exploitation. The exploitation objectives, plan, activities and exploitable results have been documented in D6.2.1 and D6.2.2.
The overall final exploitation plan of the partners towards exploiting the project results was created in order to support their own business or other activities relevant to the use of the project’s results. The exploitation objectives for i-VISION aimed mainly on the promotion of commercial exploitation for project results, the integration of those results to industrial and academic practice and the reuse in other projects. The main exploitable results are the i-VISION tool as an integrated solution, the HCOAM, the SVCM, the HIM and the SISSI.
Exploitable Result 1: HCOAM
The Human-Cockpit Operations Analysis Module is a web based application that allows the analyst to perform complex human factors methods, previously done by pen and paper, in a semi-automated way, for cockpit operations during various flight procedures. The module consists of the following main methods, which can be coarsely considered as it’s exploitable functionalities: Hierarchical Task Analysis, Error Analysis, Time Analysis, Engineer Queries and Self-Assessment. It can function as a standalone module (only the database additionally required), without communication with VR and with no engineer queries functionality.
Exploitable Result 2: SVCM
The Semantic Virtual Cockpit Module, manages integrated data that is being generated through the i-VISION tool. It consists of three distinct software artefacts and modelling work. The software artefacts are the following:
1. Semantic Engine, a rule engine for Linked Data (Linked Data Fu)
2. REST-ROS bridge (ROEST), an interface between the communication paradigm prevalent in the Virtual Cockpit Design Environment Module, publish/subscribe, and the paradigm of the Semantic Engine, Representational State Transfer
3. Writeable semantic Web resources (rwldresources), i. e. a database for semantic data, to maintain data from the HCOAM and the Virtual Cockpit Design Environment.
Exploitable Result 3: HIM
HIM is a virtual reality (VR) platform provided by OPTIS, and is the main component of VCDEM which is the graphical user interface for the i-VISION tool, including the virtual reality environment. The virtual environment loads the cockpit geometry environment and provides an immersive experience to the pilot, using data from flight simulation, managing cockpit element manipulation and sending events on the ROS Infrastructure for all detected pilot actions. An add-on of HIM has been developed especially to satisfy i-VISION requirements.
Exploitable Result 4: SISSI
SISSI is a semantic interpreter, which handles mapping between the virtual reality module and the flight simulator. SISSI also sends updates for new values that animate the virtual cockpit elements. The mappings used by SISSI to perform the interpretation are produced using the HMI annotator which provides a graphical user interface so as to create cockpit element mappings.
A list of exploitable functionalities for each individual exploitable result is defined in D6.2.2. This report also contains the final description over the IPR strategy.
Work Package 7: Project Management
This work package only consisted of on task, Task 7.1: Project Coordination, which involved the overall project and consortium management. LMS, as the project’s coordinator created and continuously updated the internal web-portal space in order to facilitate data/information storage and exchange, and support collaborative work. The project web-portal (D.1.1) contributed significantly in the efficient partner interactions and information exchange, which were also supported by the use of standard means, such as e-mailing, phone calls etc. Other management activities included planning, preparation (e.g. agendas, etc.), attending and reporting (e.g. minutes etc.) of project physical and virtual meetings. Also, monitoring of work progress, provision of guidelines for on-going technical activities, review of developments and deliverables, financial/administrative consultation, compilation of technical/administrative/financial reporting data and preparation of respective reports, including the two periodic reports submitted in month 18 and 36 respectively (D7.1.3 and D7.1.4). During the project all partners demonstrated a strong commitment to the objectives and activities according to the work plan, sustaining a smooth and effective collaboration through it that reflected on the results of the work undertaken. Having completed its final year and hence a successful period of project activities, i-VISION held its final General Assembly and Review Meeting from 9th to 10th of August 2016, in Munich, Germany.
Potential Impact:
Through its three distinct technological results, the i-VISION project applies its innovative research to the aerospace industry sector, focused on the design of cockpits using machine-interpretable knowledge of human tasks and procedures during various operating conditions. The combined technologies of i-VISION, when applied, can accelerate the design process thought the systematic reuse of knowledge while allowing for faster and more flexible creation of aircraft cockpit prototypes. In addition, the i-VISION tool allows offers a collaborative design environment that allows the effective analysis of human operations which in turn will result in highly competitive cockpits from the end-user’s perspective and will increase the utilisation of future aircraft by allowing human pilots to operate in extreme scenarios. i-VISION tool offers:
• A common platform for VR & cockpit related projects
Developing, testing and operating complex VR systems is a time and effort consuming process. The i-VISION tool will serve as a baseline for relevant future projects, providing a reusable and extendable platform for further developments.
• Reduced authoring costs for the preparation of virtual cockpit experimentations
A major cost regarding the development of VR environments is the time it takes to model the environment and setup the simulation behaviors. By integrating several modules required for a virtual cockpit experimentation into one tool, i-VISION enables easy and fast deployment of the tool.
• Set of public APIs for further developments
All i-VISION tool components can be modified to satisfy additional requirements through a set of APIs, for further research on the crucial field of virtual cockpit environments.
• Reduction of aircraft development costs by 50%
Using VR technologies for virtual prototyping aircraft cockpits can significantly reduce the costs of building physical prototypes. A cornerstone to reduce development costs is the ability to reduce errors in the early stages of the cockpit design, which in i-VISION is enabled by semantically modeling the aircraft, exploiting knowledge in the scene graph and automating parts of the human factors analyses which were previously performed by hand. i-VISION aids the engineers in assessing an aircraft long before it goes into prototype development or manufacturing, when a change is less expensive. Thus a semantic virtual model could “front-load” the analysis and evaluation work and shift those crucial tasks even more to an earlier point in the product development.
• Reduction in time-to-market for new aircraft cockpits up to 20%
Through effective virtual prototyping the need to create physical mock-ups is reduced and thus the overall time-to-market of new aircraft cockpits is reduced.
With the innovative combination of realistic virtual cockpit representation, semantic modeling and automating of procedures previously done by hand, the i-VISION tool supports the optimization of cockpit design and reduction of design tasks that would otherwise take up a lot of the engineer’s/ analyst’s time. Furthermore, the semantic descriptions in the virtual product could also be communicated to subcontractors and suppliers, thus creating the possibility to communicate certain product features in form of system simulators provided by the suppliers to the airframer. Development efforts can be shared between the partners and accelerate the cockpit development. Similarly, the “unit tests” provide opportunities for efficiency gains, which, beyond resulting in cost-savings, can also speed-up the time-to-market.
• Capture of pilot’s cognitive knowledge during pilot training
The virtual environment facilitates the verification of human performance in respect to cognitive load and human work in a modern cockpit. This use-case of applications relates to recurrent training of professional pilots, more specifically individual low-cost training with virtual co-pilot, remote collaborative training and analysis and supervision of trainees dealing with unexpected events.
• Reduction in travel charges up to 5%
The i-VISION tool provides a virtual cockpit environment that can be combined with various hardware piece of equipment, thus providing more realistic flight simulation conditions for human pilots. Also enables the pilot to operate in more difficult weather conditions, in airspaces where there is a lot of traffic and airports that are located in arias with geographic anomalies. This way the airlines will be able to reduce the travel charges through increased frequency of flights and reduced flight time which will in turn improve the fuel efficiency.
Besides the reduced direct operating costs of such aircraft, the resulting benefits of the reduced development costs and shorter time-to-markets would generate a cost benefit that the airframer would transfer to the airlines by offering less costly aircraft in comparison with the direct competitors. These reduced procurement costs will be directly forwarded to the passenger in form of reduced ticket prices in direct comparison within the ticket market.
Communication and dissemination activities are a crucial part of any research project as their aim is to attract the attention of as many relevant and interested people as possible. The dissemination activities within the i-VISION project aimed at targeting the academic, industrial and general public target groups with main scopes to:
• Create awareness, understanding, and interest within the entire European Aeronautics Community about the objectives and results of the project;
• Address all the relevant stakeholders through the communication of the right information to the right people at the right time using the right language;
• Engage the stakeholders and drive them to adopt and implement the results of the project so as to ensure the exploitation of the project’s results by the European Aeronautics Industry
• Demonstrate the added-value and positive impact of the project on demand and cost assessment and on the economic impact of semantic-based virtual environments and more specifically virtual cockpits.
To ensure that the project results have been disseminated in a consistent and constant manner to the relevant stakeholders, a carefully organized cost-efficient dissemination plan was developed during the first months of the project presenting the strategy that was designed to be followed throughout the project duration and beyond so as to disseminate the project’s results to all relevant target audiences, aiming at the effective communication of the developed knowledge through the selected dissemination actions. The i-VISION Consortium planned specific dissemination activities, either at individual level or as a joint effort of two or more partners, in order to achieve maximum spin-off, so that as many potential users as possible can/could benefit from the results of the project. Dissemination activities have been performed throughout the 36 months of the project (September 2013 – August 2016) and are planned to be performed even after the end of the project.
Most of the performed dissemination activities were focused on the scientific community (93.81%) and the Industry (89.69%), while the focus on policy makers (80.41%), Medias (56.70%) and general public (34.02%) follow. With regards to the project’s legacy i-VISION partners intend to continue the dissemination of the project’s outputs even after its completion. The intended dissemination activities are well balanced among the three main target groups (Scientific Community, Industry and Policy Makers) and focus in a similar way on Scientific Community, equally on the industry and on the policy makers, while the Medias and the general public target groups follow.
In addition, it is worth noting that the i-VISION consortium performed many dissemination activities to address the general public since a major objective of the dissemination and exploitation activities is transparency in terms of the financial resources spent. Last but not least, it is also worth highlighting the high level of i-VISION partners’ participation in workshops and conferences with dedicated booths for presenting the virtual cockpit elements. Through these participations, the i-VISION significant results have already and will be made publically known to a large number of technical and non-technical target groups.
List of Websites:
Project coordinator
Laboratory for Manufacturing Systems and Automation
Professor George Chryssolouris
xrisol@lms.mech.upatras.gr
Department of Mechanical Engineering and Aeronautics
University of Patras
Patras 26500, GREECE
Tel.: +30-2610-997262
Fax: +30-2610-997744
Project website: http://www.ivision-project.eu
YouTube channel: https://www.youtube.com/channel/UCB820w-RGBnqHJeGngOADfw
LinkedIn Group: http://www.linkedin.com/groups/iVISION-project-8385871/about
Twitter Profile: https://twitter.com/i_VISIONproject
The aerospace industry aims at reducing product development times and costs but faces a major obstacle; the need to build several physical prototypes for verifying various factors during design. Human factors considerations in the design process of aeronautic products play a crucial role for the reliability and resilience of the systems involved, from an operational and error tolerant point of view. For system design purposes there is significant utility in applying human task and cognitive workload analysis, however in existing systems the analysis and the analysed artefacts are decoupled and implemented as separate entities. Such separation leads to high manual effort for integration, while missing chances for automation and thus cost-reduction. Semantic representation of scene content and application entities is necessary for several kinds of intelligent virtual engineering tasks. Existing systems fail to implement a closed loop between semantics and 3D geometries and generally suffer from scalability and real-time performance issues. i-VISION progressed the current status of cognitive-human analysis of operations in aircraft cockpits using VR technologies, by advancing the methodologies with requirements from modern operating conditions. The i-VISION project’s outcome was designed and implemented as a platform with three distinct and complementary scientific and technological objectives, briefly described as follows:
Human-Cockpit Operations Analysis Module: This module is a web based application that accommodates advanced human factor methods (Hierarchical Task Analysis, Error Analysis, Time Analysis, Self-Assessment) for analysing the human procedures and tasks during various phases and operating conditions in a virtual reality (VR) based aircraft cockpit.
Semantic Virtual Cockpit Module: Semantic technologies are used in the i-VISION project so as to enable engineers and human factor experts to assess a virtual aircraft cockpit in a timely and cost-effective way. This module connects the Virtual Cockpit Design Environment Module and the Human-Cockpit Operations Analysis Module by integrating data and observing workflows, and allows for complex querying.
Virtual cockpit design environments: The Virtual Cockpit Design Environment serves as a reusable and low-cost simulation test bed for experimenting with various configurations allowing the human–centred assessment of future cockpit architectures. It enables the user to participate in complex simulations, tracks and records his actions which in turn act as a basis for human factors analysis.
Project Context and Objectives:
The i-VISION tool makes use of human factors background knowledge, semantic and immersive virtual reality technologies in order to support the human factors design and validation activities in aircraft cockpits, during the early phases of the product life-cycle.
The overall work that covered a 36 months period of industrial driven requirements, research, development and realization of final demonstrators, involved three stages towards the achievement of the project’s objectives. In the first phase, all the necessary input, mainly from industrial partners, were collected for developing the requirements in terms of technology and application. These requirements were analysed in order to develop the specifications of the technological modules and every component of the tool was designed. Two use cases were defined in an early stage so as to enhance the applicability of the developments in real life scenarios and validate the tool in the final stages of i-VISION.
During the second phase, implementation of the individual i-VISION components was performed, based on the detailed design that proceeded, during phase one. Second phase is the phase where the main project development took place, such Human-Cockpit Operations Analysis Module, Semantic Virtual Cockpit Module, Virtual Cockpit Design Environment and the semantic interpreter (SISSI). Finally the integration of the modules in a single tool was performed, acting as the industrial demonstrator on which the validation activities eventually took place.
In the third phase the industrial partners along with the support of research and application partners detailed the pilot cases and scenarios, previously defined, for validating the i-VISION prototypes. The validation of the tool took place where each requirement was checked and documented in an individual table. In parallel to the previous work, dissemination and exploitation activities were performed together with project management activities.
The project’s achievements were realized through the following six work packages:
Work Package 1: Requirements Analysis and Specifications
Work package 1 had as main objective the specifications’ definition for the i-VISION developments to follow in work packages 2 and 3. Industrial partners provided requirements based on their needs for a usable integrated tool that constituted the base for the specifications definition taking into account the academic and technological partners’ experience and knowledge in the applicable methods for structured human factors assessment methods, semantics and virtual reality technologies. The specifications were structure and validated throughout the four work tasks that comprise this Work Package. Realistic scenarios and use-cases were created to underpin the development of robust requirements and ensure a mutual understanding of the tools to be developed and integrated in the following work packages. Finally, a major objective of this work package was to define formats and interfaces that would allow a seamless transition of methodical toolsets into the virtual reality platform.
Work Package 2: Human-Cockpit Operations Analysis Module
The objective of this work package was the development of the Human-Cockpit Operations Analysis Module (HCOAM). The work was broken down into three tasks, starting with the design of the software module as a web application, paying attention to creating a user-friendly, familiar interface, with a feel as close to the conventional methods. The next step was the development of an ontology for the human tasks and procedure model and finally the development of the software prototype to be used together with the semantic module and the integrated virtual environment.
Work Package 3: Semantic Virtual Cockpit Module
The objective of this work package was the development of the Semantic Virtual Cockpit Module. The work was broken down into three tasks, starting with the design of the software module then continuing with the development of the semantic knowledge manager and the development of the software prototype to be used together with the semantic module and the integrated virtual environment. WP3 relies on input from WP1 regarding the interface definitions and WP2 regarding task modelling.
Work Package 4: Virtual Cockpit Design Environment
This work package’s goal was the development of the Integrated Virtual Cockpit Environment. This was based on the HIM VR Platform, developed by OPTIS, where an add-on was developed for i-VISION purposes. The final Virtual Cockpit Design environment aimed at integrating all the technological components of i-VISION. This work package covers the creation of the 3D cockpit model, the adaptation of the VR platform, and the integration and interconnection of all research and development results coming from the others partners in view to provide a fully integrated virtual cockpit environment.
Work Package 5: Industrial Use-cases and Validation
In the context of this work package the industrial pilot scenarios were aimed to be defined and executed as a basis for the validation of the developed technologies. Usability tests with operators to assess the developed concepts were planned to be performed during these tests. Since the main objective of this work package was to validate the integration results of WP4 according to the requirements of WP1, a validation plan was to be defined as a basis to set-up the necessary equipment in order to assess the developed technologies under realistic conditions in an industrial environment.
Work Package 6: Dissemination and Exploitation
The objective of this work package was to prepare and support the exploitation and dissemination of the results of the i-VISION project. That included activities for the effective and sustainable dissemination of knowledge among and beyond the members of the consortium (beyond the life time of the project). Also, activities to promote the i-VISION project and European RTD efforts and make research results available to a larger scientific community, providing visibility to the results and attracting interest. Furthermore coordination of knowledge management and other innovation-related activities and engaging with policy makers, industrial stakeholders, SMEs, and other interested actors in a constant interaction and consultation mode with the project, as well as with their EU counterparts. More activities included promoting the exploitation of the results, providing for the adequate and effective protection of know-ledge created in the project, having due regard to the legitimate interests of the contractors concerned and reporting about these activities. Finally accelerating knowledge transfer and dissemination of know-how for future projects.
Work Package 7: Project Management
This WP was meant to cover the overall legal, contractual, financial and administrative management of the project and to establish a reliable contact with the EC throughout the project’s duration. The overall coordination and control of the international collaborative work was done by LMS as the coordinating partner, together with the management board. Moreover, this work package would ensure that the work is carried out in a time and cost effective manner and supervise the preparation and the overall quality of deliverables.
Project Results:
Work Package 1: Requirements Analysis and Specifications
The first task of the work package was Task 1.1: Product Requirements Analysis and Specifications. This task was responsible for the requirements elicitation of aerospace industry regarding the function and operation of the i-VISION tool the results of which are reported in D1.1.1. Based on these requirements, the specifications for the technology developments were built. On D1.1.2 an early cockpit model was described in terms of geometric components, on which the semantic developments were based. The model was designed to be easily scalable down to a smaller aircraft or up to an airliner. Task 1.2: Human Factors Requirements Analysis and Specifications focused on human factors requirements both for the technology and methods used in the Human-Cockpit Operations Analysis module, but also for the usability of the VR environments that were used in WP4. The requirements set by the industrial partners during this task provided the basis for the functionality of the Human-Cockpit Operations Analysis Module. The specifications provided description on the procedures and conditions taking place in the cockpit that are reported in D1.2.1. D1.2.2 describes an early model of the human procedures to be analysed that provided input for WP2 in order to drive the Human Tasks/Procedures Model task. In Task 1.3: Use-cases and Scenarios the use-cases and scenarios for i-VISION technologies were defined. These scenarios formed the basis for driving the technology development while laying the ground for the detailed definition in Task 5.1 where the pilot demonstrators were used for validating the developments. These scenarios and use-cases were used to generate a common understanding of the different scientific domains gathered in the i-VISION project and were reported in D1.3.1. Placing the use-case generation in such an early stage of the project enabled the consortium to agree on a final vision of the project results. The first use-case was human-centred cockpit design which included semi-automated work analysis, immersive human factor analysis and evaluation of design changes through virtual flight simulation. The second use-case of applications related to recurrent training of professional pilots, more specifically individual low-cost training with virtual co-pilot, remote collaborative training and analysis and supervision of trainees dealing with unexpected events. In order to have a smooth integration process towards the implementation of the pilot demonstrators, in addition to securing the robustness of the modules and environment cooperation, Task 1.4: Formats and Interfaces provided the framework in which the developments would take place. To facilitate the development process, an account of two things was needed: first, how to model the corresponding data in a way such that data handled by different modules could be aligned easily. The proposed method to achieve this was by resource-oriented modelling. Second, how to communicate between the components of the i-VISION software system in a lightweight fashion so as to ease the integration of the i-VISION tool’s components. The request-response message exchange pattern had proven successful for loosely coupled data exchange on the Web and RDF was presented as a flexible data model. In addition, in D1.4.1 high-level ontologies that can be used to represent virtual cockpits and hierarchical tasks were described, and the corresponding data serialisation and transport protocols were reported so as to ensure a homogeneous design of all components of i-VISION.
Work Package 2: Human-Cockpit Operations Analysis Module
In Task 2.1: Design of Human-Cockpit Operations Analysis Module the final down selection of Human Factors methods for use in i-VISION was performed and the method development processes that have been undertaken to enable the implementation of the methods in a virtual environment were documented in D2.1.1. The down selection was conducted in the context of mapping methods to the Human Factors requirements and mapping methods to the i-VISION use cases and scenarios. The results of Task 2.2: Human Tasks/Procedures Model were reported in D2.2.1and involved the detail of the Human-Task Procedures Model that underpins the Human Factors analysis methods in the i-VISION tool. The predominant method which acted as a baseline for the model was the Hierarchical Task Analysis (HTA). The deliverable summarised the method and provided a reference HTA based on an ILS approach and auto-landing task. That reference HTA was used as a template for the continuous development of the i-VISION tool and the subsequent evaluation and validation of the HF analyses with the tool. Work package 2 closed with Task 2.3: Development of Human-Cockpit Operations Analysis Module where the software module was progressed in two stages, the first prototype, described in D2.3.1 and the final prototype, described in D2.3.2. Human-Cockpit Operations analysis module was developed as a web application and provided to the stakeholders via a graphical user interface. In D2.3.1 the components of the module were described in a more technical level and a validation of the prototype’s usability in different stages of its development was included, in order to evaluate and justify the design decisions made through the development cycle, and finally set guidelines towards evolving into the final version of the HCOA module. The development of the first prototype implemented the design of Task 2.1 and was delivered in month 24 in order to conduct preliminary validations before continuing with the refinement of the final prototype to ensure the robustness of the software and the satisfaction of the industrial requirements.
The final prototype, described in D2.3.2 can work with captured human task and procedure data from the simulation (virtual reality environment) or manual input from the human factors expert. For some functionalities of the tool data is handled by the semantic module. The output it produces is analyses results which are displayed graphically in the interface of the module, and optionally highlighted in the virtual environment.
Work Package 3: Semantic Virtual Cockpit Module
Task 3.1: Design of Semantic Virtual Cockpit Module was about the design of the Semantic Virtual Cockpit Module. Reported in D3.1.1 were the constraints, requirements and necessary architecture for accessing, managing and manipulating semantic descriptions of scene graphs and the human task model design. In this deliverable there was also included the semantic modelling of different views on the elements of a cockpit. The interfaces for operating the semantic data management component were designed in detail, based on interface definitions from T1.4. An important aspect of the semantic module is integration and querying of semantic descriptions. In Task 3.2: Semantic Knowledge Management, the part of the Semantic Virtual Cockpit module that is responsible for managing the semantic data in the integrated environment was developed. D3.2.1 focused on how to access and combine the 3D geometric data and semantic data (high-level scene descriptions and human task descriptions) in a unified way, in order to facilitate analysis and simulation of interconnected components and tasks. In addition, the intelligent querying functions necessary for manipulating and analysing semantic data based on the input and functionality needed for the rest of the modules were also presented. The Knowledge Management Prototype contained the description of the data that are exchanged and distributed across the system starting from the development of the VR concepts that were used to detect the user interactions inside the Virtual Environment, formation of Semantic Queries and an initial mapping between the Linked Data - fu engine and a VR Scenegraph that was experimentally used.
Task 3.3: Development of Semantic Virtual Cockpit Module produced the first prototype of the module that hosted the semantic technologies developed for i-VISION. Two major components of the Semantic Virtual Cockpit module are described for this prototype; one is the VR Connector, a semantic stateful Linked Data interface to the stream of events in the Robot Operating System (ROS) network, which connects the components of the VR platform in the i-VISION tool, X-Plane and HIM. Second, a query processing and reasoning component which can evaluate queries while taking into account inferences, based on the data accessible from the VR platform. The semantic engine supports active components, i.e. is able to call functions in the VR platform based on inferences drawn. The first version of the prototype was delivered in month 24 in order to conduct preliminary validations before continuing with the refinement of the final prototype. In the final prototype, described in D3.3.2 all the four different steps required to access, integrate, query, and store data can be encoded in a high-level specification based on rules. Linked Data-Fu is an interpreter for such a specification, able to perform data retrieval, data integration via reasoning over retrieved schemata, and query evaluation. It allows for cohesive and seamless data processing that fulfils the performance requirements of a VR environment. It has been adapted to the i-VISION project’s specific needs, namely an HTTP interface for posting queries and programs directly has been developed as required in the final version of the Human-Cockpit Operations Analysis Module. Moreover, we developed a rule-based specification that interprets HTAs from the Human-Cockpit Operations Analysis Module such that we can track HTAs performed by pilots in the i-VISION tool. Further interpretations of HTAs allow for integrated querying of both the virtual aircraft and HTAs at the run-time of the i-VISION tool.
Work Package 4: Virtual Cockpit Design Environment
In Task 4.1: Design of Virtual Cockpit Design Environment the environment was designed to act as the means for information visualization and provisioning of the knowledge and analysis results in a highly interactive manner. D4.1.1 described the OPTIS HIM VR platform and as the Virtual Cockpit Design Environment refers to the architectural design of the i-VISION tool it also described the architecture of the design environment by detailing the functionality and the properties of its sub-components. This environment was designed to work with the Human-Cockpit Operations Analysis and Semantic modules in order to design virtual cockpits taking into account the use cases and scenarios described in deliverable D1.31. The design environment was carefully designed to be modular, easily updated and act as support for the evaluation of future cockpit designs. Also the flight simulation capabilities that were described, present a new solution in vast and innovative pilot training.
The next step was Task 4.2: Virtual Cockpit Model where the focus of the work was the description of the virtual cockpit in detail, providing screenshots with geometry and functional details. Also, cockpit elements correlation with the existing scenarios were made and finally cockpit modularity, versatility and testing oriented design was presented. An ontology of the virtual cockpit model was developed to be managed by the semantic knowledge manager. The model, presented in D.2.1 was used as a basis for the further development of the i-VISION tool. With this task as a bases, the prototype of Virtual Cockpit Design Environment that integrated the first versions of all the technological components, known as modules of the i-VISION tool was materialized during Task 4.3: Development of Virtual Cockpit Design Environment. This first prototype that was described in D4.3.1 took part in the preliminary validation activities of Task 4.4 in order to gather comments from the industrial partners and improve the functionality through refinement. It enabled a pilot to fly a virtual aircraft in virtual reality and with his actions during simulation being tracked. This, although it was the minimal environment that could demonstrate the i-VISION tool, it was the first step towards virtual cockpit evaluation with virtual reality and a good opportunity to observe how the tool would behave in harsh conditions of lighting, mobility and user expertise in the domain. Hardware and software concerns were taken into account and the final version of the virtual cockpit design environment prototype described in D4.3.2 integrated the final versions of all software modules. Major improvements of the final prototype included performance improvements for the 3D simulation, more realistic interactions with cockpit elements, integration of a virtual human for pilot or co-pilot simulation and support of semantic queries and physical human factors. The integrated prototype was developed in Task 4.4: Integration of Virtual Cockpit Design Environment and hosted all the technologies developed in i-VISION creating the final i-VISION tool.
D4.4.1 reports how developments from other work packages, namely the Human-Cockpit Operations Analysis Module, the Semantic Virtual Cockpit Module, the Virtual Reality Module, the Semantic Interpreter and the Flight Simulation Module, have been adjusted and interconnected. The resulting prototype acted as the final demonstrator of the project where the validation activities took place in work package 5.
Work Package 5: Industrial Use-cases and Validation
Task 5.1: Definition of Pilot Demonstrators prepared the validation of i-VISION prototypes against requirements from WP1. Deliverable 5.1.1 was comprised of refined scenarios, examples of supporting meta-data and a preliminary validation plan regarding i-VISION technologies. It described the amendments made to scenarios introduced previously and presented the common flying task of interest as well as the flight model chosen for the pilot demonstrator. It also gave examples of meta-data required for the industrial pilot demonstrator and showed several semantic mappings, serialized in JSON-LD (LD standing for Linked Data), a web development friendly format based on a data model equivalent to RDF. The preliminary validation plan was sketched based on the on-going progress and i-VISION requirements. Based on the definition of use-cases of this task, Task 5.2: Installation and Set-up of Demonstrators several real-life set-ups were built up to be able validate the different developments and modules of the i-VISION project. Several key components were provided by the partners to allow an integrated approach of the various use-cases. The first stage was described in D5.2.1 where a preliminary demonstrator was setup that provided a solid experimentation ground for the hardware set-ups that finally constituted the i-VISION tool, including some software concerns implied by the set-ups. The virtual reality hardware and some experimentations already made with it were presented. In stage two the final and mature demonstrators were setup in order to carry out the validation phase, through the execution and evaluation of the i-VISION scenarios. The sites that the pilot demonstrators were installed are the three industrial premises of Airbus G, located at Ottobrunn, Germany, Airbus F located in Suresnes, France and OPTIS located in Sophia Antipolis and Toulouse, France. In addition, two academic locations were also involved, especially during the development phase, those are the KIT at Karlsruhe, Germany and LMS at Patras, Greece. At the aforementioned sites, the prototypes developed in WP2, WP3 and WP4 were installed, based on a plan that defined which use-cases will be executed and what integrated prototypes were needed for the validation in the context of WP1 and WP6. Details about the demonstrator’s installation like requirements and instructions can be found in D5.2.2.
The purpose of Task 5.3: Validation was to assure the overall quality of the developed systems within i-VISION. Therefore within this task the various developments were intensively tested in respect to the different scenarios/use-cases, the software requirements and the overall system requirements as defined in Tasks 1.1 1.2 1.3 and 5.1. D5.3.1 presents a preliminary validation report where the different components developed in WP2, WP3 and WP4 were tested against the requirements defined in WP1. In this the validation parameters were gathered and the validation tests and activities that had been carried out in LMS Greece and at the industrial partners’ premises in Germany and France were described. The validation variables that were defined in this report served as the basis for the final validation of the i-VISION tool and were aligned in assuring the overall quality of the software developed. The validation efforts and results within the final validation phase of the i-VISION project were summarized in D5.3.2. The task of the document was to validate every requirement, consisting in total of 46 items from which 40 are mandatory while 6 are marked as nice-to-have. Each requirement was checked and documented in an individual table. The conclusion 39 out of 46 requirements are fully satisfied, while only three nice-to-have requirements were not validated.
Work Package 6: Dissemination and Exploitation
Promoting the use and dissemination of research results in the European industrial and the universal scientific communities was a key objective of the i-VISION project. The European Aeronautics Science Network (EASN) was in charge of Task 6.1: Dissemination where several dissemination activities were performed and detailed in D6.1.1 for the first period, in D6.1.2 until the end of the project. The dissemination activities included internal dissemination of results and knowledge generated within the i-VISION project by all partners to the relevant departments of their organisations and external dissemination beyond the consortium by the participating entities through publications of i-VISION related information in scientific and simple-language to either technical and/or non-technical networks. Also scientific publications in highly ranked journals, international and national conferences as well as workshops, the publication of project related information in the i-VISION public website which was designed for providing information about the project’s objectives, progress and results for different target groups and levels of dissemination, collaboration with other FP7 projects and display of the i- VISION simulator and videos in major aeronautics exhibitions (e.g. AERODAYS 2015, Farnborough 2016).
Due to the novelty research realized within i-VISION, the necessary measures were required to be applied towards protecting the legitimate interests of the involved parties with respect to the background introduced to the project and the foreground developed. To this end, special focus was placed on ensuring appropriate knowledge management and protection (Consortium Agreement). Finally, it is worth mentioning that the i-VISION partners will continue to disseminate the project’s results even after its completion.
The second task of this work package is Task 6.2: Exploitation. The exploitation objectives, plan, activities and exploitable results have been documented in D6.2.1 and D6.2.2.
The overall final exploitation plan of the partners towards exploiting the project results was created in order to support their own business or other activities relevant to the use of the project’s results. The exploitation objectives for i-VISION aimed mainly on the promotion of commercial exploitation for project results, the integration of those results to industrial and academic practice and the reuse in other projects. The main exploitable results are the i-VISION tool as an integrated solution, the HCOAM, the SVCM, the HIM and the SISSI.
Exploitable Result 1: HCOAM
The Human-Cockpit Operations Analysis Module is a web based application that allows the analyst to perform complex human factors methods, previously done by pen and paper, in a semi-automated way, for cockpit operations during various flight procedures. The module consists of the following main methods, which can be coarsely considered as it’s exploitable functionalities: Hierarchical Task Analysis, Error Analysis, Time Analysis, Engineer Queries and Self-Assessment. It can function as a standalone module (only the database additionally required), without communication with VR and with no engineer queries functionality.
Exploitable Result 2: SVCM
The Semantic Virtual Cockpit Module, manages integrated data that is being generated through the i-VISION tool. It consists of three distinct software artefacts and modelling work. The software artefacts are the following:
1. Semantic Engine, a rule engine for Linked Data (Linked Data Fu)
2. REST-ROS bridge (ROEST), an interface between the communication paradigm prevalent in the Virtual Cockpit Design Environment Module, publish/subscribe, and the paradigm of the Semantic Engine, Representational State Transfer
3. Writeable semantic Web resources (rwldresources), i. e. a database for semantic data, to maintain data from the HCOAM and the Virtual Cockpit Design Environment.
Exploitable Result 3: HIM
HIM is a virtual reality (VR) platform provided by OPTIS, and is the main component of VCDEM which is the graphical user interface for the i-VISION tool, including the virtual reality environment. The virtual environment loads the cockpit geometry environment and provides an immersive experience to the pilot, using data from flight simulation, managing cockpit element manipulation and sending events on the ROS Infrastructure for all detected pilot actions. An add-on of HIM has been developed especially to satisfy i-VISION requirements.
Exploitable Result 4: SISSI
SISSI is a semantic interpreter, which handles mapping between the virtual reality module and the flight simulator. SISSI also sends updates for new values that animate the virtual cockpit elements. The mappings used by SISSI to perform the interpretation are produced using the HMI annotator which provides a graphical user interface so as to create cockpit element mappings.
A list of exploitable functionalities for each individual exploitable result is defined in D6.2.2. This report also contains the final description over the IPR strategy.
Work Package 7: Project Management
This work package only consisted of on task, Task 7.1: Project Coordination, which involved the overall project and consortium management. LMS, as the project’s coordinator created and continuously updated the internal web-portal space in order to facilitate data/information storage and exchange, and support collaborative work. The project web-portal (D.1.1) contributed significantly in the efficient partner interactions and information exchange, which were also supported by the use of standard means, such as e-mailing, phone calls etc. Other management activities included planning, preparation (e.g. agendas, etc.), attending and reporting (e.g. minutes etc.) of project physical and virtual meetings. Also, monitoring of work progress, provision of guidelines for on-going technical activities, review of developments and deliverables, financial/administrative consultation, compilation of technical/administrative/financial reporting data and preparation of respective reports, including the two periodic reports submitted in month 18 and 36 respectively (D7.1.3 and D7.1.4). During the project all partners demonstrated a strong commitment to the objectives and activities according to the work plan, sustaining a smooth and effective collaboration through it that reflected on the results of the work undertaken. Having completed its final year and hence a successful period of project activities, i-VISION held its final General Assembly and Review Meeting from 9th to 10th of August 2016, in Munich, Germany.
Potential Impact:
Through its three distinct technological results, the i-VISION project applies its innovative research to the aerospace industry sector, focused on the design of cockpits using machine-interpretable knowledge of human tasks and procedures during various operating conditions. The combined technologies of i-VISION, when applied, can accelerate the design process thought the systematic reuse of knowledge while allowing for faster and more flexible creation of aircraft cockpit prototypes. In addition, the i-VISION tool allows offers a collaborative design environment that allows the effective analysis of human operations which in turn will result in highly competitive cockpits from the end-user’s perspective and will increase the utilisation of future aircraft by allowing human pilots to operate in extreme scenarios. i-VISION tool offers:
• A common platform for VR & cockpit related projects
Developing, testing and operating complex VR systems is a time and effort consuming process. The i-VISION tool will serve as a baseline for relevant future projects, providing a reusable and extendable platform for further developments.
• Reduced authoring costs for the preparation of virtual cockpit experimentations
A major cost regarding the development of VR environments is the time it takes to model the environment and setup the simulation behaviors. By integrating several modules required for a virtual cockpit experimentation into one tool, i-VISION enables easy and fast deployment of the tool.
• Set of public APIs for further developments
All i-VISION tool components can be modified to satisfy additional requirements through a set of APIs, for further research on the crucial field of virtual cockpit environments.
• Reduction of aircraft development costs by 50%
Using VR technologies for virtual prototyping aircraft cockpits can significantly reduce the costs of building physical prototypes. A cornerstone to reduce development costs is the ability to reduce errors in the early stages of the cockpit design, which in i-VISION is enabled by semantically modeling the aircraft, exploiting knowledge in the scene graph and automating parts of the human factors analyses which were previously performed by hand. i-VISION aids the engineers in assessing an aircraft long before it goes into prototype development or manufacturing, when a change is less expensive. Thus a semantic virtual model could “front-load” the analysis and evaluation work and shift those crucial tasks even more to an earlier point in the product development.
• Reduction in time-to-market for new aircraft cockpits up to 20%
Through effective virtual prototyping the need to create physical mock-ups is reduced and thus the overall time-to-market of new aircraft cockpits is reduced.
With the innovative combination of realistic virtual cockpit representation, semantic modeling and automating of procedures previously done by hand, the i-VISION tool supports the optimization of cockpit design and reduction of design tasks that would otherwise take up a lot of the engineer’s/ analyst’s time. Furthermore, the semantic descriptions in the virtual product could also be communicated to subcontractors and suppliers, thus creating the possibility to communicate certain product features in form of system simulators provided by the suppliers to the airframer. Development efforts can be shared between the partners and accelerate the cockpit development. Similarly, the “unit tests” provide opportunities for efficiency gains, which, beyond resulting in cost-savings, can also speed-up the time-to-market.
• Capture of pilot’s cognitive knowledge during pilot training
The virtual environment facilitates the verification of human performance in respect to cognitive load and human work in a modern cockpit. This use-case of applications relates to recurrent training of professional pilots, more specifically individual low-cost training with virtual co-pilot, remote collaborative training and analysis and supervision of trainees dealing with unexpected events.
• Reduction in travel charges up to 5%
The i-VISION tool provides a virtual cockpit environment that can be combined with various hardware piece of equipment, thus providing more realistic flight simulation conditions for human pilots. Also enables the pilot to operate in more difficult weather conditions, in airspaces where there is a lot of traffic and airports that are located in arias with geographic anomalies. This way the airlines will be able to reduce the travel charges through increased frequency of flights and reduced flight time which will in turn improve the fuel efficiency.
Besides the reduced direct operating costs of such aircraft, the resulting benefits of the reduced development costs and shorter time-to-markets would generate a cost benefit that the airframer would transfer to the airlines by offering less costly aircraft in comparison with the direct competitors. These reduced procurement costs will be directly forwarded to the passenger in form of reduced ticket prices in direct comparison within the ticket market.
Communication and dissemination activities are a crucial part of any research project as their aim is to attract the attention of as many relevant and interested people as possible. The dissemination activities within the i-VISION project aimed at targeting the academic, industrial and general public target groups with main scopes to:
• Create awareness, understanding, and interest within the entire European Aeronautics Community about the objectives and results of the project;
• Address all the relevant stakeholders through the communication of the right information to the right people at the right time using the right language;
• Engage the stakeholders and drive them to adopt and implement the results of the project so as to ensure the exploitation of the project’s results by the European Aeronautics Industry
• Demonstrate the added-value and positive impact of the project on demand and cost assessment and on the economic impact of semantic-based virtual environments and more specifically virtual cockpits.
To ensure that the project results have been disseminated in a consistent and constant manner to the relevant stakeholders, a carefully organized cost-efficient dissemination plan was developed during the first months of the project presenting the strategy that was designed to be followed throughout the project duration and beyond so as to disseminate the project’s results to all relevant target audiences, aiming at the effective communication of the developed knowledge through the selected dissemination actions. The i-VISION Consortium planned specific dissemination activities, either at individual level or as a joint effort of two or more partners, in order to achieve maximum spin-off, so that as many potential users as possible can/could benefit from the results of the project. Dissemination activities have been performed throughout the 36 months of the project (September 2013 – August 2016) and are planned to be performed even after the end of the project.
Most of the performed dissemination activities were focused on the scientific community (93.81%) and the Industry (89.69%), while the focus on policy makers (80.41%), Medias (56.70%) and general public (34.02%) follow. With regards to the project’s legacy i-VISION partners intend to continue the dissemination of the project’s outputs even after its completion. The intended dissemination activities are well balanced among the three main target groups (Scientific Community, Industry and Policy Makers) and focus in a similar way on Scientific Community, equally on the industry and on the policy makers, while the Medias and the general public target groups follow.
In addition, it is worth noting that the i-VISION consortium performed many dissemination activities to address the general public since a major objective of the dissemination and exploitation activities is transparency in terms of the financial resources spent. Last but not least, it is also worth highlighting the high level of i-VISION partners’ participation in workshops and conferences with dedicated booths for presenting the virtual cockpit elements. Through these participations, the i-VISION significant results have already and will be made publically known to a large number of technical and non-technical target groups.
List of Websites:
Project coordinator
Laboratory for Manufacturing Systems and Automation
Professor George Chryssolouris
xrisol@lms.mech.upatras.gr
Department of Mechanical Engineering and Aeronautics
University of Patras
Patras 26500, GREECE
Tel.: +30-2610-997262
Fax: +30-2610-997744
Project website: http://www.ivision-project.eu
YouTube channel: https://www.youtube.com/channel/UCB820w-RGBnqHJeGngOADfw
LinkedIn Group: http://www.linkedin.com/groups/iVISION-project-8385871/about
Twitter Profile: https://twitter.com/i_VISIONproject
