Periodic Reporting for period 2 - VISTA (Vision-based Inspection Systems for automated Testing of Aircraft interiors)
Reporting period: 2019-08-01 to 2021-01-31
Post-assembly testing of aircraft interior installations in both cabin and cargo units is fundamental to ensure quality and reliability of located components. Currently, most of aircraft interior testing activities are carried out by human surveyors, often in non-ergonomic conditions. Process chains are extremely complex and do not guarantee a sufficient level of transparency of the assembly status at all times.
Innovative automated testing systems aimed to support human work force are therefore needed, in order to reduce time and costs and to increase reliability, flexibility and transparency of tolerance control and quality check of the cabin and cargo interior final assembly.
The overall goal of the project is the development and implementation of a multi-sensor platform and sensor processing algorithms to be integrated on-board a mobile system (AGV) to provide efficient automated testing of cabin and cargo installations.
The project was initially planned to span 30 months. However, COVID-19 situation impacted on scheduling, with the project being granted an extension by JU and IFAM of 6 months. Moreover, due to logistic constraints, the venue for the final experiments was changed from Stade (Germany) to partners facilities in Bari (Italy). In the end, the VISTA project successfully identified and developed technologies and methodologies aimed at spotting both geometrical and surface defects, developing a complete networked system consisting of an AGV (automated guided vehicle) fitted with sensors for the acquisition of panels, and a networked supervisor and server for processing data and hosting results. Additionally, VISTA implemented novel reporting tools that could be used by officers for working better, both on-site and off-site, using tools customized for augmented reality or virtual reality, while validating the identified defects.
Innovative automated testing systems aimed to support human work force are therefore needed, in order to reduce time and costs and to increase reliability, flexibility and transparency of tolerance control and quality check of the cabin and cargo interior final assembly.
The overall goal of the project is the development and implementation of a multi-sensor platform and sensor processing algorithms to be integrated on-board a mobile system (AGV) to provide efficient automated testing of cabin and cargo installations.
The project was initially planned to span 30 months. However, COVID-19 situation impacted on scheduling, with the project being granted an extension by JU and IFAM of 6 months. Moreover, due to logistic constraints, the venue for the final experiments was changed from Stade (Germany) to partners facilities in Bari (Italy). In the end, the VISTA project successfully identified and developed technologies and methodologies aimed at spotting both geometrical and surface defects, developing a complete networked system consisting of an AGV (automated guided vehicle) fitted with sensors for the acquisition of panels, and a networked supervisor and server for processing data and hosting results. Additionally, VISTA implemented novel reporting tools that could be used by officers for working better, both on-site and off-site, using tools customized for augmented reality or virtual reality, while validating the identified defects.
The work for the reported period has been mostly devoted for performing a close assessment of the measurement needs to reach the overall objectives and in devising the inspection strategy, the imaging sensor needed and in general the overall system architecture.
In particular it has addressed the requirements of the automated measurement system and the related test strategy needed to detect the panel defects that might arise because of an automated assembly of representative part of the internal aircraft linings. In order, this has required to analyze the standard procedures already in place while assembling a single isle civil aircraft, with the objective of identifying a set of guidelines useful for the automated assembly of ceiling panels, side walls panels, hatracks and cargo panels, that are considered in the VISTA project.
Following a classification already being used in Airbus, this list of requirements has been split between cabin and cargo area. Additionally, these requirements have been identified either as related to geometrical defects or to surface defects.
The requirements of the automated measurement system have then been used to compare different competing technologies for finding geometrical or surface defects. Suitable technologies that have been examined included laser profile scanners, structured light, time of flight, stereo and color cameras. This comparison identified a much smaller subset of suitable technologies. These early experiments have therefore enabled the selection of two specific sensors for the detection of geometric and surface defects, usable both in the cabin and cargo area.
The final developed solution was built around the design of an automated guided vehicle, able to navigate a structured environment with the objective of accomplishing acquisition missions tailored for specific environments and aircraft panels, using a sensor specialized for the identification of alignment-related defects and geometric defects and a sensor for color and texture related defects. The mobile robot can inspect cargo and cabin areas. Specific challenges related to these environments were addressed while designing an acquisition system that could reach highly place vantage points for the inspection of hatracks while moving in the cabin area, and at the same time being able to move inside the cargo area, where the ceiling is very low and the moving space is at a premium. In order to achieve this, a lightweight robot arm (Universal Robots UR10e) aided by a lift kit was chosen.
Acquisition strategies were tested in a simulated environment. A planner, able to identify the number of overlapped acquisitions needed to find surface and geometrical defects related to panels and the best acquisition point of view for each acquisition was developed. The planned solutions, tailored to panels in cargo and cabin environments were then tested in the real world, with a different computer node tasked for collecting acquired data, process and identify possible defects and store them. Additionally, VISTA implemented novel reporting tools that could be used by officers for working better, both on-site and off-site, using tools customized for augmented reality or virtual reality, while validating the identified defects.
Dissemination activities are ongoing, even with the organization of special sessions in metrology and aerospace related conferences, while partners companies are focusing on plans for exploiting results in industrial applications.
In particular it has addressed the requirements of the automated measurement system and the related test strategy needed to detect the panel defects that might arise because of an automated assembly of representative part of the internal aircraft linings. In order, this has required to analyze the standard procedures already in place while assembling a single isle civil aircraft, with the objective of identifying a set of guidelines useful for the automated assembly of ceiling panels, side walls panels, hatracks and cargo panels, that are considered in the VISTA project.
Following a classification already being used in Airbus, this list of requirements has been split between cabin and cargo area. Additionally, these requirements have been identified either as related to geometrical defects or to surface defects.
The requirements of the automated measurement system have then been used to compare different competing technologies for finding geometrical or surface defects. Suitable technologies that have been examined included laser profile scanners, structured light, time of flight, stereo and color cameras. This comparison identified a much smaller subset of suitable technologies. These early experiments have therefore enabled the selection of two specific sensors for the detection of geometric and surface defects, usable both in the cabin and cargo area.
The final developed solution was built around the design of an automated guided vehicle, able to navigate a structured environment with the objective of accomplishing acquisition missions tailored for specific environments and aircraft panels, using a sensor specialized for the identification of alignment-related defects and geometric defects and a sensor for color and texture related defects. The mobile robot can inspect cargo and cabin areas. Specific challenges related to these environments were addressed while designing an acquisition system that could reach highly place vantage points for the inspection of hatracks while moving in the cabin area, and at the same time being able to move inside the cargo area, where the ceiling is very low and the moving space is at a premium. In order to achieve this, a lightweight robot arm (Universal Robots UR10e) aided by a lift kit was chosen.
Acquisition strategies were tested in a simulated environment. A planner, able to identify the number of overlapped acquisitions needed to find surface and geometrical defects related to panels and the best acquisition point of view for each acquisition was developed. The planned solutions, tailored to panels in cargo and cabin environments were then tested in the real world, with a different computer node tasked for collecting acquired data, process and identify possible defects and store them. Additionally, VISTA implemented novel reporting tools that could be used by officers for working better, both on-site and off-site, using tools customized for augmented reality or virtual reality, while validating the identified defects.
Dissemination activities are ongoing, even with the organization of special sessions in metrology and aerospace related conferences, while partners companies are focusing on plans for exploiting results in industrial applications.
The project focus is now shifting towards the definition of suitable processing techniques able to analyze images and 3d profiles, looking for geometrical and surface defects. An important expected result from defects assessment done by human operators to defects assessment made by an AGV and then reported to a human operator is related to more objective assessment. Although keeping the human in the loop might suggest that subjectivity could still be an issue, it is expected that this approach will increase the reliability of the results for several reasons: first, it will provide reliable results that are less affected by measurement tools positioning mistakes done by humans; second, since possible defects reporting is performed after the whole aircraft have been inspected, all defects will be considered in a shorter time-span, enabling human operators to provide more consistent decisions since all the possible defects can be compared together in a shorter time and when human focus has not yet been affected by fatigue. Last but not least, the planned ability to operate in a virtual reality or augmented reality settings will provide additional flexibility: the inspector is no longer constrained to operate and take decisions near the assembly facilities. Instead it can work from everywhere and even share the burden of a decision with another inspector, connected elsewhere. Indeed, there are different benefits from operating by using in an augmented reality scenario, where the inspector can over-impose important information on the part of the aircraft panels being inspected and come to a more informed decision. The potential impact of this vision for the factory of the future is therefore considerable, with benefits in the accuracy of the measurements, the consistency of the decisions, the flexibility of being able to operate everywhere, overcoming any geographical constraints.