Final Report Summary - WING REFLECTOMETRY (In-Flight Monitoring of Wing Surface with Quasi tangential Reflectometry and Shadow Casting)
Executive Summary:
Laminar Flow is a key technology to reduce aircraft drag and fuel consumption. As part of the Clean Sky Program the Smart Fixed Wing Aircraft (SFWA) project has designed a new high speed laminar wing. The challenge to have a laminar flow at high speeds is an optimized wing profile with the smallest possible deviation between design and production and the smallest possible degradation during flight.
To test and particularly measure the principles, the outboard 9 meters of the wing of the High Speed Flight demonstrator, an A340-300, will be replaced with a new laminar profile wing and equipped with a specially designed measurement system.
However, the measurement method shall not affect the behavior of the wing profile deformation. This is preferably accomplished by a contactless measurement system.
Therefore FTI’s reflectometry methodology was developed to detect any local deformations of the laminar wing surface whilst airborne.
The reflectometry measurement is based on the measurement of the unknown real deformation of a known pattern combined with the as designed, not deformed wing surface. The set-up allows the installation on an aircraft without affecting its aerodynamic and structural characteristics.
The geometric limitation of the aircraft installation creates challenges for the measurement system. The location of the pattern, cameras and illumination require the reflection to be measured under a very shallow angle. The software algorithm must take into consideration that the pattern is already deformed in a defined way by the curved wing surface. As a system with multiple cameras is in use, the software must also be able to stitch picture overlap. An illumination system is also under development to overcome further constraints, such as bright sunlight, highly reflective paint schemes, vibration and so on. As the measurement system will be installed on a flying aircraft, it must also conform to requirements concerned with aircraft operation and safety.
Project Context and Objectives:
The shape of the optimal laminar wing has a specific curvature and makes available only limited space to integrate the reflectometry components such as cameras, pattern and illuminators. The shallow angle of the measurement system towards wing surface and reflection pattern - in combination with available optical systems depth of view, sensor resolution, light sensitivity and light availability – presented further challenges in transforming the system from laboratory to a practical application on the aircraft while maintaining measuring accuracy. The project has focused on establishing and validating solutions to these challenges.
The first developmental phases aimed at assuring the feasibility of detection and measurement of local waviness deformations on the upper surface of a wing using the reflectometry method and a static reference pattern. For validation of the approach and verifying the first software iterations and hardware architecture a laboratory set-up followed by a wing mock-up has been deployed. A provisional non-conforming measurement system with a camera array and conventional switches was used.
The full scale mock-up at FTI has proved to be a great benefit as it has allowed us to perform the development tests at an early stage in multiple configurations.
Objectives of the Laboratory set up and the full scale Mock-Up:
Quantify the measuring envelope and the resolution of the method
Ensure the feasibility of the technique
Test the surface properties like color and reflectivity of the wing for use of reflectometry.
Check the reachable zones of interest over the wing surface
Ensure the location of the area of interest on a deformable wing
Determine optimal amount, combination and position of the cameras
Validate illumination of the area of interest and resulting picture quality
Verify assumptions concerning the sensitivity of the system, picture noise.
Determine achievable picture quality and find improvement methods like filters, exposure times and noise reduction.
Determine external influences like sun-light, wing tip pod glass and small changes of camera positions.
Camera, lens and filter choice has been performed on the mock-up for the infrared illumination system. As a change to a visible light system is expected, the filters are under review.
Camera group and illuminator assembly position determined for an optimal coverage of the areas of interest.
The system architecture consists of 2 groups of cameras (4 each) and a signal distribution and power supply unit in the Wing Tip Pod and a control computer in the cabin.
Illumination design is ongoing as the current design iterations using IR light were not able to meet the requirements. The illuminator is a powerful beamed LED light source flashing near infrared light synchronously with the camera shutter opening. 2 Illuminators are planned to be installed in each wing tip pod.
The first design and function module was functional but not powerful enough in combination with the selected cameras and filters.
A second prototype was designed and assembled. Tests revealed that insufficient light power in combination with the selected cameras and filters was available and that the LEDs were not reliable enough when working at the specified maximum power setting.
Intensive market research for sufficient LEDs has shown that no adequate LEDs are presently available. Nonetheless it is expected that LEDs with the required power will be shortly available.
The ADU is a device designed to power 8 cameras, to collect the video signals from these cameras and to transfer the video data by means of optic fibers to a control computer located in the aircraft cabin. The design of the ADU is completed.
Eight cameras are installed in the wing tip pod and connected to the ADU.
The ADU is located near the cameras in the Wing Tip Pod. The acquisition and recording of the picture is done by a computer located in the cabin. This cabin computer is connected to the ADU using 2 multimode optical fiber connections.
The reflectometry system has to be adjusted and calibrated. The adjustment has to be done on the hardware; the calibration is done within the analysis software basing on hardware references.
The calibration itself has only been tested with the proposed method using artificial deformations. Current methods for calibration with e.g. laser distance measuring equipment will only work in a static laboratory environment as these sytems must be placed directly over the area of interest.
Shadow casting is an optical method to measure the height of step-like surface defects by evaluation of the length of a shadow cast by the defect when illuminated obliquely with quasi parallel light. The feasibility of the shadow casting method has been confirmed in the laboratory. The same hardware equipment as used for reflectometry will be applied: only position of the camera module need to be exchanged with illuminator position. The mechanical interfaces will be designed to support this position exchange. Due to the required lighting angles, shadow casting necessitates a flight profile relative to the sun to generate an optimal lighting direction.
Project Results:
The analysis software uses the images captured and stored during flight by the cabin computer.
The analysis software is installed on an independent ground computer. It is controlled via a graphical user interface (GUI). It is used to check image quality and usability. User input is required to select an area of interest containing the pattern on the fairing and its reflection. Additionally, the user has to select a marker, which needs to be in the area of interest, and whose position is known very accurately for later camera-pixel to surface correlation. This correlation is also the basis for calibration. Following the manual preparation, the computer based post processing and analysis of the recorded images takes place to calculate the absolute height of the wing’s surface. The result can be displayed as an elevation profile of the area of interest at a defined time during flight.
A high contrast diagonal stripes pattern and its reflection are analyzed. The output result is the surface deformed by aerodynamic loads. The deformed wing is then computationally compared to the designed wing to determine absolute height and position of the deformation.
Either a general calculation for line distance and angle of the upper stripes is performed, which is needed to accurately determine the surface deformations, or the deformed surface itself is calculated. From user input, the analysis SW knows an approximate position of the marking in the image. After extracting a cropped image around this position the software searches the highest intensity in the miniature image and estimates the position of the marking with a two-dimensional gauss fit with sub-pixel accuracy. As the analysis SW can only work with the part of the input image which contains the pattern and its reflection, the user selected area of interest is cut out of the input image for further analysis. After filtering, the image is binarized using a local mean value as threshold.
The camera control software has been realized in C#, integrating interfaces to 8 cameras, data handling and system health monitoring controlled and set up from a user interface (GUI).
Key Features
• Interfaces up to eight Cameras in parallel
• Displayed camera live image is user selectable
• Synchronizes all cameras to concurrent 2.5Hz image acquisition rate
• Acquired camera images are saved to hard disk drives in uncompressed bitmap
format upon user request
• System health is observed automatically
• Network failures, camera image acquisition and recording problems are recognized
• Camera temperature faults are detected and handled
• System drive usage percentage is monitored
The analysis software has been realized in Matlab, integrating processing of surface data acquired with camera control software and providing a user interface (GUI). The basic evaluation software produced usable results in a completely controlled laboratory environment on flat surfaces and ideal, small pattern reflections.
Following the move to the double curved surface of the wing mock-up, some parasitic effects had a much stronger influence than anticipated.
The low angle from the cameras to the surface resulted in a requirement for a long field of view and a large aperture setting which resulted in lower contrast than needed and a compensation method in the analysis software to filter out noise.
The low angle and the curved suface resulted in a highly non-linear distribution of resolution in the pictures. As a rule of thumb, the further away from the camera, the lower the resolution. Additionally, to cover the whole area of interest, the set of 4 pictures per measurement needed to be stitched to one picture similar to a panoramic picture. The evaluation software needed to be equipped with several compensation algorithms to take the non-linear resolution and the distortion at the lenses edges into account.
The integration of the aircraft wing point cloud as a calibration dataset proved to be challenging due to the large data to be processed for each measurement and the slight, at that point unquantified, deviations to the mock-up surface.
Key Features:
• Processing of surface data acquired with camera control software
• Analysis of four overlapping camera image streams displaying a reflected stripe pattern
• Calculation of absolute and relative height of the reflecting surface
• Correlation of camera pixels to aircraft’s coordinate system
• Display of height vs time plots for selectable surface points
• Integrated user interface (GUI)
The measurement system hardware consists of 2 modules with 4 cameras each per wing. The data stream of the cameras is collected in a single distribution unit per wing. This ADU transforms the local camera LAN signals to a dual optical fiber signal. The fiber optic signals from each wing ADU are collected by the cabin computer to be stored on solid state drives. To support the design of the hard- and software, laboratory equipment and a wing mock-up were produced.
During the conceptual phase for the reflectometry and shadow casting systems it was determined that an illumination of the areas of interest would be required. The illumination should provide a better contrast of the pictures, a defined light source and the possibility to fly at night. Moreover it was thought that an illumination in the infrared spectrum with respective filters on the cameras would reduce disturbances by natural sun light.
The first iteration of an off the shelf system did not produce sufficient power to provide availability of night flights. The second iteration used specially pulsed infrared LED arrays. This approach showed to be promising in lab tests but under long duration tests the chosen LEDs where not reliable enough for practical application. A new high power visible light LED was unfortunately not available in 2013. An optimization of the camera software to increase light sensitivity of the cameras resulted in increased noise of the pictures taken. Picture noise reduction algorithms were implemented to compensate.
Besides designing the hardware to be able to perform highly sensitive measurements during flight it had also to be designed to conform to regulations concerned with aircraft safety and operation.
Potential Impact:
Laminar flow technology promises significant energy savings in aviation. To enable more detailed and accurate research of laminar flow wings, a highly accurate, contactless measurement method must be made available.
During the project, the dissemination activity has mostly been performed by Airbus. FTI has presented the reflectometry system at several SFWA events but has not followed any other public or internal activities. Public dissemination is expected to take place after the BLADE flight tests and has been moved to follow up CfPs.
The exploitation of the research into the feasibility and realization of a surface measurement system will take place when the equipment is actually built and deployed on the BLADE Aircraft. It is expected that the method of measuring small deformations in flight will prove itself on the BLADE aircraft and may be available for other flight tests and customers who may have very similar requirements.
List of Websites:
FTI Engineering Network GmbH
Schmiedestrasse 2
15745 Wildau
http://www.cleansky.eu/
Laminar Flow is a key technology to reduce aircraft drag and fuel consumption. As part of the Clean Sky Program the Smart Fixed Wing Aircraft (SFWA) project has designed a new high speed laminar wing. The challenge to have a laminar flow at high speeds is an optimized wing profile with the smallest possible deviation between design and production and the smallest possible degradation during flight.
To test and particularly measure the principles, the outboard 9 meters of the wing of the High Speed Flight demonstrator, an A340-300, will be replaced with a new laminar profile wing and equipped with a specially designed measurement system.
However, the measurement method shall not affect the behavior of the wing profile deformation. This is preferably accomplished by a contactless measurement system.
Therefore FTI’s reflectometry methodology was developed to detect any local deformations of the laminar wing surface whilst airborne.
The reflectometry measurement is based on the measurement of the unknown real deformation of a known pattern combined with the as designed, not deformed wing surface. The set-up allows the installation on an aircraft without affecting its aerodynamic and structural characteristics.
The geometric limitation of the aircraft installation creates challenges for the measurement system. The location of the pattern, cameras and illumination require the reflection to be measured under a very shallow angle. The software algorithm must take into consideration that the pattern is already deformed in a defined way by the curved wing surface. As a system with multiple cameras is in use, the software must also be able to stitch picture overlap. An illumination system is also under development to overcome further constraints, such as bright sunlight, highly reflective paint schemes, vibration and so on. As the measurement system will be installed on a flying aircraft, it must also conform to requirements concerned with aircraft operation and safety.
Project Context and Objectives:
The shape of the optimal laminar wing has a specific curvature and makes available only limited space to integrate the reflectometry components such as cameras, pattern and illuminators. The shallow angle of the measurement system towards wing surface and reflection pattern - in combination with available optical systems depth of view, sensor resolution, light sensitivity and light availability – presented further challenges in transforming the system from laboratory to a practical application on the aircraft while maintaining measuring accuracy. The project has focused on establishing and validating solutions to these challenges.
The first developmental phases aimed at assuring the feasibility of detection and measurement of local waviness deformations on the upper surface of a wing using the reflectometry method and a static reference pattern. For validation of the approach and verifying the first software iterations and hardware architecture a laboratory set-up followed by a wing mock-up has been deployed. A provisional non-conforming measurement system with a camera array and conventional switches was used.
The full scale mock-up at FTI has proved to be a great benefit as it has allowed us to perform the development tests at an early stage in multiple configurations.
Objectives of the Laboratory set up and the full scale Mock-Up:
Quantify the measuring envelope and the resolution of the method
Ensure the feasibility of the technique
Test the surface properties like color and reflectivity of the wing for use of reflectometry.
Check the reachable zones of interest over the wing surface
Ensure the location of the area of interest on a deformable wing
Determine optimal amount, combination and position of the cameras
Validate illumination of the area of interest and resulting picture quality
Verify assumptions concerning the sensitivity of the system, picture noise.
Determine achievable picture quality and find improvement methods like filters, exposure times and noise reduction.
Determine external influences like sun-light, wing tip pod glass and small changes of camera positions.
Camera, lens and filter choice has been performed on the mock-up for the infrared illumination system. As a change to a visible light system is expected, the filters are under review.
Camera group and illuminator assembly position determined for an optimal coverage of the areas of interest.
The system architecture consists of 2 groups of cameras (4 each) and a signal distribution and power supply unit in the Wing Tip Pod and a control computer in the cabin.
Illumination design is ongoing as the current design iterations using IR light were not able to meet the requirements. The illuminator is a powerful beamed LED light source flashing near infrared light synchronously with the camera shutter opening. 2 Illuminators are planned to be installed in each wing tip pod.
The first design and function module was functional but not powerful enough in combination with the selected cameras and filters.
A second prototype was designed and assembled. Tests revealed that insufficient light power in combination with the selected cameras and filters was available and that the LEDs were not reliable enough when working at the specified maximum power setting.
Intensive market research for sufficient LEDs has shown that no adequate LEDs are presently available. Nonetheless it is expected that LEDs with the required power will be shortly available.
The ADU is a device designed to power 8 cameras, to collect the video signals from these cameras and to transfer the video data by means of optic fibers to a control computer located in the aircraft cabin. The design of the ADU is completed.
Eight cameras are installed in the wing tip pod and connected to the ADU.
The ADU is located near the cameras in the Wing Tip Pod. The acquisition and recording of the picture is done by a computer located in the cabin. This cabin computer is connected to the ADU using 2 multimode optical fiber connections.
The reflectometry system has to be adjusted and calibrated. The adjustment has to be done on the hardware; the calibration is done within the analysis software basing on hardware references.
The calibration itself has only been tested with the proposed method using artificial deformations. Current methods for calibration with e.g. laser distance measuring equipment will only work in a static laboratory environment as these sytems must be placed directly over the area of interest.
Shadow casting is an optical method to measure the height of step-like surface defects by evaluation of the length of a shadow cast by the defect when illuminated obliquely with quasi parallel light. The feasibility of the shadow casting method has been confirmed in the laboratory. The same hardware equipment as used for reflectometry will be applied: only position of the camera module need to be exchanged with illuminator position. The mechanical interfaces will be designed to support this position exchange. Due to the required lighting angles, shadow casting necessitates a flight profile relative to the sun to generate an optimal lighting direction.
Project Results:
The analysis software uses the images captured and stored during flight by the cabin computer.
The analysis software is installed on an independent ground computer. It is controlled via a graphical user interface (GUI). It is used to check image quality and usability. User input is required to select an area of interest containing the pattern on the fairing and its reflection. Additionally, the user has to select a marker, which needs to be in the area of interest, and whose position is known very accurately for later camera-pixel to surface correlation. This correlation is also the basis for calibration. Following the manual preparation, the computer based post processing and analysis of the recorded images takes place to calculate the absolute height of the wing’s surface. The result can be displayed as an elevation profile of the area of interest at a defined time during flight.
A high contrast diagonal stripes pattern and its reflection are analyzed. The output result is the surface deformed by aerodynamic loads. The deformed wing is then computationally compared to the designed wing to determine absolute height and position of the deformation.
Either a general calculation for line distance and angle of the upper stripes is performed, which is needed to accurately determine the surface deformations, or the deformed surface itself is calculated. From user input, the analysis SW knows an approximate position of the marking in the image. After extracting a cropped image around this position the software searches the highest intensity in the miniature image and estimates the position of the marking with a two-dimensional gauss fit with sub-pixel accuracy. As the analysis SW can only work with the part of the input image which contains the pattern and its reflection, the user selected area of interest is cut out of the input image for further analysis. After filtering, the image is binarized using a local mean value as threshold.
The camera control software has been realized in C#, integrating interfaces to 8 cameras, data handling and system health monitoring controlled and set up from a user interface (GUI).
Key Features
• Interfaces up to eight Cameras in parallel
• Displayed camera live image is user selectable
• Synchronizes all cameras to concurrent 2.5Hz image acquisition rate
• Acquired camera images are saved to hard disk drives in uncompressed bitmap
format upon user request
• System health is observed automatically
• Network failures, camera image acquisition and recording problems are recognized
• Camera temperature faults are detected and handled
• System drive usage percentage is monitored
The analysis software has been realized in Matlab, integrating processing of surface data acquired with camera control software and providing a user interface (GUI). The basic evaluation software produced usable results in a completely controlled laboratory environment on flat surfaces and ideal, small pattern reflections.
Following the move to the double curved surface of the wing mock-up, some parasitic effects had a much stronger influence than anticipated.
The low angle from the cameras to the surface resulted in a requirement for a long field of view and a large aperture setting which resulted in lower contrast than needed and a compensation method in the analysis software to filter out noise.
The low angle and the curved suface resulted in a highly non-linear distribution of resolution in the pictures. As a rule of thumb, the further away from the camera, the lower the resolution. Additionally, to cover the whole area of interest, the set of 4 pictures per measurement needed to be stitched to one picture similar to a panoramic picture. The evaluation software needed to be equipped with several compensation algorithms to take the non-linear resolution and the distortion at the lenses edges into account.
The integration of the aircraft wing point cloud as a calibration dataset proved to be challenging due to the large data to be processed for each measurement and the slight, at that point unquantified, deviations to the mock-up surface.
Key Features:
• Processing of surface data acquired with camera control software
• Analysis of four overlapping camera image streams displaying a reflected stripe pattern
• Calculation of absolute and relative height of the reflecting surface
• Correlation of camera pixels to aircraft’s coordinate system
• Display of height vs time plots for selectable surface points
• Integrated user interface (GUI)
The measurement system hardware consists of 2 modules with 4 cameras each per wing. The data stream of the cameras is collected in a single distribution unit per wing. This ADU transforms the local camera LAN signals to a dual optical fiber signal. The fiber optic signals from each wing ADU are collected by the cabin computer to be stored on solid state drives. To support the design of the hard- and software, laboratory equipment and a wing mock-up were produced.
During the conceptual phase for the reflectometry and shadow casting systems it was determined that an illumination of the areas of interest would be required. The illumination should provide a better contrast of the pictures, a defined light source and the possibility to fly at night. Moreover it was thought that an illumination in the infrared spectrum with respective filters on the cameras would reduce disturbances by natural sun light.
The first iteration of an off the shelf system did not produce sufficient power to provide availability of night flights. The second iteration used specially pulsed infrared LED arrays. This approach showed to be promising in lab tests but under long duration tests the chosen LEDs where not reliable enough for practical application. A new high power visible light LED was unfortunately not available in 2013. An optimization of the camera software to increase light sensitivity of the cameras resulted in increased noise of the pictures taken. Picture noise reduction algorithms were implemented to compensate.
Besides designing the hardware to be able to perform highly sensitive measurements during flight it had also to be designed to conform to regulations concerned with aircraft safety and operation.
Potential Impact:
Laminar flow technology promises significant energy savings in aviation. To enable more detailed and accurate research of laminar flow wings, a highly accurate, contactless measurement method must be made available.
During the project, the dissemination activity has mostly been performed by Airbus. FTI has presented the reflectometry system at several SFWA events but has not followed any other public or internal activities. Public dissemination is expected to take place after the BLADE flight tests and has been moved to follow up CfPs.
The exploitation of the research into the feasibility and realization of a surface measurement system will take place when the equipment is actually built and deployed on the BLADE Aircraft. It is expected that the method of measuring small deformations in flight will prove itself on the BLADE aircraft and may be available for other flight tests and customers who may have very similar requirements.
List of Websites:
FTI Engineering Network GmbH
Schmiedestrasse 2
15745 Wildau
http://www.cleansky.eu/