Periodic Reporting for period 2 - NewSense (Evaluation of 5G Network and mmWave Radar Sensors to Enhance Surveillance of the Airport Surface)
Période du rapport: 2021-11-01 au 2022-12-31
NewSense is a research and development project that aims at improving safety and efficiency of operations primarily in secondary airports with innovative low-cost surface surveillance solutions allowing the implementation of affordable Advanced-Surface Movement Guidance and Control System (A-SMGCS). It also aims at developing gap-filler solutions that could be deployed at larger airports to cover up current system limitations such as coverage issues (e.g. parking, apron areas).
The project will explore a long-term opportunity to position objects on the airport surface by using 5G based signals in addition to low-cost millimetre waves (mmWave) radar combined with Artificial Intelligence (AI). The project has the following detailed objectives:
• Objective 1: To propose an initial system design for an A-SMGCS system based on 5G Software Design Radio (SDR) receivers with 3D Vector Antennas (VA) and relying on mmWave radar augmented with AI. (WP1)
- Identify and analyse the operational, technical performance, safety, and security requirements applicable to such system for use in an A-SMGCS from the existing standards. (D1.1)
- Propose an initial system design and guidelines for building an A-SMCGS system based on NewSense sensors. (D1.2)
• Objective 2: To design a 5G-signal-based surveillance function for use in A-SMGCS including: (WP1 & WP2)
- 3D vector antenna (VA): source of Angle of Arrival (AOA) estimation. (D2.1 D2.2 & D2.3)
- A 5G positioning function identifying and calculating cooperative targets position using AOA and estimating Time of Arrival (TOA) from their transmitted 5G Radio Frequency (RF) signals. (D2.1 D3.1 & D3.2)
- A radar-like system relying on 5G signals to calculate all targets position from AOA and TOA of reflected 5G Base Station (BS) RF signals. (D3.1)
• Objective 3: To evaluate low-cost mmWave radar as a non-cooperative surveillance solution for use in A-SMGCS. (WP4)
- Consolidate a preliminary mmWave technology capabilities assessment. (D4.1)
- Provides an assessment of the mmWave radar augmented with AI to recognize target types from reflected mmWave radar signals. (D4.2)
The project will explore a long-term opportunity to position objects on the airport surface by using 5G based signals in addition to low-cost millimetre waves (mmWave) radar combined with Artificial Intelligence (AI). The project has the following detailed objectives:
• Objective 1: To propose an initial system design for an A-SMGCS system based on 5G Software Design Radio (SDR) receivers with 3D Vector Antennas (VA) and relying on mmWave radar augmented with AI. (WP1)
- Identify and analyse the operational, technical performance, safety, and security requirements applicable to such system for use in an A-SMGCS from the existing standards. (D1.1)
- Propose an initial system design and guidelines for building an A-SMCGS system based on NewSense sensors. (D1.2)
• Objective 2: To design a 5G-signal-based surveillance function for use in A-SMGCS including: (WP1 & WP2)
- 3D vector antenna (VA): source of Angle of Arrival (AOA) estimation. (D2.1 D2.2 & D2.3)
- A 5G positioning function identifying and calculating cooperative targets position using AOA and estimating Time of Arrival (TOA) from their transmitted 5G Radio Frequency (RF) signals. (D2.1 D3.1 & D3.2)
- A radar-like system relying on 5G signals to calculate all targets position from AOA and TOA of reflected 5G Base Station (BS) RF signals. (D3.1)
• Objective 3: To evaluate low-cost mmWave radar as a non-cooperative surveillance solution for use in A-SMGCS. (WP4)
- Consolidate a preliminary mmWave technology capabilities assessment. (D4.1)
- Provides an assessment of the mmWave radar augmented with AI to recognize target types from reflected mmWave radar signals. (D4.2)
NewSense project evaluates the use of low-cost innovative sensors for use in A-SMGCS and proposes two Surveillance solutions; the 5G Surveillance solution, including a cooperative 5G signal based surveillance sensor and a non-cooperative Synthetic-aperture radar (SAR) imaging with 5G signals sensor, and the mmWave Surveillance solution augmented with Artificial Intelligence for non-cooperative targets positioning and classification.
The proposed 5G Surveillance Solution works in two configurations:
• An uplink (UL) configuration, with base stations equipped with 3D VA antennas and positioning of the user equipment (UE) done at the base station side.
• A downlink (DL) configuration, with UE equipped with 3D VA antennas and positioning of the user equipment (UE) done at the UE side.
5G positioning using both angle and time estimates from different 5G reference signals in both DL and UL directions are promising for secondary airport surveillance systems. For the timing estimates, correlation results of reference signals are to be used with the assumption of receiver system having some prior information about these reference signals. For the angle estimates, 3D-VA antenna structures are used due to their direction detection capabilities. The output of these antenna structures is processed by subspace-based MUSIC AoA estimation algorithm to estimate the angles. 5G positioning explored the combination of 3D-VA and innovative signal processing techniques based on time and angle measurements, as well as Machine Learning based Line of Sight (LOS) detection mechanisms for use in airport surveillance. The performances obtained with the 5G Surveillance solution are promising especially for LOS scenario that concerns mainly the Manoeuvring area and taxiways as there are less obstacles and thus less multi-path near these areas (depending on the position of the Base Station relative to these areas). The performances decrease significantly for Non LOS (NLOS) which could concern mainly apron and stands area.
The use of mmWave radar for positioning is a promising complementary tool in future airport surveillance. The mmWave Radar Surveillance Solution is based on the use of a non-cooperative mmWave radar to position and detect the type of targets (e.g. aircraft, vehicle and person) on the airport surface using mmWave signals and Machine Learning. The technology is based on a Frequency Modulated Continuous Wave (FMCW) mmWave radar operating in the 77 – 81 GHz frequency band. The mmWave radar includes the RF front-end, the ADC and processing modules used for positioning, velocity calculation and classification using Machine Learning algorithms. The maximum range of the radar depends on the radar configuration, the Radar Cross Section (RCS) of the target and the radar transmit power. The mmWav radar used within this study has a transmit power up to 12 dBm (16 mW) and it was able to detect a truck up to 137 meters (as example, typical Surface Movement Radar (SMR) has a transmitting power superior to 180 Watt and its coverage is between 150m to 2500m). We estimate that Commercial aircraft could be detected at a range of 200 meters within the same conditions and we assume that a maximum range of 500 meters can be reached with a transmitted power of 1 Watt. Measurements demonstrate that the mmWave radar offer a high accuracy which makes possible to differentiate two objects that are close and to detect accurately some events such as ATOT, ALDT, AIBT, AOBT, etc. Machine Learning applied to radar data makes possible to classify the target. An accuracy of 90% could be achieved using the range-angle heatmaps with YOLOv4 network.
NewSense project participated to multiple dissemination events to present the project objectives and results (D5.3).
The proposed 5G Surveillance Solution works in two configurations:
• An uplink (UL) configuration, with base stations equipped with 3D VA antennas and positioning of the user equipment (UE) done at the base station side.
• A downlink (DL) configuration, with UE equipped with 3D VA antennas and positioning of the user equipment (UE) done at the UE side.
5G positioning using both angle and time estimates from different 5G reference signals in both DL and UL directions are promising for secondary airport surveillance systems. For the timing estimates, correlation results of reference signals are to be used with the assumption of receiver system having some prior information about these reference signals. For the angle estimates, 3D-VA antenna structures are used due to their direction detection capabilities. The output of these antenna structures is processed by subspace-based MUSIC AoA estimation algorithm to estimate the angles. 5G positioning explored the combination of 3D-VA and innovative signal processing techniques based on time and angle measurements, as well as Machine Learning based Line of Sight (LOS) detection mechanisms for use in airport surveillance. The performances obtained with the 5G Surveillance solution are promising especially for LOS scenario that concerns mainly the Manoeuvring area and taxiways as there are less obstacles and thus less multi-path near these areas (depending on the position of the Base Station relative to these areas). The performances decrease significantly for Non LOS (NLOS) which could concern mainly apron and stands area.
The use of mmWave radar for positioning is a promising complementary tool in future airport surveillance. The mmWave Radar Surveillance Solution is based on the use of a non-cooperative mmWave radar to position and detect the type of targets (e.g. aircraft, vehicle and person) on the airport surface using mmWave signals and Machine Learning. The technology is based on a Frequency Modulated Continuous Wave (FMCW) mmWave radar operating in the 77 – 81 GHz frequency band. The mmWave radar includes the RF front-end, the ADC and processing modules used for positioning, velocity calculation and classification using Machine Learning algorithms. The maximum range of the radar depends on the radar configuration, the Radar Cross Section (RCS) of the target and the radar transmit power. The mmWav radar used within this study has a transmit power up to 12 dBm (16 mW) and it was able to detect a truck up to 137 meters (as example, typical Surface Movement Radar (SMR) has a transmitting power superior to 180 Watt and its coverage is between 150m to 2500m). We estimate that Commercial aircraft could be detected at a range of 200 meters within the same conditions and we assume that a maximum range of 500 meters can be reached with a transmitted power of 1 Watt. Measurements demonstrate that the mmWave radar offer a high accuracy which makes possible to differentiate two objects that are close and to detect accurately some events such as ATOT, ALDT, AIBT, AOBT, etc. Machine Learning applied to radar data makes possible to classify the target. An accuracy of 90% could be achieved using the range-angle heatmaps with YOLOv4 network.
NewSense project participated to multiple dissemination events to present the project objectives and results (D5.3).
The findings so far in NewSense show that the 5G signal based surveillance solution and the mmWave radar surveillance have promising potential for use in airport surveillance especially for small- and medium-sized airport areas and can support the FlightPath 2050 European vision towards increased safety and security.
Regarding the 5G Surveillance solution, extending the simulations to outdoors measurements with multiple base stations would be a transition in the 5G surveillance solution development and maturity level. Considering the performances obtained with the 5G positioning (i.e. high accuracy in LOS scenario), the use of the 5G Surveillance Solution could be extended to Terminal Manoeuvring Area (TMA) and complex airports.
For the mmWave radar, the next steps would be to look for solution to increase the maximum range of the radar, either by searching existing products available in the short/medium term taking into consideration the rapidly growth of mmWave industry, or by prototyping a radar that combines a higher transmit power, providing at least 500 meters of maximum range, and a synchronized rotating pedestal in order to increase the Field of View of the radar. Machine Learning algorithms for classification should be investigated further to enhance the performances and to evaluate the processing resources and time. In addition to that, it would be interesting to assess the mmWave radar performances in low visibility and all-weather conditions (e.g. night-time, fog and heavy rain, etc.).
Regarding the 5G Surveillance solution, extending the simulations to outdoors measurements with multiple base stations would be a transition in the 5G surveillance solution development and maturity level. Considering the performances obtained with the 5G positioning (i.e. high accuracy in LOS scenario), the use of the 5G Surveillance Solution could be extended to Terminal Manoeuvring Area (TMA) and complex airports.
For the mmWave radar, the next steps would be to look for solution to increase the maximum range of the radar, either by searching existing products available in the short/medium term taking into consideration the rapidly growth of mmWave industry, or by prototyping a radar that combines a higher transmit power, providing at least 500 meters of maximum range, and a synchronized rotating pedestal in order to increase the Field of View of the radar. Machine Learning algorithms for classification should be investigated further to enhance the performances and to evaluate the processing resources and time. In addition to that, it would be interesting to assess the mmWave radar performances in low visibility and all-weather conditions (e.g. night-time, fog and heavy rain, etc.).