Community Research and Development Information Service - CORDIS

Periodic Report Summary 2 - SUNNY (Smart UNattended airborne sensor Network for detection of vessels used for cross border crime and irregular entrY)

Project Context and Objectives:
The control of the EU border is one of the issues that the EU is facing. The length of the border partially on land and partially over sea poses a challenge to the organisation in charge of the border control. National authorities cooperate with transnational organisation such FRONTEX agency, but the areas to be controlled are very large and sometimes remote and the resources available to cope with such tasks are insufficient to achieve the desired levels of effectiveness.

The SUNNY project aims to develop system solutions capable of improving the effectiveness of the EU border monitoring compared to the legacy systems whilst keeping the affordability and interoperability as key enabling factors. It is recognised that the legacy sensors and communications system developed for military applications are not optimised for border monitoring and their interoperability with civil standards is limited. Moreover, it is acknowledged that the diffusion of the information is tailored in service systems to highly skilled personnel and the number of operators to conduct the activity is high. Via the integration of technologies developed across different initiatives, the SUNNY approach will deliver pre-processed information with meaningful decision support tools enabling the reduction of number and required level of expertise of border surveillance personnel.

SUNNY represents a step beyond existing research projects due to the following main features:

• A two-tier intelligent heterogeneous UAV sensor network will be integrated in order to provide both large field and focused surveillance capabilities. In this network, the first-tier sensors, carried by Vertical Take-Off and Landing (VTOL) UAVs are used to patrol large border areas to detect suspicious targets and provide global situation awareness. Fed with the information collected by the first-tier sensors, the second-tier sensors will be deployed to provide more focused surveillance capability by tracking the targets and collecting further evidence for more accurate target recognition and threat evaluation. Novel algorithms will be developed to analyse the data collected by the sensors for robust and accurate target identification and event detection.

• Novel sensors and on-board processing generation, integrated on UAV system. The focus will be on developing and integrating sensors of low weight, low cost, high resolution, and can operate under variable conditions such as darkness, snow, and rain. In particular, SUNNY will develop sensors that can generate both (Visible, Near Infrared (NIR)-SWIR) and LWIR images and hyperspectral data and that can use RGB data and radar information to detect, discriminate and track objects of interest inside complex environment, over land and sea. SUNNY also aims to couple sensor processing and preliminary detection results (on-board) with local UAV control, leading to innovative active sensing techniques, replacing low level sensor data communication by a higher abstraction level of information communication.

The exploitation and adaptation of emerging standard wireless technologies and architectures to the SUNNY scenarios towards the EUROSUR’s goal of defining European wide standards. Existing wireless standard technologies, such as IEEE 802.11a/g/n, IEEE 802.11p, DVB-T2, Mobile WiMAX, LTE, and Wi-Fi@700MHz will be considered due to their low cost and features such as radio resource management, provisioning of high bitrates, and mobility management.

Project Results:
The System Architecture was produced providing
1. High level system requirements
2. Definition of the UAS generic constraints and requirements
3. Definition of user interface requirements and baseline system requirements.
4. Functional baseline including system functional design with architecture definition
5. Functional System architecture verification

The Sensor Specification was produced providing
1. Definition of novel passive sensors system requirements specifications
2. Definition of novel active sensors system requirements specifications

The Data Processing specification was produced providing
1. Definition of on-board data system requirements specifications
2. Definition of on-board data system requirements specifications.

The SUNNY Communications specification was produced providing
1. Definition of communication system requirements specification and architecture
2. Communications Interoperability and Protocol

The LWIR and SWIR sensors were developed and tested.

Data analysis proved that the radiometric quality of the airborne Hyperspectral data was good and image sharpness excellent. The image data could be automatically georectified without manual adjustments, proving that the time sync between image and GNSS/IMU was correct. Altogether the entire hyperspectral system proved to work according to the design target specifications.

Airborne miniSAR data was successfully collected in a maritime scenario in the Netherlands, in which different ships and vessel were detected. Acquired data has been used for testing and tuning the processing algorithms, in particular automated detection and generation of 2D images by ISAR techniques. Once the flight test was finalized, the radar system was shipped to CINAV facilities in Portugal for actual installation on the ANTEX platform and relative certification process.

A sensor alignment and fusion module for the SUNNY system has been developed including
• Time based sensor alignment algorithm
• World coordinate based sensor alignment algorithm
• Image super-resolution algorithms

Control/perception algorithms were tested in simulation to develop remote mixed initiative target validation using a team of UAVs. During the course of the work and due to safety and regulation concerning the piloting and manoeuvring of the UAVs without pilot, some of the develop features could not be yet tested in real UAVs. To overcome this issue the sensing network will enable the tracking algorithms using the sensors that allow target tracking e.g. Gimbal, and will also feed the UAVs Ground Control Station with a new set of waypoints to follow a specific target.

An on-board data processing architecture was defined, and a SUNNY on-board Software System Prototype has been developed. The consortium conducted the development and validation of real-time perception algorithms to see and detect targets at sea, using a heterogeneous sensor payload.

Dataset campaigns using piloted and unmanned operational means were also performed to collect synchronized data in order to foster SUNNY Data Processing System developments.

An automated target identification module has been developed providing
• Single-sensor based target identification algorithms
• Multi-sensor based target identification algorithms
• Active learning algorithm for target identification

Decision fusion algorithms have been developed.

The Visualisation and User Interface has been developed. Vessels are positioned according to threat fusion data and AIS data, UAV’s are positioned via a direct data from the sensor network and ground stations.

The solution selected for the air-air datalink was the IEEE 802.11 technology. Broadband radio transceivers from Doodle Labs were bought which operate at 700 MHz band.

Microhard’s DDL OFDM modem at 2.3 GHz was selected as the most appropriate technology for UAV to SBS communications.

An initial Integration Plan was defined in the order to detail which components and subsystems of the SUNNY project should be integrated, and how they are to be assessed.

A detailed demonstration scenario has been defined.

Potential Impact:
SUNNY is an innovative concept for fully integrated development of Optimum Airborne Sensors with Adapted Data Link Solution(s) to Enhance Border Security, driven by real End User requirements and appropriate interoperability and standardizations. SUNNY will address the following key topics:
- Innovative airborne active and passive sensors for civil security applications. Specific constraints for applicability in related scenario will be analysed and taken into account (weather and terrain conditions, physical constraints, affordability, endurance, resolution, mission timing and characteristics, operating procedures).
- Adapted communications and data link, civil frequencies data link and protocols for airborne sensor network for combining large field and focused/tracking surveillance capabilities to be inter-connected and supervised by a ground control centre with large parallel processing capabilities.
- Sensor Fusion and Data exploitation. Novel fusion algorithms for an effective data processing, analysis, fusion and visualization focusing attention on techniques, and human computer interface for fusing and visualization of multi-source sensor data captured in real time for assist in global situation awareness and prompt decision making. Man-in-the-loop learning mechanism will also be addressed for learning from human feedback.
- Cooperative networking. Cooperative mission by optimized sensors network, teams of platforms complementing other assets (space and surface), data transformation into complete information to support decision makers.
- Improving the sensing and perception autonomy of UAVs by innovative on-board processing capabilities using state of the art of parallel processing hardware.
- Appropriate interoperability and Standardization will be addressed.
- Concept Validation through on-ground and in-flight test demonstration (primary site trials I Greece, NAMFI test range Crete, and alternative site in Portugal).
- Assessment of cost-effectiveness of the proposed border monitoring system via cost-benefit analysis.

SUNNY intends to deal with the above mentioned challenges in accordance the layered concept presented in Figure 1.

Figure 1: SUNNY operational view

In the following for each layer we present the key scientific objectives and measurable outcomes:
1. In the Sensing layer the applicable objectives are:
a) Develop optimum airborne sensors for the purpose of maritime/terrestrial surveillance and situation awareness enhancing EU-border security (optics/optronics, radar and 3D) that are suitable for civil security applications. The focus is on developing and integrating sensors of low weight, low cost, high resolution, and can operate under variable conditions such as darkness, snow, and rain. In particular, SUNNY will develop sensors that can generate Near Infrared (NIR) images and hyperspectral data and that can use radar information to detect, discriminate and track objects of interest inside complex environment, over land and sea. The sensors will be integrated on UAV system and tested in flight in relevant conditions.
b) Develop and implement on-board real-time data processing. It will lead to reduced communication requirements and taking in advantage the multi UAV configuration and mobility to perform active sensing. This processing will be performed with current state-of the art low power parallel hardware computation systems and will provide abstract information to be incorporated in the overall system leading to increased situational awareness. The on-board processing system will be integrated on UAV and tested in flight.

2. In the Communication layer the applicable objectives are:
a) Tailor existing datalink technology and to develop adapted data link solutions. It will enable using civilian frequencies and protocols to support the operationally critical communications between UAVs and sensor network ground control centre. A reliable communication system will be integrated to enable the communication between UAVs and between the UAVs and the ground operator. The communication system is validated in the final demonstration.

3. In the System layer the applicable objectives are:
a) Develop and integrate an intelligent airborne heterogeneous sensor network. The focus is on integrating various airborne sensors to form a two-tier intelligent heterogeneous network in order to provide both large field and focused surveillance capabilities. Effective path planning, collision avoidance, and coordination between UAVs will be investigated in order to achieve more effective large field surveillance with low cost and energy consumption. The solution will be tested in flight.
b) Develop and integrate capabilities to enable fast (near real-time) detection and tracking of people, vehicles and ships. The capability based on fast image processing algorithms will enabled the detection and tracking of humans, vehicles and ships in various terrains and from operational relevant distances. The focus will be on novel and advanced algorithms for image segmentation, object recognition and detection. In particular, super-resolution and active learning techniques will be investigated for enhance the image quality and reduce the number of false alarms. The capability will be tested in flight.

4. In the Information layer the applicable objectives are:
a) Develop and implement effective data fusion and focus of attention techniques, and human-computer interface. It is about the fusion of multi-source sensor data. The focus is on development of advanced data fusion algorithms for effectively combining information collected from multiple sensors which could be noisy and contradictory to each other. In particular, different sensory data will have different resolutions. Novel algorithms are to be developed to align the data of different spectral to reconstruct high resolution images for all spectral. These algorithms will be validated with datasets and in final demonstration.
b) Develop and implement effective human-machine interface. The focus is to assist in global situation awareness and prompt decision making. The developed human computer interface will also enable efficient human feedback so that the intelligent system can learn from the feedback for improving target/intrusion. This system will be validated in simulation and final demonstration.
c) Check the compatibility of the SUNNY system concept with the EUROSUR interoperability recommendations

SUNNY will demonstrate the objectives mentioned above across two levels:
1) By conducting System Demonstration which will allow the testing and validation of the proposed system. Real-life use case scenarios are foreseen for each of the project objectives. These use case scenarios are defined by the end-user partners of the consortium. Two main scenarios are considered for demonstrating the integrated results:
a. A maritime scenario for detecting border crossings at sea. This will be achieved in collaboration with sea border control end user partners, who can deliver the required testing facilities.
b. A ground scenario for detecting border crossings at land. This will be achieved in collaboration with land border control end user partners, who can deliver the required testing facilities.

2) By System simulation, aimed to support the development and check the robustness of the innovations under controlled conditions to reduce the risk associated to the conduct on flight testing and to enable the extrapolation of the real life testing to a wider set of conditions than those covered by the in-flight system demonstrations.

Beyond the technical objectives defined above, the SUNNY project will
1) Ensure that the proposed solution is compliant with all privacy, security and ethical aspects as well as European societal values and citizen’s right.
As the proposed SUNNY solution is heavily geared towards surveillance activities, it is of vital importance that the technology does not infringe upon citizen’s rights. For this purpose, the project will put great emphasis on evaluating and monitoring legal and ethical aspects on privacy and societal values, leading to the potential acceptability of the system.
2) Promote the solution for the adoption by civil agencies in order to ensure the pan-European uptake of the SUNNY system.
For the development of the optimum sensors and data links (including data fusion for enhanced situational awareness), the project will also put special emphasis on critical factors which will determine the up-take and adoption of the solution by civil agencies. These factors include miniaturisation, affordability, endurance, special adherence to the timing and characteristics of mission and operating procedures. Furthermore, the project will aim to promote the proposed technology to playing a pivotal role in the contribution to law enforcement and immigration control. This will also reinforce the up-take of the proposed solution.

List of Websites:

Related information

Documents and Publications

Reported by

United Kingdom
Follow us on: RSS Facebook Twitter YouTube Managed by the EU Publications Office Top