Skip to main content

Snapshot spEctral imagEr for cost effective IR Surveillance

Periodic Reporting for period 3 - SEERS (Snapshot spEctral imagEr for cost effective IR Surveillance)

Reporting period: 2017-02-01 to 2018-01-31

SEERS project has developed a highly competitive system, in terms of cost, performance, and adaptability, for multispectral sensing and intelligent surveillance of a wide variety of critical infrastructures. These features enable access to a large portion of the safety and security market, since critical infrastructure surveillance represents a high volume market, due to its worldwide demand and to the large scale of the corresponding surveillance deployments.
The overall objective of SEERS project was to develop a compact snapshot multispectral imaging system in the range 0.7-14 µm wavelength based on three uncooled infrared monolithic technologies compatible with Si-CMOS (PbSe, CMOS-based microbolometers, CMOS). Firstly, a snapshot multispectral imager in the IR domain has been developed. Secondly, an embedded approach has been adopted for image reconstruction, cognitive image fusion, video pre-processing and event-driven operation. Thirdly, the benefits of a novel video analytics solution (VAS) for smart networked operation have been demonstrated in terms of performance and persistence, thanks to multispectral imaging. SEERS approach enables robust intelligent surveillance with event-driven and smart performance. Remarkable capabilities are robustness to variable visibility conditions (e.g. fog, rain, fumes), gas discrimination, fire and burst imaging with accurate temperature measurement independent on emissivity. Demonstration of the system in real operational conditions for three different scenarios have been carried out: firefighting, road traffic surveillance and coastal surveillance.
SEERS has developed a modular, compact and cost-effective snapshot spectral imaging system for the infrared domain (0.7-14um). A multi-sensor approach based on a multi-aperture (MA) optical arrangement has been adopted, including embedded processing for cognitive image fusion and image-based measurement at video-rate. A multispectral video analytics for the IR has been developed and integrated in a real video management software framework, demonstrating the SEERS system capabilities in three operational scenarios relative to safety and security. The SEERS consortium expects to launch the project results on the security and safety market, and specifically the critical infrastructure protection sector. Five key exploitable results have been identified:
KER1: VIS/NIR/LWIR Video Surveillance System. The proposed results is a cost effective multi-spectral (VIS/NIR/LWIR) video surveillance system for traffic surveillance and infrastructure perimeter protection, composed of a snapshot multispectral camera and a video analytics suite, unique in the market, which exploits and combines information from all available bands for enhanced analytics performance.
KER2: VIS/NIR/SWIR Video Surveillance System. SEERS VIS/NIR/SWIR camera offers a highly persistent vision solution for aerial maritime surveillance at a competitive cost compared to high-end VIS-SWIR gimbals.
KER 3: High-speed MWIR multispectral camera. The proposed result is an uncooled multi-spectral MWIR camera, designed with variable number of apertures depending on the application needs, allowing super resolution or multispectral imaging. An MWIR prototype with three apertures, was demonstrated for measuring temperature independent on emissivity in a wide range [100-1000]º C with an error below 3.5%.
KER 4: Modular IR building blocks. The SEERS camera is based on a set of modular units that can be combined in many ways. In this KER, instead of positioning in the market a preconfigured camera, we provide a configuration service to customers. The multi-sensor technology as developed in the SEERS project is applied to the requirements of each customer. This would result in low-volume sales, but due to the built-to-spec nature of each camera, at a relatively high cost.
KER 5: Multispectral Video Analytics System. The SEERS Multispectral Video Analytics System (MS VAS) is capable of exploiting and combining information from multiple spectral bands with overlapping FOV, resulting in enhanced performance compared to single-sensor analytics. The solution is versatile and accepts different sets of input spectral bands, as required by each particular surveillance scenario.

Dissemination has been focused on presentation of project results in different fairs and events, personal contact with potential stakeholders and other EU projects and initiatives. Other activities include demonstration of the system in a public seminar, dissemination through the project webpage and social networks and several press releases. Besides, strong effort on scientific dissemination has been carried out with eleven scientific publications.
SEERS has brought into play expertise and state-of-the-art approaches borrowed from different disciplines, namely sensor technology, multi-aperture optics and image reconstruction techniques, embedded vision and image analysis, advanced video analytics for surveillance, and integration of surveillance solutions for safety and civil security. The novel features of the system are:
- Multisensor and uncooled because SEERS integrates by the first time the three unique uncooled monolithic SiCMOS compatible infrared technologies covering three IR windows (NIR, LWIR and MWIR). Low manufacturing cost of CMOS compatible sensors, enabling technology to develop new applications not previously possible and fast sensor evolution.
- A snapshot spectral and super-resolution imager, as a result of combining a multi-aperture arrangement and computational imaging;
- A modular optoelectronic arrangement that allow to obtain different combinations of bands in a straightforward manner and at a cost scaled to the number of bands. Agile spectral reconfiguration and optimization;
- Smart, since it features embedded processing of acquired images for reconstructing the multispectral stack, extracting measurements, fusing images,pre-processing video, and sending data and alarms to the Video Management Software (VMS); -Versatile and compliant with realistic requirements that a real surveillance scenario for safety and civil security demands. The system features an advanced video analytics system suitable to be integrated in a real VMS platforms.

SEERS has also an impact on an important environmental and social aspect: the increase on the security and safety of European large infrastructures, with strong implications on the security and safety of population and environments. In the case of the traffic scenario, regional and European administrations are highly concerned with the security and safety of public transportation infrastructures. Coastal surveillance, on its part, covers a wide range of security and safety threats, each one having a great social impact, among them, smuggling and illegal migrant detection. Another great concern of the European society that is addressed in SEERS is the great environmental impact that safety weaknesses of critical infrastructures potentially present. Accidents in infrastructures such as oil and gas refineries or nuclear plants may have catastrophic environmental consequences that greatly worry citizens and administrations. SEERS provides a useful tool for the display of an early response in case of accidents, thus limiting the environmental consequences in such situations.