Final Report Summary - CONSORTIS (Concealed Objects Stand-Off Real-Time Imaging for Security)
Executive Summary:
This project developed a prototype real-time dual-mode submillimetre-wave (SMMW, a.k.a. “THz”) security screening system for use at airports. This system is capable of high-throughput, improved security screening of people and is less restrictive for people using it than existing systems. The goal was to achieve this through the following overall objectives:
• Non-intrusiveness, safety and accuracy – SMMW systems are non-ionizing.
• Speed with near-real time video imaging (10 frames/s) for “walk-by” security.
• SMMW (“THz”) operation with passive sensors at about 250 and 500 GHz and radar centered at 340 GHz.
• Multi-frequency operation – the passive system with two frequency bands.
• Automated threat recognition capability.
• Privacy issues addressed with strong involvement of ethics experts.
• Pilot/demonstration – The system to be tested and validated against realistic threats in a controlled environment.
CONSORTIS was a 48-month project which adopted a systems engineering approach. The end-user requirements were first determined with the aid of external experts through interviews and advisors and translated into technology requirements. In parallel, ethical aspects were evaluated for deployment scenarios and provided ethical inputs to the end-user requirements, to help ensure societal acceptability of the solution.
The preliminary design of the system was produced and reviewed, followed by hardware manufacturing, sub-systems characterisation and testing, and system integration and testing. The Automatic Anomaly Detection (AAD) research focused on the development of algorithms. In the field trial the system was tested and validated in a laboratory at VTT against realistic threats, collecting over 1000 data sets relevant to aviation security with typical walk-through times of some seconds. An airport demonstration at Helsinki was defined and planned, but changed into a laboratory technological demonstration to allow some additional modifications delayed by resourcing challenges to be completed. The technological achievements include:
• World first 16-channel 340GHz radar imaging at 10Hz video frame rate
• World first 8000-element passive sub-millimetre wave security imaging array based on Kinetic Inductance Detectors (KIDs)
The major impact of the project is the hardware which offers a unique capability in Europe to collect security data at SMMW frequencies, not only in aviation but in the security of public crowded spaces and high value infrastructure. The hardware still needs further development before it becomes market ready, however, both the passive and active sensors represent a quantum step in performance. The concept is novel and has been protected by a system-level patent, as is the readout of the KIDs array. A number of peer-reviewed journal papers have been published, also the volume of dissemination activity has been considerable with many conference and workshop papers at international symposia. Significant progress in the sub-systems, particularly the radar components, has been extensively disseminated via scientific presentations and papers, together with briefings for academic and government stakeholders. In total 63 dissemination events, including presentations and briefings to stakeholders, have been performed during the project, with a further four planned beyond the end of the project, together with seven additional events used to engage with potential exploitation customers and partners and to gather market intelligence.
Project Context and Objectives:
This project developed a prototype real-time dual-mode submillimetre-wave (SMMW, a.k.a. “THz”) security screening system for use at airports. This system is capable of high-throughput, improved security screening of people and is less restrictive for people using it than existing systems. The goal was to achieve this through the combination of a multi-band passive SMMW imager operating at 250 and 500 GHz, integrated with a 340 GHz 3D imaging radar.
The combination of an active imaging radar system with a dual-band passive imaging radiometer was chosen to exploit the complementary image features revealed by the two modalities which in turn gives rise to improved security. Passive systems tend to be more mature in terms of imaging wide fields-of-view (FoV) at near-video frame rates but suffer from limited dynamic range, given the weak signatures associated with passive SMMW imaging, especially in situations where there are substantial clothing material losses present. Active systems, however, especially when operated in radar mode, can achieve very high dynamic ranges capable of overcoming the effects of clothing losses, but tend to only cover limited FoV at near-video frame rates. This is due to the technical challenges associated with building large arrays of SMMW transceivers. One of the challenges of this project was to bring these disparate sensors together in a common enclosure as part of the same system.
These overall objectives of the CONSORTIS project address the topics stated in the call as follows:
• Non-intrusiveness, safety and accuracy – SMMW systems are non-ionizing
• Speed – given that the system will be capable of near-real time imaging (10Hz) it is applicable for “walk-by” security screening solutions for subjects walking at normal pace
• SMMW (“THz”) operation – the system will incorporate passive sensors covering frequencies at about 250 and 500 GHz, while the active radar is centred at 340 GHz
• Multi-frequency operation – the passive system will cover two frequency bands for materials differentiation capability
• Automated threat recognition capability – this was addressed by Automated Anomaly Detection (AAD) algorithm development to make use of the complementary image outputs of the passive and active sensors
• Privacy issues – Strong involvement of ethics experts and the incorporation of the AAD capability minimises the potential issues with privacy
• Pilot/demonstration – The system was planned to be tested and validated against realistic threats in a controlled environment and also to be validated in a real (airport) environment.
Description of the Work Performed and the Main Results Achieved
CONSORTIS was a 48-month project which adopted a systems engineering approach. Work Package 2 (WP2) ‘Solution Specification and Validation’ identified end-user requirements with the aid of external experts through interviews and advisors (the Exploitation, Industrialisation and Ethics Advisory Board) and translated them into technology requirements (system requirements) appropriate for implementation by the technology teams. WP3 ‘Ethical Aspects Advisory’ ran in parallel to WP2, evaluating deployment scenarios, providing ethical inputs to the end-user requirements and providing guidance on societal issues to inform the design and testing of the system throughout the project, to help ensure societal acceptability of the solution.
WP4 ‘Automatic Anomaly Detection’ focused on the development of the AAD algorithms, guided by the WP2 system requirements, for implementation within WP7 ‘Field Trial’. The preliminary design of the system was produced in WP5 ‘Camera Design’, and review of the final designs was followed by hardware manufacturing, sub-systems characterisation and testing, and system integration and testing in WP6 ‘Camera Manufacture, Characterisation and Test’.
In WP7 ‘Field Trial’ the final system was tested and validated in a laboratory at VTT against realistic threats as defined by end users and test procedures developed within the project, based on best practise. A demonstration at Helsinki Airport was defined and planned in WP8 ‘Airport Demonstration’ to allow testing and validation in a realistic environment as defined by the end users. The airport demonstration was postponed until after the project to allow some additional necessary technology integration measures delayed by resourcing challenges to be completed.
In parallel with the whole project, which has been managed by WP1 ‘Management’, WP9 ‘Dissemination and Exploitation’ ensured the protection of foreground IPR, the dissemination of the project results through publications and conferences etc., and developed exploitation roadmaps for the sub-system technologies and the overall system to provide a route to market.
The CONSORTIS project has made several remarkable achievements in applying SMMW technology to aviation security. These include:
• World’s first 16-channel 340 GHz radar imaging at 10 Hz frame rate based on homodyne radar transceivers
• World’s first 8000-element passive sub-millimetre wave security scanner focal plane array based on Kinetic Inductance Detectors (KIDs)
• Multi-mode active and passive sensors working in the same enclosure
• Real time sub-millimetre wave imaging of passengers in a walkthrough mode.
Bringing the Radar and Radiometer systems together in a common enclosure for screening people is unique and a major step forward, and promises to produce ground-breaking performance in terms of security.
The CONSORTIS system was used in data collection mode to collect over 1000 data sets relevant to aviation security.
Potential Impact and Use
The market context for CONSORTIS has changed during the duration of the project. The aviation need to protect aircraft from IEDs carried by passengers or in-hold baggage continues. However, the need for improved affordable systems for the protection of urban crowded places and critical infrastructures has grown significantly, as highlighted by attacks in Manchester (2017), Copenhagen and Paris (2015), and other attacks outside the EU in crowded areas such as the Boston marathon and a Nairobi shopping mall (2013). EUROPOL has stated that the overall threat to EU security is high, with the level of activity in the EU attributed to jihadist terrorism continuing to rise.
The major impact of the project is the hardware which offers a unique capability in Europe to collect security data at SMMW frequencies. This will inform future system procurements and determine if SMMW technology offers advantages not only in aviation security but also other security applications such as public crowded spaces and high value infrastructure. The hardware as it stands still needs further development before it becomes market ready. However it represents much more than a step towards a new aviation security system as individually both the passive and active sensors represent a quantum step in performance. These sensors could in turn be applied in other market segments such as remote sensing or surveillance, providing them with a new capability. The walk-through concept for security screening using active and passive sensors is novel and has been protected by a system-level patent, as is the readout of the KIDs array. In conjunction to these patents there have been a number of peer-reviewed papers which capture the expertise of the consortium. These papers cover the KIDs array and the optics of both the passive and active systems, which are particularly challenging as the passenger is very close to the system requiring well-corrected optics over a wide field of view.
The volume of dissemination activity has been considerable leading to many conference and workshop papers at international symposia. Significant progress in the sub-systems, particularly the radar components, has been extensively disseminated via scientific presentations and papers, together with briefings for academic and government stakeholders. Sixty-three dissemination events, including presentations and briefings to stakeholders, have been performed during the project, with a further four planned beyond the end of the project to engage with potential exploitation customers and partners and to gather market intelligence.
Two Technical Demonstrations were held, primarily targeted at aviation technical experts. The first was held at VTT on 14th December 2017, involving customer technical experts as attendees. A further demonstration to show the performance of the system to the European Commission Project Officer and Reviewer was also held at VTT on 6th February 2018. The results are sufficiently promising that partners have agreed to collaborate to take the CONSORTIS concept/technologies forward for both aviation and non-aviation security applications.
Project Results:
4.1.3.1 General
CONSORTIS was proposed for the FP7 call SEC-2012.3.4-5 ‘Further research and pilot implementation of Terahertz detection techniques (T-ray)’ issued in August 2011. This call required:
• Frequencies >300 GHz and <1 THz
• Prototype system with single or multiple frequencies
• Safe and respecting privacy
• Demonstrator/pilot test/validation in a live operational environment e.g. airport
• More effective security
It was aimed at next generation aviation security systems that would increase passenger freedom by reducing divesting, queuing and the number of constraints on passengers. Furthermore it should satisfy the current ethical requirements.
To satisfy this call the CONSORTIS project team elected to design and fabricate a walkthrough aviation security system. The selection of walkthrough provided greater freedom for passengers with fewer constraints. More effective security was planned to be achieved by higher probability of detection, lower false alarm rates and being able to detect the unexpected. This goal was to be achieved by combining a dual band sub millimetre wave passive system with sub millimetre wave radar capable of measuring 3D shape. This combination increases probability of detection because it combines sensors which use different methods to form an image and this combination also reduces false alarms. For both sensors the specification was such that the performance would be a step change over in-service equipment.
Such a system has some top level challenges that have to be overcome to be successful. They include:
1. Be simple to use
2. Aviation security has demanding fields of view as passengers are close to the system ~1 m requires 60° vertical and 27° horizontal just to cover a person
3. Combining the passive and radar sub-systems in one enclosure must not impede the performance of either sensor
4. Combing real time imagery from three submillimetre wave sensors (two passive and one active) to generate improved security has not been done before
5. Ethical best practise.
A systems engineering approach was adopted and a requirements capture was conducted, against this a systems requirement document was developed. The sensors are accommodated in a tower and they track and scan the person as they walk past it. In the final system two towers will be required to see both sides of the person but funding was only available to demonstrate a single tower in this project.
This planned full system would consist of a 340 GHz radar, a 250 and 500 GHz radiometer, an automatic anomaly detection (AAD) system, a command module and a passenger control system. The specification for each of these sub-systems flowed from the systems requirement document.
The ethical requirements were described and fed into the requirements capture. The ethics work also had a wider remit looking at the ‘fair process effect’ in systems like body scanners and addressing the future in a world where security systems will be connected by the Internet and how ethics needs to be considered.
State of the art security systems analyze the image data generated by the system using algorithms on a computer to indicate the presence of a threat to the operator. In CONSORTIS this function was to be fulfilled by the Automatic Anomaly Detection system (AAD) and with three sub-millimetre wave sensors this is clearly a key function. The development of the AAD is described below in chapter 3.
The design of the CONSORTIS hardware is described in chapter 4 as the requirement driven specification was iterated with what could be achieved in hardware and some prototyping was done. In this work package the architecture of Command Module which takes command of the whole system and the Passenger Control System which controls the flow of passengers were also defined. Some prototyping was also done. The Preliminary Design Review (PDR) and Critical Design Review (CDR) were conducted.
The build of the hardware and its integration is described in chapter 4, this includes all the sensors, the enclosures, passenger control system and command module.
CONSORTIS was tested in the laboratory at VTT with performance tests taking place with airport staff participation and this work is described in chapter 5. Testing was conducted against dummy weapons, explosive simulants and real explosives with the help of the Finnish police.
Preparation for the airport trials were made and this work is described in chapter 5, however due to severe time constraints at the end of the project, as well as the inavailability of finished radiometer and AAD, it was decided not to conduct these trials at Helsinki-Vantaa airport. However, as much as possible of the methodology and operational knowhow were included in the field trial.
Instead of the originally planned demonstration at Helsinki-Vantaa airport, a technical demonstration was held at VTT on 14th December 2017, involving user community technical experts as attendees (chapter 6).
Dissemination and Exploitation are described in a subsequent section of this report which brings together all the publications and events and the plan for taking CONSORTIS to the market.
4.1.3.2 Requirement Capture
This activity was responsible for capturing the needs of the airport users for a CONSORTIS solution (and the user-needs from adjacent markets where possible) and for validating these needs throughout the project. The aim was to guide the project towards an exploitable solution that, as far as practicable, met user requirements and aspirations. .
The project adopted a Systems Engineering approach to the design in which three major activities were undertaken. They were:
a. Definition of what users want, stated in their own operational terms (The User Requirements Document or URD)
b. A technical description of what is needed to meet the URD was developed in a Systems Requirement Document (SRD)
c. The design solution was produced, using real hardware and software, that met the technical requirement (Candidate Architecture(s))
d. Throughout the project these documents were used to validate sub-systems and the final hardware to ensure that the user requirements had been met.
Visits were made to three regulators, five airport operators, three industry experts, one equipment operator and (indirectly) three bodies representing the views of the general public. Structured interviews were conducted with a balanced set of stakeholders which included Europe’s largest user of body scanners plus an important airport trade association.
It was necessary to revisit the contents of URD in the light of what is technically possible and to implement trade-offs. This iteration was also part of the work involving all consortium members, under the leadership of the Technical Manager, as part of the design reviews process led from WP2.
Generating the Concept of Operations (CONOPS) for meeting all the requirements was very demanding and over 20 alternative approaches were considered and discussed with potential users. The airports strongly preferred an unencumbered walk through approach. This solution comprised CONSORTIS system to be positioned either side of a transit path along which the passenger walked with arms raised, to allow images from under the arm to be visible.
Comparison of the technological solutions and the user needs was also very challenging and it was necessary to iterate around the systems engineering cycle three times. Key outputs from this work were:
• 48 specific user requirements.
• The definition, review and agreement of the CONOPS for the equipment, heavily influenced by the needs of the airport operators for a simple walk-through system.
• Definition of the trade-off space balancing detection probability, false alarms and throughputs which, together with cost, will be critical in positioning the CONSORTIS system in the market.
Once the architectures had been defined and performance modelled, this output was used to generate detailed equipment design specifications.
There was also the ongoing task of monitoring operational airport developments and technical developments to identify and implement any further changes that may be needed during the life of the project. To this end, good communication with stakeholders was developed and maintained by this task. The project held meetings with the European Commission (principally through the offices of DG Move), airport operators, including their trade association, ACI (Airport Council International), and public bodies through the European Association of Airport and Seaport Police (a member of the EIEAB).
4.1.3.3 Automatic Anomaly Detection
The objectives of Automatic Anomaly Detection (AAD) research were to develop and implement image processing and data fusion methods for privacy preserving detection of anomalies in multi spectral scanner data.
The pre-existing data from previous systems was identified and collected. The collected data was analysed and reported including a description and analysis of the example datasets together with an indicative evaluation of what sensor specifications were required by the AAD. It was concluded that data to initiate AAD for radiometer was already available. A shortage was identified on data to support the development of AAD algorithms for radar and for developing fusion algorithms.
In the development of AAD method, four main activities were undertaken: 1) initial development of AAD methods for radiometer and radar data, 2) synthetic data generation, 3) design of a strategy for using results from the two sensor modalities and 4) the design of a support sensor system.
Initial Development of AAD in Radiometer Data
A literature study was undertaken trying to establish the current state-of-art in anomaly detection in passive mm-wave imagery. From this study a set of promising image processing strategies was identified, implemented, and evaluated on existing data. Auxiliary algorithms needed for e.g. foreground detection (=subject to be scanned) and silhouette outlining was developed and evaluated. It was concluded that a candidate set of algorithms was now available and that final development and evaluation could continue as soon as final data was available having the characteristics and properties of the final CONSORTIS passive imager.
Simulation Tool Chain
To mitigate the absence of relevant radar data before the completion of the radar, efforts were made to generate synthetic radar data. A complete tool chain was established spanning from character animations, cloth simulation and electromagnetic field propagation and simulation. From simulated data, studies were made to assure that CONSORTIS system requirements were appropriately set to yield data with e.g. sufficient resolution performance. Synthetic data was also used for initial development of AAD algorithms in radar data.
Initial Development of AAD in Radar Data
Using synthetic data and, later on, data from the ‘pathFinder’ radar, a single channel device built to establish feasibility, initial algorithm development was undertaken for AAD on radar data. Data was explored for its basic characteristics and tools and algorithms were developed for data filtering. Filters were constructed to find the ‘first’ and ‘last’ surface in data (detect outermost layer of clothing vs body’s skin), to find sharp edges in surface geometries, to find strong curvatures etc. An algorithm was also developed to compensate for motion artefacts in data due to subject movements during data capture.
Sensor Fusion and Support Sensor
Strategies on how to fuse results from the two modalities were outlined. It was found that the most robust and flexible solution for sensor fusion is to run separate AAD’s on each sensor (radar and radiometer), and fuse the results in a post-processing step. It was identified that a support sensor is needed for the sensor fusion, and for other reasons also which include to analyse the scene and to align data from both sensors. A support sensor system was designed and a prototype was developed and tested. The support sensor system detects the position and poses of the subject and provides all information necessary for data alignment.
AAD Implementation and Evaluation
The original objective was to make a final implementation and parametrization of the AAD prototype algorithms, using data from camera laboratory tests and system performance field trial. A re-scope of this task was necessary to mitigate the early enough lack of data from the CONSORTIS sub-system sensors. The main achievements of this task are described below.
Implementation of AAD in Radiometer Data
Based on the data from the commercial Argon passive SMMW imaging system of AQA, a final pick was made on which algorithm to deploy in the CONSORTIS prototype as soon as data would be available. The algorithm code was migrated to execute on the CONSORTIS platform and within the AAD sub-system component. It was tested that the algorithm could process multiband passive imagery at 10 Hz video rate.
Implementation of AAD in Radar Data
In close collaboration with USTAN and TU Delft, progress was made on the radar based signal processing. A number of small measurements campaigns were conducted by USTAN using the pathFinder system, and later on the radar sub-system component, to support the algorithm development. Major work was undertaken to mathematically describe the optical system of the radar and to find a method to conduct image formation that is geometrically correct.
AAD Integration
A software environment was designed and implemented to embed the core AAD-algorithms as soon as they were available. This software environment included the functionality needed for integration with the CONSORTIS software architecture and to receive and buffer data from the sensors. The environment also included functionality to distribute calculations and perform multi-threaded (i.e. parallel) processing of data frames from the radar and radiometer. Using ‘dummies’ as core AAD-algorithms, the software environment was fully integrated with the CONSORTIS software environment and successfully stress-tested against heavy data loads.
AAD Post-Trial Evaluation
After the system performance trial was completed, a large effort was made in a short timescale to analyse the collected data and to compile a tool-chain to process the data for the best possible use in the system performance analysis.
4.1.3.4 Camera Design and Manufacture
The camera design work was conducted in two phases: preliminary design culminating in the Preliminary Design Review (PDR) which was completed in October 2014 and detailed or critical design culminating in the Critical Design Review (CDR) which took place in April 2015.
4.1.3.4.1 Imaging radar
Sub-mm Wave Transceiver
The 16 sub-mmw transceivers were supplied by GOTMIC (x8 MMIC frequency multipliers) and WMW (170 GHz doublers and 340GHz doubler-mixers) to USTAN for incorporation into the radar subsystem. All units were successfully delivered by November 2016 and met specification. For this operating frequency, the flatness per unit and the spread across units is extremely good.
Active Front End
In active front end manufacture and characterisation the work was concerned with the chirp generator and 16-channel receiver/demodulator built by USTAN. The chirp generator was successfully manufactured, optimised and met specification. A smooth-walled feedhorn was designed and manufactured. All 16 feedhorns were measured and achieved specification with state-of-the-art performance. The 16-channel receiver/demodulator was designed and manufactured and met specification with close channel-to-channel balance.
Optics and Optical System Mechanical Manufacturing
Camera optics and opto-mechanics manufacture and characterisation concerned the manufacture of the optics for the radar and radiometer sensors and their characterisation to confirm that the optical performance meets specifications. The radar optics and opto-mechanics were manufactured by VTT and delivered to USTAN in May 2016. The optics comprises five large reflectors and a mini-mirror array mounted to the transceiver horn assembly. The sequence starting at the transceivers is: mini-mirror array, Dragonian secondary (fixed, curved mirror, oval shape), Dragonian primary (reciprocating, curved mirror, oval shape), two counter-rotating circular plane mirrors (forming the ‘double-disc scanner’), and a large ‘pan’ mirror steerable in the azimuth axis (flat, oval shape).
The key characteristics of these elements were:
Mini-mirror array: 16 mirrors alternating between +45° and -45° to direct feedhorn beams from left and right sides into a single line array which was machined from a single piece of aluminium with thin walls between mirrors to avoid spillover between beams.
Dragonian mirrors: doubly-curved focusing surfaces in aluminium with a required profile accuracy of λ/20 ≈ 40 µm. The primary mirror should be lightweight and stiff as it is driven in a reciprocating motion for the elevation scan.
Double-disc mirrors: circular plane mirrors whose surfaces are tilted off-axis and which counter rotate. Rotation rate of 2700 rpm is required for 10 Hz imaging frame rate. Light weighted and mass-balanced body framework was manufactured with a plane aluminium reflecting surface cemented on top.
Pan mirror: aluminium body framework and cemented aluminium sheet as a reflector surface, driven by a stepper motor.
The optical elements are mounted on a sub-chassis assembled from commercially available aluminium extrusion profiles built around a thick solid aluminium base plate which supports the majority of reflectors, particularly the rotating double-discs with their drive mechanism. FEM simulations were carried out for the whole radar assembly to ensure that any (resonant) vibrations generated by the double-disc scanner are at frequencies much lower than the natural frequency of the entire radar assembly.
The double-disc scanner, a form of Lissajous scanner, scans a long thin figure-of-eight pattern in azimuth, approximating a linear scan. Using rotating elements makes it feasible to achieve the required 11 ms horizontal scan time. A single 230 Vac motor drives a shaft from which the precise and synchronised counter-rotation of the two discs is achieved using belts and pulleys. The scanner operates successfully at the rated speed of 2700 rpm (45 Hz) with minimal vibrations but the system does generate quite a lot of noise.
During beam profiling it was discovered that the optics were not focusing correctly and this was traced to the Dragonian mirrors being distorted in shape. Whilst they had been milled from single pieces of aluminium to high precision they had deformed after tooling. They had to be completely redesigned and remanufactured paying special attention to minimise shape deformation after machining. This meant using a more stable grade of aluminium, machining in several stages and special mounting methods. Moreover, the backside of the reflectors was designed to have a deep spider’s web support structure, which greatly improved the rigidity from the original design. Accurate inspection on a coordinate measuring machine (CMM) verified that the reflectors met the shape specification. Subsequent beam measurements confirmed that correct focusing was achieved – see Radar Subsystem Characterisation and Testing section below.
Radar Back End
Back-End manufacture and characterisation concerned the back-end electronics for both the radar and radiometer. For the radar this comprised COTS data acquisition cards (16 channels, 25 MS/s), a desktop PC and high speed, multi-threaded parallel-processing code written in C. The underpinning code was based on a successful radar processing architecture developed during a previous USTAN radar imaging project which was modified for the CONSORTIS data acquisition cards and radar parameters. Various engineering versions of the code were also written for low level control and analysis during the radar manufacture. The radar software also performs all the control functions for the various motors in the system, control of the chirp generator, and the BIT/BITE functions. Raw intensity imagery is displayed by the software to aid testing and characterisation. Ultimately raw data will not be displayed as all image analysis would be handled by the AAD to ensure privacy.
Radar Subsystem Integration
The integration of all the radar components and modules was undertaken by USTAN and encompassed many tasks.
Radar Subsystem Characterisation and Testing
Key parameters which were confirmed in characterisation and testing were the frame rate (10 Hz), field of view (1 x 1 m2), depth of field (1 m), beam spot size and range bin width, all of which are critical to meeting the imaging requirements.
Radar Subsystem Compliance with Original Objectives
Two aspects of the radar subsystem’s performance fell short of the original objectives.
1) The field of view was limited to 1 x 1 m2 which is less than required to cover a whole person (the aim was 1 x 2 m2). This was due to reaching a compromise in the design of the optics
2) The frame rate achieved during data collection had to be limited to 7 Hz (instead of 10 Hz) due to a timing latency issue, straightforward to be corrected in the future. Both the optomechanics and signal processing software were proven to be capable of running at 10 Hz.
Despite these two shortcomings the radar was able to collect significant quantities of data during the data collection trial. All the other key parameters of the radar’s performance met the original objectives, as described in the following:
• Design of a radar chirp generator by USTAN (with output in the microwave range) which achieves very high linearity, low phase noise, short duration chirps over large fractional bandwidths (>10%).
• Design of an advanced five-mirror quasi-optical system for the radar by TU Delft capable of imaging at up to 10 Hz frame rates over a wide 1 x 1 m2 field of view at close range (~2.5 m) with low aberration and scan loss. This comprises a Dragonian mirror pair for focussing, a high speed double-disc scanner combines with the reciprocating Dragonian primary mirror to achieve azimuth-elevation beam scanning, and a pan mirror to steer the radar field of view to follow the subject.
• Design of novel self-mixing multiplier radar transceivers which combine MMIC and Schottky diode technology from GOTMIC and WMW. They have shown state-of-the-art simulation results, achieving wideband operation with excellent power flatness and low signal distortion. These units remove the need for separate duplexing components to separate transmit from receive and hence enabled a focal plane array of 16 radar channels – believed to be the world’s biggest submillimetre wave radar array.
• De-risking the radar design by building a single transceiver, mechanically scanned, lens focused, 220 GHz ‘pathFinder’ radar. This proved the chirp generator architecture and the radar transceiver design methodology. Additionally, it yielded high fidelity radar data on mannequins and test targets with 1cm3 voxel resolution which was valuable for the AAD development.
• Use of a Microsoft Kinect v2 as the Support Sensor by FOI to provide accurate, high resolution skeleton tracking and pose estimation of the subject to assist passenger control, panning of the primary sensors and data processing.
• Design of a Passenger Control System (PCS) by RSN and VTT which meets the user requirements of airport operation and controls the passenger flow suitably for reliable imaging of each person. The PCS makes use of COTS components.
• Design of a Command Module (CM) by VTT and FOI which coordinates all the subsystems in the system ensuring reliable high speed data interchange and separate low speed control information exchange. The CM uses COTS components.
• Mechanical design of all system and subsystem hardware using 3D CAD modelling software by VTT and USTAN to ensure correct part manufacture and successful system integration. This included finite element analysis of the high speed moving parts within the radar scanning mechanism which was performed to support the mechanical design, ensuring balanced, vibration free motion and reliable operation.
4.1.3.4.2 Imaging radiometer
Passive Front End
The passive front-end manufacture and characterisation concerns the build of the passive focal plane array (FPA), cryostat and read-out electronics being developed by VTT and AQA.
The core of the imaging system, the focal plane with 8208 detectors was fabricated at VTT’s clean room facility using multi-layer superconducting fabrication line integrated with a deep etching process to realize the nanomembrane structure needed for thermal optimization. The requirement of the dual-band operation generated the need to develop a novel fabrication technique based on stacked wafers to realize the designed quasi-optical coupling. For this end, fabrication test runs prior to actual detector fabrication were executed. .
The detectors were integrated into Gifford-McMahon type cold head and compressor integrated with the custom designed and manufactured vacuum system with quasi-optical feedthroughs. As the system is relying on a large 2D focal plane, the filtering scheme intended to reject ambient radiative thermal loading was designed and integrated into the system. For the RF-based readout, readout-band (40 - 400 MHz) wiring was designed and manufactured, based on flexible RF stripline connections.
The detector readout electronics was manufactured and integrated with the system. In addition to the basic readout concepts, circuitry for temperature stabilization was implemented.
Radiometer Optics and Optical System Mechanical Manufacturing
The lenses for the radiometer optical system were manufactured and thoroughly characterised. The lenses were made of material, cyclic olefin copolymer, with exceptionally low loss at sub-millimetre waves. The trade name of the material is TOPAS™. The material is well suited for CNC machining and the RMS error of the manufactured surface was 11-37 μm – almost according to specification. The symmetrical inaccuracies were compensated with re-optimizing the lens positions for all focusing distances. Performance was very close to design predictions, differing only slightly due to mechanical tolerances in fabrication. The pan/dither mirror was successfully manufactured.
The framework of passive system was constructed utilizing aluminium profiles where different subassemblies are mounted with adapters taking care of the correct positioning relative to each other. Some of the subassemblies like lens moving assembly were delivered to another location before the complete system was ready to be assembled due to delays and project schedule. The final assembly of the passive subsystem was performed in VTT’s Espoo site.
Back End
The back end of the system consists of the analog-to-digital converters, microprosessor elelctronics with firmware realising the readout sequences, and PC software for image high-level data processing and image formation.
Radiometer Subsystem Characterization and Testing
Various components of the radiometer were separately tested to verify functionality. This includes readout-band characterisation indicating the basic functionality and good yield (typically >97% within the readout channels), the video-band response of the detectors, and point-spread functions indicating the functionality of the system optics, as well as detector quasioptics, as integrated into the radiometer subsystem.
The electronics-firmware interface was tested by calibrating the readout-sequences, and observing the signal chain from the detectors to the readout software. The data corresponds to one image frame with an integration time of 0.1 s, and illustrates a row of pixels as indicated in the optical photograph. In addition to the illustration of the basic functionality, the data verifies the basic capacity of the approach to detect concealed objects.
4.1.3.4.3 Command Module and Related Subsystems
Command Module
The Command Module (CM) manufacture and test concerns the procurement and building a system from COTS components to construct the CM (principally a PC and power supply parts). The CM hardware was assembled and tested. It includes the following hardware that was used in the test setup: Power distribution units; Ethernet router/switch, industrial PC with user interface hardware and interconnection cables.
The system control software is ready and has been tested with the real CM, PCS, HMI, support sensor and radar modules and with radiometer and AAD dummy software modules. The test setup and the software are working as designed. The software provides coordination, control and monitoring for the CONSORTIS system. A graphical user interface (GUI) was also developed for a system level control and maintenance purposes. A control-link communication specification has been made and documented, including a list of all control commands and requests, detailed control data format descriptions etc. The power and safety functions under the CM were also tested and programmed.
Passenger Control System
The Passenger Control system (PCS) hardware has been assembled and tested. The passenger control system functional testing was performed at first by using commercial Advantech’s AdamApax.NET control and measurement software package, and after this by the software integrated into the CM Python code. The PCS control-link uses Ethernet interface controlled by an industrial box computer. All the outputs (system status lights, audio alarm, traffic lights, power outputs, etc.) and inputs (four optical sensors, operator flow control switches, emergency switch, power output measurement readings etc.) are working functionally as designed.
After this the system control software (inside the CM) and its graphical user interface were used to control the PCS and read its sensors and other inputs. The PCS control software block uses Modbus compatible function sets through control-link with Gigabit Ethernet.
The Passenger Control System (PCS) hardware has been procured and is deployed according to the functional layout derived from the CONOPS.
Both the Command module and the Passenger Control System were fully integrated to the CONSORTIS system and successfully utilized during the field trials and the demonstration to control the system and guide the passengers/test persons through the scanning flow. The system flow is fully automated so that the operator only has to push “OK” from the GUI for allowing the next person into the system.
Support Sensor
Support Sensor manufacture and test concerns the procurement of a COTS support sensor. The chosen sensor, Microsoft Kinect v2, was acquired and tested and code to interface it with the system was written. Passenger Control System (PCS) manufacture and test concerned the procurement and building a system of COTS components to construct a PCS. The subsystem hardware has been assembled and tested.
AAD and HMI Interface
AAD and HMI interface development concerns the implementation of the data interface between the AAD, HMI and primary sensors and the implementation of the HMI. An HMI front-end has been produced and reviewed from an ethical and operator experience perspective. The HMI back-end and the data interface software architecture have both been implemented and completed. The architecture has been tested successfully with most key-components in the loop and with dummy modules where needed (the radar and radiometer were not yet interfaced with the data-link).
4.1.3.4.4 Highlights
Particular highlights of the manufacturing phase are given below:
Completion of radar 16 channel, 340 GHz, transceiver focal plane array, incorporating state-of-the-art transceivers and smooth-walled feedhorns. This is believed to be the highest channel count used in a radar at this frequency.
Completion of wide field-of-view focusing optics with high speed optomechanical beam steering capable of scanning at 10 Hz frame rate. This is believed to achieve the highest combination of frame rate, field of view and resolution of any mechanical beam steering system in a millimetre / sub-millimetre wave imager.
Completion of the 16 channel, 340 GHz radar subsystem and its successful integration into the CONSORTIS system..
Successful collection of state-of-the-art 340 GHz radar imagery acquired during data collection trials. More than 1000 data sets were collected by the radar during the trials in an operational scenario under system control. Additionally, hundreds of data sets were acquired during testing to investigate imaging response to a wider variety of targets, subject motion, subject tracking etc.
Development of bolometric kinetic inductance technology from single detector demonstrators into large-scale arrays. Prior to CONSORTIS the passive detector technology was demonstrated only with single detector characterization not integrated to any macroscopic optics system. In this project we demonstrated that large detector arrays can be fabricated and integrated with optics, and that the entity has capacity for concealed object detection.
Development of novel readout concept for kinetic inductance detectors. CONSORTIS developed a readout concept, with patent application filed, that enables the readout of large detector arrays. As compared to the competing techniques our solution enables the realisation of the readout system with affordable components, making it more attractive in commercial solutions.
Successful realization of state-of-the-art wide field-of-view dual-band inversely-magnified dual lens design for the radiometer. This doublet lens, manufactured from TOPAS™ material, focusses both the 250 and 500 GHz band radiation into the KID detector array and can refocus on moving subjects. The lens system was tested at a near-field measurement range at 500 GHz. The test procedures included re-focusing the lens system to the extreme distances designed, as well as the different sensor positions measured by offsetting the transmitter in place of the KID sensors. The constant beam properties were verified by measurements.
4.1.3.5 Field Trial
The objectives of the field trial were to develop and implement image processing and data fusion methods for privacy preserving detection of anomalies in multi spectral scanner data. In trial planning, input was collected from user requirements document (URD) and from external documents describing accepted test procedures (the International Wireless Industry Consortium (IWPC) protocol was mainly used). From this input, large efforts were made to produce a proper experimental design and compile a trial plan that would include all measurements needed to generate reliable system performance evaluations, but still would be possible to execute within the six-day duration of the trial. Additionally, administrative procedures were investigated and practical arrangements were planned to allow safe and secure handling of explosive materials during the trial.
A significant deviation from the original plan came with the consortium decision to re-locate the trial from FOI’s premises at Grindsjön, Sweden, to VTT’s premises in Espoo, Finland. The trial was then held at VTT premises on 13th -17th and 20th of November 2017. The trial was conducted using the CONSORTIS prototype system (except for the radiometer which was not operational) running in the intended CONOPS. On 13th -17th, the trial focused on tests with explosive simulants. The test plan was designed to span over the factors:
• Different body types: {average male, rotund male, average female}
• Different positions: {stomach, under of arm, back, inside of thigh}
• Clothing: indoor, outdoor (e.g. a pullover, no coats)
• Different simulants material: three different slab materials + four different granular materials
• Object size/weight: Small (250gr/ml), Medium (500gr/ml), Large(1000gr/ml).
In addition to simulants testing, tests were also made on a few chemicals and with different weapons. Test runs were also made on a variety of different ‘clean’ test persons. On 20th of November, the tests were performed with real plastic explosives (PENO).
Altogether, 1026 tests were properly collected and recorded within the duration of the trial. All tests have then been properly annotated and stored, together with their corresponding sensor data, in a database for easy access and later use. A method was also set-up on how to evaluate the data in a suitable way. Due to the fact that a fully automatic AAD method was not available, it was decided to evaluate trial data using a scheme in which the target/anomaly detection was operator assisted. An evaluation-plan was prepared to determine which subsets of the trial dataset were feasible to be used in an operator assisted setup. A framework to pre-process all necessary radar data presented to the operator was also constructed. Results from the operator assisted evaluation were collected and analysed.
4.1.3.6 Technical (Planned Airport) Demonstration
The planning of the airport demonstration was performed by FNA and VTT and an airport demo plan was prepared. FNA was ready for a demonstration at the airport. However, because of the delay in the development of the imaging radiometer and the automatic anomaly detection (AAD), and the consequent unavailability of those subsystems, it was decided not to have the demonstration at the airport. Instead, a laboratory demonstration for mostly technical specialists was performed at VTT in Espoo, Finland on 14 December, 2017. The arrangement in the demonstration was very similar as in the trial in WP7, but with a limited scale and documentation.
Potential Impact:
4.1.4.1 Dissemination
4.1.4.1.1 Stakeholder Engagement Strategy
The strategy used for dissemination was based on the need to engage with stakeholders and potential users and customers to exploit the results of the project, while at the same time protecting IP and the competitive position of CONSORTIS. Dissemination routes have included the public website, engagements with stakeholders as part of the user requirements analysis and exploitation strategy, presentations at appropriate conferences and peer reviewed publications. In total, there have been 63 dissemination events during the project and four events scheduled for after the project end.
Dissemination has been carefully controlled due to the overriding need to avoid premature disclosure of emerging results and the planned implementation of the CONSORTIS system, to avoid alerting competitors, and the need to protect IP at a stage when the technology is still immature. Therefore, dissemination has focused on presentations at key conferences to engage with stakeholders, rather than producing peer reviewed academic papers giving extensive descriptions of the approach and underlying know-how.
An approval process was implemented during the project to screen proposals for conference presentations and publication. As the design and development work progressed, technology advances that may require protection were reviewed to determine if the foreground IP developed was patentable, and any necessary IP protection activity was initiated. The volume of dissemination activity within these constraints has been considerable. Significant progress in the sub-systems, particularly the radar component, has been extensively disseminated via scientific presentations and papers, together with briefings for academic and government stakeholders.
Dissemination activity for the CONSORTIS system as a whole has been constrained by the delay in obtaining system performance data and the postponement of the Airport Demonstration. Consequently dissemination of CONSORTIS system results during the project has been limited to a Technical Demonstration targeting aviation technical experts but also involving some potential non-aviation users at VTT on 14th December 2017. A further demonstration was held to show the performance of the system to the European Commission Project Officer and Project Reviewer in February 2018, and system performance results will be used and disseminated in follow-on exploitation activities, in particular seeking Government and Industry support between January and December 2018.
4.1.4.1.2 Website, social media, publications
Website
The main responsibility for the development of the CONSORTIS project website lies with VTT, the Coordinator. The first version of the website was up and running on 31 March 2014 and has been updated three times during the project. The site contains a public section and a password protected partner-only section, which was originally planned to be used to exchange information within the project. However, VTT’s secure SharePoint server was found more practical for this purpose, and consequently it has been almost exclusively used for internal project information exchange purposes.
Use of Social Media
The importance of social media is increasing also in scientific and industrial communication. Because of the rapid developments, the use of social media in CONSORTIS was not envisaged in the original project plan made in 2011-2012. The other reasons not to emphasize the use of social media are the need to protect sensitive information and also because of the uncontrollable nature of information spread in said media. Those reasons still hold at the end of the project, and therefore social media did not become a mechanism for internal or external CONSORTIS information sharing during the project lifetime.
Publication of (Peer Reviewed) Journal Articles
Dissemination has focused on presentations at key conferences as part of the exploitation activities to engage with stakeholders, rather than producing academic papers. These were carefully controlled to avoid disclosure of details of the CONSORTIS technology to protect IP at a stage when the technology was immature. Peer-reviewed journal articles generally require more technical details than conference presentations. As a consequence, there have only been a limited number of peer-reviewed papers during the project.
Other Publications
The public sharing of project information has taken place in the following main channels: 1) Scientific journals, 2) Scientific and technical conferences and 3) Personal meetings with stakeholders. A wider publicity through trade press and other media has not been deemed necessary or even desirable, and has not been used.
4.1.4.1.3 Scientific Exchange
Interaction with other European Commission-Funded projects
The FP7 LASIE (LArge Scale Information Exploitation of Forensic Data) project invited the CONSORTIS team to participate in a project workshop in London on 15-16 July, 2015. The data being considered by LASIE include CCTV surveillance data. Since ethics and privacy were discussed there, Andreas Wolkenstein of EKUT, who is leading on these issues within CONSORTIS, attended the workshop. Ethical issues are an area where, more than in purely technological topics, common ground between the projects exists, and these were investigated in the workshop.
Discussions have been ongoing during the project with the FP7 TeraSCREEN (Multi-frequency multi-mode Terahertz screening for border checks). A joint demonstration to stakeholders was considered but not pursued as both projects had technological issues and did not reach the required maturity to make this appropriate. The TeraSCREEN project was invited to contribute papers to the European Microwave Conference Workshop in October 2017, which was organised by ISEC and Rohde and Schwarz.
Interaction with National Research Projects
There has been interaction with Academy of Finland, Centre of Excellence in Low Temperature Quantum Phenomena and Devices (CoE-LTQ). The Academy of Finland’s Centres of Excellence are the flagships of Finnish research. CoE-LTQ is formed by partners in Aalto University, University of Jyväskylä and VTT to pursue research in low temperature physics, technology and nanophysics. The detector solution chosen for the passive front-end in CONSORTIS has initially been developed within CoE-LTQ. The same technology is being applied in other VTT projects, for example in space applications.
4.1.4.2 Exploitation
4.1.4.2.1 Status
The overall project exploitation plan, described in submitted deliverables and periodic reports, comprised the following four elements:
1. Targeted dissemination of emerging CONSORTIS project results
2. Development and implementation of individual partner exploitation plans.
3. Demonstration of system functionality and performance at an airport
4. Development and implementation of a system level roadmap and plan for commercialisation and market entry
Significant progress in the sub-systems, particularly the radar, has been extensively disseminated via scientific presentations and papers, together with briefings for academic and government stakeholders.
Individual partner exploitation plans have focused on the development and protection, where appropriate, of partner IP, and the foreground IP developed during the project. Exploitation of the partners’ sub-systems and components beyond the end of the project, including plans for licensing to the system integrators, has been addressed.
System performance trials have been held and detailed plans were developed for the demonstration at Helsinki airport, as described in the Airport Demo Plan. However, the airport demonstration was postponed until the fully integrated CONSORTIS system could be tested. The exploitation plan was therefore revised to replace this with a Technical Demonstration.
The CONSORTIS system comprising the integrated radar, radiometer, AAD and passenger control system has reached a TRL of 5/6, apart from the Automatic Anomaly Detection (due to lack of data and time).
Five patent applications have been submitted, three of them concerned with the same invention. One of these three applications is listed in the patent table of this Final Report. The other two inventions could not be included in the patent table, because as confidential they did not yet have the required URLs.
4.1.4.2.2 Target User and other Stakeholder Groups
The target user and stakeholder groups have been revised in the light of market developments and the potential USPs of the CONSORTIS system for exploitation. These include government and law enforcement agencies concerned with both the aviation security and wider non-aviation people screening sectors.
The main stakeholder groups that have been targeted include:
• EU Legislative Authorities e.g. EC DG MOVE on EU aviation security policy, European Civil Aviation Conference (ECAC) on civil aviation security policies and practices.
• National Legislative Authorities e.g. Finnish Transport Safety Agency (Trafi), UK Department for Transport (DfT).
• National Security Authorities e.g. UK Home Office, UK Border Force.
• Airport Operators and trade bodies e.g. Finavia, Airports Council International (ACI).
• US Government agencies, e.g. Department of Homeland Security (DHS), Transportation Security Administration (TSA).
• Critical infrastructure agencies e.g. UK Centre for the Protection of National Infrastructure.
• Potential Industry exploitation partners, system integrators, etc.
4.1.4.2.3 Conclusions on Exploitation
The final exploitation plan has been made for commercialisation and market entry for the CONSORTIS technology solution and sub-systems developed by the project for people screening. It is an update of the second draft exploitation plan issued two years earlier and reflects market changes over that period, in particular in alternate markets other than aviation, and the technical advances made by the project that concluded with a technical demonstration in December 2017.
Significant progress in the sub-systems has been made and extensively disseminated via scientific presentations and papers, together with briefings for academic and government stakeholders. Individual partner exploitation plans have focused on the development and protection, where appropriate, of partner IP, and significant foreground IP has been developed during the project. Exploitation of the partners’ sub-systems and components beyond the end of the project, including plans for licensing to the system integrators, has been addressed.
Exploitation activities in the final two years of the project have included attending relevant events involving potential exploitation customers, partners and competitors to gather market intelligence, and this has been used to shape the commercialisation plan. System performance trials have been held and detailed plans were developed for a demonstration at Helsinki airport, as described in Airport Demo Plan. The original strategy was to use the Airport Demonstration as the main mechanism to engage key stakeholders in support of the exploitation plan. However due to delays in the development of the passive radiometer system which delayed the gathering of system performance data it was not possible to develop the CONSORTIS system to a sufficiently advanced level that would be credible to Airport customers. The Airport Demonstration was therefore postponed from October 2017 and a Technical Demonstration involving aviation customer and industry technical experts as attendees was carried out at VTT in December 2017.
At the culmination of the project (December 2017) the CONSORTIS system has reached a TRL of 5/6 apart from the Automatic Anomaly Detection (due to lack of data and time), but without a demonstration being made in an operational environment (TRL7) at an Airport. Given the complexity and risk of the project it is not surprising that this point has not been reached. However the results of the technical demonstration are sufficiently promising that partners have agreed that further investment funding will be sought to take the CONSORTIS concept/technologies forward.
4.1.4.3 Ethical Aspects
4.1.4.3.1 Introduction
As a part of the wider societal implications of the project, research partner EKUT was tasked with conducting ethical research about the CONSORTIS system. The work was divided into two parts. The first consisted of an overview of ethically relevant values with regard to security and security scanners. The second part consisted of two empirical studies examining the so-called fair process effect.
4.1.4.3.2 Security and Values
In the first part of the work an overview of what values are was presented. Based on a distinction between the act of valuing, i.e. holding something valuable, and the object of values themselves, a working definition of values was given:
“Values are entities that are judged to make a positive contribution to the judger’s fulfillment of motivational psychological states if brought about, realized or expressed through one’s actions or character dispositions (a process called valuations, or attributing value). By reference to values we express a normative recommendation with differing force concerning a practical decision. Values serve different functions: among others, they can serve as reasons for actions (in the form of orienting action, but also in the form of criticizing actions).”
The action-guiding nature of values is hampered by the existence of value-pluralism. Since people differ in their valuing things, they will inevitably assume the existence of a variety and plurality of values. Value-pluralism might be overcome by certain (argumentative) strategies, but we still need to expect that there are situation where there is insurmountable conflict between values. This holds true for both the intra-individual case (where a person cannot chose between to action-guiding values regarding a given decision she has to make) and the inter-individual case. This means that different people might vary between what they think are important or even final values.
Undeniably, value-pluralism is a fact in the real world. However, this does not preclude us from assuming the existence of value monism at the foundational level. A theory of value-monism thus understood is not about the relation between individuals and the things they value (and subsequently understand as reified value), but about the source of value. Here, the idea was presented that the notion of aim-fulfillment provides the basis for valuing and values. In other words, every instance of valuing something (and holding that something as a value or seeing it in the value-perspective) is indexed to a person who, by valuing something, fulfills her aim of leading a good life. Therefore, leading a value-based life is exactly what gives values their basis and resource and, as we shall see, provides the resource for solving inter-individual pluralism.
Inter-individual value pluralism is particularly important in the domain of (homeland, national) security since security itself concerns many people. When we think about putting value to security (and to derive political consequences), different people will come to different conclusions. The tradition of political philosophy has discussed this problem which might even be considered to be at the roots of political philosophy itself. Various ways to deal with the problem of value pluralism have been proposed: democracy, cost-benefit analysis, public reason and libertarianism. Each of these solutions exhibits valuable insights, but in the end they fail to appreciate the most basic value, i.e. the individual person leading a good life. The solution that was pursued in light of these reflections involved the idea of individuals being able to exert a strong influence on inter-individual decision-making processes, an influence that is not granted by democracy alone, not by other forms of political organization. An ideal, but somewhat utopian solution to the problem of inter-individual pluralism consists of introducing flexibility and competition in the political process. Another insight derived from these theoretical arguments is that many problems with security and technology do not (only) address particular technologies, but decision-making processes as well as political and social institutions. And it makes room for the idea of looking at processes and their ethical features.
Given these rather theoretical remarks, a closer look at the issue of security (technology) revealed the existence of the following values and their importance for guiding the process of producing technology and making decisions: Privacy, discrimination, health, democracy, proliferation, responsibility, corrosion of civil liberties, cost-benefit analysis, passenger convenience, right to travel and costs. This initial list provided the background for a final list of ethical values in security technology as given below:
The identified list of ethical values in security is given in the following:
Security level
A certain level of security must be secured, especially if one technology (FBS, Full-Body Scanner) is thought to replace another (metal detector).
Action points: Tests and comparisons, societal participation in defining acceptable risk.
Liberty
Liberty as the central political value is regulating interactions between people and between the state and citizens.
Action points: Reconsider security policy-making by allowing participatory elements. Respecting rule of law.
Privacy (related: health issues, anti-discrimination)
Is related to liberty in that it defines what liberty is all about: choice, self-direction, data protection through control rights over information.
Action points: Implementing privacy-by-design features: disabling storage (of raw data), disabling connectivity with alternative means. Providing choice of security measures as demanded by customers; using FBS along with other technologies to enhance choice.
Responsibility for acceptable use
Individual businesses are responsible for rights-respecting use through Corporate Social Responsibility (CSR).
Action points: Codes of conduct for responsible business behaviour. Subscribe to European human rights standards.
Procedural justice
Fairness of security procedures involving FBS must be respected in order to respect individuals and to reach acceptance.
Action points: Design security procedure with elements of procedural justice: Allowing voice, respectful treatment through security personnel, enhancing trust in authorities through transparency.
4.1.4.3.3 Procedural Justice
The second part of the ethical research consisted of empirical studies about the fair-process effect. It thus follows up on the idea that procedures matter. Political procedures could not be studied in the project. Instead, the procedures at airports where security scanners are used were focused on. The main justification for pursuing this path is given below:
(1) Procedural justice accounts for many (positive) effects in decision-making contexts such as organizations and the workplace justice and law enforcement. Knowledge about fair procedures, together with the positive effects they have, might then lead to an improved way of introducing and using security scanners. Since acceptance even of non-beneficial decisions is enhanced through fair procedures, making people get screened by a (often) negatively viewed scanner might also become more accepted through fair procedures.
(2) It provides a novel approach both to learning about what is going on in people’s mind when they enter airport security and to make practical recommendation about how to organize airport security. This is particularly the case given the fact that in this study, factors driving people’s susceptibility to the fair process effect are being examined. A security architecture that takes these facts into account can thus be tailored to the specific needs of different groups of people.
(3) Existing research and the insights we can derive from it for practical purposes (e.g. crisis and risk communication) deserve further and more detailed analysis in order to qualify what we can say about the (practical) value of procedural justice.
(4) Finally, respecting people’s fairness-related preferences is required by moral reflection and is called for by recent politico-philosophical reasoning. Moreover, the beneficial effects of procedural fairness are not a way of tricking people into accepting what they would not accept otherwise. The point of procedural fairness is that it respects people’s preferences and views and is thus fully compatible with autonomy and freedom of choice.
In summary, the present research contributes to a body of knowledge that relates procedural fairness with people’s acceptance of security measures and technology. It improves our empirical knowledge, which in turn can influence ethical judgments. Finally, it sheds light on a particular form of human-policy-interaction.
4.1.4.3.4 Study 1
We conducted two studies. In the first study, we were interested in finding out whether or not fair procedures lead to people’s acceptance of security scanners. We hypothesized:
(1) The Fair Process Effect (FPE) will also occur at security controls in airports, that is: Test subjects that were given a choice will perceive the situation as more fair and it will be more likely for them to accept body scanners than subjects who didn’t have a choice in the matter.
(2) Fair procedures will have a larger effect in a low privacy condition than in a high privacy condition, which means that the difference of acceptance ratings between fair and unfair treatment will be larger within the low privacy condition than within the high privacy condition.
Manipulating choice served as an instrument to express fair procedures, where the existence of choice represents fairness, according to previous insights from psychological studies. Moreover, we wanted to show that fair procedures are particularly important when it comes to negative evaluations of security scanners, hence the manipulation of privacy conditions. Note that the study was not intended to represent a realistic scenario of using the CONSORTIS system, but to find out about underlying psychological processes in terms of accepting security technology.
Unfortunately our hypotheses were not confirmed, a conclusion that study 1 summarizes. However, future studies are needed to improve the study’s design since the fair process effect is well-known in other domains so that it would be surprising not to find it in security procedures.
4.1.4.3.5 Study 2
The second study followed up on the first one while introducing new parameters that might have an influence on the fair process effect. The hypotheses tested were:
(1) People with low numeracy tend to overestimate a security scanner’s effectiveness
(2) Procedural fairness trumps effectiveness ratings when people judge their acceptance of a security scanner, meaning that people who face a highly effective scanner are less likely to accept it if they perceive themselves to be treated unfairly
(3) People with low numeracy pay less attention to procedural fairness, meaning that the effect described in (2) is less strong for those people who are less numerate.
Numeracy, i.e. the ability to deal with numerical information (i.e. calculate probabilities correctly etc.), might be particularly important since security itself is a matter of risk and probability. Both the risk of, say, terrorism, is subject to probabilistic reasoning and risk perception, but also the perception of a scanner’s capacity to actually provide security (given information about false positives, false negatives, and the probability of detection).
Again, the hypotheses were not confirmed. This might be due to decision about the study design and information that participants received. Further studies are needed to improve the study and examine a psychological effect that is needed to enhance acceptance while at the same time respecting individual’s choices and values. This combination of acceptance and people’s values is what links the empirical studies back to what was said in the theoretical ethical reflections in the first part of the ethics work.
4.1.4.3.6 Conclusion on Ethics
The general conclusions of the CONSORTIS ethics research are:
1. More research is needed to examine the fair process effect in the domain of security. This can help to enhance acceptance in a way that is fair, open, and transparent. This, in turn, improves a societal discussion about security and will make societies more secure. Therefore, political decision-makers should have an enormous interest in funding more research on security and the fair process effect.
2. More research is needed to define the role an ethics of security should assume. As was repeatedly indicated, academic ethics should not only speak with (and to) engineers and their product, but also to those who are responsible of funding the engineers and make political decisions. But how exactly does research policy in security look like when this is taken seriously, i.e. when ethics is included in the process about how to design research policy itself? And how can ethical insights into research and security policy be inserted in the political institutions responsible for those policy domains?
3. Security technology such as the security scanner need to be assessed as part of a larger security architecture. Since, for instance, data provided by the scanner can be put in relation with other data (from CCTV etc.), a larger security apparatus emerges. Ethics needs to take this seriously. One way to do so would be to fund ethics projects not as part of a technology project, but as cross-sectional. Ethics projects would accompany several technological research projects and study the interaction between various measures; they would reveal problems of increasing security by combining technology and data, but also find ways to ease these problems. And through being in close interaction with these projects, they would be close to the technological reality while being able use their insights to feed the political and societal process.
List of Websites:
CONSORTIS website: http://CONSORTIS.eu/
Email contact: consortiscoordinator@vtt.fi
This project developed a prototype real-time dual-mode submillimetre-wave (SMMW, a.k.a. “THz”) security screening system for use at airports. This system is capable of high-throughput, improved security screening of people and is less restrictive for people using it than existing systems. The goal was to achieve this through the following overall objectives:
• Non-intrusiveness, safety and accuracy – SMMW systems are non-ionizing.
• Speed with near-real time video imaging (10 frames/s) for “walk-by” security.
• SMMW (“THz”) operation with passive sensors at about 250 and 500 GHz and radar centered at 340 GHz.
• Multi-frequency operation – the passive system with two frequency bands.
• Automated threat recognition capability.
• Privacy issues addressed with strong involvement of ethics experts.
• Pilot/demonstration – The system to be tested and validated against realistic threats in a controlled environment.
CONSORTIS was a 48-month project which adopted a systems engineering approach. The end-user requirements were first determined with the aid of external experts through interviews and advisors and translated into technology requirements. In parallel, ethical aspects were evaluated for deployment scenarios and provided ethical inputs to the end-user requirements, to help ensure societal acceptability of the solution.
The preliminary design of the system was produced and reviewed, followed by hardware manufacturing, sub-systems characterisation and testing, and system integration and testing. The Automatic Anomaly Detection (AAD) research focused on the development of algorithms. In the field trial the system was tested and validated in a laboratory at VTT against realistic threats, collecting over 1000 data sets relevant to aviation security with typical walk-through times of some seconds. An airport demonstration at Helsinki was defined and planned, but changed into a laboratory technological demonstration to allow some additional modifications delayed by resourcing challenges to be completed. The technological achievements include:
• World first 16-channel 340GHz radar imaging at 10Hz video frame rate
• World first 8000-element passive sub-millimetre wave security imaging array based on Kinetic Inductance Detectors (KIDs)
The major impact of the project is the hardware which offers a unique capability in Europe to collect security data at SMMW frequencies, not only in aviation but in the security of public crowded spaces and high value infrastructure. The hardware still needs further development before it becomes market ready, however, both the passive and active sensors represent a quantum step in performance. The concept is novel and has been protected by a system-level patent, as is the readout of the KIDs array. A number of peer-reviewed journal papers have been published, also the volume of dissemination activity has been considerable with many conference and workshop papers at international symposia. Significant progress in the sub-systems, particularly the radar components, has been extensively disseminated via scientific presentations and papers, together with briefings for academic and government stakeholders. In total 63 dissemination events, including presentations and briefings to stakeholders, have been performed during the project, with a further four planned beyond the end of the project, together with seven additional events used to engage with potential exploitation customers and partners and to gather market intelligence.
Project Context and Objectives:
This project developed a prototype real-time dual-mode submillimetre-wave (SMMW, a.k.a. “THz”) security screening system for use at airports. This system is capable of high-throughput, improved security screening of people and is less restrictive for people using it than existing systems. The goal was to achieve this through the combination of a multi-band passive SMMW imager operating at 250 and 500 GHz, integrated with a 340 GHz 3D imaging radar.
The combination of an active imaging radar system with a dual-band passive imaging radiometer was chosen to exploit the complementary image features revealed by the two modalities which in turn gives rise to improved security. Passive systems tend to be more mature in terms of imaging wide fields-of-view (FoV) at near-video frame rates but suffer from limited dynamic range, given the weak signatures associated with passive SMMW imaging, especially in situations where there are substantial clothing material losses present. Active systems, however, especially when operated in radar mode, can achieve very high dynamic ranges capable of overcoming the effects of clothing losses, but tend to only cover limited FoV at near-video frame rates. This is due to the technical challenges associated with building large arrays of SMMW transceivers. One of the challenges of this project was to bring these disparate sensors together in a common enclosure as part of the same system.
These overall objectives of the CONSORTIS project address the topics stated in the call as follows:
• Non-intrusiveness, safety and accuracy – SMMW systems are non-ionizing
• Speed – given that the system will be capable of near-real time imaging (10Hz) it is applicable for “walk-by” security screening solutions for subjects walking at normal pace
• SMMW (“THz”) operation – the system will incorporate passive sensors covering frequencies at about 250 and 500 GHz, while the active radar is centred at 340 GHz
• Multi-frequency operation – the passive system will cover two frequency bands for materials differentiation capability
• Automated threat recognition capability – this was addressed by Automated Anomaly Detection (AAD) algorithm development to make use of the complementary image outputs of the passive and active sensors
• Privacy issues – Strong involvement of ethics experts and the incorporation of the AAD capability minimises the potential issues with privacy
• Pilot/demonstration – The system was planned to be tested and validated against realistic threats in a controlled environment and also to be validated in a real (airport) environment.
Description of the Work Performed and the Main Results Achieved
CONSORTIS was a 48-month project which adopted a systems engineering approach. Work Package 2 (WP2) ‘Solution Specification and Validation’ identified end-user requirements with the aid of external experts through interviews and advisors (the Exploitation, Industrialisation and Ethics Advisory Board) and translated them into technology requirements (system requirements) appropriate for implementation by the technology teams. WP3 ‘Ethical Aspects Advisory’ ran in parallel to WP2, evaluating deployment scenarios, providing ethical inputs to the end-user requirements and providing guidance on societal issues to inform the design and testing of the system throughout the project, to help ensure societal acceptability of the solution.
WP4 ‘Automatic Anomaly Detection’ focused on the development of the AAD algorithms, guided by the WP2 system requirements, for implementation within WP7 ‘Field Trial’. The preliminary design of the system was produced in WP5 ‘Camera Design’, and review of the final designs was followed by hardware manufacturing, sub-systems characterisation and testing, and system integration and testing in WP6 ‘Camera Manufacture, Characterisation and Test’.
In WP7 ‘Field Trial’ the final system was tested and validated in a laboratory at VTT against realistic threats as defined by end users and test procedures developed within the project, based on best practise. A demonstration at Helsinki Airport was defined and planned in WP8 ‘Airport Demonstration’ to allow testing and validation in a realistic environment as defined by the end users. The airport demonstration was postponed until after the project to allow some additional necessary technology integration measures delayed by resourcing challenges to be completed.
In parallel with the whole project, which has been managed by WP1 ‘Management’, WP9 ‘Dissemination and Exploitation’ ensured the protection of foreground IPR, the dissemination of the project results through publications and conferences etc., and developed exploitation roadmaps for the sub-system technologies and the overall system to provide a route to market.
The CONSORTIS project has made several remarkable achievements in applying SMMW technology to aviation security. These include:
• World’s first 16-channel 340 GHz radar imaging at 10 Hz frame rate based on homodyne radar transceivers
• World’s first 8000-element passive sub-millimetre wave security scanner focal plane array based on Kinetic Inductance Detectors (KIDs)
• Multi-mode active and passive sensors working in the same enclosure
• Real time sub-millimetre wave imaging of passengers in a walkthrough mode.
Bringing the Radar and Radiometer systems together in a common enclosure for screening people is unique and a major step forward, and promises to produce ground-breaking performance in terms of security.
The CONSORTIS system was used in data collection mode to collect over 1000 data sets relevant to aviation security.
Potential Impact and Use
The market context for CONSORTIS has changed during the duration of the project. The aviation need to protect aircraft from IEDs carried by passengers or in-hold baggage continues. However, the need for improved affordable systems for the protection of urban crowded places and critical infrastructures has grown significantly, as highlighted by attacks in Manchester (2017), Copenhagen and Paris (2015), and other attacks outside the EU in crowded areas such as the Boston marathon and a Nairobi shopping mall (2013). EUROPOL has stated that the overall threat to EU security is high, with the level of activity in the EU attributed to jihadist terrorism continuing to rise.
The major impact of the project is the hardware which offers a unique capability in Europe to collect security data at SMMW frequencies. This will inform future system procurements and determine if SMMW technology offers advantages not only in aviation security but also other security applications such as public crowded spaces and high value infrastructure. The hardware as it stands still needs further development before it becomes market ready. However it represents much more than a step towards a new aviation security system as individually both the passive and active sensors represent a quantum step in performance. These sensors could in turn be applied in other market segments such as remote sensing or surveillance, providing them with a new capability. The walk-through concept for security screening using active and passive sensors is novel and has been protected by a system-level patent, as is the readout of the KIDs array. In conjunction to these patents there have been a number of peer-reviewed papers which capture the expertise of the consortium. These papers cover the KIDs array and the optics of both the passive and active systems, which are particularly challenging as the passenger is very close to the system requiring well-corrected optics over a wide field of view.
The volume of dissemination activity has been considerable leading to many conference and workshop papers at international symposia. Significant progress in the sub-systems, particularly the radar components, has been extensively disseminated via scientific presentations and papers, together with briefings for academic and government stakeholders. Sixty-three dissemination events, including presentations and briefings to stakeholders, have been performed during the project, with a further four planned beyond the end of the project to engage with potential exploitation customers and partners and to gather market intelligence.
Two Technical Demonstrations were held, primarily targeted at aviation technical experts. The first was held at VTT on 14th December 2017, involving customer technical experts as attendees. A further demonstration to show the performance of the system to the European Commission Project Officer and Reviewer was also held at VTT on 6th February 2018. The results are sufficiently promising that partners have agreed to collaborate to take the CONSORTIS concept/technologies forward for both aviation and non-aviation security applications.
Project Results:
4.1.3.1 General
CONSORTIS was proposed for the FP7 call SEC-2012.3.4-5 ‘Further research and pilot implementation of Terahertz detection techniques (T-ray)’ issued in August 2011. This call required:
• Frequencies >300 GHz and <1 THz
• Prototype system with single or multiple frequencies
• Safe and respecting privacy
• Demonstrator/pilot test/validation in a live operational environment e.g. airport
• More effective security
It was aimed at next generation aviation security systems that would increase passenger freedom by reducing divesting, queuing and the number of constraints on passengers. Furthermore it should satisfy the current ethical requirements.
To satisfy this call the CONSORTIS project team elected to design and fabricate a walkthrough aviation security system. The selection of walkthrough provided greater freedom for passengers with fewer constraints. More effective security was planned to be achieved by higher probability of detection, lower false alarm rates and being able to detect the unexpected. This goal was to be achieved by combining a dual band sub millimetre wave passive system with sub millimetre wave radar capable of measuring 3D shape. This combination increases probability of detection because it combines sensors which use different methods to form an image and this combination also reduces false alarms. For both sensors the specification was such that the performance would be a step change over in-service equipment.
Such a system has some top level challenges that have to be overcome to be successful. They include:
1. Be simple to use
2. Aviation security has demanding fields of view as passengers are close to the system ~1 m requires 60° vertical and 27° horizontal just to cover a person
3. Combining the passive and radar sub-systems in one enclosure must not impede the performance of either sensor
4. Combing real time imagery from three submillimetre wave sensors (two passive and one active) to generate improved security has not been done before
5. Ethical best practise.
A systems engineering approach was adopted and a requirements capture was conducted, against this a systems requirement document was developed. The sensors are accommodated in a tower and they track and scan the person as they walk past it. In the final system two towers will be required to see both sides of the person but funding was only available to demonstrate a single tower in this project.
This planned full system would consist of a 340 GHz radar, a 250 and 500 GHz radiometer, an automatic anomaly detection (AAD) system, a command module and a passenger control system. The specification for each of these sub-systems flowed from the systems requirement document.
The ethical requirements were described and fed into the requirements capture. The ethics work also had a wider remit looking at the ‘fair process effect’ in systems like body scanners and addressing the future in a world where security systems will be connected by the Internet and how ethics needs to be considered.
State of the art security systems analyze the image data generated by the system using algorithms on a computer to indicate the presence of a threat to the operator. In CONSORTIS this function was to be fulfilled by the Automatic Anomaly Detection system (AAD) and with three sub-millimetre wave sensors this is clearly a key function. The development of the AAD is described below in chapter 3.
The design of the CONSORTIS hardware is described in chapter 4 as the requirement driven specification was iterated with what could be achieved in hardware and some prototyping was done. In this work package the architecture of Command Module which takes command of the whole system and the Passenger Control System which controls the flow of passengers were also defined. Some prototyping was also done. The Preliminary Design Review (PDR) and Critical Design Review (CDR) were conducted.
The build of the hardware and its integration is described in chapter 4, this includes all the sensors, the enclosures, passenger control system and command module.
CONSORTIS was tested in the laboratory at VTT with performance tests taking place with airport staff participation and this work is described in chapter 5. Testing was conducted against dummy weapons, explosive simulants and real explosives with the help of the Finnish police.
Preparation for the airport trials were made and this work is described in chapter 5, however due to severe time constraints at the end of the project, as well as the inavailability of finished radiometer and AAD, it was decided not to conduct these trials at Helsinki-Vantaa airport. However, as much as possible of the methodology and operational knowhow were included in the field trial.
Instead of the originally planned demonstration at Helsinki-Vantaa airport, a technical demonstration was held at VTT on 14th December 2017, involving user community technical experts as attendees (chapter 6).
Dissemination and Exploitation are described in a subsequent section of this report which brings together all the publications and events and the plan for taking CONSORTIS to the market.
4.1.3.2 Requirement Capture
This activity was responsible for capturing the needs of the airport users for a CONSORTIS solution (and the user-needs from adjacent markets where possible) and for validating these needs throughout the project. The aim was to guide the project towards an exploitable solution that, as far as practicable, met user requirements and aspirations. .
The project adopted a Systems Engineering approach to the design in which three major activities were undertaken. They were:
a. Definition of what users want, stated in their own operational terms (The User Requirements Document or URD)
b. A technical description of what is needed to meet the URD was developed in a Systems Requirement Document (SRD)
c. The design solution was produced, using real hardware and software, that met the technical requirement (Candidate Architecture(s))
d. Throughout the project these documents were used to validate sub-systems and the final hardware to ensure that the user requirements had been met.
Visits were made to three regulators, five airport operators, three industry experts, one equipment operator and (indirectly) three bodies representing the views of the general public. Structured interviews were conducted with a balanced set of stakeholders which included Europe’s largest user of body scanners plus an important airport trade association.
It was necessary to revisit the contents of URD in the light of what is technically possible and to implement trade-offs. This iteration was also part of the work involving all consortium members, under the leadership of the Technical Manager, as part of the design reviews process led from WP2.
Generating the Concept of Operations (CONOPS) for meeting all the requirements was very demanding and over 20 alternative approaches were considered and discussed with potential users. The airports strongly preferred an unencumbered walk through approach. This solution comprised CONSORTIS system to be positioned either side of a transit path along which the passenger walked with arms raised, to allow images from under the arm to be visible.
Comparison of the technological solutions and the user needs was also very challenging and it was necessary to iterate around the systems engineering cycle three times. Key outputs from this work were:
• 48 specific user requirements.
• The definition, review and agreement of the CONOPS for the equipment, heavily influenced by the needs of the airport operators for a simple walk-through system.
• Definition of the trade-off space balancing detection probability, false alarms and throughputs which, together with cost, will be critical in positioning the CONSORTIS system in the market.
Once the architectures had been defined and performance modelled, this output was used to generate detailed equipment design specifications.
There was also the ongoing task of monitoring operational airport developments and technical developments to identify and implement any further changes that may be needed during the life of the project. To this end, good communication with stakeholders was developed and maintained by this task. The project held meetings with the European Commission (principally through the offices of DG Move), airport operators, including their trade association, ACI (Airport Council International), and public bodies through the European Association of Airport and Seaport Police (a member of the EIEAB).
4.1.3.3 Automatic Anomaly Detection
The objectives of Automatic Anomaly Detection (AAD) research were to develop and implement image processing and data fusion methods for privacy preserving detection of anomalies in multi spectral scanner data.
The pre-existing data from previous systems was identified and collected. The collected data was analysed and reported including a description and analysis of the example datasets together with an indicative evaluation of what sensor specifications were required by the AAD. It was concluded that data to initiate AAD for radiometer was already available. A shortage was identified on data to support the development of AAD algorithms for radar and for developing fusion algorithms.
In the development of AAD method, four main activities were undertaken: 1) initial development of AAD methods for radiometer and radar data, 2) synthetic data generation, 3) design of a strategy for using results from the two sensor modalities and 4) the design of a support sensor system.
Initial Development of AAD in Radiometer Data
A literature study was undertaken trying to establish the current state-of-art in anomaly detection in passive mm-wave imagery. From this study a set of promising image processing strategies was identified, implemented, and evaluated on existing data. Auxiliary algorithms needed for e.g. foreground detection (=subject to be scanned) and silhouette outlining was developed and evaluated. It was concluded that a candidate set of algorithms was now available and that final development and evaluation could continue as soon as final data was available having the characteristics and properties of the final CONSORTIS passive imager.
Simulation Tool Chain
To mitigate the absence of relevant radar data before the completion of the radar, efforts were made to generate synthetic radar data. A complete tool chain was established spanning from character animations, cloth simulation and electromagnetic field propagation and simulation. From simulated data, studies were made to assure that CONSORTIS system requirements were appropriately set to yield data with e.g. sufficient resolution performance. Synthetic data was also used for initial development of AAD algorithms in radar data.
Initial Development of AAD in Radar Data
Using synthetic data and, later on, data from the ‘pathFinder’ radar, a single channel device built to establish feasibility, initial algorithm development was undertaken for AAD on radar data. Data was explored for its basic characteristics and tools and algorithms were developed for data filtering. Filters were constructed to find the ‘first’ and ‘last’ surface in data (detect outermost layer of clothing vs body’s skin), to find sharp edges in surface geometries, to find strong curvatures etc. An algorithm was also developed to compensate for motion artefacts in data due to subject movements during data capture.
Sensor Fusion and Support Sensor
Strategies on how to fuse results from the two modalities were outlined. It was found that the most robust and flexible solution for sensor fusion is to run separate AAD’s on each sensor (radar and radiometer), and fuse the results in a post-processing step. It was identified that a support sensor is needed for the sensor fusion, and for other reasons also which include to analyse the scene and to align data from both sensors. A support sensor system was designed and a prototype was developed and tested. The support sensor system detects the position and poses of the subject and provides all information necessary for data alignment.
AAD Implementation and Evaluation
The original objective was to make a final implementation and parametrization of the AAD prototype algorithms, using data from camera laboratory tests and system performance field trial. A re-scope of this task was necessary to mitigate the early enough lack of data from the CONSORTIS sub-system sensors. The main achievements of this task are described below.
Implementation of AAD in Radiometer Data
Based on the data from the commercial Argon passive SMMW imaging system of AQA, a final pick was made on which algorithm to deploy in the CONSORTIS prototype as soon as data would be available. The algorithm code was migrated to execute on the CONSORTIS platform and within the AAD sub-system component. It was tested that the algorithm could process multiband passive imagery at 10 Hz video rate.
Implementation of AAD in Radar Data
In close collaboration with USTAN and TU Delft, progress was made on the radar based signal processing. A number of small measurements campaigns were conducted by USTAN using the pathFinder system, and later on the radar sub-system component, to support the algorithm development. Major work was undertaken to mathematically describe the optical system of the radar and to find a method to conduct image formation that is geometrically correct.
AAD Integration
A software environment was designed and implemented to embed the core AAD-algorithms as soon as they were available. This software environment included the functionality needed for integration with the CONSORTIS software architecture and to receive and buffer data from the sensors. The environment also included functionality to distribute calculations and perform multi-threaded (i.e. parallel) processing of data frames from the radar and radiometer. Using ‘dummies’ as core AAD-algorithms, the software environment was fully integrated with the CONSORTIS software environment and successfully stress-tested against heavy data loads.
AAD Post-Trial Evaluation
After the system performance trial was completed, a large effort was made in a short timescale to analyse the collected data and to compile a tool-chain to process the data for the best possible use in the system performance analysis.
4.1.3.4 Camera Design and Manufacture
The camera design work was conducted in two phases: preliminary design culminating in the Preliminary Design Review (PDR) which was completed in October 2014 and detailed or critical design culminating in the Critical Design Review (CDR) which took place in April 2015.
4.1.3.4.1 Imaging radar
Sub-mm Wave Transceiver
The 16 sub-mmw transceivers were supplied by GOTMIC (x8 MMIC frequency multipliers) and WMW (170 GHz doublers and 340GHz doubler-mixers) to USTAN for incorporation into the radar subsystem. All units were successfully delivered by November 2016 and met specification. For this operating frequency, the flatness per unit and the spread across units is extremely good.
Active Front End
In active front end manufacture and characterisation the work was concerned with the chirp generator and 16-channel receiver/demodulator built by USTAN. The chirp generator was successfully manufactured, optimised and met specification. A smooth-walled feedhorn was designed and manufactured. All 16 feedhorns were measured and achieved specification with state-of-the-art performance. The 16-channel receiver/demodulator was designed and manufactured and met specification with close channel-to-channel balance.
Optics and Optical System Mechanical Manufacturing
Camera optics and opto-mechanics manufacture and characterisation concerned the manufacture of the optics for the radar and radiometer sensors and their characterisation to confirm that the optical performance meets specifications. The radar optics and opto-mechanics were manufactured by VTT and delivered to USTAN in May 2016. The optics comprises five large reflectors and a mini-mirror array mounted to the transceiver horn assembly. The sequence starting at the transceivers is: mini-mirror array, Dragonian secondary (fixed, curved mirror, oval shape), Dragonian primary (reciprocating, curved mirror, oval shape), two counter-rotating circular plane mirrors (forming the ‘double-disc scanner’), and a large ‘pan’ mirror steerable in the azimuth axis (flat, oval shape).
The key characteristics of these elements were:
Mini-mirror array: 16 mirrors alternating between +45° and -45° to direct feedhorn beams from left and right sides into a single line array which was machined from a single piece of aluminium with thin walls between mirrors to avoid spillover between beams.
Dragonian mirrors: doubly-curved focusing surfaces in aluminium with a required profile accuracy of λ/20 ≈ 40 µm. The primary mirror should be lightweight and stiff as it is driven in a reciprocating motion for the elevation scan.
Double-disc mirrors: circular plane mirrors whose surfaces are tilted off-axis and which counter rotate. Rotation rate of 2700 rpm is required for 10 Hz imaging frame rate. Light weighted and mass-balanced body framework was manufactured with a plane aluminium reflecting surface cemented on top.
Pan mirror: aluminium body framework and cemented aluminium sheet as a reflector surface, driven by a stepper motor.
The optical elements are mounted on a sub-chassis assembled from commercially available aluminium extrusion profiles built around a thick solid aluminium base plate which supports the majority of reflectors, particularly the rotating double-discs with their drive mechanism. FEM simulations were carried out for the whole radar assembly to ensure that any (resonant) vibrations generated by the double-disc scanner are at frequencies much lower than the natural frequency of the entire radar assembly.
The double-disc scanner, a form of Lissajous scanner, scans a long thin figure-of-eight pattern in azimuth, approximating a linear scan. Using rotating elements makes it feasible to achieve the required 11 ms horizontal scan time. A single 230 Vac motor drives a shaft from which the precise and synchronised counter-rotation of the two discs is achieved using belts and pulleys. The scanner operates successfully at the rated speed of 2700 rpm (45 Hz) with minimal vibrations but the system does generate quite a lot of noise.
During beam profiling it was discovered that the optics were not focusing correctly and this was traced to the Dragonian mirrors being distorted in shape. Whilst they had been milled from single pieces of aluminium to high precision they had deformed after tooling. They had to be completely redesigned and remanufactured paying special attention to minimise shape deformation after machining. This meant using a more stable grade of aluminium, machining in several stages and special mounting methods. Moreover, the backside of the reflectors was designed to have a deep spider’s web support structure, which greatly improved the rigidity from the original design. Accurate inspection on a coordinate measuring machine (CMM) verified that the reflectors met the shape specification. Subsequent beam measurements confirmed that correct focusing was achieved – see Radar Subsystem Characterisation and Testing section below.
Radar Back End
Back-End manufacture and characterisation concerned the back-end electronics for both the radar and radiometer. For the radar this comprised COTS data acquisition cards (16 channels, 25 MS/s), a desktop PC and high speed, multi-threaded parallel-processing code written in C. The underpinning code was based on a successful radar processing architecture developed during a previous USTAN radar imaging project which was modified for the CONSORTIS data acquisition cards and radar parameters. Various engineering versions of the code were also written for low level control and analysis during the radar manufacture. The radar software also performs all the control functions for the various motors in the system, control of the chirp generator, and the BIT/BITE functions. Raw intensity imagery is displayed by the software to aid testing and characterisation. Ultimately raw data will not be displayed as all image analysis would be handled by the AAD to ensure privacy.
Radar Subsystem Integration
The integration of all the radar components and modules was undertaken by USTAN and encompassed many tasks.
Radar Subsystem Characterisation and Testing
Key parameters which were confirmed in characterisation and testing were the frame rate (10 Hz), field of view (1 x 1 m2), depth of field (1 m), beam spot size and range bin width, all of which are critical to meeting the imaging requirements.
Radar Subsystem Compliance with Original Objectives
Two aspects of the radar subsystem’s performance fell short of the original objectives.
1) The field of view was limited to 1 x 1 m2 which is less than required to cover a whole person (the aim was 1 x 2 m2). This was due to reaching a compromise in the design of the optics
2) The frame rate achieved during data collection had to be limited to 7 Hz (instead of 10 Hz) due to a timing latency issue, straightforward to be corrected in the future. Both the optomechanics and signal processing software were proven to be capable of running at 10 Hz.
Despite these two shortcomings the radar was able to collect significant quantities of data during the data collection trial. All the other key parameters of the radar’s performance met the original objectives, as described in the following:
• Design of a radar chirp generator by USTAN (with output in the microwave range) which achieves very high linearity, low phase noise, short duration chirps over large fractional bandwidths (>10%).
• Design of an advanced five-mirror quasi-optical system for the radar by TU Delft capable of imaging at up to 10 Hz frame rates over a wide 1 x 1 m2 field of view at close range (~2.5 m) with low aberration and scan loss. This comprises a Dragonian mirror pair for focussing, a high speed double-disc scanner combines with the reciprocating Dragonian primary mirror to achieve azimuth-elevation beam scanning, and a pan mirror to steer the radar field of view to follow the subject.
• Design of novel self-mixing multiplier radar transceivers which combine MMIC and Schottky diode technology from GOTMIC and WMW. They have shown state-of-the-art simulation results, achieving wideband operation with excellent power flatness and low signal distortion. These units remove the need for separate duplexing components to separate transmit from receive and hence enabled a focal plane array of 16 radar channels – believed to be the world’s biggest submillimetre wave radar array.
• De-risking the radar design by building a single transceiver, mechanically scanned, lens focused, 220 GHz ‘pathFinder’ radar. This proved the chirp generator architecture and the radar transceiver design methodology. Additionally, it yielded high fidelity radar data on mannequins and test targets with 1cm3 voxel resolution which was valuable for the AAD development.
• Use of a Microsoft Kinect v2 as the Support Sensor by FOI to provide accurate, high resolution skeleton tracking and pose estimation of the subject to assist passenger control, panning of the primary sensors and data processing.
• Design of a Passenger Control System (PCS) by RSN and VTT which meets the user requirements of airport operation and controls the passenger flow suitably for reliable imaging of each person. The PCS makes use of COTS components.
• Design of a Command Module (CM) by VTT and FOI which coordinates all the subsystems in the system ensuring reliable high speed data interchange and separate low speed control information exchange. The CM uses COTS components.
• Mechanical design of all system and subsystem hardware using 3D CAD modelling software by VTT and USTAN to ensure correct part manufacture and successful system integration. This included finite element analysis of the high speed moving parts within the radar scanning mechanism which was performed to support the mechanical design, ensuring balanced, vibration free motion and reliable operation.
4.1.3.4.2 Imaging radiometer
Passive Front End
The passive front-end manufacture and characterisation concerns the build of the passive focal plane array (FPA), cryostat and read-out electronics being developed by VTT and AQA.
The core of the imaging system, the focal plane with 8208 detectors was fabricated at VTT’s clean room facility using multi-layer superconducting fabrication line integrated with a deep etching process to realize the nanomembrane structure needed for thermal optimization. The requirement of the dual-band operation generated the need to develop a novel fabrication technique based on stacked wafers to realize the designed quasi-optical coupling. For this end, fabrication test runs prior to actual detector fabrication were executed. .
The detectors were integrated into Gifford-McMahon type cold head and compressor integrated with the custom designed and manufactured vacuum system with quasi-optical feedthroughs. As the system is relying on a large 2D focal plane, the filtering scheme intended to reject ambient radiative thermal loading was designed and integrated into the system. For the RF-based readout, readout-band (40 - 400 MHz) wiring was designed and manufactured, based on flexible RF stripline connections.
The detector readout electronics was manufactured and integrated with the system. In addition to the basic readout concepts, circuitry for temperature stabilization was implemented.
Radiometer Optics and Optical System Mechanical Manufacturing
The lenses for the radiometer optical system were manufactured and thoroughly characterised. The lenses were made of material, cyclic olefin copolymer, with exceptionally low loss at sub-millimetre waves. The trade name of the material is TOPAS™. The material is well suited for CNC machining and the RMS error of the manufactured surface was 11-37 μm – almost according to specification. The symmetrical inaccuracies were compensated with re-optimizing the lens positions for all focusing distances. Performance was very close to design predictions, differing only slightly due to mechanical tolerances in fabrication. The pan/dither mirror was successfully manufactured.
The framework of passive system was constructed utilizing aluminium profiles where different subassemblies are mounted with adapters taking care of the correct positioning relative to each other. Some of the subassemblies like lens moving assembly were delivered to another location before the complete system was ready to be assembled due to delays and project schedule. The final assembly of the passive subsystem was performed in VTT’s Espoo site.
Back End
The back end of the system consists of the analog-to-digital converters, microprosessor elelctronics with firmware realising the readout sequences, and PC software for image high-level data processing and image formation.
Radiometer Subsystem Characterization and Testing
Various components of the radiometer were separately tested to verify functionality. This includes readout-band characterisation indicating the basic functionality and good yield (typically >97% within the readout channels), the video-band response of the detectors, and point-spread functions indicating the functionality of the system optics, as well as detector quasioptics, as integrated into the radiometer subsystem.
The electronics-firmware interface was tested by calibrating the readout-sequences, and observing the signal chain from the detectors to the readout software. The data corresponds to one image frame with an integration time of 0.1 s, and illustrates a row of pixels as indicated in the optical photograph. In addition to the illustration of the basic functionality, the data verifies the basic capacity of the approach to detect concealed objects.
4.1.3.4.3 Command Module and Related Subsystems
Command Module
The Command Module (CM) manufacture and test concerns the procurement and building a system from COTS components to construct the CM (principally a PC and power supply parts). The CM hardware was assembled and tested. It includes the following hardware that was used in the test setup: Power distribution units; Ethernet router/switch, industrial PC with user interface hardware and interconnection cables.
The system control software is ready and has been tested with the real CM, PCS, HMI, support sensor and radar modules and with radiometer and AAD dummy software modules. The test setup and the software are working as designed. The software provides coordination, control and monitoring for the CONSORTIS system. A graphical user interface (GUI) was also developed for a system level control and maintenance purposes. A control-link communication specification has been made and documented, including a list of all control commands and requests, detailed control data format descriptions etc. The power and safety functions under the CM were also tested and programmed.
Passenger Control System
The Passenger Control system (PCS) hardware has been assembled and tested. The passenger control system functional testing was performed at first by using commercial Advantech’s AdamApax.NET control and measurement software package, and after this by the software integrated into the CM Python code. The PCS control-link uses Ethernet interface controlled by an industrial box computer. All the outputs (system status lights, audio alarm, traffic lights, power outputs, etc.) and inputs (four optical sensors, operator flow control switches, emergency switch, power output measurement readings etc.) are working functionally as designed.
After this the system control software (inside the CM) and its graphical user interface were used to control the PCS and read its sensors and other inputs. The PCS control software block uses Modbus compatible function sets through control-link with Gigabit Ethernet.
The Passenger Control System (PCS) hardware has been procured and is deployed according to the functional layout derived from the CONOPS.
Both the Command module and the Passenger Control System were fully integrated to the CONSORTIS system and successfully utilized during the field trials and the demonstration to control the system and guide the passengers/test persons through the scanning flow. The system flow is fully automated so that the operator only has to push “OK” from the GUI for allowing the next person into the system.
Support Sensor
Support Sensor manufacture and test concerns the procurement of a COTS support sensor. The chosen sensor, Microsoft Kinect v2, was acquired and tested and code to interface it with the system was written. Passenger Control System (PCS) manufacture and test concerned the procurement and building a system of COTS components to construct a PCS. The subsystem hardware has been assembled and tested.
AAD and HMI Interface
AAD and HMI interface development concerns the implementation of the data interface between the AAD, HMI and primary sensors and the implementation of the HMI. An HMI front-end has been produced and reviewed from an ethical and operator experience perspective. The HMI back-end and the data interface software architecture have both been implemented and completed. The architecture has been tested successfully with most key-components in the loop and with dummy modules where needed (the radar and radiometer were not yet interfaced with the data-link).
4.1.3.4.4 Highlights
Particular highlights of the manufacturing phase are given below:
Completion of radar 16 channel, 340 GHz, transceiver focal plane array, incorporating state-of-the-art transceivers and smooth-walled feedhorns. This is believed to be the highest channel count used in a radar at this frequency.
Completion of wide field-of-view focusing optics with high speed optomechanical beam steering capable of scanning at 10 Hz frame rate. This is believed to achieve the highest combination of frame rate, field of view and resolution of any mechanical beam steering system in a millimetre / sub-millimetre wave imager.
Completion of the 16 channel, 340 GHz radar subsystem and its successful integration into the CONSORTIS system..
Successful collection of state-of-the-art 340 GHz radar imagery acquired during data collection trials. More than 1000 data sets were collected by the radar during the trials in an operational scenario under system control. Additionally, hundreds of data sets were acquired during testing to investigate imaging response to a wider variety of targets, subject motion, subject tracking etc.
Development of bolometric kinetic inductance technology from single detector demonstrators into large-scale arrays. Prior to CONSORTIS the passive detector technology was demonstrated only with single detector characterization not integrated to any macroscopic optics system. In this project we demonstrated that large detector arrays can be fabricated and integrated with optics, and that the entity has capacity for concealed object detection.
Development of novel readout concept for kinetic inductance detectors. CONSORTIS developed a readout concept, with patent application filed, that enables the readout of large detector arrays. As compared to the competing techniques our solution enables the realisation of the readout system with affordable components, making it more attractive in commercial solutions.
Successful realization of state-of-the-art wide field-of-view dual-band inversely-magnified dual lens design for the radiometer. This doublet lens, manufactured from TOPAS™ material, focusses both the 250 and 500 GHz band radiation into the KID detector array and can refocus on moving subjects. The lens system was tested at a near-field measurement range at 500 GHz. The test procedures included re-focusing the lens system to the extreme distances designed, as well as the different sensor positions measured by offsetting the transmitter in place of the KID sensors. The constant beam properties were verified by measurements.
4.1.3.5 Field Trial
The objectives of the field trial were to develop and implement image processing and data fusion methods for privacy preserving detection of anomalies in multi spectral scanner data. In trial planning, input was collected from user requirements document (URD) and from external documents describing accepted test procedures (the International Wireless Industry Consortium (IWPC) protocol was mainly used). From this input, large efforts were made to produce a proper experimental design and compile a trial plan that would include all measurements needed to generate reliable system performance evaluations, but still would be possible to execute within the six-day duration of the trial. Additionally, administrative procedures were investigated and practical arrangements were planned to allow safe and secure handling of explosive materials during the trial.
A significant deviation from the original plan came with the consortium decision to re-locate the trial from FOI’s premises at Grindsjön, Sweden, to VTT’s premises in Espoo, Finland. The trial was then held at VTT premises on 13th -17th and 20th of November 2017. The trial was conducted using the CONSORTIS prototype system (except for the radiometer which was not operational) running in the intended CONOPS. On 13th -17th, the trial focused on tests with explosive simulants. The test plan was designed to span over the factors:
• Different body types: {average male, rotund male, average female}
• Different positions: {stomach, under of arm, back, inside of thigh}
• Clothing: indoor, outdoor (e.g. a pullover, no coats)
• Different simulants material: three different slab materials + four different granular materials
• Object size/weight: Small (250gr/ml), Medium (500gr/ml), Large(1000gr/ml).
In addition to simulants testing, tests were also made on a few chemicals and with different weapons. Test runs were also made on a variety of different ‘clean’ test persons. On 20th of November, the tests were performed with real plastic explosives (PENO).
Altogether, 1026 tests were properly collected and recorded within the duration of the trial. All tests have then been properly annotated and stored, together with their corresponding sensor data, in a database for easy access and later use. A method was also set-up on how to evaluate the data in a suitable way. Due to the fact that a fully automatic AAD method was not available, it was decided to evaluate trial data using a scheme in which the target/anomaly detection was operator assisted. An evaluation-plan was prepared to determine which subsets of the trial dataset were feasible to be used in an operator assisted setup. A framework to pre-process all necessary radar data presented to the operator was also constructed. Results from the operator assisted evaluation were collected and analysed.
4.1.3.6 Technical (Planned Airport) Demonstration
The planning of the airport demonstration was performed by FNA and VTT and an airport demo plan was prepared. FNA was ready for a demonstration at the airport. However, because of the delay in the development of the imaging radiometer and the automatic anomaly detection (AAD), and the consequent unavailability of those subsystems, it was decided not to have the demonstration at the airport. Instead, a laboratory demonstration for mostly technical specialists was performed at VTT in Espoo, Finland on 14 December, 2017. The arrangement in the demonstration was very similar as in the trial in WP7, but with a limited scale and documentation.
Potential Impact:
4.1.4.1 Dissemination
4.1.4.1.1 Stakeholder Engagement Strategy
The strategy used for dissemination was based on the need to engage with stakeholders and potential users and customers to exploit the results of the project, while at the same time protecting IP and the competitive position of CONSORTIS. Dissemination routes have included the public website, engagements with stakeholders as part of the user requirements analysis and exploitation strategy, presentations at appropriate conferences and peer reviewed publications. In total, there have been 63 dissemination events during the project and four events scheduled for after the project end.
Dissemination has been carefully controlled due to the overriding need to avoid premature disclosure of emerging results and the planned implementation of the CONSORTIS system, to avoid alerting competitors, and the need to protect IP at a stage when the technology is still immature. Therefore, dissemination has focused on presentations at key conferences to engage with stakeholders, rather than producing peer reviewed academic papers giving extensive descriptions of the approach and underlying know-how.
An approval process was implemented during the project to screen proposals for conference presentations and publication. As the design and development work progressed, technology advances that may require protection were reviewed to determine if the foreground IP developed was patentable, and any necessary IP protection activity was initiated. The volume of dissemination activity within these constraints has been considerable. Significant progress in the sub-systems, particularly the radar component, has been extensively disseminated via scientific presentations and papers, together with briefings for academic and government stakeholders.
Dissemination activity for the CONSORTIS system as a whole has been constrained by the delay in obtaining system performance data and the postponement of the Airport Demonstration. Consequently dissemination of CONSORTIS system results during the project has been limited to a Technical Demonstration targeting aviation technical experts but also involving some potential non-aviation users at VTT on 14th December 2017. A further demonstration was held to show the performance of the system to the European Commission Project Officer and Project Reviewer in February 2018, and system performance results will be used and disseminated in follow-on exploitation activities, in particular seeking Government and Industry support between January and December 2018.
4.1.4.1.2 Website, social media, publications
Website
The main responsibility for the development of the CONSORTIS project website lies with VTT, the Coordinator. The first version of the website was up and running on 31 March 2014 and has been updated three times during the project. The site contains a public section and a password protected partner-only section, which was originally planned to be used to exchange information within the project. However, VTT’s secure SharePoint server was found more practical for this purpose, and consequently it has been almost exclusively used for internal project information exchange purposes.
Use of Social Media
The importance of social media is increasing also in scientific and industrial communication. Because of the rapid developments, the use of social media in CONSORTIS was not envisaged in the original project plan made in 2011-2012. The other reasons not to emphasize the use of social media are the need to protect sensitive information and also because of the uncontrollable nature of information spread in said media. Those reasons still hold at the end of the project, and therefore social media did not become a mechanism for internal or external CONSORTIS information sharing during the project lifetime.
Publication of (Peer Reviewed) Journal Articles
Dissemination has focused on presentations at key conferences as part of the exploitation activities to engage with stakeholders, rather than producing academic papers. These were carefully controlled to avoid disclosure of details of the CONSORTIS technology to protect IP at a stage when the technology was immature. Peer-reviewed journal articles generally require more technical details than conference presentations. As a consequence, there have only been a limited number of peer-reviewed papers during the project.
Other Publications
The public sharing of project information has taken place in the following main channels: 1) Scientific journals, 2) Scientific and technical conferences and 3) Personal meetings with stakeholders. A wider publicity through trade press and other media has not been deemed necessary or even desirable, and has not been used.
4.1.4.1.3 Scientific Exchange
Interaction with other European Commission-Funded projects
The FP7 LASIE (LArge Scale Information Exploitation of Forensic Data) project invited the CONSORTIS team to participate in a project workshop in London on 15-16 July, 2015. The data being considered by LASIE include CCTV surveillance data. Since ethics and privacy were discussed there, Andreas Wolkenstein of EKUT, who is leading on these issues within CONSORTIS, attended the workshop. Ethical issues are an area where, more than in purely technological topics, common ground between the projects exists, and these were investigated in the workshop.
Discussions have been ongoing during the project with the FP7 TeraSCREEN (Multi-frequency multi-mode Terahertz screening for border checks). A joint demonstration to stakeholders was considered but not pursued as both projects had technological issues and did not reach the required maturity to make this appropriate. The TeraSCREEN project was invited to contribute papers to the European Microwave Conference Workshop in October 2017, which was organised by ISEC and Rohde and Schwarz.
Interaction with National Research Projects
There has been interaction with Academy of Finland, Centre of Excellence in Low Temperature Quantum Phenomena and Devices (CoE-LTQ). The Academy of Finland’s Centres of Excellence are the flagships of Finnish research. CoE-LTQ is formed by partners in Aalto University, University of Jyväskylä and VTT to pursue research in low temperature physics, technology and nanophysics. The detector solution chosen for the passive front-end in CONSORTIS has initially been developed within CoE-LTQ. The same technology is being applied in other VTT projects, for example in space applications.
4.1.4.2 Exploitation
4.1.4.2.1 Status
The overall project exploitation plan, described in submitted deliverables and periodic reports, comprised the following four elements:
1. Targeted dissemination of emerging CONSORTIS project results
2. Development and implementation of individual partner exploitation plans.
3. Demonstration of system functionality and performance at an airport
4. Development and implementation of a system level roadmap and plan for commercialisation and market entry
Significant progress in the sub-systems, particularly the radar, has been extensively disseminated via scientific presentations and papers, together with briefings for academic and government stakeholders.
Individual partner exploitation plans have focused on the development and protection, where appropriate, of partner IP, and the foreground IP developed during the project. Exploitation of the partners’ sub-systems and components beyond the end of the project, including plans for licensing to the system integrators, has been addressed.
System performance trials have been held and detailed plans were developed for the demonstration at Helsinki airport, as described in the Airport Demo Plan. However, the airport demonstration was postponed until the fully integrated CONSORTIS system could be tested. The exploitation plan was therefore revised to replace this with a Technical Demonstration.
The CONSORTIS system comprising the integrated radar, radiometer, AAD and passenger control system has reached a TRL of 5/6, apart from the Automatic Anomaly Detection (due to lack of data and time).
Five patent applications have been submitted, three of them concerned with the same invention. One of these three applications is listed in the patent table of this Final Report. The other two inventions could not be included in the patent table, because as confidential they did not yet have the required URLs.
4.1.4.2.2 Target User and other Stakeholder Groups
The target user and stakeholder groups have been revised in the light of market developments and the potential USPs of the CONSORTIS system for exploitation. These include government and law enforcement agencies concerned with both the aviation security and wider non-aviation people screening sectors.
The main stakeholder groups that have been targeted include:
• EU Legislative Authorities e.g. EC DG MOVE on EU aviation security policy, European Civil Aviation Conference (ECAC) on civil aviation security policies and practices.
• National Legislative Authorities e.g. Finnish Transport Safety Agency (Trafi), UK Department for Transport (DfT).
• National Security Authorities e.g. UK Home Office, UK Border Force.
• Airport Operators and trade bodies e.g. Finavia, Airports Council International (ACI).
• US Government agencies, e.g. Department of Homeland Security (DHS), Transportation Security Administration (TSA).
• Critical infrastructure agencies e.g. UK Centre for the Protection of National Infrastructure.
• Potential Industry exploitation partners, system integrators, etc.
4.1.4.2.3 Conclusions on Exploitation
The final exploitation plan has been made for commercialisation and market entry for the CONSORTIS technology solution and sub-systems developed by the project for people screening. It is an update of the second draft exploitation plan issued two years earlier and reflects market changes over that period, in particular in alternate markets other than aviation, and the technical advances made by the project that concluded with a technical demonstration in December 2017.
Significant progress in the sub-systems has been made and extensively disseminated via scientific presentations and papers, together with briefings for academic and government stakeholders. Individual partner exploitation plans have focused on the development and protection, where appropriate, of partner IP, and significant foreground IP has been developed during the project. Exploitation of the partners’ sub-systems and components beyond the end of the project, including plans for licensing to the system integrators, has been addressed.
Exploitation activities in the final two years of the project have included attending relevant events involving potential exploitation customers, partners and competitors to gather market intelligence, and this has been used to shape the commercialisation plan. System performance trials have been held and detailed plans were developed for a demonstration at Helsinki airport, as described in Airport Demo Plan. The original strategy was to use the Airport Demonstration as the main mechanism to engage key stakeholders in support of the exploitation plan. However due to delays in the development of the passive radiometer system which delayed the gathering of system performance data it was not possible to develop the CONSORTIS system to a sufficiently advanced level that would be credible to Airport customers. The Airport Demonstration was therefore postponed from October 2017 and a Technical Demonstration involving aviation customer and industry technical experts as attendees was carried out at VTT in December 2017.
At the culmination of the project (December 2017) the CONSORTIS system has reached a TRL of 5/6 apart from the Automatic Anomaly Detection (due to lack of data and time), but without a demonstration being made in an operational environment (TRL7) at an Airport. Given the complexity and risk of the project it is not surprising that this point has not been reached. However the results of the technical demonstration are sufficiently promising that partners have agreed that further investment funding will be sought to take the CONSORTIS concept/technologies forward.
4.1.4.3 Ethical Aspects
4.1.4.3.1 Introduction
As a part of the wider societal implications of the project, research partner EKUT was tasked with conducting ethical research about the CONSORTIS system. The work was divided into two parts. The first consisted of an overview of ethically relevant values with regard to security and security scanners. The second part consisted of two empirical studies examining the so-called fair process effect.
4.1.4.3.2 Security and Values
In the first part of the work an overview of what values are was presented. Based on a distinction between the act of valuing, i.e. holding something valuable, and the object of values themselves, a working definition of values was given:
“Values are entities that are judged to make a positive contribution to the judger’s fulfillment of motivational psychological states if brought about, realized or expressed through one’s actions or character dispositions (a process called valuations, or attributing value). By reference to values we express a normative recommendation with differing force concerning a practical decision. Values serve different functions: among others, they can serve as reasons for actions (in the form of orienting action, but also in the form of criticizing actions).”
The action-guiding nature of values is hampered by the existence of value-pluralism. Since people differ in their valuing things, they will inevitably assume the existence of a variety and plurality of values. Value-pluralism might be overcome by certain (argumentative) strategies, but we still need to expect that there are situation where there is insurmountable conflict between values. This holds true for both the intra-individual case (where a person cannot chose between to action-guiding values regarding a given decision she has to make) and the inter-individual case. This means that different people might vary between what they think are important or even final values.
Undeniably, value-pluralism is a fact in the real world. However, this does not preclude us from assuming the existence of value monism at the foundational level. A theory of value-monism thus understood is not about the relation between individuals and the things they value (and subsequently understand as reified value), but about the source of value. Here, the idea was presented that the notion of aim-fulfillment provides the basis for valuing and values. In other words, every instance of valuing something (and holding that something as a value or seeing it in the value-perspective) is indexed to a person who, by valuing something, fulfills her aim of leading a good life. Therefore, leading a value-based life is exactly what gives values their basis and resource and, as we shall see, provides the resource for solving inter-individual pluralism.
Inter-individual value pluralism is particularly important in the domain of (homeland, national) security since security itself concerns many people. When we think about putting value to security (and to derive political consequences), different people will come to different conclusions. The tradition of political philosophy has discussed this problem which might even be considered to be at the roots of political philosophy itself. Various ways to deal with the problem of value pluralism have been proposed: democracy, cost-benefit analysis, public reason and libertarianism. Each of these solutions exhibits valuable insights, but in the end they fail to appreciate the most basic value, i.e. the individual person leading a good life. The solution that was pursued in light of these reflections involved the idea of individuals being able to exert a strong influence on inter-individual decision-making processes, an influence that is not granted by democracy alone, not by other forms of political organization. An ideal, but somewhat utopian solution to the problem of inter-individual pluralism consists of introducing flexibility and competition in the political process. Another insight derived from these theoretical arguments is that many problems with security and technology do not (only) address particular technologies, but decision-making processes as well as political and social institutions. And it makes room for the idea of looking at processes and their ethical features.
Given these rather theoretical remarks, a closer look at the issue of security (technology) revealed the existence of the following values and their importance for guiding the process of producing technology and making decisions: Privacy, discrimination, health, democracy, proliferation, responsibility, corrosion of civil liberties, cost-benefit analysis, passenger convenience, right to travel and costs. This initial list provided the background for a final list of ethical values in security technology as given below:
The identified list of ethical values in security is given in the following:
Security level
A certain level of security must be secured, especially if one technology (FBS, Full-Body Scanner) is thought to replace another (metal detector).
Action points: Tests and comparisons, societal participation in defining acceptable risk.
Liberty
Liberty as the central political value is regulating interactions between people and between the state and citizens.
Action points: Reconsider security policy-making by allowing participatory elements. Respecting rule of law.
Privacy (related: health issues, anti-discrimination)
Is related to liberty in that it defines what liberty is all about: choice, self-direction, data protection through control rights over information.
Action points: Implementing privacy-by-design features: disabling storage (of raw data), disabling connectivity with alternative means. Providing choice of security measures as demanded by customers; using FBS along with other technologies to enhance choice.
Responsibility for acceptable use
Individual businesses are responsible for rights-respecting use through Corporate Social Responsibility (CSR).
Action points: Codes of conduct for responsible business behaviour. Subscribe to European human rights standards.
Procedural justice
Fairness of security procedures involving FBS must be respected in order to respect individuals and to reach acceptance.
Action points: Design security procedure with elements of procedural justice: Allowing voice, respectful treatment through security personnel, enhancing trust in authorities through transparency.
4.1.4.3.3 Procedural Justice
The second part of the ethical research consisted of empirical studies about the fair-process effect. It thus follows up on the idea that procedures matter. Political procedures could not be studied in the project. Instead, the procedures at airports where security scanners are used were focused on. The main justification for pursuing this path is given below:
(1) Procedural justice accounts for many (positive) effects in decision-making contexts such as organizations and the workplace justice and law enforcement. Knowledge about fair procedures, together with the positive effects they have, might then lead to an improved way of introducing and using security scanners. Since acceptance even of non-beneficial decisions is enhanced through fair procedures, making people get screened by a (often) negatively viewed scanner might also become more accepted through fair procedures.
(2) It provides a novel approach both to learning about what is going on in people’s mind when they enter airport security and to make practical recommendation about how to organize airport security. This is particularly the case given the fact that in this study, factors driving people’s susceptibility to the fair process effect are being examined. A security architecture that takes these facts into account can thus be tailored to the specific needs of different groups of people.
(3) Existing research and the insights we can derive from it for practical purposes (e.g. crisis and risk communication) deserve further and more detailed analysis in order to qualify what we can say about the (practical) value of procedural justice.
(4) Finally, respecting people’s fairness-related preferences is required by moral reflection and is called for by recent politico-philosophical reasoning. Moreover, the beneficial effects of procedural fairness are not a way of tricking people into accepting what they would not accept otherwise. The point of procedural fairness is that it respects people’s preferences and views and is thus fully compatible with autonomy and freedom of choice.
In summary, the present research contributes to a body of knowledge that relates procedural fairness with people’s acceptance of security measures and technology. It improves our empirical knowledge, which in turn can influence ethical judgments. Finally, it sheds light on a particular form of human-policy-interaction.
4.1.4.3.4 Study 1
We conducted two studies. In the first study, we were interested in finding out whether or not fair procedures lead to people’s acceptance of security scanners. We hypothesized:
(1) The Fair Process Effect (FPE) will also occur at security controls in airports, that is: Test subjects that were given a choice will perceive the situation as more fair and it will be more likely for them to accept body scanners than subjects who didn’t have a choice in the matter.
(2) Fair procedures will have a larger effect in a low privacy condition than in a high privacy condition, which means that the difference of acceptance ratings between fair and unfair treatment will be larger within the low privacy condition than within the high privacy condition.
Manipulating choice served as an instrument to express fair procedures, where the existence of choice represents fairness, according to previous insights from psychological studies. Moreover, we wanted to show that fair procedures are particularly important when it comes to negative evaluations of security scanners, hence the manipulation of privacy conditions. Note that the study was not intended to represent a realistic scenario of using the CONSORTIS system, but to find out about underlying psychological processes in terms of accepting security technology.
Unfortunately our hypotheses were not confirmed, a conclusion that study 1 summarizes. However, future studies are needed to improve the study’s design since the fair process effect is well-known in other domains so that it would be surprising not to find it in security procedures.
4.1.4.3.5 Study 2
The second study followed up on the first one while introducing new parameters that might have an influence on the fair process effect. The hypotheses tested were:
(1) People with low numeracy tend to overestimate a security scanner’s effectiveness
(2) Procedural fairness trumps effectiveness ratings when people judge their acceptance of a security scanner, meaning that people who face a highly effective scanner are less likely to accept it if they perceive themselves to be treated unfairly
(3) People with low numeracy pay less attention to procedural fairness, meaning that the effect described in (2) is less strong for those people who are less numerate.
Numeracy, i.e. the ability to deal with numerical information (i.e. calculate probabilities correctly etc.), might be particularly important since security itself is a matter of risk and probability. Both the risk of, say, terrorism, is subject to probabilistic reasoning and risk perception, but also the perception of a scanner’s capacity to actually provide security (given information about false positives, false negatives, and the probability of detection).
Again, the hypotheses were not confirmed. This might be due to decision about the study design and information that participants received. Further studies are needed to improve the study and examine a psychological effect that is needed to enhance acceptance while at the same time respecting individual’s choices and values. This combination of acceptance and people’s values is what links the empirical studies back to what was said in the theoretical ethical reflections in the first part of the ethics work.
4.1.4.3.6 Conclusion on Ethics
The general conclusions of the CONSORTIS ethics research are:
1. More research is needed to examine the fair process effect in the domain of security. This can help to enhance acceptance in a way that is fair, open, and transparent. This, in turn, improves a societal discussion about security and will make societies more secure. Therefore, political decision-makers should have an enormous interest in funding more research on security and the fair process effect.
2. More research is needed to define the role an ethics of security should assume. As was repeatedly indicated, academic ethics should not only speak with (and to) engineers and their product, but also to those who are responsible of funding the engineers and make political decisions. But how exactly does research policy in security look like when this is taken seriously, i.e. when ethics is included in the process about how to design research policy itself? And how can ethical insights into research and security policy be inserted in the political institutions responsible for those policy domains?
3. Security technology such as the security scanner need to be assessed as part of a larger security architecture. Since, for instance, data provided by the scanner can be put in relation with other data (from CCTV etc.), a larger security apparatus emerges. Ethics needs to take this seriously. One way to do so would be to fund ethics projects not as part of a technology project, but as cross-sectional. Ethics projects would accompany several technological research projects and study the interaction between various measures; they would reveal problems of increasing security by combining technology and data, but also find ways to ease these problems. And through being in close interaction with these projects, they would be close to the technological reality while being able use their insights to feed the political and societal process.
List of Websites:
CONSORTIS website: http://CONSORTIS.eu/
Email contact: consortiscoordinator@vtt.fi
