Final Report Summary - HECTOS (Harmonized Evaluation, Certification and Testing of Security Products)
Executive Summary:
HECTOS is an EU FP7 security research project exploring the issue that there are very few evaluation and certification procedures for physical security products which are mutually recognised by EU Member States. This leads to fragmentation of the market, as identified in the EC Communication on Security Industrial Policy, with negative impacts on both suppliers and users. The HECTOS objective is to support the harmonisation of the European market for physical security products, by producing a certification framework and template for these products as well as a roadmap for the implementation of the framework.
Physical security equipment is very diverse in technology, concept of operation, application area and performance, and similar security products are difficult to compare in terms of performance, accuracy, usage, trust and validation of functionality. Furthermore, in a number of product areas, there is a lack of product and measurement standards at European or International level which are a necessary prerequisite for harmonised certification schemes.
The HECTOS project focuses on work towards harmonisation of evaluation and certification schemes for physical security products and has developed a framework for conformity assessment systems and schemes for this group of products. It has explored how the international standards for conformity assessment (ISO/IEC 17000 series) and the existing certification systems used in other areas could be applied, adapted or developed for products used for security of people, property and infrastructure. This has been achieved by taken into account the particular security-specific features that need to be built into performance measurement and threshold performance schemes for physical security products to address:
• Realistic and adversarial testing
• Consistency in testing – in particular for adversarial testing
• Measurement of complex performance parameters
• Security sensitivity
• Continually evolving requirements
• The wide range of applications and performance requirements
• The diverse range of types of product, markets and levels of maturity
The HECTOS harmonised certification framework comprises a three level structure and approach which can accommodate all types of physical security product and application, covering all forms of conformity assessment including pass/fail testing against a threshold and performance measurement schemes.
The development of the framework and its supporting implementation template has been supported by experimental case studies for Explosives & Weapons (E&W) and biometric recognition products and validated by applying them to several different product groups as case studies including; E&W detection, biometric recognition, CBRN detection, locks and Video Surveillance Systems.
The framework and template for the establishment of evaluation and certification schemes for physical security products are the main content of a CEN/CENELEC Workshop Agreement (CWA), initiated by the HECTOS project. The CWA was developed by the project team together with a wide range of stakeholders such as representatives from certification bodies, industrial associations and standards bodies. The CWA published as CWA 17260:2018 is downloadable at no cost from the CEC/CENELEC website.
Project Context and Objectives:
HECTOS is a European project focusing on harmonisation of evaluation and certification procedures for physical security products. Physical security equipment and systems are very diverse in technology, concept of operation, application area and performance, and similar security products are difficult to compare in terms of performance, accuracy, usage, trust and validation of functionality. Currently, there are very few test, evaluation and certification procedures in Europe that are mutually recognised by different Member States (MS). This leads to fragmentation of the market, as identified in the 2012 EC Communication on Security Industrial Policy, with negative impacts on both suppliers and users. Through the harmonised approach developed in HECTOS, product functional performance and certification schemes can be recognised, leading to a more homogeneous market with enhanced possibility to demonstrate and compare physical security products.
Focussing on the functional Performance of Physical security Products (the so-called HECTOS ‘three Ps’), the project has investigated conformity assessment schemes both for consistent performance measurement and for conformity with threshold performance values. The scope of the project covers physical security products used for security of people, property and infrastructure, including:
• Barriers
• Access management
• Surveillance
• Detection equipment
This wide range of products and applications, the need to operate in both regulated and unregulated environments as well as products with very different maturity and market sizes, means that a range of different types of scheme is needed. HECTOS covers security products which protect against criminal and terrorist attacks, rather than products for safety, such as protection from natural disasters. The project has not focussed on certification of other product attributes, for example interoperability or safety, nor does it cover the evaluation and certification of security systems, security services or cyber security – although these topics are also important and need to be addressed in other projects.
HECTOS has explored how existing certification schemes and systems used in other areas could be applied, adapted or developed for products used for physical security by taking into account the particular features which applies to certification of security products. HECTOS has identified the current state-of-play and the level of harmonisation across all types of physical security product and has developed a roadmap showing how harmonised European certification systems and schemes could be introduced.
Accommodation of the disparate types of security product, whilst enabling harmonisation is proposed by ensuring or encouraging:
• Performance measurement schemes which provide certified measurement of performance attributes, as well as threshold performance schemes which certify conformity with defined threshold performance values
• Common definitions of performance attributes and their evaluation methods in measurement standards or test methods
• Common threshold minimum performance values or grade definitions for conformity assessment
• Common, or mutually accepted, processes for evaluation, certification and accreditation to ensure consistency over time and between participants
• Common requirements and mutually accepted processes for ongoing surveillance to ensure the consistency of product manufacturing
• Common or mutually accepted security mark and/or database
Security products differ from other types of product in that they help protect against attacks by an intelligent adversary with malicious intent. Therefore, a number of security-specific features need to be built into performance measurement and conformity assessment processes in security certification schemes:
• Realistic and adversarial testing – users need to understand how a product will perform in real-life situations
• Consistency in testing – tests involving human strength and skill need to be standardised. Procedures are needed to ensure the consistency of tests over time and between test laboratories
• Complex performance information – performance attributes such as detection rates are often complex and need to be precisely defined
• Security sensitivity – it may be necessary to restrict access to performance data and test methods to avoid revealing weaknesses and gaps to a potential attacker
• Continually evolving requirements– changing threats as attackers adapt to exploit weaknesses in security systems mean that security product requirements are always changing
• Wide range of applications and performance requirements – many security products are used in a variety of different applications subject to different threats. Schemes need to be carefully managed to ensure testing covers the range of requirements without imposing too large burden on manufacturers
• Diverse range of types of product, markets and maturity – from low-cost consumer products selling in large volumes to high-end and costly products with only a handful of sales per year, mean that schemes need to be based on what the market needs and can support.
The following sections summarise the main activities in HECTOS.
Review of schemes and requirements
A review of physical security products was made early in the project. The approach used a set of categories based on the functions that the products provide as well as taking into account the categories used by stakeholders in the EU and around the world. The survey also took into account categories/structures established by European standards bodies. This was followed by a review of application categories and scenarios in which these products are used. Six application categories and eighteen underlying scenarios were identified and described, including an assessment of the market size. These were used to help guide the work and ensure that a wide range of different application needs were taken into account.
During the first year of the project, stakeholder requirements for physical security product evaluation and certification schemes were also identified and analysed. This survey was based on a questionnaire, interviews and a workshop held with a wide range of stakeholders. Input from previous and ongoing work on certification schemes in the EU was also been taken into account. The requirements were grouped into standards-, evaluation-, certification-, and accreditation-related requirements. Product-specific requirements were also identified.
Certification systems are often based on product and measurement standards, including performance requirements and test methods, respectively. The landscape of existing standards for physical security products were therefore investigated. This revealed product areas in which standards are lacking or insufficient in terms of performance requirements, or areas in which there are many (possibly overlapping) standards. It also provided an overview of the use of European standards versus other standards such as industry, national and international standards.
Based on the initial surveys on products, applications, standards and stakeholder requirements, HECTOS studied existing certification systems. This provided knowledge on how certification systems could be adopted, adapted or developed for products used for physical security. More than 20 certification systems in use in different countries in Europe and around the world were analysed. Most of them were deployed for certification of security products, but also systems for non-security products were studied in order to understand the features which potentially could be applied to the security sector.
Framework development
During the initial surveys, it was understood that a single certification system for physical security products cannot accommodate the wide range of products and applications, varieties in market size and different level of maturity between product types. In order to accommodate these varieties, HECTOS decided to develop a generic certification framework for security products with guidelines (template) on the establishment and maintenance of systems and schemes. It was found that the generic ISO/IEC 17000 series of standards for conformity assessment, based on the functional approach, is widely applied and involves mechanisms for evaluation and certification, also applicable for certification of physical security products. Thus, the certification framework developed in HECTOS is based on this ISO/IEC 17000 series of standards supplemented with features that focus on security-specific aspects, which currently make certification of security products challenging.
The HECTOS framework accommodates:
• A three-layer structure: system group, systems and schemes
• System and scheme key building blocks
• Security specific features
• Performance measurement schemes
• Threshold performance schemes
• System management structure
• Actors and roles, including System Committees and scheme Working Groups
• Recommendations on a single security mark with system and scheme identifiers
• A common database for certified physical security products
• Controlling documents for requirements as well as operational processes and procedures
The HECTOS certification framework is supported by an implementation template covering topics to be considered when establishing a scheme as well as guidance on the maintenance of systems and schemes, including operational and management features. The development of the certification framework and template was monitored from an ethical perspective by the ethics experts in the consortium and issues were raised and discussed when necessary.
The HECTOS certification framework and template for establishing a scheme became the basis for a CEN/CENELEC Workshop Agreement (CWA 17260:2018), initiated by HECTOS and developed together with external stakeholders.
Experimental investigation
Two experimental case studies were carried out in order to assist the development of the certification framework and template. These studies focused on Biometric products and Explosives and Weapons (E&W) detection equipment and covered primarily evaluation (testing) aspects. Additional scheme elements were included as paper studies.
The case study on biometric products analysed three topics:
• Image quality of contactless fingerprint sensors
• Presentation attack detection/resistance of fingerprint systems
• Secure biometric access control
Certification criteria across multiple technologies, harmonisation of requirements and inter-laboratory consistency of adversarial testing are examples of issues explored in detail.
The E&W detection equipment case study examined explosives trace detectors (ETD) and people screening portals. Elements in test methods critical for inter- and intra-laboratory consistency as well as the applicability of Receiver Operating Characteristic (ROC) curve-based evaluation are two examples of topics investigated.
Validation
A final validation of the HECTOS certification framework and template was performed by considering how they could be applied to a number of security products:
• Biometric products
• E&W detection products (aviation security and non-aviation security)
• Radiological and Nuclear detection products (for illicit trafficking)
• Chemical and Biological detection products
• Locks
• Video Surveillance Systems (VSS)
These paper studies fulfilled two purposes: (i) they provided valuable feedback to the refinement of the framework and template (and thus the CWA), and (ii) they provided guidance to the development of the implementation roadmap for individual certification systems and schemes. The studies applied the template for establishing a scheme and assessed the current status of harmonised certification for the different types of product. Opportunities and challenges for further implementation of harmonised certification systems were also investigated.
Roadmap
The HECTOS view on how the European standardisation, evaluation, certification and accreditation for physical security products can be taken forward within a holistic framework was finally studied and presented. This includes recommendations on actions to be taken, how they can be taken and in what order. A suggested design of the three-layer structure of system group, systems and schemes was elaborated, to show how the framework could accommodate all the different categories and applications of physical security product.
A high-level so-called enabling infrastructure roadmap was developed which focussed on the major activities to be taken by the stakeholder community in order to implement the HECTOS certification framework. The finalisation of the HECTOS CWA is a first step in this process.
Conclusions from the paper studies described above were summarised in a number of system roadmaps describing the phased development of certification systems and schemes for individual product categories.
Project Results:
Security product and application surveys
The objective of this activity was to carry out a broad survey of the different categories & types of physical security product as well as the scenarios in which they are used and their evaluation and certification requirements.
Security product survey
The security product survey provided a foundation for the work of HECTOS by categorising and describing the different types of physical security product that are used in the provision of physical security solutions and describing their principal characteristics. The aim was to compare and contrast the different categories to support the development of the harmonised certification framework and template.
The work included definitions and scope of physical security and physical security products as adopted by HECTOS, and provided an overview of physical security products and their categorisation. Each product category was then analysed and broken down into product types together with a description of the key characteristics of each product type. The product categories are:
• Barriers (Fences & gates, Building components, Vehicle barriers)
• Access Management (Locks, Safes and Security containers, Access control systems, Biometrics)
• Surveillance (Video Surveillance Systems (VSS), Security lighting)
• Detection equipment (Intruder detection alarms, CBRNE detection)
Under each of these product category, there are a number of identified product types (also sub-types where appropriate) which have been described in more detail. A number of relevant characteristics of each category and type are given in terms of product function; technologies used, application, market and user aspects. The product survey was used throughout HECTOS as the basis when e.g. studying the standards landscape and the development of the implementation roadmap.
Security application survey
The security application survey identified the principal security application categories and defined a set of typical application scenarios which was used to help identify product evaluation and certification requirements. They were chosen to be representative scenarios which ‘span the space’ of security application areas and product types.
The application survey described eighteen application scenarios which, whilst not exhaustive, are representative of security requirements and the way that physical security products are deployed to help fulfil these requirements. The scenarios were briefly described in terms of the application area, the types of threat that exist, the physical security measures that are deployed to mitigate the threat and other relevant information about regulations, standards, operator requirements and the impact of the measures on users and others.
The survey also included an estimate of the number of instances of each scenario in the EU and the value of products deployed. The application scenarios guided and supported the research carried out during the HECTOS project, for example to guide and illustrate the development of concepts used in the experimental case studies.
Stakeholder requirements
Requirements for standards, evaluation methods and certification schemes across the different categories of physical security products and application areas were identified. The requirements were synthesised from a number of sources:
• Previous and ongoing work, where available, including other EU research projects, studies and initiatives
• Stakeholders, through a questionnaire, interviews and a workshop
• Consortium knowledge and experience as experts and participants in the development, test, evaluation and use of security products from different categories and application areas
Requirements were presented and discussed from the perspective of the principal stakeholder groups including end users, specifiers, their representatives and advisers; product manufacturers, distributors, system designers and integrators; others with a financial stake such as insurers; standards, evaluation and certification bodies; national and EU governments and regulators. Requirements were grouped into standards-, evaluation-, certification-, and accreditation-related requirements. Product-specific requirements were also identified.
It was found that requirements often vary between different product categories and between application areas. These differences are also identified, since this is important information for investigating the applicability of different potential certification systems and whether or not schemes for products can have similar rules and procedures and thus form a certification system. This was further explored during the development of the HECTOS certification framework and implementation roadmap.
Review of standardisation landscape
The objective of the standardisation landscape review was to get an overview of the maturity of the standardisation activities for the different product categories. The survey revealed details on sectors where standards are existing and utilised, as well as areas where standards are missing or insufficient.
The results of the standards research were divided in different fields according to the defined product categories and the analysis considered national, European and international standards. Standards developed by user and specifier oriented industry organisations and independent certification bodies (such as VdS, LPCB, SSF) were also identified.
It was difficult to carry out a survey across the whole physical security area but the survey provided a reasonably complete and representative picture of the state of standardisation in each of the HECTOS product categories and it is probably the most complete survey in existence.
Taking the HECTOS product categories in turn:
Barriers
• Fences & Gates There appear to be no European or International standards covering fences and gates. There are standards at a national level as well as other standards from governmental organisations and building-industry organisations such as LPCB, particularly for high security applications.
• Building Components A range of European standards cover security functionality of building components such as doors, windows, walling, grilles and shutters. They include standards for burglary/intrusion resistance, bullet and explosion resistance. Various national and other standards also cover this area, especially to define more stringent requirements for higher security applications.
• Vehicle Barriers There is a CWA and IWA which provides a test method for impact crash tests, developed in turn from two UK PAS and elements from an ASTM standard. This provides a good basis for mutually accepted testing. There are however no standards on performance and other performance aspects.
Access Management
• Locks This is a very mature area both for products and standardisation. There are European standards in place with requirements for locks providing different grades of security. As with several other categories, there are national and other standards (e.g. LPCB, VdS) that provide requirements and test methods for higher grades of security – particularly against lock-picking as well as the use of force. There is also a comprehensive range of lock standards in the USA. There do not appear to be international standards in this area.
• Safes and Security Containers This is another mature area. Standardisation is similar to the locks category with European and US standards, but none on an international level. It appears that the requirements and security grades in the EN standards meet the needs of most types of user.
• Access Control Systems In the field of access control systems shows some relevant European standards as well as some of relevance from CPNI (Centre for the protection of National Infrastructure, UK). Some relevant standards can be found as well in the product category of biometrics.
• Biometrics There are comprehensive IEC and ISO standards across many areas of biometric technology. These standards are widely accepted in Europe but are not formally adopted as EN standards. European standardisation activities (i.e. by CEN/CENELEC/ETSI) are in their infancy, but there are some activities starting especially in the automated border control application area.
Surveillance
• Video Surveillance Systems There is the newly adopted EN 62676 family of standards covering Video Surveillance Systems (VSS) based on the IEC 62676. These focus almost exclusively on interoperability aspects and do not cover functional performance. The emerging video analytics area for which functional performance is less well covered by performance standards and test methods. There are some national test and evaluation programs of note, such as the UK iLIDS and US activities at NIST, but no international or EU standards.
• Security Lighting There are no standards specifically covering the requirements of lighting products used for physical security applications. However, there is no perceived need for these, since the requirements are very similar to those which can be found in other lighting applications.
Detection
• Intruder Detection Alarms are covered by a comprehensive series of EN standards which mirrors a corresponding series of international IEC standards. These provide a graded set of system and component product requirements and test methods. Some Member States have a small number of country-specific or additional requirements leading to separate national standards (for example to meet the needs of local policies and procedures for police response). It is possible that some standards in the national or other category provide more comprehensive test methods than those in the EN/IEC standards.
• Chemical Threat Detection There are no European or International standards in this area and only very limited coverage in other standards (just three ASTM standards covering chemical vapour detection products).
• Biological Threat Detection There are no European or International standards in this area and only very limited coverage in other standards (just one ASTM standard).
• Explosives & Weapons Detection There are no European standards for explosives and weapons detection products and only two IEC standards, which apply to just one of the many product types in this category. Products of various types used in aviation security, which is the largest market for explosives detection equipment, are covered by ‘other’ standards in the form of the EU regulations on aviation security equipment and the ECAC test methods, both of which are classified documents. Outside aviation only a limited number of standards exists; that is for people screening equipment, X-ray systems for baggage/cargo inspection, and Explosives trace detection.
• Radiological & Nuclear Threat Detection is well covered by international, mainly IEC, standards. A number of these have been formally adopted as EN standards.
From the above, it is clear that there are several areas where there are few standards at the European or International levels. Clearly, if standards are not available or adequate, it will be impossible to implement harmonized certification schemes.
There are a number of areas where the requirements of particular application areas, such as high security levels demanded by government, critical infrastructure and other users, have led to standards which add additional requirements to those in the European standards and/or more rigorous tests. These test often involve more realistic (if less objective and repeatable) tests in addition to those in the EN standards. This needs to be investigated since it limits the degree to which product evaluation and certification to EN standards is sufficient as the basis of a European ’quality mark’.
Review of existing certification systems
A review of existing certification systems and schemes was carried out in order to provide a categorised list of evaluation and certification schemes used in the area of physical security products, on European and national level. The aim was to describe each evaluation and certification scheme by several common elements, both in tabular and in narrative format, and by the test method that is involved. The final goal was to enable a comparison of existing schemes and to identify common and missing process elements. The overview and comparison of the evaluation and certification schemes was used for the development of the certification framework and implementation template. The results provided an overview of certification systems/schemes in a number of European countries (driven by the consortium composition) as well as relevant schemes in the USA. It also included approaches that already take steps towards harmonisation of certification systems, like for example the European Fire and Security Group (EFSG), Keymark and the framework for conformity assessment presented in ISO/IEC 17067. Furthermore, some evaluation and certification systems outside the security domain are given as well since these provided interesting elements that can also be used for the HECTOS harmonised approach. Taking the HECTOS product categories in turn, most products are certified through schemes operated at a national level where they apply their own marks which are mostly recognised at a regional level. For some product types, such as chemical and biological detectors, no certification system or scheme exist neither at national or European level.
What is special about Security?
Security products differ from other types of product in that they help protect against attacks by an intelligent adversary with malicious intent. Attackers are likely to:
• Constantly probe for and exploit weaknesses in the products
• Attack in predictable and unpredictable ways
• Constantly adapt and change their method of attack
Consequently, schemes need to focus on realistic and adversarial testing; support evolving threats and requirements; and support mechanisms to handle security sensitive information. Products of a given type may also have a very wide range of performance requirements depending on the application. Therefore, a number of special features need to be built into performance measurement and conformity assessment processes in security certification schemes, in order to take these aspects into account. These include features covering:
• Realistic and adversarial testing. Performance is often defined in terms of the protection from real threats, for example how long a barrier resists an attack. Users require realistic testing and, in some cases adversarial testing where the tester exploits weaknesses in the product in the same way as an attacker.
• Consistency in testing. Tests involving human strength and skills need to be standardised, for example so that testers do not apply too much or too little strength. Procedures are needed to ensure testing remains consistent over time and between different test laboratories.
• Complex performance information. Performance information such as detection rates is often complex and test methods needs to be precisely defined.
• Security sensitivity. It may be necessary to restrict access to product performance requirements, test methods and test results in order to prevent weaknesses and gaps from being identified and exploited by an attacker.
• Continually evolving requirements. Changing threats lead to constant change in product requirements. Standards and test methods for security products need to be updated regularly to accommodate these changes, without placing too large burden on manufacturers
• Wide range of applications and performance requirements. Many security products are used in a variety of different applications subject to different threats. Schemes need to be carefully managed to ensure that testing covers the range of requirements and that there is clarity over the applications for which a product is and is not certified.
• Diverse range of types of product, markets and levels of maturity from low cost consumer products selling in millions to specialist equipment with only a handful of sales per year. Certification schemes need to be based on what the market needs and can support.
The certification framework and implementation template described in the following sections accommodate these features.
The HECTOS framework for harmonised certification of physical security products
The HECTOS certification framework was developed to provide a mechanism for the creation of harmonised conformity assessment schemes that enable the mutual recognition between EU Member States and other participants, support a Single Market and international trade and enable end-users to implement better security capabilities to mitigate the risks they face.
The certification framework is based on the ISO/IEC 17000 Conformity Assessment family of standards, adapted and supplemented by features to support the special requirements of security products. The certification framework comprises:
• A top-level coordinating structure
• Certification systems for related product and application areas
• Certification schemes with common rules and procedures applying to all the products in the scheme
The framework accommodates all forms of conformity assessment including pass/fail testing against a threshold and performance measurement schemes. Typically, for a threshold performance scheme, product or application requirements will be set out in a specification standard. For a performance measurement scheme, the evaluation process will be defined in a measurement standard (also known as test method).
The system group level provides an overall common structure and a security mark - a security-specific quality mark indicating that a product has been certified according to the common framework. A database over certified products should be available and administrated at system group level.
Individual systems manage the certification of different product groups (and sometimes applications) each having their own specialist subject matter expertise and application requirements.
Schemes within each of these systems certify one or a group of related product types each with its own standards and/or test methods, setting out a specific set of requirements. These individual product certifications within a scheme are also defined by other controlling documents for ensuring consistency of conformity assessment activities.
So-called System Committees and scheme Working Groups bring together the product-specific technical expertise to manage and maintain systems and schemes. They link with standards bodies to help maintain and facilitate updates to standards.
At the system group level, a framework set of documents will exist, describing overall basic rules for participation and the terms for usage of a common security mark. The system level will further develop a set of key documents describing common rules and procedures for all constituent schemes.
Each product certification within a scheme is defined by a set of controlling documents which set out the specific technical product and evaluation requirements as well as the product-specific technical aspects of operational processes and procedures such as proficiency testing and surveillance. Threshold performance schemes have a specification standard, which sets out the functional performance (and other) requirements, together with a scope defining the product types and applications to which the certification applies.
Performance measurement schemes are defined by a measurement standard (test method), which needs to be sufficiently detailed to remove ambiguities which might cause inconsistency in application.
Proficiency testing documents define how intra and inter-laboratory comparisons, peer assessment and other techniques should be carried out to ensure consistency over time and across test laboratories. Surveillance methods, where included in the scheme, define technical details of procedures to ensure consistent production quality over time.
The templates for the establishment and maintenance of schemes
The HECTOS certification framework accommodates a wide variety of schemes, defined by product type and/or application area. To support its implementation, two templates have been prepared providing guides and step-by-step checklists of the activities that need to be carried out in establishing and maintaining schemes and systems within the overall framework. The templates describe the steps that need to be considered to facilitate consistent implementation of systems and schemes. It consists of nine activities and designed as a “pick and choose” guideline – i.e. schemes that do not benefit from all of these prerequisites do not need to apply them. The activities are:
I. Identify Scope
a. Product type(s)
b. Application
c. Identify purpose of scheme
II. Identify scheme fundamentals
a. Establish the prospective scheme owner and a preliminary working group
b. Identify stakeholder groups
c. Perform an initial survey of existing standards and requirements landscape
d. Identify a financial model
III. Identify system structure
a. Identify existing systems
b. Identify system owner and management
c. Identify security specific management
IV. Identify standards
a. Identify relevant product and measurement standards
b. Identify harmonised and local standards
c. Identify need to develop new standards
d. Review stakeholders acceptance of identified standards
V. Identify detailed requirements
a. Establish performance measurement and threshold performance scope
b. Identify functional & non-functional requirements
c. Identify threshold performance and performance measurement requirements
d. Identify different and conflicting requirements
e. Identify steps for achieving common view in case of conflicting requirements
f. Identify the security sensitivity of information
g. Review acceptance of requirements
VI. Identify test methods
a. Survey and identify existing test methods
b. Adopt existing test methods
c. Develop new test methods
d. Identify the security sensitivity level for test methods
e. Identify ethical and legal compliance requirements
f. Review acceptance of test methods
VII. Identify scheme structure
a. Select ISO/IEC 17067 scheme types 1-5
b. Identify scheme owner and management
c. Scheme certificate
d. Identify scheme rules and classification
VIII. Identify and establish qualification methods
a. Identify operator qualification requirements
b. Identify accreditation needs
c. Identify and establish laboratory consistency methods
IX. Identify periodic surveillance methods for maintenance of certification
a. Determine surveillance needs
b. Identify scope of surveillance test methods
c. Identify procedures and frequency of surveillance activities
d. Identify validity of certificate
Case study: Biometric products
To analyse, develop, enhance, and experimentally validate evaluation and certification schemes, HECTOS conducted an experimental case study on biometrics products. The following topics were selected:
Topic 1 — Image quality of contactless fingerprint sensors
Topic 2 — Presentation attack (spoof) detection capability and presentation attack resistance of biometric systems
Topic 3 — Products for secure biometric access control to critical infrastructure
For each of these topics, performance requirements and associated evaluation methodologies were specified, based on a risk assessment for the selected applications, as well as requirements and associated test methods of existing standards:
For topic 1 — Established requirements and test methods associated with 2D fingerprint sensors
For topic 2 — Common Criteria guidance, published protection profiles for fingerprint spoof detection, output from the EU project BEAT, and related ISO/IEC standards under development
For topic 3 — A proposed CEN technical specification for biometric authentication for critical infrastructure access control (which itself was based on two similar national standards addressing biometric access control in critical infrastructure)
This biometrics study analysed and assessed the evaluation and certification processes and outcomes for the biometric case study topic against the elements of the preliminary HECTOS certification framework and template and provided recommendations to enhance the viability of a harmonised certification scheme for security products. Key issues examined include:
• Development and agreement of conformity assessment certification criteria across multiple technologies (e.g. both legacy and new biometric technologies)
• Harmonisation of requirements between related applications
• Inter-laboratory comparison, in particular identifying suitable procedures to ensure repeatability for adversarial tests (i.e. presentation attack resistance / presentation attack detection in the case of biometrics)
• Consideration of the possibility for certified evaluation independent of conformity assessment with quantified performance requirements, or thresholds, and
• Consideration of the requirements for accreditation (e.g. interlaboratory comparison, publication of test results) to enable re-use and mutual recognition of evaluation results.
Case study: Weapons and Explosives detection products
The second case study which assisted the development of the certification framework and template was on Explosives & Weapons detection equipment:
• Explosives trace detection equipment (ETD)
• People screening portals
The study developed test methods whose elements were investigated in detail for both ETD and people screening portals.
ETD
The review and comparison of existing test methods (in measurement standards, such as ASTM E2520-07) showed that these still do not solve the problem of a harmonised evaluation of ETD in areas outside aviation security. Various requirements for harmonised evaluation were assessed and it was concluded that the most important of them are concerning security performance by measuring of detection rate and false alarm rate. These two are not fixed parameters but are very dependent on the amount and the selection of threat and background substances. Therefore scenario related requirements have been reported in order to allow scenario adjusted evaluation.
Interlaboratory comparisons were extensively investigated and the results were used to contribute to the testing relevant parts of the HECTOS certification template. These studies showed that sample preparation was the most crucial part in the test methodology. Though the amount of explosives traces can easily be controlled by dispensing diluted solutions, the amount actually available for sampling of this traces from surfaces cannot be controlled precisely. Different preparation methods were tested with several different surfaces and explosives. It turned out that apparently no method fits all and the optimum preparation method has to be chosen from case to case. A two-step evaluation process was recommended (i) the use of samples of high accuracy (and thus higher degree of repeatability) but somewhat less relevancy and (ii) more realistic samples at the expense of larger variance.
The result of the interlaboratory test showed some interesting discrepancies. Especially the description of the sampling-method of volatile explosives within the test method was not clear enough and caused apparently misunderstandings and in consequence big differences concerning the detection results.
People screening portals
Intra-lab repeatability was investigated by carrying out twice a Security Scanner (SSc) test, using different test managers. For both tests the test managers were responsible to carry out the test according to the same test methods, but the tests differed in test persons, the exact way of carrying out the test and alarm indication interpretation. ROC-curve evaluation was also carried out for Walk Through Metal Detectors (WTMD) as well as a theoretical study on the technology independence of test methods for people screening.
The ROC-curve based evaluation showed that person-based testing allows for a more realistic assessment of the WTMD than tests without test persons. Due to a number of deliberately uncontrolled but realistic parameters, the measured detection rate shows a high variance. Person-based, realistic testing comes hence at the expense of accuracy and repeatability (which are of interest for mutually comparing the performance of WTMD systems). It was also concluded that mixed threat item / innocuous item testing gives a more realistic impression of the actual behavior in the field because of the interference that can occur between two or more items. An important consequence of this interference is that threat items may be found in mixed mode while missed in full divestment mode. The detection rate in mixed mode may thus be higher than for fully divested screening. The ROC-curve based evaluation seems to be especially useful when operated in partially divested mode. For non-divested mode full detection is obtained, even at the lowest sensitivity and with and without threat items. For full divested operation, the False alarm rate is zero and no ROC-curve is obtained. It is believed that the ROC-curve approach enables type certification based on its intended application, i.e. divestment level and sensitivity setting.
The intra-lab repeatability of the SSc test was reasonably good but with some statistically significant differences. Based on observations made during the test, the repeatability is thought to increase if a larger number of test persons are used and when the exact threat item location, orientation and way of attachment to the body is prescribed in the test protocol.
The technology independence of test methodologies was investigated on metal detectors based on passive magnetometry as used in some walk-by metal detectors (WBMD), the signatures that they detect and a discussion of testing considerations that apply to this technology. This showed that test methods developed for WTMD and SSc will not be suitable for passive magnetometer testing unless the magnetic signatures of test objects and environmental influences are very carefully taken into account as well. Consequently, a one-test protocol-fits-all for people screening detection systems is not realistic. It must be ensured that the assumptions made in the development of a test method are valid for all the different types of equipment that may be tested. The operating principles of the device under test have to be examined to determine whether the proposed test method is suitable or needs adjustment to fit those operating principles.
Final verification of the certification framework and template
HECTOS investigated how the harmonised certification framework could be applied to several physical security products used in biometrics, explosives & weapons detection equipment, radiological & nuclear detection equipment, chemical and biological detection equipment and locks. Additionally, the applicability of the HECTOS framework and template for security systems was explored for video surveillance systems.
These studies initially summarised the historical needs and developments in the field of evaluation and certification for the types of product and the current international, European and national activities with respect to testing, standards and certification. Next, the HECTOS certification framework and template for establishing a new certification system and/or schemes were elaborated and verified, via case studies of the above-mentioned product categories and the needs of end-users. The template itself was utilised as guide in this process to identify the current maturity with respect to the implementation of standardisation and harmonised evaluation and certification schemes, and to provide suggestions on how harmonised schemes could be introduced, identifying activities for scheme functions and features that yet need to be addressed. The analysis indicated that E&W detection equipment for aviation security is a relatively mature, while E&W detection equipment for non-avsec applications is very immature with respect to the implementation of harmonised evaluation and certification schemes. Biometrics, RN detection equipment and locks have a similar maturity with respect to the implementation of harmonised evaluation and certification schemes but face different challenges. CB detection equipment is another product type showing low maturity. There is a need in nearly all areas for further work on harmonised standardisation, both in product and measurement standards, which is a prerequisite for establishing harmonised schemes.
The case studies have shown that the framework provides a basis on which sets of harmonised schemes could be developed for the different categories of physical security products. The framework and template have been found to be useful in identifying elements that exist; gaps where further work is needed in terms of requirements development, standardisation and development of schemes; and the potential types and structure of certification schemes that are required in each area.
Implementation roadmap
The validation of the certification framework and template assisted the development of the implementation roadmap that shows a possible way to implement the proposed framework for certification and evaluation of physical security products on a long-scale time frame. It consists of two types of roadmaps:
Enabling infrastructure roadmap – comprising a number of roadmap high-level elements ranging from dissemination and awareness building of the proposed concept through its piloting to the expansion into various physical security certification systems. The system group coordinator has a key role in these activities.
System roadmaps – describing the highlights of the way ahead towards implementation of the proposed harmonised European certification schemes for biometric products, explosives & weapons detection equipment, radiological & nuclear detection equipment, chemical and biological detection equipment and locks. By assigning values for the status quo and the complexity to further implement harmonised schemes for each of the analysed products, arbitrary indicative scores were obtained for the anticipated complexity to further implement harmonised certification for each product studied. The assessed complexity was analysed and compared with the status quo. The following conclusions could be drawn:
• Explosives and Weapon Detection system for aviation security is relatively mature as a harmonised system, but the remaining steps might be challenging
• RN Detection system is quite mature in terms of standardisation, although there are no certification schemes outside the USA, and seems to be the less complex case to implement, provided there is a business case
• Biometrics and Locks are both quite mature but remaining harmonisation is anticipated to be challenging
• CB Detection is very immature and many steps are expected to be quite complex to implement
• Explosives and Weapon Detection system for non-aviation security is the least mature system and expected to be the most complex to implement
Considering the degree of complexity and maturity of the template steps needed to be taken as presented above, the RN detection system, Explosives and Weapon Detection system for aviation security and the Biometrics product system are foreseen as the most suitable candidates for a pilot study on the implementation of a certification scheme using the HECTOS template and framework.
Testing of early stage technologies
HECTOS also studied the development of approaches to the testing of low-TRL prototypes in the physical security domain, aiming to deliver the following anticipated benefits:
• To enable researchers and technology developers to assess the maturity of their work and the potential of the technology
• To guide researchers by helping them understand the real world environment their technology will need to operate in and identify the areas where further development work is needed
• To enable technology developers to communicate effectively with their peers and with their sponsors or funding agencies about the progress and potential of their work
Early in the process it was decided to divide physical security products into two broad categories; technologies which ‘Detect’ and those which ‘Delay’ an adversary. It was identified as necessary to take a different approach towards testing in each case. The majority of the effort in this exercise was focused on developing early stage testing approaches for the ‘Detect’ technologies. This was done using case studies of two different types of explosives and weapons detection technology.
Using background research into other existing early stage testing concepts in both security non-security markets, as well as prior experience of working with early stage security technology, an initial high level approach to the generic low-TRL test methodology was developed. This was tested and refined through the development of two technology specific case studies. The first case was test and evaluation of optical Raman trace explosive detection techniques and second case was test and evaluation of millimetre wave techniques used for detecting concealed explosives and weapons.
The draft Raman test method was used to test actual hardware prototypes under development to explore which aspects worked well and identify areas for improvement. The draft millimetre wave test method was reviewed by a number of independent subject matter experts comprising developers, system integrators and end-users. For both test methods, feedback was very positive. As well as providing the basis for the methodology development, it was felt that both test methods could be used by the R&D community to support a number of ongoing development projects. Using the experience and insight gained in the case study process, the generic methodology has been refined and developed to the extent that it now stands alone as, hopefully, an accessible guide for users to understand the principles and apply this approach for a new detection technology development.
It was found through the process of reviewing approaches in other fields that although there were no directly applicable methods, there were lessons that could be learnt, and good generic principles that could be extracted from other early stage testing examples. The process by which the test methodology was generated was found to work well. By building on the high level initial approach to develop the two test cases, then challenging them through lab testing or peer review, the findings were found to be valuable in refining the generic low-TRL test methodology. The defining principles were generated at the start of the process as high level goals for the activity, and refined throughout by a process of retrospective review. It is considered that the methodology produced in HECTOS meets these principles, and thus hopefully will act as a useful tool to support and guide the development of new technologies. There is clear scope to continue this activity, and exploit the outputs of this research and these potential activities are outlined in the section below about exploitation activities.
Law and ethics
HECTOS has studied the ethical and legal aspects of evaluation and certification of physical security products, such as:
• Ethics and Human Rights Risks of Security Products
• Ethical/human rights aspects of application scenarios
• Privacy by Design and Ethics by Design
• Legal issues raised by the evaluation and testing of security products
The work was led by University of Warwick and it started with an overview of the ethics and human rights issues that arise in connection with the use of security products. Distinctions and overlaps between ethics and human rights were described. The following ethical issues were analysed: privacy; data protection; mission creep; discrimination; error; consent; dual use; freedom of movement, expression, and association; and health and safety concerns, as well as how these issues arise in connection with each of the 4 product categories focused on by the HECTOS project (i.e. Barriers, Access Management; Surveillance, and Detection).
For the ethics and human rights risks of the technology use described in the application scenarios defined by HECTOS, the most commonly arising issues were identified to be: privacy, data protection, freedom of expression, association, and movement, health and safety concerns, proportionality, and issues around consent. It was found that some of these issues can be addressed at the stage of product design and manufacture. Others can only be addressed by changes to the processes and procedures around technology use, such as training of staff, codes of conduct, protocols and standard operating procedures.
University of Warwick organised two meetings, each of which address the Privacy by Design and Ethics by Design aspects of Person Screening products and Biometric Access Control Products. The meetings gathered expert opinion on inclusion of ethical and legal compliance assessment in evaluation and certification schemes and explored the use of specific ethical standards such as privacy-by-design within this context. Speakers included HECTOS Case-Study leads, manufacturers, Privacy by Design, Data Protection, and Privacy Impact Assessment experts, lawyers, and commercial providers of Privacy by Design Certification. This resulted in a number of recommendations for the incorporation of Privacy-by-Design and Data Protection principles into products via certification.
Competition issues that arise currently in relation to the standardisation and certification of security products within the EU, and how a harmonised approach may affect these were also investigated through interviews with experts, such as competition lawyers and other relevant stakeholders. Key themes that arose during the discussions included: the ambiguous role of both of standards and standards-bodies and of testing houses in facilitating competition; the tension between, on the one hand, conceded national prerogatives with respect to the procurement, development and sale of products, and on the other, the need for larger competitiveness across the EU; and the challenges to entry into the market for smaller players in light of the power of larger players to influence standard-setting and to inclusion in procurement processes. It was found that harmonisation encourages the creation of common standards and, though standards are desirable, they must be established in an open way and then made generally available, without participation being limited to big or well-resourced players.
Expert and Advisory Group
The HECTOS Expert Advisory Group (EAG) with stakeholders drawn from manufacturers, trade associations, users, government security agencies, representatives of the standardisation, evaluation and certification communities, was established initially from the 12 organisations that offered to join such a group and who wrote letters of support for the proposal. It was expanded as the consortium identified and engaged with other stakeholders identified during the project, in particular to include the IEC Conformity Assessment Board (CAB) and DIN CERTCO. The EAG members at the end of HECTOS were:
• European Organisation for Security – EOS
• Perimeter Security Suppliers Association – PSSA
• Euralarm
• Smiths Detection
• Bruker Daltronics
• UK Home Office Centre for Applied Science & Technology - CAST
• UK Centre for Protection of National Infrastructure - CPNI
• JRC IRMM
• BMI – German Interior Ministry
• AFNOR
• BKK – German Civil Protection Office
• The Swedish Theft Prevention Association, SSF
• British Standards Institute –BSI
• German Federal Police Technical Centre –FPTC
• Bundeskriminalamt – BKA
• Loss Prevention Certification Board –LPCB
• European Fire & Safety Group – EFSG
• DIN CERTCO
• IEC Conformity Assessment Board
In addition, the project continued to engage with other stakeholders such as Assa Abloy, the Swedish evaluation and certification body RISE (former SP), the CRISP project, DG Home (in its role as EC DG Home - Innovation and Industry for Security), CEN TC/391 Societal Security committee as well as a number of others from various areas.
Meetings and discussions were held with EAG members throughout the project to keep them up to date with our work, seek their guidance and discuss certification issues. In particular the EAG members provided the bulk of the stakeholder requirements, helped in the review of the HECTOS certification framework and the decision to develop it as a CWA and made significant contributions to the CWA itself.
Stakeholder meetings were held in Brussels in 2015 and 2016 and attended by a good range of stakeholders. These meetings were a good opportunity to present the project activities and obtain feedback to validate the approach being taken,
The third stakeholder meeting and HECTOS project final event was held at the Bouche à Oreille (BaO) conference centre in Brussels on December 6th 2017. The event was held as part of the 9th meeting of the European Commission’s Community of Users on Secure, Safe and Resilient Societies.
Dissemination Plan & Reports
An initial Dissemination Plan was produced for internal use within the project during the first few months of the project. This was revised and expanded at the end of first year of the projects in the light of the consortium’s growing knowledge and experience and to incorporate the results of discussions with the Project Officer on project strategy, objectives and alignment with other initiatives.
Public and Targeted Dissemination Activities
A good level of dissemination activities has taken place throughout the project including the project including the website and project booklet; stakeholder events; presentations at conferences and specialist meetings; discussions with individual stakeholders; dissemination of deliverables; and the project website.
The project website has been maintained throughout the project at http://hectos-fp7.eu/. This contains an overview of the project, news items, summaries of all deliverables and downloads of the public deliverables from the project.
A ten-page printed project booklet providing an overview of the project and its main results was prepared for use at the HECTOS final project event. Copies were provided to partners for distribution to their stakeholder contacts. A pdf version of the booklet is also available for download on the HECTOS website.
Liaison activities with the FP7 CRISP project continued throughout the project and representatives of each project attended the others stakeholder meetings. Deliverables were shared with CRISP and with other relevant FP7 and H2020 projects.
Team members attended and presented at a range of meetings and conferences throughout the project, giving presentations on the overall project, the HECTOS Certification Framework and on individual specialist topics from the explosives & weapons detection and biometrics case studies.
Towards the end of the project a ten-page printed booklet was produced describing the key results from the project. This was published on the HECTOS website alongside the summaries of project deliverables and downloads of the public deliverables from the project.
Standardisation Activities
A formal liaison with CEN TC/391 “Societal and Citizen Security” was finalised in September 2016. HECTOS representatives attended meetings of the CEN/TC 391 to facilitate knowledge exchange and HECTOS obtained relevant documents dealing with current work of CEN/TC 391, as well as information about the current and future work programme.
As the second period progressed and with the development of the HECTOS certification framework and implementation template, it became clear that HECTOS could raise its standardisation ambitions and attempt to progress the framework through a CEN/CENELEC workshop, resulting in the publication of a so called ‘CEN/CENELEC Workshop Agreement (CWA).
CEN/CENELEC also agreed an ‘open access’ model for CWAs resulting from European research, whereby publication costs would be paid by DIN from their project budget and the CWA would be available for download free-of-charge
After the successful initiation of the CEN/CENELEC Workshop HECTOS with the aim to publish a CEN/CENELEC Workshop Agreement (CWA) on Guidelines on evaluation systems and schemes for physical security products two physical meetings (including the kick-off) as well as four web conferences were conducted to develop a draft document. Due to the strong participation of external stakeholders in the workshop several fruitful discussions took place in order to adapt the proposed HECTOS results to market and user needs.
In December 2017 a first draft of the CWA was finished and provided to the public for a commenting phase. More than 300 comments from both workshop participants as well as external, non-workshop members were received. The final document was then developed and approved by the registered workshop members by end of January. After approval of the document by a majority of participants and final editorial improvement, the CWA was sent to CEN/CENELEC Management Centre for publication on 31th January 2018.
The final CWA document was approved by 12 of the participating organisations:
• Swedish Defence Research Agency, FOI (Chair)
• Iconal Technology Ltd (Vice-Chair)
• Asociatia Romana pentru Tehnica de Securitate, ARTS
• BRE Global Ltd
• DIN CERTCO Gesellschaft für Konformitätsbewertung mbH
• European Certification Body (ECB) GmbH
• European Security Systems Association (ESSA) e.V.
• IDEMIA
• Fraunhofer-Institut für Chemische Technologie
• Fraunhofer-Institut für Grafische Datenverarbeitung
• National Physical Laboratory (NPL)
• The Netherlands Organisation for Applied Scientific Research (TNO)
No organisation disapproved the CWA and only one abstained.
The CWA will be published as CWA 17260:2018 Guidelines on evaluation systems and schemes for physical security products, free of charge and publicly available via the CEN-CENELEC webpage.
Overall, HECTOS has ensured that it is visible to and engaged with a wide range of stakeholders representing different aspects of the field. In particular, the project has sought to listen to and take input from stakeholders in order to be aligned with and relevant to both user and supplier industry needs. As discussed above, channels have been identified for the dissemination and exploitation of project results.
As the research on the project starts to yield results, these are being disseminated to the security, standards and evaluation/certification communities so that the results of the project will be exploited in due course.
The continuing stakeholder contacts and posters/presentations at conferences have maintained awareness in the community. Technical results from the case studies are starting to be disseminated and exploited.
Potential Impact:
Potential Impact
HECTOS could have a significant impact on physical security, helping protect citizens from serious crime and terrorism, whilst supporting economic development through the single market. Improved physical security product evaluation and certification mechanisms, such as those enabled through the work of HECTOS, should lead to improved security systems and capability for end-user organisations as well as increasing the understanding of and trust in the performance of the component products from which they are constructed. Harmonised evaluation and certification schemes, focused on the particular characteristics and needs of security products, such as those based on the HECTOS certification framework, should reduce the cost and time-to-market for product manufacturers by removing the need for different tests and certification activities in different Member States. They should also make it easier for end-user organisations to understand and select appropriate products for their needs, thus encouraging the deployment of more effective solutions. Benefits are foreseen for a range of industry stakeholders:
End-user organisations, specifiers and advisers
- Better and trusted information on the performance of security products
- Better information on which to base the design of security systems and processes to protect against attacks from attackers with malicious intent
- Reduced product costs
- Improved availability of new capabilities
Manufacturers
- Reduced costs of evaluation and certification by avoiding the need for multiple certifications in different countries
- Faster time to market, leading to competitive advantage
Government and regulators
- Improved security of the citizen and the state
- Protection of fundamental rights
- Improved economic position of Member States
- A potential roadmap for the introduction of harmonised evaluation and certification schemes for security products
Security research & development community
- Improved test and evaluation approaches for early stage detection technology development to guide and support the R&D process
- Specific test and evaluation techniques to address key technical challenges in the case study areas of biometric products and explosives & weapons detection
Standardisation, Test & Evaluation, Certification organisations
- Improved procedures and guidelines for the development and implementation of evaluation and certification schemes, recognising the particular requirements of security products
- Improved test methods in the areas of biometrics and explosives and weapons detection, addressing issues of repeatability and consistency – over time and across tests both within a test laboratory and across different test laboratories
The HECTOS vision is to implement harmonised certification schemes based on the HECTOS framework across all physical security products. Whilst it is recognised that this is a long-term ambition which will require the involvement and cooperation of governmental, industry and end-user stakeholders across Europe, much can be done today with the results of the project:
• The HECTOS roadmap provides for an incremental introduction of harmonised evaluation and certification schemes. These could be introduced initially in one or two areas, perhaps where a good level of standardisation exists, but where formal certification schemes are lacking.
• The concept of Measurement Schemes as well as Threshold Performance Schemes enables certification of products where the wide range of applications, each with their own performance requirements, makes the concept of certifying ‘minimum performance levels’ impractical
• The low-TRL testing methodology work to support the security R&D community is already being exploited.
• Test methods and techniques investigated and developed as part of the HECTOS case studies are already being considered and used in the explosives detection and biometrics communities.
Dissemination
The project has worked closely with the stakeholder community (end-user and specifier, manufacturer, governmental, R&D, standardisation, test & evaluation and certification bodies) throughout the project. A dialogue has been maintained both to help guide and shape the work of the project and to disseminate the vision and results of the project as they have emerged. The principal dissemination activities during the project have been:
• Development of a dissemination plan analysing and defining the project’s communication and dissemination strategy to all the various stakeholders and the general public.
• Project website http://hectos-fp7.eu/ with general information, deliverable summaries and public deliverables
• Engagement and relationship building with stakeholders of all types across the wide range of different physical security products
• Regular stakeholder meetings to communicate the results of the research
• One-to-one interactions with key stakeholders – meetings and presentations throughout the project.
• Sharing of project deliverables with other EU research projects and stakeholders
• Presentations at industry conferences and published papers and posters – at a wide range of events in Europe and USA.
• Engagement with stakeholders during the HECTOS laboratory test-based case studies in explosives & weapons detection and biometrics, as well as in the paper studies on potential future certification schemes in other areas.
• Engagement with and participation of stakeholders in the development, review and approval of the CEN/CENELEC Workshop Agreement CWA 17260 Guidelines on evaluation systems and schemes for physical security products
• HECTOS project Final Event and the associated project booklet distributed via the partners and on the project website
• Dissemination of the HECTOS Roadmap (D8.1 D8.2 titles) as public deliverables
• Publication of the HECTOS Certification Framework as a formal European standardisation product with stakeholder approval for credibility and open access publication to maximise its dissemination.
Exploitable Results
The exploitable results of the project divide into three groups:
1. Framework & Roadmap
- Overall HECTOS physical security product certification approach, Certification framework and its supporting template, plus the HECTOS Roadmap
2. Case study results
- Specific knowhow, test methods and approaches developed to explore and address specific challenges in the evaluation of biometrics and explosives & weapons detection products
- Low TRL testing methodology and example case study test methods
3. Physical security and certification expertise
1. Framework & Roadmap. The overall HECTOS physical security product certification approach, certification framework and its supporting template, plus the HECTOS Roadmap are the main results of the project and are the items which have the potential to achieve the impact described above. Exploitation of these requires action from certification bodies who will own and operate schemes, supported by a broad spread of physical security product manufacturers and user organisations. Direct support from the European Commission, either in the form of legislation to force behaviour or support actions to encourage it is almost certainly also required. At the very least the Framework needs to be endorsed at European level, as described in the HECTOS roadmap.
None of the project partners are certification bodies able to drive this exploitation. Consequently, these principal results have been placed in the public domain as public deliverables from the project, or in the CEN/CENELEC Workshop Agreement CWA 17260 Guidelines on evaluation systems and schemes for physical security products submitted to the CEN/CENELEC Management Committee (CCMC) on 31th of January 2018 for proofreading and publication. The CWA will be available to download at no cost from the CEN/CENELEC website, which should increase its dissemination. The HECTOS roadmap is another public deliverable (D8.2) showing how the overall Framework could be implemented, together with certification systems and schemes for individual product categories.
The HECTOS partners will support organisations who wish to establish schemes based on the HECTOS Framework through the provision of consultancy services. The partners will also support other organisations who wish to provide similar services (A former employee of partner Iconal has already set up a consultancy organisation with this aim in the RN detection area. The HECTOS partners have agreed to make deliverables available to this organisation).
2. Case study results. The HECTOS case studies have also led to significant exploitable results including:
• An evaluation scheme for contactless fingerprint acquisition from the biometrics case study scheme, which has already been accepted and used by the scheme owner. The updated scheme may now be integrated in the next revision of ISO/IEC 19794-4 standard.
• Work on repeatability of results of Presentation Attack Detection evaluations which has helped to define new methodologies. Further work in this area may help harmonise results from evaluation bodies.
• The biometric case study on Biometric Access Control to Critical Infrastructure which has provided relevant data and recommendations which are currently being standardised.
• Interlaboratory comparison of biometrics Presentation Attack Detection (PAD) testing, leading to input to standards.
• Establishing (potential) baseline performance requirements on PAD and inputs to technical specifications and evaluation standards in CEN & ISO
• Work on the repeatability and interlaboratory comparisons of explosives trace detection product evaluations, which are being discussed with the European Civil Aviation Conference (ECAC) for the evaluation and certification of aviation security equipment.
• New ideas on test method development & testing of people screening portals, especially for ROC curve measurement, which are also being discussed with ECAC for future aviation security test and evaluation
• The low-TRL testing methodology for detection technology which is already being used by Iconal and FOI in their own detection technology R&D activities and which is being exploited by Iconal for the benefit of UK Government funded explosives and weapons detection research projects in aviation security and non-aviation applications.
3. Physical security and certification expertise. All of the partners have gained considerable experience and expertise in:
• The wide range of different types of physical security product
• Standardisation of security products
• Test and evaluation of products
• Certification systems and schemes
• The specific needs and challenges of security product evaluation and certification.
All of this knowledge and expertise is exploitable. In particular, the project has enabled the partners to gain expertise across the whole range of different types of security product and application. Few organisations have this breadth of knowledge, making it very exploitable.
This expertise will be used to further the security research and development as well as the consultancy and services businesses of the various partners.
Partner exploitation plans
The specific exploitation plans of each partner are summarised below:
FOI plans to support the adoption of the HECTOS Framework and Template by maintaining the dialogue with EC, Certification and Evaluation Bodies, Standards bodies and other key stakeholder, To the extent possible, FOI will use the CWA (or parts of it) in our own evaluation activities, thereby showing its usefulness to the security community. If the HECTOS Framework is implemented for explosives detection products, FOI may participate as an evaluation body. FOI will also use the HECTOS low-TRL testing methodology in its own research activities.
Iconal Technology is already exploiting the low TRL testing methodology work, both by using it on two of our own research projects and in a UK Government project to make it available to other explosives and weapons detection research projects in other organisations. Iconal also plans to use the experience and expertise it has gained in development of standards, evaluation and certification in its business activities supporting technical and policy makers in UK, European and US government organisations.
TNO plans to apply expertise gained on physical security products in general; on the certification framework and guidelines for physical security products; and, on the possible implementation steps for certification schemes for VSS, explosives and weapons, chemical and biological detection in its consultancy role to policy makers and test and certification bodies. TNO will apply the results of the case studies and expertise gained on test method development & testing of people screening portals in work as an ECAC test centre and member of the ECAC CEP Management Group and Technical Task Force. TNO also plans to promote of the HECTOS certification framework and implementation template whenever possible and appropriate during national and international meetings and conferences.
NPL will use the results of HECTOS to extend interlaboratory comparison methods to encompass measurement of security systems and more generally to extend its techniques developed for biometric evaluation to other types of security product. Practical experience and knowhow gained on the project will feed in to its biometrics standards projects in SC37, CEN.
Fraunhofer IGD will seek to exploit the HECTOS results, in particular the results from the biometrics case studies, in an ongoing project supporting the German government in standardisation in the field of biometrics, and in other standardisation projects dealing with security evaluation of biometric technology. Furthermore, IGD will exploit project results in its own biometrics evaluation laboratory, which offers security evaluation services for biometric technologies to industry.
Idemia (Morpho) is already using the project results to extend an existing FBI certification scheme to new technology (contactless fingerprint acquisition). This involves new 3D targets which considers an acquisition volume rather than an acquisition surface. The methodology has been accepted by the scheme owner and may be integrated in a revision of ISO/IEC 19794-4: Fingerprint Image. Idemia is planning to exploit HECTOS case study results in biometrics Presentation Attack Detection certification, which is an important subject as PAD is a key feature to demonstrate to allow mass-market deployment of biometric products. Results from Idemia work on the Biometric Access for Critical Infrastructure case study are currently being used in the standardisation process at CEN WG18 on Biometrics. Recommendations and requirements will be taken into account for next update of Access Control products line
Fraunhofer ICT will exploit its work on explosives & weapons detection through further internal and external dissemination of the results. It will continue the discussion with manufacturers and suppliers of detection devices and with the Federal police & BKA on implementation of the HECTOS framework and template in the explosives detection community and about plans how to introduce changes. Discussions will be held with ECAC and end users on how to bring certification into the non-aviation applications. Experience gained during test method development and testing will improve the ICT test and evaluation services offered to customers.
DIN is a national standards body and does not participate in product certification schemes. However it will continue to support the exploitation of the results of HECTOS through standardisation and will use the expertise gained on physical security products and security product certification in its ongoing standardisation work.
The University of Warwick (UW) has published the results of its work on the project as publicly available deliverables, for use by the ethics research community and by policy makers working in the security area. Warwick will use the knowledge and expertise of physical security products and certification gained during the project in its future ethics research activities.
Further research
Further European research and actions would be beneficial in several areas to extend and support the development and exploitation of the HECTOS work. These include:
• Development of measurement and product standards for product categories where few or no European/International standards exist – including CB detection, Explosives & Weapons detection outside of aviation security, video analytics products for Video Surveillance Systems, fences and gates.
• Harmonisation of European Standards (although not necessarily leading to ‘Harmonised EN Standards’ driven by European Directives) in mature product areas such as locks and for high security products where national ‘top up’ standards exist.
• Development of guidance material on how to write measurement standards (test methods) for security products, focussing especially on the need to ensure consistency in realistic and adversarial testing.
The contact details for the HECTOS partners are:
Totalförsvarets Forskningsinstitut, FOI (SE)
Website: www.foi.se
Contact: Anders Elfving
anders.elfving@foi.se
Nederlandse Organisatie voor Toegepast Natuurwetenschappelijk Onderzoek, TNO (NL)
Website: www.tno.nl
Contact: Martijn Koolloos
martijn.koolloos@tno.nl
Fraunhofer Gesellschaft (DE)
Website: https://www.fraunhofer.de
Contact: Frank Schnürer
frank.schnuerer@ict.fraunhofer.de
Iconal Technologies (UK)
Website: http://www.iconal.com/
Contact: Mike Kemp
mike.kemp@iconal.com
Idemia (FR)
Website: www.idemia.com
Contact: Pierre Gacon
pierre.gacon@idemia.com
University of Warwick (UK)
Website: https://warwick.ac.uk/
Contact: Katerina Hadjimatheou
K.Hadjimatheou@warwick.ac.uk
National Physical Laboratories, NPL (UK)
Website: http://www.npl.co.uk/
Contact: Tony Mansfield
tony.mansfield@npl.co.uk
Deutsches Institut für Normung, DIN (DE)
Website: www.din.de
Contact: Christopher Liedtke
Christopher.Liedtke@din.de
HECTOS is an EU FP7 security research project exploring the issue that there are very few evaluation and certification procedures for physical security products which are mutually recognised by EU Member States. This leads to fragmentation of the market, as identified in the EC Communication on Security Industrial Policy, with negative impacts on both suppliers and users. The HECTOS objective is to support the harmonisation of the European market for physical security products, by producing a certification framework and template for these products as well as a roadmap for the implementation of the framework.
Physical security equipment is very diverse in technology, concept of operation, application area and performance, and similar security products are difficult to compare in terms of performance, accuracy, usage, trust and validation of functionality. Furthermore, in a number of product areas, there is a lack of product and measurement standards at European or International level which are a necessary prerequisite for harmonised certification schemes.
The HECTOS project focuses on work towards harmonisation of evaluation and certification schemes for physical security products and has developed a framework for conformity assessment systems and schemes for this group of products. It has explored how the international standards for conformity assessment (ISO/IEC 17000 series) and the existing certification systems used in other areas could be applied, adapted or developed for products used for security of people, property and infrastructure. This has been achieved by taken into account the particular security-specific features that need to be built into performance measurement and threshold performance schemes for physical security products to address:
• Realistic and adversarial testing
• Consistency in testing – in particular for adversarial testing
• Measurement of complex performance parameters
• Security sensitivity
• Continually evolving requirements
• The wide range of applications and performance requirements
• The diverse range of types of product, markets and levels of maturity
The HECTOS harmonised certification framework comprises a three level structure and approach which can accommodate all types of physical security product and application, covering all forms of conformity assessment including pass/fail testing against a threshold and performance measurement schemes.
The development of the framework and its supporting implementation template has been supported by experimental case studies for Explosives & Weapons (E&W) and biometric recognition products and validated by applying them to several different product groups as case studies including; E&W detection, biometric recognition, CBRN detection, locks and Video Surveillance Systems.
The framework and template for the establishment of evaluation and certification schemes for physical security products are the main content of a CEN/CENELEC Workshop Agreement (CWA), initiated by the HECTOS project. The CWA was developed by the project team together with a wide range of stakeholders such as representatives from certification bodies, industrial associations and standards bodies. The CWA published as CWA 17260:2018 is downloadable at no cost from the CEC/CENELEC website.
Project Context and Objectives:
HECTOS is a European project focusing on harmonisation of evaluation and certification procedures for physical security products. Physical security equipment and systems are very diverse in technology, concept of operation, application area and performance, and similar security products are difficult to compare in terms of performance, accuracy, usage, trust and validation of functionality. Currently, there are very few test, evaluation and certification procedures in Europe that are mutually recognised by different Member States (MS). This leads to fragmentation of the market, as identified in the 2012 EC Communication on Security Industrial Policy, with negative impacts on both suppliers and users. Through the harmonised approach developed in HECTOS, product functional performance and certification schemes can be recognised, leading to a more homogeneous market with enhanced possibility to demonstrate and compare physical security products.
Focussing on the functional Performance of Physical security Products (the so-called HECTOS ‘three Ps’), the project has investigated conformity assessment schemes both for consistent performance measurement and for conformity with threshold performance values. The scope of the project covers physical security products used for security of people, property and infrastructure, including:
• Barriers
• Access management
• Surveillance
• Detection equipment
This wide range of products and applications, the need to operate in both regulated and unregulated environments as well as products with very different maturity and market sizes, means that a range of different types of scheme is needed. HECTOS covers security products which protect against criminal and terrorist attacks, rather than products for safety, such as protection from natural disasters. The project has not focussed on certification of other product attributes, for example interoperability or safety, nor does it cover the evaluation and certification of security systems, security services or cyber security – although these topics are also important and need to be addressed in other projects.
HECTOS has explored how existing certification schemes and systems used in other areas could be applied, adapted or developed for products used for physical security by taking into account the particular features which applies to certification of security products. HECTOS has identified the current state-of-play and the level of harmonisation across all types of physical security product and has developed a roadmap showing how harmonised European certification systems and schemes could be introduced.
Accommodation of the disparate types of security product, whilst enabling harmonisation is proposed by ensuring or encouraging:
• Performance measurement schemes which provide certified measurement of performance attributes, as well as threshold performance schemes which certify conformity with defined threshold performance values
• Common definitions of performance attributes and their evaluation methods in measurement standards or test methods
• Common threshold minimum performance values or grade definitions for conformity assessment
• Common, or mutually accepted, processes for evaluation, certification and accreditation to ensure consistency over time and between participants
• Common requirements and mutually accepted processes for ongoing surveillance to ensure the consistency of product manufacturing
• Common or mutually accepted security mark and/or database
Security products differ from other types of product in that they help protect against attacks by an intelligent adversary with malicious intent. Therefore, a number of security-specific features need to be built into performance measurement and conformity assessment processes in security certification schemes:
• Realistic and adversarial testing – users need to understand how a product will perform in real-life situations
• Consistency in testing – tests involving human strength and skill need to be standardised. Procedures are needed to ensure the consistency of tests over time and between test laboratories
• Complex performance information – performance attributes such as detection rates are often complex and need to be precisely defined
• Security sensitivity – it may be necessary to restrict access to performance data and test methods to avoid revealing weaknesses and gaps to a potential attacker
• Continually evolving requirements– changing threats as attackers adapt to exploit weaknesses in security systems mean that security product requirements are always changing
• Wide range of applications and performance requirements – many security products are used in a variety of different applications subject to different threats. Schemes need to be carefully managed to ensure testing covers the range of requirements without imposing too large burden on manufacturers
• Diverse range of types of product, markets and maturity – from low-cost consumer products selling in large volumes to high-end and costly products with only a handful of sales per year, mean that schemes need to be based on what the market needs and can support.
The following sections summarise the main activities in HECTOS.
Review of schemes and requirements
A review of physical security products was made early in the project. The approach used a set of categories based on the functions that the products provide as well as taking into account the categories used by stakeholders in the EU and around the world. The survey also took into account categories/structures established by European standards bodies. This was followed by a review of application categories and scenarios in which these products are used. Six application categories and eighteen underlying scenarios were identified and described, including an assessment of the market size. These were used to help guide the work and ensure that a wide range of different application needs were taken into account.
During the first year of the project, stakeholder requirements for physical security product evaluation and certification schemes were also identified and analysed. This survey was based on a questionnaire, interviews and a workshop held with a wide range of stakeholders. Input from previous and ongoing work on certification schemes in the EU was also been taken into account. The requirements were grouped into standards-, evaluation-, certification-, and accreditation-related requirements. Product-specific requirements were also identified.
Certification systems are often based on product and measurement standards, including performance requirements and test methods, respectively. The landscape of existing standards for physical security products were therefore investigated. This revealed product areas in which standards are lacking or insufficient in terms of performance requirements, or areas in which there are many (possibly overlapping) standards. It also provided an overview of the use of European standards versus other standards such as industry, national and international standards.
Based on the initial surveys on products, applications, standards and stakeholder requirements, HECTOS studied existing certification systems. This provided knowledge on how certification systems could be adopted, adapted or developed for products used for physical security. More than 20 certification systems in use in different countries in Europe and around the world were analysed. Most of them were deployed for certification of security products, but also systems for non-security products were studied in order to understand the features which potentially could be applied to the security sector.
Framework development
During the initial surveys, it was understood that a single certification system for physical security products cannot accommodate the wide range of products and applications, varieties in market size and different level of maturity between product types. In order to accommodate these varieties, HECTOS decided to develop a generic certification framework for security products with guidelines (template) on the establishment and maintenance of systems and schemes. It was found that the generic ISO/IEC 17000 series of standards for conformity assessment, based on the functional approach, is widely applied and involves mechanisms for evaluation and certification, also applicable for certification of physical security products. Thus, the certification framework developed in HECTOS is based on this ISO/IEC 17000 series of standards supplemented with features that focus on security-specific aspects, which currently make certification of security products challenging.
The HECTOS framework accommodates:
• A three-layer structure: system group, systems and schemes
• System and scheme key building blocks
• Security specific features
• Performance measurement schemes
• Threshold performance schemes
• System management structure
• Actors and roles, including System Committees and scheme Working Groups
• Recommendations on a single security mark with system and scheme identifiers
• A common database for certified physical security products
• Controlling documents for requirements as well as operational processes and procedures
The HECTOS certification framework is supported by an implementation template covering topics to be considered when establishing a scheme as well as guidance on the maintenance of systems and schemes, including operational and management features. The development of the certification framework and template was monitored from an ethical perspective by the ethics experts in the consortium and issues were raised and discussed when necessary.
The HECTOS certification framework and template for establishing a scheme became the basis for a CEN/CENELEC Workshop Agreement (CWA 17260:2018), initiated by HECTOS and developed together with external stakeholders.
Experimental investigation
Two experimental case studies were carried out in order to assist the development of the certification framework and template. These studies focused on Biometric products and Explosives and Weapons (E&W) detection equipment and covered primarily evaluation (testing) aspects. Additional scheme elements were included as paper studies.
The case study on biometric products analysed three topics:
• Image quality of contactless fingerprint sensors
• Presentation attack detection/resistance of fingerprint systems
• Secure biometric access control
Certification criteria across multiple technologies, harmonisation of requirements and inter-laboratory consistency of adversarial testing are examples of issues explored in detail.
The E&W detection equipment case study examined explosives trace detectors (ETD) and people screening portals. Elements in test methods critical for inter- and intra-laboratory consistency as well as the applicability of Receiver Operating Characteristic (ROC) curve-based evaluation are two examples of topics investigated.
Validation
A final validation of the HECTOS certification framework and template was performed by considering how they could be applied to a number of security products:
• Biometric products
• E&W detection products (aviation security and non-aviation security)
• Radiological and Nuclear detection products (for illicit trafficking)
• Chemical and Biological detection products
• Locks
• Video Surveillance Systems (VSS)
These paper studies fulfilled two purposes: (i) they provided valuable feedback to the refinement of the framework and template (and thus the CWA), and (ii) they provided guidance to the development of the implementation roadmap for individual certification systems and schemes. The studies applied the template for establishing a scheme and assessed the current status of harmonised certification for the different types of product. Opportunities and challenges for further implementation of harmonised certification systems were also investigated.
Roadmap
The HECTOS view on how the European standardisation, evaluation, certification and accreditation for physical security products can be taken forward within a holistic framework was finally studied and presented. This includes recommendations on actions to be taken, how they can be taken and in what order. A suggested design of the three-layer structure of system group, systems and schemes was elaborated, to show how the framework could accommodate all the different categories and applications of physical security product.
A high-level so-called enabling infrastructure roadmap was developed which focussed on the major activities to be taken by the stakeholder community in order to implement the HECTOS certification framework. The finalisation of the HECTOS CWA is a first step in this process.
Conclusions from the paper studies described above were summarised in a number of system roadmaps describing the phased development of certification systems and schemes for individual product categories.
Project Results:
Security product and application surveys
The objective of this activity was to carry out a broad survey of the different categories & types of physical security product as well as the scenarios in which they are used and their evaluation and certification requirements.
Security product survey
The security product survey provided a foundation for the work of HECTOS by categorising and describing the different types of physical security product that are used in the provision of physical security solutions and describing their principal characteristics. The aim was to compare and contrast the different categories to support the development of the harmonised certification framework and template.
The work included definitions and scope of physical security and physical security products as adopted by HECTOS, and provided an overview of physical security products and their categorisation. Each product category was then analysed and broken down into product types together with a description of the key characteristics of each product type. The product categories are:
• Barriers (Fences & gates, Building components, Vehicle barriers)
• Access Management (Locks, Safes and Security containers, Access control systems, Biometrics)
• Surveillance (Video Surveillance Systems (VSS), Security lighting)
• Detection equipment (Intruder detection alarms, CBRNE detection)
Under each of these product category, there are a number of identified product types (also sub-types where appropriate) which have been described in more detail. A number of relevant characteristics of each category and type are given in terms of product function; technologies used, application, market and user aspects. The product survey was used throughout HECTOS as the basis when e.g. studying the standards landscape and the development of the implementation roadmap.
Security application survey
The security application survey identified the principal security application categories and defined a set of typical application scenarios which was used to help identify product evaluation and certification requirements. They were chosen to be representative scenarios which ‘span the space’ of security application areas and product types.
The application survey described eighteen application scenarios which, whilst not exhaustive, are representative of security requirements and the way that physical security products are deployed to help fulfil these requirements. The scenarios were briefly described in terms of the application area, the types of threat that exist, the physical security measures that are deployed to mitigate the threat and other relevant information about regulations, standards, operator requirements and the impact of the measures on users and others.
The survey also included an estimate of the number of instances of each scenario in the EU and the value of products deployed. The application scenarios guided and supported the research carried out during the HECTOS project, for example to guide and illustrate the development of concepts used in the experimental case studies.
Stakeholder requirements
Requirements for standards, evaluation methods and certification schemes across the different categories of physical security products and application areas were identified. The requirements were synthesised from a number of sources:
• Previous and ongoing work, where available, including other EU research projects, studies and initiatives
• Stakeholders, through a questionnaire, interviews and a workshop
• Consortium knowledge and experience as experts and participants in the development, test, evaluation and use of security products from different categories and application areas
Requirements were presented and discussed from the perspective of the principal stakeholder groups including end users, specifiers, their representatives and advisers; product manufacturers, distributors, system designers and integrators; others with a financial stake such as insurers; standards, evaluation and certification bodies; national and EU governments and regulators. Requirements were grouped into standards-, evaluation-, certification-, and accreditation-related requirements. Product-specific requirements were also identified.
It was found that requirements often vary between different product categories and between application areas. These differences are also identified, since this is important information for investigating the applicability of different potential certification systems and whether or not schemes for products can have similar rules and procedures and thus form a certification system. This was further explored during the development of the HECTOS certification framework and implementation roadmap.
Review of standardisation landscape
The objective of the standardisation landscape review was to get an overview of the maturity of the standardisation activities for the different product categories. The survey revealed details on sectors where standards are existing and utilised, as well as areas where standards are missing or insufficient.
The results of the standards research were divided in different fields according to the defined product categories and the analysis considered national, European and international standards. Standards developed by user and specifier oriented industry organisations and independent certification bodies (such as VdS, LPCB, SSF) were also identified.
It was difficult to carry out a survey across the whole physical security area but the survey provided a reasonably complete and representative picture of the state of standardisation in each of the HECTOS product categories and it is probably the most complete survey in existence.
Taking the HECTOS product categories in turn:
Barriers
• Fences & Gates There appear to be no European or International standards covering fences and gates. There are standards at a national level as well as other standards from governmental organisations and building-industry organisations such as LPCB, particularly for high security applications.
• Building Components A range of European standards cover security functionality of building components such as doors, windows, walling, grilles and shutters. They include standards for burglary/intrusion resistance, bullet and explosion resistance. Various national and other standards also cover this area, especially to define more stringent requirements for higher security applications.
• Vehicle Barriers There is a CWA and IWA which provides a test method for impact crash tests, developed in turn from two UK PAS and elements from an ASTM standard. This provides a good basis for mutually accepted testing. There are however no standards on performance and other performance aspects.
Access Management
• Locks This is a very mature area both for products and standardisation. There are European standards in place with requirements for locks providing different grades of security. As with several other categories, there are national and other standards (e.g. LPCB, VdS) that provide requirements and test methods for higher grades of security – particularly against lock-picking as well as the use of force. There is also a comprehensive range of lock standards in the USA. There do not appear to be international standards in this area.
• Safes and Security Containers This is another mature area. Standardisation is similar to the locks category with European and US standards, but none on an international level. It appears that the requirements and security grades in the EN standards meet the needs of most types of user.
• Access Control Systems In the field of access control systems shows some relevant European standards as well as some of relevance from CPNI (Centre for the protection of National Infrastructure, UK). Some relevant standards can be found as well in the product category of biometrics.
• Biometrics There are comprehensive IEC and ISO standards across many areas of biometric technology. These standards are widely accepted in Europe but are not formally adopted as EN standards. European standardisation activities (i.e. by CEN/CENELEC/ETSI) are in their infancy, but there are some activities starting especially in the automated border control application area.
Surveillance
• Video Surveillance Systems There is the newly adopted EN 62676 family of standards covering Video Surveillance Systems (VSS) based on the IEC 62676. These focus almost exclusively on interoperability aspects and do not cover functional performance. The emerging video analytics area for which functional performance is less well covered by performance standards and test methods. There are some national test and evaluation programs of note, such as the UK iLIDS and US activities at NIST, but no international or EU standards.
• Security Lighting There are no standards specifically covering the requirements of lighting products used for physical security applications. However, there is no perceived need for these, since the requirements are very similar to those which can be found in other lighting applications.
Detection
• Intruder Detection Alarms are covered by a comprehensive series of EN standards which mirrors a corresponding series of international IEC standards. These provide a graded set of system and component product requirements and test methods. Some Member States have a small number of country-specific or additional requirements leading to separate national standards (for example to meet the needs of local policies and procedures for police response). It is possible that some standards in the national or other category provide more comprehensive test methods than those in the EN/IEC standards.
• Chemical Threat Detection There are no European or International standards in this area and only very limited coverage in other standards (just three ASTM standards covering chemical vapour detection products).
• Biological Threat Detection There are no European or International standards in this area and only very limited coverage in other standards (just one ASTM standard).
• Explosives & Weapons Detection There are no European standards for explosives and weapons detection products and only two IEC standards, which apply to just one of the many product types in this category. Products of various types used in aviation security, which is the largest market for explosives detection equipment, are covered by ‘other’ standards in the form of the EU regulations on aviation security equipment and the ECAC test methods, both of which are classified documents. Outside aviation only a limited number of standards exists; that is for people screening equipment, X-ray systems for baggage/cargo inspection, and Explosives trace detection.
• Radiological & Nuclear Threat Detection is well covered by international, mainly IEC, standards. A number of these have been formally adopted as EN standards.
From the above, it is clear that there are several areas where there are few standards at the European or International levels. Clearly, if standards are not available or adequate, it will be impossible to implement harmonized certification schemes.
There are a number of areas where the requirements of particular application areas, such as high security levels demanded by government, critical infrastructure and other users, have led to standards which add additional requirements to those in the European standards and/or more rigorous tests. These test often involve more realistic (if less objective and repeatable) tests in addition to those in the EN standards. This needs to be investigated since it limits the degree to which product evaluation and certification to EN standards is sufficient as the basis of a European ’quality mark’.
Review of existing certification systems
A review of existing certification systems and schemes was carried out in order to provide a categorised list of evaluation and certification schemes used in the area of physical security products, on European and national level. The aim was to describe each evaluation and certification scheme by several common elements, both in tabular and in narrative format, and by the test method that is involved. The final goal was to enable a comparison of existing schemes and to identify common and missing process elements. The overview and comparison of the evaluation and certification schemes was used for the development of the certification framework and implementation template. The results provided an overview of certification systems/schemes in a number of European countries (driven by the consortium composition) as well as relevant schemes in the USA. It also included approaches that already take steps towards harmonisation of certification systems, like for example the European Fire and Security Group (EFSG), Keymark and the framework for conformity assessment presented in ISO/IEC 17067. Furthermore, some evaluation and certification systems outside the security domain are given as well since these provided interesting elements that can also be used for the HECTOS harmonised approach. Taking the HECTOS product categories in turn, most products are certified through schemes operated at a national level where they apply their own marks which are mostly recognised at a regional level. For some product types, such as chemical and biological detectors, no certification system or scheme exist neither at national or European level.
What is special about Security?
Security products differ from other types of product in that they help protect against attacks by an intelligent adversary with malicious intent. Attackers are likely to:
• Constantly probe for and exploit weaknesses in the products
• Attack in predictable and unpredictable ways
• Constantly adapt and change their method of attack
Consequently, schemes need to focus on realistic and adversarial testing; support evolving threats and requirements; and support mechanisms to handle security sensitive information. Products of a given type may also have a very wide range of performance requirements depending on the application. Therefore, a number of special features need to be built into performance measurement and conformity assessment processes in security certification schemes, in order to take these aspects into account. These include features covering:
• Realistic and adversarial testing. Performance is often defined in terms of the protection from real threats, for example how long a barrier resists an attack. Users require realistic testing and, in some cases adversarial testing where the tester exploits weaknesses in the product in the same way as an attacker.
• Consistency in testing. Tests involving human strength and skills need to be standardised, for example so that testers do not apply too much or too little strength. Procedures are needed to ensure testing remains consistent over time and between different test laboratories.
• Complex performance information. Performance information such as detection rates is often complex and test methods needs to be precisely defined.
• Security sensitivity. It may be necessary to restrict access to product performance requirements, test methods and test results in order to prevent weaknesses and gaps from being identified and exploited by an attacker.
• Continually evolving requirements. Changing threats lead to constant change in product requirements. Standards and test methods for security products need to be updated regularly to accommodate these changes, without placing too large burden on manufacturers
• Wide range of applications and performance requirements. Many security products are used in a variety of different applications subject to different threats. Schemes need to be carefully managed to ensure that testing covers the range of requirements and that there is clarity over the applications for which a product is and is not certified.
• Diverse range of types of product, markets and levels of maturity from low cost consumer products selling in millions to specialist equipment with only a handful of sales per year. Certification schemes need to be based on what the market needs and can support.
The certification framework and implementation template described in the following sections accommodate these features.
The HECTOS framework for harmonised certification of physical security products
The HECTOS certification framework was developed to provide a mechanism for the creation of harmonised conformity assessment schemes that enable the mutual recognition between EU Member States and other participants, support a Single Market and international trade and enable end-users to implement better security capabilities to mitigate the risks they face.
The certification framework is based on the ISO/IEC 17000 Conformity Assessment family of standards, adapted and supplemented by features to support the special requirements of security products. The certification framework comprises:
• A top-level coordinating structure
• Certification systems for related product and application areas
• Certification schemes with common rules and procedures applying to all the products in the scheme
The framework accommodates all forms of conformity assessment including pass/fail testing against a threshold and performance measurement schemes. Typically, for a threshold performance scheme, product or application requirements will be set out in a specification standard. For a performance measurement scheme, the evaluation process will be defined in a measurement standard (also known as test method).
The system group level provides an overall common structure and a security mark - a security-specific quality mark indicating that a product has been certified according to the common framework. A database over certified products should be available and administrated at system group level.
Individual systems manage the certification of different product groups (and sometimes applications) each having their own specialist subject matter expertise and application requirements.
Schemes within each of these systems certify one or a group of related product types each with its own standards and/or test methods, setting out a specific set of requirements. These individual product certifications within a scheme are also defined by other controlling documents for ensuring consistency of conformity assessment activities.
So-called System Committees and scheme Working Groups bring together the product-specific technical expertise to manage and maintain systems and schemes. They link with standards bodies to help maintain and facilitate updates to standards.
At the system group level, a framework set of documents will exist, describing overall basic rules for participation and the terms for usage of a common security mark. The system level will further develop a set of key documents describing common rules and procedures for all constituent schemes.
Each product certification within a scheme is defined by a set of controlling documents which set out the specific technical product and evaluation requirements as well as the product-specific technical aspects of operational processes and procedures such as proficiency testing and surveillance. Threshold performance schemes have a specification standard, which sets out the functional performance (and other) requirements, together with a scope defining the product types and applications to which the certification applies.
Performance measurement schemes are defined by a measurement standard (test method), which needs to be sufficiently detailed to remove ambiguities which might cause inconsistency in application.
Proficiency testing documents define how intra and inter-laboratory comparisons, peer assessment and other techniques should be carried out to ensure consistency over time and across test laboratories. Surveillance methods, where included in the scheme, define technical details of procedures to ensure consistent production quality over time.
The templates for the establishment and maintenance of schemes
The HECTOS certification framework accommodates a wide variety of schemes, defined by product type and/or application area. To support its implementation, two templates have been prepared providing guides and step-by-step checklists of the activities that need to be carried out in establishing and maintaining schemes and systems within the overall framework. The templates describe the steps that need to be considered to facilitate consistent implementation of systems and schemes. It consists of nine activities and designed as a “pick and choose” guideline – i.e. schemes that do not benefit from all of these prerequisites do not need to apply them. The activities are:
I. Identify Scope
a. Product type(s)
b. Application
c. Identify purpose of scheme
II. Identify scheme fundamentals
a. Establish the prospective scheme owner and a preliminary working group
b. Identify stakeholder groups
c. Perform an initial survey of existing standards and requirements landscape
d. Identify a financial model
III. Identify system structure
a. Identify existing systems
b. Identify system owner and management
c. Identify security specific management
IV. Identify standards
a. Identify relevant product and measurement standards
b. Identify harmonised and local standards
c. Identify need to develop new standards
d. Review stakeholders acceptance of identified standards
V. Identify detailed requirements
a. Establish performance measurement and threshold performance scope
b. Identify functional & non-functional requirements
c. Identify threshold performance and performance measurement requirements
d. Identify different and conflicting requirements
e. Identify steps for achieving common view in case of conflicting requirements
f. Identify the security sensitivity of information
g. Review acceptance of requirements
VI. Identify test methods
a. Survey and identify existing test methods
b. Adopt existing test methods
c. Develop new test methods
d. Identify the security sensitivity level for test methods
e. Identify ethical and legal compliance requirements
f. Review acceptance of test methods
VII. Identify scheme structure
a. Select ISO/IEC 17067 scheme types 1-5
b. Identify scheme owner and management
c. Scheme certificate
d. Identify scheme rules and classification
VIII. Identify and establish qualification methods
a. Identify operator qualification requirements
b. Identify accreditation needs
c. Identify and establish laboratory consistency methods
IX. Identify periodic surveillance methods for maintenance of certification
a. Determine surveillance needs
b. Identify scope of surveillance test methods
c. Identify procedures and frequency of surveillance activities
d. Identify validity of certificate
Case study: Biometric products
To analyse, develop, enhance, and experimentally validate evaluation and certification schemes, HECTOS conducted an experimental case study on biometrics products. The following topics were selected:
Topic 1 — Image quality of contactless fingerprint sensors
Topic 2 — Presentation attack (spoof) detection capability and presentation attack resistance of biometric systems
Topic 3 — Products for secure biometric access control to critical infrastructure
For each of these topics, performance requirements and associated evaluation methodologies were specified, based on a risk assessment for the selected applications, as well as requirements and associated test methods of existing standards:
For topic 1 — Established requirements and test methods associated with 2D fingerprint sensors
For topic 2 — Common Criteria guidance, published protection profiles for fingerprint spoof detection, output from the EU project BEAT, and related ISO/IEC standards under development
For topic 3 — A proposed CEN technical specification for biometric authentication for critical infrastructure access control (which itself was based on two similar national standards addressing biometric access control in critical infrastructure)
This biometrics study analysed and assessed the evaluation and certification processes and outcomes for the biometric case study topic against the elements of the preliminary HECTOS certification framework and template and provided recommendations to enhance the viability of a harmonised certification scheme for security products. Key issues examined include:
• Development and agreement of conformity assessment certification criteria across multiple technologies (e.g. both legacy and new biometric technologies)
• Harmonisation of requirements between related applications
• Inter-laboratory comparison, in particular identifying suitable procedures to ensure repeatability for adversarial tests (i.e. presentation attack resistance / presentation attack detection in the case of biometrics)
• Consideration of the possibility for certified evaluation independent of conformity assessment with quantified performance requirements, or thresholds, and
• Consideration of the requirements for accreditation (e.g. interlaboratory comparison, publication of test results) to enable re-use and mutual recognition of evaluation results.
Case study: Weapons and Explosives detection products
The second case study which assisted the development of the certification framework and template was on Explosives & Weapons detection equipment:
• Explosives trace detection equipment (ETD)
• People screening portals
The study developed test methods whose elements were investigated in detail for both ETD and people screening portals.
ETD
The review and comparison of existing test methods (in measurement standards, such as ASTM E2520-07) showed that these still do not solve the problem of a harmonised evaluation of ETD in areas outside aviation security. Various requirements for harmonised evaluation were assessed and it was concluded that the most important of them are concerning security performance by measuring of detection rate and false alarm rate. These two are not fixed parameters but are very dependent on the amount and the selection of threat and background substances. Therefore scenario related requirements have been reported in order to allow scenario adjusted evaluation.
Interlaboratory comparisons were extensively investigated and the results were used to contribute to the testing relevant parts of the HECTOS certification template. These studies showed that sample preparation was the most crucial part in the test methodology. Though the amount of explosives traces can easily be controlled by dispensing diluted solutions, the amount actually available for sampling of this traces from surfaces cannot be controlled precisely. Different preparation methods were tested with several different surfaces and explosives. It turned out that apparently no method fits all and the optimum preparation method has to be chosen from case to case. A two-step evaluation process was recommended (i) the use of samples of high accuracy (and thus higher degree of repeatability) but somewhat less relevancy and (ii) more realistic samples at the expense of larger variance.
The result of the interlaboratory test showed some interesting discrepancies. Especially the description of the sampling-method of volatile explosives within the test method was not clear enough and caused apparently misunderstandings and in consequence big differences concerning the detection results.
People screening portals
Intra-lab repeatability was investigated by carrying out twice a Security Scanner (SSc) test, using different test managers. For both tests the test managers were responsible to carry out the test according to the same test methods, but the tests differed in test persons, the exact way of carrying out the test and alarm indication interpretation. ROC-curve evaluation was also carried out for Walk Through Metal Detectors (WTMD) as well as a theoretical study on the technology independence of test methods for people screening.
The ROC-curve based evaluation showed that person-based testing allows for a more realistic assessment of the WTMD than tests without test persons. Due to a number of deliberately uncontrolled but realistic parameters, the measured detection rate shows a high variance. Person-based, realistic testing comes hence at the expense of accuracy and repeatability (which are of interest for mutually comparing the performance of WTMD systems). It was also concluded that mixed threat item / innocuous item testing gives a more realistic impression of the actual behavior in the field because of the interference that can occur between two or more items. An important consequence of this interference is that threat items may be found in mixed mode while missed in full divestment mode. The detection rate in mixed mode may thus be higher than for fully divested screening. The ROC-curve based evaluation seems to be especially useful when operated in partially divested mode. For non-divested mode full detection is obtained, even at the lowest sensitivity and with and without threat items. For full divested operation, the False alarm rate is zero and no ROC-curve is obtained. It is believed that the ROC-curve approach enables type certification based on its intended application, i.e. divestment level and sensitivity setting.
The intra-lab repeatability of the SSc test was reasonably good but with some statistically significant differences. Based on observations made during the test, the repeatability is thought to increase if a larger number of test persons are used and when the exact threat item location, orientation and way of attachment to the body is prescribed in the test protocol.
The technology independence of test methodologies was investigated on metal detectors based on passive magnetometry as used in some walk-by metal detectors (WBMD), the signatures that they detect and a discussion of testing considerations that apply to this technology. This showed that test methods developed for WTMD and SSc will not be suitable for passive magnetometer testing unless the magnetic signatures of test objects and environmental influences are very carefully taken into account as well. Consequently, a one-test protocol-fits-all for people screening detection systems is not realistic. It must be ensured that the assumptions made in the development of a test method are valid for all the different types of equipment that may be tested. The operating principles of the device under test have to be examined to determine whether the proposed test method is suitable or needs adjustment to fit those operating principles.
Final verification of the certification framework and template
HECTOS investigated how the harmonised certification framework could be applied to several physical security products used in biometrics, explosives & weapons detection equipment, radiological & nuclear detection equipment, chemical and biological detection equipment and locks. Additionally, the applicability of the HECTOS framework and template for security systems was explored for video surveillance systems.
These studies initially summarised the historical needs and developments in the field of evaluation and certification for the types of product and the current international, European and national activities with respect to testing, standards and certification. Next, the HECTOS certification framework and template for establishing a new certification system and/or schemes were elaborated and verified, via case studies of the above-mentioned product categories and the needs of end-users. The template itself was utilised as guide in this process to identify the current maturity with respect to the implementation of standardisation and harmonised evaluation and certification schemes, and to provide suggestions on how harmonised schemes could be introduced, identifying activities for scheme functions and features that yet need to be addressed. The analysis indicated that E&W detection equipment for aviation security is a relatively mature, while E&W detection equipment for non-avsec applications is very immature with respect to the implementation of harmonised evaluation and certification schemes. Biometrics, RN detection equipment and locks have a similar maturity with respect to the implementation of harmonised evaluation and certification schemes but face different challenges. CB detection equipment is another product type showing low maturity. There is a need in nearly all areas for further work on harmonised standardisation, both in product and measurement standards, which is a prerequisite for establishing harmonised schemes.
The case studies have shown that the framework provides a basis on which sets of harmonised schemes could be developed for the different categories of physical security products. The framework and template have been found to be useful in identifying elements that exist; gaps where further work is needed in terms of requirements development, standardisation and development of schemes; and the potential types and structure of certification schemes that are required in each area.
Implementation roadmap
The validation of the certification framework and template assisted the development of the implementation roadmap that shows a possible way to implement the proposed framework for certification and evaluation of physical security products on a long-scale time frame. It consists of two types of roadmaps:
Enabling infrastructure roadmap – comprising a number of roadmap high-level elements ranging from dissemination and awareness building of the proposed concept through its piloting to the expansion into various physical security certification systems. The system group coordinator has a key role in these activities.
System roadmaps – describing the highlights of the way ahead towards implementation of the proposed harmonised European certification schemes for biometric products, explosives & weapons detection equipment, radiological & nuclear detection equipment, chemical and biological detection equipment and locks. By assigning values for the status quo and the complexity to further implement harmonised schemes for each of the analysed products, arbitrary indicative scores were obtained for the anticipated complexity to further implement harmonised certification for each product studied. The assessed complexity was analysed and compared with the status quo. The following conclusions could be drawn:
• Explosives and Weapon Detection system for aviation security is relatively mature as a harmonised system, but the remaining steps might be challenging
• RN Detection system is quite mature in terms of standardisation, although there are no certification schemes outside the USA, and seems to be the less complex case to implement, provided there is a business case
• Biometrics and Locks are both quite mature but remaining harmonisation is anticipated to be challenging
• CB Detection is very immature and many steps are expected to be quite complex to implement
• Explosives and Weapon Detection system for non-aviation security is the least mature system and expected to be the most complex to implement
Considering the degree of complexity and maturity of the template steps needed to be taken as presented above, the RN detection system, Explosives and Weapon Detection system for aviation security and the Biometrics product system are foreseen as the most suitable candidates for a pilot study on the implementation of a certification scheme using the HECTOS template and framework.
Testing of early stage technologies
HECTOS also studied the development of approaches to the testing of low-TRL prototypes in the physical security domain, aiming to deliver the following anticipated benefits:
• To enable researchers and technology developers to assess the maturity of their work and the potential of the technology
• To guide researchers by helping them understand the real world environment their technology will need to operate in and identify the areas where further development work is needed
• To enable technology developers to communicate effectively with their peers and with their sponsors or funding agencies about the progress and potential of their work
Early in the process it was decided to divide physical security products into two broad categories; technologies which ‘Detect’ and those which ‘Delay’ an adversary. It was identified as necessary to take a different approach towards testing in each case. The majority of the effort in this exercise was focused on developing early stage testing approaches for the ‘Detect’ technologies. This was done using case studies of two different types of explosives and weapons detection technology.
Using background research into other existing early stage testing concepts in both security non-security markets, as well as prior experience of working with early stage security technology, an initial high level approach to the generic low-TRL test methodology was developed. This was tested and refined through the development of two technology specific case studies. The first case was test and evaluation of optical Raman trace explosive detection techniques and second case was test and evaluation of millimetre wave techniques used for detecting concealed explosives and weapons.
The draft Raman test method was used to test actual hardware prototypes under development to explore which aspects worked well and identify areas for improvement. The draft millimetre wave test method was reviewed by a number of independent subject matter experts comprising developers, system integrators and end-users. For both test methods, feedback was very positive. As well as providing the basis for the methodology development, it was felt that both test methods could be used by the R&D community to support a number of ongoing development projects. Using the experience and insight gained in the case study process, the generic methodology has been refined and developed to the extent that it now stands alone as, hopefully, an accessible guide for users to understand the principles and apply this approach for a new detection technology development.
It was found through the process of reviewing approaches in other fields that although there were no directly applicable methods, there were lessons that could be learnt, and good generic principles that could be extracted from other early stage testing examples. The process by which the test methodology was generated was found to work well. By building on the high level initial approach to develop the two test cases, then challenging them through lab testing or peer review, the findings were found to be valuable in refining the generic low-TRL test methodology. The defining principles were generated at the start of the process as high level goals for the activity, and refined throughout by a process of retrospective review. It is considered that the methodology produced in HECTOS meets these principles, and thus hopefully will act as a useful tool to support and guide the development of new technologies. There is clear scope to continue this activity, and exploit the outputs of this research and these potential activities are outlined in the section below about exploitation activities.
Law and ethics
HECTOS has studied the ethical and legal aspects of evaluation and certification of physical security products, such as:
• Ethics and Human Rights Risks of Security Products
• Ethical/human rights aspects of application scenarios
• Privacy by Design and Ethics by Design
• Legal issues raised by the evaluation and testing of security products
The work was led by University of Warwick and it started with an overview of the ethics and human rights issues that arise in connection with the use of security products. Distinctions and overlaps between ethics and human rights were described. The following ethical issues were analysed: privacy; data protection; mission creep; discrimination; error; consent; dual use; freedom of movement, expression, and association; and health and safety concerns, as well as how these issues arise in connection with each of the 4 product categories focused on by the HECTOS project (i.e. Barriers, Access Management; Surveillance, and Detection).
For the ethics and human rights risks of the technology use described in the application scenarios defined by HECTOS, the most commonly arising issues were identified to be: privacy, data protection, freedom of expression, association, and movement, health and safety concerns, proportionality, and issues around consent. It was found that some of these issues can be addressed at the stage of product design and manufacture. Others can only be addressed by changes to the processes and procedures around technology use, such as training of staff, codes of conduct, protocols and standard operating procedures.
University of Warwick organised two meetings, each of which address the Privacy by Design and Ethics by Design aspects of Person Screening products and Biometric Access Control Products. The meetings gathered expert opinion on inclusion of ethical and legal compliance assessment in evaluation and certification schemes and explored the use of specific ethical standards such as privacy-by-design within this context. Speakers included HECTOS Case-Study leads, manufacturers, Privacy by Design, Data Protection, and Privacy Impact Assessment experts, lawyers, and commercial providers of Privacy by Design Certification. This resulted in a number of recommendations for the incorporation of Privacy-by-Design and Data Protection principles into products via certification.
Competition issues that arise currently in relation to the standardisation and certification of security products within the EU, and how a harmonised approach may affect these were also investigated through interviews with experts, such as competition lawyers and other relevant stakeholders. Key themes that arose during the discussions included: the ambiguous role of both of standards and standards-bodies and of testing houses in facilitating competition; the tension between, on the one hand, conceded national prerogatives with respect to the procurement, development and sale of products, and on the other, the need for larger competitiveness across the EU; and the challenges to entry into the market for smaller players in light of the power of larger players to influence standard-setting and to inclusion in procurement processes. It was found that harmonisation encourages the creation of common standards and, though standards are desirable, they must be established in an open way and then made generally available, without participation being limited to big or well-resourced players.
Expert and Advisory Group
The HECTOS Expert Advisory Group (EAG) with stakeholders drawn from manufacturers, trade associations, users, government security agencies, representatives of the standardisation, evaluation and certification communities, was established initially from the 12 organisations that offered to join such a group and who wrote letters of support for the proposal. It was expanded as the consortium identified and engaged with other stakeholders identified during the project, in particular to include the IEC Conformity Assessment Board (CAB) and DIN CERTCO. The EAG members at the end of HECTOS were:
• European Organisation for Security – EOS
• Perimeter Security Suppliers Association – PSSA
• Euralarm
• Smiths Detection
• Bruker Daltronics
• UK Home Office Centre for Applied Science & Technology - CAST
• UK Centre for Protection of National Infrastructure - CPNI
• JRC IRMM
• BMI – German Interior Ministry
• AFNOR
• BKK – German Civil Protection Office
• The Swedish Theft Prevention Association, SSF
• British Standards Institute –BSI
• German Federal Police Technical Centre –FPTC
• Bundeskriminalamt – BKA
• Loss Prevention Certification Board –LPCB
• European Fire & Safety Group – EFSG
• DIN CERTCO
• IEC Conformity Assessment Board
In addition, the project continued to engage with other stakeholders such as Assa Abloy, the Swedish evaluation and certification body RISE (former SP), the CRISP project, DG Home (in its role as EC DG Home - Innovation and Industry for Security), CEN TC/391 Societal Security committee as well as a number of others from various areas.
Meetings and discussions were held with EAG members throughout the project to keep them up to date with our work, seek their guidance and discuss certification issues. In particular the EAG members provided the bulk of the stakeholder requirements, helped in the review of the HECTOS certification framework and the decision to develop it as a CWA and made significant contributions to the CWA itself.
Stakeholder meetings were held in Brussels in 2015 and 2016 and attended by a good range of stakeholders. These meetings were a good opportunity to present the project activities and obtain feedback to validate the approach being taken,
The third stakeholder meeting and HECTOS project final event was held at the Bouche à Oreille (BaO) conference centre in Brussels on December 6th 2017. The event was held as part of the 9th meeting of the European Commission’s Community of Users on Secure, Safe and Resilient Societies.
Dissemination Plan & Reports
An initial Dissemination Plan was produced for internal use within the project during the first few months of the project. This was revised and expanded at the end of first year of the projects in the light of the consortium’s growing knowledge and experience and to incorporate the results of discussions with the Project Officer on project strategy, objectives and alignment with other initiatives.
Public and Targeted Dissemination Activities
A good level of dissemination activities has taken place throughout the project including the project including the website and project booklet; stakeholder events; presentations at conferences and specialist meetings; discussions with individual stakeholders; dissemination of deliverables; and the project website.
The project website has been maintained throughout the project at http://hectos-fp7.eu/. This contains an overview of the project, news items, summaries of all deliverables and downloads of the public deliverables from the project.
A ten-page printed project booklet providing an overview of the project and its main results was prepared for use at the HECTOS final project event. Copies were provided to partners for distribution to their stakeholder contacts. A pdf version of the booklet is also available for download on the HECTOS website.
Liaison activities with the FP7 CRISP project continued throughout the project and representatives of each project attended the others stakeholder meetings. Deliverables were shared with CRISP and with other relevant FP7 and H2020 projects.
Team members attended and presented at a range of meetings and conferences throughout the project, giving presentations on the overall project, the HECTOS Certification Framework and on individual specialist topics from the explosives & weapons detection and biometrics case studies.
Towards the end of the project a ten-page printed booklet was produced describing the key results from the project. This was published on the HECTOS website alongside the summaries of project deliverables and downloads of the public deliverables from the project.
Standardisation Activities
A formal liaison with CEN TC/391 “Societal and Citizen Security” was finalised in September 2016. HECTOS representatives attended meetings of the CEN/TC 391 to facilitate knowledge exchange and HECTOS obtained relevant documents dealing with current work of CEN/TC 391, as well as information about the current and future work programme.
As the second period progressed and with the development of the HECTOS certification framework and implementation template, it became clear that HECTOS could raise its standardisation ambitions and attempt to progress the framework through a CEN/CENELEC workshop, resulting in the publication of a so called ‘CEN/CENELEC Workshop Agreement (CWA).
CEN/CENELEC also agreed an ‘open access’ model for CWAs resulting from European research, whereby publication costs would be paid by DIN from their project budget and the CWA would be available for download free-of-charge
After the successful initiation of the CEN/CENELEC Workshop HECTOS with the aim to publish a CEN/CENELEC Workshop Agreement (CWA) on Guidelines on evaluation systems and schemes for physical security products two physical meetings (including the kick-off) as well as four web conferences were conducted to develop a draft document. Due to the strong participation of external stakeholders in the workshop several fruitful discussions took place in order to adapt the proposed HECTOS results to market and user needs.
In December 2017 a first draft of the CWA was finished and provided to the public for a commenting phase. More than 300 comments from both workshop participants as well as external, non-workshop members were received. The final document was then developed and approved by the registered workshop members by end of January. After approval of the document by a majority of participants and final editorial improvement, the CWA was sent to CEN/CENELEC Management Centre for publication on 31th January 2018.
The final CWA document was approved by 12 of the participating organisations:
• Swedish Defence Research Agency, FOI (Chair)
• Iconal Technology Ltd (Vice-Chair)
• Asociatia Romana pentru Tehnica de Securitate, ARTS
• BRE Global Ltd
• DIN CERTCO Gesellschaft für Konformitätsbewertung mbH
• European Certification Body (ECB) GmbH
• European Security Systems Association (ESSA) e.V.
• IDEMIA
• Fraunhofer-Institut für Chemische Technologie
• Fraunhofer-Institut für Grafische Datenverarbeitung
• National Physical Laboratory (NPL)
• The Netherlands Organisation for Applied Scientific Research (TNO)
No organisation disapproved the CWA and only one abstained.
The CWA will be published as CWA 17260:2018 Guidelines on evaluation systems and schemes for physical security products, free of charge and publicly available via the CEN-CENELEC webpage.
Overall, HECTOS has ensured that it is visible to and engaged with a wide range of stakeholders representing different aspects of the field. In particular, the project has sought to listen to and take input from stakeholders in order to be aligned with and relevant to both user and supplier industry needs. As discussed above, channels have been identified for the dissemination and exploitation of project results.
As the research on the project starts to yield results, these are being disseminated to the security, standards and evaluation/certification communities so that the results of the project will be exploited in due course.
The continuing stakeholder contacts and posters/presentations at conferences have maintained awareness in the community. Technical results from the case studies are starting to be disseminated and exploited.
Potential Impact:
Potential Impact
HECTOS could have a significant impact on physical security, helping protect citizens from serious crime and terrorism, whilst supporting economic development through the single market. Improved physical security product evaluation and certification mechanisms, such as those enabled through the work of HECTOS, should lead to improved security systems and capability for end-user organisations as well as increasing the understanding of and trust in the performance of the component products from which they are constructed. Harmonised evaluation and certification schemes, focused on the particular characteristics and needs of security products, such as those based on the HECTOS certification framework, should reduce the cost and time-to-market for product manufacturers by removing the need for different tests and certification activities in different Member States. They should also make it easier for end-user organisations to understand and select appropriate products for their needs, thus encouraging the deployment of more effective solutions. Benefits are foreseen for a range of industry stakeholders:
End-user organisations, specifiers and advisers
- Better and trusted information on the performance of security products
- Better information on which to base the design of security systems and processes to protect against attacks from attackers with malicious intent
- Reduced product costs
- Improved availability of new capabilities
Manufacturers
- Reduced costs of evaluation and certification by avoiding the need for multiple certifications in different countries
- Faster time to market, leading to competitive advantage
Government and regulators
- Improved security of the citizen and the state
- Protection of fundamental rights
- Improved economic position of Member States
- A potential roadmap for the introduction of harmonised evaluation and certification schemes for security products
Security research & development community
- Improved test and evaluation approaches for early stage detection technology development to guide and support the R&D process
- Specific test and evaluation techniques to address key technical challenges in the case study areas of biometric products and explosives & weapons detection
Standardisation, Test & Evaluation, Certification organisations
- Improved procedures and guidelines for the development and implementation of evaluation and certification schemes, recognising the particular requirements of security products
- Improved test methods in the areas of biometrics and explosives and weapons detection, addressing issues of repeatability and consistency – over time and across tests both within a test laboratory and across different test laboratories
The HECTOS vision is to implement harmonised certification schemes based on the HECTOS framework across all physical security products. Whilst it is recognised that this is a long-term ambition which will require the involvement and cooperation of governmental, industry and end-user stakeholders across Europe, much can be done today with the results of the project:
• The HECTOS roadmap provides for an incremental introduction of harmonised evaluation and certification schemes. These could be introduced initially in one or two areas, perhaps where a good level of standardisation exists, but where formal certification schemes are lacking.
• The concept of Measurement Schemes as well as Threshold Performance Schemes enables certification of products where the wide range of applications, each with their own performance requirements, makes the concept of certifying ‘minimum performance levels’ impractical
• The low-TRL testing methodology work to support the security R&D community is already being exploited.
• Test methods and techniques investigated and developed as part of the HECTOS case studies are already being considered and used in the explosives detection and biometrics communities.
Dissemination
The project has worked closely with the stakeholder community (end-user and specifier, manufacturer, governmental, R&D, standardisation, test & evaluation and certification bodies) throughout the project. A dialogue has been maintained both to help guide and shape the work of the project and to disseminate the vision and results of the project as they have emerged. The principal dissemination activities during the project have been:
• Development of a dissemination plan analysing and defining the project’s communication and dissemination strategy to all the various stakeholders and the general public.
• Project website http://hectos-fp7.eu/ with general information, deliverable summaries and public deliverables
• Engagement and relationship building with stakeholders of all types across the wide range of different physical security products
• Regular stakeholder meetings to communicate the results of the research
• One-to-one interactions with key stakeholders – meetings and presentations throughout the project.
• Sharing of project deliverables with other EU research projects and stakeholders
• Presentations at industry conferences and published papers and posters – at a wide range of events in Europe and USA.
• Engagement with stakeholders during the HECTOS laboratory test-based case studies in explosives & weapons detection and biometrics, as well as in the paper studies on potential future certification schemes in other areas.
• Engagement with and participation of stakeholders in the development, review and approval of the CEN/CENELEC Workshop Agreement CWA 17260 Guidelines on evaluation systems and schemes for physical security products
• HECTOS project Final Event and the associated project booklet distributed via the partners and on the project website
• Dissemination of the HECTOS Roadmap (D8.1 D8.2 titles) as public deliverables
• Publication of the HECTOS Certification Framework as a formal European standardisation product with stakeholder approval for credibility and open access publication to maximise its dissemination.
Exploitable Results
The exploitable results of the project divide into three groups:
1. Framework & Roadmap
- Overall HECTOS physical security product certification approach, Certification framework and its supporting template, plus the HECTOS Roadmap
2. Case study results
- Specific knowhow, test methods and approaches developed to explore and address specific challenges in the evaluation of biometrics and explosives & weapons detection products
- Low TRL testing methodology and example case study test methods
3. Physical security and certification expertise
1. Framework & Roadmap. The overall HECTOS physical security product certification approach, certification framework and its supporting template, plus the HECTOS Roadmap are the main results of the project and are the items which have the potential to achieve the impact described above. Exploitation of these requires action from certification bodies who will own and operate schemes, supported by a broad spread of physical security product manufacturers and user organisations. Direct support from the European Commission, either in the form of legislation to force behaviour or support actions to encourage it is almost certainly also required. At the very least the Framework needs to be endorsed at European level, as described in the HECTOS roadmap.
None of the project partners are certification bodies able to drive this exploitation. Consequently, these principal results have been placed in the public domain as public deliverables from the project, or in the CEN/CENELEC Workshop Agreement CWA 17260 Guidelines on evaluation systems and schemes for physical security products submitted to the CEN/CENELEC Management Committee (CCMC) on 31th of January 2018 for proofreading and publication. The CWA will be available to download at no cost from the CEN/CENELEC website, which should increase its dissemination. The HECTOS roadmap is another public deliverable (D8.2) showing how the overall Framework could be implemented, together with certification systems and schemes for individual product categories.
The HECTOS partners will support organisations who wish to establish schemes based on the HECTOS Framework through the provision of consultancy services. The partners will also support other organisations who wish to provide similar services (A former employee of partner Iconal has already set up a consultancy organisation with this aim in the RN detection area. The HECTOS partners have agreed to make deliverables available to this organisation).
2. Case study results. The HECTOS case studies have also led to significant exploitable results including:
• An evaluation scheme for contactless fingerprint acquisition from the biometrics case study scheme, which has already been accepted and used by the scheme owner. The updated scheme may now be integrated in the next revision of ISO/IEC 19794-4 standard.
• Work on repeatability of results of Presentation Attack Detection evaluations which has helped to define new methodologies. Further work in this area may help harmonise results from evaluation bodies.
• The biometric case study on Biometric Access Control to Critical Infrastructure which has provided relevant data and recommendations which are currently being standardised.
• Interlaboratory comparison of biometrics Presentation Attack Detection (PAD) testing, leading to input to standards.
• Establishing (potential) baseline performance requirements on PAD and inputs to technical specifications and evaluation standards in CEN & ISO
• Work on the repeatability and interlaboratory comparisons of explosives trace detection product evaluations, which are being discussed with the European Civil Aviation Conference (ECAC) for the evaluation and certification of aviation security equipment.
• New ideas on test method development & testing of people screening portals, especially for ROC curve measurement, which are also being discussed with ECAC for future aviation security test and evaluation
• The low-TRL testing methodology for detection technology which is already being used by Iconal and FOI in their own detection technology R&D activities and which is being exploited by Iconal for the benefit of UK Government funded explosives and weapons detection research projects in aviation security and non-aviation applications.
3. Physical security and certification expertise. All of the partners have gained considerable experience and expertise in:
• The wide range of different types of physical security product
• Standardisation of security products
• Test and evaluation of products
• Certification systems and schemes
• The specific needs and challenges of security product evaluation and certification.
All of this knowledge and expertise is exploitable. In particular, the project has enabled the partners to gain expertise across the whole range of different types of security product and application. Few organisations have this breadth of knowledge, making it very exploitable.
This expertise will be used to further the security research and development as well as the consultancy and services businesses of the various partners.
Partner exploitation plans
The specific exploitation plans of each partner are summarised below:
FOI plans to support the adoption of the HECTOS Framework and Template by maintaining the dialogue with EC, Certification and Evaluation Bodies, Standards bodies and other key stakeholder, To the extent possible, FOI will use the CWA (or parts of it) in our own evaluation activities, thereby showing its usefulness to the security community. If the HECTOS Framework is implemented for explosives detection products, FOI may participate as an evaluation body. FOI will also use the HECTOS low-TRL testing methodology in its own research activities.
Iconal Technology is already exploiting the low TRL testing methodology work, both by using it on two of our own research projects and in a UK Government project to make it available to other explosives and weapons detection research projects in other organisations. Iconal also plans to use the experience and expertise it has gained in development of standards, evaluation and certification in its business activities supporting technical and policy makers in UK, European and US government organisations.
TNO plans to apply expertise gained on physical security products in general; on the certification framework and guidelines for physical security products; and, on the possible implementation steps for certification schemes for VSS, explosives and weapons, chemical and biological detection in its consultancy role to policy makers and test and certification bodies. TNO will apply the results of the case studies and expertise gained on test method development & testing of people screening portals in work as an ECAC test centre and member of the ECAC CEP Management Group and Technical Task Force. TNO also plans to promote of the HECTOS certification framework and implementation template whenever possible and appropriate during national and international meetings and conferences.
NPL will use the results of HECTOS to extend interlaboratory comparison methods to encompass measurement of security systems and more generally to extend its techniques developed for biometric evaluation to other types of security product. Practical experience and knowhow gained on the project will feed in to its biometrics standards projects in SC37, CEN.
Fraunhofer IGD will seek to exploit the HECTOS results, in particular the results from the biometrics case studies, in an ongoing project supporting the German government in standardisation in the field of biometrics, and in other standardisation projects dealing with security evaluation of biometric technology. Furthermore, IGD will exploit project results in its own biometrics evaluation laboratory, which offers security evaluation services for biometric technologies to industry.
Idemia (Morpho) is already using the project results to extend an existing FBI certification scheme to new technology (contactless fingerprint acquisition). This involves new 3D targets which considers an acquisition volume rather than an acquisition surface. The methodology has been accepted by the scheme owner and may be integrated in a revision of ISO/IEC 19794-4: Fingerprint Image. Idemia is planning to exploit HECTOS case study results in biometrics Presentation Attack Detection certification, which is an important subject as PAD is a key feature to demonstrate to allow mass-market deployment of biometric products. Results from Idemia work on the Biometric Access for Critical Infrastructure case study are currently being used in the standardisation process at CEN WG18 on Biometrics. Recommendations and requirements will be taken into account for next update of Access Control products line
Fraunhofer ICT will exploit its work on explosives & weapons detection through further internal and external dissemination of the results. It will continue the discussion with manufacturers and suppliers of detection devices and with the Federal police & BKA on implementation of the HECTOS framework and template in the explosives detection community and about plans how to introduce changes. Discussions will be held with ECAC and end users on how to bring certification into the non-aviation applications. Experience gained during test method development and testing will improve the ICT test and evaluation services offered to customers.
DIN is a national standards body and does not participate in product certification schemes. However it will continue to support the exploitation of the results of HECTOS through standardisation and will use the expertise gained on physical security products and security product certification in its ongoing standardisation work.
The University of Warwick (UW) has published the results of its work on the project as publicly available deliverables, for use by the ethics research community and by policy makers working in the security area. Warwick will use the knowledge and expertise of physical security products and certification gained during the project in its future ethics research activities.
Further research
Further European research and actions would be beneficial in several areas to extend and support the development and exploitation of the HECTOS work. These include:
• Development of measurement and product standards for product categories where few or no European/International standards exist – including CB detection, Explosives & Weapons detection outside of aviation security, video analytics products for Video Surveillance Systems, fences and gates.
• Harmonisation of European Standards (although not necessarily leading to ‘Harmonised EN Standards’ driven by European Directives) in mature product areas such as locks and for high security products where national ‘top up’ standards exist.
• Development of guidance material on how to write measurement standards (test methods) for security products, focussing especially on the need to ensure consistency in realistic and adversarial testing.
The contact details for the HECTOS partners are:
Totalförsvarets Forskningsinstitut, FOI (SE)
Website: www.foi.se
Contact: Anders Elfving
anders.elfving@foi.se
Nederlandse Organisatie voor Toegepast Natuurwetenschappelijk Onderzoek, TNO (NL)
Website: www.tno.nl
Contact: Martijn Koolloos
martijn.koolloos@tno.nl
Fraunhofer Gesellschaft (DE)
Website: https://www.fraunhofer.de
Contact: Frank Schnürer
frank.schnuerer@ict.fraunhofer.de
Iconal Technologies (UK)
Website: http://www.iconal.com/
Contact: Mike Kemp
mike.kemp@iconal.com
Idemia (FR)
Website: www.idemia.com
Contact: Pierre Gacon
pierre.gacon@idemia.com
University of Warwick (UK)
Website: https://warwick.ac.uk/
Contact: Katerina Hadjimatheou
K.Hadjimatheou@warwick.ac.uk
National Physical Laboratories, NPL (UK)
Website: http://www.npl.co.uk/
Contact: Tony Mansfield
tony.mansfield@npl.co.uk
Deutsches Institut für Normung, DIN (DE)
Website: www.din.de
Contact: Christopher Liedtke
Christopher.Liedtke@din.de