Final Report Summary - BEAT (Biometrics Evaluation and Testing)
Executive Summary:
Identity management using Biometrics is a reality because of the e-passport (Biometric passport). Similar biometric technology has also become more prevalent on personal computers with more biometric-enabled functions and applications to recognize nomadic users through biometrics emerged as mobile devices are equipped with more sensors. Unfortunately the reliability of these biometric technologies is not always known and therefore cant be guaranteed. In particular the three criteria of (1) the performance of the underlying biometric system, (2) the robustness to vulnerabilities such as direct (spoofing) or indirect attacks, and (3) the strength of privacy preservation techniques, are often unknown or impossible to compare to competitors.
The lack of standard operational evaluations is the reason that we cannot measure the reliability of these biometric technologies. Some initiatives exist in Europe, the United States of America, and Asia. However, these initiatives are: isolated (focusing only on one or two biometric modalities), disorganized (teams from the same institution can work on different biometrics without talking to each other), or limited in time (very few are organizing ongoing evaluations). This leads to discontinuous and non-integrated efforts which have a limited life span. Thus there is a need for establishing a framework to evaluate, in a systematic way, the performance of biometric technologies using several metrics and criteria (performance, vulnerability, privacy).
Project Context and Objectives:
The goal of BEAT is to propose a framework of standard operational evaluations for biometric technologies. This was achieved by
- developing an online and open platform to transparently and independently evaluate biometric systems against validated benchmarks,
- designing protocols and tools for vulnerability analysis, and
- developing standardization documents for Common Criteria evaluations.
Additionally, legal aspects were considered to address the issues of both privacy data protection and Intellectual Property and so ensure that the BEAT framework can be used by the research community and companies.
There are three outcomes of this project. The first is that the reliability of biometric systems is measurable and thus should lead to a meaningful increase in performance. The second is that technology transfer from research to companies is much easier as there will be an interoperable framework. Finally, decision-makers and authorities are informed about the progress that is made in biometrics as the results have an impact on standards. Given these outcomes we are convinced that BEAT significantly contributed to the development of a European Identification Certification System.
BEAT has identified 8 concrete objectives in the description of work. We list them below and shortly mention how they were achieved.
1. Evaluation of the performance of biometric systems:
All deliverables from WorkPackage "Evaluation of Biometric Performance" (D3.1 to D3.7) were completed and submitted. In addition some of the biometric algorithms, metrics and databases are available on the BEAT platform. The reader is referred to the section 2 of the deliverable D7.6 and to the main S&T result 1-2 in the next section.
2. Evaluation of the vulnerabilities of biometric systems:
All deliverables from WorkPackage "Evaluation of Vulnerabilities" (D4.1 to D4.8) were completed and submitted. In addition some of the biometric algorithms, metrics and databases are available on the BEAT platform. The reader is referred to the section 2 of the deliverable D7.6 and to the main S&T result 2 in the next section.
3. Evaluation of privacy preservation mechanisms:
All deliverables from WorkPackage "Evaluation of Privacy Preservation" (D5.1 to D5.5) were completed and submitted. The reader is referred to the main S&T result 2 in the next section.
4. Integration and deployment of the BEAT platform:
All deliverables from WorkPackages "Specification and Design" (D2.1 to D2.6) and \Integration and Deployment" (D7.1 to D7.6) were completed and submitted. The reader is referred to the main S&T result 5 in the next section. The BEAT platform is operational and running ( https://www.beat-eu.org/platform/ ) and open source ( https://gitlab.idiap.ch/groups/beat ).
5. Dissemination on research activities:
Deliverables from WorkPackage "Dissemination and Exploitation" (D8.1 to D8.3) were completed and submitted. BEAT research activities were disseminated to a large audience through the organization and participation to events and the publication of scientific papers in major journals and international conferences.
6. Exploitation activities:
Deliverables from WorkPackage "Dissemination and Exploitation" (D8.4 to D8.6) were completed and submitted. In addition, BEAT has generated 2 patents, 1 license agreement and prepared its legacy as the \Swiss Center for Biometrics Research and Testing" ( http://www.
biometrics-center.ch/testing ). This center will maintain the BEAT platform and host public standardization documents such as the D6.5 \Toward Common Criteria evaluations of biometric systems".
7. Developing standardization documents:
Deliverables from WorkPackage "Standards and Certification" (D6.1 to D6.5) were completed and submitted. The reader is referred to the main S&T result 3 in the next section. We contributed to international standardization activities (SC27 and SC37) by proposing security evaluation standards for biometrics. In January 2016 the "Information technology - Biometric presentation attack detection - Part 1: Framework" standard
ISO/IEC 30107-1:2016 ( http://www.iso.org/iso/catalogue_detail.htm?csnumber=53227 ) was published resulting from many contributions by BEAT partners. In addition, the deliverable D6.5 "Toward Common Criteria evaluations of biometric systems" was completed and will be proposed to Common Criteria for testing biometric products in the future.
8. Legal aspects:
Deliverables from WorkPackage "Legal Aspects of Privacy and Intellectual Property" (D9.1 to D9.4) were completed and submitted. The reader is refered to the main S&T result 4 in the next section.
Project Results:
cf report D1.7 chapter 1.3 Main S&T Results - Foregrounds, attached to this form
Potential Impact:
cf report D1.7 chapter 1.4 Societal Impacts, attached to this form
List of Websites:
https://www.beat-eu.org/
Identity management using Biometrics is a reality because of the e-passport (Biometric passport). Similar biometric technology has also become more prevalent on personal computers with more biometric-enabled functions and applications to recognize nomadic users through biometrics emerged as mobile devices are equipped with more sensors. Unfortunately the reliability of these biometric technologies is not always known and therefore cant be guaranteed. In particular the three criteria of (1) the performance of the underlying biometric system, (2) the robustness to vulnerabilities such as direct (spoofing) or indirect attacks, and (3) the strength of privacy preservation techniques, are often unknown or impossible to compare to competitors.
The lack of standard operational evaluations is the reason that we cannot measure the reliability of these biometric technologies. Some initiatives exist in Europe, the United States of America, and Asia. However, these initiatives are: isolated (focusing only on one or two biometric modalities), disorganized (teams from the same institution can work on different biometrics without talking to each other), or limited in time (very few are organizing ongoing evaluations). This leads to discontinuous and non-integrated efforts which have a limited life span. Thus there is a need for establishing a framework to evaluate, in a systematic way, the performance of biometric technologies using several metrics and criteria (performance, vulnerability, privacy).
Project Context and Objectives:
The goal of BEAT is to propose a framework of standard operational evaluations for biometric technologies. This was achieved by
- developing an online and open platform to transparently and independently evaluate biometric systems against validated benchmarks,
- designing protocols and tools for vulnerability analysis, and
- developing standardization documents for Common Criteria evaluations.
Additionally, legal aspects were considered to address the issues of both privacy data protection and Intellectual Property and so ensure that the BEAT framework can be used by the research community and companies.
There are three outcomes of this project. The first is that the reliability of biometric systems is measurable and thus should lead to a meaningful increase in performance. The second is that technology transfer from research to companies is much easier as there will be an interoperable framework. Finally, decision-makers and authorities are informed about the progress that is made in biometrics as the results have an impact on standards. Given these outcomes we are convinced that BEAT significantly contributed to the development of a European Identification Certification System.
BEAT has identified 8 concrete objectives in the description of work. We list them below and shortly mention how they were achieved.
1. Evaluation of the performance of biometric systems:
All deliverables from WorkPackage "Evaluation of Biometric Performance" (D3.1 to D3.7) were completed and submitted. In addition some of the biometric algorithms, metrics and databases are available on the BEAT platform. The reader is referred to the section 2 of the deliverable D7.6 and to the main S&T result 1-2 in the next section.
2. Evaluation of the vulnerabilities of biometric systems:
All deliverables from WorkPackage "Evaluation of Vulnerabilities" (D4.1 to D4.8) were completed and submitted. In addition some of the biometric algorithms, metrics and databases are available on the BEAT platform. The reader is referred to the section 2 of the deliverable D7.6 and to the main S&T result 2 in the next section.
3. Evaluation of privacy preservation mechanisms:
All deliverables from WorkPackage "Evaluation of Privacy Preservation" (D5.1 to D5.5) were completed and submitted. The reader is referred to the main S&T result 2 in the next section.
4. Integration and deployment of the BEAT platform:
All deliverables from WorkPackages "Specification and Design" (D2.1 to D2.6) and \Integration and Deployment" (D7.1 to D7.6) were completed and submitted. The reader is referred to the main S&T result 5 in the next section. The BEAT platform is operational and running ( https://www.beat-eu.org/platform/ ) and open source ( https://gitlab.idiap.ch/groups/beat ).
5. Dissemination on research activities:
Deliverables from WorkPackage "Dissemination and Exploitation" (D8.1 to D8.3) were completed and submitted. BEAT research activities were disseminated to a large audience through the organization and participation to events and the publication of scientific papers in major journals and international conferences.
6. Exploitation activities:
Deliverables from WorkPackage "Dissemination and Exploitation" (D8.4 to D8.6) were completed and submitted. In addition, BEAT has generated 2 patents, 1 license agreement and prepared its legacy as the \Swiss Center for Biometrics Research and Testing" ( http://www.
biometrics-center.ch/testing ). This center will maintain the BEAT platform and host public standardization documents such as the D6.5 \Toward Common Criteria evaluations of biometric systems".
7. Developing standardization documents:
Deliverables from WorkPackage "Standards and Certification" (D6.1 to D6.5) were completed and submitted. The reader is referred to the main S&T result 3 in the next section. We contributed to international standardization activities (SC27 and SC37) by proposing security evaluation standards for biometrics. In January 2016 the "Information technology - Biometric presentation attack detection - Part 1: Framework" standard
ISO/IEC 30107-1:2016 ( http://www.iso.org/iso/catalogue_detail.htm?csnumber=53227 ) was published resulting from many contributions by BEAT partners. In addition, the deliverable D6.5 "Toward Common Criteria evaluations of biometric systems" was completed and will be proposed to Common Criteria for testing biometric products in the future.
8. Legal aspects:
Deliverables from WorkPackage "Legal Aspects of Privacy and Intellectual Property" (D9.1 to D9.4) were completed and submitted. The reader is refered to the main S&T result 4 in the next section.
Project Results:
cf report D1.7 chapter 1.3 Main S&T Results - Foregrounds, attached to this form
Potential Impact:
cf report D1.7 chapter 1.4 Societal Impacts, attached to this form
List of Websites:
https://www.beat-eu.org/