Periodic Reporting for period 1 - TENSOR (Reliable biomeTric tEchNologies to asSist Police authorities in cOmbating terrorism and oRganized crime)
Période du rapport: 2023-01-01 au 2023-12-31
Biometrics address a longstanding concern to prove one's identity, irrefutably, by using what makes a person unique. Over the past few years, biometrics have become an essential part of daily life, as more and more people leverage biometric authentication to gain access to devices and services. Today, biometric identification is being combined with other advanced technologies such as behavioural detection and emotion recognition to serve an array of purposes, including healthcare, law enforcement, and border control.
In the law enforcement domain, the undisputable power of biometrics is being fully harnessed by Police Authorities and forensic investigators who have been relying since the 1980s on the AFIS to tackle illegal activities. However, matching the evidence - often smudged, incomplete, or deposited on top of other markings - with complete prints in AFIS databases is not a simple task. To overcome this, recently forensic investigators have resorted to the combination of multiple biometric modalities such as face and voice biometrics.
TENSOR proposes going beyond the traditional fingerprint-based identification by combining multiple emerging biometric modalities such as face, voice, and gait biometrics, through advanced AI to produce court-proof evidence. More specifically, lawful evidence derived from CCTVs (face, gait, voice), mobile devices (behavioural patterns) and fingerprints, will be fused with more or less distinctive features towards accurate and multimodal identification.
At the heart of the TENSOR project lies the creation of a unique European Biometric Data Space, which will enable seamless sharing of biometric data between security stakeholders, allowing for faster and more accurate identification of suspects.
In the law enforcement domain, the undisputable power of biometrics is being fully harnessed by Police Authorities and forensic investigators who have been relying since the 1980s on the AFIS to tackle illegal activities. However, matching the evidence - often smudged, incomplete, or deposited on top of other markings - with complete prints in AFIS databases is not a simple task. To overcome this, recently forensic investigators have resorted to the combination of multiple biometric modalities such as face and voice biometrics.
TENSOR proposes going beyond the traditional fingerprint-based identification by combining multiple emerging biometric modalities such as face, voice, and gait biometrics, through advanced AI to produce court-proof evidence. More specifically, lawful evidence derived from CCTVs (face, gait, voice), mobile devices (behavioural patterns) and fingerprints, will be fused with more or less distinctive features towards accurate and multimodal identification.
At the heart of the TENSOR project lies the creation of a unique European Biometric Data Space, which will enable seamless sharing of biometric data between security stakeholders, allowing for faster and more accurate identification of suspects.
WP2: A gap analysis (T2.1) was initially conducted to assess and report on the current state-of-the-art of the offered technologies for digital forensics and biometrics and report the desired status that these technologies can be extended within TENSOR. T2.5 provided the TENSOR architecture, which follows a security-and-privacy by design approach and acts as the roadmap for the integration of the individual modules into an interoperable, highly scalable and flexible platform.
WP3: Physiological biometrical components were developed and provided for face, person, voice and fingerprint recognition (T3.1) together with a set of derived information e.g. about the gender, the age range, spoken language etc. In case of behavioral biometrics, WP3 developed components for gait recognition as well as device related usage patterns under T3.2. Selected biometric components were used to analyze their fairness properties as well as to provide explanations of achieved results (explainability) (T3.4). WP3 also developed a first implementation version of the European biometric data space. This implementation is based on the IDSA RAM, which enables trusted data exchange between connected European LEAs. All these components are available in the form of a first implementation version.
WP4: WP4 has developed local data Pods and established API endpoints for diverse actions related to data sharing such as creating new data Pods and users, uploading resources, modifying permissions, and retrieving resources from other Pods. It also explored the integration of Ethereum blockchain into the SOLID platform, focusing on linking user IDs between smart contract users and SOLID Pods' owners to enhance decentralized and distributed data sharing recording and monitoring.
On the aspect of data encryption and privacy preservation technologies, TENSOR has adopted lightweight homomorphic encryption. Technical accomplishments encompass the successful implementation of lightweight HE through the SEAL library. Fuzzy extractor was explored to examine the initial comparison of biometric data. We also introduced the "Investigation Recommendation & Enhancement" (IRE) component which aims at fusing biometric matching scores and providing recommendations for potential suspects, as well as assisting the investigation with incident narration generation. WP4 also attempts to improve the rapid and efficient retrieval of large-scale biometric data, specifically facial images, by advanced indexing techniques and Deep Learning based search methods. WP4 also developed the initial version of the toolkit that was designed to exploit common vulnerabilities in authentication mechanisms, providing the capability for even non-expert LEAs to unlock a suspect's mobile device.
WP3: Physiological biometrical components were developed and provided for face, person, voice and fingerprint recognition (T3.1) together with a set of derived information e.g. about the gender, the age range, spoken language etc. In case of behavioral biometrics, WP3 developed components for gait recognition as well as device related usage patterns under T3.2. Selected biometric components were used to analyze their fairness properties as well as to provide explanations of achieved results (explainability) (T3.4). WP3 also developed a first implementation version of the European biometric data space. This implementation is based on the IDSA RAM, which enables trusted data exchange between connected European LEAs. All these components are available in the form of a first implementation version.
WP4: WP4 has developed local data Pods and established API endpoints for diverse actions related to data sharing such as creating new data Pods and users, uploading resources, modifying permissions, and retrieving resources from other Pods. It also explored the integration of Ethereum blockchain into the SOLID platform, focusing on linking user IDs between smart contract users and SOLID Pods' owners to enhance decentralized and distributed data sharing recording and monitoring.
On the aspect of data encryption and privacy preservation technologies, TENSOR has adopted lightweight homomorphic encryption. Technical accomplishments encompass the successful implementation of lightweight HE through the SEAL library. Fuzzy extractor was explored to examine the initial comparison of biometric data. We also introduced the "Investigation Recommendation & Enhancement" (IRE) component which aims at fusing biometric matching scores and providing recommendations for potential suspects, as well as assisting the investigation with incident narration generation. WP4 also attempts to improve the rapid and efficient retrieval of large-scale biometric data, specifically facial images, by advanced indexing techniques and Deep Learning based search methods. WP4 also developed the initial version of the toolkit that was designed to exploit common vulnerabilities in authentication mechanisms, providing the capability for even non-expert LEAs to unlock a suspect's mobile device.
As this concerns the first year of the project implementation, we have been focusing on the research aspects and we have identified that indeed further research is required in multimodal biometric identification systems, as well as for searchable encryption techniques applied specifically to biometric data. The remaining impacts will be assessed during the following reporting periods.