Periodic Reporting for period 2 - TENSOR (Reliable biomeTric tEchNologies to asSist Police authorities in cOmbating terrorism and oRganized crime)
Periodo di rendicontazione: 2024-01-01 al 2024-12-31
Biometrics address a longstanding concern to prove one's identity, irrefutably, by using what makes a person unique. Over the past few years, biometrics have become an essential part of daily life, as more and more people leverage biometric authentication to gain access to devices and services. Today, biometric identification is being combined with other advanced technologies such as behavioural detection to serve an array of purposes, including healthcare, law enforcement, and border control.
In the law enforcement domain, the undisputable power of biometrics is being fully harnessed by Police Authorities and forensic investigators who have been relying since the 1980s on the AFIS to tackle illegal activities. However, matching the evidence - often smudged, incomplete, or deposited on top of other markings - with complete prints in AFIS databases is not a simple task. To overcome this, recently forensic investigators have resorted to the combination of multiple biometric modalities such as face and voice biometrics.
TENSOR proposes going beyond the traditional fingerprint-based identification by combining multiple emerging biometric modalities such as face, voice, and gait biometrics, through advanced AI to produce court-proof evidence. More specifically, lawful evidence derived from CCTVs (face, gait, voice), mobile devices (behavioural patterns) and fingerprints, will be fused with more or less distinctive features towards accurate and multimodal identification.
The TENSOR solutions are complemented by the European Biometric Data Space, which will enable seamless sharing of biometric data between security stakeholders, allowing for faster and more accurate identification of suspects.
In the law enforcement domain, the undisputable power of biometrics is being fully harnessed by Police Authorities and forensic investigators who have been relying since the 1980s on the AFIS to tackle illegal activities. However, matching the evidence - often smudged, incomplete, or deposited on top of other markings - with complete prints in AFIS databases is not a simple task. To overcome this, recently forensic investigators have resorted to the combination of multiple biometric modalities such as face and voice biometrics.
TENSOR proposes going beyond the traditional fingerprint-based identification by combining multiple emerging biometric modalities such as face, voice, and gait biometrics, through advanced AI to produce court-proof evidence. More specifically, lawful evidence derived from CCTVs (face, gait, voice), mobile devices (behavioural patterns) and fingerprints, will be fused with more or less distinctive features towards accurate and multimodal identification.
The TENSOR solutions are complemented by the European Biometric Data Space, which will enable seamless sharing of biometric data between security stakeholders, allowing for faster and more accurate identification of suspects.
During RP2, TENSOR achieved the following key results:
1. Multi-Modal Biometric Fusion for Identification
• TENSOR has successfully integrated multiple biometric modalities, including face recognition, voice recognition, gait analysis, and behavioral biometrics from mobile devices, into a unified identification framework.
• Beyond SOTA: Unlike many existing systems that rely on a single biometric trait, TENSOR’s fusion engine enhances accuracy and robustness by dynamically combining complementary biometric evidence using TOPSIS.
2. Extraction of Data of Interest from Mobile Devices
• Developed methodologies for extracting data of interest (e.g. app usage patterns, location traces, media files, text messages, etc.) from seized mobile devices, bypassing the inherent encryption mechanisms.
• Beyond SOTA: This represents an innovative and impactful advancement in digital forensics analysis, enabling LEAs with no technical expertise and specialized equipment to identify the owner of an orphan device found at a crime scene.
3. Explainable AI (XAI) and Human-in-the-Loop Integration
• Embedded explainability mechanisms and expert-in-the-loop workflows, enhancing the system’s decision-making pipeline.
• Beyond SOTA: TENSOR empowers LEAs not just with accurate suspect recommendations, but with transparent, interpretable reasoning. This fosters trust and ensures compliance with EU ethical and legal standards.
4. Biometrics Dataspace for Cross-Border LEA Collaboration
• Established the first prototype of a sovereign and privacy-preserving Biometrics Dataspace ecosystem, enabling secure and trustworthy cross-border data exchange between LEAs.
• Beyond SOTA: This goes beyond traditional centralized databases by aligning with EU data sovereignty principles and IDSA-based data sharing frameworks, proposing a new paradigm in biometric data exchange.
1. Multi-Modal Biometric Fusion for Identification
• TENSOR has successfully integrated multiple biometric modalities, including face recognition, voice recognition, gait analysis, and behavioral biometrics from mobile devices, into a unified identification framework.
• Beyond SOTA: Unlike many existing systems that rely on a single biometric trait, TENSOR’s fusion engine enhances accuracy and robustness by dynamically combining complementary biometric evidence using TOPSIS.
2. Extraction of Data of Interest from Mobile Devices
• Developed methodologies for extracting data of interest (e.g. app usage patterns, location traces, media files, text messages, etc.) from seized mobile devices, bypassing the inherent encryption mechanisms.
• Beyond SOTA: This represents an innovative and impactful advancement in digital forensics analysis, enabling LEAs with no technical expertise and specialized equipment to identify the owner of an orphan device found at a crime scene.
3. Explainable AI (XAI) and Human-in-the-Loop Integration
• Embedded explainability mechanisms and expert-in-the-loop workflows, enhancing the system’s decision-making pipeline.
• Beyond SOTA: TENSOR empowers LEAs not just with accurate suspect recommendations, but with transparent, interpretable reasoning. This fosters trust and ensures compliance with EU ethical and legal standards.
4. Biometrics Dataspace for Cross-Border LEA Collaboration
• Established the first prototype of a sovereign and privacy-preserving Biometrics Dataspace ecosystem, enabling secure and trustworthy cross-border data exchange between LEAs.
• Beyond SOTA: This goes beyond traditional centralized databases by aligning with EU data sovereignty principles and IDSA-based data sharing frameworks, proposing a new paradigm in biometric data exchange.
TENSOR has delivered a set of results with the potential to significantly enhance law enforcement capabilities in suspect identification and digital investigations, while aligning with EU standards on ethics, privacy, and cross-border cooperation. The indicative impacts per key result are presented below:
1. Improve suspect identification performance, greater resilience to identity spoofing or poor-quality data, and increased trust in biometric outcomes.
2. Provide small and less technically equipped LEAs with a practical and cost-efficient alternative to advanced and costly digital forensics tools, expanding the access to critical investigative capabilities across the EU.
3. Reinforce human oversight, ensure legal admissibility, and promote transparency in biometric-based decision support systems.
4. Pave the way for more secure, GDPR-compliant, and scalable biometric data exchange between LEAs across the EU.
Additionally, to ensure further uptake and success, the following key needs have been identified:
• Additional research is needed to strengthen the reliability of emerging biometric modalities such as gait recognition.
• Expand testing with operational LEAs in real-case environments for the TENSOR results to go beyond TRL7.
• Support from EU innovation programs and public-private partnerships can accelerate the adoption of TENSOR’s solutions. Engagement with industry partners is strongly required to translate research outcomes into practical, market-ready tools.
• Targeted support is needed for exploitation planning. TENSOR is receiving assistance through the Horizon Booster and is exploring collaboration with EACTDA to leverage opportunities under the ISF.
• Significant standardization gaps remain, particularly for emerging biometric technologies like voice recognition.
1. Improve suspect identification performance, greater resilience to identity spoofing or poor-quality data, and increased trust in biometric outcomes.
2. Provide small and less technically equipped LEAs with a practical and cost-efficient alternative to advanced and costly digital forensics tools, expanding the access to critical investigative capabilities across the EU.
3. Reinforce human oversight, ensure legal admissibility, and promote transparency in biometric-based decision support systems.
4. Pave the way for more secure, GDPR-compliant, and scalable biometric data exchange between LEAs across the EU.
Additionally, to ensure further uptake and success, the following key needs have been identified:
• Additional research is needed to strengthen the reliability of emerging biometric modalities such as gait recognition.
• Expand testing with operational LEAs in real-case environments for the TENSOR results to go beyond TRL7.
• Support from EU innovation programs and public-private partnerships can accelerate the adoption of TENSOR’s solutions. Engagement with industry partners is strongly required to translate research outcomes into practical, market-ready tools.
• Targeted support is needed for exploitation planning. TENSOR is receiving assistance through the Horizon Booster and is exploring collaboration with EACTDA to leverage opportunities under the ISF.
• Significant standardization gaps remain, particularly for emerging biometric technologies like voice recognition.