Periodic Reporting for period 3 - iMARS (image Manipulation Attack Resolving Solutions)
Período documentado: 2023-05-01 hasta 2024-11-30
The main purpose of iMARS is to fight against identity document fraud.
Fraudulent identity documents are used for a number of illegal activities. For instance, a wanted criminal may try to enter from or escape to a foreign country using a fraudulent ID document. Therefore, fraud on identity documents is a major problem for society as it facilitates transborder crime.
Some fraudulent identity documents are genuine documents, regularly issued by government entities, that carry false information. This is possible because the breeder documents and enrolment data on which they rely are false or manipulated. In particular, the face image on an identity document can be manipulated in such a way that the image on the identity document resembles two different persons. When two face images representing two different persons are merged to form a single image that resembles both persons, the combined face image is called a morphed image.
iMARS considers three use cases in the life cycle of identity documents: document issuing, document control at the border and forensic investigation.
For all three use-cases, iMARS focuses on two sets of technical tools useful in the fight against fraudulent documents:
• A set of tools to detect image manipulation, especially morphing. They are called manipulation attack detection (MAD) tools.
• A set of tools to detect fraudulent identity documents: they are called document validation and fraud detection (DVFD) solutions.
iMARS also considers a number of critical enablers for the uptake of its technological tools and further research in the same field:
• Legal, ethical and societal acceptance studies in relation with the iMARS project as well as its tools and technologies
• Improvement of the enrolment phase in ID document issuing
• Skill assessment and training of the key personnel involved in the use-cases of iMARS
• Development of a testing database for MAD tools that will be maintained 20 years after iMARS
• Development of standards in the field of face image related to quality, manipulation attack detection, presentation attack detection, and to the metrics to measure them
Fraudulent identity documents are used for a number of illegal activities. For instance, a wanted criminal may try to enter from or escape to a foreign country using a fraudulent ID document. Therefore, fraud on identity documents is a major problem for society as it facilitates transborder crime.
Some fraudulent identity documents are genuine documents, regularly issued by government entities, that carry false information. This is possible because the breeder documents and enrolment data on which they rely are false or manipulated. In particular, the face image on an identity document can be manipulated in such a way that the image on the identity document resembles two different persons. When two face images representing two different persons are merged to form a single image that resembles both persons, the combined face image is called a morphed image.
iMARS considers three use cases in the life cycle of identity documents: document issuing, document control at the border and forensic investigation.
For all three use-cases, iMARS focuses on two sets of technical tools useful in the fight against fraudulent documents:
• A set of tools to detect image manipulation, especially morphing. They are called manipulation attack detection (MAD) tools.
• A set of tools to detect fraudulent identity documents: they are called document validation and fraud detection (DVFD) solutions.
iMARS also considers a number of critical enablers for the uptake of its technological tools and further research in the same field:
• Legal, ethical and societal acceptance studies in relation with the iMARS project as well as its tools and technologies
• Improvement of the enrolment phase in ID document issuing
• Skill assessment and training of the key personnel involved in the use-cases of iMARS
• Development of a testing database for MAD tools that will be maintained 20 years after iMARS
• Development of standards in the field of face image related to quality, manipulation attack detection, presentation attack detection, and to the metrics to measure them
In period 1 (September 2020 to December 2021), the focus was on preparing the ground for the research and the uptake of the technology that the project develops.
This encompassed:
• Defining the internal management, security and communication framework.
• Use case analysis and requirement definition.
• Designing the iMARS testing framework: data, methodology and plans.
• Sharing with the technical teams of the first analysis of legal requirements.
• Ethical monitoring and delivery of a first ethical analysis.
• Establishing the Advisory Board.
In Period 1, the research activity started in the following domains:
• Secure enrolment processes
• Vulnerabilities to morphing of face recognition systems and human observers
• Document validation and fraud detection (DVFD)
• Manipulation attack detection when only the potentially manipulated face image is available (S-MAD) and when the face image is presented by a person who is supposedly represented on the face image (D-MAD).
• Standardisation related to biometric sample quality and biometric presentation attack detection (ISO/IEC 29794-1, ISO/IEC 29794-5 and ISO/IEC 30107-3)
In period 2 (January 2022 – April 2023), research activity reached its full speed.
Achievements within this period included:
• Completion of the studies on vulnerability started in period 1.
• New presentation attack detection methods and morphing methods.
• New tools to make passports more robust to attacks.
• Near completion of new mobile tools to check passports and travellers’ identity at the border.
• Methods to detect morphing based on the effect of morphing on face images.
• New S-MAD and D-MAD algorithms.
• Extension of morphing and its detection to other modalities than face images (iris, fingerprints and 3D faces).
• Face quality assessment methods and their standardisation.
• Societal acceptability of iMARS technologies.
Communication, dissemination and exploitation activities started during P2. Most notably, a workshop was organised in March 2023 to share the results obtained to that date.
In Period 3 (May 2023 - November 2024), the technical research started during P2 was completed to meet KPIs. The technical results were tested thanks to the dedicated MAD testing platform built during iMARS, which will be publicly available after the project. Additional research in the field of face image quality was carried out.
Research on legal topics was completed taking into account the new EU Artificial Intelligence (AI) Act that affects iMARS technologies, which are mostly based on algorithms using AI. A second ethical report was delivered, which touched on the operational usage of iMARS technologies.
An important aspect of P3 was dissemination: a second stakeholder workshop was organised in March 2024; a conference in Washington DC defined a common approach to morphing and other biometric attacks between Europe and the US; and the iMARS final conference was held in November 2024. Overall, iMARS partners took part in more than 150 dissemination events (workshops, tradeshows, conferences) and published more than 60 scientific articles. The exploitation roadmap was defined and included results that are immediately exploitable such as training and secure enrolment processes.
This encompassed:
• Defining the internal management, security and communication framework.
• Use case analysis and requirement definition.
• Designing the iMARS testing framework: data, methodology and plans.
• Sharing with the technical teams of the first analysis of legal requirements.
• Ethical monitoring and delivery of a first ethical analysis.
• Establishing the Advisory Board.
In Period 1, the research activity started in the following domains:
• Secure enrolment processes
• Vulnerabilities to morphing of face recognition systems and human observers
• Document validation and fraud detection (DVFD)
• Manipulation attack detection when only the potentially manipulated face image is available (S-MAD) and when the face image is presented by a person who is supposedly represented on the face image (D-MAD).
• Standardisation related to biometric sample quality and biometric presentation attack detection (ISO/IEC 29794-1, ISO/IEC 29794-5 and ISO/IEC 30107-3)
In period 2 (January 2022 – April 2023), research activity reached its full speed.
Achievements within this period included:
• Completion of the studies on vulnerability started in period 1.
• New presentation attack detection methods and morphing methods.
• New tools to make passports more robust to attacks.
• Near completion of new mobile tools to check passports and travellers’ identity at the border.
• Methods to detect morphing based on the effect of morphing on face images.
• New S-MAD and D-MAD algorithms.
• Extension of morphing and its detection to other modalities than face images (iris, fingerprints and 3D faces).
• Face quality assessment methods and their standardisation.
• Societal acceptability of iMARS technologies.
Communication, dissemination and exploitation activities started during P2. Most notably, a workshop was organised in March 2023 to share the results obtained to that date.
In Period 3 (May 2023 - November 2024), the technical research started during P2 was completed to meet KPIs. The technical results were tested thanks to the dedicated MAD testing platform built during iMARS, which will be publicly available after the project. Additional research in the field of face image quality was carried out.
Research on legal topics was completed taking into account the new EU Artificial Intelligence (AI) Act that affects iMARS technologies, which are mostly based on algorithms using AI. A second ethical report was delivered, which touched on the operational usage of iMARS technologies.
An important aspect of P3 was dissemination: a second stakeholder workshop was organised in March 2024; a conference in Washington DC defined a common approach to morphing and other biometric attacks between Europe and the US; and the iMARS final conference was held in November 2024. Overall, iMARS partners took part in more than 150 dissemination events (workshops, tradeshows, conferences) and published more than 60 scientific articles. The exploitation roadmap was defined and included results that are immediately exploitable such as training and secure enrolment processes.
iMARS has achieved significant impact in the fight against identity document fraud through:
• An increased awareness of the community (including academics, industrials and practitioners) of the morphing threat, its risks, and how to address it.
• Significant progress in the field of morphing attack detection (MAD): at the beginning of iMARS, MAD technology was in its infancy, now it is close to be deployable and robust to a large variety of morphing techniques and algorithms.
• Better knowledge about the ability of people working with ID documents to detect morphing attacks.
• A very large variety of published MAD algorithms, classified by type, usable as a basis for further research.
• A legal, ethical and societal framework for the deployment of iMARS technologies.
• An open European testing platform to assess MAD algorithms accuracy.
• An e-learning platform to detect morphing attacks “visually” that can be accessed by governmental agencies worldwide.
• A description of secure enrolment practices.
• New mobile tools to detect ID document fraud which are robust to more fraud attempts.
• New ways to make passport more robust against forgery and counterfeiting.
• New standards in the field of morphing and face image quality.
• Improved methods to detect presentation attacks and adversarial attacks.
The e-learning platform; legal, ethical and societal framework; and secure enrolment processes are ready for exploitation. iMARS MAD tools are sufficiently robust enough for an operational attempt in Europe, which would allow measuring the extent to which criminals are currently using morphing in fraudulent documents.
• An increased awareness of the community (including academics, industrials and practitioners) of the morphing threat, its risks, and how to address it.
• Significant progress in the field of morphing attack detection (MAD): at the beginning of iMARS, MAD technology was in its infancy, now it is close to be deployable and robust to a large variety of morphing techniques and algorithms.
• Better knowledge about the ability of people working with ID documents to detect morphing attacks.
• A very large variety of published MAD algorithms, classified by type, usable as a basis for further research.
• A legal, ethical and societal framework for the deployment of iMARS technologies.
• An open European testing platform to assess MAD algorithms accuracy.
• An e-learning platform to detect morphing attacks “visually” that can be accessed by governmental agencies worldwide.
• A description of secure enrolment practices.
• New mobile tools to detect ID document fraud which are robust to more fraud attempts.
• New ways to make passport more robust against forgery and counterfeiting.
• New standards in the field of morphing and face image quality.
• Improved methods to detect presentation attacks and adversarial attacks.
The e-learning platform; legal, ethical and societal framework; and secure enrolment processes are ready for exploitation. iMARS MAD tools are sufficiently robust enough for an operational attempt in Europe, which would allow measuring the extent to which criminals are currently using morphing in fraudulent documents.