Periodic Reporting for period 2 - iMARS (image Manipulation Attack Resolving Solutions) Reporting period: 2022-01-01 to 2023-04-30 Summary of the context and overall objectives of the project iMARS is a European H2020 project, funded under Grant Agreement ID 883356. It started in September 2020 and will continue until August 2024. iMARS purpose is to help with the fight against identity document Fraud. Fraudulent Identity documents are used for illegal activities. For instance, a wanted criminal may try to escape to a foreign country, using a fraudulent ID document. In other words, fraud on Identity documents is a major problem for the society as it makes a number of illegal activities easier, including trans-border major criminal activities. Some fraudulent identity documents are genuine documents, regularly issued by government entities: they simply carry false information. This is possible because the breeder documents and enrolment data on which they rely are false. In particular, the face image of an identity document can be manipulated in such a way that an identity document could look suitable for two different persons. When two face images representing two different persons are merged to form a single image, looking equally like both persons at the origin of the fake face image, the combined face image is called a morphed image.iMARS considers 3 use-cases in the life cycle of identity documents:• Document issuing• Document control at the border• Forensic investigation For all three use-cases, iMARS focuses on two sets of technical tools useful in the fight against fraudulent documents.• A set of tools to detect image manipulation and especially morphing. They are called MAD tools ( Manipulation Attack Detection)• A set of tools to detect fraudulent Identity documents: they are called DVFD solutions (Document Validation and Fraud Detection). iMARS also develops a number of critical enablers for the uptake of its technological tools and further research in the same field: • Legal and societal acceptance studies in relation with iMARS tools, • Ethical studies related to the iMARS project itself and the operational usage of its technologies, • Improvement of the enrolment phase in ID document issuing• Skill assessment and training of the key personnel involved in the use-cases of iMARS, • Development of a testing database for MAD tools that will be maintained 20 years after iMARS, • Standards in the field of face image, related to quality, manipulation attack detection and presentation attack detection, and to the metrics to measure them. Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far During period 1 (Sept 2020, Dec 2021), the work performed in iMARS mostly concentrated on enabling the research, and the uptake of the technology that the project develops. This consisted in:• Definition of management rules of the project. • Put in place the communication infrastructure of iMARS • Use case analysis and requirement definition • iMARS testing methodology and test plans • iMARS test database gathering • Analysis of legal requirements and in depth presentation of them to technical people • Monitor the project from an ethical perspective and deliver a first ethical analysis • Put in place an Advisory board including experts on the fight against identity document fraudIn Period 1, the research activity started in the following domains• Definition of secure enrolment processes• Study the vulnerabilities to morphing of face recognition systems (FRS) and human observers • Document validation and Fraud Detection (DVFD)• Manipulation attack detection (S-MAD when only the potentially manipulated face image is available, and D-MAD when the face image is presented by a person who is supposedly represented on the face image).• Standardization (ISO/IEC 29794-1, ISO/IEC 29794-5 and ISO/IEC 30107-3)This resulted in numerous dissemination activities: • 14 publications• 33 presentations of iMARS in conferences, workshops, etc.In period 2, communication dissemination and standardisation activities continued. Examples:• a workshop mostly dedicated to EU agencies was held • 20 publications and 24 iMARS presentation in events happened,• Face quality workshops were organized with impacts on ISO/IEC 29794-5• a new metric for face recognition vulnerability, invented in iMARS, was proposed for standardisation.iMARS also started to study how the technologies it develops can be further exploited.The societal acceptability of iMARS technologies was analysed using a survey and the result was published.Period 2 was mostly dedicated to research activity. • Completion of the studies on the vulnerability with regards to face morphing of o face recognition systems o Human observers• New presentation attack detection methods • New mobile tool to check passports and travellers identity at the border (almost complete)• New tools to make passports more robust to attacks • New innovative morphing models and how they impact vulnerability of FRS • Study on the effect of morphing on images and how this can help to detect morphing• New S-MAD and D-MAD algorithms • Work on the extension of morphing to other modalities than face images (iris, fingerprints and 3D faces)• Detection of other face manipulation than morphing• Face quality assessment methods and their standardisation Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far) During period 1 and 2, important results were obtained in the fields listed below:• Dedicated study grouping all the legal aspects to be considered for Identity document fraud technologies development and deployment. It was made publicly available • Completion by an independent ethical advisor of a report, further made publicly available• Very large survey on the societal acceptability of iMARS technologies• Document explaining how reliable enrolment can be achieved, and ID document issuing can be safer. From this document, a policy brief was built.• Contribution to standardization: ISO/IEC 29794-1, 29794-5 30107-3 and 20059 (new).• Platform to measure the accuracy of MAD algorithms (Bologna Evaluation Platform). Will be increased with additional data and morphing methods.• Study of the vulnerability of FRS to morphing (evaluation on 5 face recognition algorithms, on a number of morphing methods, including the most recent ones).• Large-scale study about the ability of human observers to detect morphed face images (almost 500 people), showing that S-MAD is more difficult than D-MAD, and that observers learn on D-MAD better.• Design of at least three innovative ways to detect forged and counterfeited Identity documents, suitable for mobile control. • Security measures to increase the robustness of passports to forgery: 2 methods were developed with their associated detection, and their robustness tested.• New PAD method based on one class learning was developed• Numerous new algorithms in the field of S-MAD and D-MAD. They reached good results in NIST FRVT evaluation and Bologna evaluation platform, much better than when iMARS started. o S-MAD accuracy level has still an equal error rate higher than 2%. o D-MAD accuracy level is now very close to operational,• Tool to simulate print-scan effect, because the effect of print-scan is so important on morphed images.• Early work on morphing in other modalities than 2D face: morphing methods and morphing attackts detection.• New ways of detecting morphed images with forensic tools (designed for human observers).• New ways of detecting adversarial attacks on faces. imars-fraud-image.jpg imars-logo-complete-rgb-highres.jpg
