Computer Vision Enabled Multimedia Forensics and People Identification

Project Information

IDENTITY

Grant agreement ID: 690907

DOI

10.3030/690907

Project closed

EC signature date 20 November 2015

Start date 1 January 2016

End date 31 December 2019

Funded under

• EXCELLENT SCIENCE - Marie Skłodowska-Curie Actions

Total cost

€ 2 025 000,00

EU contribution

€ 2 025 000,00

Coordinated by

UNIVERSITY OF WARWICK
United Kingdom

Periodic Reporting for period 2 - IDENTITY (Computer Vision Enabled Multimedia Forensics and People Identification)

Reporting period: 2018-01-01 to 2019-12-31

Summary of the context and overall objectives of the project

Forensics is mainly concerned with proving and investigating infringements, identifying perpetrators and describing modus operandi. Biometrics, on the other hand, is a relatively new science that aims at measuring and analysing a person's unique characteristics, both behavioural and physical. The potential of applying biometrics to forensics comes natural as several forensic questions rely on identifying, or verifying the identity, of people allegedly involved in crimes. Although these two scientific communities have operated in relative isolation over the past couple of decades, forensic biometrics have been successfully applied through the development automatic fingerprint identification systems (AFIS), and most recently, through the development of face recognition systems. The potential of forensics biometrics, however, can be extended to other biometric traits, such as iris and gait analysis.
This project aims at continuing supporting the development of the forensic biometric community. In the past five years, several improvements in the biometric community have resulted in more robust technologies for face, fingerprint, iris and gait recognition, which can now be applied to practical scenarios.

This project aims at consolidating the integration of multimedia forensics into forensic science. Multimedia forensics is concerned with the development of scientific methods to extract, analyse and categorise digital evidence derived from multimedia sources.. Since the enabling technologies in multimedia forensics are similar to those used for identification and verification purposes in biometric forensics, the integration of these areas is seamless. The proposed project will support the integration of knowledge and expertise from the forensics, biometrics and multimedia forensics communities.

Multimedia forensic and biometric techniques are useful in combating crime. For example multimedia forensic techniques can be used for source device identification, source device verification common source inference, content authentication and source-oriented image clustering. On the other hand, biometric techniques for identifying and recognising people find their use in for example, access control border control. From the scientific point of view, both sets of techniques are enabled by computer vision, image processing and machine learning. Therefore, experience gained from one domain is transferrable to the other and can facilitate cross-fertilisation of ideas.


The main objective of IDENTITY is to enhance international and European collaborations in research, entrepreneurial development and innovation within the area of multimedia and biometric forensics. To this end, IDENTITY has the following specific objectives:
1. To promote knowledge transfer among research institutions and companies about methodologies for identification within a forensic context. Two main lines of identification are considered, imaging device identification for multimedia forensics, and people identification, for biometric forensics.
2. To enhance research programs by incorporating experience of private companies and police investigators from real identification scenarios and forensic cases.
3. To disseminate knowledge and technologies internationally to ensure a wide impact and a continuing fostering of the multimedia forensics and biometric forensics communities.

Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far

Through 352 secondment months, the consortium exchanged 74 researchers among 17 participating organisation. More than 100 articles were published in prestigious international journals, conference proceedings, and book chapters. Two edited books were also published in collaboration with researchers from outside the consortium. The researchers of the consortium delivered more than 30 keynote speeches, tutorials, and invited talks to disseminate the outcomes and help advance the field of the their research. The consortium also organised four international conferences and four summer schools targeting the themes of the project, namely multimedia forensics and people identification. These two series of events (open to researchers outside the consortium) provided stimulating forums to facilitate cross-fertilisation of ideas, training of early career researchers, and dissemination of research outcomes.

Since the start of the project, in the multimedia forensics area, the consortium has developed a number of new methods for attenuating various sources of distortions to sensor pattern noise (a form of device fingerprint). This allows the quality of the device fingerprint to be enhanced to aide forensic investigations. There is a new method for reducing the dimensionality of the sensor pattern noise (i.e. a more compact representation of the sensor pattern noise) so that forensic investigation can be performed in a more efficient manner. The consortium developed a blind image clustering method using sensor pattern noise, which is able to classify images into group so that each group contains images taken with the same camera. The significance of this clustering method is that the method allows investigators to establish the relationship of the images acquired at different times at different locations, hence enabling the investigators to narrow down the investigation.

In the biometrics domain, the main contribution has been the use of deep learning in people re-identification, which can be used for video surveillance. The method is capable of learning the feature adaptively without the user providing specific features to guide the analysis and re-identification. Iris recognition is another sub-area the consortium has made significant improvement in terms of accuracy and efficiency (i.e. low computational complexity).

Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far)

In addition to the academic outputs and impact, the consortium engaged the general public by contributing to university open days (talking to the prospective undergraduate students and their parents/family members), disseminating research outcomes at national events and in several non-technical magazines and media interviews.

One patent application on Video Anomaly Detection and Action Recognition was filed to protect innovative intellectual property and to pave the way for further commercial exploitation. Research outcomes were also adopted by the law enforcement sector and banking industry. For example, a researcher acted as an expert witness to help Guildford Crown Court to secure the conviction of a voyeur using the device fingerprinting technology developed at the University of Warwick through this project. The technology was later incorporated into an EU bank’s authentication process. One Warwick researcher’s face image pre-processing skills developed through this project were also used by the Metropolitan Police Services (Scotland Yard) in the UK to help improve the performance of their existing face recognition algorithms.

Many PhD students and early career researchers have improved their employability through the international and inter-sectoral secondments, training and networking activities, and are now at many international institutions.
The consortium has delivered profound and tangible socio-economic impact with the adoptions of the research outcomes by the law enforcement agencies in their fight against crime and by the commercial sector in their information security processes.

International Biometrics Summer School, Sardinia, 2017