Periodic Reporting for period 2 - IDENTITY (Computer Vision Enabled Multimedia Forensics and People Identification)
Reporting period: 2018-01-01 to 2019-12-31
This project aims at continuing supporting the development of the forensic biometric community. In the past five years, several improvements in the biometric community have resulted in more robust technologies for face, fingerprint, iris and gait recognition, which can now be applied to practical scenarios.
This project aims at consolidating the integration of multimedia forensics into forensic science. Multimedia forensics is concerned with the development of scientific methods to extract, analyse and categorise digital evidence derived from multimedia sources.. Since the enabling technologies in multimedia forensics are similar to those used for identification and verification purposes in biometric forensics, the integration of these areas is seamless. The proposed project will support the integration of knowledge and expertise from the forensics, biometrics and multimedia forensics communities.
Multimedia forensic and biometric techniques are useful in combating crime. For example multimedia forensic techniques can be used for source device identification, source device verification common source inference, content authentication and source-oriented image clustering. On the other hand, biometric techniques for identifying and recognising people find their use in for example, access control border control. From the scientific point of view, both sets of techniques are enabled by computer vision, image processing and machine learning. Therefore, experience gained from one domain is transferrable to the other and can facilitate cross-fertilisation of ideas.
The main objective of IDENTITY is to enhance international and European collaborations in research, entrepreneurial development and innovation within the area of multimedia and biometric forensics. To this end, IDENTITY has the following specific objectives:
1. To promote knowledge transfer among research institutions and companies about methodologies for identification within a forensic context. Two main lines of identification are considered, imaging device identification for multimedia forensics, and people identification, for biometric forensics.
2. To enhance research programs by incorporating experience of private companies and police investigators from real identification scenarios and forensic cases.
3. To disseminate knowledge and technologies internationally to ensure a wide impact and a continuing fostering of the multimedia forensics and biometric forensics communities.
Since the start of the project, in the multimedia forensics area, the consortium has developed a number of new methods for attenuating various sources of distortions to sensor pattern noise (a form of device fingerprint). This allows the quality of the device fingerprint to be enhanced to aide forensic investigations. There is a new method for reducing the dimensionality of the sensor pattern noise (i.e. a more compact representation of the sensor pattern noise) so that forensic investigation can be performed in a more efficient manner. The consortium developed a blind image clustering method using sensor pattern noise, which is able to classify images into group so that each group contains images taken with the same camera. The significance of this clustering method is that the method allows investigators to establish the relationship of the images acquired at different times at different locations, hence enabling the investigators to narrow down the investigation.
In the biometrics domain, the main contribution has been the use of deep learning in people re-identification, which can be used for video surveillance. The method is capable of learning the feature adaptively without the user providing specific features to guide the analysis and re-identification. Iris recognition is another sub-area the consortium has made significant improvement in terms of accuracy and efficiency (i.e. low computational complexity).
One patent application on Video Anomaly Detection and Action Recognition was filed to protect innovative intellectual property and to pave the way for further commercial exploitation. Research outcomes were also adopted by the law enforcement sector and banking industry. For example, a researcher acted as an expert witness to help Guildford Crown Court to secure the conviction of a voyeur using the device fingerprinting technology developed at the University of Warwick through this project. The technology was later incorporated into an EU bank’s authentication process. One Warwick researcher’s face image pre-processing skills developed through this project were also used by the Metropolitan Police Services (Scotland Yard) in the UK to help improve the performance of their existing face recognition algorithms.
Many PhD students and early career researchers have improved their employability through the international and inter-sectoral secondments, training and networking activities, and are now at many international institutions.
The consortium has delivered profound and tangible socio-economic impact with the adoptions of the research outcomes by the law enforcement agencies in their fight against crime and by the commercial sector in their information security processes.