Community Research and Development Information Service - CORDIS

Periodic Report Summary 1 - MMFP (Multimodal Face Processing)

Facial image processing and analysis is the task of automatically analyzing face images in order to acquire information about the depicted persons. This includes, for example, a person’s identity, emotional state, facial gestures, age, and gender. There are two main objectives of this project. The first one is building a common framework to derive information from face images and the second one is joint maximization of information extraction performance. In addition, the project will address the task of benchmarking face processing under ambient conditions.

Towards these goals, significant progress has been achieved. In terms of common framework, a fast and practical, discrete cosine transform based face representation has been opted for. Age estimation, gender, and facial expression classification systems based on this face representation have been developed. In order to test the developed systems and assess influence of facial attributes on each other’s classification performance a benchmark has been established. Moreover, to evaluate face analysis algorithms across age variations and under adverse condition, a dataset from Harry Potter movies has been prepared. For facial expression, a novel hierarchical model based on partial least squares (PLS) has been introduced, which can handle non-linearities in the data by utilizing combinations of locally linear PLS sub-models, each of which is learned at different levels of detail.

As in many areas of computer vision, deep learning based approaches has also become the state-of-the-art in automatic face analysis. A common face representation based on a deep face model would also be possible. During the second period, it will be considered to benefit such an approach, if deep face models become publicly available.

Besides the progress in technical approaches and benchmarking, innovative application ideas that utilize automatic face analysis technologies have also been explored with the focus on facilitating face-to-face communication and interaction. A multiplayer mobile game application, which aims at enabling individuals play paintball or laser tag style games using their smartphones, and a facial expression imitation game has been developed.

The researcher has passed his habilitation exam successfully and received “docent” title in September 2013. Habilitation is a requirement in Turkey to progress in academic career. He is currently an associate professor in the Department of Computer Engineering at Istanbul Technical University. He has established his own research group SiMiT Lab (Smart Interaction, Mobile Intelligence, and Multimedia Technologies Lab in ITU. He has founded a start-up by obtaining seed funding through Innovation and Entrepreneurship award from the Scientific and Technological Research Council of Turkey. The start-up focuses on utilizing computer vision and machine learning algorithms for mobile and social gaming.

Many activities have been carried out to disseminate the results of the proposed research to the larger scientific community, as well as to the general public, including publications in prestigious conferences, public website, demo videos, participation in CeBIT Bilisim Eurasia fair, open lab days, and a magazine article.

Reported by



Life Sciences
Follow us on: RSS Facebook Twitter YouTube Managed by the EU Publications Office Top