Community Research and Development Information Service - CORDIS

Periodic Report Summary 1 - IRIS (IRIS: Towards Natural Interaction and Communication)

The overall goal of IRIS is to contribute to the existence of a Human-Computer Interaction platform accessible and adapted for all users, in indoor scenarios, particularly for people with speech impairments and elderly. This purpose is supported on two complementary research objectives:
- The first objective, Advances in Human-Computer interaction, is concerned with the definition of the initial settings and the definition of the technical platform. The definition of the initial settings encompasses the target users and the contexts of use. It addresses studying and defining user profiles, based on the target user specifications (e.g., disabilities, pathologies), and context scenarios. The definition of the technical platform focuses on novel multimodal interaction paradigms. The concept of multimodal interaction, within the IRIS project, covers a set of input modalities (speech, silent speech, gestures, and gaze), a set of output modalities (animated characters, pictograms and personalized synthetic voices), as well as their combinations that assure a flexible and enjoyable interaction experience”
- The second objective, Moving to real world scenarios, relies on the proposed technologies to advance into real world scenarios. To support this stage of the project, one of the most important aspects concerns the assessment of how the proposed systems and technologies are serving the overall goal. By employing the technical infrastructure achieved within the context of the first objective, IRIS project partners establish a framework for multimodal usability evaluation. The concept of usability evaluation within IRIS is based on a set of psychophysical measures, such as Electromyography (EMG) and eye tracking, among other sensing techniques. The usability evaluation is a continuous process, grounding the entire project in order to guide the development and to assess user performance using the proposed systems in virtual and real-world situations.

Overall, IRIS has been advancing very well and the milestone objectives for the first two-years have been successfully accomplished. In the first 24 months (M1-M24), users’ profiles and case scenarios have been defined. Based on the platform workflow establishment, the interaction modalities have been specified and their integration to the platform have been initiated. The partners have contributed to the development of the modalities, in particular animated characters and pictograms, as well as the fusion of various modalities, such as speech and gaze. Contributions to the state-of-the-art have also been made in other domains, such as authentication based on face and voice biometrics, sign language and children speech recognition for European Portuguese (EP). The Automatics Speech Recognition (ASR) component of the platform has been localized to Turkish for user interaction in native language. The acoustic models for EP were adapted to recognize elderly speech. Also for EP, a personalized elderly synthetic voice is already available to be integrated in the platform as an output modality. Corresponding results have been presented in top scientific meetings, journals and workshops in the respective fields of research, in a total of 62 research papers and 8 academic theses. For a detailed list of publications, please go to .

Both the kickoff meeting and the Mid-Term project review took place at Microsoft in M1 and M22 respectively, in the project coordinator facilities in Lisbon, Portugal. Consortium meetings took place in M09, M15 and M22 in Zaragoza, Spain (hosted by UNIZAR), in Aveiro, Portugal (hosted by UAVR) and in Lisbon, Portugal (hosted by MSFT), respectively. Regular meetings for discussing technical items and planning have been executed with WP leaders and fellows.

During the last 24 months 16 training activities were performed, taking great advantage of the 34 months of secondments and #17,75 months of recruitments already performed. In particular, during July-August 2014 (M07-M08) and 2015 (M19-M20) experts from all the consortium came together not only to plan and design project execution but also to exchange knowledge on multiple disciplines with people from both industry and academia. In this context, several seminars/workshops organized to general public took place, such as the Seminars in GPU Programming and Optimization and Audio Analysis for Environment Awareness and Assisting Hearing Disabilities, the Seminar on Adapting Communications Tools for People with Special Needs, or the Summer Workshop Microsoft on Applied Computing (two editions were already performed). Regular visits from students to Microsoft, University of Zaragoza and Middle East Technical University premises occurred in order to show R&D activities and studies, which included demonstrations of the prototypes developed in IRIS. Real-time Markerless Animation Demos are another example of an activity that spread the progress of the project to the general public. Last, for a detail list of activities, please visit and

A full time Project Manager was hired in M5 with the main goal of ensuring that all planned objectives are successfully met during the lifetime of the project. The project management has included coordination and supervision of all work packages and secondments, financial administration, the organization of the project sync up meetings, research meetings, project reporting, and the coordination of the dissemination by drawing plans of publications. IRIS project has been presented in different social media in newspapers and via the webpage, which had received just for the first six-months of existence, an average of 11,746 views (with 1,302 returning visits).

Reported by



Life Sciences
Follow us on: RSS Facebook Twitter YouTube Managed by the EU Publications Office Top