Final Report Summary - MARIE (Multimodal activity recognition for interactive environments) Your eyes are the closest thing you have to a window on your mind. While we cannot easily discern what you are thinking, we can get some idea about what you are looking at. A major contribution of this work has been to extend this idea towards using your eyes as a means of finding out what you are doing: of recognising your activity. We introduced the idea that using eye movements alone we can collect sufficient information to automatically recognise certain activities. We developed a complete recognition methodology: from devising features based on an analysis on the fundamentals of eye movement; selecting those features most useful for each activity; to the classification of movement sequences with varying complexity. We published two complete multi-subject datasets of eye movement for future use by other researchers. This work, in partnership with ETH Zurich, has directly spawned a new EPSRC-funded project which will continue where MARIE left off. With already three major publications in the first two years, another two in progress, and a growing number of early-stage researchers working on the idea (there are now four researchers directly working on the topic), eye-based activity recognition is an important topic with a huge potential for future success.Evaluation is an essential component of any scientific research field. Activity recognition is no different. For many years the field continued with severe drawbacks in the way researchers measured and evaluated their work. Some of these drawbacks were identified in the researcher's earlier PhD work. Through the MARIE project we introduced a complete system of evaluation - results comparison, error scoring, metric calculation - and demonstrated these on widely available activity recognition datasets. We demonstrated the utility of our approach in comparison to the standard methods. This work has recently been accepted for publication in a forthcoming ACM journal, and has already drawn a number of interested users from the activity recognition research community.In a parallel work through collaboration with over 30 of the leading researchers in activity recognition, we drew together a consensus on how activity recognition can best proceed in future. A publication summarising this work is now under way. This has a strong the potential to influence the entire research field in future.
