Objetivo
HERMES concentrates on how to extract descriptions of human behaviour from videos in a circumscribable discourse domain, such as:
(i) pedestrians crossing inner-city roads and pedestrians approaching or waiting at stops of busses or trams, and
(ii) humans in indoor worlds like an airport hall, or a train station.
These discourse domains allow exploring a coherent evaluation of human movements and facial expressions across a wide variation of scale. This general approach lends itself to various cognitive surveillance scenarios at varying degrees of resolution: from wide-field-of-view multiple-agent scenes, through to more specific inferences of emotional state that could be elicited from high-resolution imagery of faces. The true challenge will consist in the development of a system facility, which starts with basic knowledge about pedestrian behaviour in the chosen discourse domain, but could cluster evaluation results into semantically meaningful subsets of behaviours.
The envisaged system will comprise an internal logic-based representation, which enables it to comment each individual subset, giving natural language explanations of why the system has created the subset in question. Inference and causal reasoning will be based on finding sequences of agent states that are consistent with the observation data and the agent models. Moreover, the sensing process will be tightly integrated with the reasoning process as part of a perception-action cycle: we aim to consider how cooperating pan-tilt-zoom sensors can enhance the process of cognition via controlled responses to uncertain or ambiguous interpretations.
The system will be exposed to video recordings from different parts of Europe in order to prevent over-adaptation to local habits and, in addition, to learn systematically occurring differences between pedestrian habits in different countries. The system's explanatory and arguing capabilities are expected to ease an assessment of its strengths and weaknesses.
Ámbito científico
Convocatoria de propuestas
Data not availableRégimen de financiación
STREP - Specific Targeted Research ProjectCoordinador
08193 BELLATERRA (CERDANYOLA DEL VALLES)
España