The proposed project adopts an appearance-based approach to visual recognition. The project addresses the scientific question: Can rich local image descriptions from foveal-like image sensors, selected by a hierarchal visual attention process and processed using task, scene, function and object contextual knowledge improve image-based recognition processes? This clearly addresses questions central to the cognitive vision approach.
To investigate this and more specific questions we propose to research methods for
l)foveated feature extraction and grouping,
2) integrating feature, object and top-down priming for spatial and temporal attention,
3) representing and recognising objects, contexts and situations,
4) learning representation models from visual evidence and
5) reactive and top-down control of the recognition process. We will integrate the results in a complete closed-loop object and situation recognition system.
The main objective is to develop the theory of context-aware visual recognition systems. We will implement the theory in a complete closed-loop vision system, and apply it to two applications (city street surveillance and customer behaviour analysis). To achieve these objectives, we will develop new feature grouping, attention and appearance-based recognition processes. This will also require development of new techniques for acquiring, representing and using visual context and situation knowledge.
The main objective is to develop the theory of context-aware visual recognition systems. The key elements of the project are: use of foveal sensing, descriptive features extracted from the foveated image data, selective attention using context priming as well as image feature salience to steer the sensor and recognition using learned appearance models of object, context and situation. We will investigate individual components of this problem and integrate working modules into a complete working object, context and situation recognition system. To evaluate the performance of the approach, we will acquire model s and apply the system to analysis of image sequences of city centre pedestrian and potential consumer behaviour.
Five scientific workpackages will investigate:
1) foveal camera control, foveal image acquisition, foveal image feature extraction and spatial and temporal grouping.
2) system architectures that allow incremental and continual project refinement and alternative process controllers.
3) feature-based, hierarchical attention processes that receive top-down priming about interesting objects, contexts and situations.
4) object, context and situation representations, how to invoke instances of these to explain image data, how to evaluate the similarity of these to the image data and how to recover from incorrect decisions.
5) tools and methods for specifying and automatically learning object, scene and context representations and control strategies. A sixth workpackage on performance characterisation allows continual validation of individual component and full system progress. A seventh workpackage is focussed on dissemination of project results.
The expected project results are the theory behind and the software implementation of: Month 12) a complete but rudimentary working object, situation and context recognition system Month 24) improved system with modules based on foveal features, selective attention and situation models Month 30) addition of grouping, top-down priming and property learning Month 36) addition of automatic context model learning and error recovery
Funding SchemeCSC - Cost-sharing contracts
75794 Paris Cedex 16
78153 Le Chesnay
38031 Grenoble Cedex 1
38400 Saint Martin D'heres