Objective
The proposed project adopts an appearance-based approach to visual recognition. The project addresses the scientific question: Can rich local image descriptions from foveal-like image sensors, selected by a hierarchal visual attention process and processed using task, scene, function and object contextual knowledge improve image-based recognition processes? This clearly addresses questions central to the cognitive vision approach.
To investigate this and more specific questions we propose to research methods for
l)foveated feature extraction and grouping,
2) integrating feature, object and top-down priming for spatial and temporal attention,
3) representing and recognising objects, contexts and situations,
4) learning representation models from visual evidence and
5) reactive and top-down control of the recognition process. We will integrate the results in a complete closed-loop object and situation recognition system.
Objectives:
The main objective is to develop the theory of context-aware visual recognition systems. We will implement the theory in a complete closed-loop vision system, and apply it to two applications (city street surveillance and customer behaviour analysis). To achieve these objectives, we will develop new feature grouping, attention and appearance-based recognition processes. This will also require development of new techniques for acquiring, representing and using visual context and situation knowledge.
Work description:
The main objective is to develop the theory of context-aware visual recognition systems. The key elements of the project are: use of foveal sensing, descriptive features extracted from the foveated image data, selective attention using context priming as well as image feature salience to steer the sensor and recognition using learned appearance models of object, context and situation. We will investigate individual components of this problem and integrate working modules into a complete working object, context and situation recognition system. To evaluate the performance of the approach, we will acquire model s and apply the system to analysis of image sequences of city centre pedestrian and potential consumer behaviour.
Five scientific workpackages will investigate:
1) foveal camera control, foveal image acquisition, foveal image feature extraction and spatial and temporal grouping.
2) system architectures that allow incremental and continual project refinement and alternative process controllers.
3) feature-based, hierarchical attention processes that receive top-down priming about interesting objects, contexts and situations.
4) object, context and situation representations, how to invoke instances of these to explain image data, how to evaluate the similarity of these to the image data and how to recover from incorrect decisions.
5) tools and methods for specifying and automatically learning object, scene and context representations and control strategies. A sixth workpackage on performance characterisation allows continual validation of individual component and full system progress. A seventh workpackage is focussed on dissemination of project results.
Milestones:
The expected project results are the theory behind and the software implementation of: Month 12) a complete but rudimentary working object, situation and context recognition system Month 24) improved system with modules based on foveal features, selective attention and situation models Month 30) addition of grouping, top-down priming and property learning Month 36) addition of automatic context model learning and error recovery
Fields of science
CORDIS classifies projects with EuroSciVoc, a multilingual taxonomy of fields of science, through a semi-automatic process based on NLP techniques.
CORDIS classifies projects with EuroSciVoc, a multilingual taxonomy of fields of science, through a semi-automatic process based on NLP techniques.
Call for proposal
Data not availableFunding Scheme
CSC - Cost-sharing contractsCoordinator
EH8 9YL EDINBURGH
United Kingdom