Final Report Summary - TS-VICI (Task Specific Description of Visual Color Information) The reintegration grant has helped the beneficial to perform independent research and to improve his career opportunities in academy. The grant has been primarily used to pay the salaries of a PhD student (18 months), a postdoctoral researcher (3 months) and travel to conferences. The grant has provided the beneficial with the financial independence which has accelerated the integration into the Computer Vision Center. Furthermore, the grant allowed the beneficial to keep the relation with his thesis supervisor at the university of Amsterdam (leading to 3 journals during the project) and with the leading researcher of the Intraeuropean Marie Curie at INRIA Rhone-alps (leading to 1 journal publication). The beneficial has currently obtained funding from the Spanish government and is successfully supervising three PhD students. The main goal of the project is to develop new methods to combine colour and shape description for object recognition. In accordance with this we set out the following objectives: I. Automatic adaptation colour description: Colour image description should automatically balance discriminative power and photometric information depending on task, and object category. a) For this purpose, a single theory should be developed which combines both photometric invariance and discriminative power. b) Learning techniques should be used which allow for class specific colour description. II. Field specific adaptation colour descriptor: The theory as developed in the Marie Curie is also applicable to related computer vision fields such as robotic vision, image and video retrieval, automatic image annotation and tracking. For optimal use in these fields specific adaptations to the descriptor are likely to be necessary. As an example we mention video retrieval, were one should consider how to correctly process colour information along the temporal axis. III. Performance evaluation: The performance of the developed techniques should be evaluated against competing colour description approaches, and luminance-based (non-colour) approaches. Within the context of bag-of-words based object recognition to combination of colour and shape has been found to give suboptimal results (only a few percentage gain was obtained with colour, and sometimes even losses were found by adding colour information).The research in this joint project suggests the usage of colour to guide the shape description of objects. Results were found to outperform existing methods on colour and shape combination. This project has led to ICCV 2009 paper. An extension which allows for class-specific weights which are automatically learned from training data is in minor review at IJCV (please see http://www.cat.uab.cat/Research/object-recognition/ online). Another research path has focussed on physically realistic recoloring of objects. The project has potential implication in photo editing, publicity and gaming industry. In current available photo edit software users need to pre-segment an object, after which they can choose to change the colour. This research will be presented at ICCV 2011 (please see http://www.cat.uab.cat/~shida/Research/ObjectRecoloring.html online). Further research has been conducted on improving colour based feature detection. The extension to colour feature descriptors has been proven to be beneficial. However, the extension to colour of feature detectors has had little attention. This work has been presented at ICPR 2010 and is available at http://cat.cvc.uab.es/~darojas/FeatureDetection/FeatureDetection.html.
