Final Activity Report Summary - PHIOR (Photometric Robust Features for Object Recognition in Colour Images)
For computers the recognition of objects in pictures is a very challenging task. If the same object is taken from another angle, under indoor lighting or outdoor lighting, or partially occluded by another object, the appearance of the object in the picture changes enormously. Computer vision researchers have dedicated a lot of research in attempting to find representations of objects which are robust to these variations. Finding such representations would allow computers to recognize objects, which would have potential use in many industry areas, such as robotics, health care, defence, space exploration, surveillance, transportation, image and video search engines.
One of the remaining questions in this research area is how can we determine that the colour of two objects taken under varying lighting conditions, e.g. sun light and candle light, is the same. During his thesis research at the University of Amsterdam, Joost van de Weijer, conducted research to the photometric invariant description of object colours. Depending on the physical variations different descriptors needed to be designed. From physical models of the reflectance of light, descriptions of object colour were derived. These descriptions were proven to be invariant to multiple physical variations, among which shadow changes, viewpoint variation, object position, and illuminant colour.
In a two year Marie Curie fellowship at INRIA Rhone-Alpes, Joost van de Weijer sought to apply his colour research to object recognition. His research was conducted at the LEAR team at INRIA Rhone-Alpes, which is one of the leading research groups in Europe in this field. Object recognition systems operate as follows: they break up the image in a lot of small image pieces, much like puzzle pieces. Given enough images of a certain object, for example a car, the computer learns that some pieces are often reoccurring, such as tires, bumpers, and license plates. We call these often occurring pieces the car model. Given an unseen image, the computer will again divide the image in many small pieces. If enough pieces resemble the car model, the image will be classified as 'containing a car'. Until recently, most object recognition systems only used luminance information and ignored colour information, which was believed to be untrustworthy. Hence, the description of the picture pieces only described the shapes of the pieces (such as corner-ness or blob-ness). During the fellowship the descriptors were extended with a colour description. Subsequently, every piece was described by both its shape and its colour. The hope was that this would help recognizing coloured objects classes such as faces, trees, and traffic lights.
During the fellowship extensive experiments were performed to evaluate the usefulness of colour for object recognition. One of these experiments is called image classification. In image classification the task is to answer for a number of object classes, for example trees, cars, buildings, if they are present in an image or not. To compare the results of various object recognition methods PASCAL, a European research network, organises a yearly challenge in which research groups from all over the world participate. In the 2007 competition, INRIA Rhone-Alpes won the image classification task. An analysis of the results showed that the colour descriptors significantly helped in obtaining the results.
The research showed that although colour varies a lot in appearance under varying imaging conditions, such as viewing angle, and light source colour, it is possible to extract meaningful colour information. This information can then be used to improve object recognition systems, helping computer to better recognise coloured objects in the world.
One of the remaining questions in this research area is how can we determine that the colour of two objects taken under varying lighting conditions, e.g. sun light and candle light, is the same. During his thesis research at the University of Amsterdam, Joost van de Weijer, conducted research to the photometric invariant description of object colours. Depending on the physical variations different descriptors needed to be designed. From physical models of the reflectance of light, descriptions of object colour were derived. These descriptions were proven to be invariant to multiple physical variations, among which shadow changes, viewpoint variation, object position, and illuminant colour.
In a two year Marie Curie fellowship at INRIA Rhone-Alpes, Joost van de Weijer sought to apply his colour research to object recognition. His research was conducted at the LEAR team at INRIA Rhone-Alpes, which is one of the leading research groups in Europe in this field. Object recognition systems operate as follows: they break up the image in a lot of small image pieces, much like puzzle pieces. Given enough images of a certain object, for example a car, the computer learns that some pieces are often reoccurring, such as tires, bumpers, and license plates. We call these often occurring pieces the car model. Given an unseen image, the computer will again divide the image in many small pieces. If enough pieces resemble the car model, the image will be classified as 'containing a car'. Until recently, most object recognition systems only used luminance information and ignored colour information, which was believed to be untrustworthy. Hence, the description of the picture pieces only described the shapes of the pieces (such as corner-ness or blob-ness). During the fellowship the descriptors were extended with a colour description. Subsequently, every piece was described by both its shape and its colour. The hope was that this would help recognizing coloured objects classes such as faces, trees, and traffic lights.
During the fellowship extensive experiments were performed to evaluate the usefulness of colour for object recognition. One of these experiments is called image classification. In image classification the task is to answer for a number of object classes, for example trees, cars, buildings, if they are present in an image or not. To compare the results of various object recognition methods PASCAL, a European research network, organises a yearly challenge in which research groups from all over the world participate. In the 2007 competition, INRIA Rhone-Alpes won the image classification task. An analysis of the results showed that the colour descriptors significantly helped in obtaining the results.
The research showed that although colour varies a lot in appearance under varying imaging conditions, such as viewing angle, and light source colour, it is possible to extract meaningful colour information. This information can then be used to improve object recognition systems, helping computer to better recognise coloured objects in the world.