Final Report Summary - ALLEGRO (Active large-scale learning for visual recognition)
A massive and ever growing amount of digital image and video content is available today, on sites such as Flickr and YouTube, in audiovisual archives such as those of BBC and INA, and in personal collections. In most cases, it comes with additional information, such as text, audio or other metadata, that forms a rather sparse and noisy, yet rich and diverse source of annotation, ideally suited to emerging weakly supervised and active machine learning technology. The ALLEGRO project has taken visual recognition to the next level by using this largely untapped source of data to automatically learn visual models. We have developed new algorithms and computer software capable of autonomously exploring evolving data collections, selecting the relevant information, and determining the visual models most appropriate for different object, scene, and activity categories. We have also designed ways of generating synthetic data for learning visual model, an approach complementary to weak supervision. Moreover, we have significantly moved forward the state-of-the-art on learning visual models from video and on the representation and recognition of human activities.