Final Activity Report Summary - BUSIAP (Hierarchical classification, mining and semantic retrieval in remote sensing image archives)
Remotely sensed imagery has become an invaluable tool or scientists, governments and the general public to understand the world and its surrounding environment. Advances in satellite and airborne sensor technology have significantly increased the amount of remotely sensed image data and the need for their effective processing and storage. Therefore, automatic content extraction and efficient access to this content have become highly desired goals for developing intelligent systems in applications such as land cover/land use classification, urban mapping and monitoring, crop yield estimation, disaster management, monitoring of climate change, and ecological analysis.
The main objective of this project was the development of a hierarchical scene representation where the modelling of remotely sensed imagery was extended to three levels: pixel level, region level and scene level. This hierarchical scene representation aimed to bridge the gap between data and high-level expert interpretation by incorporating spatial content in the classification process for semantic image understanding. Pixel level representations corresponded to land cover labels for individual pixels such as water, grass, soil, asphalt and concrete. Region level representations included labels for groups of pixels such as lake, field, forest, road, building. Scene level representations corresponded to land cover and land use by modelling scenes using the relative spatial arrangements of regions such as urban settlement patterns that are modelled by arrangements of buildings and other urban structures, orchards that are modelled by arrangements of individual trees, and high-level searches such as queries for residential areas that are close to an industrial zone, etc.
The important objective about complementing pixel-based information with spatial context was achieved with the development of a novel segmentation algorithm that produced a hierarchical decomposition of an image where the regions at each level of the hierarchy corresponded to candidate structures at different scales. Then, the ones that were spectrally homogeneous and spatially consistent with their neighbourhood were automatically selected as meaningful structures. Furthermore, we developed a probabilistic algorithm that automatically selected coherent groups of segments from multiple segmentations where each group corresponded to an object such as road, building, tree, etc.
Another important achievement was made in the modelling of boundary-based, distance-based, and directional spatial relationships between region pairs and the between relationship among three regions. We developed efficient algorithms to compute such relationships and obtained results that were also more intuitive than those of alternative techniques. Furthermore, we extended these representations to scenes that contained multiple regions using attributed relational graph models that were automatically built using the segmentation results and pairwise relationships.
We demonstrated the effectiveness of this approach in query scenarios that could not be expressed by traditional approaches but where the proposed models could capture both feature and spatial characteristics of scenes and could retrieve similar areas according to their high-level semantic content. Finally, we developed measures of repetitiveness and periodicity for modelling spatial patterns of simple objects for the detection of complex geospatial objects and scene structures. We applied these measures to the modelling of building patterns to classify neighbourhoods in high-resolution images as organized and unorganised to study urban growth.
In summary, we made important scientific achievements in all objectives of the project and presented our contributions in many publications. We also identified new lines of research and new applications of the developed methods to other computer vision and pattern recognition problems. The reintegration was very successful in terms of setting up a new research group at the host institution, contributing to the undergraduate and graduate education, and establishing new collaborations.
The main objective of this project was the development of a hierarchical scene representation where the modelling of remotely sensed imagery was extended to three levels: pixel level, region level and scene level. This hierarchical scene representation aimed to bridge the gap between data and high-level expert interpretation by incorporating spatial content in the classification process for semantic image understanding. Pixel level representations corresponded to land cover labels for individual pixels such as water, grass, soil, asphalt and concrete. Region level representations included labels for groups of pixels such as lake, field, forest, road, building. Scene level representations corresponded to land cover and land use by modelling scenes using the relative spatial arrangements of regions such as urban settlement patterns that are modelled by arrangements of buildings and other urban structures, orchards that are modelled by arrangements of individual trees, and high-level searches such as queries for residential areas that are close to an industrial zone, etc.
The important objective about complementing pixel-based information with spatial context was achieved with the development of a novel segmentation algorithm that produced a hierarchical decomposition of an image where the regions at each level of the hierarchy corresponded to candidate structures at different scales. Then, the ones that were spectrally homogeneous and spatially consistent with their neighbourhood were automatically selected as meaningful structures. Furthermore, we developed a probabilistic algorithm that automatically selected coherent groups of segments from multiple segmentations where each group corresponded to an object such as road, building, tree, etc.
Another important achievement was made in the modelling of boundary-based, distance-based, and directional spatial relationships between region pairs and the between relationship among three regions. We developed efficient algorithms to compute such relationships and obtained results that were also more intuitive than those of alternative techniques. Furthermore, we extended these representations to scenes that contained multiple regions using attributed relational graph models that were automatically built using the segmentation results and pairwise relationships.
We demonstrated the effectiveness of this approach in query scenarios that could not be expressed by traditional approaches but where the proposed models could capture both feature and spatial characteristics of scenes and could retrieve similar areas according to their high-level semantic content. Finally, we developed measures of repetitiveness and periodicity for modelling spatial patterns of simple objects for the detection of complex geospatial objects and scene structures. We applied these measures to the modelling of building patterns to classify neighbourhoods in high-resolution images as organized and unorganised to study urban growth.
In summary, we made important scientific achievements in all objectives of the project and presented our contributions in many publications. We also identified new lines of research and new applications of the developed methods to other computer vision and pattern recognition problems. The reintegration was very successful in terms of setting up a new research group at the host institution, contributing to the undergraduate and graduate education, and establishing new collaborations.