Skip to main content
European Commission logo print header

Deep Structure, Singularities, and Computer Vision

Deliverables

All the results produced by the Liverpool team are theoretical, as envisaged in the original proposal. Deliverable 34 is a first investigation of 'pre-symmetry sets' of surfaces in 3D space. The idea here is that the symmetry set is too complex but contains much valuable information not present in the medial axis, therefore some construction more primitive than the symmetry set is in order. Many results are obtained on the local structure of pre-symmetry sets, particularly of their local parameterisations. This deliverable also contains the latest information on symmetry sets of families of surfaces in 3-space. These theoretical results are not directly applicable to technology. That will be a later stage for someone else to consider.
The application of this methodology is mainly in the evaluation of information contents in image features. As such, it has already within this project been used for showing that edges and blobs are sufficient fully to characterize an image. This is trivially extended any other feature type. Hence, in R&D in feature-based image analysis this tool substitutes the researchers' intuition by object measurements leading to faster and better innovation.
We have studied combinatorial tree matching problems primarily focusing on problems based on local transitions or edit operations in the tree. These types of problems are suitable for MSST matching since transitions in the underlying images naturally correspond to transitions in the trees. The key contributions of the work are a number of papers, including a comprehensive survey of tree matching problems and two theoretical papers studying two specific tree-matching problems. The latter two papers present new algorithms significantly improving the previously known complexity bounds.
We have addressed the nontrivial task of finding a fiducial object, captured by a digital image ("query image"), in another image, or database of such images, consisting of complex configurations of various objects ("scene image"). The requirements posed to this object retrieval or structural search problem include robustness under various degrading factors, notably noise, occlusion, and invariance under similarity transformations, i.e. relative translation, rotation, and scaling of reference frames of scene object relative to query object. In addition the algorithm should be operationally defined and mathematically rigorous. Elaborating on pioneering heuristics by Lowe, further developed by Mikolajczyk, and Schmid (which has led to promising results, but lacking mathematical underpinning) we have realized a prototype object retrieval algorithm that combines the strengths of Lowe's heuristics and mathematical rigor. This has the benefit that the object retrieval problem has been cast into a well-understood conceptual framework and a transparent algorithm. A performance comparison shows that our object retrieval algorithm competes with state-of-the-art heuristic algorithms despite its very generic nature (no application specific tuning or hidden parameters have been incorporated). As such our framework is quite susceptible to future improvements by the scientific community. The ultimate goal of a fully operational generic object retrieval algorithm for structural search in image databases (e.g. object google, forensic research or comparisons of clinically similar cases) has thus come within closer reach. However, several weaknesses will have to be overcome; in particular computational efficiency will have to be improved, and deviations from similarity transformations between query object and matching scene object will have to be accounted for.
A performance evaluation on shape matching based on the pre-symmetry diagram has been made. The algorithm has been developed and implemented including a distance measure on shapes. A database of 99 shapes divided in 11 classes. The same shape database is used by Kimia, Sebastian et al. The results are that the total amount of errors compared to the human observer classification is given by (0, 3, 6, 6, 12, 17, 24, 28, 42, 43, 58). Each number in the tupple is in the range 0 to 99. The first position is the sum of the errors among all the best matches to each shape in the database. An error is 0 if the match belongs to the class as query shape and is 1 if the match belongs to another class than the query. The matching algorithm does not in its current form handle severe occlusion. If the three most clear occlusion-caused cases are left out of the match, the above result is improved to (0, 0, 3, 5, 9, 15, 22, 25, 39, 41, 56). This means that the two best suggested matches by the method will always give a shape in the right class, etc. The results are worse than the current state-of-the-art on this database and comparable to early results using the medial axis as shape representation. The current state-of-the-art result from the latest tuned method on the used database is (0, 0, 0, 1, 1, 3, 4, 5, 13). This result shows that the symmetry sets may be used for characterizing shapes and effective in shape retrieval. This can be the core technology in content-based image database queries.
Prior to this project, the gradient magnitude singularity trees have shown instrumental in segmentation of 2D and 3D images. This e.g. led to the spin-off company Generic Vision employing the technology for the segmentation of medical images. The current result makes the foundation of incorporating also object recognition in such an approach.
Symmetry sets have been demonstrated to be efficient for shape characterisation in 2D. This work makes the foundation of extending this to 3D shapes. In this way it may be shown possible to carry over results of efficiency and improved quality from 2D to 3D as in e.g. medical image databases.
All the results produced by the Liverpool team are theoretical, as envisaged in the original proposal. In Deliverable 23 we gave the latest information on the symmetry sets of isophote curves in all the generic cases. These matters have not been investigated before. We obtained a new subdivision of the hyperbolic region of a surface into two sub-regions separated by the 'vertex transition curve'. This separates the two generic behaviours of the vertices (maxima/minima of curvature) of sections of a surface at a given hyperbolic point. No global definition of this curve is known, and in fact this question has been taken up by other mathematicians at INRIA in France following a visit by myself to INRIA in December 2005. I hope indeed to pursue it myself. These theoretical results are not directly applicable to technology. That will be a later stage for someone else to consider.
In this report, we present and evaluate an algorithm that exploits the Multi-Scale Singularity Trees (MSSTs) for image matching. Two versions of the algorithm is presented: an exact and an approximation. Several experiments are conducted to empirically evaluate the MSST matching algorithm under image distortions. Further, the performance of the MSST matching algorithm is measured on three databases: the ORL face database, magazine covers, and the COIL database. Finally, the performance is compared with algorithms based on the Scale Invariant Feature Transform (SIFT) and the Position of Catastrophes (CAT). The intended application is image matching and this highlights the strengths and weaknesses of the three methods MSST, the SIFT, or the CAT. The potential end-user application is image databases, which are found as medical X-ray images, newspaper archives, and private digital image archives. Our work has focused on the MSST structure and algorithms for matching these. Our results do not compare with the state-of-the-art of the SIFT algorithm, since we are limited algorithmically to handle only a few number of catastrophes, and thus can only obtain state-of-the-art results for blurry images. However, for these few number of parameters, our matching results are better than the SIFT algorithms, which indicate that the tree structure is closer linked to the important image information than the SIFT features. Thus, it is our opinion that the MSSTs and the matching algorithm presented in this report deserve further attention.
We have studied combinatorial tree matching problems primarily focusing on problems based on local transitions or edit operations in the tree. These types of problems are suitable for MSST matching since transitions in the underlying images naturally correspond to transitions in the trees. The key contributions of the work are a number of papers, including a comprehensive survey of tree matching problems and two theoretical papers studying two specific tree-matching problems. The latter two papers present new algorithms significantly improving the previously known complexity bounds.

Searching for OpenAIRE data...

There was an error trying to search data from OpenAIRE

No results available