>>2006,Wu Bo,A Reliable Stereo Image Matching Method Based on the Self-adaptive Triangle Constraint

---

Abstract

Digital photogrammetry technology is now generally used for the derivation of geometric, radiometric, and semantic information of objects in the 3D real world from 2D digital stereo images. Along with the easy availability of more image data and the increasing requirements for the application of 3D space information, the performances of perfect automaticity, good efficiency and high reliability to the digital photogrammetry are more considered in nowadays. One of the difficult tasks in digital photogrammetry is stereo image matching, which is the process to automatically search for the corresponding points in the overlapping area of the stereo pairs by making use of computers to substitute or simulate human eyes, so as to compute object coordinates in a 3D ground coordinate system. Stereo image matching is also one of the most important and challenging subjects in computer vision and image understanding.

The existing image matching methods work well in open areas with relatively smooth terrain. However, they still meet many problems in the case of larger scale images of nature, especially in the dense city centers, forestry areas and the textureless regions such as water bodies. The reliability of image matching is still a big issue in these cases. For example, the corners of buildings and some object boundary points sometimes cannot be matched successfully, and there are many blunders in the automatically derived digital surface models that need to be manually removed, which leads to the decrease of the automaticity in digital photogrammetry. For the purpose of reliable image matching, Zhu et al. (2005) and Zhao (2004) presented a novel stereo image matching method based on the self-adaptive triangle constraint. They firstly triangulate a few seed points on the edge of irregular areas to form a coarse corresponding Delaunay triangulation pair, each triangle in these triangulations is considered as a local continuous area; then detect certain amount of interest points within each triangle, match these interest points under the triangle constraint, and obtain a pair of corresponding points with maximum reliability; after that, insert the new matched corresponding points into the triangulations, and update the triangulations dynamically, then handle the next pair of triangles, and repeat the same matching process until the termination conditions of the propagation are met. Because the most distinctive point is always successfully matched first, the dynamic updating of triangulations is just the process of self-adaptive matching propagation. Along with the matching propagation, the local geometry constraint of the dynamically updated triangles can adapt to the changes in image textures automatically, and will finally produce more reliable matching results. Based on the previous works of Zhu et al. (2005) and Zhao (2004), this dissertation researches further the key issues of this reliable image matching method and the technical details in realizing this method as follows:

• An interest point detecting method with good repeatability and information content. This dissertation introduces several stereo image interest point detectors generally used in digital photogrammetry, and compares them with respect to their repeatability and information content through experimental analysis. The Harris-Laplace detector gives better results than other detectors in areas of good texture; however, in areas of poor texture, the Harris-Laplace detector may be not the best choice. A feature-related filtering strategy is then designed for the Harris-Laplace detector to improve the repeatability and information content for imagery with both good and poor texture by picking out those points with less interest strength, which benefits the subsequent stereo image matching and the 3D object reconstruction. The performance of this filtering strategy is supported by experimental analysis with actual stereo images.
• A subpixel location method for interest points. The interest points detected by the Harris-Laplace detector are preferable for stereo image matching and the 3D object reconstruction. However, Harris-Laplace detector clearly suffers from delocalization and forgets a part of the shape. So, this dissertation presents a subpixel location method to improve the precision of the interest points by means of the Harris interest strength. Firstly, a least squares fit of a paraboloid function to the Harris interest strength in the image grayscale surface is designed. Then, the precise coordinates of the interest points are obtained by calculating the extremities of the fitting surface. This method can be used to locate interest points including corners, junction points, dominant points, blob-like points and all other types of distinctive points with important two-dimensional texture features. Experimental analysis from the theoretical and the practical point of view both prove that the location accuracy of 0.15 pixels is obtained. Results from intensiveexperimental work also indicate that this method has the desired performance of anti-interference, is invariant to image rotation and is highly efficient.
• Triangle constrained cross-correlation (TC3) matching. This dissertation proposes the theoretical framework of the TC3 matching method, and designs the detailed TC3 algorithm. Firstly, the essential matching constraints of TC3are introduced. Then, the measurement criteria of matching reliability in TC3 is defined by integrating these matching constraints, the contribution of different constraint to the matching reliability is also experimental analyzed. Furthermore, this dissertation studies the matching strategies to decrease the mismatching problems in difficult texture areas such as in textureless areas, homogeneous texture areas, surface discontinuity areas and image occlusion areas, and the actual performance of these strategies is confirmed by experimental analysis with actual stereo images.
• Matching propagation. The matching propagation is the most important issue in the self-adaptive triangle constrained image matching method. This dissertation summarizes the different matching propagation manners in the existing image matching methods, and proposes a novel image matching propagation strategy of “triangle-to-triangle”. Firstly, because the triangle constrained matching propagation begins with the initial corresponding triangulations formed by a few seed points in the stereo pairs, this dissertation studies the affects of different numbers and different distributions of seed points on the image matching results through experimental analysis, and proposes a seed point selection strategy for different image texture conditions, an automatic selection algorithm is also designed that gives good distribution quality for a defined number of seed points. Then, to carry out the triangle constrained matching propagation, the fast construction method, the data structure and storage manners as well as the local dynamic updating method of the corresponding Delaunay triangulations with mass data are introduced. Based on these former works, this dissertation proposes three propagation strategies: the stochastic propagation, the adjacent propagation based on the topological relationship of triangles, and self-adaptive propagation, which considers the texture feature. The detailed algorithms of these three propagation strategies are also presented. To compare these propagation strategies, an intensive experimental analysis is illustrated with different stereo pairs, and the results prove that the self-adaptive propagation gives best matching reliability than the others due to the “best first” strategy.

The stereo image matching method this dissertation proposed is programmed using VC++ 6.0, and several typical stereo pairs that cover different texture areas are selected to test the performance of this method. From the experimental results, this method is preferable to process aerial stereo pairs with different texture features, the main object boundary points and the terrain feature points can be matched successfully, and the matched result triangulation is accommodated to the image textures. Experimental analysis also proves that the precison of the derived digital surface models from this method is better than the existing methods without eliminating the blunders.

This dissertation proposes a reliable stereo image matching method based on the self-adaptive triangle constraint, which employs a novel dynamic triangle constraint to the image matching. This method gives a new solution for the reliable stereo image matching, and the research of this dissertation provides a theoretical foundation for extending and enhancing some of the related functions in the digital photogrammetry system, some methods and algorithms in this dissertation can be applied to the existing digital photogrammetry workstation directly.

Key Words

Stereo image matching; triangle constraint; interest point; matching propagation