# Adaptive Image Matching Using Discrimination of Deformable Objects

^{1}

^{2}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Related Works

#### 2.1. Neighborhood-Based Matching

#### 2.2. Statistical-Based Matching

#### 2.3. Deformable Object-Based Matching

## 3. Proposed Algorithm

#### 3.1. Feature Detection, and Making a Matched Pair

_{1}and d

_{2}are the nearest and second nearest neighbor distances, D

_{A}is the target descriptor, D

_{B}and D

_{C}are its closest two neighbors, and the symbol $\parallel \u2022\parallel $ denotes the Euclidean distance. Lowe demonstrated that the probability of a false match (e.g., a feature with a similar pattern) significantly increases when NNDR > 0.8 [17]. Thus, matching pairs with an NNDR higher than 0.8 are not employed. Numerous studies showed that forming 1:1 matching pairs using NNDR leads to the best performance. However, matching for deformable objects the single matching-pair can be outliers, which would disrupt performance. Therefore, considering deformable-object matching, up to k candidates are selected in decreasing order of ratio, rather than selecting a single candidate with NNDR. A feature point forms 1:k matching pairs using k-NNDR. For rigid-object matching, matching pairs with a k = 1 are used, and in the deformable object-matching method, k = 2 or 3 is used.

_{k}) consist of each of the feature points from the reference and query images. In Equation (3), M

_{k}(p

_{k}) denotes the positions of the two feature points. Here, ${p}_{k}=\left[{p}_{k}^{R},{p}_{k}^{Q}\right]$, where ${p}_{k}^{R}$ is the position of the feature point extracted from the reference image, and ${p}_{k}^{Q}$ is the position of the feature point extracted from the query image. If ${p}_{i}^{R}$ is equal to ${p}_{j}^{R}$, or if ${p}_{i}^{Q}$ is equal to ${p}_{j}^{Q}$ when comparing the i-th matching pair (M

_{i}) with the j-th matching pair (M

_{j}), it is recognized as a repetition, and 1 is assigned to $ovlp\left[i,j\right]$. In this manner, 1 or 0 is assigned to all $ovlp\left[i,j\right]$, eventually generating an NM × NM overlap matrix, which has $ovlp\left[i,j\right]$ as elements. The generated overlap matrix is used for clustering during deformable-object matching.

#### 3.2. Geometric Verification for Rigid Object-Matching

_{1},y

_{1}),…,(x

_{n},y

_{n}), the LDR set Z is obtained with the following equation:

_{i,j}is a coordinate of the reference image, and y

_{i,j}and x

_{i,j}are the matched-feature coordinates.

**r**are calculated using D

**r**= u

**r**, used in Equation (11).

**r**is sorted in descending order, and the upper-range matching pairs corresponding to the calculated number of inliers are finally determined to be the inliers. Figure 4b shows a diagrammatic representation of this process.

#### 3.3. Discrimination of Deformable Object Images

_{1},c

_{1}), (x

_{2},c

_{2}),…,(x

_{n},c

_{n}), x

_{i}for the i-th properties, and let c

_{i}be the class index of x

_{i}, which represents non-matching (w

_{1}) and deformable-matching (w

_{2}). X is defined as the ratio between the inlier number from matching information and matching-pair weight, as shown in Equation (13):

#### 3.4. Deformable-Object Matching

_{i}, and M

_{j}, transformations T

_{i}and T

_{j}and translations t

_{i}, and t

_{j}can be calculated from the characteristics of each matching pair, using the enhanced weak geometric consistency (WGC) [20], as shown in Equation (14).

_{x}and t

_{y}represent coordinate translations. Using Equation (9), the matching pairs can be expressed as ${M}_{i}=\left(\left({x}_{i},{y}_{i}\right),({x}_{i}^{\prime},{y}_{i}^{\prime}),{T}_{i}\right)$ and ${M}_{j}=\left(\left({x}_{j},{y}_{j}\right),({x}_{j}^{\prime},{y}_{j}^{\prime}),{T}_{j}\right),$ and the geometric similarity of the two matching pairs is calculated using Equation (15):

_{geo}(M

_{i},M

_{j}) will be close to 0. A graphical representation of this is shown in Figure 7.

_{geo}(M

_{i},M

_{j}) have a similar geometric relation. Therefore, similarity is computed by calculating the geometric transformation between each matching pair, rather than by defining a transformation model of the whole image.

_{m}. Secondly, a cluster is determined to be a valid cluster if the area of the matching pairs that form the cluster is larger than a certain portion of the entire area (τ

_{a}). The area of the matching pairs that form the cluster is calculated using a convex hull. Figure 10 shows the results of removing the invalid clusters through clustering validation for each case of inlier matching and outlier matching. From inlier matching, it was observed that the clusters with a small area are removed, and for outlier matching, the accuracy was enhanced by preventing the false positives that occur due to small clusters.

## 4. Experiment Results

#### 4.1. Geometric Verification Test for Rigid Object Matching

_{m}is the threshold value. Matching was determined for a matching score >0.5. Accordingly, the experiment was conducted by altering the T

_{m}value. The optimum value for T

_{m}was determined through the receiver operating characteristic (ROC) curve, and set at T

_{m}= 3, allowing a comparison of the performances of standard matching methods. A rigid-object image from the SMVS dataset was employed as an experimental image.

#### 4.2. Discriminating Deformable Objects Using Voting Methods

#### 4.3. Deformable Object-Matching Performance Test

_{m}). The optimum value was determined through experiment to be τ

_{m}= 5. Figure 15b shows the calculated ratio of each cluster area and the entire image area. A false cluster was deemed to occur when the ratio (τ

_{a}) is low. From the experiment, the best value was confirmed to be τ

_{a}= 0.01, and hence, this value was employed.

#### 4.4. Performance Evaluation for the Proposed Matching Method

_{m}= 3 for the matching score was employed for rigid-object matching. Moreover, rigid matching was deemed to occur when the calculated matching score was higher than 0.5; voting > 1, which determines the best performance, was employed for voting on discrimination of deformable object images. Finally, cutoff = 30 was employed for cluster matching in deformable-object matching, while values of τ

_{m}= 5 and τ

_{a}= 0.01 were employed for cluster validation. For the performance comparison, the same parameters were employed for DISTRAT [9] and ACC [15].

## 5. Conclusions

## Acknowledgments

## Author Contributions

## Conflicts of Interest

## References

- Krizhevsky, A.; Sutskever, I.; Hinton, G.E. Imagenet classification with deep convolutional neural networks. In Proceedings of the Advances in Neural Information Processing Systems (NIPS), Lake Tahoe, NV, USA, 3–8 December 2012; pp. 1097–1105.
- Won, I.; Yang, H.; Jang, H.; Jeong, D. Adaptive Matching Method of Rigid and Deformable Object Image using Statistical Analysis of Matching-pairs. J. Inst. Electron. Inform. Eng.
**2015**, 52, 102–110. [Google Scholar] [CrossRef] - Liu, S.; Song, Z.; Liu, G.; Xu, C.; Lu, H.; Yan, S. Street-to-shop: Cross-scenario clothing retrieval via parts alignment and auxiliary set. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Providence, RI, USA, 16–21 June 2012; pp. 3330–3337.
- Na, S.; Oh, W.; Jeong, D. A Frame-Based Video Signature Method for Very Quick Video Identification and Location. ETRI J.
**2013**, 35, 281–291. [Google Scholar] [CrossRef] - Kahaki, S.; Nordin, M.J.; Ashtari, A.H.; Zahra, S.J. Deformation invariant image matching based on dissimilarity of spatial features. Neurocomputing
**2016**, 175, 1009–1018. [Google Scholar] [CrossRef] - Awrangjeb, M.; Lu, G. Contour-Based Corner Detection and Robust Geometric Point Matching Techniques. Ph.D. Thesis, Monash University, Melbourne, Australia, 2008. [Google Scholar]
- Fischler, M.A.; Bolles, R.C. Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM
**1981**, 24, 381–395. [Google Scholar] [CrossRef] - Tsai, S.S.; Chen, D.; Takacs, G.; Chandrasekhar, V.; Vedantham, R.; Grzeszczuk, R.; Girod, B. Fast geometric re-ranking for image-based retrieval. In Proceedings of the International Conference on Image Processing (ICIP), Hong Kong, 26–29 September 2010; pp. 1029–1032.
- Lepsøy, S.; Francini, G.; Cordara, G.; Gusmão, D.; Buarque, P.P. Statistical modelling of outliers for fast visual search. In Proceedings of the International Conference Multimedia and Expo (ICME), Barcelona, Spain, 11–15 July 2011; pp. 1–6.
- Duan, L.; Chandrasekhar, V.; Chen, J.; Lin, J.; Wang, Z.; Huang, T.; Girod, B.; Gao, W. Overview of the MPEG-CDVS Standard. IEEE Trans. Image Process.
**2016**, 25, 179–194. [Google Scholar] [CrossRef] [PubMed] - Zhu, J.; Hoi, S.C.; Lyu, M.R. Nonrigid shape recovery by Gaussian process regression. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Miami, FL, USA, 20–25 June 2009; pp. 1319–1326.
- Pilet, J.; Lepetit, V.; Fua, P. Real-time nonrigid surface detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), San Diego, CA, USA, 20–26 June 2009; pp. 822–828.
- Kettani, O.; Ramdani, F.; Tadili, B. An Agglomerative Clustering Method for Large Data Sets. Int. J. Comp. Appl.
**2014**, 92, 1–7. [Google Scholar] [CrossRef] - Zhou, F.; Torre, F. Deformable graph matching. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Portland, OR, USA, 23–28 June 2013; pp. 2922–2929.
- Cho, M.; Lee, J.; Lee, K.M. Feature correspondence and deformable object matching via agglomerative correspondence clustering. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), Kyoto, Japan, 27 September–4 October 2009; pp. 1280–1287.
- Yang, H.; Won, I.; Jeong, D. On the Improvement of Deformable Object Matching. In Proceedings of the Korea-Japan Joint Workshop on Frontiers of Computer Vision (FCV), Okinawa, Japan, 4–6 February 2014; pp. 1–4.
- Lowe, D.G. Distinctive image features from scale-invariant keypoints. Int. J. Comp. Vis.
**2004**, 60, 91–110. [Google Scholar] [CrossRef] - Krystian, M.; Cordelia, S. A Performance Evaluation of Local Descriptors. IEEE Trans. Pattern Anal. Mach. Intell.
**2005**, 27, 1615–1630. [Google Scholar] - Chapelle, O.; Haffner, P.; Vapnik, V.N. Support vector machines for histogram-based image classification. IEEE Trans. Neural Netw.
**1999**, 10, 1055–1064. [Google Scholar] [CrossRef] [PubMed] - Zhao, W.; Wu, X.; Ngo, C. On the annotation of web videos by efficient near-duplicate search. IEEE Trans. Multimed.
**2010**, 12, 448–461. [Google Scholar] [CrossRef] - Chandrasekhar, V.R.; Chen, D.M.; Tsai, S.S.; Cheung, N.; Chen, H.; Takacs, G.; Reznik, Y.; Vedantham, R.; Grzeszczuk, R.; Bach, J. The Stanford mobile visual search data set. In Proceedings of the Second Annual ACM Conference on Multimedia Systems, Santa Clara, CA, USA, 23–25 February 2011; pp. 117–122.
- Bookstein, F.L. Principal warps: Thin-plate splines and the decomposition of deformations. IEEE Trans. Pattern Anal. Mach. Intell.
**1989**, 10, 567–585. [Google Scholar] [CrossRef] - Nister, D.; Stewenius, H. Scalable recognition with a vocabulary tree. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), New York, NY, USA, 17–22 June 2006; pp. 2161–2168.
- Arya, S.; Mount, D.M.; Netanyahu, N.S.; Silverman, R.; Wu, A.Y. An optimal algorithm for approximate nearest neighbor searching fixed dimensions. J. ACM
**1998**, 45, 891–923. [Google Scholar] [CrossRef]

**Figure 3.**(

**a**) Results of feature matching using NNDR (

**left**: incorrectly matched images,

**right**: correctly matched images); (

**b**) the lines point to the positions of the matched features in the other image; and (

**c**) the LDR histogram of the model function.

**Figure 4.**Analysis for geometric verification based on LDR (k = 25): (

**a**) example of LDR histogram h(k), model function f(k), and difference d(k) (

**top**: correctly matched images,

**bottom**: incorrectly matched images); and (

**b**) eigenvector

**r**for finding inliers (

**top**), and the results in descending order (

**bottom**).

**Figure 5.**Normal distribution model of matching information from matching pairs of deformable and non-matching images: (

**a**) number of matching pairs; (

**b**) number of inliers; (

**c**) sum of all matching pairs’ distances; (

**d**) sum of inliers’ distances; and (

**e**) matching information of x.

**Figure 6.**Comparison of d(k) between matching images: (

**a**) d(k) of rigid matching pair; (

**b**) d(k) of deformable-object matching pair; and (

**c**) d(k) of non-matching pair.

**Figure 9.**Deformable image-matching results using hierarchical clustering of geometric similarity: (

**a**) matching results for rigid objects with geometric transformation; and (

**b**) matching results of a deformable object.

**Figure 10.**Comparison results from before (

**left**) and after (

**right**) clustering validation: (

**a**) inlier-matching pairs; and (

**b**) outlier-matching pairs.

**Figure 11.**Examples of images for the matching test: (

**a**) SMVS datasets; (

**b**) deformable transformation images (normal, light, medium, and heavy); and (

**c**) clothing images.

**Figure 13.**Performance per voting values (

**N**: normal images;

**L**: light deformable images;

**M**: medium deformable images;

**H**: heavy deformable images; and

**C**: clothing images): (

**a**) true positive rate for voting values; (

**b**) false positive rate for voting values; (

**c**) accuracy for voting values; and (

**d**) execution time for voting values.

**Figure 14.**Experimental results according to parameters in clustering of geometric similarity for deformable-object matching: (

**a**) experimental result based on the cutoff of geometric similarity; and (

**b**) experimental result based on k of a k-NN linkage.

**Figure 15.**Results of the parameter experiment for cluster validation: (

**a**) threshold of matching-pairs constituting a cluster; and (

**b**) experimental results based on the ratio of clusters to the entire image area.

**Figure 16.**Performance results of the matching methods: (

**a**) comparison of true positive rate; and (

**b**) comparison of accuracy.

Methods | TPR | FPR | Accuracy | Matching Time (s) |
---|---|---|---|---|

DISTRAT [9] | 83.51% | 6.41% | 88.55% | 0.446 |

ANN [24] | 70.27% | 3.03% | 83.62% | 0.536 |

ACC [15] | 86.45% | 6.46% | 89.99% | 3.759 |

CDVS(Global) [10] | 67.27% | 0.35% | 83.46% | 0.003 |

CDVS(Local) [10] | 74.94% | 0.28% | 87.33% | 0.005 |

Proposed | 89.78% | 7.12% | 91.33% | 0.521 |

© 2016 by the authors; licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC-BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Won, I.; Jeong, J.; Yang, H.; Kwon, J.; Jeong, D.
Adaptive Image Matching Using Discrimination of Deformable Objects. *Symmetry* **2016**, *8*, 68.
https://doi.org/10.3390/sym8070068

**AMA Style**

Won I, Jeong J, Yang H, Kwon J, Jeong D.
Adaptive Image Matching Using Discrimination of Deformable Objects. *Symmetry*. 2016; 8(7):68.
https://doi.org/10.3390/sym8070068

**Chicago/Turabian Style**

Won, Insu, Jaehyup Jeong, Hunjun Yang, Jangwoo Kwon, and Dongseok Jeong.
2016. "Adaptive Image Matching Using Discrimination of Deformable Objects" *Symmetry* 8, no. 7: 68.
https://doi.org/10.3390/sym8070068