# Dual-ISM: Duality-Based Image Sequence Matching for Similar Image Search

^{1}

^{2}

^{3}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Related Work

#### 2.1. Sequence Matching Methods

#### 2.2. Low-Dimensional Transformation Techniques

_{opt}, that maximizes the coefficient of determination of the total scatter matrix [16].

## 3. Dual-ISM (Proposed Method)

- Step 1: Feature point extraction; feature points are extracted from the original image, and a feature vector is constructed.
- Step 2: Low-dimensional transformation; for efficient subsequence matching, feature vectors that are extracted from the image are configured as disjoint windows and low-dimensional transformation is performed.
- Step 3: Candidate set construction; a set of candidates for subsequence matching is configured with windows in which the distance between the low-dimensional transformed data sequence and the query sequence is less than a given allowable value, ε.
- Step 4: Searching for similar image sequences; similar image sequences are searched for the constructed candidate set using a distance calculation that is based on the original feature vector.

#### 3.1. Step 1: Feature Point Extraction

#### 3.2. Step 2: Low-Dimensional Transformation

#### 3.3. Step 3: Candidate Set Construction

#### 3.4. Step 4: Searching for Similar Image Sequences

## 4. Experimental Results

#### 4.1. Comparative Experiment of Similar Image Search Accuracy—Low-Dimensional Transformation Methods

- 128 d+ED: Euclidean distance calculation that is based on the original feature vector
- LDA+ED: Euclidean distance calculation after LDA low-dimensional transformation
- Dual-ISM (LDA): the proposed method using LDA low-dimensional transformation
- PCA+Dual-ISM (PCA): the proposed method using PCA low-dimensional transformation
- NMF+Dual-ISM (NMF): the proposed method using NMF low-dimensional transformation

#### 4.2. Comparative Experiment of Similar Image Search Accuracy—Subsequence Matching

#### 4.3. Comparison of Similar Image Search Results

#### 4.4. Comparison of Similar Image Search Speeds

## 5. Conclusions

## Author Contributions

## Funding

## Data Availability Statement

## Conflicts of Interest

## References

- Lu, J.; Liong, V.E.; Zhou, J. Deep Hashing for Scalable Image Search. IEEE Trans. Image Process.
**2017**, 26, 2352–2367. [Google Scholar] [CrossRef] [PubMed] - Choi, D.; Park, S.; Dong, K.; Wee, J.; Lee, H.; Lim, J.; Bok, K.; Yoo, J. An efficient distributed in-memory high-dimensional indexing scheme for content-based retrieval in spark environments. J. KIISE
**2020**, 47, 95–108. [Google Scholar] [CrossRef] - Shao, Y.; Mao, J.; Liu, Y.; Zhang, M.; Ma, S. Towards context-aware evaluation for image search. In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, Paris, France, 21–25 July 2019; pp. 1209–1212. [Google Scholar]
- Deng, C.; Yang, E.; Liu, T.; Li, J.; Liu, W.; Tao, D. Unsupervised Semantic-Preserving Adversarial Hashing for Image Search. IEEE Trans. Image Process.
**2019**, 28, 4032–4044. [Google Scholar] [CrossRef] [PubMed] - Krizhevsky, A.; Sutskever, I.; Hinton, G.E. ImageNet classification with deep convolutional neural networks. Adv. Neural Inf. Process. Syst.
**2012**, 25, 1097–1105. [Google Scholar] [CrossRef] - Girshick, R.; Donahue, J.; Darrell, T.; Malik, J. Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation. In Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA, 23–28 June 2014; pp. 580–587. [Google Scholar] [CrossRef][Green Version]
- Ren, S.; He, K.; Girshick, R.; Sun, J. Faster r-cnn: Towards real-time object detection with region proposal networks. Adv. Neural Inf. Process. Syst.
**2015**, 28, 91–99. [Google Scholar] [CrossRef] [PubMed][Green Version] - Zhou, T.; Li, J.; Wang, S.; Tao, R.; Shen, J. MATNet: Motion-Attentive Transition Network for Zero-Shot Video Object Segmentation. IEEE Trans. Image Process.
**2020**, 29, 8326–8338. [Google Scholar] [CrossRef] [PubMed] - Faloutsos, C.; Ranganathan, M.; Manolopoulos, Y. Fast subsequence matching in time-series databases. ACM SIGMOD Rec.
**1994**, 23, 419–429. [Google Scholar] [CrossRef][Green Version] - Moon, Y.S.; Whang, K.Y.; Loh, W.K. Duality-based subsequence matching in time-series databases. In Proceedings of the 17th International Conference on Data Engineering, Heidelberg, Germany, 2–6 April 2001; pp. 263–272. [Google Scholar]
- Ihm, S.-Y.; Nasridinov, A.; Lee, J.-H.; Park, Y.-H. Efficient duality-based subsequent matching on time-series data in green computing. J. Supercomput.
**2014**, 69, 1039–1053. [Google Scholar] [CrossRef] - Alghamdi, N.; Zhang, L.; Zhang, H.; Rundensteiner, E.A.; Eltabakh, M.Y. ChainLink: Indexing Big Time Series Data for Long Subsequence Matching. In Proceedings of the 2020 IEEE 36th International Conference on Data Engineering (ICDE), Dallas, TX, USA, 20–24 April 2020; pp. 529–540. [Google Scholar] [CrossRef]
- Feng, K.; Wang, P.; Wu, J.; Wang, W. L-Match: A Lightweight and Effective Subsequence Matching Approach. IEEE Access
**2020**, 8, 71572–71583. [Google Scholar] [CrossRef] - Wu, J.; Wang, P.; Pan, N.; Wang, C.; Wang, W.; Wang, J. KV-Match: A Subsequence Matching Approach Supporting Normalization and Time Warping. In Proceedings of the 2019 IEEE 35th International Conference on Data Engineering (ICDE), Macao, China, 8–11 April 2019; pp. 866–877. [Google Scholar] [CrossRef][Green Version]
- Mueen, A.; Zhu, Y.; Yeh, M.; Kamgar, K.; Viswanathan, K.; Gupta, C.K.; Keogh, E. The Fastest Similarity Search Algorithm for Time Series Subsequences under EUCLIDEAN Distance. 2015. Available online: http://www.cs.unm.edu/~mueen/FastestSimilaritySearch.html (accessed on 30 November 2021).
- Ye, F.; Shi, Z.; Shi, Z. A comparative study of PCA, LDA and Kernel LDA for image classification. In Proceedings of the International Symposium on Ubiquitous Virtual Reality, Guangju, Korea, 8–11 July 2009; pp. 51–54. [Google Scholar]
- Belhumeur, P.; Hespanha, J.; Kriegman, D. Eigenfaces vs. Fisherfaces: Recognition using class specific linear projection. IEEE Trans. Pattern Anal. Mach. Intell.
**1997**, 19, 711–720. [Google Scholar] [CrossRef][Green Version] - Jafri, R.; Arabnia, H.R. A Survey of Face Recognition Techniques. J. Inf. Process. Syst.
**2009**, 5, 41–68. [Google Scholar] [CrossRef][Green Version] - Karg, M.; Jenke, R.; Seiberl, W.; Kühnlenz, K.; Schwirtz, A.; Buss, M. A comparison of PCA, KPCA and LDA for feature extraction to recognize affect in gait kinematics. In Proceedings of the 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops, Amsterdam, The Netherlands, 10–12 September 2009; pp. 1–6. [Google Scholar]
- Lee, D.D.; Seung, H.S. Learning the parts of objects by non-negative matrix factorization. Nature
**1999**, 401, 788–791. [Google Scholar] [CrossRef] [PubMed] - Cai, D.; He, X.; Wu, X.; Han, J. Non-negative Matrix Factorization on Manifold. In Proceedings of the 2008 Eighth IEEE International Conference on Data Mining, Pisa, Italy, 15–19 December 2008; pp. 63–72. [Google Scholar] [CrossRef][Green Version]
- Qian, Y.; Xiong, F.; Zeng, S.; Zhou, J.; Tang, Y.Y. Matrix-Vector Nonnegative Tensor Factorization for Blind Unmixing of Hyperspectral Imagery. IEEE Trans. Geosci. Remote Sens.
**2017**, 55, 1776–1792. [Google Scholar] [CrossRef][Green Version] - Alcantarilla, P.F.; Bartoli, A.; Davison, A.J. KAZE features. In Proceedings of the European Conference on Computer Cision, Florence, Italy, 7–13 October 2012; pp. 214–227. [Google Scholar]
- Tareen, S.A.K.; Saleem, Z. A comparative analysis of SIFT, SURF, KAZE, AKAZE, ORB, and BRISK. In Proceedings of the 2018 International Conference on Computing, Mathematics and Engineering Technologies (iCoMET), Sukkur, Pakistan, 3–4 March 2018; pp. 1–10. [Google Scholar]
- Ma, X.; Xie, Q.; Kong, X. Improving KAZE feature matching algorithm with alternative image gray method. In Proceedings of the 2nd International Conference on Computer Science and Application Engineering, Hohhot, China, 22–24 October 2018; pp. 1–5. [Google Scholar]
- Demchev, D.; Volkov, V.; Kazakov, E.; Alcantarilla, P.F.; Sandven, S.; Khmeleva, V. Sea Ice Drift Tracking from Sequential SAR Images Using Accelerated-KAZE Features. IEEE Trans. Geosci. Remote Sens.
**2017**, 55, 5174–5184. [Google Scholar] [CrossRef] - Nilsback, M.-E.; Zisserman, A. A Visual Vocabulary for Flower Classification. In Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’06), New York, NY, USA, 17–22 June 2006; pp. 1447–1454. [Google Scholar] [CrossRef]
- Russakovsky, O.; Deng, J.; Su, H.; Krause, J.; Satheesh, S.; Ma, S.; Huang, Z.; Karpathy, A.; Khosla, A.; Bernstein, M.; et al. Imagenet large scale visual recognition challenge. Int. J. Comput. Vis.
**2015**, 115, 211–252. [Google Scholar] [CrossRef][Green Version]

**Figure 2.**Feature point extraction results using the KAZE algorithm. (

**a**) Flowers-17 (Daffodil); (

**b**) ILSVRC2012 (Pineapple).

**Figure 3.**Example of an image feature point vector. (

**a**) Flowers-10 (Daffodil); (

**b**) ILSVRC (pineapple).

**Figure 4.**Comparison of similar image search accuracy. (

**a**) Accuracy comparison for the Flowers dataset and (

**b**) the accuracy comparison for the ILSVRC dataset.

**Figure 5.**Comparison of similar image search accuracy. (

**a**) Accuracy comparison for the Flowers dataset and (

**b**) the accuracy comparison for the ILSVRC dataset.

Class | Component1 | Component2 | … | Component8 | Actual Distance | Rank Score |
---|---|---|---|---|---|---|

0 | 0.4148 | 0.8275 | … | 0.7073 | 0.95332 | 30 |

6 | 1.1050 | −0.3521 | … | 0.9043 | 1.87587 | |

0 | −0.5677 | 1.0200 | … | −0.0351 | 2.07946 | 28 |

6 | 0.5831 | 0.0016 | … | 1.0581 | 2.11729 | |

3 | 0.4737 | 0.8336 | … | −0.2374 | 2.21694 | |

6 | −0.6978 | 1.1347 | … | 0.5265 | 2.22265 | |

6 | −0.4595 | 0.3928 | … | 0.6221 | 2.24890 | |

3 | 0.2888 | 0.7531 | … | −0.3324 | 2.29725 | |

… | … | … | … | … | … | … |

0 | 0.1168 | 0.9978 | … | 0.1012 | 2.57962 | 1 |

Class | Component1 | Component2 | … | Component128 | Actual Distance | Rank Score |
---|---|---|---|---|---|---|

0 | 0.0010 | −0.0074 | … | 0.0590 | 1.31587 | 30 |

0 | 0.0012 | 0.0016 | … | 0.0609 | 1.36660 | 29 |

0 | 0.0008 | 0.0036 | … | 0.0655 | 1.41247 | 28 |

0 | −0.0096 | −0.0198 | … | 0.1271 | 1.47892 | 27 |

0 | −0.0226 | 0.0163 | … | 0.0195 | 1.52239 | 26 |

0 | −0.0475 | 0.0212 | … | 0.0300 | 1.57971 | 25 |

0 | 0.0456 | 0.0255 | … | 0.1115 | 1.60463 | 24 |

0 | 0.0411 | 0.0069 | … | 0.0980 | 1.611103 | 23 |

… | … | … | … | … | … | … |

9 | −0.0002 | 0.0004 | … | 0.0129 | 9.13522 |

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Lee, H.-J.; Kwon, Y.; Ihm, S.-Y. Dual-ISM: Duality-Based Image Sequence Matching for Similar Image Search. *Appl. Sci.* **2022**, *12*, 1609.
https://doi.org/10.3390/app12031609

**AMA Style**

Lee H-J, Kwon Y, Ihm S-Y. Dual-ISM: Duality-Based Image Sequence Matching for Similar Image Search. *Applied Sciences*. 2022; 12(3):1609.
https://doi.org/10.3390/app12031609

**Chicago/Turabian Style**

Lee, Hye-Jin, Yongjin Kwon, and Sun-Young Ihm. 2022. "Dual-ISM: Duality-Based Image Sequence Matching for Similar Image Search" *Applied Sciences* 12, no. 3: 1609.
https://doi.org/10.3390/app12031609