Robust Long-Term Visual Object Tracking via Low-Rank Sparse Learning for Re-Detection
Abstract
:1. Introduction
2. Related Work
2.1. DCF-Based Trackers
2.2. Long-Term Trackers
3. Tracking Algorithm Design
3.1. Tracking Module
3.2. Re-Detection Module
3.3. Reliability Estimation
Algorithm 1: Our proposed tracking approach |
|
4. Experiment
4.1. Experimental Setup
4.2. Results and Analysis
4.2.1. OTB-2013 Benchmark
- (1)
- Comparison with the baseline
- (2)
- Integration into Different DCF Trackers
- (3)
- Comparison with the state-of-the-art trackers
- (4)
- Qualitative Analysis
- (a)
- Out of view
- (b)
- Occlusion
- (c)
- Scale variation
- (d)
- Out-of-plane rotation
- (e)
- Low resolution
4.2.2. OTB-2015 Benchmark
4.2.3. Temple Color-128 Benchmark
4.2.4. Speed Analysis
5. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- Smeulders, A.W.; Chu, D.M.; Cucchiara, R.; Calderara, S.; Dehghan, A.; Shah, M. Visual tracking: An experimental survey. IEEE Trans. Pattern Anal. Mach. Intell. 2013, 36, 1442–1468. [Google Scholar]
- Lu, H.; Li, P.; Wang, D. Visual object tracking: A survey. Pattern Recognit. Artif. Intell. 2018, 31, 61–76. [Google Scholar]
- Kim, I.S.; Choi, H.S.; Yi, K.M.; Choi, J.Y.; Kong, S.G. Intelligent Visual Surveillance—A Survey. Int. J. Control. Autom. Syst. 2010, 8, 926–939. [Google Scholar] [CrossRef]
- Reddy, K.R.; Priya, K.H.; Neelima, N. Object Detection and Tracking—A Survey. In Proceedings of the 2015 International Conference on Computational Intelligence and Communication Networks (CICN), Jabalpur, India, 12–14 December 2015; pp. 418–421. [Google Scholar]
- Deori, B.; Thounaojam, D.M. A Survey On Moving Object Tracking in Video. Int. J. Inf. Theory 2014, 3, 31–46. [Google Scholar] [CrossRef]
- Pan, Z.; Liu, S.; Fu, W. A Review of Visual Moving Target Tracking. Multimed. Tools Appl. 2017, 76, 16989–17018. [Google Scholar] [CrossRef]
- Baker, S.; Matthews, I. Lucas-Kanade 20 Years on: A Unifying Framework. Int. J. Comput. Vis. 2004, 56, 221–255. [Google Scholar] [CrossRef]
- Oron, S.; Bar-Hillel, A.; Levi, D.; Avidan, S. Locally Orderless Tracking. Int. J. Comput. Vis. 2015, 111, 213–228. [Google Scholar] [CrossRef]
- Ross, D.A.; Lim, J.; Lin, R.S.; Yang, M.H. Incremental Learning for Robust Visual Tracking. Int. J. Comput. Vis. 2008, 77, 125–141. [Google Scholar] [CrossRef]
- Danelljan, M.; Shahbaz Khan, F.; Felsberg, M.; Van de Weijer, J. Adaptive Color Attributes for Real-time Visual Tracking. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA, 24–27 June 2014; pp. 1090–1097. [Google Scholar]
- Bolme, D.S.; Beveridge, J.R.; Draper, B.A.; Lui, Y.M. Visual Object Tracking Using Adaptive Correlation Filters. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, San Francisco, CA, USA, 13–18 June 2010; pp. 2544–2550. [Google Scholar]
- Henriques, J.F.; Caseiro, R.; Martins, P.; Batista, J. High-speed tracking with kernelized correlation filters. IIEEE Trans. Pattern Anal. Mach. Intell. 2014, 37, 583–596. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Danelljan, M.; Häger, G.; Khan, F.S.; Felsberg, M. Discriminative Scale Space Tracking. IEEE Trans. Ppattern Anal. Mach. Intell. 2016, 39, 1561–1575. [Google Scholar] [CrossRef] [Green Version]
- Li, Y.; Zhu, J. A Scale Adaptive Kernel Correlation Filter Tracker with Feature Integration. In Proceedings of the European Conference on Computer Vision, Zurich, Switzerland, 5–12 September 2014; pp. 254–265. [Google Scholar]
- Bertinetto, L.; Valmadre, J.; Golodetz, S.; Miksik, O.; Torr, P.H. Staple: Complementary Learners for Real-time Tracking. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 26 June–1 July 2016; pp. 1401–1409. [Google Scholar]
- Danelljan, M.; Hager, G.; Shahbaz, K.F.; Felsberg, M. Convolutional Features for Correlation Filter based Visual Tracking. In Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile, 11–18 December 2015; pp. 58–66. [Google Scholar]
- Danelljan, M.; Hager, G.; Shahbaz, K.F.; Felsberg, M. Learning Spatially Regularized Correlation Filters for Visual Tracking. In Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile, 11–18 December 2015; pp. 4310–4318. [Google Scholar]
- Danelljan, M.; Bhat, G.; Shahbaz Khan, F.; Felsberg, M. Eco: Efficient Convolution Operators for Tracking. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, 22–25 July 2017; pp. 6638–6646. [Google Scholar]
- Danelljan, M.; Robinson, A.; Khan, F.S.; Felsberg, M. Beyond Correlation Filters: Learning Continuous Convolution Operators for Visual Tracking. In Proceedings of the European Conference on Computer Vision, Amsterdam, The Netherlands, 8–16 October 2016; pp. 472–488. [Google Scholar]
- Dai, K.; Wang, D.; Lu, H.; Sun, C.; Li, J. Visual Tracking via Adaptive Spatially-regularized Correlation Filters. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Beach, CA, USA, 16–20 June 2019; pp. 4670–4679. [Google Scholar]
- Li, F.; Tian, C.; Zuo, W.; Zhang, L.; Yang, M.H. Learning Spatial-temporal Regularized Correlation Filters for Visual Tracking. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 19–21 June 2018; pp. 4904–4913. [Google Scholar]
- Xu, T.; Feng, Z.H.; Wu, X.J.; Kittler, J. Learning Adaptive Discriminative Correlation Filters via Temporal Consistency Preserving Spatial Feature Selection for Robust Visual Object Tracking. IEEE Trans. Image Process. 2019, 28, 5596–5609. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Kiani Galoogahi, H.; Fagg, A.; Lucey, S. Learning Background-aware Correlation Filters for Visual Tracking. In Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy, 19–22 October 2017; pp. 1135–1143. [Google Scholar]
- Luo, S.; Li, B.; Yuan, X. An Anti-Drift Background-aware Correlation Filter for Visual Tracking in Complex Scenes. IEEE Access 2019, 7, 185857–185867. [Google Scholar] [CrossRef]
- Xue, X.; Li, Y.; Shen, Q. Unmanned Aerial Vehicle Object Tracking by Correlation Filter with Adaptive Appearance Model. Sensors 2018, 18, 2751. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Yang, Y.; Zhang, Y.; Li, D.; Wang, Z. Parallel Correlation Filters for Real-time Visual Tracking. Sensors 2019, 19, 2362. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Shin, J.; Kim, H.; Kim, D.; Paik, J. Fast and Robust Object Tracking Using Tracking Failure Detection in Kernelized Correlation Filter. Appl. Sci. 2020, 10, 713. [Google Scholar] [CrossRef] [Green Version]
- He, W.; Li, H.; Liu, W.; Li, C.; Guo, B. rStaple: A Robust Complementary Learning Method for Real-Time Object Tracking. Appl. Sci. 2020, 10, 3021. [Google Scholar] [CrossRef]
- Wang, W.; Liu, C.; Xu, B.; Li, L.; Chen, W.; Tian, Y. Robust Visual Tracking Based on Fusional Multi-Correlation-Filters with a High-Confidence Judgement Mechanism. Appl. Sci. 2020, 10, 2151. [Google Scholar] [CrossRef] [Green Version]
- Wu, Y.; Lim, J.; Yang, M.H. Online Object Tracking: A benchmark. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Portland, OR, USA, 25–27 June 2013; pp. 2411–2418. [Google Scholar]
- Liang, P.; Blasch, E.; Ling, H. Encoding Color Information for Visual Tracking: Algorithms and Benchmark. IEEE Trans. Image Process. 2015, 24, 5630–5644. [Google Scholar] [CrossRef] [PubMed]
- Kristan, M.; Leonardis, A.; Matas, J.; Felsberg, M.; Pflugfelder, R.; Čehovin Zajc, L.; Vojir, T.; Bhat, G.; Lukezic, A.; Eldesokey, A.; et al. The sixth Visual Object Tracking VOT2018 Challenge Results. In Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany, 8–14 September 2018. [Google Scholar] [CrossRef] [Green Version]
- Ma, C.; Yang, X.; Zhang, C.; Yang, M.H. Long-term Correlation Tracking. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA, 8–10 June 2015; pp. 5388–5396. [Google Scholar]
- Lebeda, K.; Hadfield, S.; Matas, J.; Bowden, R. Long-term Tracking through Failure Cases. In Proceedings of the IEEE International Conference on Computer Vision Workshops, Sydney, Australia, 1–8 December 2013; pp. 153–160. [Google Scholar]
- Hong, Z.; Chen, Z.; Wang, C.; Mei, X.; Prokhorov, D.; Tao, D. Multi-store Tracker (muster): A Cognitive Psychology Inspired Approach to Object Tracking. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA, 8–10 June 2015; pp. 749–758. [Google Scholar]
- Wang, N.; Zhou, W.; Li, H. Reliable Re-detection for Long-term Tracking. IEEE Trans. Circuits Syst. Video Technol. 2018, 29, 730–743. [Google Scholar] [CrossRef]
- Chen, Z.; Liu, P.; Du, Y.; Luo, Y.; Guo, J. Long-term Correlation Tracking via Spatial-temporal Context. Vis. Comput. 2020, 36, 425–442. [Google Scholar] [CrossRef]
- Tang, M.; Feng, J. Multi-kernel Correlation Filter for Visual Tracking. In Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile, 11–18 December 2015; pp. 3038–3046. [Google Scholar]
- Lukezic, A.; Vojir, T.; Čehovin Zajc, L.; Matas, J.; Kristan, M. Discriminative Correlation Filter with Channel and Spatial Reliability. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, 21–26 July 2017; pp. 6309–6318. [Google Scholar]
- Mueller, M.; Smith, N.; Ghanem, B. Context-aware Correlation Filter Tracking. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, 21–26 July 2017; pp. 1396–1404. [Google Scholar]
- Lukežič, A.; Zajc, L.Č.; Kristan, M. Deformable Parts Correlation Filters for Robust Visual Tracking. IEEE Trans. Cybern. 2017, 48, 1849–1861. [Google Scholar]
- Kalal, Z.; Mikolajczyk, K.; Matas, J. Tracking-learning-detection. IEEE Trans. Pattern Anal. Mach. Intell. 2011, 34, 1409–1422. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Felzenszwalb, P.F.; Girshick, R.B.; McAllester, D.; Ramanan, D. Object Detection with Discriminatively Trained Part-based Models. IEEE Trans. Pattern Anal. Mach. Intell. 2009, 32, 1627–1645. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Huang, B.; Xu, T.; Liu, B.; Yuan, B. Context Constraint and Pattern Memory for Long-term Correlation Tracking. Neurocomputing 2020, 377, 1–15. [Google Scholar] [CrossRef]
- Baisa, N.L.; Bhowmik, D.; Wallace, A. Long-term Correlation Tracking using Multi-layer Hybrid Features in Sparse and Dense Environments. J. Vis. Commun. Image Represent. 2018, 55, 464–476. [Google Scholar] [CrossRef] [Green Version]
- Dalal, N.; Triggs, B.; Schmid, C. Human Detection Using Oriented Histograms of Flow and Appearance. In Proceedings of the European Conference on Computer Vision, Graz, Austria, 7–13 May 2006; pp. 428–441. [Google Scholar]
- Chunyu, Y.; Jun, F.; Jinjun, W.; Yongming, Z. Video Fire Smoke Detection Using Motion and Color Features. Fire Technol. 2010, 46, 651–663. [Google Scholar] [CrossRef]
- Tu, F.; Ge, S.S.; Tang, Y.; Hang, C.C. Robust Visual Tracking via Collaborative Motion and Appearance Model. IEEE Trans. Ind. Inform. 2017, 13, 2251–2259. [Google Scholar] [CrossRef]
- Wang, M.; Liu, Y.; Huang, Z. Large Margin Object Tracking with Circulant Feature Maps. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, 21–26 July 2017; pp. 4021–4029. [Google Scholar]
- Zeng, H.; Peng, N.; Yu, Z.; Gu, Z.; Liu, H.; Zhang, K. Visual Tracking using Multi-channel Correlation Filters. In Proceedings of the 2015 IEEE International Conference on Digital Signal Processing (DSP), Singapore, 21–24 July 2015; pp. 211–214. [Google Scholar]
- Horn, B.K.; Schunck, B.G. Determining Optical Flow. Tech. Appl. Image Underst. 1981, 281, 319–331. [Google Scholar] [CrossRef] [Green Version]
- Lin, Z.; Liu, R.; Su, Z. Linearized Alternating Direction Method with Adaptive Penalty for Low-rank Representation. arXiv 2011, preprint. arXiv:1109.0367. [Google Scholar]
- Du, H.; Hu, Q.; Qiao, D.; Pitas, I. Robust Face Recognition via Low-rank Sparse Representation-based Classification. Int. J. Autom. Comput. 2015, 12, 579–587. [Google Scholar] [CrossRef]
- Ding, Y.; Chong, Y.; Pan, S. Sparse and Low-rank Representation with Key Connectivity for Hyperspectral Image Classification. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2020, 13, 5609–5622. [Google Scholar] [CrossRef]
- Liu, J.; Ji, S.; Ye, J. Multi-task Feature Learning via Efficient L2, 1-norm Minimization. arXiv 2012, preprint. arXiv:1205.2631. [Google Scholar]
- Zitnick, C.L.; Dollár, P. Edge boxes: Locating Object Proposals from Edges. In Proceedings of the European Conference on Computer Vision, Zurich, Switzerland, 6–12 September 2014; pp. 391–405. [Google Scholar]
BC | MB | DEF | IV | IPR | LR | OCC | OPR | OV | SV | FM | |
---|---|---|---|---|---|---|---|---|---|---|---|
Ours | 0.796 | 0.732 | 0.871 | 0.766 | 0.761 | 0.628 | 0.848 | 0.815 | 0.865 | 0.775 | 0.723 |
Baseline (Staple) | 0.739 | 0.512 | 0.693 | 0.643 | 0.667 | 0.508 | 0.701 | 0.685 | 0.659 | 0.658 | 0.576 |
Ours | ECOhc | STAPLE | SRDCF | BACF | SAMF_CA | CRS_DCF | KCF | SAMF | DSST | TLD | LCT | |
---|---|---|---|---|---|---|---|---|---|---|---|---|
Avg.FPS (CPU) | 19.1 | 37.6 | 67.7 | 12.8 | 41.6 | 27.7 | 17.8 | 315.1 | 29.4 | 63.9 | 60.6 | 33.5 |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Luo, S.; Li, B.; Yuan, X.; Liu, H. Robust Long-Term Visual Object Tracking via Low-Rank Sparse Learning for Re-Detection. Appl. Sci. 2021, 11, 1963. https://doi.org/10.3390/app11041963
Luo S, Li B, Yuan X, Liu H. Robust Long-Term Visual Object Tracking via Low-Rank Sparse Learning for Re-Detection. Applied Sciences. 2021; 11(4):1963. https://doi.org/10.3390/app11041963
Chicago/Turabian StyleLuo, Shanshan, Baoqing Li, Xiaobing Yuan, and Huawei Liu. 2021. "Robust Long-Term Visual Object Tracking via Low-Rank Sparse Learning for Re-Detection" Applied Sciences 11, no. 4: 1963. https://doi.org/10.3390/app11041963
APA StyleLuo, S., Li, B., Yuan, X., & Liu, H. (2021). Robust Long-Term Visual Object Tracking via Low-Rank Sparse Learning for Re-Detection. Applied Sciences, 11(4), 1963. https://doi.org/10.3390/app11041963