Adaptive Fractional-Order Multi-Scale Optimization TV-L1 Optical Flow Algorithm
Abstract
:1. Introduction
- •
- The TV-L1 optical flow objective function is discretized by the fractional-order differential operator, which makes the model pay extra attention to the local texture-scale optical flow features, increases the participation of effective optical flow, and improves the robustness of the model in multi-scale computation.
- •
- The fractional order is regarded as one of the objects of the optimization problem, and the Ant Lion algorithm is used to realize the local sampling search, and the global smoothness advantage of the TV-L1 optical flow model is retained. The multi-scale objective function is optimized by multi-parameter optimization of the smoothness term weight and fractional order.
- •
- The significant performance of our proposed model is verified on the MPI_Sintel and Middleburry datasets.
2. Materials and Methods
2.1. Construction Process of TV-L1 Optical Flow Model
2.2. Subsection
2.3. Iterative Calculation of Fractional TV-L1 Optical Flow Based on Ant Lion Optimization
- The initial values of pixels and optical flow vectors are initialized in the two populations, and the pheromone matrix of the Ants is established to record the pheromone concentration between each pixel. The search space of the Ants is defined as follows (29):
- 2
- The second step is to randomly initialize the positions of Ants and Ant Lions on the solution space, calculate the fitness value of each pixel, and take the pixel corresponding to the lowest value as the Ant Lion.
- 3
- For each Ant, a roulette wheel is used to select an Ant Lion as the target. Concurrently, the Ant will randomly move around the chosen Ant Lion as the best solution, and update its position and speed, calculate the position and speed of the next moment and the pheromone increment of the current position, and update the pheromone, that is, the path that the Ant has passed and the local optimal pixel solution.
- 4
- When the fitness value of an Ant exceeds the optimal value from the previous iteration, the position is then updated to match the position of that particular Ant. In the selection and update of the Ant Lion, assuming that the random walk of goes to and the random walk of goes to , the movement of the individual Ant around the Ant Lion trap domain is as follows (34):
- 5
- The fifth step is to determine whether the convergence condition is satisfied, output the fractional order , penalty parameter , and solution vector corresponding to the current optimal solution, and stop the iteration, or otherwise return to step 3.
3. Experiments and Analysis
3.1. Ablation Experiment
3.2. Comparative Experiments
4. Discussion
5. Conclusions
Author Contributions
Funding
Data Availability Statement
Conflicts of Interest
References
- Bolme, D.; Beveridge, J.R.; Draper, B.A.; Lui, Y.M. Visual object tracking using adaptive correlation filters. In Proceedings of the 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Francisco, CA, USA, 13–18 June 2010; pp. 2544–2550. [Google Scholar] [CrossRef]
- Zhang, J.-X.; Yang, T.; Chai, T. Neural Network Control of Underactuated Surface Vehicles With Prescribed Trajectory Tracking Performance. IEEE Trans. Neural Netw. Learn. Syst. 2023, 1–14. [Google Scholar] [CrossRef] [PubMed]
- Zhang, J.-X.; Xu, K.-D.; Wang, Q.-G. Prescribed performance tracking control of time-delay nonlinear systems with output constraints. IEEE/CAA J. Autom. Sin. 2023, 67, 6904–6911. [Google Scholar]
- Wang, G.; Luo, C.; Sun, X.; Xiong, Z.; Zeng, W. Tracking by Instance Detection: A Meta-Learning Approach. In Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA, 13–19 June 2020; pp. 6288–6297. [Google Scholar] [CrossRef]
- Danelljan, M.; Bhat, G.; Khan, F.S.; Felsberg, M. ECO: Efficient Convolution Operators for Tracking. In Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, 21–26 July 2017; pp. 6638–6646. [Google Scholar] [CrossRef]
- Wei, H.; Qian, Z.; Bo, Q.; Yang, Y. Research on HS Optical Flow Algorithm Based on Motion Estimation Optimization. J. Comput. Commun. 2018, 6, 171–184. [Google Scholar] [CrossRef]
- Brox, T.; Bruhn, A.; Papenberg, N.; Weickert, J. High Accuracy Optical Flow Estimation Based on a Theory for Warping. In Proceedings of the European Conference on Computer Vision, Prague, Czech Republic, 11–14 May 2004; pp. 25–36. [Google Scholar] [CrossRef]
- Zhang, X.; Boutat, D.; Liu, D. Applications of Fractional Operator in Image Processing and Stability of Control Systems. Fractal Fract. 2023, 7, 359. [Google Scholar] [CrossRef]
- Zhang, J.-X.; Chai, T. Proportional-integral funnel control of unknown lower-triangular nonlinear systems. IEEE Trans. Autom. Control 2023, 69, 1921–1927. [Google Scholar] [CrossRef]
- Tu, Z.; Li, H.; Xie, W.; Liu, Y.; Zhang, S.; Li, B.; Yuan, J. Optical flow forvideo super-resolution: A survey. Artif. Intell. Rev. 2022, 55, 6505–6546. [Google Scholar] [CrossRef]
- Zhang, J.-X.; Yang, G.-H. Fault-Tolerant Output-Constrained Control of Unknown Euler–Lagrange Systems with Prescribed Tracking Accuracy. Automatica 2020, 111, 108606. [Google Scholar] [CrossRef]
- Zhang, X.; Lin, C.; Chen, Y.Q.; Boutat, D. A Unified Framework of Stability Theorems for LTI Fractional Order Systems with 0 < alpha < 2. IEEE Trans. Circuits Syst. II Express Briefs 2020, 67, 3237–3241. [Google Scholar] [CrossRef]
- Zhang, X.; Chen, S.; Zhang, J.-X. Adaptive Sliding Mode Consensus Control Based on Neural Network for Singular Fractional Order Multi-Agent Systems. Appl. Math. Comput. 2022, 434, 127442. [Google Scholar] [CrossRef]
- Yang, Q.; Chen, D.; Zhao, T.; Chen, Y. Fractional calculus in image processing: A review. Fract. Calc. Appl. Anal. 2016, 19, 1222–1249. [Google Scholar] [CrossRef]
- Zhang, J.-X.; Wang, Q.-G.; Ding, W. Global Output-Feedback Prescribed Performance Control of Nonlinear Systems with Unknown Virtual Control Coefficients. IEEE Trans. Autom. Control. 2022, 67, 6904–6911. [Google Scholar] [CrossRef]
- Zhang, G.-M.; Hu, Q.; Guo, L.-J. Medical image non-rigid registration based on adaptive fractional order. Acta Autom. Sin. 2020, 46, 1941–1951. [Google Scholar] [CrossRef]
- Wang, Y. Research and Application of Antlion Optimization Algorithm Based on Distance; Hunan University: Changsha, China, 2017. [Google Scholar]
- Kumar, P.; Kumar, S.; Raman, B. A fractional order variational model for the robust estimation of optical flow from image sequences. Optik 2016, 127, 8710–8727. [Google Scholar] [CrossRef]
- Khan, M.; Kumar, P. A nonlinear modeling of fractional order based variational model in optical flow estimation. Optik 2022, 261, 169136. [Google Scholar] [CrossRef]
- Kumar, P. A duality based approach for fractional order Tv-model in optical flow estimation. In Proceedings of the 2020 5th International Conference on Computing, Communication and Security (ICCCS), Patna, India, 14–16 October 2020; pp. 1–5. [Google Scholar]
Parameter | Description | Default Value |
---|---|---|
Time step | 0.25 | |
Threshold | 0.01 | |
Zoom factor | 0.5 | |
Number of scales | 5 | |
Number of warps | 5 |
Parameter Selection | MPI_Sintel Sequence () | ||||||||
---|---|---|---|---|---|---|---|---|---|
Ambush2 | Ambush6 | Shamen2 | Bandage1 | Market2 | Temple3 | Sleeping2 | Cave4 | ||
AEPE | 44.30 | 40.70 | 1.45 | 6.35 | 5.22 | 25.79 | 4.86 | 13.05 | |
44.26 | 39.78 | 1.40 | 6.30 | 4.86 | 16.99 | 7.15 | 13.10 | ||
44.17 | 39.65 | 1.35 | 6.29 | 4.63 | 12.73 | 11.76 | 13.37 | ||
44.29 | 39.64 | 1.40 | 6.27 | 4.21 | 11.89 | 4.75 | 13.04 | ||
44.78 | 40.72 | 1.66 | 6.45 | 4.10 | 14.38 | 5.22 | 13.05 | ||
45.90 | 44.48 | 2.11 | 6.66 | 4.75 | 19.17 | 6.12 | 13.09 | ||
50.91 | 49.45 | 2.53 | 7.00 | 5.21 | 32.52 | 7.86 | 13.67 | ||
AAE | 1.54 | 1.51 | 1.48 | 1.29 | 1.70 | 1.49 | 1.66 | 1.53 | |
1.61 | 1.42 | 1.69 | 1.42 | 1.79 | 1.70 | 1.68 | 1.51 | ||
1.66 | 1.83 | 1.81 | 1.40 | 1.89 | 1.69 | 1.69 | 1.50 | ||
1.57 | 1.8 | 1.50 | 1.33 | 2.29 | 1.44 | 1.77 | 1.62 | ||
1.54 | 1.42 | 1.40 | 1.29 | 1.88 | 1.45 | 1.88 | 1.63 | ||
1.61 | 1.74 | 1.43 | 1.41 | 1.66 | 1.44 | 1.84 | 1.63 | ||
1.60 | 1.92 | 1.53 | 1.30 | 1.85 | 1.45 | 1.67 | 1.63 |
Parameter Selection | MiddleBury Sequence () | ||||||||
---|---|---|---|---|---|---|---|---|---|
Teddy | Urban | Woden | Mequn | Army | Scheflera | Grove | Yosemite | ||
AEPE | 0.68 | 0.69 | 0.15 | 0.21 | 0.13 | 0.26 | 0.50 | 0.10 | |
0.65 | 0.63 | 0.13 | 0.19 | 0.11 | 0.25 | 0.54 | 0.10 | ||
0.65 | 0.56 | 0.15 | 0.18 | 0.12 | 0.25 | 0.57 | 0.13 | ||
0.56 | 0.43 | 0.11 | 0.17 | 0.09 | 0.29 | 0.54 | 0.15 | ||
0.63 | 0.66 | 0.15 | 0.20 | 0.09 | 0.27 | 0.51 | 0.09 | ||
0.73 | 0.69 | 0.14 | 0.21 | 0.11 | 0.37 | 0.53 | 0.13 | ||
0.79 | 0.90 | 0.20 | 0.24 | 0.14 | 0.42 | 0.58 | 0.18 | ||
AAE | 2.46 | 2.20 | 3.00 | 3.03 | 3.35 | 4.72 | 2.61 | 2.71 | |
2.45 | 2.19 | 2.99 | 3.03 | 3.03 | 4.69 | 2.63 | 2.63 | ||
2.43 | 2.17 | 3.01 | 2.97 | 2.99 | 4.65 | 2.63 | 2.60 | ||
2.43 | 2.16 | 2.94 | 3.01 | 2.98 | 4.65 | 2.60 | 2.62 | ||
2.45 | 2.17 | 3.01 | 3.00 | 3.04 | 4.62 | 2.58 | 2.64 | ||
2.47 | 2.18 | 3.03 | 3.00 | 3.04 | 4.61 | 2.61 | 2.68 | ||
2.66 | 2.30 | 3.33 | 3.44 | 3.07 | 5.11 | 3.05 | 3.01 |
Evaluation Index | Test Sequence—Cave2 | |||||
---|---|---|---|---|---|---|
TVL1 (GD) | TV-L1 (CG) | FO-TVL1 () | FO-TVL1 () | TVL1 (ALO) | FO-TVL1 (ALO) | |
AEPE | 18.40 | 15.44 | 17.58 | 15.33 | 15.94 | 14.58 |
PSNR | 12.94 | 12.93 | 12.97 | 12.85 | 12.94 | 12.99 |
SSIM | 0.0295 | 0.0266 | 0.0249 | 0.0299 | 0.0276 | 0.0340 |
Parameter Selection | Evaluation Index | MPI_Sintel Sequence | |||
---|---|---|---|---|---|
Ambush2 | alley_1 | Market_5 | Cave2 | ||
AEPE | 80.30 | 278.86 | 109.60 | 71.25 | |
PSNR | 11.25 | 13.73 | 10.60 | 22.91 | |
SSIM | 0.0130 | 0.0313 | 0.0062 | 0.1988 | |
AEPE | 80.01 | 247.92 | 101.71 | 65.80 | |
PSNR | 11.24 | 13.75 | 10.62 | 22.87 | |
SSIM | 0.0119 | 0.0280 | 0.0032 | 0.1900 | |
AEPE | 78.98 | 155.15 | 84.36 | 50.29 | |
PSNR | 11.24 | 13.76 | 10.65 | 22.88 | |
SSIM | 0.0127 | 0.0247 | 0.0113 | 0.2147 | |
AEPE | 78.81 | 138.70 | 80.79 | 44.79 | |
PSNR | 11.24 | 13.78 | 10.65 | 22.89 | |
SSIM | 0.0130 | 0.0229 | 0.0114 | 0.211 | |
AEPE | 77.44 | 96.22 | 69.40 | 26.09 | |
PSNR | 11.24 | 13.72 | 10.59 | 22.69 | |
SSIM | 0.0132 | 0.0370 | 0.0010 | 0.2074 | |
AEPE | 77.21 | 71.92 | 64.96 | 18.48 | |
PSNR | 11.24 | 13.69 | 10.55 | 22.61 | |
SSIM | 0.0132 | 0.0332 | 0.0017 | 0.2174 | |
AEPE | 76.53 | 49.79 | 61.50 | 14.33 | |
PSNR | 11.24 | 13.72 | 10.49 | 22.53 | |
SSIM | 0.0150 | 0.0278 | 0.0081 | 0.2165 |
AEPE | Comparative Algorithms | MPI_Sintel Sequence | |||
Ambush | Wall | Market | Cave | ||
TVL1-Flow | 44.62 | 6.51 | 4.11 | 18.23 | |
HS | 43.87 | 8.39 | 3.64 | 19.82 | |
FO-FFlow | 38.69 | 6.45 | 3.64 | 18.05 | |
FOVOFM | 44.15 | 6.55 | 4.23 | 17.71 | |
ADFOVOFM | 42.35 | 6.27 | 3.58 | 16.18 | |
Ours | 41.72 | 6.24 | 3.96 | 15.32 |
AEPE | Comparative Algorithms | MiddleBury Sequence | |||
Mequn | Yosemite | Grove | Teddy | ||
TVL1-Flow | 0.24 | 0.16 | 0.62 | 0.41 | |
HS | 0.18 | 0.19 | 0.49 | 0.32 | |
FO-FFlow | 0.21 | 0.15 | 0.56 | 0.47 | |
FOVOFM | 0.22 | 0.17 | 0.67 | 0.51 | |
ADFOVOFM | 0.19 | 0.13 | 0.61 | 0.42 | |
Ours | 0.17 | 0.11 | 0.57 | 0.39 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Yang, Q.; Wang, Y.; Liu, L.; Zhang, X. Adaptive Fractional-Order Multi-Scale Optimization TV-L1 Optical Flow Algorithm. Fractal Fract. 2024, 8, 179. https://doi.org/10.3390/fractalfract8040179
Yang Q, Wang Y, Liu L, Zhang X. Adaptive Fractional-Order Multi-Scale Optimization TV-L1 Optical Flow Algorithm. Fractal and Fractional. 2024; 8(4):179. https://doi.org/10.3390/fractalfract8040179
Chicago/Turabian StyleYang, Qi, Yilu Wang, Lu Liu, and Xiaomeng Zhang. 2024. "Adaptive Fractional-Order Multi-Scale Optimization TV-L1 Optical Flow Algorithm" Fractal and Fractional 8, no. 4: 179. https://doi.org/10.3390/fractalfract8040179
APA StyleYang, Q., Wang, Y., Liu, L., & Zhang, X. (2024). Adaptive Fractional-Order Multi-Scale Optimization TV-L1 Optical Flow Algorithm. Fractal and Fractional, 8(4), 179. https://doi.org/10.3390/fractalfract8040179