Image Motion Extraction of Structures Using Computer Vision Techniques: A Comparative Study
Abstract
:1. Introduction
2. Computer Vision-Based Motion Extraction Techniques
2.1. Optical Flow with Lucas–Kanade Method
2.2. Digital Image Correlation with Bilinear Interpolation
2.3. In-Plane Motion Magnification
3. Experiment Verification
3.1. Experimental Setup
3.2. Motion Extraction
3.2.1. Optical Flow with the Lucas–Kanade Method
3.2.2. Digital Image Correlation with Bilinear Interpolation
3.2.3. Phase-Based Motion Magnification Using the Riesz Pyramid
3.3. Discussions of Motion Extraction Results
3.4. System Identification
4. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Conflicts of Interest
References
- Brownjohn, J.M. Structural health monitoring of civil infrastructure. Philos. Trans. Ser. A Math. Phys. Eng. Sci. 2007, 365, 589–622. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Li, J.; Hao, H.; Fan, K.; Brownjon, J. Development and application of a relative displacement sensor for structural health monitoring of composite bridges. Struct. Control Health Monit. 2014, 22, 726–742. [Google Scholar] [CrossRef] [Green Version]
- Xu, K.; Mita, A. Absolute displacement-based formulation for peak inter-story drift identification of shear structures using only one accelerometer. Sensors 2021, 21, 3629. [Google Scholar] [CrossRef] [PubMed]
- Feng, M.; Fukuda, Y.; Mizuta, M.; Ozer, E. Citizen sensors for SHM: Use of accelerometers data from smartphones. Sensors 2015, 15, 2980–2998. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Chen, J.G.; Wadhwa, N.; Cha, Y.J.; Durand, F.; Freeman, W.T.; Buyukozturk, O. Modal identification of simple structures with high-speed video using motion magnification. J. Sound Vib. 2015, 345, 58–71. [Google Scholar] [CrossRef]
- Yoon, H.; Elanwar, H.; Choi, H.; Fard, G.; Spencer, B.F. Target-free approach for vision-based structural system identification using consumer-grade cameras. Struct. Control Health Monit. 2016, 23, 1405–1416. [Google Scholar] [CrossRef]
- Singh, A. Optic Flow Computation: A Unified Perspective; IEEE Computer Society Press: Washington, DC, USA, 1991; Volume 3. [Google Scholar]
- Zhu, J.; Lu, Z.; Zhang, C. A marker-free method for structural dynamic displacement measurement based on optical flow. Struct. Infrastruct. Eng. 2020, 1–13. [Google Scholar] [CrossRef]
- Won, J.; Park, J.W.; Park, K.; Yoon, H.; Moon, D.S. Non-target structural displacement measurement using reference frame-based deepflow. Sensors 2019, 19, 2992. [Google Scholar] [CrossRef] [Green Version]
- Dong, C.Z.; Celik, O.; Necati Catbas, F.; O’Brien, E.J.; Taylor, S. Structural displacement monitoring using deep learning-based full field optical flow methods. Struct. Infrastruct. Eng. 2020, 16, 51–71. [Google Scholar] [CrossRef]
- Brox, T.; Bregler, C.; Malik, J. Large displacement optical flow. In Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA, 20–25 June 2009. [Google Scholar]
- Chu, T.C.; Ranson, W.F.; Sutton, M.A.; Peters, W.H. Applications of digital-image-correlation techniques to experimental mechanics. Exp. Mech. 1985, 25, 232–244. [Google Scholar] [CrossRef]
- Keating, M.A.; Wolf, P.R.; Scarpace, F.L. An improved method of digital image correlation. Photogramm. Eng. Remote Sens. 1975, 41, 993–1002. [Google Scholar]
- Zappa, E.; Matinmanesh, A.; Mazzoleni, P. Evaluation and improvement of digital image correlation uncertainty in dynamic conditions. Opt. Lasers Eng. 2014, 59, 82–92. [Google Scholar] [CrossRef]
- Tian, L.; Zhao, J.; Pan, B.; Wang, Z. Full-field bridge deflection monitoring with off-axis digital image correlation. Sensors 2021, 21, 5058. [Google Scholar] [CrossRef]
- Castellini, P.; Chiariotti, P.; Martarelli, M.; Zappa, E.; Lavetelli, A. Experimental modal analysis on vibration data measured by digital image correlation. In Conference Proceedings of the Society for Experimental Mechanics; Springer: Cham, Switzerland, 2017. [Google Scholar]
- Uehara, D.; Sirohi, J. Full-field optical deformation measurement and operational modal analysis of a flexible rotor blade. Mech. Syst. Signal Process. 2019, 133, 106265. [Google Scholar] [CrossRef]
- Molina-Viedma, Á.; López-Alba, E.; Felipe-Sesé, L.; Díaz, F. Full-Field Operational Modal Analysis of an Aircraft Composite Panel from the Dynamic Response in Multi-Impact Test. Sensors 2021, 21, 1602. [Google Scholar] [CrossRef] [PubMed]
- Stoilov, G.; Kavardzhikov, V.; Pashkouleva, D. A comparative study of random patterns for digital image correlation. J. Theor. Appl. Mech. 2012, 42, 55–66. [Google Scholar] [CrossRef] [Green Version]
- Luu, L.; Wang, Z.; Vo, M.; Hoang, T.; Ma, J. Accuracy enhancement of digital image correlation with B-spline interpolation. Opt. Lett. 2011, 36, 3070–3072. [Google Scholar] [CrossRef] [PubMed]
- Zhao, T.B.; Zhang, W.; Gua, W.Y. Digital image correlation analysis of displacement based on corrected three surface fitting algorithm. Hindawi 2019, 2019, 4620858. [Google Scholar] [CrossRef] [Green Version]
- Wadhwa, N.; Rubinstein, M.; Durand, F.; Freeman, W.T. Phase-based video motion processing. ACM Trans. Graph. 2013, 32, 1–10. [Google Scholar] [CrossRef] [Green Version]
- Cha, Y.J.; Chen, J.G.; Büyüköztürk, O. Output-only computer vision based damage detection using phase-based optical flow and unscented Kalman filters. Eng. Struct. 2017, 132, 300–313. [Google Scholar] [CrossRef]
- Harmanci, Y.E.; Gülan, U.; Holzner, M.; Chatzi, E. A novel approach for 3D-structural identification through video recording: Magnified tracking. Sensors 2019, 19, 1229. [Google Scholar] [CrossRef] [Green Version]
- Shang, Z.; Shen, Z. Multi-point vibration measurement and mode magnification of civil T structures using video-based motion processing. Autom. Constr. 2018, 93, 231–240. [Google Scholar] [CrossRef]
- Sarrafi, A.; Mao, Z.; Nierecki, C.; Poozesh, P. Vibration-based damage detection in wind turbine blades using Phase-based Motion Estimation and motion magnification. J. Sound Vib. 2018, 421, 300–318. [Google Scholar] [CrossRef] [Green Version]
- Lucas, B.D.; Kanade, T. An interactive image registration technique with an application to stereo vision. In Proceedings of the 7th International Joint Conferences on Artificial Intelligence, Vancouver, BC, Canada, 24–28 August 1981; Volume 81. [Google Scholar]
- Bruck, H.A.; McNeill, S.R.; Sutton, M.A.; Peters, W.H. Digital image correlation using Newton-Raphson method of partial differential correction. Exp. Mech. 1989, 29, 261–267. [Google Scholar] [CrossRef]
- Wadhwa, N.; Rubinstein, M.; Durand, F.; Freeman, W.T. Riesz pyramid for fast phase-based video magnification. In Proceedings of the 2014 IEEE International Conference on Computational Photography (ICCP), Santa Clara, CA, USA, 2–4 May 2014; pp. 1–10. [Google Scholar]
- Canny, J.F. A computational approach to edge detection. IEEE Trans. Pattern Anal. Mach. Intell. 1986, 8, 679–697. [Google Scholar] [CrossRef] [PubMed]
- Brinker, R.; Zhang, L.; Anderson, P. Modal identification of output-only systems using frequency domain decomposition. Smart Mater. Struct. 2001, 10, 441–445. [Google Scholar] [CrossRef] [Green Version]
- Pastor, M.; Binda, M.; Harčarik, T. Modal assurance criterion. Procdia Eng. 2012, 48, 543–548. [Google Scholar] [CrossRef]
Four-Story Steel-Frame Building | Camera System | ||
---|---|---|---|
Story height | 2.2 m | Resolution | |
Story width | 3.15 m | Frame Rate | 30 fps |
Story weight | 6 tons | distance | ~3 m |
Beam cross section | H-type | ||
Sensors | 18 LVDTs |
LVDT | Optical Flow | DIC | Phase-Based | |||||
---|---|---|---|---|---|---|---|---|
Dispmax | RMSref | Errormax | RMSE | Errormax | RMSE | Errormax | RMSE | |
1st Floor | 16.96 | 6.5202 | 5.17 | 1.2661 (19.4%) | 4.52 | 1.2193 (18.7%) | 5.44 | 1.1034 (16.9%) |
2nd Floor | 19.35 | 6.8226 | 4.75 | 1.3124 (19.2%) | 4.85 | 1.0652 (15.6%) | 4.66 | 1.5071 (22.1%) |
3rd Floor | 20.24 | 7.0277 | 5.36 | 1.1202 (15.9%) | 5.51 | 1.2534 (17.8%) | 9.72 | 1.6095 (22.9%) |
4th Floor | 21.45 | 7.1746 | 5.02 | 1.4465 (20.2%) | 7.16 | 1.4107 (19.7%) | 5.15 | 1.4330 (20.0%) |
Optical Flow | DIC | Phase-Based | |
---|---|---|---|
Speed | 45.94 s | 56.30 s | 11.86 s |
Optical Flow | DIC | Phase-Based | ||||
---|---|---|---|---|---|---|
MAC | MAC | MAC | ||||
1st Mode | 1.23 Hz (−1.6%) | 1.00 | 1.23 Hz (−1.6%) | 1.00 | 1.23 Hz (−1.6%) | 1.00 |
2nd Mode | 4.01 Hz (+0.5%) | 0.97 | 4.01 Hz (+0.5%) | 0.92 | 3.92 Hz (−1.8%) | 0.95 |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Chou, J.-Y.; Chang, C.-M. Image Motion Extraction of Structures Using Computer Vision Techniques: A Comparative Study. Sensors 2021, 21, 6248. https://doi.org/10.3390/s21186248
Chou J-Y, Chang C-M. Image Motion Extraction of Structures Using Computer Vision Techniques: A Comparative Study. Sensors. 2021; 21(18):6248. https://doi.org/10.3390/s21186248
Chicago/Turabian StyleChou, Jau-Yu, and Chia-Ming Chang. 2021. "Image Motion Extraction of Structures Using Computer Vision Techniques: A Comparative Study" Sensors 21, no. 18: 6248. https://doi.org/10.3390/s21186248
APA StyleChou, J.-Y., & Chang, C.-M. (2021). Image Motion Extraction of Structures Using Computer Vision Techniques: A Comparative Study. Sensors, 21(18), 6248. https://doi.org/10.3390/s21186248