Structural Vibration Detection Using the Optimized Optical Flow Technique and UAV After Removing UAV’s Motions

Abstract

Traditional structural damage detection relies on multi-sensor arrays (e.g., total stations, accelerometers, and GNSS). However, these sensors have some inherent limitations such as high cost, limited accuracy, and environmental sensitivity. Advances in computer vision technology have driven the research on vision-based structural vibration analysis and damage identification. In this study, an optimized Lucas–Kanade optical flow algorithm is proposed, and it integrates feature point trajectory analysis with an adaptive thresholding mechanism, and improves the accuracy of the measurements through an innovative error vector filtering strategy. Comprehensive experimental validation demonstrates the performance of the algorithm in a variety of test scenarios. The method tracked MTS vibrations with 97% accuracy in a laboratory environment, and the robustness of the environment was confirmed by successful noise reduction using a dedicated noise-suppression algorithm under camera-induced interference conditions. UAV field tests show that it effectively compensates for UAV-induced motion artifacts and maintains over 90% measurement accuracy in both indoor and outdoor environments. Comparative analyses show that the proposed UAV-based method has significantly improved accuracy compared to the traditional optical flow method, providing a highly robust visual monitoring solution for structural durability assessment in complex environments.

1. Introduction

Traditional damage detection primarily involves installing sensors on the structure, such as laser ranging sensors, resistive strain sensors, and acceleration sensors. Employing these sensors incurs substantial labor costs and installation time [1,2]. Additionally, different sensor types have their own limitations. For example, they may have a limited measurement response range, and their accuracy can be affected by temperature and humidity changes. Moreover, the pre-installation wiring layout of sensors, as well as subsequent maintenance and replacement, are also expensive undertakings.

Therefore, many scholars have explored indirect non-contact measurement methods from several angles. For example, Luo et al. [3] utilized a total station to measure the vertical settlement and horizontal displacement before tunnel construction, but the application of total station was limited by the deployment points, such as a large number of equipments, e.g., cables and lighting equipment. Luo et al. [4] proposed an improved Variable Mode Decomposition (VMD) method to obtain accurate displacement trends from the initial GNSS signals. Dai et al. [5] combined the GNSS and Inertial Measurement Unit (IMU) methods and conducted simulation experiments on a shaking table using a three-antenna GNSS/IMU integrated device. However, the GNSS-based method is affected by factors such as signal strength and transmission frequency during satellite transmission, thus affecting the positioning accuracy. Some scholars have investigated the use of Laser Doppler Vibrometer (LDV) for detection. Klun et al. [6] utilized LDV to monitor hydroelectric power plant structures under dynamic conditions and determined the degree of damage of each structural component through the change of eigenfrequency. Schewe et al. [7] validated a drone-mounted LDV inspection system and successfully detected surface defects in cylindrical concrete specimens. Cosoli et al. [8] utilized LDV to scan concrete beams in order to obtain multiple test point data and detect the location of structural damage. Khun et al. [9] placed metal blocks at structural offsets and used LDV to measure the free vibration response of a building model under different offset loads to obtain the material parameters of the building model. However, the limitation of this LDV equipment is the reflection of light through the test points, which is not suitable for comprehensive measurements of large-scale structures. Yaghoubzadehfard et al. [10] scanned and obtained the modal vibration patterns of a footbridge using a device with a radar participation system (IBIS-FS). Morris et al. [11] used Ground Penetrating Radar (GPR) to survey a footbridge and successfully detected defects within the concrete. Although radar systems are suitable for non-contact, high-frequency measurements, the vibration signals are weak and prone to distortion, making accurate measurements challenging.

In recent years, computer vision-related programs and algorithms have been developing rapidly, and more and more scholars have begun to explore the intelligent recognition method that combines image processing technology with deep learning models, and the mainstream methods currently applied to vibration displacement testing include Digital Image Correlation (DIC) method, template matching method, and optical flow method. The DIC method is to process the image data and use the related processing software to calculate the displacement and deformation of the selected area of the object under test, etc. Xiao et al. [12] combined the template matching method with the sub-pixel method to realize the high-precision identification of multi-point damage of the components, and Durand-Texte et al. [13] verified a method that relies on a high-speed camera to measure the unidirectional displacement of the curved flat plate. Durand-Texte et al. [13] validated a method to measure the unidirectional displacement of a curved flat plate by a high-speed camera. Mastrodicasa et al. [14] used a low-speed camera in combination with the DIC method to perform a modal analysis of a wind turbine blade. Wang et al. [15] used DIC to analyze the crack cracking and development mechanism of a concrete beam. The template matching method works as follows: a region of the object under test is selected as a template, and each pixel point in the image is compared and calculated in turn; the higher the similarity, the higher the probability that the object is located at that position. Kimiya et al. [16] developed a camera calibration method combined with the oucc algorithm for high-frequency, large-scale structural vibration testing. Bai et al. [17] successfully eliminated the effects of camera translation and rotation by using the Zernike moments sub-pixel edge detection, the normalized cross-correlation based template matching method, and the pixel fitting method. Zhao et al. [18] designed a strain sensor based on a template-matching algorithm and proved its high accuracy in static and dynamic load tests of structures. Wu et al. [19] successfully extracted displacements and tilts of a large structure using a phase-based algorithm and a template-matching method. Hacıefendioğlu et al. [20] combined the LK optical flow method with an Autore Gressive Moving Averaging (ARMA) model. Enhanced Frequency Domain Decomposition (EFDD) validation experiments were conducted to demonstrate the reliability of the method for vibration measurements. Luan et al. [21] proposed a phase-based enhancement method to successfully track the large displacement motion of an object. Dong et al. [22] effectively eliminated the measurement errors caused by camera motion by phase subtraction based on immobile points. Zheng et al. [23] used the eigen-optical flow method to record the dynamic displacement response of transformer feature points in simulated earthquake experiments. The displacement data obtained were consistent with the sensor data. Al-Qudah and Yang [24] improved the LK optical flow method by using the second-order derivative of the Sobel operator via an algorithm.

Existing optical flow algorithms are prone to displacement calculation bias in complex scenes due to feature point mismatch and platform motion artifacts. In this study, an improved LK optical flow algorithm is proposed, which filters abnormal optical flow vectors through the angle constraint mechanism and combines with the threshold filtering strategy to improve the accuracy of displacement calculation in noisy environments. The system was validated using the MTS system, which provides stable inputs of amplitude and frequency, and helps to quantify the effect of angular constraint thresholds on the measurement error to determine the optimal parameters. The improved algorithm is further applied to vibration camera and UAV live video scenes, and the results show that the accuracy of the method is better than the traditional algorithm, which can effectively eliminate the platform vibration interference and extract more accurate displacement curves.

2. Methods Used in This Study

2.1. LK Optical Flow Method

When an observer views a moving object, the human eye’s retina captures the light from the brightness changes caused by the object’s motion. This allows the eye to track the object’s position at any given moment. In computer vision, the optical flow method leverages this principle. It tracks the position of an object moving relative to a stationary background by calculating the pixel changes in a continuous grayscale image due to the object’s movement [25,26]. The optical flow method primarily computes the velocity vectors along the x-and y-axes of the image to obtain the displacement information of the moving object. Before applying the optical flow method, certain constraints need to be established. Specifically, in a continuous grayscale image, the brightness of the moving object should remain essentially constant between two adjacent frames, and the object’s speed cannot be too high. Equation (1) can be derived from the constant brightness I of the object [27].

$$\mathit{I}\left(\mathit{x},\mathit{y},\mathit{t}\right)=\mathit{I}\left(\mathit{x}+\mathsf{\Delta }\mathit{x},\mathit{y}+\mathit{\Delta }\mathit{y},\mathit{t}+\mathit{\Delta }\mathit{t}\right)$$

(1)

In the Equation (1), $\Delta x, \Delta y$ represent the image in which the coordinate point (x,y) has elapsed time $\Delta t$ to move the distance along the x-axis and y-axis, respectively. A Taylor series expansion of the right-hand side of the equation to first-order terms yields Equation (2), which is then brought to a simplification to yield Equation (3).

$$I\left(x+\Delta x,y+\Delta y,t+\Delta t\right)=I\left(x,y,t\right)+{\displaystyle \frac{\partial I}{\partial x}}\Delta x+{\displaystyle \frac{\partial I}{\partial y}}\Delta y+{\displaystyle \frac{\partial I}{\partial t}}\Delta t$$

(2)

$${\displaystyle \frac{\partial I}{\partial x}}\Delta x+{\displaystyle \frac{\partial I}{\partial y}}\Delta y+{\displaystyle \frac{\partial I}{\partial t}}\Delta t=0$$

(3)

In the Equation (3), both sides of the equation are divided by $\Delta t$, then Equation (4) can be obtained, where $I_x$=${\displaystyle \frac{\partial I}{\partial x}},I_y={\displaystyle \frac{\partial I}{\partial y}},I_t$=${\displaystyle \frac{\partial I}{\partial t}},u={\displaystyle \frac{\Delta x}{\Delta t}},v={\displaystyle \frac{\Delta y}{\Delta t}}$.

$${\mathit{I}}_{\mathit{x}}\mathit{u}+{\mathit{I}}_{\mathit{y}}\mathit{v}+{\mathit{I}}_{\mathit{t}}=0$$

(4)

A window is set up with the coordinate point (x,y) as the center, and the n pixel points within the window have the same motion, combining these pixel points in Equation (4) can be expressed in a matrix as Equation (5).

$$\left[\begin{array}{cc}{ I}_{x_1}& I_{y_1} \\ I_{x_2}& I_{y_2} \\ .& .\\ .& .\\ .& .\\ I_{x_n}& I_{y_n} \end{array}\right] \cdot \left[\begin{array}{c} u \\ v \end{array}\right]=-\left[\begin{array}{c}{ I}_{t1 }\\ { I}_{t2}\\ .\\ .\\ .\\ I_{tn}\end{array} \right]$$

(5)

In Equation (5), the two-dimensional optical flow vectors u and v can be found by applying the least squares method.

$$\left[\begin{array}{c} \mathit{u} \\ \mathit{v} \end{array}\right]={\left[\begin{array}{cc}{\mathbf{\sum }}_{\mathit{i}}{\mathit{I}}_{\mathit{x}\mathit{i}}^{\mathbf{2}}& {\mathbf{\sum }}_{\mathit{i}}{\mathit{I}}_{\mathit{x}\mathit{i}}{\mathit{I}}_{\mathit{y}\mathit{i}}\\ {\mathbf{\sum }}_{\mathit{i}}{\mathit{I}}_{\mathit{x}\mathit{i}}{\mathit{I}}_{\mathit{y}\mathit{i}}& {\mathbf{\sum }}_{\mathit{i}}{\mathit{I}}_{\mathit{y}\mathit{i}}^{\mathbf{2}}\end{array}\right]}^{\mathbf{-}\mathbf{1}}\mathbf{\cdot }\left[\begin{array}{c}{\mathbf{\sum }}_{\mathit{i}}{\mathit{I}}_{\mathit{x}\mathit{i}}{\mathit{I}}_{\mathit{t}\mathit{i}} \\ {\mathbf{\sum }}_{\mathit{i}}{\mathit{I}}_{\mathit{y}\mathit{i}}{\mathit{I}}_{\mathit{t}\mathit{i}} \end{array}\right]$$

(6)

2.2. Improved Methods for the LK Optical Flow Method

When an object vibrates, it may move at a high speed, resulting in large displacements. Additionally, irregular surface reflections or changes in the distance from the light source can cause sharp brightness variations. These factors introduce computational errors in the optical flow method, leading to incorrect tracking of the object’s motion direction. As shown in Figure 1, to obtain more accurate calculation results, when the object undergoes rigid-body motion, the motion direction of each feature point is consistent and can represent the overall motion direction of the object. The motion direction of all feature points can be calculated using Formula (7), and the calculated angle is denoted as A.

$$A={\displaystyle \frac{\mathit{atan}\left(\frac{u_i}{v_i}\right)\cdot 180}{\pi }}$$

(7)

$$M={\displaystyle \frac{\mathit{atan}\left(\frac{\sum _{i=1}^n{u}_i}{\sum _{i=1}^n{v}_i}\right)\cdot 180}{\pi }}$$

(8)

$$S=\sqrt{{\displaystyle \frac{\sum _{i=1}^n{\left(A_i-M\right)}^2}{n}}}$$

(9)

$$D=|A-M|$$

(10)

After computing the mean M and standard deviation S of the angles for all feature points, the angular deviation of each feature point from the object’s overall motion direction can be derived. The magnitude of the angular deviation |D| is then compared with the standard deviation to filter out feature points with incorrect motion directions. The threshold is mainly set as a multiple of the standard deviation. A smaller threshold can eliminate feature points with obvious errors, but an excessively large threshold might exclude accurate displacement data. Hence, the threshold setting can be adjusted by comparing the number of retained and excluded points and observing the motion trajectories and displacement lengths of each feature point plotted via the optical flow method. Averaging the data from the retained feature points generates the average motion component for the current frame. When combined with the calculation results from the previous frame, the overall displacement data of the object can be obtained. Before using in Equations (7) and (8), the program will calculate the displacement component of each pixel point through Equation (6), and substitute it into the right side of the above equation to get the value of the left side of the equation, and then substitute the calculated angle A and the average value M into Equations (9) and (10) to get the angular deviation D and standard deviation S.

2.3. Camera Shake Elimination

When a camera is recording an object vibrating outdoors, the surrounding environment can induce camera shake. As a result, the motion of the object captured in the video is a combination of the camera’s panning and flipping movements, rather than the object’s actual motion.

2.3.1. Calibrating the Camera’s Flip Motion

When the camera shakes, it may rotate along different axes. During camera rotations, the object being measured is projected onto the screen, causing the object’s shape to be distorted. Thus, to calculate the real-world displacement information of the object, the image must be corrected first. This distorted projection can be rectified using the Planar Homography Transformation Method.

Planar Homography Transformation means that when the projection of an object is mapped in two different planes, the projected images in these different planes can be transformed into one another. In the process of planar homography transformation, four stationary reference points and their pixel coordinates X_i (x_i, y_i) need to be found first. The image projected from the stationary reference points onto the first frame is regarded as the original image. When the lens rotates, the image projected from the stationary reference points onto the second frame is considered the image to be corrected, with its coordinates Xi’ (x_i’, y_i’), as depicted in Figure 2.

The second or nth frame image can be corrected to the original image by the operation of the transformation matrix, which is calculated as shown in Equation (11).

$$\left[\begin{array}{c}{x}_i^{\prime }\\ y_i^{\prime }\\ 1\end{array}\right]\cdot H=\left[\begin{array}{c}{x}_i\\ y_i\\ 1\end{array}\right]$$

(11)

where the original image and the uncorrected stationary reference point coordinates can be obtained by the program, the transformation matrix H between the two frames is a 3 × 3 matrix, and one of the unknowns in the H matrix is converted to 1, which in fact only needs to be solved for eight unknowns, which can be solved by the least squares method.

2.3.2. Scale Factor Calculation

After correcting the image, the pixel coordinates of the current frame are converted to real world coordinates by a scale factor $a$, which means the ratio of the actual size of the object L, to the recognized size of the object $dp$ in the image [28], as in Equation (12) is shown.

$$a={\displaystyle \frac{L}{dp}}$$

(12)

2.3.3. Eliminate Camera Panning Motion

When the camera moves translationally in a plane, the true displacement of the object can be derived by subtracting the displacement of the stationary reference. As illustrated in Figure 3, the displacement of the stationary reference can be determined by tracking the movement of the stationary reference point in the video. Let the motion displacement of the object without eliminating camera shake be x₀, the motion displacement of camera shake be x_d, and the motion displacement of the object after eliminating camera shake be x_r [29]. The elimination methods are presented in Equation (13).

$$x_r=x_0-x_d$$

(13)

Figure 4 shows the computational process of obtaining the real displacement from the drone video, the bridge vibration video captured by the drone is grayed out, the pixel coordinates of the reference point and the test point are tracked simultaneously by the improved optical flow method described above. Then images are established to correspond to the mono clinic relationship, and the computed transformation matrix is used for calibration, and after the image plane is calibrated, the drone’s own movement is filtered through the static reference points of the trajectory. The final result is the true displacement of the test point.

3. Experiments and Results

3.1. Experimental Setup

To test the effectiveness of the improved method based on the traditional optical flow method and the accuracy of the data obtained in practical applications, an experiment is carried out. The experiment involves measuring the simulated vibration of an MTS hydraulic piston and comparing the vibration displacement data calculated by the traditional optical flow method and its improved program with the actual data output from the experiment, aiming to test the optimization effect of the improved method.

The experimental equipment setup is shown in Figure 5. The camera was placed directly in front of the MTS hydraulic press by using a tripod with the aim of capturing the vibration video at the correct camera angle and position. The MTS hydraulic press was connected to a computer terminal and first, the relevant parameters were set. A simple sinusoidal form of motion is set up by the computer to adjust the different frequencies and amplitudes of vibration required. After start-up, the machine can run stably for a long time. The video recorded by the camera has a resolution of 1920 × 1080 and a frame rate of 30 fps. The captured vibration video is then fed into the computer for processing, and the MATLAB (2024) program detects the characteristic points of the vibration video captured by the camera using the Harris angle detection function. Finally, the displacement increments in the x-axis and y-axis directions for two adjacent frames are output by a program that calculates the equations using the optical flow method, and the results for all frames are summed.

Method Validation

In order to verify the improved method developed based on the conventional optical flow method, a MATLAB program is written in accordance with the principle of this improved method. When calculating the optical flow between each pair of adjacent frames, for each feature corner point, the increment dx along the x-axis and the increment dy along the y-axis are computed. Then, the above Formula (7) is applied to determine the corresponding included angle. Feature points with an included angle smaller than the set threshold are removed. The thresholds are set with the standard deviation σ, which is calculated from the arrow angles of all detected feature corner points, as a common factor. Specifically, the thresholds are set as 1×σ, 0.5×σ, and 0.3×σ respectively. The feature corner points eliminated by setting these different thresholds are illustrated in Figure 6.

In Figure 6, a total of 223 feature angular points are detected.

Table 1. Quantities of Retained and Removed Feature Corners Under Different Threshold Values.

Thresholds	1×σ	0.5×σ	0.3×σ
reservations	186	130	97
removals	37	93	126

presents all the retained feature points and the number of deleted ones when different thresholds are set. The data in the table indicate that when the threshold is set relatively large, fewer feature points with insufficient accuracy are removed. Conversely, as the threshold is set smaller, more angular feature points are eliminated, and the removal effect is more significant. This results in retaining more feature points for which the computed data are more accurate [30].

3.2. Results

3.2.1. No Noise Condition

The experimental setup for the vibration testing of the MTS hydraulic system adopted a sine wave form. The amplitude was set at 10 mm, and the frequencies were set to 0.5 Hz and 1.5 Hz, respectively. The displacement curves of the objects were derived using MATLAB programs implementing both the traditional optical flow method and the improved method. These curves were then compared with the displacement curves output by the MTS. The test results for the vibration at a frequency of 0.5 Hz are depicted in Figure 7. The calculated data show a high degree of synchronization with the real position at each instant. Among them, the displacement curve with a threshold of 0.3×σ is the closest to the displacement curve output by the MTS, yielding the most accurate results, especially in the vicinity of the peak where the comparison is more pronounced.

As illustrated in Figure 7, the main computational error is concentrated in the peak region. The region of interest was defined as centered around the peak point, with a range spanning five frames of the picture before and after the peak as the area for calculating the average accuracy.

The Normalized cross-correlation (NCC) method was used to calculate accuracy between two vibrations. And the how to calculate the NCC is shown in Equation (14). The accuracy for the vibration curve at different regions with a frequency of 0.5 Hz are presented in

Table 2. Accuracy Comparison of Region of Interest Computation: Traditional LK Method vs. Improved LK Method with a Threshold of 0.3×σ (at 0.5 Hz).

ROI	Max Pt Region1	Min Pt Region1	Max Pt Region2	Min Pt Region2
Traditional LK	95.7%	95.9%	95.9%	96.05%
Improved LK	97.0%	97.1%	97.3%	96.9%

Marginal notes: ROI (region of interest) refers to the region of interest. “Max pt region1” and “Min pt region2” represent the region near the maximum extreme-value point and the region near the minimum extreme-value point, respectively, and the numbers 1 and 2 denote the first or second (region).

. The accuracy of the region of interest stabilizes at approximately 97.0% by using the proposed Improved LK Method, comparing with accuracy at 96% by using the traditional LK Method.

$$\begin{array}{c} NCC={\displaystyle \frac{{\sum }_x\left[f\left(x\right)-\overline{f}\right]\left[t\left(x\right)-\overline{t}\right]}{\{{\sum }_x\left[f\right(x)-\overline{f}{]}^2[t\left(x\right)-\overline{t}{]}^2{\}}^{0.5}}}\end{array}$$

(14)

where f is displacement value of the first vibration; $\overline{f}$ is the mean of the first vibration; t is displacement value of the second vibration; $\overline{t }$ is the mean of the second vibration.

In the vibration test with a frequency of 0.5 Hz, the average error of the calculation results for all frames is 2.0%. The distribution of the calculation errors for all consecutive frames is shown in Figure 8. Here, the errors are mainly concentrated at the peak positions. The distribution pattern of the calculation errors of the improved method is roughly the same as that of the traditional LK method.

The test results for the vibration at a frequency of 1.5 Hz are presented in Figure 9. The overall waveform of the displacement curve obtained by the improved method generally aligns well with the actual displacement curve output by the MTS machine. Based on the assumptions of the optical flow method, it can be inferred that the object moves faster at 1.5 Hz compared to 0.5 Hz. Moreover, the motion displacement of the object in each frame is greater at 1.5 Hz. As a result, the optical flow method calculation produces larger errors, and these errors are mainly concentrated in the vicinity of the peak values.

The calculation results of the accuracy of the region of interest for the vibration curve with a frequency of 1.5 Hz are shown in

Table 3. Accuracy Comparison of Region of Interest Computation: Traditional LK Method versus Improved LK Method with a Threshold of 0.3×σ (at 1.5 Hz).

ROI	Max Pt Region1	Min Pt Region1	Max Pt Region2	Min Pt Region2
Traditional LK	91.4%	90.0%	86.2%	87.3%
Improved LK	98.6%	99.3%	94.4%	95.9%

. The accuracy remains stable at 97.0%, which is much higher than that using the traditional LK method. The average error of the calculation results across all frames is 2.0%.

3.2.2. Noise Condition

To verify the applicability of the improved optical flow method under non-laboratory noisy conditions, vibration was artificially introduced to the fixed camera during the recording of the vibration video [31,32]. Then, in combination with the camera vibration elimination method proposed in this study, comparative experiments were conducted before and after noise removal. The relevant experimental set-ups were the same as those described above. The vibrations of the camera were triggered by hand. In the presence of noise, the improved program with a threshold value of 0.3×σ was used in conjunction with the noise removal method. The test results for the vibration at a frequency of 1.5 Hz are shown in Figure 10. The displacement curves obtained after noise removal using the improved method show a significant improvement in accuracy compared to those before removal. The displacement curves obtained by the improved program are much closer to the displacement curves output by the MTS. The optimization effect is particularly evident near the peak values.

The calculation results of the accuracy of the region of interest for the vibration curve with a frequency of 1.5 Hz are shown in

Table 4. Comparison of accuracy of region of interest computation (1.5 Hz) after eliminating camera vibration with the improved LK method with a threshold of 0.3×σ.

ROI	Min Pt Region1	Max Pt Region1	Min Pt Region2	Max Pt Region2
Before elimination	94.8%	85.5%	95.3%	79.9%
After elimination	96.7%	95.4%	96.9%	96.7%

. Its accuracy is stabilized at 95.6%. Significantly, the data after noise elimination exhibit a substantial improvement in accuracy compared to the original calculated data, with the maximum improvement being approximately 16%.

When using the improved procedure with a threshold value of 0.3×σ and the noise-elimination method for calculations, the test results for the vibration at a frequency of 3 Hz are presented in Figure 11. By applying the improved method to process the noise, it is evident that the results derived from the improved procedure exhibit the same optimization effect.

The calculation results of the accuracy of the region of interest for the 3 Hz frequency vibration curve are presented in

Table 5. Comparison of accuracy of region of interest computation (3 Hz) after eliminating camera vibration with the improved LK method with a threshold of 0.3×σ.

ROI	Min Pt Region1	Max Pt Region1	Min Pt Region2	Max Pt Region2
Before elimination	59.7%	65.2%	56.1%	70.4%
After elimination	97.7%	98.6%	96.2%	95.7%

. The accuracy stabilizes at 97.0%. Moreover, the accuracy of the data after noise elimination is also significantly improved, with a maximum increase of approximately 25%.

4. UAV Experiments

4.1. UAV Experiment in Lab

When a UAV’s on-board camera is used to record video, during flight in the air, the UAV is likely to be influenced by air currents and wind, causing the fuselage to drift and shake. To verify the reliability of the improved method in removing the noise generated by the UAV, comparative experiments were carried out both before and after noise elimination. The experimental setup is illustrated in Figure 12.

The experimental configuration was as follows: The MTS hydraulic system was made to vibrate at frequencies of 0.5 Hz and 1 Hz, respectively, with the amplitude set at 10 mm. The UAV was operated, and the camera mounted on the UAV captured the vibration video at a frame rate of 30 frames per second. The experimental equipment is depicted in Figure 13.

The test results at a frequency of 0.5 Hz are presented in Figure 14. Owing to the instability of the UAV’s hovering flight, when the UAV moves away from the MTS, the horizontal distance between the UAV and the vibrating object increases, which leads to a relatively small amplitude of the curves from 0 to 2 s. As illustrated in Figure 15, there is an increase in the horizontal distance from the vibrating object during the movement away from the MTS. Conversely, when the UAV approaches the MTS, the horizontal distance from the vibrating object decreases, causing the amplitude of the curve to be larger in the time interval of 2 to 4 s. Through a comparison with the curves of the MTS, it can be seen that the data obtained after noise removal using the improved method are accurate.

As presented in

Table 6. Accuracy Comparison of Region of Interest Calculation for Improved LK Method After UAV Wobble Elimination (0.5 Hz).

ROI	Min Pt Region1	Max Pt Region1	Min Pt Region2	Max Pt Region2
Before elimination	90.4%	88.3%	93.3%	87.5%
After elimination	97.9%	99.1%	98.0%	97.9%

, the accuracy prior to the elimination of the UAV’s wobble was approximately 89.9%, and after the elimination, it reached around 98.2%, marking an improvement of 8.3%.

The test results at 1 Hz are shown in Figure 12. When the horizontal distance between the UAV and the object changes significantly, the improved method is still capable of effectively eliminating the noise and yielding accurate results.

As shown in

Table 7. Comparison of the accuracy of the region of interest calculation (1 Hz) for the improved LK method after eliminating UAV wobble.

ROI	Min Pt Region1	Max Pt Region1	Min Pt Region2	Max Pt Region2
Before elimination	36.6%	70.2%	53.9%	16.6%
After elimination	98.5%	97.8%	98.3%	98.5%

, the accuracy before eliminating drone drift was only about 44.3% and after eliminating it was about 98.3%, an improvement of 54.0%, due to the drone’s violent shaking.

4.2. UAV Experiment Outside Lab

An outdoor setup was arranged with a vibration table. On this table, a test object was firmly secured. This setup allowed for the adjustment of frequency and amplitude parameters to subject the test object to periodic reciprocating motion. As depicted in Figure 16, there were several stationary background reference points behind the vibration table. Two complementary imaging systems were employed for data collection. One was an unmanned aerial vehicle (UAV) fitted with a high-speed camera, and the other was a stationary ground-based camera placed in front of the vibration table. The vibration table produced sinusoidal vibrations at a frequency of 3 Hz and an amplitude of 10 mm. The UAV-mounted camera and the fixed camera recorded videos at frame rates of 60 fps and 30 fps, respectively.

At the experimental site, multiple stationary objects were selected as reference targets for calculating the UAV’s trajectory. Variations in distances between these stationary objects and the measured object can lead to inaccurate results [33]. To identify the optimal stationary target, comparative analyses were conducted using different stationary objects positioned near the experimental setup.

Three stationary objects—the tool bucket (a), cardboard box (b), and iron mesh (c)—were selected for comparative analysis based on their distance relationships to the experimental setup, as illustrated in Figure 17.

The calculation results using the tool bucket (a) as the stationary target are presented in Figure 18. The UAV motion trajectory derived from this calculation exhibits higher accuracy, with the final displacement curve demonstrating closer alignment to the MTS output.

The calculation results using the cardboard box (b) as the stationary target are presented in Figure 19. The derived UAV trajectory exhibits significant errors, because the cardboard box vibrated due to the wind. The final displacement curve shows substantial deviation from the MTS output.

The calculation results using the iron mesh (c) as the stationary target are presented in Figure 20. The derived UAV trajectory exhibits the largest error, with the final displacement curve demonstrating significant deviation from the MTS output. The possible reason is that iron mesh is too dark to be detected precisely.

As presented in

Table 8. Precision of calculation results output from the improved LK method (3 Hz) after elimination of UAV trajectories when different reference points are set up.

Stationary Object	Tool Bucket (a)	Cardboard Box (b)	Iron Mesh (c)
After elimination	95.5%	79.7%	52.7%

, the precision of calculation results output from the improved LK method (3 Hz) after elimination of UAV trajectories with different reference points are shown.

As indicated in Figure 21, the displacement curve obtained from the UAV camera, which was calculated using the tool bucket (a) as a stationary target, shows a general similarity to the curve trajectory of the stationary camera. However, there are distinct disparities at the peak points. It is also evident that the curve of the stationary camera is more in line with the curve output by the MTS.

As shown in

Table 9. Accuracy Comparison of Region of Interest Calculation for Displacement Curves Derived from the Improved LK Method Based on Fixed and UAV Camera Data.

ROI	Max Pt Region1	Min Pt Region1	Max Pt Region2	Min Pt Region2
Fixed Camera	97.4%	94.5%	98.4%	98.9%
UAV Camera	92.4%	92.8%	96.1%	90.4%

, in the vicinity of the peak, the calculation results of the stationary camera are more accurate, with an average accuracy of about 97.3%, and the calculation error of the UAV camera is slightly larger, with an average accuracy of about 92.9%. As presented in Table 9, within the peak regions, the computed results from the stationary camera demonstrated higher accuracy, achieving an average accuracy of approximately 97.3%. Conversely, the UAV camera’s calculation error was marginally higher, resulting in an average accuracy of approximately 92.9%.

5. Conclusions

The proposed optimized optical flow-based displacement sensor accurately detected MTS vibrations in laboratory experiments, achieving better accuracy of approximately 97% comparing with the existed optical flow-based displacement sensors. The proposed method maintained structural vibration detection accuracy exceeding 90% when utilizing UAV-recorded vibration videos. Effective compensation for six distinct UAV motions resulted in significant accuracy improvements.

• The optimized optical flow displacement sensor proposed in this work obtains more accurate vibration detection results compared to the original optical flow method.
• The proposed method effectively eliminated the motions of the UAV. In laboratory settings, when the UAV was employed along with the proposed method to measure vibrations, the resulting accuracy was satisfactory.
• The proposed method has successfully eliminated the motions of the UAV. In the field experiment, when using the UAV in conjunction with the proposed method to obtain vibration data, the achieved accuracy is satisfactory.
• In the field experiment, the accuracy of the measurement utilizing the UAV was approximately 93%, whereas the accuracy achieved with the fixed camera was 97%.Potential sources of error may be the lighting condition changes and the blurred image.
• When the proposed method is applied to eliminate the motions of the UAV, different fixed points selected in the background will give rise to varying degrees of errors. There are some ways to avoid these errors, such as fixing the reference point well, selecting reference targets with high contrast, and keeping the reference targets close to the structure.

In summary, the angle-constrained optical flow algorithm proposed in this study realizes the measurement of the motion and structural response of the UAV platform, effectively eliminates the interference of UAV vibration artifacts, and significantly improves the vibration detection accuracy. It provides a new technical path for the application of visual displacement measurement in noisy environments. The limitations of traditional vision sensors are solved, and lightweight, non-contact structural vibration monitoring is realized.