Experimental Validation of Non-Marker Simple Image Displacement Measurements for Railway Bridges

Abstract

Simple bridge displacement measurement using a video camera is effective in realizing the efficient management of numerous railway structures via condition-based maintenance. Although non-marker image measurement is significantly influenced by the measuring environment, its practical applicability considering the displacement measurement accuracy of non-marker images and the influence of various environments is not completely understood. In this study, the accuracy of non-marker image displacement measurement and the influence of illuminance are confirmed using a model bridge, and the accuracy and applicable range are discussed. Moreover, field tests on two bridges—a steel and a concrete bridge—on low-speed and high-speed railways confirm the accuracy and practical application of non-marker image measurement in a real environment. The displacement was observed to be measured with an accuracy of ~1/30 pixel (error of ~0.4 mm at 20 m position) in the daytime with sufficient brightness. Moreover, the settings for subset positions and post-processing methods to ensure accuracy in non-marker image measurement on concrete bridges with low surface contrast are discussed.

1. Introduction

There is a long history of railway construction, and railway bridges exist that were constructed over 100 years ago [1]. In addition, in developed countries, there has been a decrease in the number of personnel available to manage these bridges. Thus, improvement is required in the efficiency of bridge management systems based on existing periodic visual inspection. In particular, a transition to condition-based maintenance (CBM) has been proposed. CBM aims at improving the efficiency of maintenance while ensuring safety by appropriately evaluating the structural performance of railway bridges and optimizing the types and implementation times of measures [2,3]. The efficient and quantitative evaluation of bridge performance based on measurements is indispensable for realizing CBM. One of the typical property indicators of railway bridges is the deflection (vertical displacement) under the passage of trains [4,5]. However, the traditional contact-type or wire-type deflection measurement methods are required to work at large heights and may not be possible depending on the conditions under the bridge (e.g., river bridges). Because of this problem, several developments of simple deflection measurement techniques have been conducted [6,7,8]. This study contributes to the realization of CBM from the perspective of improving the efficiency of the measurement method by focusing on the displacement measurement method of bridges via image processing.

This study’s concept for image displacement measurements is based on the premise that it will be used for patrols and visual inspections of existing bridges. This point differs from previous studies aimed at developing highly accurate measurement techniques. Therefore, non-marker image displacement measurements that use only the minimum portable image measurement items (i.e., camera and tripod only) and do not require access to the bridge will be considered in this study. This concept aims at realizing primary screening based on bridge performance by establishing a simple bridge displacement measurement method that can be applied in the current inspection and patrol system. Furthermore, the quantitative records of structural performance are maintained in daily patrol visual inspections, which provide vital and objective data for future managers’ decision making in the long-term management of bridges.

The measurement accuracy and applicable conditions are critical issues for this type of simple measurement method. In this study, the displacement measurement accuracy and brightness conditions for non-marker image measurement were confirmed by a laboratory model test [9]. The applicable scope was then determined based on the accumulated deflection data of railway bridges during the passage of a train. Furthermore, field tests using two bridges, on a local low- and a high-speed railway, indicate the effectiveness of non-maker image measurement in real-world situations.

In this paper, Section 2 reviews the related works and Section 3 explains the non-marker image measurement method. Section 4 describes the results of accuracy verification tests using model bridges and the calculation results of the applicable scope. Section 5 presents the results from the application on two bridges and additional considerations.

2. Related Contributions

Due to advancements in cost and video capability, such as the development of high speeds and high resolutions, image measuring using a video camera is gradually becoming more common. Because railway bridge displacement is the primary structural index, in this section previous major contributions are reviewed, with a particular emphasis on bridge displacement measurements and application instances using images.

The method of measuring bridge displacement using images and markers is long standing. Lee and Shinozuka [10] reported the measurement of actual bridges using industrial video cameras in 2006. Subsequently, numerous improvements and applications of bridge displacement measurement methods have been made using industrial video cameras. For example, Choi et al. [11] proposed a method of measuring bridge dynamic displacement using a handheld video recorder via an image processing method that resizes the region of interest and updates the coefficients. Ribeiro et al. [12] established a displacement measurement system for railway bridges using an industrial video camera. They proposed a method using a dedicated marker to track a similar position in the image obtained by digital image correlation (DIC). Moreover, lights and windshields were employed to ensure adequate illumination and limit the camera’s instability. In video measurement in which accuracy is a problem, their method achieved a relatively high accuracy of 0.1 mm (shooting from a 10 m point). Cigada et al. [13] examined the effects of the camera type, image processing techniques, and magnification levels on image-based methods of measuring bridge displacement during train passage. They noted that a significant increase in measurement error is unavoidable in non-marker displacement measurements. Feng et al. established a displacement measurement system for railway bridges through several studies using industrial video cameras [14,15,16]. Moreover, the accuracy was empirically verified, not only when using markers, but also when measuring the displacement without markers [16]. They proposed a sub-pixel estimation method for high-resolution displacement measurement [15]. Pan et al. proposed a DIC-based image displacement measurement system for railway bridges [17]. Here, a computer-controlled industrial video camera was proposed for use in non-marker deflection measurements and a real-time system that allows the results to be confirmed at the measurement site.

Furthermore, several active studies and contributions related to image displacement measurements are underway, as detailed in two review papers [18,19]. Feng and Feng (2018) summarized the importance of multipoint measurement data obtained from images from the damage detection perspective [18]. Moreover, Xu and Brownjohn (2018) discussed the use of structural health monitoring and the difficulties that should be addressed in the future for each elemental technology (camera calibration, target tracking, and structural displacement calculation) [19].

Recently, Ribeiro et al. [20] realized a remote image measurement method using aerial vehicles. The automation and simplification of imaging using aerial vehicles and robots have significant potential to simplify the patrol and inspection of structures, which is the premise of this study. Although the use of aerial vehicles is an important research subject, it is outside the scope of this study and a future’s task.

Several of the aforementioned studies confirmed their measurements on actual bridges after sufficient confirmation of their accuracy in the laboratory. Therefore, there are few instances in which the scope of application is discussed after understanding the limits of the camera distance and the measurable environmental conditions, in terms of both measurable and unmeasurable data. However, the quantification of the relationship between the camera distance and the measurement accuracy performed by Ribeiro et al. [12] is a remarkable contribution. To determine if the image measurement method can be reasonably used in actual practice, at least two relationships, namely, those between the measurable distance and the bridge displacement, and between the environment (especially brightness) and the measurement accuracy, are required. The authors of this study confirmed the relationship between the illuminance and the measurement accuracy by testing under various lighting conditions in the laboratory. The results were close to the limit of measurement accuracy and their analysis provides considerable information for future developments.

3. Processing and Test Methods

3.1. Image Processing Method

In this study, the displacement is calculated from the image obtained via the DIC method [12]. A random pattern or marker is often applied to the object to be measured [12,21]. Here, using the high-resolution image obtained by a video camera, the displacement is calculated via DIC [22] without applying a random pattern or setting markers.

Figure 1 shows an example of DIC processing in the model bridge in this study. The subset A of the red frame in the figure is set at the time point $t_0$. The length and width of the subset A are equal, and the size of subset A is $L_A\times L_A$. Further, the local coordinate system whose origin is the control point, which is the center of subset A, is set as $\left(i^{\prime },j\prime \right)$. Moreover, subset B is set as the search area for the amount of movement of subset A at different time points. Here, the center of subset B is the origin $\left(i^{\prime },j\prime \right)=\left(0,0\right)$ of the local coordinate system. Furthermore, for the size $L_B\times L_B$ of subset B, $L_B>L_A$. In this study, $L_B=2L_A$ was set primarily to verify the measurement accuracy of minute displacements of 0.1 pixels or less.

The amount of image movement $\left(\delta i_t,\delta j_t\right)$ at time point t is estimated by searching for the position in subset B that has the highest correlation with the lightness vector $D_A$, which is the vectorization of the lightness series of each pixel of subset A. Indices for the Sum of Squared Difference (SSD), Sum of Absolute Difference (SAD), and Normalized Cross-Correlation (NCC) are used for calculating the amount of image movement [22,23,24]. The NCC shown in Equation (1) was used to eliminate the effect of changes in brightness of the entire subset. Let $d_r\left(i^{\prime },j^{\prime }\right)$ be the lightness of the reference at local coordinates $\left(i^{\prime },j^{\prime }\right)$, and $d_t\left(i^{\prime },j^{\prime }\right)$ be the light level at time, t, at the same local coordinates. The NCC in the number of movements, $\left(u^{\prime },v\prime \right)$, is determined by Equation (1).

$$NCC\left(u^{\prime },v^{\prime }\right)=\frac{{\sum }_{i^{\prime },j^{\prime }}\left\{\left(d_r\left(i^{\prime },j^{\prime }\right)-{\overline{d}}_r\right)\left(d_t\left(i^{\prime }-u^{\prime },j^{\prime }-v^{\prime }\right)-{\overline{d}}_t\right)\right\}}{\sqrt{{\sum }_{i^{\prime },j^{\prime }}{\left(d_r\left(i^{\prime },j^{\prime }\right)-{\overline{d}}_r\right)}^2{\sum }_{i^{\prime },j^{\prime }}{\left(d_t\left(i^{\prime }-u^{\prime },j^{\prime }-v^{\prime }\right)-{\overline{d}}_t\right)}^2}},$$

(1)

where ${\overline{d}}_r$ represents the average value of the light level in subset A, and ${\overline{d}}_t$ represents the average value of the light level within the range of size $L_A\times L_A$ at a time point t. Therefore, as shown in Equation (2), the amount of movement $\left(\delta i_t^{\prime },\mathsf{\delta }{j}_t^{\prime }\right)$ that maximizes the NCC is the estimated value of the displacement (in pixels) at the time point t.

$$\left(\delta i_t^{\prime },\mathsf{\delta }{j}_t^{\prime }\right)={\mathrm{argmax}}_{u^{\prime },v^{\prime }}\left\{NCC\left(u^{\prime },v^{\prime }\right)\right\}.$$

(2)

Figure 2 shows the parabola fitting used to estimate the displacement below 1 pixel (sub-pixel estimation) [23,25]. In parabola fitting, the 2D parabola function is fitted to the correlation value of the pixel with the highest correlation value $\left(\delta i_t^{\prime },\mathsf{\delta }{j}_t^{\prime }\right)$ and the surrounding 8 pixels. The positions $\delta i_t,\mathsf{\delta }{j}_t$ that represent the maximum value of the fitted parabolic function are calculated as sub-pixel displacements in 1/1000-pixel units.

In sub-pixel estimation using the fitting function, a phenomenon called pixel locking may occur, in which the number of movements is biased to an integer value [25]. The estimation error cancelation (EEC) method [25] was introduced in the DIC of this study to minimize the effect of pixel locking. In the EEC, sub-pixel estimation is performed using the reference of the image translated by 0.5 pixels and estimated separately. The movement amount adjusted by 0.5 pixels is estimated from the movement amount of these images, and the bias to the integer value is offset by considering the average of the movement amount estimated from the original image (see reference [25] for details).

In the displacement estimation by DIC and parabola fitting, the influence of the subset size $L_A$ is frequently mentioned. In particular, if the texture of the surface to be measured is a random pattern, the accuracy is generally higher when the size $L_A$ of the subset A is set to large value [19]. However, if the size $L_A$ of subset A is increased, objects and backgrounds that move differently from the bridge will be mixed in the subset, and the accuracy will decrease. Moreover, as shown in the discussion in Section 5, the brightness variation of the measurement surface has a greater effect on the displacement estimation error than the effect of the subset size in the case of non-marker DIC. Considering these factors, the subset size $L_A$ is based on 15 pixels, and the effect of brightness variation on the measurement surface accuracy is empirically analyzed.

The obtained pixel-by-pixel displacement is converted from pixels to actual dimensions (m) using structural dimensions comparable to the drawing, such as the girder height at the measurement position. The effects of camera calibration [19] were investigated; however, the effects could not be confirmed, so their description is omitted from this study.

3.2. Model Experiment Method

Table 1. Specifications of model bridge.

Specifications	Values
Bridge length	3048 mm
Bridge span	2800 mm
Mass (without noise barrier)	230 kg/197 kg
Girder	2 × H-100
Deck	3048 × 400 × 9 mm
Calculated frequency	37 Hz

and Figure 3 show the specifications and view of the model bridge [9] used for the measurement confirmation in this study. A steel bridge with a total length of ~3 m is supported by four bearings with a span length of 2.8 m. Horizontal and rotating mechanisms were introduced to ensure the bearings achieved simple support conditions. The main girder is H-steel with a height and width of 100 mm, the deck is a steel plate with a width of 400 mm and a thickness of 9 mm, and the noise barriers are Al plates with a height of 165 mm and a thickness of 1.5 mm. The goal of this experiment was to confirm the basic performance and measurement accuracy, and to evaluate the marginal performance of the measurement system, such that the amount of deflection at a shooting distance of 2.5 m was set at ~0.01–0.1 pixels. The noise barrier was removed, and only two H-beams and the deck were used for the experiments. The frequency of the girder primary mode in this configuration is 37 Hz in the calculated value and ~35 Hz in the actual measurement.

Table 2. Specifications of the measuring equipment.

Equipment	Product Models	Companies	Specifications
Contact displacement gauge	CFP-10	Tokyo Sokki (Shinagawa, Tokyo, Japan)	Range: 10 mm Output:5 mV/V Sensitivity: 0.001 ε/mm
Data recording	Ni cDAQ-9172	National Instruments Japan (Minato, Tokyo, Japan)	Sampling rate: 2000 Hz
	Ni9220
	LabVIEW
Camera	Flare 4M180-MCX	IO industries (Hamamatsu, Shizuoka, Japan)	Resolution: 2048 × 800 pixel
Recorder	DVR Express CORE2	IO industries (Hamamatsu, Shizuoka, Japan)	Sampling rate: 350 Hz
Lens	LM8XC2	Kowa Company. Ltd. (Nagoya, Aichi, Japan)	Focal length: 8.5 mm

shows the measuring equipment specifications, and Figure 4 shows the measuring equipment layout. To measure the deflection shape of the model bridge girder, seven contact displacement gauges were installed at equal intervals (L/8, 2L/8, 3L/8, 4L/8, 5L/8, 6L/8, and 7L/8). These measurement points are called Ch0–Ch6, from the left-hand side of the figure. The video camera was installed 2.5 m from the model bridge, and the displacement was measured at the same location as the displacement meter measurement point. LEDs were used for lighting. The effect of the brightness of the shooting environment on the measurement accuracy, determined by turning the LED light on and off, was examined. A distance of one pixel above and below in the image taken under the measurement conditions corresponds to 1.45 mm above and below the model bridge.

The excitation was performed using a shaking machine, and the midspan and the L/4 point of the girder, which was the antinode of the secondary mode, were vibrated. The vibration frequency was set to 35 Hz at the midspan and 120 Hz at the L/4 point. The excitation frequencies corresponded to the resonance frequencies in the primary and secondary deflection modes. The displacement amplitudes were about ±30 μm for the midspan excitation, and about ±4 μm for the L/4-point excitation, which were about ±0.02 pixels and ±0.003 pixels, respectively. These amounts were set to be close to the limit of displacement measurement accuracy by the DIC.

3.3. Field Measurements Method

In this study, two railway bridges with different properties were investigated.

Table 3. Specifications of bridge A.

Types	Simply Supported Bridge
Maximum speed	100 km/h
Span length	20 m
Girder height	1.4 m
Girder type	I type steel girder × 2
Track	Wooden sleeper single track (without slab)
Moment of inertia of area	0.062 m4
Frequency	13.03 Hz

and Figure 5 show the outline of bridge A of the local low-speed railway that was measured for confirmation in this study. It comprises two steel I girders with a 20 m span length and a 1.4 m girder height. For the main girder, two steel I girders with a length of 10 m are connected with bolts at the midspan. Wooden sleepers are directly laid on the upper flanges of the girder. In the field test, the accuracy of non-marker image measurement was confirmed by comparing it with the existing displacement measurement system using a laser Doppler velocimeter, as explained below. The train speed was <100 km/h and the contribution of the dynamic response was insignificant compared with that of the high-speed railway bridge; the bridge has a single track and is simply supported, and the behavior of train passage is relatively simple. Moreover, a steel bridge constructed by connecting several members has a clear subset texture (surface pattern), thus making it relatively easy to ensure accuracy.

Table 4. Specifications of bridge B.

Types	Simply Supported Bridge
Maximum speed	260 km/h
Span length	89.2 m + 77.2 m
Girder height	1.5–3.2 m
Girder type	Concrete box
Track	Ballastless double track

and Figure 6 show the specifications and pictures of the high-speed railway bridge B that was measured. Bridge B is a two-span continuous prestressed concrete rigid frame bridge whose dynamic behavior under train passage is relatively complicated; the length of the measured span is 89.2 m. Because it is a concrete box girder bridge, there are few clear textures on the bridge taken by non-marker image measurement. Therefore, it is difficult to ensure the displacement estimation accuracy by DIC. Moreover, the maximum speed of bridge B is 260 km/h, the frequency at which displacement occurs is higher than that of bridge A, and a contribution of the dynamic response is anticipated [26,27]. For bridge B, the point 34.5 m from the starting point was set as the target point for deflection measurement. The height from the ground to the bottom of the girder is 5.7 m, and other measurement devices that require space were placed beneath the girder.

Table 5. Specifications of the measuring equipment on bridge A.

Measurement Methods	Measurement Principles	Specifications
Self-vibration correction laser Doppler velocimeter (Laser measurement)	Displacement is calculated by integrating the vibration velocity measured with a laser Doppler velocimeter	Non-contact With reflective marker Sampling rate: 500 Hz
Non-marker image measurement	Displacement is calculated from the image by DIC (Section 3.1)	Resolution: 1920 × 600 pixel Sampling rate: 120 fps Camera: GS3-U3-23S6M-C Lens: LM8XC2

shows the specifications of the measuring equipment on bridge A, and Figure 7 shows the installation status of each measuring device. In addition to the suggested method, a laser Doppler velocimeter (referred to as laser measurement) [6] was used to measure the girder deflection during the train passage to verify the accuracy of the proposed non-marker image measurement. Laser measurement estimates the displacement by integrating the measurement response of a laser Doppler velocimeter. Furthermore, the laser Doppler velocimeter has a built-in acceleration sensor, making it possible to separate the vibration of the meter itself from the measurement response [6,27]. For non-marker image measurement, a video camera was installed 16.5 m from the bridge such that the entire bridge could be seen on the screen. A single vertical pixel in the image taken under the measurement conditions corresponds to a 9.2 mm vertical distance at the bridge position. In the non-marker image measurement, the displacement at the midspan was measured for verification with a different measurement method, and the deflection shape of the girder was measured by taking advantage of the non-marker image measurement. In measuring the deflection shape of the girder, 44 points were set at equal intervals along the longitudinal direction of the girder as measurement points, and the displacement was calculated at each position. Two passing trains (6 cars at 45 km/h and 2 cars at 35 km/h) were involved. The measurement was performed between 11:00 and 15:00. The weather was cloudy and there was almost no wind.

Figure 8 shows the installation position of each method on bridge B, and

Table 6. Specifications of the measuring equipment on bridge B.

Measurement Methods	Measurement Principles	Specifications
Ring type gauge Wire fixing jig, gauge	Displacement is converted from the strain of the ring gauge installed on a piano wire connecting the girder to the ground.	Sampling rate: 1000 Hz
Self-vibration correction laser Doppler velocimeter (laser measurement) Reflective marker, Doppler meter	Displacement is calculated by integrating the vibration velocity measured with a laser Doppler velocimeter	Non-contact With reflective marker Sampling rate: 500 Hz Measurement distance: 0.1–100 m
Non-marker image measurement A PC-controlled industrial camera	Displacement is calculated from images by DIC (Section 3.1)	Resolution: 4096 × 2160 pixels Camera: Flare 4M180-MCX Sampling rate: 200 fps Lens: wide-angle or standard
Non-marker image measurement B Compact digital camera	Displacement is calculated from images by DIC (Section 3.1)	Resolution: 2048 × 1088 pixel Camera: DSC-RX1004M Sampling rate: 60 Hz

shows the equipment specifications and installation status of each displacement measurement method performed on the bridge. The bridge displacements for bridge B during the train passage were simultaneously measured by four measurement methods: a ring-type gauge, a laser [6], non-marker image measurement A (industrial camera), and non-marker image measurement B (compact digital camera). Non-marker image measurement B used the high-speed shooting function of a general compact digital camera. This is a scenario that assumes simple measurement using a digital camera carried on a portable structure. The ring-type gauge and laser measurements were undertaken by directly arranging the equipment under the measurement point. In the ring-type gauge, a jig was attached to the underside of the girder, and a piano wire was stretched between the gauge and the jig to measure the displacement when the train passed the gauge output. For non-marker image measurements A and B, cameras were installed 50 and 20 m from the girder measurement point, and the side surface of the girder was examined. The girder displacement was estimated by DIC using the existing contrast of the surface of the measurement point (side surface of the girder). The measurement sampling of the bridge displacement was 1 kHz for the ring-type gauge and the laser, 200 fps for the non-marker A (an industrial camera), and 60 fps for the non-marker B (digital camera). The bridge displacement during the train passage was simultaneously measured by each measurement method. The trains were targeted to run at ~220–260 km/h, and ~4 trains were measured for each up and down passage. The measurements were performed from about 11:00 to 15:00. The weather was sunny and almost calm.

4. Verification Results by Model Bridge

4.1. Measurement Results

Figure 9 shows the displacement responses obtained by non-marker image measurement and contact measurement when the midspan was vibrated at 35 Hz. Here, subset A was set for the model bridge web with a few measurement surface patterns such that the conditions were the strictest. Because the vibration applied was close to the resonance frequency in the primary deflection mode, the displacement amplitude was the highest at Ch3 in the midspan, and smaller at Ch0 and Ch6 near the supports. The maximum displacement amplitude was ±50 μm at Ch3 and the minimum was ±25 μm at Ch0 based on the measured values of the contact displacement meter. Therefore, on the measured image, the Ch3 measurement was a maximum of ±0.035 pixels, whereas the Ch0 measurement was a minimum of ±0.017 pixels.

The displacement responses obtained by non-marker image measurement and contact displacement measurement were similar at all measurement points. However, a long periodic error was confirmed, such that the maximum amplitude fluctuated in the non-marker image measurement. These errors may be attributed to the vibration transmitted through the laboratory floor, which shook the tripod or camera. Moreover, the responses of Ch2, Ch4, and Ch5 of non-marker image measurement were mediated by random noise; the conversion of these errors from the maximum amplitude was ~20 μm. In Ch6, the displacement amplitude of the non-marker image measurement was smaller than that of the contact measurement, and random noise was observed.

To examine the effect of lens aberration and position in the image on these measurement errors, the number of pixels at each measurement position was converted from the I-digit height at each measurement point; however, the error due to aberration did not exist at all points. Thus, the error was not attributed to the position of the object in the image. Furthermore, as in the case of Ch6, Ch0 is located at the edge of the image, and has a small measurement error. This confirms that the error is not specific to the object position in the image. Another effect is that the brightness distribution in the measurement area is biased toward a certain value and does not have sufficient variance. In particular, Ch6 appears to be white in Figure 6, and so-called overexposure occurred. Therefore, when estimating the displacement amount, its measurement error is considered to be high because of the characteristics of the overexposed part due to reflection, in addition to the surface pattern peculiar to the structure. Thus, in practical application, it is necessary to predict the overexposed areas in advance, exclude them from the measurement points, and perform interpolation using the peripheral measurement points.

4.2. Measurement Accuracy

Figure 10 shows examples of the accuracy verification of the non-marker image measurement when the midspan was vibrated at 35 Hz. The horizontal axis is the displacement of the contact measurement, whereas the vertical axis is the displacement of the non-marker image. Hence, the straight line with a slope of 1 indicated by the black line in Figure 10 has an error of 0. Moreover, the regression line of the measured value and the reliability interval of ±2σ is shown by the red bold and dotted lines, respectively.

The regression line at points other than Ch6 is similar to the straight line with a slope of 1. In this study, the pixel displacement was converted from the height of the H steel and the number of pixels at the relevant location in the image. Because the effect of this simple method on the measurement accuracy is insignificant, the absence of any practical problem is confirmed. When the effect of overexposure is significant, as in Ch6, the error of the displacement amplitude obtained by the non-marker image measurement is large; therefore, an apparent error occurs in the pixel-displacement conversion.

The reliability interval of ±2σ represents the random error included in the non-marker image measurement. The reliability intervals of ±2σ were about ±20 μm (0.014 pixels), with the exception of Ch6 in the overexposed area. Furthermore, the amplitude dependence of the random error was not confirmed; thus, the effect of pixel locking is small. Moreover, it can be confirmed that the error is characterized by white noise. However, as shown in the previous section, Ch6 has a larger random error than the other measurement points because of overexposure.

Figure 11 shows the standard deviation σ (average value of the absolute error) obtained from the regression analysis of the non-marker measurement error for the contact measurement at each measurement point. In Ch4, there was a problem with the contact displacement meter during measurement; therefore, it was excluded from the evaluation. The standard deviation, σ, is about 0.004–0.007 pixels, with the exception of Ch6, which is the overexposed region. About 95% of the data is in the range of ±2σ for a normal error distribution. Therefore, if ±2σ or more is considered as the measurement range in which the noise and signal can be separated, the limit of non-marker image measurement is considered to be 0.015 pixels for the single-sided amplitude and 0.03 pixels for the double-sided amplitude. However, it is necessary to set the measurement points to avoid overexposed regions such as Ch6.

Figure 11 shows the results of high-pass filtering with a cutoff frequency of 2 Hz on the measured waveform to eliminate the long-period fluctuations observed in the displacement responses of the non-marker measurement. Although the measurement error of the filter response is reduced compared to the original responses, the amount of reduction is ~0.002 pixels, and no significant reduction effect can be observed. Hence, as discussed in the previous section, most errors in the non-marker image measurement are white noise and are difficult to eliminate by simple filtering. However, the above consideration is for a minute error of about ±10 μm (0.007 pixels), and no problem exists in its application to bridge displacement measurement.

4.3. Measurement Results with Minute Amplitude

Here, the measurement results of the displacement amplitude below the measurement limit are shown.

Figure 12 shows the displacement responses obtained by non-marker image and contact measurements when the bridge L/4 point is vibrated at 120 Hz. Because the vibration was applied close to the resonance frequency in the secondary deflection mode, the displacement amplitudes of Ch0 and Ch1 are large, and the displacement amplitudes of the interstitial centers Ch2 and Ch3 are small. Based on the contact measurement values, the displacement amplitude was ±4 μm at the maximum for Ch1 and ±2 μm at the minimum for Ch2. Therefore, small vibrations with a maximum of ±0.003 pixels for Ch1 and a minimum of ±0.0015 pixels for Ch2 on the measured image were obtained.

Figure 12 shows that the non-marker image measurement contains a large noise component, making it difficult to confirm the shape of the original responses. The magnitude of the noise varies slightly depending on the measurement point; it is at most ±20 μm (0.014 pixels), which is consistent with the result of the vibration at the midspan of 35 Hz. However, there are parts at which the vibration cycle is similar to the contact type result; although it is below the measurement limit (0.03 pixels), it may be useful for evaluating the predominant frequency in the frequency domain.

Figure 13 shows the frequency spectra of Ch0 and Ch1 obtained by the 35 Hz excitation at the midspan and the 120 Hz excitation at the L/4 point. The non-marker image measurements shown in blue in the figure have flat frequency characteristics, with the exception of the peak that occurs at the excitation frequency. Therefore, it can be confirmed that the mixed error is mostly white noise. The peak at 35 Hz can be satisfactorily evaluated with a midspan of 35 Hz excitation. However, for 120 Hz excitation where the vibration amplitude of the original waveform is small, the amplitude is less than the noise of the non-marker image measurement in most frequency bands. Moreover, it can be confirmed that the peak amplitude slightly exceeds the noise level near 120 Hz, which is the vibration frequency. Therefore, it may be possible to evaluate the predominant frequency even if the displacement amplitude is about ±0.003 pixels, which is an extremely small level of pixels.

4.4. Discussion for Practical Application I: Camera Distance

The measurement accuracy of the midspan 35 Hz excitation was 0.03 pixels at a distance of 2.5 m from the camera to the object to be measured; the displacement was 44 μm. Assuming that the measurement accuracy in pixels does not vary with the measurement distance (i.e., the influence of distance attenuation and air fluctuation is insignificant), the measurement limit displacement at any measurement distance x (m) can be calculated as 0.0176 × x (mm). If bridge displacement when the train passes is larger than this measurement limit displacement, the non-marker image measurement examined in this study can be applied. Here, the feasibility rate is the probability that the measured bridge displacement will exceed the measurement limit displacement.

Figure 14 shows the relationship between the maximum displacements and the bridge span [28] used for calculating the feasibility rate. We observed all of the 246 bridges constructed for high-speed railways. The maximum displacement value at low speed was used as a representative value in each case. The span length of the bridges in Figure 14 was 10 to 110 m, and the measuring distance was set every 5 m from 5 to 50 m. The feasibility rate was computed as the proportion of bridges whose maximum displacement exceeded the measured limit displacement calculated at 0.0176 × (mm) for each span length.

Table 7. Number of bridges and the span length used in the calculation.

Bridge Spans	Bridge Number	Bridge Spans	Bridge Number
Less than 10 m	4	30–40 m	66
10–20 m	13	40–50 m	54
20–30 m	44	Over 50 m	65

shows the number of bridges used for the trial calculation in this study for each span length.

Figure 15 shows the feasibility rate calculated from the measured bridge and the limit displacements in Table 7 and Figure 14. Figure 15 shows that the bridges with a span length of 20 m have a feasibility rate of ~90% or more at measurement distances up to 50 m. However, because the bridge displacement is small for bridges with a span length of <20 m, it is necessary to provide a measurement distance below 20 m. If a bridge with a short span length must be measured from a distance of 20 m or more, it may be necessary to increase the displacement per pixel using a telephoto lens. However, the above consideration disregards the influence of distance attenuation and fluctuation, and separate verification is required when the measurement distance is long; this is a matter for future research.

4.5. Discussion for Practical Application II: Effect of Weather (Illuminance)

In the practical application of non-marker image measurement, environmental conditions are a major concern and influence measurement accuracy. Here, the authors focus on the brightness of the shooting environment and consider this environment from the perspective of illuminance based on the results of experiments on model bridges.

Figure 16 shows the vibration test setting of a model bridge used to examine the effect of illuminance. The illuminance was varied in this test by turning off the LED light located on the right-hand side of the figure, which was one of two LED lights used as lighting. Figure 17 shows the illuminance when the LED light was on and off at each measurement point. The illuminance exhibited certain errors because it was converted from the brightness on the captured image through the illuminance performance of the LED light. In fact, although Ch0 and Ch2 at LED OFF have higher illuminance than at LED ON, this is an error due to the effect of conversion from brightness to illuminance. However, it is similar to the measured value with the illuminometer, and no problem was encountered in making a rough evaluation of the brightness of the surrounding environment. When the LED light was on, an illuminance of ~1000–3000 lx was obtained at all measurement points. By turning off the LED light, the measurement environment was set to 800 lx for Ch3 and under 500 lx for Ch4–Ch6.

Figure 18 shows the influence of turning the LED light on and off on the accuracy of the non-marker image measurement. The result of turning the LED light on was the same as that shown in Figure 11, and the standard deviation was ~0.004–0.007 pixels, with the exception of Ch6. However, when the LED light was off, a large error occurred in Ch4–Ch6, where the illuminance was <500 lx. Because the highest displacement was between 0.035 and 0.017 pixels, it is safe to assume that, on average, an error of the same magnitude as the original waveform amplitude occurred on Ch4–Ch6, thus making measurement difficult. Because the precision of Ch3 was nearly identical to that when the LED light was on, 500 lx was used as a guideline for the non-target image measurement.

Table 8. Guideline for general illuminance and brightness [29].

Situation	Time	Illuminance (klx)	Applicability
Outdoor/Sunny	Noon	100	○
	10 AM	65	○
	3 PM	35	○
	1 h before sunset	1	○
Outdoor/Cloudy	Noon	32	○
	10 AM	25	○
	1 h after sunrise	2	○
	Sunrise/sunset	0.3	×
Under the street light	Night	0.05–0.1	×
Moonlight	Night	0.0005–0.001	×

shows a guideline for general illuminance under each climatic and environmental condition in Japan (Ref. [29]). The table shows the applicability of the non-marker image measurement for illuminance. If 500 lx is used as the applicability guideline, the illuminance may be insufficient immediately after sunset or just before sunrise; however, it can be measured even if it is sunny or cloudy during the day. In addition to illuminance, the application in rainy weather conditions is challenging because of the effects of raindrops moving into the image and the waterproofness of the equipment; these concerns will be addressed in the future.

5. Verification Results of Field Tests

5.1. Measurement Results of Bridge A

For bridge A, the bridge midspan during the passage of six- and two-car trains was obtained by non-marker image and laser measurements. As per the estimation results, it is difficult to apply the non-marker image measurement to some bridges with a camera distance of 16.5 m.

Figure 19 shows the measured displacement responses during the train passage. The maximum amounts of deflection were ~5.5 and 7.2 mm when the six- and two-car trains passed, respectively. The measuring distance in this environment was ~9.2 mm per pixel. Thus, the maximum displacement corresponds to 0.6–0.8 pixels, and sufficient displacement occurs compared with the measurement limit accuracy (0.03 pixels in total amplitude). The results of the non-marker image measurement correlate with those of the laser measurement.

Table 9. Maximum displacements of bridge A.

Methods	Tests	Maximum Displacements (mm)	Errors (%)
Laser measurement	Six-car train	5.47	-
non-marker image measurement		5.64	3.1
Laser measurement	Two-car train	7.16	-
non-marker image measurement		7.06	1.4

shows the maximum displacements obtained by each measurement system during the two tests. Moreover, the error of the maximum displacement of the non-marker image measurement, using the laser measurement as the correct value, is shown. The error of the maximum displacement when the train passed was ~3% or less relative to the laser measurement. Therefore, under the current measurement conditions, it can be confirmed that the maximum deflection can be sufficiently evaluated by non-marker image measurement.

Figure 20 shows examples of calculating the deflection shape of bridge A during the train passage by taking advantage of the non-marker image measurement. Here, 44 measurement points were placed in the longitudinal direction of the bridge, and the deflection at the time of the train passage was calculated at each measurement point. The deflection shape varies based on the bogie position of the passing vehicle. Because the texture of the measurement surface is relatively clear at all points on the steel bridge, the dynamic behavior of the girder can be spatially captured by calculating the deflection shape in this manner. Since the bridge is a simple support girder on which the train runs at a relatively low speed, the bending behavior when the train passes is relatively simple, and depends only on the bogie position. These applications may be useful for understanding the dynamic behavior of continuous girders and cable-stayed bridges having more complex structural forms.

5.2. Measurement Results of Bridge B

Vertical displacements of bridge B under two, up- and down-bound, eight-car trains were obtained using a ring-type gauge, laser measurement, and non-marker image measurements A (industrial video camera) and B (compact digital camera). The compact digital camera is not an industrial camera but a general consumer product, and the displacement was measured using its video recording function (See Table 6 for the equipment details).

Figure 21 shows the displacement responses during the train passage using each method. Bridge B is a two-span continuous pre-stressed concrete rigid frame bridge. Thus, an uplift that causes displacement on the upper side occurs when the train passes on adjacent spans. Further, the timing is reversed for up- and down-bound train passages. Rach figure shows, in addition to the entire views, magnified images of the displacement measurement results when the maximum and minimum displacements occurred.

Figure 22 shows the maximum displacement of bridge B. The maximum displacement was slightly larger during the passage of the down train. In the non-marker image measurement A, an error of about −0.8 mm occurred in the case of down train 1. Figure 21b shows the measured response, and it can be confirmed that spike-like noise is accidentally duplicated when the maximum deflection occurs (~8.9 s). Moreover, the entire displacement response of non-marker image measurement A, in this case, tends to shift downward. Therefore, it may be possible to reduce these errors after measurement by performing smoothing and drift correction. This post-processing is considered in the next section. With the except of the above cases, it was possible to evaluate the maximum displacements of a concrete bridge with a relatively smooth surface with an error of ±0.3 mm or less compared to the ring-type gauge, even using the non-marker image measurement under the current conditions.

5.3. Discussion of Displacement Measurement of Bridge B (Concrete Bridge)

Figure 23 shows the measurement position on the side of the girder using the compact digital camera for non-marker image measurement B. Here, the effect of the contrast at a subset position on the image measurement is discussed using three locations with significantly different contrasts as an example. The subset point I in the figure is the target area for evaluating the influence of the contrast, and exhibits a clear high contrast in black and white. The subset point II is the location used for image processing of non-marker image measurement B of bridge B, and is the boundary between the side surface and the bottom surface of the girder. Subset III is the concrete surface on the side of the girder, having an unclear black and white and low contrast.

Figure 24 shows the displacement responses of bridge B during the train passage, which were calculated for each location. It can be confirmed that the error increases with a decrease in contrast. To investigate the increase and decrease in the error due to the high and low contrast, the standard deviations were calculated for the section where the original displacement was 0, except when the train passed, and the magnitude of the errors was arranged by the 2σ values.

Table 10. Errors (2σ value) of each measurement subset position with different contrasts.

Subset Position	I	II (Reference)	III
Contrasts	high	-	low
2σ values	0.23 mm	0.46 mm	1.16 mm
Variations to the reference II	49%	100%	252%
Smoothing (0.1 s moving average)	0.18 mm	0.27 mm	0.79 mm

shows the 2σ values for each subset. The 2σ value of subset II is about 0.5 mm, whereas, in the low contrast area III, the noise more than doubles to 1.16 mm. The 2σ value of the high contrast point I is halved to 0.23 mm.

If the noise included in the result of the non-marker image measurement is random, a certain degree of error can be minimized by smoothing the moving average. However, because smoothing may round the maximum and minimum values, it is suggested that it is used within a maximum of about 0.1 s. When smoothing in a range higher than this, or when the span is short and the high frequency is predominant, it is necessary to focus on the length for calculating the moving average.

Figure 25 shows the result of smoothing with a 0.1 s moving average filter. Table 10 shows the 2σ values calculated in the section where the original displacement is 0, except when the train passes. When the contrast of subsets such as II and I is high, the noise reduction effect tends to be high. However, at subset III, an error in the 2σ value of about 0.8 mm remains, even after smoothing. From the above, it can be observed that the capacity to improve the accuracy by smoothing is limited.

As mentioned above, the improvement of the measurement surface contrast significantly contributes to the improvement of the image measurement accuracy. Therefore, if high accuracy is required for measurement, it is desirable to have a marker with clear contrast and structural feature points, even during non-marker image measurement. In particular, for concrete bridges it is useful to embed a marker-like color scheme on the side of the girder during construction, such as using a spacer of a different color from the base when placing the concrete.

6. Conclusions

In this study, the measurement accuracy of railway bridge displacement obtained by the non-marker image measurement method was investigated. The main findings obtained are shown below.

• The multipoint displacement measurement accuracy of non-marker image measurement was verified by exciting a model bridge at seven measurement points and exciting minute displacements with a vertical amplitude of ±25 to 50 μm (±0.017 to 0.035 pixels). Consequently, it was established that displacements greater than 0.03 pixels can be quantified using the non-marker image measurement by avoiding the overexposed areas.
• From the measurement results with varying illuminance, it was estimated that the illuminance required to apply the non-marker image measurement is ~500 lx, and the proposed method can be applied during the daytime.
• The displacements during train passages of the steel bridge A (span length 20 m), which had a clear contrast at the image measurement point, were measured using non-marker image measurement and compared with the result of laser measurement. It was confirmed that the non-marker image measurement can be used to estimate the displacement response at the bridge midspan with the same accuracy as that of other measurement approaches.
• The displacements during train passages of the high-speed railway bridge B (continuous concrete bridge), which had an unclear contrast, were measured using two types of non-marker image measurement and compared with the results obtained from the ring-type gauge and laser measurement. Consequently, the measurement error of the maximum displacement was ~0.3 mm at a camera distance of ~20 m from the bridge.
• By examining the error reduction effect of the subset position and moving average processing when the contrast of the image measurement point was not clear, the error was halved by setting the subset on the side of the bottom boundary with a clear texture and using 0.1 s moving average processing.

Of the methods proposed in this study, the use of a compact digital camera has already been widely employed, not only for bridge surveys on Japanese railways, but also for surveys of vibration of tracks (rails and sleepers) and railway poles [27,30,31].