Holographic Wide-Angle System for Deformation Measurement of Extended Structures

Abstract

We introduce a holographic wide angle system that combines the accuracy of a long focal length with the extended field of view of a wide angle lens. To accomplish this, we use a computer-generated hologram (CGH) in front of the lens to diffract light from (a discrete number of) specific angular locations. This method is tested in laboratory conditions, as well as under real-world conditions. This measurement system was developed as a possible tool for real-time movement tracking and control of extended dynamic structures, such as bridges and high-rise buildings. Within that application, the obtained measurement uncertainty is 10 μ$\mathrm{m}$ in object space at 10 $\mathrm{m}$ distance spanning 10 $\mathrm{m}$ width.

1. Introduction

The measurement of positions is a main goal for many vision-based measurement applications. Within that domain, the measurement of deformations of extended structures, especially bridges and high-rise buildings, is of great interest in civil engineering. Large deformations are warning signs for deficiencies in the structural integrity [1,2]. Adaptive structures usually come with the necessary sensory equipment already integrated [1,3,4]. However, there could be a large number of structures which could benefit from an installation of deformation sensors, even though they are not adaptive and were not meant to be equipped with them.

Especially for adaptive systems, which are feedback controlled, it is imperative to achieve high spatial accuracy (sub-mm), high temporal accuracy ( 100 $\mathrm{Hz}$) as well as low latency (few $\mathrm{m}$$\mathrm{s}$) for multi-modal oscillation detection/measurement.

Time-of-flight sensors typically have distance accuracies of 0.1–1 of the measuring distance, but not better than one millimeter [5]. Furthermore, for a large field of view (FOV), the scanning method results in either low lateral resolutions or slow refresh rates (a few frames per second). Total stations and terrestrial laser scanners are far more common, especially when used in the construction of buildings and bridges. Nowadays, they come with absolute distance meters, which offer sub-$\mathrm{m}$$\mathrm{m}$ longitudinal and a few $\mathrm{m}$$\mathrm{m}$ lateral accuracy over a 100 m distance [6]. Yet, they also lack high refresh rates for large FOVs. Interferometric methods can be quite accurate but are costly, slow and not feasible for continuous and fast measurement over a long time. In general, the scanning-based methods presented above are not applicable if fast movements (e.g., vibrations) of extended structures are to be continuously monitored.

Photogrammetric measurement systems easily meet the requirements for high temporal resolution and high lateral accuracies [7,8]. Naturally, temporal or spatial averaging can further improve accuracy, but air turbulence limits the maximum achievable accuracy to about $0.05$ pixel in image space [9].

Even photogrammetric measurement systems are limited to a low accuracy when using a small focal length for investigating large scenes. The following sections are focused on eliminating this shortcoming for a discrete number of measurement points.

2. Image Replication and Holographic Lens

When high accuracy is required for photogrammetrically measuring or monitoring a remote position, the first logical step is to employ long focal length imaging systems. As a result, the magnification is large. However, for a given sensor size, the field is reduced. For the measurement of deformations of large structures, this is a problem since one is interested in the relative displacement of points on the large structure.

One possible approach is to use sensor networks, e.g., consisting of multiple cameras, which are calibrated with respect to each other [10]. In practice, however, it is difficult to maintain (extrinsic) calibration of the cameras. Even extremely slight angular changes lead to large measurement errors for deformation measurements.

Another approach is to avoid this problem by using a short focal length in combination with extremely accurate image processing. We have shown such an approach in previous publications [11,12,13,14] where we employed computer-generated holograms (CGH) in order to achieve accuracies in the range of hundreds to thousands of a pixel.

As an illustration of the problem, an extended structure (bridge) is to be monitored at three measurement points. As measurement points, LED or laser emitters are used, which are placed at a large horizontal distance to each other at one side of the structure (beginning, end, and middle of the bridge). The imaging system (achromat with long focal length and camera) is positioned at a large distance in the center of the front side. Figure 1a) shows that due to the long focal length, only the light coming from the middle emitter—which is located on the optical axis—can be imaged on the sensor.

To circumvent the problem of imaging just a small section of the bridge onto the sensor, a CGH is used to image the additional axially distant emitters onto the sensor. For this purpose, the CGH is put in front of the achromatic lens as shown in Figure 1b). The incident peripheral beams are diffracted at the CGH to become approximately on-axis beams. This enables the imaging of (a limited discrete number of) distant image sections. Therefore, high-resolution monitoring of measurement points by a single imaging system becomes possible.

The CGH is optimized using an iterative Fourier transform algorithm (IFTA) to diffract light to a limited number of points [15,16]. Obviously, if there is a total of three emitters, the CGH must produce −1st, 0th and 1st order diffraction for light coming from each emitter. For an ideal CGH (100% diffraction efficiency, 2/3 of the light would be lost into unwanted diffraction orders. This situation becomes worse the more LEDs or image sections are to be imaged on the sensor, and one has to design the system such that the unwanted diffraction orders will not overlap with the desired orders, and the stray light is not disturbing.

3. Experimental Setup

The imaging system (shown in Figure 2) consists of the CGH, a $f^{\prime }=$ 200 $\mathrm{m}$$\mathrm{m}$ tube lens, a narrow band-pass filter (at $\lambda =$ 532 $\mathrm{n}$$\mathrm{m}$ with FWHM = 2 $\mathrm{n}$$\mathrm{m}$) and a FLIR USB3.0 Grasshopper 3 monochrome image sensor (GS3-U3-23S6M-C: 2.3 MP, 163 FPS, SONY IMX174). The band-bass filter is used to reduce unwanted background light. The FOV of the camera, considering the magnification and sensor size, is $3.2^{\circ }\times 2.0^{\circ }$, or $660\mathrm{mm}\times 350\mathrm{mm}$ at 10 m distance. Adding the CGH replicates this FOV to specific angular locations reaching $\pm {25}^{\circ }$. This widens the (discrete) FOV by a factor of $15.6$.

For the laboratory setup we use laser diodes as emitters (Laserfuchs LFD532-1-3(12 × 60)-001 with 5 $\mathrm{m}$$\mathrm{W}$ optical power) housed inside an aluminum box in combination with a planoconvex lens (LA1540-A-ML) to realize a light emitting cone. One of the emitters is mounted on a motorized labjack from Thorlabs (MLJ150) which moves upwards and downwards as shown in Figure 3 in order to simulate deformations. The other two emitters are positioned $\pm {25}^{\circ }$ apart, which is ≈$\pm 5$ m.

The captured images are processed using the open-source software package, ITOM [17]. For each spot, the center of gravity (COG) is computed—as described in [8]—where a threshold t is subtracted to reduce noise, and the values (after subtraction) are limited to positive values:

$$I^{\prime }(x_i,y_j)=\left\{\begin{array}{c}\begin{array}{cc}\hfill I\left(x_i,y_j\right)-t,& \mathrm{if}I\left(x_i,y_j\right)>t\hfill \\ \hfill 0,& \mathrm{if}I\left(x_i,y_j\right)<t\hfill \end{array}\hfill \end{array}\right.\forall i,j$$

(1)

where I is the recorded intensity, and $\left(x_i,y_j\right)$ denotes the image coordinates in pixels.

The COG is computed using

$$x_{COG}=\frac{{\sum }_i{\sum }_j{x}_{i}I\left(x_i,y_j\right)}{{\sum }_i{\sum }_{j}I\left(x_i,y_j\right)}{y}_{COG}=\frac{{\sum }_i{\sum }_j{y}_{i}I\left(x_i,y_j\right)}{{\sum }_i{\sum }_{j}I\left(x_i,y_j\right)}$$

(2)

Figure 4 shows the upwards and downwards movement over the time of the light emitter mounted on the stage. To better enhance the visibility of the positional uncertainty of each measurement, the core movement of the stage can be fitted by a third-degree polynomial. The nonlinear movement of the stage due to acceleration and deceleration is visible. If one subtracts this core movement, the residual variation is the measurement uncertainty.

The calculated measurement uncertainties $\sigma$ of the individual segments range from 0.036pixel–0.042pixel, which are within the range of achievable sub-pixel accuracies discussed by Feng et al. [18]. By using the magnification equation and considering the pixel size of 5.86 µm, the positional accuracy can be calculated from the sensor accuracy without further calibration since the relative spot movement is within a few pixel. This leads to an object-sided measurement uncertainty of 10 μ$\mathrm{m}$. The resulting measurement accuracy was the best that could be achieved considering the accuracy of the stage and the visible air turbulence.

4. Outdoor Test

To further evaluate the scalability of our approach, we conducted our outdoor experiment with five spots in total, where the outermost ones are imaged onto the sensor via their second diffraction order. Due to eye safety considerations, the laser emitters were not usable anymore and we utilized Roithner G58A5111P LEDs ($I_V$ = 100 $\mathrm{A}$ at $I_F$ = 50 $\mathrm{m}$$\mathrm{A}$, $U\approx 3.8\mathrm{V}$). Moreover, we observed in other laboratory tests that contamination with dust and dirt—which is common when using equipment outdoors—can cause trouble with speckles.

Figure 5 shows the outdoor setup using a wooden prototype bridge, which is exclusively used by pedestrians and bicycles. LEDs are positioned with a pitch of $12.5^{\circ }$ viewing angle from the camera, ≈2.6 m apart. For the vertical force application, a person with a mass of 85 $\mathrm{k}$$\mathrm{g}$ was positioned above the spot $+1$. To achieve maximum bridge deflection with a single person, the load was applied dynamically by jumping. The deflection—and, consequentially, the oscillation of the bridge—was then measured with the same imaging system we used indoors.

Spots $+2$ and $-2$ which are positioned at $\pm {25}^{\circ }$, cannot be reconstructed and evaluated with the employed exposure time of 5 $\mathrm{m}$$\mathrm{s}$ due to the low diffraction efficiency of the the corresponding orders of the CGH.

Spot 1 (Figure 6) is striking to observe. We find a kind of bisection of the single spot. One possible explanation for this phenomenon is the large magnification combined with the camera focusing the LED. Due to the design of the LED, a two-part spot is created, which reflects the inner circuitry (mainly the anode and cathode) of the LED. The theory seems confirmed when comparing the central spot (0th order) with a macro picture of the LED (Figure 7).

Several test runs and test jumps were performed to obtain different intensities and frequencies of the applied force (shown in Figure 8). It is possible that during the damped oscillation, further force was applied. The nearly exact overlap of the y-positions of the single spots at the force application times and the lack of reaching the rest position support the thesis.

Spot $+1$, which is located directly below the position where the force is applied, experiences the largest positional shift. In line with this, spot $-1$, which is located at the greatest distance to the applied force, experiences the smallest displacement.

The field test shows that a relative position deviation can be measured and evaluated with the presented system. In order to improve the comparability of different bridge excitation frequencies and to enable the calculation of the resulting relative position deviations, a defined mechanical, periodic force application should be used for future tests.

Due to the periodic signal waveform in Figure 8 at a high bridge excitation frequency, the spectrum of the excited pedestrian and bicycle bridge can be determined by the fast Fourier transform (FFT), shown in Figure 9. In this way, individual frequencies acting on the structure can be analyzed.

The highest peak at $6.5$ $\mathrm{Hz}$ and a second visible peak at $2.5$ $\mathrm{Hz}$ probably represent the two fundamental oscillation frequencies of the bridge, where the first one has a significantly lower amplitude than the highest peak.

5. Conclusions

We demonstrated a new method for a camera-based measurement of slight deformations of large extended structures. The core idea is to use a computer-generated hologram in combination with conventional imaging to realize long focal-length imaging with a large field of view. Compared to conventional imaging-based measurements, the field-related accuracy can be strongly increased using a very simple extension of a conventional camera (additional CGH). Compared to sensor networks consisting of multiple cameras, the cost is strongly reduced, the system is much more stable, and continuous long-term operation becomes possible. One important limitation for the proposed method is the inherent need for light emitters on the structure to be measured. Only clearly separated light emitters can be tracked with high accuracy, so the method is not applicable for measuring whole field deformations, e.g., deformations for each pixel of the image sensor.

The basic functionality of the presented system was demonstrated under laboratory conditions using laser emitters and on a real bridge using LEDs. Especially for adaptive systems, the relative position measurement is accurate enough without the need for extensive calibration as with absolute measurement. To achieve stability within a closed-loop feedback system, it is not important that sensory data are perfectly calibrated. Even if a displacement is 1% off, if the displacement is in the order of millimeters, the error is negligible. Finally, measurement uncertainties of 10 μ $\mathrm{m}$ for a 10 $\mathrm{m}$ structure at a distance of 10 $\mathrm{m}$ could be achieved at high temporal resolution using a standard monochrome image sensor. This corresponds to an angular measurement uncertainty of 1 µrad.

In the future, the method may be combined with the multi-point method [9] to further increase the accuracy.