Indoor and Outdoor Surface Measurement of 3D Objects under Different Background Illuminations and Wind Conditions Using Laser-Beam-Based Sinusoidal Fringe Projections

Measurement of 3D Objects under Different Background Illuminations and Wind Conditions Using Laser-Beam-Based Sinusoidal Fringe Projections. Abstract: In this study, both theoretical analysis and experimental validation are carried out for 3D surface measurement under different indoor/outdoor environmental conditions via combining the projected laser-beam-based sinusoidal optical signal, the optical ﬁltering technique, and the single-shot approach based on Fourier transform proﬁlometry. The designed optical signal generator used in this work is capable of ensuring that the projected fringe pattern is monochromatic, higher-contrast, time-invariant, and truly sinusoidal. The proposed and developed optical setup of 3D surface measurement is portable and is used for in-situ experiments of 3D surface measurements that have been carried out under different sunlight illuminations. The experimental results indicate that accurate reconstructions of measured objects with even or varying surface reﬂectivity can be obtained under windy conditions and strong environmental illuminations such as the background illuminance of 5600–35,000 Lux. The generated fringe-pattern signal is not sensitive to vibrations from environmental inﬂuences including the effects of the wind, which has overcome the outdoor-measurement restrictions of the traditional interferometric system and the proﬁlometry approaches based on phase-shifting methods.


Introduction
There has been a demand of methods and technologies for non-contact and highresolution shape measurement of three-dimensional (3D) objects, which is expected to have applications in in-situ industrial inspection of manufactured parts, target recognition, corrosion analysis, measurement of surface roughness, and biomedical applications such as 3D intra-oral dental measurements and non-invasive 3D imaging and monitoring of vascular wall deformations [1,2]. 3D surface measurement using the structured light techniques [3][4][5] has been attracting increasing attention in the field of optical profilometry and related applications. As indicated in recent review articles [2,6], one of the challenging issues in this field is still the development of new fringe-projection technology and hardware, for improving the accuracy, resolution, and speed of a 3D-shape imaging system.
In the diagram of a typical system with structured light, there are two basic parts: one part is the projector for generating structured illuminating patterns onto the observed object, and the other part is a camera for measuring distorted structured patterns on the target, while the projection of structured light patterns has been considered as the critically important part. For generating the structured signal of projecting light, there have been some useful and successful approaches [7], such as the interferometry with monochromatic light [8][9][10][11], projection using grating with white light [12], the acoustooptic fringe projection [13][14][15], the encoded projection techniques [16], the approach with a sinusoidal phase grating [17], and the DMD-chip-based digital-light-processing (DLP) projector [1,2,18]. However, the structured optical signal generated using interferometry is very sensitive to vibrations from environmental influences including the structural vibration of the system and the air disturbance, which can weaken the temporal coherence, and yield low-contrast sinusoidal signal since it is difficult to adjust the spatial frequency and obtain a high-spatial-frequency signal. For the method using white-light grating, it usually uses Ronchi grating for which it is difficult to obtain the sinusoidal structured signal. Note that the white-light grating method is basically unable to complete a dynamic and real-time measurement with adjustable spatial frequencies. The higher spatial frequency can be obtained using the acousto-optic approach, but the generating system is complicated compared to others. Noises are possibly introduced when the encoded projection is used in profilometry. The spatial frequency of the sinusoidal structured signal generated by a DLP projector is usually lower, while the illuminating area is also relatively limited.
Even sinusoidal patterns generated with a DLP projector are widely used, it does have its limitations such as the requirement of precise synchronization, speed limit of measurement, and the nonlinear gamma effect of the DLP projector [19]. Note that the laser-based DLP projector still uses a DLP chipset, but its light source is laser-based instead of light-based, while the laser-based DLP projector also has the drawbacks of a typical DLP projector. Thus, there has still been a very challenging need for developing methods that can generate sinusoidal fringe patterns for improving the 3D surface profilometry accuracy.
Moreover, 3D surface measurement under outdoor environmental conditions has been extremely challenging, since the available methods are unable to go through the restrictions of vibrations from environmental influences and effects of background illumination from sunlight. Meanwhile, for another challenging issue of carrying out 3D surface measurement of objects with a wide range of reflectance [20], there have been some recently developed techniques such as the dynamic projection theory [21], the approach utilizing the transitioning state of digital micromirror device [22], and the method with a dual optical sensing configuration via combining the diffuse reflective light and the specular surface light [23]. However, these techniques might only be allowed for applications of indoor measurements or under dark environmental conditions. Generally speaking, the most existing techniques basically suffer the main drawbacks as follows: First of all, phase-shifting-based methods and techniques have a rigorous requirement that the observed target has to remain still during the measuring time, thus vibrations caused by the surrounding environment such as the wind or the observed target itself such as the human breathing during a medical scan will prohibit the practical measurements. Next, if the projected pattern is based on the sinusoidal fringe generated by DLP projector, the intensity deviation of the sinusoidal-fringe curve affected by the Gamma effect and defocusing issue will result in errors of target measurement. Finally, for structured-pattern projections with white light, background illumination is mostly not allowed during the measurement, which indicates that outdoor in-situ measurements of usual objects will be impossible.
Therefore, the purpose of this work will focus on the introduction and analysis of the designed optical system for generating laser-beam-based sinusoidal optical fringe patterns and the experimental measurements and reconstruction of objects for both indoor and outdoor conditions. Note that, in addition to the object surface with the same material and isotropic reflectance, we will apply our approach for measuring the surface shape of object with a wide range of reflectance under different background illuminance, which is based on the combination of the laser-beam-based sinusoidal optical fringe patterns generated using our designed optical system, Fourier transform profilometry (FTP), and the optical filtering processing technique.

The Designed Optical System and Theoretical Analysis
An important part of this work is to design and develop an optical system for the sinusoidal optical signal generator. The basic goal of our design is twofold: the first objective is to obtain monochromatic, high contrast, high quality, and truly sinusoidal fringes, and the second one is to overcome the shortcomings of the traditional interferometric system. We know that a interferometric system is usually complex and sensitive to vibrations from environmental influences including the structural vibrations of the operating system and air disturbances from the surrounding environment, which have imposed restrictions on the applications of outdoor and production line in-situ measurement. Figure 1 is the photo of experimental setup of in-situ measurement under the sunlight, while the parts S, G, L, and F in Figure 1 are the main components of the sinusoidal optical signal generator. The optical system starts with a CW-laser source S with the wavelength λ = 532 nm, maximum output power of 200 mW, and a small divergence angle (≤ 1.5 • ). A rectangular grating G illuminated by the laser beam is placed at a front focal point of a positive lens L. On the conjugate plane of S, an adjustable spatial-frequency filter F is set, which is considered that the part F plays a critical role in this system. For this spatial-frequency filter, a V-shape aperture with a width of 0.5 mm is carved with high precision, we can then choose and allow the selected order of the spatial frequency to pass through. The output of a sinusoidal fringe pattern can then be observed on the plane P, as indicated in Figure 1. Note that the design of this adjustable spatial-frequency filter F containing three accurately-cut slits is actually based on our previous work of measuring modulation transfer function [24] and a recent study on human face imaging [25].

P
F L G S C Figure 1. Photo of in-situ measurement under the sunlight via using the projection of sinusoidal fringe patterns generated by the designed optical system. C-Camera; F-spatial-frequency filter; G-grating; L-lens; P-observation plane; S-CW laser source.
Since the focus of this investigation is the indoor and outdoor surface measurement of 3D Objects under different background illuminations and wind conditions, there will be two environmental parameters, the background illuminance E, and the wind speed v wind , to be measured. For both indoor and outdoor measurements, the real-time background illuminance E was measured using a digital luxmeter placed beside the observation plane P. Note that the digital luxmeter should not be illuminated by the signal of the projected fringe pattern when the background illuminance is being measured. The wind speed of indoor measurement is set to zero. For outdoor measurement, the real-time wind speed v wind of the measuring period was monitored and measured using a digital anemometer.

Mathematical Description of the Sinusoidal Optical Signal Generator
The theoretical description of the sinusoidal fringe pattern generated by the optical system described in Figure 1 is given as follows. A schematic diagram of Figure 1 can be sketched using Figure 2, which will be used to derive a mathematical expression for the sinusoidal fringe pattern obtained at the observation plane P in Figure 1, i.e., the plane P 3 in Figure 2. For other planes in Figure 2, the grating G is located at plane P 0 , the positive lens L is at plane P 1 , and the plane P 2 is the position of the spatial-frequency filter F. The filed U(r) from a continuous-wave point source laser S can be described as where A is a complex constant related to the laser power, and r = x 2 + y 2 + z 2 . Taking A = 1 for simplicity and using the paraxial approximation, the field U 0 (x 0 , y 0 ) behind the grating G at the plane P 0 is given by where Z 0 is the distance between the laser source S and grating G, and t 0 (x 0 , y 0 ) the transmission function of G. Considering that G is one-dimensional, t 0 (x 0 , y 0 ) can be described using where x 0 and y 0 are the coordinate variables of P 0 , and a and d are the optical parameters of grating G. H is the grating width, which is assumed, for the sake of simplicity, to be the diameter of the illumination spot of the laser. Note that the propagation of field U 0 (x 0 , y 0 ) from plane P 0 to plane P 1 is within the Fresnel region. The field U 1 (x 1 , y 1 ) in front of the lens L at plane P 1 is given by where e ikZ 1 is a complex constant, and Z 1 is the distance between the planes P 0 and P 1 . The field U 1 (x 1 , y 1 ) after the lens L at plane P 1 can be written as where t L (x 1 , y 1 ) = e −i k 2 f (x 2 1 +y 2 1 ) denotes the transmission function of the lens L. The propagating field U 1 (x 1 , y 1 ) at plane P 1 to U 2 (x 2 , y 2 ) at plane P 2 can be accurately computed using Fresnel diffraction. U 2 (x 2 , y 2 ) denotes the field in front of the spatialfrequency filter F, and is given by where C 2 = 1 iλZ 2 e ikZ 2 is a complex constant, and Z 2 is the distance between the planes P 1 and P 2 . Combining Equations (2)-(6) as well as the expression of t L (x 1 , y 1 ), we have where is the Bessel function of the first kind, ω = 2π f x with the spatial frequency f x at plane P 2 defined by f x = x 2 λ f , and C 3 is a complex constant given by The role of the adjustable spatial-frequency filter F at plane P 2 is to select a ±m-th order spectrum described in Equation (7), and let them pass through it, and here we take m = 1. We then get the field U 2 (x 2 , y 2 ) right behind the filter F in the following form: The propagation of field U 2 (x 2 , y 2 ) from plane P 2 to P 3 can also be analyzed using Fresnel diffraction, which let the field U 3 (x 3 , y 3 ) at plane P 3 to be given by where C 4 = 1 iλZ 3 e ikZ 3 is also a complex constant, and Z 3 is the distance between the planes P 2 and P 3 . Considering that the size of Σ 2 (x 2 , y 2 ) (≤ 4 mm in diameter) is much less than that of Σ 3 (x 3 , y 3 ), i.e., the spot size of the sinusoidal fringe pattern (≥ 100 mm in diameter), we take an approximation that λ for further derivation of Equation (9). Equation (9) then becomes where the complex parameter C 5 is given by . Equation (10) indicates that the field at the observation plane P 3 is an inverse Fourier transform of the field output from the spatial-frequency filter F. Combining Equations (8) and (10), we have Obviously, C 6 is a complex constant for determined spatial distances Z 0 and Z 3 , and the modulus of C 6 represents the amplitude of the generated sinusoidal signal. We see that the fringe intensity will decrease as Z 3 increases since f and Z 0 are generally parameters with fixed values. The part of circ in Equation (11) results in a definition of the range of the fringe pattern with a circle of radius HZ 3 2 f . The term of cos is the key part of output of the designed optical system, i.e., the expression of a sinusoidal curve.

Physical Characteristics of the Generated Sinusoidal Fringe Pattern
From Equation (11), it can be seen that the cosine term in Equation (11) has only one variable x 3 at the plane P 3 along the x 3 -axis, which indicates that the output of this system is monochromatic, high contrast, and truly sinusoidal fringes. The cosine term also includes the information that the fringe width (related to the spatial frequency of the output fringe pattern) will increase as Z 3 increases, which might be useful and important for 3D surface measurement.
The light-intensity distribution of the sinusoidal fringe pattern generated using the developed optical system is measured using a CMOS camera, and the camera uses an Aptina CMOS sensor with the sensor size of 5.70 × 4.28 mm and each pixel size of 2.2 × 2.2 µm. Due to the fact that the laser power is high, the exposure time and the γ value of the camera must be carefully adjusted for obtaining a color picture of the fringe pattern with good quality, as shown in Figure 3a.  The measured light-intensity distribution of the sinusoidal fringe pattern is compared with the standard sine function in Figure 3b . The difference or the error between the measured intensity curve and that of the standard sine function, as indicated in Figure 3b, may have over-estimated the real error value, since the measured intensity distribution is affected by the nonlinear gamma effect of the CMOS camera, dark current, lab background light noises, and other possible items from the measuring instruments and related hardware devices. Figure 3c indicates that the fringe contrast will decrease as the background illumination goes up. However, the optical filter can be employed for removing the effect of background illumination to remain the fringe contrast and improve the accuracy of reconstruction.
Based on Equation (11) and the related derivation process, it should be noted that the output fringe pattern is not affected by the focusing issue at all, which is different from the focusing feature of the fringe pattern generated by a DLP projector, an image with programmed intensity distribution.

3D Measurement Framework with Optical Filtering
For the processing of measured data for the retrievals of phase and 3D-surface height, a typical triangulation framework of 3D surface measurement based on FTP is adopted. However, as indicated by Figure 3c, the fringe contrast will decrease dramatically if the background illumination increases. Considering that the illuminated fringe pattern is monochromatic with wavelength at λ = 532 nm, an optical bandpass filter centered at 532 nm with FWHM being 20 nm is used for removing the effects of background illuminations. Based on the descriptions above, the sinusoidal fringe pattern can be described using where U 3 (x 3 , y 3 ) = U 3 (x 3 ) that is given in Equation (11), and x 3 is the only variable in this equation at the object plane along the x 3 -axis for describing the illuminated fringe patterns. After the optical filtering, the algorithm of phase retrieval and surface-height computation is basically similar to the theoretical analyses of Takeda and Mutoh [3], Rajoub et al. [26], and Maurel and co-workers [27].

Measuring a Coin with Different Background Illuminances
A coin of the diameter 27 mm is selected as the first object to be measured. For indoor measurement, the background illuminance E was adjusted using LED lights, for the cases of E = 0 Lux and E = 1000 Lux, respectively. For outdoor measurement, the wind speed v wind of the measuring period was measured using a digital anemometer and recorded as v wind = 3.2 ∼ 6.5 m/s. The outdoor background illuminance E was adjusted using natural sunlight and an umbrella, for the cases of E = 12,200 Lux and E = 35,000 Lux, respectively, while the weather condition was clear when the outdoor experiments were being carried out as indicated in Figure 1.
A sinusoidal optical fringe pattern generated using the optical setup described in Figure 1 with spatial frequency of 8 lp/mm is used to illuminate the selected coin. Figure 4 shows the illuminated coin and its retrieval for background illuminance E = 0 Lux. We see from Figure 4 that the illuminated coin of Figure 4a is a little brighter than that of Figure 4b, even E = 0 Lux, which is due to the fact that the optical filtering affects the exposure compensation of the camera. For the retrieved coins shown in Figure 4c,d, both yield accurate reconstructions, and it does not indicate any distinct difference, which means that optical filtering is not necessary if there is no environmental illumination. When environmental illumination exists, the illuminated sinusoidal optical signal will be modulated by the noise due to the background illumination, as indicated by Figures 5a and 6a below, or even be almost buried by the noise, as shown in Figure 7a below.  When the background illuminance changes from E = 1000 Lux to E = 35,000 Lux as shown in Figures 5-8, the optical filtering plays a critically important role for realizing accurate retrievals. Figures 4a, 5a, 6a, and 7a are the coin images with illuminated fringe patterns without optical filtering, and Figures 4b, 5b, 6b, and 7b are the coin images with illuminated fringe patterns with optical filtering. The reason that one cannot see the fringe pattern could be that the spatial frequency of the projected fringe pattern is very high (8 lp/mm). Figure 7e,f are the enlarged parts for Figure 7a,b, respectively, which is capable of telling the difference between the images of the object projected with fringe pattern for two measuring conditions:  Figures 5d, 6d, and 7d is the same as in Figure 4d. Even the quality of retrieved image of Figure 7d is slightly lower than other cases, it still shows that the proposed approach is capable of obtaining accurate 3D shape measurement under strong background illumination.

Measuring a Human-Head Model with Different Background Illuminances
We now present the experimental results for measuring a model of human head with a size of 45 mm in diameter. Such a model of human head does have very different geometrical features compared to the object coin described and discussed above. The coin is relatively flat, and its height variation is within a small range. The curvature and surface-height difference will lead to relatively severe fringe distortion, which will affect the accuracy of the image processing algorithm used in this study.
Note that the severe fringe distortion due to height variation within a small range of such an observed object can usually result in the overlapping of the zero-order spectrum and the spectrum of fundamental frequency [28][29][30], which has been solved in this investigation via using a higher spatial frequency of the projected fringe pattern, such as 3 lp/mm that might be 10 times larger than the researchers usually use with DPL projectors.
Similar to the coin measurement, for the indoor measurement of human-head model, the background illuminance E was adjusted using LED lights, for the cases of E = 0 Lux, E = 500 Lux, and E = 3500 Lux, respectively. For outdoor measurement, the wind speed v wind of the measuring period was measured using a digital anemometer and recorded as v wind = 3.5 ∼ 5.7 m/s. The outdoor background illuminance E was contributed by natural sunlight, while the weather condition was clear when the outdoor experiments were being carried out. For all the measurements of both indoor and outdoor situations, a sinusoidal optical fringe pattern generated using the optical setup described in Section 2 with a spatial frequency of 3 lp/mm is used to illuminate the human-head model. Figure 8 highlights that optical filtering is not necessary if the measurement is carried in a dark environment (i.e., E = 0 Lux) since we cannot find any difference between the Figure 8a,b, and their retrievals in Figure 8c,d are also almost identical. However, if environmental illumination appears as shown in Figures 9a, 10a, and 11a, the optical filtering is needed in the measurements for getting accurate reconstructions of the object. (c) This is the retrieval using the measurement of (b), while it is unable to obtain satisfied retrieval using the measurement of (a). (c) Figure 11. Human-head-model retrieval as background illuminance E = 12,000 Lux for outdoor measurement. (a) without filtering; (b) with filtering; (c) This is the retrieval using the measurement of (b), while it is unable to obtain satisfied retrieval using the measurement of (a).
We know that E = 500 Lux is about the perfect value of room/background illumination for reading and writing conditions. When the background illuminance E = 500 Lux, Figure 9d shows that the reconstruction using measurement with optical filtering has the same quality as Figure 8d, while Figure 9c indicates that it is getting difficult to accurately reconstruct the observed object if the optical filtering technique is not used for the background illuminance E = 500 Lux. Up to E = 3500 Lux, Figure 10c indicates that this approach can yield a very good reconstruction via comparing the result of Figure 10c to that of Figure 8d. For the background illuminance of E = 12,000 Lux, a quite strong illumination under sunlight as in the case in Figure 11, we can still get a satisfied reconstruction of the observed object as shown in Figure 10c.

Measuring a 3D-Printed Color Object with Varying Surface Reflectivity
We now take a 3D-printed color object as the observed target for carrying out in-situ measurements and reconstructions. This object has the shape similar to an elongated triangular dipyramid, while it is initially generated using a 3D printer and then coated with the same type of paint but with different colors, as shown in Figure 12. The colordependent surface reflectivity of the 3D-printed color object at 532 nm is clearly displayed in Figure 13. From Figure 13, we see that, if we take the perpendicular reflectivity of the white-color region of the object surface shown in Figure 12 as unity, the reflectivities of the yellow, green, and blue regions of Figure 12 will be 48%, 20%, and 5%, respectively. The measurements and reconstructions of the 3D-printed color object have followed the same procedure as we did with the coin and human-head model above, and the results are shown in Figures 14 and 15.  Figure 15c represents a cross section perpendicular to the ridge of the 3D-printed color object, while the left slope covers the blue, the white, and the yellow regions from the bottom to the ridge, and the right slope is totally located in the blue region. The plot of Figure 15d represents a cross section parallel to the ridge of the 3D-printed color object. This cross section is within the right slope of object as shown in Figure 12 with a maximum height of 4.1 mm, and it crosses the white, the blue, the green, the yellow, and the white regions from one side to the other. Even though the background illuminance is strong (E = 5600 Lux) and the relative reflectivity of the projected fringe signal is very low (such as 5% in blue region), the deviation of the retrieval is very small as indicated in Figure 15c,d.

Discussion
Experimental results presented above show that 3D surface measurement with the proposed approach is capable of being completed under complicated outdoor conditions, such as for the windy and strong-sunlight illuminating conditions, while the approaches using DMD-chip-based DLP projector and multiple projections are unable to fulfill this task. Note that the optical filtering technique is not applicable to DLP-based-illumination approaches since the DLP-based-illumination approach is unable to generate the monochromatic signal for the purpose of 3D surface measurement. Note also that the illuminated sinusoidal signal will not be affected by the so-called gamma effect, and there is no need for precise synchronization between a fringe pattern generator and the camera. The monochromatic feature and the possibly high power of the laser beam can ensure the success of the optical filtering technique for outdoor applications.
It should be noted that the output power of the CW laser source used in this work may put a limit of the highest background illuminance E max for obtaining accurate reconstruction of the observed 3D target. In practice, a new CW laser source with higher output power can be employed, which can definitely result in outdoor in-situ measurements and accurate reconstructions under much stronger background illumination being able to be successfully completed. As far as the safety issue of using high-power laser is concerned, we have to be reminded that the energy of the reaching fringe pattern on the surface of the observed object is much less than the nominal power of the CW laser for two main reasons: (i) The power of the propagating fringe pattern decreases with the square of the propagating distance as indicated in Equations (11) and (12). (ii) The power of the propagating fringe pattern is almost evenly distributed on a 2D circular area perpendicular to the propagating direction, while the diameter of the circular area is of the order of 1 m if the observed object is placed at Z 3 = 10 m in Equation (11). Thus, the laser-power limit with numerical safety criteria will be determined using experimental measurements for different applications and situations.

Conclusions
Both theoretical and experimental results of this work have indicated that the proposed approach of 3D surface measurement is capable of being successfully completed under complicated outdoor conditions. The critical important basis for carrying outdoor 3D surface measurement is the proposed and designed novel sinusoidal optical signal generator as well as the combination of the projected laser-beam-based sinusoidal optical signal, the optical filtering technique, and the single-shot approach based on Fourier transform profilometry. The designed optical system can generate monochromatic, high contrast, and truly sinusoidal fringe patterns with high-accuracy applications to 3D-surface measurement, as we presented in our experimental results of this work. The time-invariant feature of the sinusoidal fringe pattern generated using the designed optical system is important and useful for 3D-surface measurement of both static and dynamic objects. In addition to the higher fringe contrast as discussed above, the fringe pattern is a monochromatic signal, which can be combined with an optical filter to effectively reduce or remove the influence of the environmental noisy light signals as we indicated and discussed in Section 3. Moreover, the fringe-pattern signal is not sensitive to vibrations from environmental influences including the structural vibration of the system and the air disturbance, which has overcome the outdoor and in-situ measurement restrictions of the traditional interferometric system, especially for the applications of 3D surface profilometry. Experimental results indicate that outdoor in-situ measurements and accurate reconstructions can be obtained for much stronger outdoor environmental illumination such as the background illuminance E = 5600 Lux for measuring the 3D-printed color object with varying surface reflectivity, E = 12,000 Lux for measuring the human-head model, and E = 35,000 Lux for measuring the selected coin. The portable and stable optical setup of 3D surface measurement developed in this study can be used to measure objects with even or varying surface reflectivity under windy conditions and strong environmental illuminations and obtain accurate reconstructions.