Wavelength-Selective Phase-Shifting Digital Holography: Color Three-Dimensional Imaging Ability in Relation to Bit Depth of Wavelength-Multiplexed Holograms ^†

Abstract

The quality of reconstructed images in relation to the bit depth of holograms formed by wavelength-selective phase-shifting digital holography was investigated. Wavelength-selective phase-shifting digital holography is a technique to obtain multiwavelength three-dimensional (3D) images with a full space-bandwidth product of an image sensor from wavelength-multiplexed phase-shifted holograms and has been proposed since 2013. The bit resolution required to obtain a multiwavelength holographic image was quantitatively and experimentally evaluated, and the relationship between wavelength resolution and dynamic range of an image sensor was numerically simulated. The results indicate that two-bit resolution per wavelength is required to conduct color 3D imaging.

1. Introduction

Holography [1,2] is a technique utilizing interference of light to record a complex amplitude distribution of an object wave. The recorded information is called a “hologram”. A three-dimensional (3D) image is reconstructed from the hologram by utilizing the diffraction of light. Holography can be used to record and reconstruct a 3D image of an object or a phase distribution of a wave without having to use multiple cameras or an array of lenses. Furthermore, 3D motion-picture images of any ultrafast physical phenomenon (such as light pulse propagation in 3D space) can be recorded and reconstructed with a single-shot exposure [3,4]. Digital holography (DH) [5,6,7,8,9] is used to record a digital hologram that contains an object wave and reconstructs both the 3D and quantitative phase images of an object by using a computer. DH can potentially be applied to the fields of not only ultrafast optical 3D imaging [10] but also microscopy [6,11,12], particles and flow measurements [13], quantitative phase imaging [14], lensless 3D imaging with incoherent light [15], multidimensional bio-imaging [16], multiwavelength 3D imaging [17], depth-resolved 3D imaging [18], simultaneous recording of multiple 3D images [19], and encryption [20].

Since a full space-bandwidth product of an image sensor is available, phase-shifting DH [21,22,23] is one way to capture an object wave. Using phase-shifting DH with multiple wavelengths, which is termed color/multiwavelength phase-shifting DH, 3D surface-shape measurements with multiwavelength phase unwrapping [24] and lensless color 3D image sensing [25,26] have been reported. DH using red-, green-, and blue-wavelengths is usually called “RGB digital holography”, “three-wavelength DH”, and “multiwavelength DH”. In this paper, we call DH with multiple wavelengths “two/three-wavelength DH” or “multiwavelength DH” because DH using two-green- and one-blue-wavelengths was simulated. In regard to multiwavelength phase-shifting DH, two types of representative implementations have been reported: temporal division [24] and space-division multiplexing [25,26] of multiple wavelengths. In the case of temporal division, wavelength information is sequentially recorded by changing the wavelengths of light to form a hologram. Mechanical shutters or operations to turn the light sources on and off are required for selecting the recorded wavelength. Three phase-shifted holograms are required at a wavelength [27] and nine holograms are needed for three-wavelength DH. Therefore, temporal division requires much time for multiwavelength 3D imaging. In the case of space-division multiplexing, red-, green-, and blue-wavelengths are simultaneously recorded by using a color image sensor with a Bayer color-filter array. Three exposures are required to obtain a multicolor holographic image. However, both recordable wavelength bandwidth and space-bandwidth product are determined by the array and therefore spatial information and wavelength selectivity is partially sacrificed. In the case of space-division multiplexing, crosstalk between multiwavelength object waves occurs when the wavelength selectivity of the array is insufficient [28]. The field of view (FOV) and spatial resolution of the DH system are decreased by the array due to the sacrifice of the space-bandwidth product of a hologram at each wavelength. The FOV is decreased by 75% compared to phase-shifting DH with a single wavelength.

In the case of color/multiwavelength digital holography, not only temporal-division [24] and space-division multiplexing [25,26], which are generally adopted for multiwavelength imaging in an imaging system, but also spatial division [27,28,29], temporal frequency-division multiplexing [30,31,32,33], and spatial frequency-division multiplexing [34,35,36] can be merged to record multiple wavelengths. In the case of general imaging systems, wavelength information is temporally or spatially separated. Spatial division is being actively researched because neither temporal nor spatial resolutions are sacrificed. However, in the case of space division, alignment of multiple image sensors is a problem. Numerical correction is reported to solve this problem effectively [29]. On the other hand, holographic multiplexing makes it possible to record multiwavelength/color information by using a monochrome image sensor and to reconstruct it from wavelength-multiplexed image(s). In the 1960s, Lohmann presented the concept of recording a multidimensional image by holographic multiplexing [37], which is based on spatial frequency-division multiplexing [34,35,36]. This multiplexing enables single-shot multidimensional holographic sensing and imaging; however, it sacrifices the spatial bandwidth available for recording each object wave at each wavelength as the number of wavelengths is increased. As another means of holographic multiplexing, temporal frequency-division multiplexing has been researched, and it provides a wide spatial bandwidth regardless of the number of wavelengths [29,30,31]. As for temporal frequency-division multiplexing technique, Fourier and inverse Fourier transforms are calculated for each pixel to separate wavelength information. To obtain a color 3D image, however, many wavelength-multiplexed images and an image sensor with a high frame rate are needed.

Since 2013, we have been proposing an interferometric technique which selectively extracts wavelength information by using wavelength-multiplexed phase-shifted interferograms to measure multiwavelength object waves without using a color-filter array [9,38,39,40,41,42,43]. As for the proposed interferometry, multiwavelength information is multiplexed both on the space and in the spatial-frequency domain, and it is then separated in the polar coordinate plane by using wavelength-dependent phase shifts. Hereafter, multiwavelength DH based on the proposed interferometry is termed wavelength-selective phase-shifting DH (WSPS-DH). Applying the WSPS-DH provides a full space-bandwidth product of an image sensor at each wavelength regardless of the number of wavelengths measured. Moreover, operations to change the wavelengths of light to form a hologram are not required. When the number of wavelengths is N, only 2N+1 wavelength-multiplexed images are required for multiwavelength 3D image sensing [9,38,39,40], while 3N holograms are recorded in the temporal division. Recordable wavelength bandwidth is determined by the spectral sensitivity of the monochrome image sensor. Therefore, both wavelength and spatial bandwidth of the WSPS-DH are greater than those of the space-division multiplexing with a color image sensor. After the initially reported WSPS-interferometry was reported, WSPS-DH utilizing 2N wavelength-multiplexed images was proposed [41]. In the primitive scheme [38,39,40,41], phase ambiguity of 2π was utilized to selectively extract multiwavelength object waves, and then a technique employing arbitrary phase shifts for rigorously retrieving object waves at multiple wavelengths by using 2N+1 holograms was proposed [9,42,43]. Although Doppler phase-shifting color DH [31,32] has also been proposed as another holographic multiplexing technique with a full space-bandwidth product, it requires the recording of a large number of images. WSPH-DH requires only 2N holograms at least [41] by employing two-step phase-shifting interferometry [44,45,46,47,48,49], while 512 holograms are required for Doppler three-wavelength phase-shifting DH [32]. Therefore, WSPH-DH accelerates measurement speed by more than 80 times when recording three wavelengths [39,41]. The proposed DH has the potential to obtain a multiwavelength holographic 3D image with a small number of recordings without any color absorption. It thus enables multimodal cell imaging with low light intensity when applied to biological microscopy. Moreover, in principle, it alleviates light damage to living cells during multidimensional holographic imaging.

However, it is necessary to consider the influence of bit depth of the recorded holograms on the quality of the reconstructed image. This is because the proposed DH multiplexes holograms at multiple wavelengths on a monochrome image sensor, and available bit depth per wavelength is sacrificed. Furthermore, in the case of the proposed DH, it is worth evaluating whether the dynamic range of holograms is related to wavelength resolution because wavelength information is selectively extracted by using the wavelength dependency of the intensity changes induced by the phase shifts of holograms.

In this paper, we investigate image quality in relation to the dynamic range of holograms formed by wavelength-selective phase-shifting DH. Image quality and wavelength resolution in relation to dynamic range are analyzed with numerically and experimentally obtained holograms.

2. Wavelength-Selective Phase-Shifting Digital Holography (WSPS-DH)

A schematic of WSPS-DH is shown in Figure 1. WSPS-DH is enabled by the wavelength dependency of the intensity change induced by wavelength-dependent phase shifts of interference light. By introducing wavelength-dependent phase shifts to interference light, wavelength information is separated in the polar coordinate plane. A phase shifter such as a mirror with a piezo actuator, a liquid crystal, a birefringent material, a spatial light modulator, an acousto-optic modulator, or an electro-optic modulator is used to generate the phase shifts. When multiwavelength information is recorded, light at wavelengths are not absorbed by a filter, and wavelength-multiplexed phase-shifted holograms are sequentially obtained by changing the phases of interference fringes. Object waves at multiple wavelengths are separately obtained from the recorded wavelength-multiplexed images when phase-shifting interferometry selectively extracts wavelength information [9,38,39,40,41,42,43]. Diffraction integrals are applied to the extracted object waves, and a multiwavelength holographic 3D image is then reconstructed. Since no light at wavelengths are absorbed by a filter, WSPS-DH is expected to achieve high light-use efficiency. When the WSPS technique is compared to temporal frequency-division multiplexing, the number of recordings can be reduced, and measurement speed can be increased.

Up to now, 2π ambiguity of phase [38,39,40,41] or arbitrary symmetric phase shifts [9,42,43] are utilized to extract each object wave rigorously by solving systems of equations. Combining 2π phase ambiguity and arbitrary symmetric phase shifts enables a multiwavelength holographic 3D imaging with only 2N wavelength-multiplexed holograms and in total less than 1000 nm of movement of a mirror with a piezo actuator [50,51].

3. Image Quality in Relation to Bit Depth of Wavelength-Multiplexed Holograms

3.1. Experimenal Results

The quality of the reconstructed image in relation to bit depth of wavelength-multiplexed holograms was investigated by using experimentally obtained holograms [40]. The constructed optical system, the method of generating phase shifts, phase shift α, and the specification of the lasers used are described in Reference [40]. Two lasers with oscillation wavelengths of 640 nm and 473 nm, respectively, were set to record five two-wavelength-multiplexed holograms. A monochrome complementary metal-oxide semiconductor (CMOS) image sensor was used to record the holograms. The sensor has 12 bits, 2592 × 1944 pixels, and a pixel pitch of 2.2 μm. Two overhead projector (OHP) transparency sheets were set as a color 3D object. The logo of the International Year of Light and the characters “2015” were drawn on the sheets, and blue- and red-color films were attached to the logo and characters, respectively. A red “2015” sheet and a blue logo one were set at different depths. Five wavelength-multiplexed holograms were obtained by utilizing 2π ambiguity of the phase, and a color 3D image was reconstructed with the algorithm described in Reference [40]. Holograms that have less than 8-bit resolution were generated from the recorded holograms numerically. Object images were reconstructed by using compressed holograms in which bit depth was changed from 1 to 7 bits. Then, the images obtained by holograms without compression were regarded as the true values, and the cross-correlations coefficient (CC) and root-mean-square error (RMSE) of the reconstructed images were calculated.

Color object images obtained from the compressed holograms are shown in Figure 2. The images were reconstructed by using two-wavelength-multiplexed holograms with resolution of more than 2 bits. As bit resolution was decreased, the reconstructed images degraded gradually. However, a clear color object image was reconstructed even when the number of bits was 4. Furthermore, a two-wavelength object image was reconstructed from holograms with 3-bit depth resolution. To investigate the quality of the reconstructed images quantitatively, CC and RMSE of the intensity images at respective wavelengths were calculated. Graphs of CC and RMSE are plotted in Figure 3. Maximum- and minimum-intensity values of the images were set as 255 and 0, respectively. A CC of nearly 0.8 and a RMSE of 1/10 maximum value were obtained when bit depth was 5. From the quantitative evaluations and Figure 2, it can be concluded that quite similar images are reconstructed even when the image sensor had a resolution of less than 8 bits.

3.2. Numerical Simulations

To investigate image quality and validate the experimental evaluations, in the case that bit depth of the image sensor was from 1 bit to 16 bits, image quality of reconstructed images was numerically simulated for three-wavelength WSPS-DH. A random pattern was set as the phase distribution of the object wave because scattering object waves were assumed. For color-intensity images, a photographic image of a flower and grass was prepared. For wavelengths of red-, green-, and blue-color light sources, 640 nm, 532 nm, and 488 nm were assumed. In the simulation, the distance between the object and image sensor was set to 150 mm, pixel pitch to 2.2 μm, and the number of pixels of the image sensor to 512 × 512. It was assumed that phase shifts were generated by a mirror with a piezo actuator and the mirror was moved 0 nm, 61 nm, ±244 nm, and ±488 nm sequentially. The intensity ratio between object and reference waves was 1:4 at each wavelength. Resolution of the image sensor was changed from 1 to 16 bits. Six wavelength-multiplexed phase-shifted holograms were obtained numerically and three-wavelength object waves were reconstructed by WSPS-DH [50,51]. Reconstructed images in the numerical simulation are shown in Figure 4. In the same manner as revealed by the experimental results, as bit resolution was decreased, the reconstructed images degraded gradually. However, a clear multicolor object image was reconstructed even when the number of bits was 6. As shown in Figure 5, CCs of the reconstructed amplitude images were more than 0.8 when bit depth of the image sensor was decreased to 6 bits. Furthermore, although the color of the reconstructed image differed from that of the object, a three-wavelength object image was reconstructed even when the image sensor had a 4-bit depth resolution. On the other hand, it was found that RMSE and CC of the reconstructed phase distribution were worse than those of the amplitude images. For phase measurement and 3D shape measurement with multiwavelength phase unwrapping, an image sensor with high dynamic range is required. Using an image sensor with resolution of more than 9 bits will result in performance of less than RMSE of λ/20 [rad] in phase. Analysis for smooth phase distribution is a future work.

The experimental and numerical results indicate that a resolution of at least 2 bits per wavelength in each hologram is required to obtain a multiwavelength 3D-object intensity image, and a color 3D image with a small color shift can be reconstructed when the sensor has more than 2-bit resolution per wavelength. Measurement error is reduced as bit depth is increased in the same manner as an ordinary imaging system; however, a faithful object intensity image can be reconstructed in a case of resolution of much less than 8 bits. The numerical results also show that using a low-bit image sensor causes a large error in phase measurement; therefore, an image sensor with more than 9 bits is desirable in the case of 3D shape measurement with phase information at multiple wavelengths.

4. Numerical Analysis of the Wavelength Resolution Against Dynamic Range of Holograms

The relation of wavelength resolution to bit resolution of holograms was numerically investigated. It was assumed that the optical setup is based on three-wavelength phase-shifting DH using a monochrome image sensor and a mirror with a piezo actuator under the following conditions. It was assumed that three-wavelength WSPS-DH with six wavelength-multiplexed holograms [50,51] was used and the mirror was moved 0 nm, 61 nm, ±488 nm, and ±732 nm sequentially. A color image and a rough surface were set as amplitude and phase distributions in 3D space, respectively. To investigate image quality quantitatively, CC and RMSE of the reconstructed images were calculated. It was initially assumed that the three wavelengths of light sources were λ₁ = 640 nm, λ₂ = 532 nm, and λ₃ = 488 nm. After that, wavelength λ₁ was set as 607, 589, 561, 556, 552, 546, 540, 534, 533, or 532.5 nm to investigate wavelength resolution of WSPS-DH. The wavelengths were determined from commercially available continuous wave (CW) lasers with long coherence lengths. Pixel pitch was 2.2 μm, and the number of pixels was 512 × 512. The wavelength resolution under the three conditions was investigated: an image sensor having 8-, 12-, and 16-bit resolutions. To investigate wavelength resolution of WSPS-DH under ideal conditions, no random noise such as incoherent stray light and dark-current noise was added to holograms.

Reconstructed images obtained by this numerical simulation are shown in Figure 6, and graphs of calculated RMSE and CC of the amplitude and phase images at λ₂ are plotted in Figure 7. High CC means that faithful images were obtained and low RMSE indicates that multiwavelength 3D image measurement was high precision. The numerical results clarify that high CC was obtained even when the wavelength difference was less than 10 nm when an 8-bit image sensor was used. However, in the case an 8-bit image sensor was used, it was difficult to observe an object image clearly when the wavelength difference was within 2 nm. The difference between phase shifts added at neighboring wavelengths was small and the wavelength dependency of the intensity change induced by the wavelength-dependent phase shifts also became small. It is considered that an 8-bit image sensor could not detect weak wavelength dependency of the intensity change by the quantization. In contrast, in the cases of using an image sensor with 12- and 16-bit resolutions, object waves were successfully reconstructed because the image sensor captured weak wavelength dependency of intensity changes by phase shifts. An image sensor with 16 bits can record smaller intensity changes; therefore, higher CC and lower RMSE were obtained. These results indicate that wavelength resolution can be improved by increasing the bit depth of an image sensor. It is worth noting that this feature is characteristic of WSPS-DH. Thus, a guideline for selecting an appropriate image sensor was confirmed successfully.

5. Discussion

The reason for a color shift in the numerical simulation is discussed. In comparison with the experimental results, the results of the numerical simulation show that the color of the reconstructed images shifts remarkably when bit depth of wavelength-multiplexed holograms is low. Here, the value of the wavelength difference is focused on, and λ₂ is set to 561 nm instead of 532 nm to adjust the wavelength difference for three wavelengths in the numerical simulation described in Section 3.2. The numerical results when setting the wavelengths to 488, 561, and 640 nm are shown in Figure 8. The images indicate that at least 2-bit resolution per hologram at a wavelength is required, and a color 3D image with a small color shift by using an image sensor that has more than 2-bit resolution per wavelength. However, color shift was obviously decreased by increasing the difference of neighboring wavelengths λ₂ and λ₃. This trend can be explained by the fact that, as described in Section 2, WSPS-DH is enabled by the wavelength dependency of the intensity change induced by wavelength-dependent phase shifts of interference light. When the difference between λ₂ and λ₃ was small, the effect for wavelength-dependent intensity change also became small. Selective extraction of object waves at λ₂ and λ₃ were difficult as the bit resolution was decreased because the wavelength-dependent intensity change was small and an image sensor with low bit resolution was not able to detect the change. As a result, the CC of λ₁ was relatively high and the RMSE was relatively low, so the object-intensity image at λ₁ was clearly reconstructed in comparison to those at λ₂ and λ₃. Quantitative evaluations shown in Figure 5 supported this finding because the CC was higher and the RMSE was lower at λ₁. In contrast, in the simulation shown in Figure 8, the wavelength-dependent intensity change became large by increasing wavelength-dependent phase shifts, and therefore each of three object waves was reconstructed from holograms with 4-bit resolution. As a result, the color was improved. From the experimental results presented in Section 3.1 and the numerical results presented in this section, it is clear that the color 3D-image sensing can be achieved when using an image sensor with more than 2-bit resolution per wavelength.

6. Conclusions

The quality of reconstructed images in relation to dynamic range of holograms generated by WSPS-DH was investigated. Quantitative, experimental, and numerical results clarified the required bit resolution to obtain a multiwavelength holographic image and the relationship between the wavelength resolution and dynamic range of an image sensor. Experimental and numerical results indicate that 2-bit resolution per hologram at a wavelength is required to obtain a multiwavelength 3D-object intensity image at least, and a color 3D image with a smaller color shift can be reconstructed when the sensor has more than a 2-bit resolution per wavelength. More than 3 bits per wavelength is sufficient for high-quality multiwavelength 3D imaging. Wavelength resolution can be improved by increasing bit depth of an image sensor, and this finding is characteristic of WSPS-DH. WSPS-DH will perform multiwavelength 3D imaging at high speed for low-light-intensity events. Accordingly, it will contribute to multispectral 3D imaging with high light-use efficiency and high wavelength resolution by using a monochrome image sensor with high dynamic range (such as an electron multiplying charge-coupled device (EM-CCD) camera) and an array of photo multipliers.