Expansion of Image Space in Enhanced-NA Fresnel Holographic Display

Abstract

The enhanced-NA Fresnel hologram reconstructs a holographic image at a viewing angle larger than the diffraction angle of a hologram pixel. The image space is limited by the bandwidth of a digital hologram. In this study, we investigate the property of image formation in the extended image space beyond a diffraction zone. A numerical simulation, using the phase Fresnel hologram, is carried out to observe an extension of image space and the effect of this on the changes in the angular field of view. The phase Fresnel hologram, synthesized by restricting the angular view range to a diffraction angle, can reconstruct a uniform image without high-order noises within the primary viewing zone, which is well confirmed by optical experiments. On the other hand, the overlapping of high-order images is inevitable, when the viewing angle depends on the hologram numerical aperture. In principle, the replica noises in both the hologram and image planes would be removed during the optimization algorithm considering suitable constraints. We discuss the development of a method for expanding the image space, while maintaining the viewing angle of a holographic image.

1. Introduction

A holographic display offers a three-dimensional image, in such a way that the wavefront of the propagating wave from the digital hologram is generated in free space [1]. Various attempts to implement holographic displays have been conducted because it has the ultimate merit, in that an observer can view a complete scene without visual fatigue [2,3,4]. Despite technological advancements, there still remains a fundamental limitation for realizing a commercial holographic display, owing to the finite space–bandwidth product of a pixelated spatial light modulator (SLM).

In particular, the issue of solving a narrow viewing angle is a crucial task. It is known that an SLM with a pixel pitch of the wavelength scale is necessary for seeing a holographic image at a wide angular view. A commercial modulator has a pixel pitch of several micrometers, inducing a viewing angle of a few degrees. Experiments to overcome this problem have been conducted by expanding the diffraction zone, using spatial or temporal multiplexing of modulators [3,5,6,7,8]. In this circumstance, massive data processing is required, and it would be inefficient, in view of energy consumption in informatics.

These conventional approaches for expanding the viewing zone are based on the assumption that the diffraction angle of a pixel pitch directly relates to the angular view of the holographic image. Previously, we found that the viewing angle $\mathsf{\Omega }$ of the reconstructed image is fundamentally determined by the numerical aperture (NA) of the digital hologram, rather than a diffraction angle, as demonstrated by the following equation [9,10]:

$$\mathsf{\Omega }=2{\sin }^{-1}\left(\mathrm{NA}\right).$$

(1)

The hologram numerical aperture is represented as a geometrical structure of aperture size, $l=N\mathsf{\Delta }x$, and a distance $z$, $\mathrm{NA}=\frac{N\mathsf{\Delta }x}{2z}$, where N and $\mathsf{\Delta }x$ denote the number of pixels and pixel size of the digital hologram, respectively. The viewing angle does not independently depend on the diffraction ability of the pixel pitch or the illuminating wave. This concept provides a major technological approach in expanding the viewing angle in the holographic display system. In principle, the viewing angle varies with the reconstructed distance, even when using an SLM with a constant pixel pitch. The viewing angle of the image would be larger than a diffraction angle when the image is formed below the critical distance. We called this system the enhanced-NA Fresnel holographic display, where there is still a decrease in image size because of the Nyquist sampling criterion in hologram synthesis [9,11,12,13]. However, this behavior is different from the trade-off relation, characterized by the space–bandwidth product of the digital hologram itself. For the realization of a wide viewing angle holographic display, a holographic image, without sacrificing the image size, should be reconstructed. When the image reconstruction is numerically or optically processed on the region outside of the sampling condition, the interference of high-order images is inevitable. Therefore, a detailed study on the expansion of image space and the effect of this on the changes in the viewing angle is required.

A phase modulation has some advantages, such as showing no twin image and high diffraction efficiency [14,15,16]. A phase retrieval algorithm, such as the iterative Fresnel transform, is used to calculate the phase hologram in the Fresnel diffraction regime. The enhanced-NA digital hologram creates replica fringe patterns, which fundamentally obstruct the image reconstruction in an extended space, beyond the bounds of a diffraction scope of a hologram pixel [9].

In this study, we explore the image reconstruction in the extended image space from the phase Fresnel hologram with a high numerical aperture. Firstly, the phase hologram was successfully prepared using the iterative algorithm, based on the Fresnel propagation. We analyzed an angular view of a holographic image, dependent on the hologram numerical aperture, and an expansion of image space. Secondly, optical experiments were conducted to obtain the reconstructed image without an interference of high-order noises. Subsequently, we studied the method for reconstructing a holographic image with an extended image space, at a larger viewing angle.

2. Holographic Image Reconstructed from Enhanced-NA Phase Fresnel Hologram

2.1. Analysis of Angular Field of View of Holographic Image Dependent on Hologram Numerical Aperture

The Fresnel field, which propagates from the digital hologram to the image plane, is described by the Fresnel diffraction formula. The specifications of both planes are determined by the Nyquist–Shannon sampling theorem [10,11,12]. The relationship between the pixel pitches $\left(\mathsf{\Delta }x,\mathsf{\Delta }y\right)$ and $\left(\mathsf{\Delta }{x}^{\prime },\mathsf{\Delta }{y}^{\prime }\right)$ in the hologram $\left(x,y\right)$ and image $\left(x^{\prime },y^{\prime }\right)$ coordinates is represented as follows:

$$\Delta x=\frac{\lambda z}{N\Delta x^{\prime }}, \Delta y=\frac{\lambda z}{N\Delta y^{\prime }}.$$

(2)

For convenience, a one-dimensional description is considered hereafter. The sampling condition of both planes is characterized by the critical distance having a value, $z_c=N\mathsf{\Delta }{x}^2/\lambda$, when the pixel pitches of both planes are the same. The image size, $l^{\prime }=N\mathsf{\Delta }{x}^{\prime }$, is smaller than the hologram size, $l=N\mathsf{\Delta }x$, when the image is placed at a distance less than $z_c$. We know that the aliased errors appear if they are digitally or optically processed in the regions that deviate from this specification.

A practicable phase hologram $\varphi$ for the real-valued object is extracted from the modified Gerchberg–Saxton (GS) iterative algorithm [14,17,18]. Figure 1 is a flow chart of the modified GS algorithm to obtain the phase-only hologram. The Fresnel transform, consisting of a single Fourier transform, is used in iterative procedures. Only the phase term is captured after the Fresnel transform, where the constraint in the spectral domain is a uniform modulus. The inverse Fresnel transform of the exponential term $e^{i\varphi }$ is carried out to modulate the phase hologram. To repeat this process, the amplitude of the reconstructed image is replaced by the object value. The algorithm rapidly converges to an optimized solution.

The phase holograms, composed of 1920 × 1920 pixels with an 8-µm pixel pitch, are synthesized using a letter image, as depicted in Figure 2a. Here, the critical distance, considering a wavelength of $\lambda =473 \mathrm{nm}$, is calculated to be 259.8 mm. To measure the viewing angle of a holographic image, the propagating wave far away from the reconstructed image was simulated. Figure 2b shows the propagating waves in the simulation, using the digital hologram made at a critical distance. The intensity profiles of the diffractive wave, indicated by a lateral line in the inset picture, are displayed with distance. An observer can view the image within the spreading angle of the diffractive wave. From this, the viewing angle of the reconstructed image is estimated to be 4.2°. Figure 2c is the simulation result for a digital hologram made at a half of the critical distance. The propagating wave from the image largely spreads, where the viewing angle appears to be about 7.7°.

As illustrated in Figure 2a, a holographic image is reconstructed by illuminating the digital hologram with a plane wave. Considering a point object, it appears that the point image is focused on a particular place through the digital hologram acting as a lens. From Equation (2), we can deduce that the lateral size and viewing angle $\mathsf{\Omega }$ of the reconstructed image have a trade-off relation, which is unlike the interpretation found by controlling the size of the hologram itself. The viewing angle is larger than the diffraction angle by a hologram pixel, $\theta =2{\sin }^{-1}\left(\frac{\lambda }{2\mathsf{\Delta }x}\right)$, when the image is formed at a distance less than the critical distance. The high-definition image diffracts to a larger spreading angle. We know that the angle value $\mathsf{\Omega }$ is pertinent to the hologram numerical aperture, $\mathrm{NA}=n\sin \left(\mathsf{\Omega }/2\right)$, where the refractive index n is equal to one in free space. Hence, the viewing angle of the reconstructed image obeys the relation in Equation (1). The above-measured viewing angles are consistent with the calculated values. We have also confirmed the viewing angle variation dependent on the hologram numerical aperture in optical experiments [9].

Meanwhile, a method including a double Fourier transform, such as a convolutional algorithm or angular spectrum method, cannot enhance the angular view because spatial filtering at the Fourier plane within the formula is embedded, resizing the hologram pattern according to the distance [10]. Here, one preserves the constant NA irrespective of the distance. For clarity, we defined a hologram that maintains a high numerical aperture as the enhanced-NA Fresnel hologram [13].

In previous work, we found that an aliased error arises when synthesizing the enhanced-NA Fresnel hologram because of the undersampling of the point spread function $h\left(x,y\right)$ in the front of a Fourier transform, $FT$, in the expanded form of the Fresnel diffraction formula, which is as follows [13,19]:

$$h\left(x,y\right)FT\left\{o\left(x^{\prime },y^{\prime }\right)h\left(x^{\prime },y^{\prime }\right)\right\}.$$

(3)

Since the sampling criterion with respect to the object plane is well satisfied, the spreading angle of the diffractive wave from the object $o\left(x^{\prime },y^{\prime }\right)$ covers the whole area of the digital hologram, which induces the suppression of aliased replica fringes when the object has a finite size. We explained that this phenomenon results from the near-field diffraction property within the diffraction zone, by object pixel resolution [9].

This behavior brings about an energy concentration in a low-frequency region in the digital hologram, which weakens the diffraction ability by an incident plane wave. Figure 3 illustrates the diffractive wave from the reconstructed image by a directly calculated digital hologram. We know that the image with original resolution should be restored digitally or optically because the high-frequency components of the hologram exist. However, the simulated light intensity from the finite-sized image does not spread out with distance, where it is difficult to observe a holographic image with a sufficient angular view. On the other hand, in a logarithmic scale, the spreading angle of a diffractive wave is clearly distinguished [10], which still ensures the validity of Equation (1). An effective method for mitigating the energy concentration is to add a random phase to the object in a hologram synthesis. In this case, as depicted in Figure 3c, the intensity profile of the diffractive wave progresses at an adequate spreading angle.

2.2. Expansion of Image Space and Its Reconstruction Property Depending on Viewing Angle

For the realization of a viable holographic display based on the enhanced-NA Fresnel hologram, a sufficient amount of image space should be secured. Figure 4 shows the simulation results for the generation of high-order images in the image space. The digital hologram with an 8-µm pixel pitch was conveniently calculated using a rectangular object consisting of 256 × 256 pixels, and then, the pixels were upsampled tenfold. The pixelated structure plays a role in planar gratings, and the modulated replica images were generated at an interval of $\lambda z/p$ in the Fresnel regime, where p indicates a pixel interval [10]. In this specification, it is expected that the Fraunhofer pattern starts to appear at a submillimeter distance. The digital hologram made at a distance, $2z_c$, of 69.3 mm reveals the enveloped images by a sinc function. However, there is no intensity modulation in the reconstructed image from the enhanced-NA Fresnel hologram, although the image is placed at a millimeter distance. Here, the bright areas of all the images are comparable to hologram size on the same scale. The image intensity has been known to be uniform, irrespective of its lateral location [20,21]. This makes it possible to reconstruct a uniform image in the extended image space larger than the diffraction scope by a hologram pixel.

Figure 5 displays the schematics of image reconstruction in the extended image space. We consider the hologram made using the letter object placed at a distance of half of $z_c$, 17.3 mm. The object has 512 × 512 pixels with a 4-µm pixel pitch, which is 2 times larger than the diffraction scope by a hologram pixel pitch. The angular separation of diffracted orders from the planar gratings is well described in the direction cosine space [22,23]. Each diffraction vector carries the high-order image in the new direction. As depicted in Figure 5a, the first-order images are created in the corresponding areas outside of the diffraction scope. The viewing direction of the high-order images is tilted at a diffraction angle ${\theta }_m$, which is shown in the following equation:

$$\sin {\theta }_m=\frac{m\lambda }{p}.$$

(4)

Figure 5b is the numerically reconstructed image showing up to the second-order term. The Rayleigh–Sommerfeld formula was used to preserve non-paraxial diffraction. The high-order images are unfocused because the image plane vertical to the optical axis is slightly apart from the direction cosine surface. The image blur is larger at the higher order terms.

We characterize the reconstruction behavior in two separate cases. The first case is that the viewing angle depends on the hologram numerical aperture, whose value is double a diffraction angle in this configuration. Here, the overlapped image with the high-order terms is observable. This type of high-order image would be removed through the iterative optimization algorithm, which will be discussed later. Secondly, the viewing angle could be restricted to a diffraction angle, due to the nature of algorithm. In this case, the complete image without the interference of high-order noises can be seen in the viewing direction of the zeroth order. If the viewing position is shifted to the first-order direction, the complete replica image will still reappear. This phenomenon is well observed in optical experiments.

3. Optical Reconstruction from Enhanced-NA Fresnel Hologram without Sacrificing Image Size

We carried out optical experiments by using a phase-only SLM (Holoeye PLUTO) with 1920 × 1080 pixels of an 8-µm pixel pitch and a blue laser with a wavelength of 473 nm. The critical distance for a pixel specification in a vertical direction was set as a reference to secure the imaging area with no aliased errors, which was calculated to be 146.1 mm. The phase holograms were synthesized using cameramen images located at distances of a half, a quarter, and one eighth of $z_c$, as shown in Figure 6a. It is hard to directly use the fast Fourier transform algorithm in the hologram synthesis, using the object space outside of the diffraction scope, because the pixel specifications of both planes deviate from the sampling condition in Equation (2). The digital hologram was prepared in two separate steps. First, the high-resolution holograms used to suit the extended object space in accordance with the sampling condition were calculated using the modified GS algorithm; thereafter, they were downsampled to the final resolution of the hologram.

To prepare a hologram made at half of $z_c$, 73.1 mm, the initial phase hologram with 3840 × 2160 pixels of a 4-µm pixel pitch was calculated using the object with the same pixel specifications. The final hologram in Figure 6b is the downsampled data by twice the pixel interval. We conveniently captured the image within the SLM size, by considering a direct-beam filtering by a polarizer. Figure 6c shows the reconstructed image with no replica noises. We observed that the other holograms reconstructed the original images well, without suffering from the high-order noises. All the images have a rectangular form with an 8.64 mm × 8.64 mm lateral size, according to the sampling condition. Although it is not displayed here, when the viewing direction is moved to the high-order region, the corresponding replica image reappears after the primary image is gone, which is consistent with the numerical analysis. The high-order images that overlapped in the image space are not viewable because of an inclined viewing direction. The image space is expanded, but the viewing angle is confined to a diffraction angle.

Figure 6d is the unfocused image of the image in Figure 6c. Even though an image blurring occurs, an accommodation effect is very weak. This indicates that the diffractive wave from the image spot propagates at a narrow spreading angle, where a perspective view of the image is hardly visible to the naked eye. The energy compaction of the hologram patterns invokes this weakness, as explained in Section 2.

In the second experiment, a sparse letter object was used and the down and up sampling processes were embedded within the iterative processes. Furthermore, to increase the optimization performance, the adequate background value of 37%, in comparison to the letter pixel value, was added to the object space by considering the suppression effect of the high-order noises when using the relatively dense object. A letter image is fully filled in the vacant space, whose size is equal to the hologram magnitude in vertical direction.

A phase hologram prepared at a half of the critical distance, 73.1 mm, is displayed in Figure 7a. We found that the pixel values were scattered in the whole area of the digital hologram, when compared with those of Figure 6b. The image without the interfering of the high-order noises is well reconstructed in Figure 7b, where the horizontal and vertical pixel resolutions of the images are $2.25 \mathsf{\mu }\mathrm{m}$ and $4 \mathsf{\mu }\mathrm{m}$, respectively. Figure 7c is the unfocused image captured at a distance slightly apart from half of $z_c$. We observed that an accommodation effect is very strong, which means that the spreading angle of the diffractive wave from the image spot would be larger. Figure 7d is the reconstructed image from the phase hologram synthesized at a quarter of the critical distance, 36.5 mm. Likewise, the hologram reconstructs the image well, without the high-order noises.

Figure 8 is the numerical simulation used to measure the spreading angle of the propagating wave from the image. As shown in Figure 8a, when there is no additional process, the diffraction scope of the propagating wave from the circular object image complies with the relation in Equation (1). However, the upsampled digital hologram by a simple duplication pixel deteriorates the diffracting behavior from the focused image, as shown in Figure 8b. This is the root cause for the obstruction of the viewing angle expansion of a holographic image, despite the effective removal of the high-order noises.

4. Viewing Angle Enlargement of Holographic Image and Discussions

As previously stated in Section 2, the replica fringes are formed in the enhanced-NA Fresnel hologram because of the undersampling of the point spread function, where the finite-sized object could suppress this phenomenon from the near-field diffraction property. However, it rather produces the adverse effect on the fringe concentration. To enhance the diffraction performance of a digital hologram by an incident wave, the energy compaction should be mitigated. A simple method is to diffuse an aggregated hologram pattern by adding the random phase to the real-valued object in the hologram synthesis, where the generation of the replica fringes is inevitable. In this case, it is fundamentally impossible to remove the high-order noises in the image space.

We found that, as displayed in Figure 7a, the iterative algorithm including the down and up sampling processes effectively prevents the formation of the replica fringes. The hologram pattern about the reversely propagating wave seems to be created to compensate the corresponding high-order noises during the repetition processes. This is different from a spatial filtering appearing in the algorithms with the double Fourier transforms. However, as seen in Figure 8, the down and up sampling processes still weaken the viewing angle expansion of a holographic image.

The Rayleigh–Sommerfeld formula using a definite integral can arbitrarily control the pixel specifications in both planes. Therefore, there is not a requirement for an unnecessary upsampling process within the iterative algorithm. As described in Section 2.2, the viewing angle dependent on the hologram numerical aperture would be conserved. In this circumstance, to obtain the optimized hologram, both the high-order images as well as the replica fringes in the digital hologram should be simultaneously removed during the repetition processes. Furthermore, there is still a drawback to consuming too much time in the iterative procedure.

Figure 9 shows the experimental results using the Riemann integral during the repetition processes. The phase hologram composed of 512 × 512 pixels was calculated using the letter object with a 4-µm pixel pitch located at a half of $z_c$. The fringe compaction was not completely eliminated, and there seemed to remain some replica fringes, as shown in Figure 9a. The inset in Figure 9b shows the magnified light intensity propagating from the reconstructed image. The diffracting performance of the focused image underwent no deterioration. The synthesized hologram reconstructs the image with an accommodation effect, but some high-order noises appear still, as shown in Figure 9b. The viewing angle is estimated to be about the first-order diffraction angle of 6.7° because the overlapped image is viewable. Here, the whole area of the digital hologram contributes to the optical image formation, which fulfills the viewing angle variation, dependent on the hologram numerical aperture.

It is important to develop a proper algorithm to reconstruct a holographic image that maintains an angular view, and to increase image fidelity. The high angular spectrum of a diffractive wave in both hologram and image planes should survive to form a highly focused spot. Presently, it is known that optimization algorithms, such as the stochastic gradient descent, demonstrate a considerable improvement in image fidelity. There have been several studies using an image captured by a camera or a deep learning algorithm [24,25]. In addition, a non-iterative algorithm has been studied [26]. These will become the proper methods. Further study to develop an optimization algorithm to accomplish a viewing angle enlargement will be carried out in the future. Furthermore, color holographic technology is required for realizing a holographic display. According to our analysis, the viewing angle depends only on the geometrical structure, which is irrespective of the wavelength of incident light. Thus, there is no change in the angular field of view by color in the color holographic display.

5. Conclusions

We studied the expansion of image space in the enhanced-NA Fresnel holographic display system. The reconstruction behavior largely depends on the angular view range, subjected to the synthesis algorithm of the phase Fresnel hologram. In particular, when the viewing angle is restricted to a diffraction angle, a complete image without suffering from the high-order noises is well observed in numerical and optical experiments. For the implementation of the holographic display, it is important to develop the algorithm for extending the image space, keeping the viewing angle of a holographic image. Since the viewing angle depends on the hologram numerical aperture in principle, the replica noises in both the hologram and image planes could be removed during the iterative optimization algorithm. This strategy makes it possible to recover the limitation of the narrow viewing angle problem due to the smaller pixel pitch of a pixelated modulator, which would realize the holographic display, even using a commercial spatial light modulator.