Expansion of image space in enhanced-NA Fresnel holographic display

The enhanced-NA Fresnel hologram reconstructs a holographic image at a viewing angle larger than the diffraction angle of a hologram pixel. The image space is limited by the bandwidth of a digital hologram. In this study, we investigate the property of image formation in the extended image space beyond a diffraction zone. A numerical simulation using the phase Fresnel hologram is carried out to observe an extension of image space and the effect of this on the changes in the angular field of view. The phase Fresnel hologram, synthesized by restricting the angular view range to a diffraction angle, can reconstruct a uniform image without high-order noises within the primary viewing zone, which is well confirmed by optical experiments. On the other hand, the overlapping of high-order images is inevitable when the viewing angle depends on the hologram numerical aperture. The high-order images are distributed in the direction cosine space, which could be effectively removed through an angular low-pass filter. We discuss the development of method for expanding the image space while maintaining the viewing angle of a holographic image.


I. INTRODUCTION
A holographic display offers a three-dimensional image in such a way that the wavefront of the propagating wave from the digital hologram is generated in free space [1]. Various attempts to implement holographic displays have been conducted because it has the ultimate merit in that an observer can view a complete scene without visual fatigue [2][3][4]. Despite technological advancements, there still remains a fundamental limitation for realizing a commercial holographic display owing to the finite space-bandwidth product of a pixelated spatial light modulator (SLM).
In particular, the issue of solving a narrow viewing angle is a crucial task. It is known that an SLM with a pixel pitch of the wavelength scale is necessary for seeing a holographic image at a wide angular view. A commercial modulator has a pixel pitch of several micrometers, inducing a viewing angle of a few degrees. Experiments to overcome this problem have been conducted by expanding the diffraction zone using spatial or temporal multiplexing of modulators [3,[5][6][7][8]. In this circumstance, massive data processing is required, and it would be inefficient in view of energy consumption in informatics.
These conventional approaches for expanding the viewing zone are based on the assumption that the diffraction angle of a pixel pitch directly relates to the angular view of the holographic image. Previously, we found that the viewing angle Ω V of the reconstructed image is fundamentally determined by numerical aperture (NA) of the digital hologram rather than a diffraction angle, as demonstrated by the following equation [9,10]: The hologram numerical aperture is represented as a geometrical structure of aperture size, l = N ∆x , and a distance z, NA = N∆x /2z , where N and ∆x denote the number of pixels and pixel size of the digital hologram, respectively. The viewing angle does not independently depend on the diffraction ability of the pixel pitch or the illuminating wave.
This concept can provide a major technological breakthrough in expanding the viewing angle in holographic display system. In principle, the viewing angle varies with the reconstructed distance, even when using an SLM with a constant pixel pitch. The viewing angle of the image would be larger than a diffraction angle when the image is formed below the critical distance. We called this system the enhanced-NA Fresnel holographic display, where there is still a decrease in image size because of the Nyquist sampling criterion in hologram synthesis [9,[11][12][13]. However, this behavior is different from the trade-off relation characterized by the space-bandwidth product of the digital hologram itself. For the realization of a wide viewingangle holographic display, a holographic image without sacrificing the image size should be reconstructed. When the image reconstruction is numerically or optically processed on the region outside of the sampling condition, the interference of high-order images is inevitable.
Therefore, a detailed study on the expansion of image space and the effect of this on the change in the viewing angle is required.
A phase modulation has some advantages, such as showing no twin image and high diffraction efficiency [14][15][16]. A phase retrieval algorithm such as the iterative Fresnel transform is used to calculate the phase hologram in the Fresnel diffraction regime. The enhanced-NA digital hologram creates replica fringe patterns, which fundamentally obstructs the expansion of image space, beyond the bounds of a diffraction scope of a hologram pixel [9].
In this study, we explore the image reconstruction in the extended image space from the phase Fresnel hologram with a high numerical aperture. Firstly, the phase hologram was successfully prepared using the iterative algorithm based on the Fresnel propagation.
We analyzed an angular view of a holographic image dependent on the hologram numerical aperture, and an expansion of image space. Secondly, optical experiments were conducted to obtain the reconstructed image without an interference of high-order noises. Subsequently, we studied the method for reconstructing a holographic image with an extended image volume at a larger viewing angle. The Fresnel field, which propagates from the digital hologram to image plane, is described by the Fresnel diffraction formula. The specifications of both planes are determined by the Nyquist-Shannon sampling theorem [10][11][12]. The relationship between pixel pitches (∆x , ∆y) and (∆x , ∆y ) in the hologram (x, y) and image (x , y ) coordinates is represented as follows: For convenience, one-dimensional description is considered hereafter. The sampling condition of both planes is characterized by the critical distance having a value, z c = N ∆x 2 /λ when the pixel pitches of both planes are the same. The image size, l = N ∆x is smaller than the hologram size, l = N ∆x when the image is placed at a distance less than z c . We know that aliased errors appear if they are digitally or optically processed in the regions that deviate from this specification.
A practicable phase hologram φ for the real-valued object is extracted from the modified Gerchberg-Saxton (GS) iterative algorithm [14,17,18]. The phase holograms, composed of 1920×1920 pixels with an 8-µm pixel pitch, are synthesized using a letter image, as depicted in Figure 2 the viewing angle appears to be about 7.7 • .
As illustrated in Fig. 2(a), a holographic image is reconstructed by illuminating the digital hologram with a plane wave. Considering a point object, it appears that the point image is focused on a particular place through the digital hologram acting as a lens. From Eq.
(2), we can deduce that the lateral size and viewing angle Ω V of the reconstructed image have a trade-off relation, which is unlike the interpretation found by controlling the size of hologram itself. The viewing angle is larger than the diffraction angle by a hologram pixel, θ = 2 sin −1 ( λ 2∆x ) when the image is formed at a distance less than the critical distance. The high-definition image diffracts to a larger spreading angle. We know that the angle value Ω V is pertinent to the hologram numerical aperture, NA = n sin(Ω /2), where the refractive index n is equal to one in free space. Hence, the viewing angle of the reconstructed image obeys the relation in Eq. (1). The above-measured viewing-angles are consistent with the calculated values. We have also confirmed the viewing-angle variation dependent on the hologram numerical aperture in optical experiments [9].
Meanwhile, a method including a double Fourier transform such as convolutional algorithm or angular spectrum method cannot enhance the angular view, because spatial filtering at the Fourier plane within the formula is embedded, resizing the hologram pattern according to the distance [10]. Here, one preserves the constant NA irrespective of the distance. For clarity, we defined a hologram that maintains a high numerical aperture as the enhanced-NA Fresnel hologram [13].
In previous work, we found that an aliased error arises when synthesizing the enhanced-NA Fresnel hologram because of the undersampling of the point spread function h(x, y) in the front of Fourier transform F T in the expanded form of the Fresnel diffraction formula, which is as follows [13,19]: Since the sampling criterion with respect to the object plane is well satisfied, the spreading angle of diffractive wave from the object o(x , y ) covers the whole area of digital hologram, which induces the suppression of aliased replica fringes when the object has a finite size. We explained that this phenomenon results from the near-field diffraction property within the diffraction zone by object pixel resolution [9].
This behavior brings about an energy concentration in a low-frequency region in the digital hologram, which weakens the diffraction ability by an incident plane wave. Figure 3 illustrates the diffractive wave from the reconstructed image by a directly calculated digital hologram. The simulated light intensity from the finite-sized image does not spread out with distance, where it is difficult to observe a holographic image with a sufficient angular view. On the other hand, in a logarithmic scale, the spreading angle of diffractive wave is clearly distinguished [10]. A holographic image floating in free space is induced from part of digital hologram, while most of concentrated hologram fringes directly propagates without their diffraction. An effective method for mitigating the energy concentration is to add a random phase to the object in the hologram synthesis. In this case, as depicted in Fig. 3(c), the intensity profile of diffractive wave progresses at an adequate spreading angle.
B. Expansion of image space and its reconstruction property depending on viewing angle For the realization of a viable holographic display based on the enhanced-NA Fresnel hologram, a sufficient amount of image space should be secured. Figure 4 shows the simulation results for the generation of high-order images in the image space. The digital hologram with an 8-µm pixel pitch was conveniently calculated by using a rectangular object consisting of 256×256 pixels, and then, the pixels are upsampled tenfold. The pixelated structure plays a role in planar gratings, and the modulated replica images were generated at an interval of λz/p in the Fresnel regime, where p indicates a pixel interval [10]. In this specification, it is expected that the Fraunhofer pattern starts to appear at a submillimeter distance. The digital hologram made at a distance 2z c , 69.3 mm reveals the enveloped images by a sinc function. However, there is no intensity modulation in the reconstructed image from the enhanced-NA Fresnel hologram, although the image is placed at a millimeter distance. Here, the bright areas of all the images are comparable to hologram size on the same scale. The image intensity has been known to be uniform irrespective of its lateral location [20,21].
This makes it possible to reconstruct a uniform image in the extended image space larger than the diffraction scope by a hologram pixel. We consider the hologram made using the letter object placed at a distance of half of z c , 17.3 mm. The object has 512×512 pixels with a 4-µm pixel pitch, which is 2 times larger than the diffraction scope by a hologram pixel pitch. The angular separation of diffracted orders from the planar gratings is well described in the direction cosine space [22,23]. Each The Rayleigh-Sommerfeld formula was used to preserve non-paraxial diffraction. The highorder images are unfocused because the image plane vertical to optical axis is slightly apart from the direction cosine surface. The image blur is larger at higher order terms.
We characterize the reconstruction behavior in two separate cases. The first case is that the viewing angle depends on the hologram numerical aperture, whose value is double a diffraction angle in this configuration. Here, the overlapped image with the high-order terms is observable. This type of high-order images would be removed through an angular low-pass filter, which will be discussed later. Secondly, the viewing angle could be restricted to a diffraction angle due to the nature of algorithm. In this case, the complete image without the interference of high-order noises can be seen in the viewing direction of the zeroth order. If the viewing position is shifted to the first-order direction, the complete replica image will still reappear. This phenomenon is well observed in optical experiments.

GRAM WITHOUT SACRIFICING IMAGE SIZE
We carried out optical experiments by using a phase-only SLM (Holoeye PLUTO) with 1920×1080 pixels of an 8-µm pixel pitch and a blue laser with a wavelength of 473 nm.
The critical distance for a pixel specification in a vertical direction was set as a reference to secure the imaging area with no aliased errors, which was calculated to be 146.1 mm.
The phase holograms were synthesized using a cameramen image located at distances of a half, a quarter, and one eighth of z c , in Fig. 6(a). It is hard to directly use the fast Fourier transform algorithm in the hologram synthesis using the object space outside of the diffraction scope because the pixel specifications of both planes deviate from the sampling condition in Eq. (2). The digital hologram was prepared in two separate steps. First, the high-resolution holograms used to suit the extended object space in accordance with the sampling condition were calculated using the modified GS algorithm; thereafter, they were downsampled to the final resolution of the hologram.
To prepare a hologram made at half z c , 73.1 mm, the initial phase hologram with 3840×2160 pixels of a 4-µm pixel pitch was calculated using the object with the same pixel specifications. The final hologram in Fig. 6(b) is the downsampled data by twice the pixel interval. We conveniently captured the image within the SLM size, by considering a direct-beam filtering by a polarizer.  In second experiment, a sparse letter object was used and the down and up sampling processes were embedded within the iterative processes. Furthermore, to increase the optimization performance, the adequate background value of 37% in comparison to letter pixel value was added to the object space by considering the suppression effect of the high-order noises when using the relatively dense object. A letter image is fully filled in the vacant space whose size is equal to the hologram magnitude in vertical direction.
A phase hologram prepared at a half of the critical distance, 73.1 mm is displayed in Fig.   7(a). We found that the pixel values were scattered in the whole area of the digital hologram when compared with those of Fig. 6(b). The image without interfering the high-order noises is well reconstructed in Fig. 7(b), where the horizontal and vertical pixel resolutions of images are 2.25 µm and 4 µm, respectively. Figure 7(c) is the unfocused image captured at a distance slightly apart from half of z c . We observed that an accommodation effect is very strong, which means that the spreading angle of diffractive wave from the image spot would be larger. Figure 7(d) is the reconstructed image from the phase hologram synthesized at a quarter of the critical distance, 36.5 mm. Likewise, the hologram reconstructs the image well, without the high-order noises. Figure 8 is the numerical simulation used to measure the spreading angle of propagating wave from the image. As shown in Fig. 8(a), when there is no additional process, the diffraction scope of the propagating wave from the circular object image complies with the relation in Eq. (1). However, the upsampled digital hologram by a simple duplication pixel deteriorates the diffracting behavior from the focused image, as shown in Fig. 8(b). This is the root cause for the obstruction of the viewing-angle expansion of a holographic image despite the effective removal of the high-order noises.

DISCUSSIONS
As previously stated in Section 2, the replica fringes are formed in the enhanced-NA Fresnel hologram because of the undersampling of the point spread function, where the finite-sized object could suppress this phenomenon from the near-field diffraction property.
However, it rather produces the adverse effect on the fringe concentration. To enhance the diffraction performance of a digital hologram by an incident wave, the energy compaction should be mitigated. A simple method is to diffuse an aggregated hologram pattern by adding the random phase to the real-valued object in the hologram synthesis, where the restoration of replica fringes is inevitable.
We found that as displayed in Fig. 7(a), the iterative algorithm including the down and up sampling processes effectively prevents the replica fringe formation. The hologram pattern about the reversely propagating wave seems to be created to compensate the corresponding high-order noises during the repetition processes. This is different from a spatial filtering appearing in the algorithms with the double Fourier transforms. However, as seen in Fig. 8, the down and up sampling processes still weakens the viewing-angle expansion of a holographic image.
The Rayleigh-Sommerfeld formula using a definite integral can arbitrarily control the pixel specifications in both planes. Therefore, there is not a requirement for an unnecessary upsampling process within the iterative algorithm. As described in Subsection 2.2, the viewing angle dependent on the hologram numerical aperture would be conserved. Figure 9 is the simulation results using the Riemann integral during the repetition processes. The phase hologram composed of 512×512 pixels was calculated using the letter object with a 4-µm pixel pitch located at a half of z c . The fringe compaction is not completely eliminated, and there seems to remain some replica fringes, in Fig. 9(a). The reconstructed image overlapped with the high-order terms is shown in Fig. 9(b). The inset figure shows the magnified light intensity propagating from the reconstructed image. The diffracting performance of focused image undergoes no deterioration. We know that the viewing angle is close to the first-order diffraction angle of 6.7 • because the overlapped image is viewable.
Especially, we note that the plane wave incident on the replica fringes forms the high-order images in the direction cosine surface other than the primary image plane. These images correspond to those optically generated by a pixelated structure of the SLM. Consequently, the existence of replica fringes might not be meaningful for the high-order image formation because they are created irrespective of replica fringes. In this circumstance, the high-order images could be effectively removed through an angular low-pass filter. As illustrated in Fig. 5(a), the primary image is formed by the carrier wave propagating within the angular bandwidth of θ ±1 . If an angular stop filter blocking the high-order carrier waves is placed in front of the SLM, the high-order images could not be transmitted [24]. Here, the whole area of the digital hologram contributes to the primary image formation, which still fulfills the viewing angle variation dependent on the hologram numerical aperture. For recovering an insufficiency of noise removal, the development of a proper algorithm is required. Presently, it has been known that the optimization algorithms using a captured image by a camera or deep learning technique demonstrate a considerable improvement in an image fidelity [25,26]. They would become the proper methods for accomplishing a high-quality image in this type of holographic system.   x' x' z, z'