Optical See-through 2D/3D Compatible Display Using Variable-Focus Lens and Multiplexed Holographic Optical Elements

Abstract

An optical see-through two-dimensional (2D)/three-dimensional (3D) compatible display using variable-focus lens and multiplexed holographic optical elements (MHOE) is presented. It mainly consists of a MHOE, a variable-focus lens and a projection display device. The customized MHOE, by using the angular multiplexing technology of volumetric holographic grating, records the scattering wavefront and spherical wavefront array required for 2D/3D compatible display. In particular, we proposed a feasible method to switch the 2D and 3D display modes by using a variable-focus lens in the reconstruction process. The proposed system solves the problem of bulky volume, and makes the MHOE more efficient to use. Based on the requirements of 2D and 3D displays, we calculated the liquid pumping volume of the variable-focus lens under two kinds of diopters.

1. Introduction

Augmented reality (AR) is a technology which allows computer generated virtual imagery to exactly overlay physical objects [1,2]. AR display mainly includes optical see-through type [3], video see-through type [4] and reflection type [5,6] display devices. At present, the optical see-through display device has become the mainstream technical solution, because it allows the user to directly watch the real environment light through an image combiner, and improves the user’s perception of the real world. As a kind of true three-dimensional (3D) display technology, integral imaging has a compact structure and can provide various physiological depth cues [7,8]. It is conductive for the natural integration of virtual 3D images and real scenes, and is suitable to be integrated into AR display devices [9,10]. However, due to optical modulation of the lens array, integral imaging is not compatible with two-dimensional (2D) display. Although 3D display can provide depth information that is absent in 2D display, the current 3D resolution is severely degraded. The 2D display has the advantage of high resolution and large viewing angle, but it can’t replace 3D display. Therefore, some researchers have proposed 2D/3D compatible displays, such as switching backlight units by using polarized pinhole array [11,12] and polymer dispersed liquid crystal (PDLC) [13], or using dynamic parallax grating technology [14], but these technologies cannot be applied to optical see-through AR display devices.

The technology that is key to realizing optical see-through AR display is the image combiner that overlaps the virtual images and the real scenes together. The image combiner can be further divided into two categories: reflection-type devices [15,16] and diffraction-type devices [17,18]. Compared with reflection-type devices, the diffraction-type devices have many advantages, such as unique angle selectivity and wavelength selectivity and flexibility of design, as well as high diffraction efficiency and transparency. It can replace one or several optical elements in the system and is an ideal image combiner [19]. Yeom et al. proposed a 2D/3D convertible projection system using see-through integral imaging based on HOE. The use of prism reduced the number of projectors and enabled a compact form factor, but the utilization of the HOE was only 50%, that is, one half of the HOE was used for 2D display and the other half for 3D display [20]. Chou. et al. proposed a 2D/3D hybrid integral imaging display based on a liquid crystal micro-lens array (LCMLA), in which the LCMLA has the focusing effect or no optical effect determined by the polarization state of the incident light. Therefore, the system added devices such as twisted nematic cell and polarizer to control the polarization state, and increased the redundancy [21]. In our previous work, we proposed attaching a PDLC film to the lens array HOE to constitute a 2D/3D projection screen. The lens array HOE and PDLC film were used to realize integral imaging 3D display and 2D display, respectively. However, the PDLC film degraded the transparency of the system, and the optical see-through properties could not be achieved in 2D mode [22].

In this paper, we propose an optical see-through 2D/3D compatible display by using variable-focus lens and multiplexed holographic optical elements (MHOE). Based on angular multiplexing technology, the MHOE records the scattering wavefront of a diffuser for 2D display and the spherical wavefront array of micro-lens array (MLA) for 3D display. The incident angles of the projected angles vary with the changes in the variable-focus lens to meet the two Bragg diffraction conditions of 2D and 3D displays. Therefore, our approach does not require polarization devices and reduces the systematic complexity, enhances the effective utilization of MHOE and maintains optical see-through properties for both 2D and 3D displays, all of which are highly beneficial for AR applications.

2. Structure and Principle

Figure 1 shows the structure of our proposed optical see-through 2D/3D compatible display system. It is mainly composed of a MHOE, a variable-focus lens and a projector. Essentially, the MHOE is a volumetric holographic grating. The scattering wavefront of a diffuser and the spherical wavefront array of MLA are recorded on the single volumetric holographic grating by two-step recording with angular multiplexing technology. The variable-focus lens has two optical states of diopter φ₁ and φ₂, and it modulates the light transmitted by the projector respectively to meet the two Bragg diffraction conditions of MHOE.

In Figure 1a, the probe beam from the projector contains the amplitude information of elemental image array, and illuminates the MHOE after passing through the variable-focus lens with the diopter of φ₁. Then, the MHOE’s Bragg diffraction condition 1 is satisfied, and the spherical wavefront array is diffracted out. The elemental image aligns with the element of MHOE to reconstruct the 3D image based on the theory of integral imaging. In Figure 1b, the probe beam from the projector contains the 2D image, and illuminates the MHOE after passing through the variable-focus lens with the diopter of φ₂. At this time, the MHOE’s Bragg diffraction condition 2 is satisfied, and the scattering wavefront is diffracted out to achieve 2D display. The light from the real scene whose wavelengths and incident angles don’t match the Bragg diffraction conditions, directly passes through the MHOE. Hence, the system shows good optical see-through properties.

In the reconstruction process, the variable-focus lens is the key component that plays a role in the angle selectivity of MHOE. It can accurately change the incident angles of the reconstructed beam under different diopter conditions in order to meet the Bragg matching angles. In the 3D mode, the divergent angle of projected beam is 2U₁, and it doesn’t change after passing through the variable-focus lens with the diopter of φ₁ = 0D. The conical beam still maintains the divergent angle of 2U₁ when it arrives at the MHOE, the incident angles of the two boundary beams are θ_l1 and θ_r1, respectively, as shown in Figure 2a. The θ_l1 and θ_r1 can be expressed as

$${\theta }_{l1}=90°-\beta +U_1,$$

(1)

$${\theta }_{r1}=90°-\beta -U_1,$$

(2)

where β is the complement angle between the MHOE surface and the optical axis of projector. On the 2D mode, the diopter of variable-focus lens is φ₂. The divergent angle of projected beam turns into 2U₂ after passing through the variable-focus lens. The incident angles of the two boundary beams onto the MHOE are θ_l2 and θ_r2, respectively, as shown in Figure 2b. The U₂, θ_l2 and θ_r2 can be described as

$$U_2=\frac{U_1-y\phi }{n^{\prime }},$$

(3)

$${\theta }_{l2}=90°-\beta +U_2,$$

(4)

$${\theta }_{r2}=90°-\beta -U_2,$$

(5)

where y is the height of the projected beam formed onto the back surface of the variable-focus lens, n′ is the refractive index of the liquid in the variable-focus lens.

Compared with the conventional optical elements, the HOEs possess useful properties originating from their volumetric holographic grating structures, namely their selectivity and multiplex ability. The selectivity means HOEs have optical see-through properties, and it is easy to utilize the multiplex ability to superpose different functions of optical elements in the single volumetric holographic grating. Figure 3 illustrates the design and two-step recording process of MHOE used in this paper. Figure 3a shows the first recording process that records the spherical wavefront array generated by the MLA. The reference beam illuminates the holographic plate with the divergent angle of 2U₁. The indent angles of the two boundary beams are θ_l1 and θ_r1. The signal beam illuminates the MLA vertically and generates spherical wavefront array. The spherical wavefront array is projected onto the other side of the holographic plate, and interferes with the reference beam in the region of L₁. As a result, the spherical wavefront array is recorded. Figure 3b presents the second recording process that records the scattering wavefront of a diffuser. The reference beam illuminates the holographic plate with the divergent angle of 2U₂. The indent angles of the two boundary beams are θ_l2 and θ_r2. Meanwhile, the signal beam illuminates the diffuser vertically and generates scattering wavefront. The scattering wavefront is projected onto the other side of the holographic plate, and interferes with the reference beam in the region of L₂.

3. Experiments and Results

3.1. Fabrication of Variable-Focus Liquid Lens and MHOE

In experiments, we fabricated a variable-focus liquid lens. The operating mechanism of the liquid lens is that its focal length is controlled by pumping liquid in and out of the lens chamber, which, in turn, changes the curvature of the liquid profile. Therefore, the volume of pumping liquid is controlled to adjust the diopter of the liquid lens to satisfy two Bragg diffraction conditions. The radius of curvature (R) and volume change ΔV follow the relationship [23]:

$$\Delta V=\frac{\pi }{3}\left(2R^2-\frac{d^2}{4}-2R\sqrt{R^{{}^2}-\frac{d^2}{4}}\right)\left(2R+\sqrt{R^2-\frac{d^2}{4}}\right),$$

(6)

where d is the diameter of the lens aperture. The variable-focus liquid lens is considered as a thin lens, and its effective diopter (φ) can be calculated:

$$\phi =\frac{(n^{\prime }-1)}{R},$$

(7)

where the refractive index of the liquid in the experiment is n′ = 1.33. Figure 4a shows the changes in the radius of curvature R and diopter φ with the variation in volume change ΔV. In the practical experiment, the φ was 14.29D and the R was 23 mm, which we chose while ΔV was 8.6 mL. Figure 4b,c show the lens effects with the lens diopter of 0D and 14.29D, respectively.

Figure 5a,b show the experimental setups for the two-step recording of MHOE. The fabricated variable-focus liquid lens was placed in the optical path of the reference beam to change the incident angles of the reference beam to match the Bragg diffraction conditions. The detailed parameters are shown in

Table 1. Parameters of the experiment for recording process.

Components	Parameters		Values
Bragg diffraction conditions	Bragg matching angles 1	θl1	57.2°
		θr1	32.8°
	Bragg matching angles 2	θl2	45°
		θr2	45°
Recording region	Width L1		23 mm
	Width L2		18 mm
MLA	Pitch		1 mm
	Focal length		3.3 mm
Diffuser	Scattering angle		20°
Photopolymer plate	Thickness		15 ± 1 µm
	Sensitive wavelength		532 nm
	Sensitivity		10 mJ/cm²
	Averaged refractive index		1.47
	Refractive index modulation		>0.02

. The pitch and focal length of the recorded MLA are 1 mm and 3.3 mm, respectively, and the scattering angle of the diffuser is 20°.

In order to obtain high diffraction efficiency, we actually used two holographic plates and recorded each optical element independently. The MHOE was obtained by using a frame (GCM-1301M in Daheng Optics) to clip the two holographic plates together. The fabricated MHOE was tested with reference beams under the corresponding Bragg diffraction conditions. Figure 6a,b show the diffraction effects of the MLA and diffuser, respectively. The real-world scene behind the MHOE is clearly observed without visual obstruction, which verifies the see-through properties of MHOE, as shown in Figure 6c. Figure 6d shows the relation between normalized diffraction efficiency and angle deviation. The diffraction efficiency starts to decline sharply at about ±5°. The angular difference between the two Bragg matching angles is 12.2°, which is large enough to avoid the crosstalk between the two display modes.

3.2. Experimental Results

We developed the experimental system based on the proposed scheme as shown in Figure 7a. The projector has the resolution of 1280 × 800. The projected image for the 3D mode is provided as shown in Figure 7b. The character “3” and letter “D” are in front of the MHOE with the depth of 35 mm and behind the MHOE with the depth of −35 mm, respectively. In addition, a 2D text “Sichuan University” is prepared for 2D display as shown in Figure 7c.

On the 3D display mode, the projected beam meets the Bragg diffraction condition 1. The 3D image is reconstructed, and four different perspectives from top left, top right, bottom left and bottom right viewing points are shown in Figure 8a. Obvious horizontal and vertical parallaxes can be clearly seen. Additionally, the character “3” is bigger than letter “D”, which demonstrates that “3” is reconstructed in front of the MHOE while “D” is behind the MHOE. On the 2D display mode, the projected beam satisfies the Bragg diffraction condition 2, and Figure 8b shows the displayed 2D image. The real object “dice” behind the MHOE is always visible on both 2D and 3D display modes, which verifies the good optical see-through properties of the system.

4. Conclusions

In this paper, an optical see-through 2D/3D compatible display system is proposed by using a variable-focus lens and a MHOE. The scattering wavefront and spherical wavefront array are recorded in the MHOE using angular multiplexing technology to achieve 2D/3D display, respectively. The variable-focus lens is used to adjust the incident angles of the projected beam. By analyzing the relationship between the liquid change and the radius of curvature of the variable-focus lens, the optimal diopter of the variable-focus lens is picked up to match the Bragg diffraction conditions. The incident angles of the projected beam are calculated and measured. When the MLA and diffuser were recorded on holographic plates based on angular multiplexing technology, the variable-focus lens is placed in the path of the reference beam to adjust the incident angles of the reference beam through the variation in the liquid pumping volume. During reconstruction, the incident angles of the projected beam satisfied the Bragg matching angles under the same amount of liquid pumping volume, and the spherical wavefront array of the MLA and the scattering wavefront of diffuser are reconstructed. As a result, 3D images with proper horizontal and vertical parallaxes are reconstructed on 3D display mode and 2D image is displayed on 2D display mode, and both modes show good optical see-through properties. The images; quality would be further improved if a variable-focus liquid lens with multi-chamber is used. The proposed system can be a good candidate in AR application.