Variable Clock and EM Signal Generation Scheme for Foveation-Based Driving OLED Head-Mounted Displays

Abstract

An image processing pipeline and multi-output shift register of a foveation-based driving scheme are proposed for the realization of immersive head-mounted displays in 2019. In addition, this paper describes a variable clock generation circuit to manipulate output waveforms of shift registers in the foveated display. The EM circuit for OLED displays is also introduced to support the control signal to keep OLEDs of pixels from emitting light during the compensation. Especially, the EM circuit consists of only four TFTs and one capacitor and gives rise to pulses of variable widths corresponding to the resolution of a driven display area. A variable clock generation scheme is verified with 60 Hz 1440 × 2560 monitor, eye-tracker, PSoC board and FPGA board. An EM circuit is simulated by SPICE for 9600 lines and 120 Hz foveated displays.

1. Introduction

Due to the recent global pandemic of coronavirus 2019 (COVID-19), societies are being changed from offline to online. Virtual reality (VR) is one of the key technologies to support these online activities in a variety of fields including education, conference, manufacturing and researching [1,2,3,4,5].

The realization of these VR technologies requires high resolution, wide field-of-view (FoV) and high frame rate displays that make indistinguishable visual experiences possible [6,7,8,9,10]. These high-performance displays can ameliorate screendoor effects, motion artifacts and latency, providing users with immersive experiences. However, it is too difficult to meet these three requirements at the same time. While many studies have focused on the latency reduction by means of the foveated rendering at the graphics processing unit [11], the data bandwidth reduction [12] and the deep learning for motion tracking [13], until recently, there remained one critical problem on the display side. which is the insufficient pixel charging time because of the very high resolution of several thousand lines and the frame rate faster than 60 Hz. Ideally, the visual acuity of 60 pixels per degree (ppd) and horizontal and vertical FoVs of 160${}^{°}$ and 170${}^{°}$ for each eye should be satisfied with high resolution displays of around 9600 lines. At this resolution, only a charging time smaller than 0.87 μs, which is too short to charge pixel electrodes fully, is allowed at a 120 Hz frame rate.

On the other hand, this insufficient charging time can be coped with by the foveation-based driving scheme that extends the line time by means of multiple line driving gate driver and foveated rendering in the panel [14]. The multiple lines of the high resolution input image are compressed in a line by different factors according to the distance from the foveation point and the human visual system (HVS). Then, pixels of these multiple lines are charged with the same voltage leading to the reduction on the effective number of lines by 79.0% for the resolution of 9600 lines. As a consequence, the charging time is extended from 0.87 to 4.14 μs.

The previous proposal [14] showed the overall architecture and multi-output shift registers; however, the variable clock generation block is needed to drive those shift registers because variable gate pulses are produced by clock signals. Furthermore, to apply this foveation-based method to organic light-emitting diode (OLED) displays, emission (EM) signals should be applied to the pixel circuits to keep OLEDs from emitting light when the compensation is being conducted over threshold voltage, mobility and supply voltage variations [15,16,17,18,19,20].

This paper proposes variable clock generation scheme for multi-output shift registers and EM signal circuit for the OLED pixel compensation. Consequently, the foveation-based OLED display is fully accomplished with small area multi-output shift register, simple variable clock generation, EM circuit of only four thin film transistors (TFTs) and vertical resolution reduction just by averaging. This leads to the high resolution and high frame rate display by means of line time extension as well as the reduction of data bandwidth and latency.

2. Previous Foveation-Based Driving Scheme

The perceivable spatial frequency of a displayed image is limited by the non-uniform distribution of receptor cells in the retina of HVS and the finite resolution of a display. Consequently, the maximum perceivable frequency ($f_{HVS}$) of HVS is expressed by Equation (1) and the display resolution defined as a half Nyquist frequency ($f_{RES}$) by Equation (2) for the eccentricity (e) that is the distance from the foveation point in the unit of degree. V is the vertical resolution of the display, $\beta V$ is the viewing distance between eye and head mounted display (HMD), $e_2$ is a half-resolution eccentricity constant, $C{T}_0$ is a minimal contrast threshold and $\alpha$ is a spatial frequency decay constant [21]. The resultant cutoff frequency ($f_c$) is obtained as the minimum value among $f_{HVS}$ and $f_{RES}$, as expressed in Equation (3).

$$f_{HVS}\left(e\right)=\frac{-e_{2}ln{CT}_0}{\alpha (e_2+e)}[cycles/degree]$$

(1)

$$f_{RES}\left(e\right)=\frac{\pi \beta V}{360·{cos}^2\left(\frac{\pi e}{180}\right)}[cycles/degree]$$

(2)

$$f_c=min\left(f_{HVS},f_{RES}\right)$$

(3)

When a flat panel display is used as a HMD, $f_{RES}$ is increasing and $f_{HVS}$ is decreasing over the distance from the foveation point ($d_{pixel}$). While $f_c$ is equal to $f_{RES}$ in only some region around the foveation point, $f_c$ is determined by $f_{HVS}$ in the outside area. If the resolution is reduced by $2^n$ (n = 1, 2,… ), the larger region of $f_c$ is controlled by the display resolution because a corresponding Nyquist frequency ($f_{RES/2^n}$) moves to the lower values. However, there still exists the area dominated by $f_{HVS}$ in the reduced resolutions. Therefore, resolutions of display areas ($f_{RR}$) can be further reduced while the resultant $f_c$ is perceptually equivalent to that of the original resolution image without any visible artifacts [14].

The previous foveation-based driving scheme is composed of vertical resolution reduction and multi-output driving gate circuit [22,23], as depicted in Figure 1, where the low-level gate pulses enable the pixel charging by assuming that the display panel is manufactured at a backplane of p-channel TFTs. The vertical resolution reduction curtails the vertical resolution of the input image with the much smaller number of lines by merging multiple-line pixels into one-line pixels with their average values in accordance with $f_{RR}$. Then, these merged pixel data are driven to pixels of corresponding multiple lines in a panel at a time by the multi-line driving circuit. In the end, the full resolution foveated image is given rise to directly in a panel from the reduced resolution image, which allows the pixel charging time to be extended. While the previous study proposed the overall architecture and the multi-output shift register, this paper introduces the clock generation scheme to support the variable clock waveforms for the multi-output shift register as well as the variable pulse width EM circuit for the pixel compensation in OLED displays.

3. Variable Clock Generation

The previous foveated display [14] employed eight-output shift registers in gate driver circuits. While a variety of pulse timings are programmable within the outputs of one shift register, there should exist one-clock shifts between adjacent shift registers. Consequently, it has to be ensured that output pulses of the same timing are generated only in a single shift register. Using the foveation point obtained by an eye tracking module and $f_{RR}$, upper and lower region boundaries of 1/2, 1/4 and 1/8 resolutions are calculated accordingly, where U2, U4 and U8 are the boundary line numbers for upper regions and L2, L4 and L8 for lower regions. Then, the boundaries of regions are adjusted to be multiples of tw, four and eight, which guarantees that multiple pulses with the equal timing are given rise to in a shift register. For the upper regions, the adjusted values (UB2, UB4 and UB8) are determined to be maximum multiples that are equal to or smaller than U2, U4 and U8. Conversely, the values of the lower regions (LB2, LB4 and LB8) are adjusted to be minimum multiples that are equal to or larger than L2, L4 and L8. As a result, no resolution regions are overlapped over two shift registers. This boundary adjustment is summarized in Figure 2.

After the boundaries are computed, clock pulses are controlled to generate gate pulse waveforms corresponding to regions. Since output pulses of multi-output shift registers are equivalent to clock pulses during their output periods, it is required to change clock waveforms according to the resolution of the panel region driven by gate pulses. In the region of a shift register, there are 12 possible cases, as shown in Figure 3, which can be determined from boundary values. For example, when the foveation point is placed on the center of a display, the top region can be assigned with 1/8 resolution. As the region gets closer to the foveation point, the resolutions increase up to the original resolution through the case RC1 to the case RC8. After passing the center, the resolutions decrease down to 1/8 resolution at the bottom area through the case RC9 to the case RC11, the case RC4, the case RC12 and the cases RC2 and RC1 again. The cases RC3, RC5, RC6, RC7, RC9, RC10, RC11 and RC12 represent that the boundaries are located at the inside of the region controlled by one shift register. After one region case is determined, a corresponding clock waveform is selected among 12 cases, as illustrated in Figure 4, and are transferred to shift registers. In a 1/8 resolution region, all eight clock signals give rise to pulses at the same time. In the same way, in 1/4 and 1/2 resolution regions, four and two clock signals provide pulses at the same time. Then, for the original resolution region, clock signals should be shifted over their previous ones by one pulse width.

On the other hand, as the foveation point moves, the effective number of lines in the reduced resolution image also changes. It has the maximum value for the foveation point in the center of a panel, while it gets to be the minimum for the foveation point in the uppermost and lowermost positions of a panel. To support fixed frame and line times even at this variable effective number of lines, the variation on the vertical blank length, which is recently used to support the variable frame rate in gaming and low power display applications [24,25], maintains the total number of vertical lines including vertical active and blank areas at the constant number. For example, when the effect numbers of lines at the center and uppermost positions are 1456 and 928 for the vertical resolution of 4800 lines [14], the vertical blank lengths are set to be 14 and 542, respectively. Consequently, this vertical blank adjustment leads to the constant total number of vertical lines of 1470 for any foveation points. Unlike the variable frame rate applications, this vertical blank variation scheme causes no flicker problems because the frame rate is maintained at the fixed value [22,26].

4. EM Signal Generation for OLED Pixels

As mentioned above, high quality OLED displays need to compensate for variations on TFTs’ electrical characteristics over area and time as well as supply voltage drops. The example pixel circuit shown in Figure 5 [27] is able to compensate for both threshold voltage ($V_{th}$) of T1 and voltage drop of VDD. In the $V_{th}$ detecting period, the capacitor stores the voltage of $VDD-|V_{th}|-V_{sus}$ and the following data writing period programs the gate voltage of T1 at $VDD-|V_{th}|-V_{sus}+V_{DATA}$ leading to the OLED current ($I_{OLED}$) independent of VDD and $V_{th}$ at the display period, as described in Equation (4), where k is a coefficient.

$$I_{OLED}=\frac{k}{2}{(V_{sus}-V_{DATA})}^2$$

(4)

The control signal of EM connected to a p-channel TFT (T4) is applied at the high voltage level to make the drain of T1 floating and to keep the OLED from emitting the undesired light during both $V_{th}$ detecting and data writing periods. These EM pulses, which are wider than gate pulses, have been generated by additional shift registers with wide clock pulses or programmable pulse width shift registers [28,29,30]. While previous circuits maintain the pulse width of EM signals to be equivalent for all pixels, EM pulse widths for these foveation-based driving pixels should be adjusted differently depending on the resolution of the driven panel area. The proposed scheme assigns one EM signal to eight lines driven by one multi-output shift register. That is, all pixels in these eight lines are turned off during the same period of time that is different according to the resolution of the assigned region.

Assuming that the panel is implemented at the backplane of p-channel TFTs and the high level of the EM pulse turns off the light emission, the proposed EM pulse circuit is simply composed of only four TFTs, as presented in Figure 6a. Whereas the IN1 pulse pulls up the voltage of Q and EM, the IN2 pulse pulls down Q as well as EM. As depicted in Figure 6b, EM is charged during the interval between IN1 and IN2 pulses and is maintained at the low level for the remaining period of time. Because of the Q node voltage bootstrapped through C1, EM can be driven at GND without any voltage rises. Since Q turns into a floating node at GND more quickly than EM due to the smaller capacitive load, the voltage difference between Q and EM is stored across C1, and this difference is maintained during the falling transition of EM. Therefore, Q is bootstrapped to lower than GND. The last output of the previous shift register is applied to IN1, and the first output of the following shift register is asserted to IN2, as illustrated in Figure 7 where CLK1–CLK8 are connected to odd shift registers and CLK9–CLK16 are used for even shift registers. When the example pixel circuit in Figure 5 is used, the $V_{th}$ detecting operation takes place at the same time as the last pulse of the previous shift register in all pixels of eight lines and data writing operations work for pixels of corresponding lines to pulses of the current shift register. Then, the first pulse of the next shift register triggers the light emission of those pixels.

The detail operations are explained with four periods of pulling-up, holding-high, bootstrapping and holding-low. In the pulling-up period in Figure 8a, because M1 and M3 are turned off, Q and EM are pulled up to VDD through M2 and M4. Then, in the holding-high period in Figure 8b, Q and EM are retained at VDD as all TFTs from M1 to M4 are turned off. In the bootstrapping period in Figure 8c, M1 and M3 are turned on; however, Q is pulled down more quickly than EM owing to its smaller capacitive load. Therefore, Q becomes a floating node that allows the falling transition of EM to boost the voltage of Q to the lower level than GND by the bootstrapping through C1. Consequently, EM can be pulled down to GND. In the end, in a holding-low period of Figure 8d, Q is kept at the low voltage level to turn M3 on by C1, even though there exist leakage currents at M1 and M2. EM is also held at GND during a frame time. Although this EM circuit is based on the basic structure of a shift register, the connection of M3 to a constant supply GND instead of clocks avoids the fluctuation on Q and EM nodes without additional TFTs [31].

5. Evaluation Results

5.1. Variable Clock Generation

The variable clock generation circuit was evaluated using 60 Hz 1440 × 2560 liquid crystal display (LCD) monitor, eye-tracker (Tobii Eye Tracker 4L) [32], computer, programmable system on chip (PSoC) board (Cypress PSoC 5LP) [33] and field programmable gate array (FPGA) board (Terasic DE2-115) [34], as shown in Figure 9a. The eye-tracker is attached at the bottom of a monitor to capture the foveation point, which is sent to a computer via a universal serial bus (USB) interface. The computer estimates the coordinate of the foveation point and transmits it to a PSoC board with an USB interface. Because the used FPGA board cannot handle the USB interface directly, the PSoC is used to convert the USB data into serial peripheral interface (SPI) data. In addition, because the PSoC contains a central processing unit (CPU), it calculates boundary values of UB2, UB4, UB8, LB2, LB4 and LB8 and transfers them to a FPGA board through the SPI. These resolution boundaries are calculated at the viewing distance parameter, $\beta$ of 0.35. Finally, the FPGA board outputs VBLANK, RSYNC and 16 clock signals of CLK1 to CLK16. VBLANK is the signal that indicates the vertical blank period, and RSYNC is used to show the specific region case in the oscilloscope. This connection is explained in more details in Figure 9b.

The FPGA board stores six received boundary values of each 14 bits in the internal block memory and loads them right before starting a new frame. These boundary values are compared with the current region driven by the multi-output shift register, and then one of 12 regions in Figure 3 is selected in the region case calculation, leading to the determination of a clock waveform out of 12 cases. Finally, two clock generation blocks give rise to eight clocks of CLK1 to CLK8 for even shift registers and eight clocks of CLK9 to CLK16 for odd ones. Eight clock signals are made by the 8-bit pattern sequence each line time for a selected region case. For example, the case RC2 is supported by the sequence of two 8-bit binary numbers that are 11110000 and 00001111 for CLK1 to CLK8. The case RC4 consists of four 8-bit binary numbers that are 11000000, 00110000, 00001100 and 00000011. The block diagram of this variable clock generation circuit is depicted in Figure 10. The measured waveforms for 12 cases are presented in Figure 11 where the first pulse, RSYNC, is generated to indicate the corresponding region case in an oscilloscope. The clock signals are successfully given rise to for their region cases, which are exactly matched to the target waveforms of Figure 4.

Additionally, the variation of the reduced vertical resolution from the maximum to the minimum is realized by changing the foveation point from the center position to the uppermost position on the monitor. Then, the clock signals are generated based on the foveation points obtained by the eye tracker. To keep the frame time at the constant value, the vertical blank length is adjusted as depicted in Figure 12. The pulse interval of CLK1 is wider at the higher resolution area because outputs of one shift register are generated for more line times. Thus, the foveation point area is recognized by finding where the sparser CLK1 pulses appear. In our evaluation setup where the frame time is fixed at 1000 lines, while lengths of active and vertical blank areas are 950 lines and 50 lines for the foveation point at the center, they are adjusted to be 602 lines and 398 lines at the uppermost. All clock pulses of the vertical blank period are deactivated.

5.2. EM Signal Generation

The proposed EM circuit was simulated by SmartSpice [35] with a p-channel low temperature poly silicon (LTPS) TFT model where threshold voltage, mobility and overlap capacitance of TFTs are −1.7 V, 31 ${\mathrm{cm}}^2$/V·s and 3.02 $\mathrm{fF}$/$\mathsf{\mu }$m, respectively. All channel lengths are set to be 7 μm and the channel widths of M1, M2, M3 and M4 are 10, 10, 100 and 100 μm, respectively, to drive resistive and capacitive loads of 2.2 k$\Omega$ and 120 pF [22] in a line time that is set to be 4.14 μs for a 120 Hz foveated display. The supply voltage levels of VDD and GND are 15 and 0 V. They are summarized in

Table 1. Parameters for emission (EM) circuit simulation.

PMOS Characteristics	Threshold Voltage	Mobility	Overlap Capacitance
	−1.7 V	31 ${\mathrm{cm}}^2$/V·s	3.02 $\mathrm{fF}$/μm
Transistor Sizes	Channel Length	Channel Width
	7 μm	10 μm@M1, M2 100 μm@M3, M4
Simulation Setup	Supply Voltage	Output Load	Line Time
	15 V@VDD 0 V@GND	2.2 k$\Omega$ 120 pF	4.14 μs @120 Hz

.

To determine C1, changes of holding effect, bootstrapping and falling time at the Q node are investigated regarding its capacitance. If C1 is given as 0.4 pF to be much smaller than the capacitive load at EM, Q reaches to a floating state of GND+|$V_{th}$| earlier than EM leading to the big voltage difference between Q and EM, as presented in Figure 13a. This achieves large bootstrapping at Q that results in the fast falling transition at EM. Conversely, because the large C1 of 20 pF slows down the falling transition of Q, it causes the small difference between Q and EM, degrading the bootstrapping of Q and the falling speed of EM. However, the larger C1 improves the holding effect leading to the reduced voltage rise caused by the leakage currents, as shown in Figure 13b. Consequently, there exists the trade-off between holding effect and bootstrapping where the larger C1 degrades the bootstrapping effect with better holding capability, but the smaller C1 increases the voltage rise at Q by the leakage current with larger bootstrapping. Since the maximum boosted Q voltage for a frame time ($Q_{high}$) is important to maintain the turning-on state of a pull-down TFT (M3), the capacitance of C1 is determined using the plot of $Q_{high}$, as shown in Figure 14. When C1 is less than 3 pF, the full bootstrapping effect takes place but the current leakages pull up the Q voltage level during a frame time owing to the poor holding effect. On the other hand, when C1 is larger than 3 pF, the boosted voltage level at Q goes up due to the bootstrapping degradation in spite of the improved holding performance. In addition, considering the size of the EM circuit, the final C1 has been determined as 0.4 pF with $Q_{high}$ of −6.4 V, where the falling transition of EM is completed in a line time of around 4 μs.

The simulation was conducted for the configuration of shift registers and EM circuits in Figure 7 where the first, second, third and fourth stages are assigned to $f_{RES}$, $f_{RES/2}$, $f_{RES/4}$ and $f_{RES/8}$ regions, respectively. Their corresponding cases of clock waveforms are RC8 for $f_{RES}$, RC4 for $f_{RES/2}$, RC2 for $f_{RES/4}$ and RC1 for $f_{RES/8}$ in Figure 4. The simulated waveforms are shown in Figure 15. As a consequence, the first shift register of $f_{RES}$ produces eight shifting pulses from G(0) to G(7) and the corresponding EM circuit generates the EM(0) pulse of the low level during nine line times. In the same fashion, the second shift register and EM circuit of $f_{RES/2}$ build gate pulses from G(8) to G(15) and EM(2) of five line times, the third ones of $f_{RES/4}$ from G(16) to G(23) and EM(3) of three line times and the fourth ones of $f_{RES/8}$ from G(24) to G(31) and EM(4) of two line times.

In addition, 32 p-channel TFT pixel circuits of Figure 5 are evaluated by SPICE along with four shift registers and four EM circuits of Figure 7. The $V_{th}$ and VDD variations of ±0.5 and −0.5 V are taken into account at $V_{sus}$ of 12 V. The OLED current curves over the $V_{th}$ variation are plotted in Figure 16a,b without and with pixel compensation. The OLED current characteristics over the VDD variation are shown in Figure 17a,b without and with pixel compensation. It is verified that, while the average current errors without pixel compensation show 166.6% and 34.2% over the gate voltages from 7 to 15 V, the average current errors over the data voltages from 4 to 12 V are reduced to 4.4% and 5.5% by pixel compensation circuits integrated with proposed diving circuits.

6. Conclusions

On top of previous vertical resolution reduction technology and multi-output shift register for the foveation-based driving scheme of immersive HMDs, this paper demonstrates the variable clock generation to realize multiple gate pulse waveforms for the foveated image on a panel and the EM circuit to drive the pixel compensation circuit in an OLED display. The proposed clock circuit is implemented to adjust clock pulse timings for 12 region cases and the variable clock waveforms have been verified at FPGA with various foveation points captured by an eye-tracker. The EM circuit is simply composed of four p-channel LTPS TFTs and one capacitor and is ensured by the SPICE simulation. In addition, the pixel circuits connected to shift registers and EM circuits are also evaluated by SPICE, and it is proved that the proposed driving circuits achieve the compensation for variations of threshold voltages and IR drops of supply voltages. On the other hand, this technology is applicable only to HMDs dedicated to a single user because the image is processed by foveated rendering based on one foveation point. To widen the high-resolution HMD market further, the super resolution (SR) algorithm [36] should be employed due to insufficient number of high-resolution images at several thousands of lines. If the SR technology is incorporated in the proposed foveation-based scheme, both complexity and latency can be dramatically reduced. Consequently, when the foveation-based driving scheme is supplemented by these variable clock and EM circuits, the resultant foveated display can pave the way to high resolution, wide FoV and immersive OLED HMDs in the markets earlier than expected.