An 8-Tap CMOS Lock-In Pixel Image Sensor for Short-Pulse Time-of-Flight Measurements

An 8-tap CMOS lock-in pixel image sensor that has seven carrier-capturing and a draining time window was developed for short-pulse time-of-flight (TOF) measurements. The proposed pixel for the short-pulse TOF measurements has seven consecutive time-gating windows, each of which has the width of 6 ns, which is advantageous for high-resolution range imaging, particularly for relatively longer distances (>5 m) and under high ambient light operations. In order to enhance the depth resolution, a technique for the depth-adaptive time-gating-number assignment (DATA) for the short-pulse TOF measurement is proposed. A prototype of the 8-tap CMOS lock-in pixel image sensor is implemented with a 1POLY 4METAL 0.11-μm CIS process. The maximum non-linearity error of 1.56%FS for the range of 1–6.4 m and the depth resolution of 6.4 mm was obtained at 6.2 m using the DATA technique.


Introduction
Indirect time-of-flight (TOF) range image sensors are increasingly demanded for various application areas. The application includes smart devices, augmented reality (AR)/virtual reality (VR) systems, security systems, drones, robots, slow vehicles, automated carriers, and automobiles. In order to expand the application areas of TOF imagers, improvements for depth resolution under aggressive conditions, such as longer distance and stronger ambient light, are necessary. Indirect TOF image sensors are classified into two types: a continuous-wave (CW) TOF method [1][2][3][4][5][6][7][8][9] and a short-pulse (SP) TOF method [10][11][12][13][14][15]. In CW-TOF imagers, the high depth resolution and longer distance measurements are simultaneously realized by using higher and multiple (2 or 3) modulation frequencies [5,8]. On the other hand, in the SP-TOF method, the depth resolution is improved by making the gating-time width shorter, and the distance is extended by increasing the number of signal taps of the pixel and by using range-shifting techniques [14][15][16]. For the outdoor use of TOF sensors, the SP-TOF method is more advantageous because a lower amount of ambient light charge is accumulated by using a small-duty, energy-concentrated light pulse and the short gating-time window of the pixel. This results in a high tolerance to the saturation of the pixel and less ambient light shot noise. In addition to this, the pulse-based method has a flexibility regarding the gating clock patterns to be programmed for having advanced functions and performances. Though the reported SP-TOF pixels have maximally four taps for signal outputs [14,15], a SP-TOF pixel with a further number of signal taps will create opportunities of indirect TOF imagers to be used for a variety of applications under aggressive operating conditions. Figure 1 shows the conceptual schematic of multiple-tap lock-in pixels for short-pulse TOF measurements and symbols of the parts. The detailed readout circuits used in the pixel are shown in Figure 6a, though these are not shown in Figure 1 for simplicity. The essential part of the pixel consists of a depleted diode, multiple modulation gates, charge storages (capacitors), and a drain. In short-pulse TOF measurement systems, the light is modulated as a train of light pulses and the reflected light from an object is captured by the TOF camera. The operation timing diagrams for two types of multiple-tap lock-in pixels using a short light pulse (LP) are shown in Figure 2. In the indirect TOF method using short-pulse modulation, the generated signal charge created by receiving the reflected light is transferred and shared by two capacitors, and the time-of-flight is measured by calculating the ratio of one of the two charges to the sum of them. For the 3-tap pixel, the TOF is often measured within the range of one time-gating window, as shown in Figure 2a [12,13], using two gated (G 2 , G 3 ) signals as TOF-dependent signals and another gated signal (G 1 ) for the ambient light charge component. The TOF arrow starts from the G 2 rising edge in this case because the TOF is calculated using the ratio between charges collected by G 2 and G 3 . Using the pixel of this work with 7 signal taps and a drain [18], the depth ranges of six time-gating windows can be measured as shown in the timing diagram of Figure 2b. With the assumption that the TOF measurable range (D max ) and the average light (= "peak power" × "duty cycle ratio of light pulse") for these two cases are the same, a shorter and more energy-concentrated light pulse can be used when increasing the number of signal taps in the lock-in pixels. This property is particularly effective for the operation of the TOF imager under strong ambient light. By increasing the number of signal taps and reducing the width of the resulting time-gating window, the ratio of the signal light to the ambient light intensity is increased and the depth resolution (depth noise) is greatly improved. The depth resolution of the SP-TOF imagers is proportional to the light pulse width (= time-gating width) and the signal-to-noise ratio (SNR). For the operation under no influence from the ambient light and the condition that the photon shot noise of the signal light is dominant for the SNR, the 7-tap pixel with six gating-time windows has a sixfold better depth resolution when compared with that of the 3-tap pixel with a single time-gating window. This is because of the use of the one-sixth shorter light pulse that can be used, and the SNR is unchanged. For the operation under the large influence of ambient light, where the SNR is determined by the shot noise of ambient light, the 7-tap pixel with six time-gating windows has one-sixth of the ambient light charge and 1/ √ 6 of the ambient light shot noise compared with the 3-tap pixel with a single time-gating window. Therefore, theoretically, the 7-tap pixel with six gating-time windows has 6 × √ 6 ≈ 14.7 times better depth resolution when compared with the 3-tap counterpart.
Sensors 2020, 20, 1040 3 of 16 This is because of the use of the one-sixth shorter light pulse that can be used, and the SNR is unchanged. For the operation under the large influence of ambient light, where the SNR is determined by the shot noise of ambient light, the 7-tap pixel with six time-gating windows has onesixth of the ambient light charge and 1 √6 ⁄ of the ambient light shot noise compared with the 3-tap pixel with a single time-gating window. Therefore, theoretically, the 7-tap pixel with six gating-time windows has 6 × √6 14.7 times better depth resolution when compared with the 3-tap counterpart.    This is because of the use of the one-sixth shorter light pulse that can be used, and the SNR is unchanged. For the operation under the large influence of ambient light, where the SNR is determined by the shot noise of ambient light, the 7-tap pixel with six time-gating windows has onesixth of the ambient light charge and 1 √6 ⁄ of the ambient light shot noise compared with the 3-tap pixel with a single time-gating window. Therefore, theoretically, the 7-tap pixel with six gating-time windows has 6 × √6 14.7 times better depth resolution when compared with the 3-tap counterpart.

Depth-Adaptive Time-Gating-Number Assignment for SP-TOF Measurements
In an indirect TOF measurement system, a light source mounted on a TOF camera emits light radially to the 3D measurement space. Then, the intensity of light decreases in a manner that is inversely proportional to the square of the distance from the TOF camera when the target object is assumed to have the perfectly diffusing surface. As a result, the depth resolution becomes the worst at the farthest distance because of the weak signal intensity, while a signal saturation may occur at the closest distance. To address this problem, an efficient SP-TOF measurement technique can be realized using a multiple-tap lock-in pixel imager. Figure 3 shows the SP-TOF measurement timing diagram for considering the dependency of the attenuated reflected light on the distance. In the operation shown in Figure 2b, the gating pulses from G 1 through G 7 are applied consecutively for the closest through to the farthest TOF measurements, respectively. Because of this, the gated signals for the closer objects tend to be too strong, while the gated signals for farther objects tend to be too weak to measure the TOF precisely. In Figure 3, with seven light-pulse cycles as a unit period, the number of gating times for G 1 through G 7 is gradually and adaptively increased into the distance: once per seven cycles for G 1 , twice per seven cycles for G 2 , and so on. With this operation, named the depth-adaptive time-gating-number assignment (DATA), the amount of signal charge at the farthest gated signal of G 7 is maximized while avoiding the saturation in the closest gated signals of G 1 .

Depth-Adaptive Time-Gating-Number Assignment for SP-TOF Measurements
In an indirect TOF measurement system, a light source mounted on a TOF camera emits light radially to the 3D measurement space. Then, the intensity of light decreases in a manner that is inversely proportional to the square of the distance from the TOF camera when the target object is assumed to have the perfectly diffusing surface. As a result, the depth resolution becomes the worst at the farthest distance because of the weak signal intensity, while a signal saturation may occur at the closest distance. To address this problem, an efficient SP-TOF measurement technique can be realized using a multiple-tap lock-in pixel imager. Figure 3 shows the SP-TOF measurement timing diagram for considering the dependency of the attenuated reflected light on the distance. In the operation shown in Figure 2b, the gating pulses from G1 through G7 are applied consecutively for the closest through to the farthest TOF measurements, respectively. Because of this, the gated signals for the closer objects tend to be too strong, while the gated signals for farther objects tend to be too weak to measure the TOF precisely. In Figure 3, with seven light-pulse cycles as a unit period, the number of gating times for G1 through G7 is gradually and adaptively increased into the distance: once per seven cycles for G1, twice per seven cycles for G2, and so on. With this operation, named the depth-adaptive time-gating-number assignment (DATA), the amount of signal charge at the farthest gated signal of G7 is maximized while avoiding the saturation in the closest gated signals of G1.  Figure 4 shows the timing diagram of gating pulses and the corresponding responses to the TOF of a short pulse with the pulse width of for the 7-tap lock-in pixel CMOS imager. The light pulse and the gating pulses of the TOF pixel are repeated with the cycle time of . When the time of flight of the light pulse, , is varied from zero to the maximum time (7 × ), the generated charge acquired by the nth time-gating window, , responds as a triangular shape. Then, the difference between the nth and the (n+2)th gated charge is expressed as: which responds linearly to from × to ( + 2) × , as shown in Figure 4, where n = 1, 2, 3, 4, or 5. If the light pulse is coming when the gates Gn+2 and Gn+1 are activated, the signal charges, qn+2 and qn+1 are generated. If there is an ambient light, qn+2 and qn+1 may also contain ambient light and qn contains an ambient light charge only. Therefore, Equation (1) describes the cancellation of ambient light charge. Because the light pulse and gating pulses are applied repeatedly with the cycle of , the behavior of d1 for the range of TF < 0 in Figure 4 is considered the same as that for the TOF of TC +  Figure 4 shows the timing diagram of gating pulses and the corresponding responses to the TOF of a short pulse with the pulse width of T 0 for the 7-tap lock-in pixel CMOS imager. The light pulse and the gating pulses of the TOF pixel are repeated with the cycle time of T C . When the time of flight of the light pulse, T F , is varied from zero to the maximum time (7 × T 0 ), the generated charge acquired by the nth time-gating window, q n , responds as a triangular shape. Then, the difference between the nth and the (n + 2)th gated charge is expressed as:
If the light pulse is coming when the gates G n+2 and G n+1 are activated, the signal charges, q n+2 and q n+1 are generated. If there is an ambient light, q n+2 and q n+1 may also contain ambient light and q n contains an ambient light charge only. Therefore, Equation (1) describes the cancellation of ambient light charge. Because the light pulse and gating pulses are applied repeatedly with the cycle of T C , the behavior of d 1 for the range of T F < 0 in Figure 4 is considered the same as that for the TOF of and so on. These five curves as a function of T F are used for measuring the TOF of the back-reflected light.
To do this, the variables as a function of T F , i.e., X n (n = 1, 2, . . . , 5) shown in Figure 4, is defined as: where z n is the signal amplitude estimated using q n , q n+1 , and q n+2 , and is expressed as: X n responds linearly to T F from n × T 0 to (n + 2) × T 0 and takes a value from −1 to 1. A normalized signal amplitude Z n is defined as Z n = z n /ẑ whereẑ is the total amount of the generated charges due to the signal light pulse only, and is expressed as: The response of z n andẑ to the distance is also shown in Figure 4. To choose one of the five curves (X n , (n = 1, 2, . . . , 5)) for distance measurements, the flags F n (n = 1, 2, . . . , 5) are defined as: for showing whether X n is larger than 0.5, and the flags E n (n = 1, 2, . . . , 5) are defined as: for showing whether Z n is larger than T Z , where T Z is the threshold for choosing the distance zone. Then, the resulting distance to be measured, D, is expressed as: where c is the velocity of light and n is the zone number determined using: where L n (n = 1, 2, 3, 4) are logical variables defined as L n = F n · E n .
Sensors 2020, 20, 1040 5 of 16 TF, 2TC + TF, 3TC + TF, and so on. These five curves as a function of are used for measuring the TOF of the back-reflected light. To do this, the variables as a function of , i.e., (n = 1, 2, …, 5) shown in Figure 4, is defined as: where is the signal amplitude estimated using , , and , and is expressed as: responds linearly to from × to ( + 2) × and takes a value from −1 to 1. A normalized signal amplitude is defined as = /̂ where ̂ is the total amount of the generated charges due to the signal light pulse only, and is expressed as: The response of and ̂ to the distance is also shown in Figure 4. To choose one of the five curves ( , (n = 1, 2, …, 5)) for distance measurements, the flags (n = 1, 2, …, 5) are defined as: for showing whether Xn is larger than 0.5, and the flags (n = 1, 2, …, 5) are defined as: for showing whether Zn is larger than , where is the threshold for choosing the distance zone. Then, the resulting distance to be measured, D, is expressed as: where c is the velocity of light and n is the zone number determined using: where (n = 1, 2, 3, 4) are logical variables defined as = ⋅ .
(a)   Figure 5a shows the structure (top view) of the designed 8-tap lock-in pixel. A pinned photodiode (PPD) as a photo detector is located in the central region, and the flow of photo-charges to eight directions is controlled by eight modulation gates concentrically placed around the PPD. The modulation gates ( through ) are implemented using a lateral electric field charge modulation (LEFM) [19]. The modulated charge at each tap is stored in a storage diode (SD) [20,21]. The SD basically has a structure of a pinned diode, and while having a sufficient amount of full-well capacitance (FWC) and helping the charge transfer from the SD to the floating diffusion (FD), a charge-transfer assisting (CA) gate is used for each tap [17]. During the charge transfer from the PPD to the SD, the potential well voltage of the SD is raised by the CA gate being biased to a high voltage (2.3 V). When reading the charge by transferring the accumulated charge in the SD to the FD by the transfer (TX) gate, the potential well depth of the SD is lowered by the CA gate to be biased to a low  Figure 5a shows the structure (top view) of the designed 8-tap lock-in pixel. A pinned photodiode (PPD) as a photo detector is located in the central region, and the flow of photo-charges to eight directions is controlled by eight modulation gates concentrically placed around the PPD. The modulation gates (G 1 through G 8 ) are implemented using a lateral electric field charge modulation (LEFM) [19]. The modulated charge at each tap is stored in a storage diode (SD) [20,21]. The SD basically has a structure of a pinned diode, and while having a sufficient amount of full-well capacitance (FWC) and helping the charge transfer from the SD to the floating diffusion (FD), a charge-transfer assisting (CA) gate is used for each tap [17]. During the charge transfer from the PPD to the SD, the potential well voltage of the SD is raised by the CA gate being biased to a high voltage (2.3 V). When reading the charge by transferring the accumulated charge in the SD to the FD by the transfer (TX) gate, the potential well depth of the SD is lowered by the CA gate to be biased to a low voltage (−1.0 V). This operation assists in transferring the charge from the SD to the FD while preventing backward charge retransfer from the FD to the SD.

Pixel Structure
voltage (−1.0 V). This operation assists in transferring the charge from the SD to the FD while preventing backward charge retransfer from the FD to the SD. In this pixel, three n-type doping layers are used under the p-type layer for the PPD. The first ntype layer is used entirely for the PPD, LEFM gates, and SDs; the second n-layer is mainly used for the LEFM and the SDs; and the third n-layer is used mainly for the SDs. This triple n-doping helps us to create depleted potential profiles to accelerate photo carriers from the PPD to the selected SDs (one of eight SDs) activated by one of the eight LEFM gates. Figure 5b,c shows the 3D device simulation results when the generated photo-carriers are to be transferred to . The LEFM gate and CA gates are biased to a high voltage (2.3 V) and the other LEFM gates ( through and ) are biased to a low voltage (−2 V). It can be seen that a potential gradient is generated in the photo-sensing area to transfer photo-carriers to the , as shown in Figure 5c. In Figure 5b, red points denote the initial positions (depth: 3 μm) of an electron generated by a photon and black dotted lines indicate the movement trace of an electron. The transfer time of carriers from the initial position to the , which is calculated by the velocity of carriers as a function of the electric field, is 990 ps. Figure 5d shows the potential diagrams along with the X-X' lines shown in Figure 5a when the accumulated carriers in the SD are transferred to the FD. In this case, the TX gates are biased to a high voltage (3.3 V) and the LEFM gates ( through ) and CA gates are biased to a low voltage (LEFM gate: −2 V, CA gate: −1 V). As can be seen from this result, the channel potentials around and are relatively low and these potentials prevent the charges from transferring to the untargeted SDs, while the CA gate creates a large potential gradient, which attracts carriers to be transferred to the FD. In this pixel, three n-type doping layers are used under the p-type layer for the PPD. The first n-type layer is used entirely for the PPD, LEFM gates, and SDs; the second n-layer is mainly used for the LEFM and the SDs; and the third n-layer is used mainly for the SDs. This triple n-doping helps us to create depleted potential profiles to accelerate photo carriers from the PPD to the selected SDs (one of eight SDs) activated by one of the eight LEFM gates. Figure 5b,c shows the 3D device simulation results when the generated photo-carriers are to be transferred to SD 7 . The LEFM gate G 7 and CA gates are biased to a high voltage (2.3 V) and the other LEFM gates (G 1 through G 6 and G 8 ) are biased to a low voltage (−2 V). It can be seen that a potential gradient is generated in the photo-sensing area to transfer photo-carriers to the SD 7 , as shown in Figure 5c. In Figure 5b, red points denote the initial positions (depth: 3 µm) of an electron generated by a photon and black dotted lines indicate the movement trace of an electron. The transfer time of carriers from the initial position to the SD 7 , which is calculated by the velocity of carriers as a function of the electric field, is 990 ps. Figure 5d shows the potential diagrams along with the X-X' lines shown in Figure 5a when the accumulated carriers in the SD are transferred to the FD. In this case, the TX gates are biased to a high voltage (3.3 V) and the LEFM gates (G 1 through G 8 ) and CA gates are biased to a low voltage (LEFM gate: −2 V, CA gate: −1 V). As can be seen from this result, the channel potentials around SD 1 and SD 7 are relatively low and these potentials prevent the charges from transferring to the untargeted SDs, while the CA gate creates a large potential gradient, which attracts carriers to be transferred to the FD.  Figure 6a,b shows the pixel schematic with readout circuits and a timing diagram for accumulating and reading photo signals in each pixel. The pixel has four parallel outputs, and four source follower amplifiers are shared with two adjacent SDs to save the in-pixel circuit area. As shown in Figure 6b, the TOF image acquisition cycle, or frame period, is divided into an accumulation phase and a readout phase. In the accumulation phase, the eight modulation gates are driven as shown in the timing of Figure 2b or Figure 3. After the accumulation phase, the readout phase is started. The readout process from the pixel array is done on a row-by-row basis. To read out all the accumulated signals in the SDs, the readout operation cycle is repeated twice. In the first cycle, the floating diffusion node is reset using reset gates (RT 1 and RT 2 ), then the TX 1 gate is turned on for reading out the accumulated signals in SD 1 , SD 3 , SD 5 , and SD 7 . For assisting the charge transfer from the SDs, the CA gate is used. The action of the CA gates is a little delayed (about 10 ns) from the turn-on and turn-off timings of the TX gate. In the second cycle, the floating diffusion node is reset using RT 1 ; then, the TX 2 gate is turned on for reading out the accumulated signals in SD 2 , SD 4 , and SD 6 . In the pixel operation for reading 7-tap signal outputs, the node for SD 8 is used for draining such that the FD 4 node is always reset by setting RT 2 to high during the second cycle. The four signals from the pixel are read out using analog CDS (correlated double sampling) circuits at the column and are converted to digital signals using column analog-to-digital converters (ADCs).  Figure 6a,b shows the pixel schematic with readout circuits and a timing diagram for accumulating and reading photo signals in each pixel. The pixel has four parallel outputs, and four source follower amplifiers are shared with two adjacent SDs to save the in-pixel circuit area. As shown in Figure 6b, the TOF image acquisition cycle, or frame period, is divided into an accumulation phase and a readout phase. In the accumulation phase, the eight modulation gates are driven as   Figure 7 shows a photo micrograph of a prototype 8-tap lock-in-pixel TOF imager chip implemented using a 1Poly 4Metal 0.11-µm CMOS image sensor process. The pixel array had a total of 134(H) × 128(V) pixels, including test pixels. An inverter tree and a driver for pixel gating were arranged above and below the pixel array for faster gate responses. The pixel outputs were read out by using a 19-bit column ADC [22]. The converted digital signal was horizontally read out using low-voltage differential signaling (LVDS) circuits. Table 1 shows a brief summary of the prototype TOF imager specification and basic characteristics. The chip was assembled to the cooling package, had Peltier elements, and was cooled down to roughly −15 • C to suppress the influence of the dark current in the measurements.

Implemented TOF Imager Chip
The relatively large dark current in the present design was related to the structure around the CA gate, which must be improved for further low-noise performance of the chip. The readout time of 51.84 ms was very long for the imager given the relatively small number of pixels. One reason for this long readout time was the use of a multiple-sampling based column ADC [22]. The implemented chip had a non-linearity problem if the operation clock of the ADC was increased. To reduce the non-linearity, the following measurement data were taken with a slower clock for the ADC than that of the designed specification. The readout speed could be improved by re-designing the column ADC of the chip.
Sensors 2020, 20, 1040 9 of 16 Figure 7 shows a photo micrograph of a prototype 8-tap lock-in-pixel TOF imager chip implemented using a 1Poly 4Metal 0.11-μm CMOS image sensor process. The pixel array had a total of 134(H) × 128(V) pixels, including test pixels. An inverter tree and a driver for pixel gating were arranged above and below the pixel array for faster gate responses. The pixel outputs were read out by using a 19-bit column ADC [22]. The converted digital signal was horizontally read out using lowvoltage differential signaling (LVDS) circuits. Table 1 shows a brief summary of the prototype TOF imager specification and basic characteristics. The chip was assembled to the cooling package, had Peltier elements, and was cooled down to roughly −15 °C to suppress the influence of the dark current in the measurements. The relatively large dark current in the present design was related to the structure around the CA gate, which must be improved for further low-noise performance of the chip. The readout time of 51.84 ms was very long for the imager given the relatively small number of pixels. One reason for this long readout time was the use of a multiple-sampling based column ADC [22]. The implemented chip had a non-linearity problem if the operation clock of the ADC was increased. To reduce the non-linearity, the following measurement data were taken with a slower clock for the ADC than that of the designed specification. The readout speed could be improved by re-designing the column ADC of the chip.     Figure 8 shows a measurement result of the response of seven signal-tap outputs to the light pulse delay. A laser pulse with a pulse width of 6 ns, a wavelength of 850 nm, a power of 29 mW/m 2 (0.88 W/m 2 peak) at 6 m and a laser pulse rate of 5.5 MHz was used. A lens with a large focal length of 12.5 mm was used For the sensor characterization, the sensor field-of-view (FOV) used for the measurement was 13.1 • (V) × 13.7 • (H). In this measurement, the modulation gates, G 1 through G 7 , were opened as shown in the timing of Figure 2b with T 0 = 6 ns, and the delay of the laser pulse was scanned from 0 to 60 ns with a step of 0.4 ns. The gate G 8 was used as a charge draining gate and the SD 8 was working as a drain. Figure 8a shows measured data after subtracting the dark signal from the raw data and Figure 8b shows a normalized result with a sensitivity correction. The average demodulation contrast in the charge signals stored in SDs, for q 1 through q 7 , was 0.83. As shown in Figure 8, the response of the seven signals to the pulse delay time was fast enough for the short-pulse TOF measurement with the light pulse width of 6 ns when compared with the ideal response (triangular shape), where it was rising linearly over 6 ns and falling linearly over 6 ns. Using the normalized response of Figure 8b, the differences of the two signal-tap outputs (d 1 , d 2 , d 3 , d 4 , d 5 ) were calculated and are shown in Figure 9a. The five piecewise-linear response curves, two of which overlap each other, were obtained and these were used for the TOF range measurements by switching these five curves properly, as described in Section 2.3. Figure 9b shows a curve for the calculated (measured) distance, i.e., the pseudo distance, which corresponded to twice the TOF. This linearity plot was obtained using a calibration applying the gain and offset corrections to the five piecewise-linear response curves. After this non-linearity correction, the maximum of the measured distance error was 3.7 cm and the maximum non-linearity error was 0.67%FS in the range of 0 m to 5.4 m.

Charge Modulation Characteristics
Sensors 2020, 20, 1040 10 of 16 Figure 8 shows a measurement result of the response of seven signal-tap outputs to the light pulse delay. A laser pulse with a pulse width of 6 ns, a wavelength of 850 nm, a power of 29 mW/m 2 (0.88 W/m 2 peak) at 6 m and a laser pulse rate of 5.5 MHz was used. A lens with a large focal length of 12.5 mm was used For the sensor characterization, the sensor field-of-view (FOV) used for the measurement was 13.1°(V) × 13.7°(H). In this measurement, the modulation gates, G1 through G7, were opened as shown in the timing of Figure 2b with T0 = 6 ns, and the delay of the laser pulse was scanned from 0 to 60 ns with a step of 0.4 ns. The gate G8 was used as a charge draining gate and the was working as a drain. Figure 8a shows measured data after subtracting the dark signal from the raw data and Figure 8b shows a normalized result with a sensitivity correction. The average demodulation contrast in the charge signals stored in SDs, for through , was 0.83. As shown in Figure 8, the response of the seven signals to the pulse delay time was fast enough for the short-pulse TOF measurement with the light pulse width of 6 ns when compared with the ideal response (triangular shape), where it was rising linearly over 6 ns and falling linearly over 6 ns. Using the normalized response of Figure 8b, the differences of the two signal-tap outputs ( , , , , ) were calculated and are shown in Figure 9a. The five piecewise-linear response curves, two of which overlap each other, were obtained and these were used for the TOF range measurements by switching these five curves properly, as described in Section 2.3. Figure 9b shows a curve for the calculated (measured) distance, i.e., the pseudo distance, which corresponded to twice the TOF. This linearity plot was obtained using a calibration applying the gain and offset corrections to the five piecewiselinear response curves. After this non-linearity correction, the maximum of the measured distance error was 3.7 cm and the maximum non-linearity error was 0.67%FS in the range of 0 m to 5.4 m.

Distance Measurements
Using a similar calculation method for the TOF described in Section 4.2, distance measurements were carried out with two types of gating methods, as shown in Figure 2b (ordinary time-gating method) and Figure 3 (depth-adaptive time-gating-number assignment (DATA) method). The emitted laser pulse width and pixel-gating pulse width were 6 ns. The measured object was a white panel with the reflection factor of 99% and a fully diffusing surface. The white panel was moved with a 0.1 m step from 1.0 m to 6.4 m from the TOF camera. The range linearity and depth resolution were measured using 100 frames. In the ordinary gating method, the accumulation time in all SDs was set to 0.217 ms. This setting of a short accumulation time was necessary to avoid the saturation in the SDs that received a strong light power due to the reflections with very close objects. The SD1 was the most critical and the SD2 the second most critical. Figure 10 shows the distance measurement results of the non-linearity error (Figure 10a) and depth resolution (Figure 10b).

Distance Measurements
Using a similar calculation method for the TOF described in Section 4.2, distance measurements were carried out with two types of gating methods, as shown in Figure 2b (ordinary time-gating method) and Figure 3 (depth-adaptive time-gating-number assignment (DATA) method). The emitted laser pulse width and pixel-gating pulse width were 6 ns. The measured object was a white panel with the reflection factor of 99% and a fully diffusing surface. The white panel was moved with a 0.1 m step from 1.0 m to 6.4 m from the TOF camera. The range linearity and depth resolution were measured using 100 frames. In the ordinary gating method, the accumulation time in all SDs was set to 0.217 ms. This setting of a short accumulation time was necessary to avoid the saturation in the SDs that received a strong light power due to the reflections with very close objects. The SD 1 was the most critical and the SD 2 the second most critical. Figure 10 shows the distance measurement results of the non-linearity error (Figure 10a) and depth resolution (Figure 10b). The pixel area of 11(H) × 11(V) is used for the measurements. The threshold for calculating the distance used in Equation (6) was set to 0.5. The maximum measured distance error was 6.2 cm and the maximum non-linearity error was 1.15%FS for the full range of 1-6.4 m. For the case of excluding the near end of the measurement range, the measured distance error was less than 5.4 cm, which means the non-linearity error was less than 1%FS. The depth resolution in this method was less than 20 mm up to 2.6 m, and it increased to 128 mm at the maximum distance. The degradation of the depth resolution with increasing distance was due to the attenuated reflected light and the limited FWC of the pixel. Figure 11 shows the distance measurement results using the DATA method shown in Figure 3. In this method, the accumulation times in through were 0.169 ms, 0.384 ms, 1.008 ms, 1.794 ms, 2.187 ms, 3.760 ms, and 4.153 ms, respectively. The pixel area of 11(H) × 11(V) was used for the measurements. The maximum of the measured distance error was 8.4 cm and the maximum nonlinearity error was 1.56%FS for the full range of 1-6.4 m. For the measurement ranges larger than 1.5 m, the measured distance error was less than 5.4 cm, which means the non-linearity error was less than 1%FS, as shown in Figure 11a. Figure 11b shows the depth resolution as a function of the distance. The pixel area of 11(H) × 11(V) is used for the measurements. The threshold T Z for calculating the distance used in Equation (6) was set to 0.5. The maximum measured distance error was 6.2 cm and the maximum non-linearity error was 1.15%FS for the full range of 1-6.4 m. For the case of excluding the near end of the measurement range, the measured distance error was less than 5.4 cm, which means the non-linearity error was less than 1%FS. The depth resolution in this method was less than 20 mm up to 2.6 m, and it increased to 128 mm at the maximum distance. The degradation of the depth resolution with increasing distance was due to the attenuated reflected light and the limited FWC of the pixel. Figure 11 shows the distance measurement results using the DATA method shown in Figure 3. In this method, the accumulation times in SD 1 through SD 7 were 0.169 ms, 0.384 ms, 1.008 ms, 1.794 ms, 2.187 ms, 3.760 ms, and 4.153 ms, respectively. The pixel area of 11(H) × 11(V) was used for the measurements. The maximum of the measured distance error was 8.4 cm and the maximum non-linearity error was 1.56%FS for the full range of 1-6.4 m. For the measurement ranges larger than 1.5 m, the measured distance error was less than 5.4 cm, which means the non-linearity error was less than 1%FS, as shown in Figure 11a. Figure 11b shows the depth resolution as a function of the distance. Using the DATA method, where the optimum accumulation time was assigned depending on the reflected light intensity, a good depth resolution smaller than 13 mm was obtained for the entire measurement range and the resolution of 6.4 mm was obtained at 6.2 m. The depth resolution corresponded to 0.10% of the depth. Figure 12 shows a depth image with the implemented TOF image sensors using the operation of the depth-adaptive time-gating-number assignment technique. This depth image with only the subset (80(H) × 81(V) pixels) of the entire pixel array (134(H) × 128(V)) is shown because of the relatively small irradiating angle of the laser light used. In the scene, a white box was placed at 2.1 m and 4.7 m, a manikin was placed at 3.1 m, and the white screen was placed at 6.0 m as a background. The threshold for calculating the distance used in Equation (6) was set to 0.3. This image was generated by using one frame of data. Using the DATA method, where the optimum accumulation time was assigned depending on the reflected light intensity, a good depth resolution smaller than 13 mm was obtained for the entire measurement range and the resolution of 6.4 mm was obtained at 6.2 m. The depth resolution corresponded to 0.10% of the depth. Figure 12 shows a depth image with the implemented TOF image sensors using the operation of the depth-adaptive time-gating-number assignment technique. This depth image with only the subset (80(H) × 81(V) pixels) of the entire pixel array (134(H) × 128(V)) is shown because of the relatively small irradiating angle of the laser light used. In the scene, a white box was placed at 2.1 m and 4.7 m, a manikin was placed at 3.1 m, and the white screen was placed at 6.0 m as a background. The threshold T Z for calculating the distance used in Equation (6) was set to 0.3. This image was generated by using one frame of data.

Conclusions
This paper presents an 8-tap CMOS lock-in pixel image sensor for short-pulse TOF measurements. The proposed operation of the 8-tap lock-in pixel image sensor using seven taps for signal outputs and a tap for draining was suitable for high-resolution, single-frame TOF measurements using small-duty short light pulses. The implemented prototype had a good response to a short light pulses, where a pulse width of 6 ns was used for TOF measurements. The pulse-based TOF measurements with multiple-tap pixels allowed us to use a flexible time-gating pattern setting. The proposed technique with depth adaptive time-gating-number assignment was effective for realizing a wide measurement range and a high depth resolution. Using this technique and the 8-tap pixel image sensor, the high depth resolution of 6.4 mm at 6.2 m (0.10% of depth) and better than 13 mm within whole depth range of 1 m to 6.4 m were obtained. Because of the segmented TOF measurements with seven time-gating windows, the linearity was also relatively good. For the case of pseudo distance measurements using a delay generator for pseudo TOF generation, a maximum nonlinearity error of 0.67%FS was demonstrated for the distance range from 0 m to 5.4 m. Distance calculation algorithms consecutively using the 7-tap outputs were developed and depth images covering 1 m to 6.4 m were successfully obtained.
Author Contributions: S.K. proposed the device concept and provided the overall guidance of the project. Y.S., S.K., and K.Y. designed the detector, overall circuit, and layout for the chip. S.A. designed the ADC circuit. Y.S. also measured the chip. Y.S., S.K., K.Y., and K.K. discussed the measurement results. Y.S. and S.K. drafted the manuscript and the final revision was approved by all the authors.

Conclusions
This paper presents an 8-tap CMOS lock-in pixel image sensor for short-pulse TOF measurements. The proposed operation of the 8-tap lock-in pixel image sensor using seven taps for signal outputs and a tap for draining was suitable for high-resolution, single-frame TOF measurements using small-duty short light pulses. The implemented prototype had a good response to a short light pulses, where a pulse width of 6 ns was used for TOF measurements. The pulse-based TOF measurements with multiple-tap pixels allowed us to use a flexible time-gating pattern setting. The proposed technique with depth adaptive time-gating-number assignment was effective for realizing a wide measurement range and a high depth resolution. Using this technique and the 8-tap pixel image sensor, the high depth resolution of 6.4 mm at 6.2 m (0.10% of depth) and better than 13 mm within whole depth range of 1 m to 6.4 m were obtained. Because of the segmented TOF measurements with seven time-gating windows, the linearity was also relatively good. For the case of pseudo distance measurements using a delay generator for pseudo TOF generation, a maximum non-linearity error of 0.67%FS was demonstrated for the distance range from 0 m to 5.4 m. Distance calculation algorithms consecutively using the 7-tap outputs were developed and depth images covering 1 m to 6.4 m were successfully obtained.
Author Contributions: S.K. proposed the device concept and provided the overall guidance of the project. Y.S., S.K., and K.Y. designed the detector, overall circuit, and layout for the chip. S.A. designed the ADC circuit. Y.S. also measured the chip. Y.S., S.K., K.Y., and K.K. discussed the measurement results. Y.S. and S.K. drafted the manuscript. All authors have read and agreed to the published version of the manuscript.