Next Article in Journal
Harmonic Distortion Optimization for Sigma-Delta Modulators Interface Circuit of TMR Sensors
Next Article in Special Issue
Over 100 Million Frames per Second 368 Frames Global Shutter Burst CMOS Image Sensor with Pixel-wise Trench Capacitor Memory Array
Previous Article in Journal
Recent Advances in Electrochemical and Optical Sensing of Dopamine
Previous Article in Special Issue
Fully Depleted, Trench-Pinned Photo Gate for CMOS Image Sensor Applications
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

An 8-Tap CMOS Lock-In Pixel Image Sensor for Short-Pulse Time-of-Flight Measurements

1
Graduate School of Medical Photonics, Shizuoka University, Hamamatsu 432-8011, Japan
2
Research Institute of Electronics, Shizuoka University, Hamamatsu 432-8011, Japan
3
Brookman Technology, Inc., Hamamatsu 430-0936, Japan
*
Author to whom correspondence should be addressed.
Sensors 2020, 20(4), 1040; https://doi.org/10.3390/s20041040
Submission received: 26 November 2019 / Revised: 11 February 2020 / Accepted: 11 February 2020 / Published: 14 February 2020

Abstract

:
An 8-tap CMOS lock-in pixel image sensor that has seven carrier-capturing and a draining time window was developed for short-pulse time-of-flight (TOF) measurements. The proposed pixel for the short-pulse TOF measurements has seven consecutive time-gating windows, each of which has the width of 6 ns, which is advantageous for high-resolution range imaging, particularly for relatively longer distances (>5 m) and under high ambient light operations. In order to enhance the depth resolution, a technique for the depth-adaptive time-gating-number assignment (DATA) for the short-pulse TOF measurement is proposed. A prototype of the 8-tap CMOS lock-in pixel image sensor is implemented with a 1POLY 4METAL 0.11-μm CIS process. The maximum non-linearity error of 1.56%FS for the range of 1–6.4 m and the depth resolution of 6.4 mm was obtained at 6.2 m using the DATA technique.

1. Introduction

Indirect time-of-flight (TOF) range image sensors are increasingly demanded for various application areas. The application includes smart devices, augmented reality (AR)/virtual reality (VR) systems, security systems, drones, robots, slow vehicles, automated carriers, and automobiles. In order to expand the application areas of TOF imagers, improvements for depth resolution under aggressive conditions, such as longer distance and stronger ambient light, are necessary. Indirect TOF image sensors are classified into two types: a continuous-wave (CW) TOF method [1,2,3,4,5,6,7,8,9] and a short-pulse (SP) TOF method [10,11,12,13,14,15]. In CW-TOF imagers, the high depth resolution and longer distance measurements are simultaneously realized by using higher and multiple (2 or 3) modulation frequencies [5,8]. On the other hand, in the SP-TOF method, the depth resolution is improved by making the gating-time width shorter, and the distance is extended by increasing the number of signal taps of the pixel and by using range-shifting techniques [14,15,16]. For the outdoor use of TOF sensors, the SP-TOF method is more advantageous because a lower amount of ambient light charge is accumulated by using a small-duty, energy-concentrated light pulse and the short gating-time window of the pixel. This results in a high tolerance to the saturation of the pixel and less ambient light shot noise. In addition to this, the pulse-based method has a flexibility regarding the gating clock patterns to be programmed for having advanced functions and performances. Though the reported SP-TOF pixels have maximally four taps for signal outputs [14,15], a SP-TOF pixel with a further number of signal taps will create opportunities of indirect TOF imagers to be used for a variety of applications under aggressive operating conditions.
This paper presents an 8-tap lock-in pixel image sensor [17,18] for SP-TOF measurements. The pixel has eight symmetrical modulation gates, eight charge storage diodes (SDs) with respective transfer gates, and readout source follower amplifiers. The pixel has a function of programming where one of the 8-tap nodes is used for time-gated charge draining and other seven taps are used for time-gated signal outputs. In order to increase the full well capacitance (FWC) and to assist complete charge transfer from the SDs to floating diffusions (FDs), a charge-transfer assisting (CA) gate is implemented in each SD. In Shirakawa et al. [18], a basic operation of the 8-tap lock-in pixels for TOF measurements is shown. The present paper reports a new SP-TOF measurement technique using the 8-tap lock-in pixels and the depth-adaptive time-gating-number assignment (DATA) technique is used, where the number of gating times for the 7-tap outputs are adaptively changed to the distance by considering the dependency of the attenuated reflected light to the distance. Having this technique exploit the feature of flexible programming of gating pulse patterns in SP-TOF method is very effective for obtaining relatively high depth resolution for the entire measurement range by assigning larger gating numbers at farther regions, while suppressing the saturation of signals in the nearest regions by assigning fewer gating numbers. Using this technique, the linearity and range resolution of the 7-tap TOF measurements and TOF range image samples are evaluated.
The rest of this paper is organized as follows. In Section 2, the operation of the multiple-tap lock-in pixel and the distance calculation algorithm are described. Section 3 provides the design of the proposed 8-tap lock-in pixel structure using device simulations. The experimental results of the implemented CMOS 8-tap chip are described in Section 4. Section 5 presents the conclusion.

2. Multiple-Tap Lock-In-Pixel and Short-Pulse TOF Measurements

2.1. Operation Timing of Multiple-Tap Lock-In-Pixel TOF Imagers

Figure 1 shows the conceptual schematic of multiple-tap lock-in pixels for short-pulse TOF measurements and symbols of the parts. The detailed readout circuits used in the pixel are shown in Figure 6a, though these are not shown in Figure 1 for simplicity. The essential part of the pixel consists of a depleted diode, multiple modulation gates, charge storages (capacitors), and a drain. In short-pulse TOF measurement systems, the light is modulated as a train of light pulses and the reflected light from an object is captured by the TOF camera. The operation timing diagrams for two types of multiple-tap lock-in pixels using a short light pulse (LP) are shown in Figure 2. In the indirect TOF method using short-pulse modulation, the generated signal charge created by receiving the reflected light is transferred and shared by two capacitors, and the time-of-flight is measured by calculating the ratio of one of the two charges to the sum of them. For the 3-tap pixel, the TOF is often measured within the range of one time-gating window, as shown in Figure 2a [12,13], using two gated ( G 2 , G 3 ) signals as TOF-dependent signals and another gated signal ( G 1 ) for the ambient light charge component. The TOF arrow starts from the G 2 rising edge in this case because the TOF is calculated using the ratio between charges collected by G 2 and G 3 . Using the pixel of this work with 7 signal taps and a drain [18], the depth ranges of six time-gating windows can be measured as shown in the timing diagram of Figure 2b. With the assumption that the TOF measurable range (Dmax) and the average light (= “peak power” × “duty cycle ratio of light pulse”) for these two cases are the same, a shorter and more energy-concentrated light pulse can be used when increasing the number of signal taps in the lock-in pixels. This property is particularly effective for the operation of the TOF imager under strong ambient light. By increasing the number of signal taps and reducing the width of the resulting time-gating window, the ratio of the signal light to the ambient light intensity is increased and the depth resolution (depth noise) is greatly improved. The depth resolution of the SP-TOF imagers is proportional to the light pulse width (= time-gating width) and the signal-to-noise ratio (SNR). For the operation under no influence from the ambient light and the condition that the photon shot noise of the signal light is dominant for the SNR, the 7-tap pixel with six gating-time windows has a sixfold better depth resolution when compared with that of the 3-tap pixel with a single time-gating window. This is because of the use of the one-sixth shorter light pulse that can be used, and the SNR is unchanged. For the operation under the large influence of ambient light, where the SNR is determined by the shot noise of ambient light, the 7-tap pixel with six time-gating windows has one-sixth of the ambient light charge and 1 / 6 of the ambient light shot noise compared with the 3-tap pixel with a single time-gating window. Therefore, theoretically, the 7-tap pixel with six gating-time windows has 6 × 6 14.7 times better depth resolution when compared with the 3-tap counterpart.

2.2. Depth-Adaptive Time-Gating-Number Assignment for SP-TOF Measurements

In an indirect TOF measurement system, a light source mounted on a TOF camera emits light radially to the 3D measurement space. Then, the intensity of light decreases in a manner that is inversely proportional to the square of the distance from the TOF camera when the target object is assumed to have the perfectly diffusing surface. As a result, the depth resolution becomes the worst at the farthest distance because of the weak signal intensity, while a signal saturation may occur at the closest distance. To address this problem, an efficient SP-TOF measurement technique can be realized using a multiple-tap lock-in pixel imager.
Figure 3 shows the SP-TOF measurement timing diagram for considering the dependency of the attenuated reflected light on the distance. In the operation shown in Figure 2b, the gating pulses from G1 through G7 are applied consecutively for the closest through to the farthest TOF measurements, respectively. Because of this, the gated signals for the closer objects tend to be too strong, while the gated signals for farther objects tend to be too weak to measure the TOF precisely. In Figure 3, with seven light-pulse cycles as a unit period, the number of gating times for G1 through G7 is gradually and adaptively increased into the distance: once per seven cycles for G1, twice per seven cycles for G2, and so on. With this operation, named the depth-adaptive time-gating-number assignment (DATA), the amount of signal charge at the farthest gated signal of G7 is maximized while avoiding the saturation in the closest gated signals of G1.

2.3. Distance Calculation Algorithm

Figure 4 shows the timing diagram of gating pulses and the corresponding responses to the TOF of a short pulse with the pulse width of T 0 for the 7-tap lock-in pixel CMOS imager. The light pulse and the gating pulses of the TOF pixel are repeated with the cycle time of T C . When the time of flight of the light pulse, T F , is varied from zero to the maximum time (7 × T 0 ), the generated charge acquired by the nth time-gating window, q n , responds as a triangular shape. Then, the difference between the nth and the (n + 2)th gated charge is expressed as:
d n = q n + 2 q n ,
which responds linearly to T F from n × T 0 to ( n + 2 ) × T 0 , as shown in Figure 4, where n = 1, 2, 3, 4, or 5. If the light pulse is coming when the gates Gn+2 and Gn+1 are activated, the signal charges, qn+2 and qn+1 are generated. If there is an ambient light, qn+2 and qn+1 may also contain ambient light and qn contains an ambient light charge only. Therefore, Equation (1) describes the cancellation of ambient light charge. Because the light pulse and gating pulses are applied repeatedly with the cycle of T C , the behavior of d1 for the range of TF < 0 in Figure 4 is considered the same as that for the TOF of TC + TF, 2TC + TF, 3TC + TF, and so on. These five curves as a function of T F are used for measuring the TOF of the back-reflected light. To do this, the variables as a function of T F , i.e., X n (n = 1, 2, …, 5) shown in Figure 4, is defined as:
X n = d n / z n ,
where z n is the signal amplitude estimated using q n , q n + 1 , and q n + 2 , and is expressed as:
z n = { q n + q n + 1 2 q n + 2 q n + 1 + q n + 2 2 q n ( q n q n + 2 > 0 ) ( q n q n + 2 0 ) .
X n responds linearly to T F from n × T 0 to ( n + 2 ) × T 0 and takes a value from −1 to 1. A normalized signal amplitude Z n is defined as Z n = z n / z ^ where z ^ is the total amount of the generated charges due to the signal light pulse only, and is expressed as:
z ^ = | q 3 q 1 | + q 2 + q 4 + q 6 + | q 7 q 5 | 3 2 ( | q 7 + q 5 | + | q 7 q 5 | ) .
The response of z n and z ^ to the distance is also shown in Figure 4. To choose one of the five curves ( X n , (n = 1, 2, …, 5)) for distance measurements, the flags F n (n = 1, 2, …, 5) are defined as:
F n = { 1 0 ( X n 0.5 ) ( X n > 0.5 ) ;
for showing whether Xn is larger than 0.5, and the flags E n (n = 1, 2, …, 5) are defined as:
E n = { 1 0 ( Z n > T Z ) ( Z n T Z )
for showing whether Zn is larger than T Z , where T Z is the threshold for choosing the distance zone. Then, the resulting distance to be measured, D, is expressed as:
D = c 2 T 0 ( n + X n ) ,
where c is the velocity of light and n is the zone number determined using:
n = { 1 ( if   L 1 = 1 ) 2 ( if   L 2 L 1 ¯ = 1 ) 3 ( if   L 3 L 2 ¯ L 1 ¯ = 1 ) 4 ( if   L 4 L 3 ¯ L 2 ¯ L 1 ¯ = 1 ) 5 ( if   L 4 ¯ L 3 ¯ L 2 ¯ L 1 ¯ = 1 ) ,
where L n (n = 1, 2, 3, 4) are logical variables defined as L n = F n E n .

3. Design of an 8-Tap Lock-In Pixel Image Sensor for SP-TOF Measurements

3.1. Pixel Structure

Figure 5a shows the structure (top view) of the designed 8-tap lock-in pixel. A pinned photodiode (PPD) as a photo detector is located in the central region, and the flow of photo-charges to eight directions is controlled by eight modulation gates concentrically placed around the PPD. The modulation gates ( G 1 through G 8 ) are implemented using a lateral electric field charge modulation (LEFM) [19]. The modulated charge at each tap is stored in a storage diode (SD) [20,21]. The SD basically has a structure of a pinned diode, and while having a sufficient amount of full-well capacitance (FWC) and helping the charge transfer from the SD to the floating diffusion (FD), a charge-transfer assisting (CA) gate is used for each tap [17]. During the charge transfer from the PPD to the SD, the potential well voltage of the SD is raised by the CA gate being biased to a high voltage (2.3 V). When reading the charge by transferring the accumulated charge in the SD to the FD by the transfer (TX) gate, the potential well depth of the SD is lowered by the CA gate to be biased to a low voltage (−1.0 V). This operation assists in transferring the charge from the SD to the FD while preventing backward charge retransfer from the FD to the SD.
In this pixel, three n-type doping layers are used under the p-type layer for the PPD. The first n-type layer is used entirely for the PPD, LEFM gates, and SDs; the second n-layer is mainly used for the LEFM and the SDs; and the third n-layer is used mainly for the SDs. This triple n-doping helps us to create depleted potential profiles to accelerate photo carriers from the PPD to the selected SDs (one of eight SDs) activated by one of the eight LEFM gates.
Figure 5b,c shows the 3D device simulation results when the generated photo-carriers are to be transferred to S D 7 . The LEFM gate G 7 and CA gates are biased to a high voltage (2.3 V) and the other LEFM gates ( G 1 through G 6 and G 8 ) are biased to a low voltage (−2 V). It can be seen that a potential gradient is generated in the photo-sensing area to transfer photo-carriers to the S D 7 , as shown in Figure 5c. In Figure 5b, red points denote the initial positions (depth: 3 μm) of an electron generated by a photon and black dotted lines indicate the movement trace of an electron. The transfer time of carriers from the initial position to the S D 7 , which is calculated by the velocity of carriers as a function of the electric field, is 990 ps.
Figure 5d shows the potential diagrams along with the X–X’ lines shown in Figure 5a when the accumulated carriers in the SD are transferred to the FD. In this case, the TX gates are biased to a high voltage (3.3 V) and the LEFM gates ( G 1 through G 8 ) and CA gates are biased to a low voltage (LEFM gate: −2 V, CA gate: −1 V). As can be seen from this result, the channel potentials around S D 1 and S D 7 are relatively low and these potentials prevent the charges from transferring to the untargeted SDs, while the CA gate creates a large potential gradient, which attracts carriers to be transferred to the FD.

3.2. Pixel Circuits and Readout Operations

Figure 6a,b shows the pixel schematic with readout circuits and a timing diagram for accumulating and reading photo signals in each pixel. The pixel has four parallel outputs, and four source follower amplifiers are shared with two adjacent SDs to save the in-pixel circuit area. As shown in Figure 6b, the TOF image acquisition cycle, or frame period, is divided into an accumulation phase and a readout phase. In the accumulation phase, the eight modulation gates are driven as shown in the timing of Figure 2b or Figure 3. After the accumulation phase, the readout phase is started. The readout process from the pixel array is done on a row-by-row basis. To read out all the accumulated signals in the SDs, the readout operation cycle is repeated twice. In the first cycle, the floating diffusion node is reset using reset gates ( R T 1 and R T 2 ), then the T X 1 gate is turned on for reading out the accumulated signals in S D 1 , S D 3 , S D 5 , and S D 7 . For assisting the charge transfer from the SDs, the CA gate is used. The action of the CA gates is a little delayed (about 10 ns) from the turn-on and turn-off timings of the TX gate. In the second cycle, the floating diffusion node is reset using R T 1 ; then, the T X 2 gate is turned on for reading out the accumulated signals in S D 2 , S D 4 , and S D 6 . In the pixel operation for reading 7-tap signal outputs, the node for S D 8 is used for draining such that the F D 4 node is always reset by setting R T 2 to high during the second cycle. The four signals from the pixel are read out using analog CDS (correlated double sampling) circuits at the column and are converted to digital signals using column analog-to-digital converters (ADCs).

4. Implementation and Experimental Results

4.1. Implemented TOF Imager Chip

Figure 7 shows a photo micrograph of a prototype 8-tap lock-in-pixel TOF imager chip implemented using a 1Poly 4Metal 0.11-μm CMOS image sensor process. The pixel array had a total of 134(H) × 128(V) pixels, including test pixels. An inverter tree and a driver for pixel gating were arranged above and below the pixel array for faster gate responses. The pixel outputs were read out by using a 19-bit column ADC [22]. The converted digital signal was horizontally read out using low-voltage differential signaling (LVDS) circuits. Table 1 shows a brief summary of the prototype TOF imager specification and basic characteristics. The chip was assembled to the cooling package, had Peltier elements, and was cooled down to roughly −15 °C to suppress the influence of the dark current in the measurements. The relatively large dark current in the present design was related to the structure around the CA gate, which must be improved for further low-noise performance of the chip. The readout time of 51.84 ms was very long for the imager given the relatively small number of pixels. One reason for this long readout time was the use of a multiple-sampling based column ADC [22]. The implemented chip had a non-linearity problem if the operation clock of the ADC was increased. To reduce the non-linearity, the following measurement data were taken with a slower clock for the ADC than that of the designed specification. The readout speed could be improved by re-designing the column ADC of the chip.

4.2. Charge Modulation Characteristics

Figure 8 shows a measurement result of the response of seven signal-tap outputs to the light pulse delay. A laser pulse with a pulse width of 6 ns, a wavelength of 850 nm, a power of 29 mW/m2 (0.88 W/m2 peak) at 6 m and a laser pulse rate of 5.5 MHz was used. A lens with a large focal length of 12.5 mm was used For the sensor characterization, the sensor field-of-view (FOV) used for the measurement was 13.1°(V) × 13.7°(H). In this measurement, the modulation gates, G1 through G7, were opened as shown in the timing of Figure 2b with T0 = 6 ns, and the delay of the laser pulse was scanned from 0 to 60 ns with a step of 0.4 ns. The gate G8 was used as a charge draining gate and the S D 8 was working as a drain. Figure 8a shows measured data after subtracting the dark signal from the raw data and Figure 8b shows a normalized result with a sensitivity correction. The average demodulation contrast in the charge signals stored in SDs, for q 1 through q 7 , was 0.83. As shown in Figure 8, the response of the seven signals to the pulse delay time was fast enough for the short-pulse TOF measurement with the light pulse width of 6 ns when compared with the ideal response (triangular shape), where it was rising linearly over 6 ns and falling linearly over 6 ns. Using the normalized response of Figure 8b, the differences of the two signal-tap outputs ( d 1 , d 2 , d 3 , d 4 ,   d 5 ) were calculated and are shown in Figure 9a. The five piecewise-linear response curves, two of which overlap each other, were obtained and these were used for the TOF range measurements by switching these five curves properly, as described in Section 2.3. Figure 9b shows a curve for the calculated (measured) distance, i.e., the pseudo distance, which corresponded to twice the TOF. This linearity plot was obtained using a calibration applying the gain and offset corrections to the five piecewise-linear response curves. After this non-linearity correction, the maximum of the measured distance error was 3.7 cm and the maximum non-linearity error was 0.67%FS in the range of 0 m to 5.4 m.

4.3. Distance Measurements

Using a similar calculation method for the TOF described in Section 4.2, distance measurements were carried out with two types of gating methods, as shown in Figure 2b (ordinary time-gating method) and Figure 3 (depth-adaptive time-gating-number assignment (DATA) method). The emitted laser pulse width and pixel-gating pulse width were 6 ns. The measured object was a white panel with the reflection factor of 99% and a fully diffusing surface. The white panel was moved with a 0.1 m step from 1.0 m to 6.4 m from the TOF camera. The range linearity and depth resolution were measured using 100 frames. In the ordinary gating method, the accumulation time in all SDs was set to 0.217 ms. This setting of a short accumulation time was necessary to avoid the saturation in the SDs that received a strong light power due to the reflections with very close objects. The SD1 was the most critical and the SD2 the second most critical. Figure 10 shows the distance measurement results of the non-linearity error (Figure 10a) and depth resolution (Figure 10b).
The pixel area of 11(H) × 11(V) is used for the measurements. The threshold T Z for calculating the distance used in Equation (6) was set to 0.5. The maximum measured distance error was 6.2 cm and the maximum non-linearity error was 1.15%FS for the full range of 1–6.4 m. For the case of excluding the near end of the measurement range, the measured distance error was less than 5.4 cm, which means the non-linearity error was less than 1%FS. The depth resolution in this method was less than 20 mm up to 2.6 m, and it increased to 128 mm at the maximum distance. The degradation of the depth resolution with increasing distance was due to the attenuated reflected light and the limited FWC of the pixel.
Figure 11 shows the distance measurement results using the DATA method shown in Figure 3. In this method, the accumulation times in S D 1 through S D 7 were 0.169 ms, 0.384 ms, 1.008 ms, 1.794 ms, 2.187 ms, 3.760 ms, and 4.153 ms, respectively. The pixel area of 11(H) × 11(V) was used for the measurements. The maximum of the measured distance error was 8.4 cm and the maximum non-linearity error was 1.56%FS for the full range of 1–6.4 m. For the measurement ranges larger than 1.5 m, the measured distance error was less than 5.4 cm, which means the non-linearity error was less than 1%FS, as shown in Figure 11a. Figure 11b shows the depth resolution as a function of the distance.
Using the DATA method, where the optimum accumulation time was assigned depending on the reflected light intensity, a good depth resolution smaller than 13 mm was obtained for the entire measurement range and the resolution of 6.4 mm was obtained at 6.2 m. The depth resolution corresponded to 0.10% of the depth.
Figure 12 shows a depth image with the implemented TOF image sensors using the operation of the depth-adaptive time-gating-number assignment technique. This depth image with only the subset (80(H) × 81(V) pixels) of the entire pixel array (134(H) × 128(V)) is shown because of the relatively small irradiating angle of the laser light used. In the scene, a white box was placed at 2.1 m and 4.7 m, a manikin was placed at 3.1 m, and the white screen was placed at 6.0 m as a background. The threshold T Z for calculating the distance used in Equation (6) was set to 0.3. This image was generated by using one frame of data.

5. Conclusions

This paper presents an 8-tap CMOS lock-in pixel image sensor for short-pulse TOF measurements. The proposed operation of the 8-tap lock-in pixel image sensor using seven taps for signal outputs and a tap for draining was suitable for high-resolution, single-frame TOF measurements using small-duty short light pulses. The implemented prototype had a good response to a short light pulses, where a pulse width of 6 ns was used for TOF measurements. The pulse-based TOF measurements with multiple-tap pixels allowed us to use a flexible time-gating pattern setting. The proposed technique with depth adaptive time-gating-number assignment was effective for realizing a wide measurement range and a high depth resolution. Using this technique and the 8-tap pixel image sensor, the high depth resolution of 6.4 mm at 6.2 m (0.10% of depth) and better than 13 mm within whole depth range of 1 m to 6.4 m were obtained. Because of the segmented TOF measurements with seven time-gating windows, the linearity was also relatively good. For the case of pseudo distance measurements using a delay generator for pseudo TOF generation, a maximum non-linearity error of 0.67%FS was demonstrated for the distance range from 0 m to 5.4 m. Distance calculation algorithms consecutively using the 7-tap outputs were developed and depth images covering 1 m to 6.4 m were successfully obtained.

Author Contributions

S.K. proposed the device concept and provided the overall guidance of the project. Y.S., S.K., and K.Y. designed the detector, overall circuit, and layout for the chip. S.A. designed the ADC circuit. Y.S. also measured the chip. Y.S., S.K., K.Y., and K.K. discussed the measurement results. Y.S. and S.K. drafted the manuscript. All authors have read and agreed to the published version of the manuscript.

Funding

This work was supported in part by the Japan Society for the Promotion of Science (JSPC) KAKENHI, the Grant-in-Aid for Scientific Research (S) under Grant 18H05240 through the Ministry of Education, Culture, Sports, Science and Technology (MEXT), the MEXT/JST COI-STREAM Program, and the MEXT Regional-Innovation Ecosystem Program.

Acknowledgments

VLSI Design and Education Center (VDEC), The University of Tokyo with the collaboration with Cadence Corporation, and the authors appreciate Dongbu HiTek for the CIS fabrication.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Spirig, T.; Seitz, P.; Vietze, O.; Heitger, F. The Lock-In CCD-Two-Dimensional Synchronous Detection of Light. IEEE J. Quantum Electron. 1995, 31, 1705–1708. [Google Scholar] [CrossRef]
  2. Spirig, T.; Marley, M.; Seitz, P. The multitap lock-In CCD with offset subtraction. IEEE Trans. Electron Devices 1997, 44, 1643–1647. [Google Scholar] [CrossRef]
  3. Lange, R.; Seitz, P. Solid-state time-of-flight range camera. IEEE J. Quantum Electron. 2001, 37, 390–397. [Google Scholar] [CrossRef] [Green Version]
  4. Bamji, C.S.; O’Connor, P.; Elkhatib, T.; Mehta, S.; Thompson, B.; Prather, L.A.; Snow, D.; Akkaya, O.C.; Daniel, A.; Payne, A.D.; et al. A 0.13 μm CMOS System-on-Chip for a 512 × 424 Time-of-Flight Image Sensor with Multi-Frequency Photo-Demodulation up to130 MHz and 2GS/s ADC. IEEE J. Solid-State Circuits 2015, 50, 303–319. [Google Scholar] [CrossRef]
  5. Stoppa, D.; Massari, N.; Pancheri, L.; Malfatti, M.; Perenzoni, M.; Gonzo, L. A Range Image Sensor Based on 10-μm Lock-In Pixels in 0.18-μm CMOS Imaging Technology. IEEE J. Solid-State Circuits 2011, 46, 248–258. [Google Scholar] [CrossRef]
  6. Kim, S.-J.; Kim, J.D.K.; Kang, B.; Lee, K. A CMOS Image Sensor Based on Unified Pixel Architecture with Time-Division Multiplexing Scheme for Color and Depth Image Acquisition. IEEE J. Solid-State Circuits 2011, 46, 248–258. [Google Scholar] [CrossRef]
  7. Bamji, C.S.; Mehta, S.; Thompson, B.; Elkhatib, T.; Wurster, S.; Akkaya, O.; Payne, A.; Godbaz, J.; Fenton, M.; Rajasekaran, V.; et al. IMpixel 65nm BSI 320MHz demodulated TOF image sensor with 3um global shutter pixels and analog binning. In Proceedings of the 2018 IEEE International Solid—State Circuits Conference—(ISSCC), San Francisco, CA, USA, 11–15 February 2018; pp. 94–96. [Google Scholar]
  8. Kato, Y.; Sano, T.; Moriyama, Y.; Maeda, S.; Yamazaki, T.; Nose, A.; Shina, K.; Yasu, Y.; Tempel, W.; Ercan, A.; et al. 320 × 240 back-illuminated 10um CAPD pixels for high speed modulation Time-of-Flight CMOS image sensor. In Proceedings of the 2017 Symposium on VLSI Circuits, Kyoto, Japan, 5–8 June 2017; pp. 288–289. [Google Scholar]
  9. Keel, M.; Jin, Y.; Kim, Y.; Kim, D.; Kim, Y.; Bae, M.; Chung, B.; Son, S.; Kim, H.; An, T.; et al. A 640 × 480 indirect time-of-flight CMOS image sensor with 4-tap 7μm global-shutter pixel and fixed-pattern phase noise self-compensation scheme. In Proceedings of the 2019 Symposium on VLSI Circuits, Kyoto, Japan, 9–14 June 2019; pp. 258–259. [Google Scholar]
  10. Kawahito, S.; Halin, I.A.; Ushinaga, T.; Sawada, T.; Homma, M.; Maeda, Y. A CMOS Time-of-Flight Range Image Sensor with Gates-on-Field-Oxide Structure. IEEE Sens. J. 2007, 7, 1578–1586. [Google Scholar] [CrossRef]
  11. Sawada, T.; Kawahito, S.; Nakayama, M.; Ito, K.; Halin, I.A.; Homma, M.; Ushinaga, T.; Maeda, Y. A TOF range image sensor with an ambient light charge drain and small duty-cycle light pulse. In Proceedings of the 2007 International Image Sensor Workshop, Ogunquit, ME, USA, 6–10 June 2007; pp. 254–257. [Google Scholar]
  12. Spickermann, A.; Durini, D.; Suss, A.; Ulfig, W.; Brockherde, W.; Hosticka, B.J.; Schwope, S.; Grabmaier, A. CMOS 3D image sensor based on pulse modulated time-of-flight principle and intrinsic lateral drift-field photodiode pixel. In Proceedings of the 2011 Proceedings of the ESSCIRC (ESSCIRC), Helsinki, Finland, 12–16 September 2011; pp. 111–114. [Google Scholar]
  13. Han, S.M.; Takasawa, T.; Yasutomi, K.; Aoyama, S.; Kagawa, K.; Kawahito, S. A time-of-flight range image sensor with background cancelling lock-in pixel based on lateral electric field charge modulation. IEEE J. Electron Devices Soc. 2015, 3, 267–275. [Google Scholar] [CrossRef]
  14. Lee, S.; Yasutomi, K.; Morita, M.; Kawanishi, H.; Kawahito, S. A Time-of-Flight Range Sensor Using Four-Tap Lock-In Pixels with High near Infrared Sensitivity for LiDAR Application. Sensors 2020, 20, 116. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  15. Kondo, K.; Yasutomi, K.; Yamada, K.; Komazawa, A.; Handa, Y.; Okura, Y.; Michiba, T.; Aoyama, S.; Kawahito, S. A Built-in Drift-field PD Based 4-tap Lock-in Pixel for Time-of-Flight CMOS Range Image Sensors. In Proceedings of the Extended Abstracts of the 2018 International Conference on Solid State Devices and Materials, Tokyo, Japan, 9–13 September 2018; pp. 601–602. [Google Scholar]
  16. Sawada, T.; Ito, K.; Nakayama, M.; Kawahito, S. A range-shift technique for TOF range image sensor. IEEJ Trans. Sens. Micromachines 2009, 129, 421–425. [Google Scholar] [CrossRef]
  17. Shirakawa, Y.; Seo, M.W.; Yasutomi, K.; Kagawa, K.; Teranishi, N.; Kawahito, S. Design of an 8-tap CMOS lock-in pixel with lateral electric field charge modulator for highly time-resolved imaging. In Silicon Photonics XII; SPIE: San Francisco, CA, USA, 2017. [Google Scholar]
  18. Shirakawa, Y.; Yasutomi, K.; Kagawa, K.; Aoyama, S.; Kawahito, S. An 8-tap CMOS Lock-in Pixel Image Sensor for Short-Pulse Time-of-Flight Measurements. In Proceedings of the International Image Sensor Workshop (IISW2019), Snowbird, UT, USA, 23–27 June 2019; pp. 187–190. [Google Scholar]
  19. Kawahito, S.; Baek, G.; Li, Z.; Han, S.M.; Seo, M.W.; Yasutomi, K.; Kagawa, K. CMOS lock-in pixel image sensors with lateral electric field control for time-resolved imaging. In Proceedings of the International Image Sensor Workshop (IISW), Snowbird, UT, USA, 12–16 June 2013; pp. 361–364. [Google Scholar]
  20. Seo, M.W.; Kagawa, K.; Yasutomi, K.; Kawata, Y.; Teranishi, N.; Li, Z.; Halin, I.A.; Kawahito, S. A 10 ps Time-resolution CMOS image sensor with two-tap true-CDS lock-in pixels for fluorescence lifetime imaging. IEEE J. Solid-State Circuits 2016, 51, 141–154. [Google Scholar]
  21. Seo, M.W.; Shirakawa, Y.; Kawata, Y.; Kagawa, K.; Yasutomi, K.; Kawahito, S. A time-resolved four-tap lock-in pixel CMOS image sensor for real-time fluorescence lifetime imaging microscopy. IEEE J. Solid-State Circuits 2018, 53, 2319–2330. [Google Scholar] [CrossRef]
  22. Seo, M.W.; Suh, S.H.; Iida, T.; Takasawa, T.; Isobe, K.; Watanabe, T.; Itoh, S.; Yasutomi, K.; Kawahito, S. A low-noise high intrascene dynamic range CMOS image sensor with a 13 to 19b variable-resolution column-parallel folding-integration/cyclic ADC. IEEE J. Solid-State Circuits 2012, 47, 272–283. [Google Scholar] [CrossRef]
Figure 1. Conceptual schematic of multiple-tap lock-in pixels: (a) N-tap lock-in pixel with a drain, (b) symbol for a depleted photodiode, and (c) symbol for a modulation gate.
Figure 1. Conceptual schematic of multiple-tap lock-in pixels: (a) N-tap lock-in pixel with a drain, (b) symbol for a depleted photodiode, and (c) symbol for a modulation gate.
Sensors 20 01040 g001
Figure 2. Short-pulse time-of-flight (SP-TOF) measurement using a multiple-tap lock-in pixel imager: (a) 3-tap with one window and (b) 7-tap with six windows.
Figure 2. Short-pulse time-of-flight (SP-TOF) measurement using a multiple-tap lock-in pixel imager: (a) 3-tap with one window and (b) 7-tap with six windows.
Sensors 20 01040 g002
Figure 3. Depth-adaptive time-gating-number assignment for efficient SP-TOF measurement.
Figure 3. Depth-adaptive time-gating-number assignment for efficient SP-TOF measurement.
Sensors 20 01040 g003
Figure 4. TOF operation of a 7-tap pixel with six time-gating windows: (a) principle of a short-pulse TOF measurement system and (b) diagram for the distance calculation algorithm.
Figure 4. TOF operation of a 7-tap pixel with six time-gating windows: (a) principle of a short-pulse TOF measurement system and (b) diagram for the distance calculation algorithm.
Sensors 20 01040 g004aSensors 20 01040 g004b
Figure 5. 3D device simulation results: (a) structure of the 8-tap lock-in pixel, (b) charge transportation result, (c) potential diagram of the storage diode (SD) accumulation mode (X–X’), and (d) potential diagram for the charge transfer mode from the SD to floating diffusion (FD) (X–X’).
Figure 5. 3D device simulation results: (a) structure of the 8-tap lock-in pixel, (b) charge transportation result, (c) potential diagram of the storage diode (SD) accumulation mode (X–X’), and (d) potential diagram for the charge transfer mode from the SD to floating diffusion (FD) (X–X’).
Sensors 20 01040 g005
Figure 6. Pixel circuits and readout operation: (a) equivalent circuit and (b) timing diagram for accumulating and reading photo signals.
Figure 6. Pixel circuits and readout operation: (a) equivalent circuit and (b) timing diagram for accumulating and reading photo signals.
Sensors 20 01040 g006
Figure 7. Chip micrograph. CDS: correlated double sampling, FI/C-ADC: Folding-Integration/Cyclic ADC, LEFM: lateral electric field charge modulation.
Figure 7. Chip micrograph. CDS: correlated double sampling, FI/C-ADC: Folding-Integration/Cyclic ADC, LEFM: lateral electric field charge modulation.
Sensors 20 01040 g007
Figure 8. Response of the seven signal-tap outputs to the light pulse delay: (a) measured raw data and (b) the normalized data.
Figure 8. Response of the seven signal-tap outputs to the light pulse delay: (a) measured raw data and (b) the normalized data.
Sensors 20 01040 g008
Figure 9. Differential responses and calculated distances using the normalized response curves: (a) differences of two signal-tap outputs and (b) calculated distance with seven signal-tap outputs versus the pseudo distance.
Figure 9. Differential responses and calculated distances using the normalized response curves: (a) differences of two signal-tap outputs and (b) calculated distance with seven signal-tap outputs versus the pseudo distance.
Sensors 20 01040 g009
Figure 10. Measured distance versus real distance with modulation timing being the same accumulation time: (a) non-linearity error and measured distance, and (b) depth resolution and measured distance.
Figure 10. Measured distance versus real distance with modulation timing being the same accumulation time: (a) non-linearity error and measured distance, and (b) depth resolution and measured distance.
Sensors 20 01040 g010
Figure 11. Measured distance versus real distance with an SP-TOF-specific timing: (a) non-linearity error and measured distance, and (b) depth resolution and measured distance.
Figure 11. Measured distance versus real distance with an SP-TOF-specific timing: (a) non-linearity error and measured distance, and (b) depth resolution and measured distance.
Sensors 20 01040 g011
Figure 12. Depth image.
Figure 12. Depth image.
Sensors 20 01040 g012
Table 1. Brief summary of the prototype CMOS imager performance. CIS: CMOS image sensor.
Table 1. Brief summary of the prototype CMOS imager performance. CIS: CMOS image sensor.
ParameterValue
Technology0.11-μm 1P4M CIS process
# of pixels134 (H) × 128(V)
Pixel size22.4 μm × 22.4 μm
Chip size5874 μm × 9342 μm
ADC resolution19 bit
Readout time51.84 ms
Conversion gain69.5 μ V / e
Full well capacity5600 e

Share and Cite

MDPI and ACS Style

Shirakawa, Y.; Yasutomi, K.; Kagawa, K.; Aoyama, S.; Kawahito, S. An 8-Tap CMOS Lock-In Pixel Image Sensor for Short-Pulse Time-of-Flight Measurements. Sensors 2020, 20, 1040. https://doi.org/10.3390/s20041040

AMA Style

Shirakawa Y, Yasutomi K, Kagawa K, Aoyama S, Kawahito S. An 8-Tap CMOS Lock-In Pixel Image Sensor for Short-Pulse Time-of-Flight Measurements. Sensors. 2020; 20(4):1040. https://doi.org/10.3390/s20041040

Chicago/Turabian Style

Shirakawa, Yuya, Keita Yasutomi, Keiichiro Kagawa, Satoshi Aoyama, and Shoji Kawahito. 2020. "An 8-Tap CMOS Lock-In Pixel Image Sensor for Short-Pulse Time-of-Flight Measurements" Sensors 20, no. 4: 1040. https://doi.org/10.3390/s20041040

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop