Design of a CMOS Image Sensor with Bi-Directional Gamma-Corrected Digital-Correlated Double Sampling

We present a 640 × 480 CMOS image sensor (CIS) with in-circuit bi-directional gamma correction with a proposed digital-correlated double sampling (CDS) structure. To operate the gamma correction in the CIS, the transfer function of the analog-to-digital converter can be changed by controlling the clock frequency of the counter using analog CDS. However, the analog CDS is vulnerable to capacitor mismatch, clock feedthrough, etc. Therefore, we propose a digital-CDS method with a hold-and-go counter structure to operate the bi-directional gamma correction in the CIS. The proposed CIS achieves a 10-bit resolution using a global log-exponential counter and configurable column reset counter with a resolution of 8/9 bits. The sensor was fabricated in a 0.11 μm CIS process, and the full chip area was 5.9 mm × 5.24 mm. The measurement results showed a maximum SNR improvement of 10.41% with the proposed bi-directional gamma-corrected digital-CDS with the hold-and-go counter. The total power consumption was 6.3 mW at a rate of 16.6 frames per second with analog, pixel, and digital supply voltages of 3.3 V, 3.3 V, and 1.5 V, respectively.


Introduction
Wide-dynamic-range (WDR) CMOS image sensors (CISs) have been developed as the demand for surveillance cameras, automobiles, and medical cameras has increased. Several techniques to obtain a wider dynamic range image at the circuit-algorithm level have been proposed [1,2]. Additionally, several techniques to achieve WDR without changing the pixel structure have been presented [3][4][5]. By adjusting the analog/digital gains, in-circuit gamma correction techniques can enhance image quality [6][7][8]. In general, gamma correction is performed in the image signal processor (ISP) to compensate for the limited DR of displays. Instead of memory and the complex digital circuits required for the gamma correction in ISP, the visibility of images can be improved by increasing the gain in low-light areas by adjusting the slope of the ramp generator and the frequency of the digital counter in CIS. Applying conventional gamma curvature using a digital counter has been suggested [7,8]. However, conventional gamma correction can lead to data loss after correction if the original image contains a highly illuminated, saturated area. To solve this problem, bi-directional gamma correction, which can correct both dark and bright areas simultaneously, was implemented with a logarithmic-exponential counter (LEC) and analog-correlated double sampling (A-CDS) [9]. However, the A-CDS technique can be affected by noise from capacitance mismatch caused by process variation or clock feedthrough from the switch as the signal toggles. Additionally, as higher resolution is in demand these days, the noise sources from A-CDS can affect overall image quality. Thus, we considered the implementation of digital CDS (D-CDS), which is less affected by noise from process variation and signal transition. D-CDS can be achieved using a conventional up-down (UD) counter. However, the D-CDS implemented with a conventional UD counter operates through the linear in-output response, as shown in Figure 1a. Reset voltages, including an offset voltage, are each sampled after 2 t and 5 t. After 2 t and 5 t from the moment signal voltage sampling starts, the code becomes '0', in which CDS operation is performed. However, bi-directional gamma curvature applies different gains at different luminance areas. Thus, it has a nonlinear in-output response. If bi-directional gamma correction is applied using a UD counter, the power consumption and area considerably increase, and a CDS operation error occurs. When the signal voltages are the same while the reset voltage varies, as depicted in Figure 1b, a nonlinear response causes an error that results in different timing of outputting the 0 code (t, 6 t). To remove the error generated while implementing the nonlinear response in the UD counter, reset and signal sampling should be performed separately within the column, so the counter area per column is twice as large as the linear response in the UD counter. an error that results in different timing of outputting the 0 code (t, 6 t). To remove the error generated while implementing the nonlinear response in the UD counter, reset and signal sampling should be performed separately within the column, so the counter area per column is twice as large as the linear response in the UD counter.
Thus, the hold-and-go (HG) technique [10,11], which performs the D-CDS operation using both global and column counters, is used to achieve bi-directional gamma correction, as shown in Figure 1c. In the case of the HG counter, the column counter operates only when the reset voltage is sampled, and the global counter operates when the signal voltage is sampled. Therefore, in the case of a 10-bit ADC, only an 8-bit or 9-bit column counter (8/9-bit column counter in this paper) for reset sampling is required. For the column-parallel ADC in CIS, a small area of the column circuit is required to obtain the small total area. When using a 9-bit reset counter, the column counter area is reduced by about 10%. On the other hand, since a 20-bit column counter is required to obtain a 10-bit nonlinear response with the UD counter, the column counter area can be reduced by 55% using only a 9-bit column counter with the proposed HG D-CDS, as shown in Figure 1c. Therefore, by implementing D-CDS using the HG method, the image sensor's overall power consumption and area are less than those of conventional CIS with the UD counter.

Proposed CIS Structure
The proposed CIS has a VGA resolution of 640 × 480 pixels and a 10-bit column-parallel single-slope analog-to-digital converter (SS-ADC) structure using the hold-and-go (HG) technique, as depicted in Figure 2. The prototype sensor consists of a pixel array, a 9-bit column counter for digital-correlated double sampling (D-CDS), a logarithmic-exponential counter (LEC) for bi-directional gamma correction as a global counter, and other digital periphery blocks. Additionally, the analog circuits of each column consist of the comparator, a 10-bit 2-stage static random-access memory (SRAM), and a column decoder for H-scanning. Two-stage SRAM enables outputting values at the second stage while Thus, the hold-and-go (HG) technique [10,11], which performs the D-CDS operation using both global and column counters, is used to achieve bi-directional gamma correction, as shown in Figure 1c. In the case of the HG counter, the column counter operates only when the reset voltage is sampled, and the global counter operates when the signal voltage is sampled. Therefore, in the case of a 10-bit ADC, only an 8-bit or 9-bit column counter (8/9-bit column counter in this paper) for reset sampling is required. For the columnparallel ADC in CIS, a small area of the column circuit is required to obtain the small total area. When using a 9-bit reset counter, the column counter area is reduced by about 10%. On the other hand, since a 20-bit column counter is required to obtain a 10-bit non-linear response with the UD counter, the column counter area can be reduced by 55% using only a 9-bit column counter with the proposed HG D-CDS, as shown in Figure 1c. Therefore, by implementing D-CDS using the HG method, the image sensor's overall power consumption and area are less than those of conventional CIS with the UD counter.

Proposed CIS Structure
The proposed CIS has a VGA resolution of 640 × 480 pixels and a 10-bit columnparallel single-slope analog-to-digital converter (SS-ADC) structure using the hold-and-go (HG) technique, as depicted in Figure 2. The prototype sensor consists of a pixel array, a 9-bit column counter for digital-correlated double sampling (D-CDS), a logarithmic-exponential counter (LEC) for bi-directional gamma correction as a global counter, and other digital periphery blocks. Additionally, the analog circuits of each column consist of the comparator, a 10-bit 2-stage static random-access memory (SRAM), and a column decoder for H-scanning. Two-stage SRAM enables outputting values at the second stage while getting simultaneous input into the first stage. The LEC has five operation modes as follows: linear mode, gamma mode, 1

Correlated Double Sampling (CDS)
In CIS, the Pixel Array consists of numerous pixels. These pixels have different characteristics for each pixel due to errors in the process. Even if the same light is irradiated to all pixels due to different threshold voltages of transistors in each pixel, each pixel output has different offset voltages. These different voltages result in fixed-pattern noise (FPN) that can be removed by CDS. The pixel outputs two voltages: a reset voltage and a signal voltage caused according to the intensity of light, and the difference between the two voltages is the input voltage range of the SS-ADC (=0.8 V in this paper). Since these two voltages come through the same pixel, they have a constant offset error. The CDS technique is used to remove the offset error by sampling both the pixel output during the reset process and pixel output by light and finding the difference between the two signals. The types of CDS techniques are divided into Analog-CDS using a capacitor and circuit feedback and global counter and Digital-CDS subtracting the digital code of the reset voltage and the digital code of the signal by light using a separate memory or a counter inside the column. Figure 3a shows the block diagram of A-CDS that performs CDS using feedback using a switch (sw2) and capacitor (C1). According to the law of conservation of charge, the difference between the reset voltage and signal voltage can be obtained. Such A-CDS can be operated relatively simply, but has a disadvantage in that accuracy is lower than that of D-CDS due to mismatch in the capacitor process, kT/C noise, errors caused by clock feedthrough, and charge injection of switches. Figure 3b,c show the structure and operation principle of D-CDS and has a counter in each column. As shown in Figure 3c , first the reset voltage is sampled and converted, and then the value is subtracted from the code obtained by converting the signal voltage to obtain the difference between the two signals with the offset error removed. As shown in Figure 3c, codedelta is the value obtained by subtracting codereset from codesig, confirming that Voffset is removed.

Correlated Double Sampling (CDS)
In CIS, the Pixel Array consists of numerous pixels. These pixels have different characteristics for each pixel due to errors in the process. Even if the same light is irradiated to all pixels due to different threshold voltages of transistors in each pixel, each pixel output has different offset voltages. These different voltages result in fixed-pattern noise (FPN) that can be removed by CDS. The pixel outputs two voltages: a reset voltage and a signal voltage caused according to the intensity of light, and the difference between the two voltages is the input voltage range of the SS-ADC (=0.8 V in this paper). Since these two voltages come through the same pixel, they have a constant offset error. The CDS technique is used to remove the offset error by sampling both the pixel output during the reset process and pixel output by light and finding the difference between the two signals. The types of CDS techniques are divided into Analog-CDS using a capacitor and circuit feedback and global counter and Digital-CDS subtracting the digital code of the reset voltage and the digital code of the signal by light using a separate memory or a counter inside the column. Figure 3a shows the block diagram of A-CDS that performs CDS using feedback using a switch (sw2) and capacitor (C1). According to the law of conservation of charge, the difference between the reset voltage and signal voltage can be obtained. Such A-CDS can be operated relatively simply, but has a disadvantage in that accuracy is lower than that of D-CDS due to mismatch in the capacitor process, kT/C noise, errors caused by clock feedthrough, and charge injection of switches. Figure 3b,c show the structure and operation principle of D-CDS and has a counter in each column. As shown in Figure 3c, first, the reset voltage is sampled and converted, and then the value is subtracted from the code obtained by converting the signal voltage to obtain the difference between the two signals with the offset error removed. As shown in Figure 3c, code delta is the value obtained by subtracting code reset from code sig , confirming that Voffset is removed.  Figure 4a,b show the block diagram of D-CDS with the HG method in SS-ADC and the principle of the D-CDS operation using the HG method. The reset counter (RC) starts counting when the clock cycle for the 512 code (9 bit) starts. The RC holds its output when the comparator output is flipped from 'L' to 'H' by the reset voltage. After the 9-bit clock cycle, the 9-bit + 10-bit clock cycle starts for signal voltage sampling. When the comparator output turns into 'H' from 'L' by the signal voltage, the RC counts again from the held code until it reaches the maximum code. When the RC outputs the maximum code, MSB, which is the input of the sync block, turns into 'H'. At the sync block, a sync signal is generated when the RC reaches the maximum code 111111111(2). Additionally, for energy efficiency, a configurable structure that can achieve both an 8-bit and a 9-bit resolution RC was applied. Figure 4c shows a 1-column digital block layout. The total height is 840 µm, of which the height of HG D-CDS is 228 µm. The total height of 1-column SS-ADC is 998 µm.  Figure 4a,b show the block diagram of D-CDS with the HG method in SS-ADC an the principle of the D-CDS operation using the HG method. The reset counter (RC) start counting when the clock cycle for the 512 code (9 bit) starts. The RC holds its output whe the comparator output is flipped from 'L' to 'H' by the reset voltage. After the 9-bit cloc cycle, the 9-bit + 10-bit clock cycle starts for signal voltage sampling. When the comparato output turns into 'H' from 'L' by the signal voltage, the RC counts again from the hel code until it reaches the maximum code. When the RC outputs the maximum code, MSB which is the input of the sync block, turns into 'H'. At the sync block, a sync signal i generated when the RC reaches the maximum code 111111111(2). Additionally, for energ  Figure 5a,b shows the block diagram and timing diagram of the global LEC consisting of an FLSB generator, control block, selector, and final counter. The FLSB generator consists of five Flip-Flops to generate F<0:2>, which determines the digital gain with three different frequencies. The control block generates TC<0:7>, which determines the intervals at which the digital gains are applied. The selector generates the Final LSB signal Do<0> by applying F<0:2> to each interval made by the control block. Each operation mode can be selected according to the 3-bit control signal CNTL [0:2]. The final counter generates a 10-bit final output from Do<0>. Figure 5c shows the bi-directional gamma curvature applied in the LEC. Each curvature illuminance area to be corrected is selectable.   Figure 5c shows the bi-directional gamma curvature applied in the LEC. Each curvature illuminance area to be corrected is selectable. ...

Experiment Results
The prototype of the proposed CIS was fabricated with the 0.11 μm CIS process. Figure 6a shows the chip microphotograph of the prototype CIS with an area of 5.9 mm × 5.24 mm in the full chip. To proceed with the images captured with the proposed CIS, the measurement environment is shown in Figure 6b, and the measurement sequence was as follows. An FPGA board, XEM3050 (Xilinx Spartan-3 FPGA Integration Module) was used

Experiment Results
The prototype of the proposed CIS was fabricated with the 0.11 µm CIS process. Figure 6a shows the chip microphotograph of the prototype CIS with an area of 5.9 mm × 5.24 mm in the full chip. To proceed with the images captured with the proposed CIS, the measurement Sensors 2023, 23, 1031 7 of 11 environment is shown in Figure 6b, and the measurement sequence was as follows. An FPGA board, XEM3050 (Xilinx Spartan-3 FPGA Integration Module) was used to check the control signal application and image output for driving the image sensor on a computer. Through the ISE program, the control signal was applied to the designated pin of the image sensor with Verilog, and the Verilog-A coding was completed to sequentially read the output of the sensor and display it on the computer's image viewer. Using the Opal Kelly board commercially available from Xilinx, the FPGA was driven based on the USB interface, and the measurement was conducted by checking the final image displayed in the image viewer. Figure 7 shows the captured images from the proposed CIS. The bottom-left side of Figure 7a captured in linear mode is dark and invisible. As depicted in Figure 7b, in the image corrected with the gamma curvature, the dark area becomes more visible, revealing the facial expression of the object, but the visibility of the upper-right side of the object is poor. The image with the proper bi-directional gamma curvature (Figure 7d) shows enhanced visibility in the low-illumination area, while the object is distinguishable from the background in the high-illumination area.

Modified-Absolute Mean Brightness Error
DR represents the ratio of the maximum output signal level to the noise floor level and depends on the characteristics of the pixels. Therefore, gamma correction techniques in ISP and circuits in CIS do not improve DR, and the measured DRs with different bidirectional gamma modes are identical [9]. However, as shown in Figure 7, the visibility of images can be enhanced with the bi-directional gamma correction technique. Since image quality assessment can be influenced by subjective opinion, an objective evaluation

Modified-Absolute Mean Brightness Error
DR represents the ratio of the maximum output signal level to the noise floor level and depends on the characteristics of the pixels. Therefore, gamma correction techniques in ISP and circuits in CIS do not improve DR, and the measured DRs with different bidirectional gamma modes are identical [9]. However, as shown in Figure 7, the visibility of images can be enhanced with the bi-directional gamma correction technique. Since image quality assessment can be influenced by subjective opinion, an objective evaluation concept was suggested. The absolute mean brightness error (AMBE) refers to the absolute

Modified-Absolute Mean Brightness Error
DR represents the ratio of the maximum output signal level to the noise floor level and depends on the characteristics of the pixels. Therefore, gamma correction techniques in ISP and circuits in CIS do not improve DR, and the measured DRs with different bi-directional gamma modes are identical [9]. However, as shown in Figure 7, the visibility of images can be enhanced with the bi-directional gamma correction technique. Since image quality assessment can be influenced by subjective opinion, an objective evaluation concept was suggested. The absolute mean brightness error (AMBE) refers to the absolute error value between the original image and the corrected image [12]. The AMBE is expressed in the equations below. When E[i] and A(i,j) are defined as Equation (1), the mean gray level E[X] of the image consisting of m × n array is defined as Equation (2).
Number of occurences of the intensity level i Number of intensity levels (1) The AMBE can now be defined as Equation (3).  Table 1 shows the measured signal-to-noise ratio (SNR) with different response modes of the prototype sensor. At the low illumination of the code in the digital number (DN) range between 0 to 20, the SNR of all bi-directional gamma modes is higher than that of the linear mode due to 2 × higher clock frequency that works like multisampling. As shown in Figure 9, the SNR improves at low illumination, but as the illumination increases, the SNRs of the linear mode counter output and the average of the LEC output are similar. It is presumed that this was because the SNR in bright illumination is dominated by shot noise and is not affected by variations of the digital gain.  Figure 8b,c, the Col-orChecker image captured at low illumination (20 lux) with a 3/4 point bi-directional gamma correction showed a maximum M-AMBE improvement of about 20% compared to the linear mode counter output. In the case of high illumination (550 lux), the Col-orChecker image captured with the ¼-point bi-directional gamma correction showed the best M-AMBE characteristics.

Signal-to-Noise Ratio
(a)   Table 1 shows the measured signal-to-noise ratio (SNR) with different response modes of the prototype sensor. At the low illumination of the code in the digital number (DN) range between 0 to 20, the SNR of all bi-directional gamma modes is higher than   Figure 9. Measured SNR graph of the prototype CIS.

Signal-to-Noise Ratio
The figure of merit (FoM) defined in [13] was used for comparison with other stateof-the-art sensors. The FoM defined in [13] was used for comparison.
(4) Table 2 shows the performance of the proposed CIS and other state-of-the-art technologies. The techniques [7][8][9] refer to the comparison. Except for the proposed CIS, the A-CDS method was used.  The figure of merit (FoM) defined in [13] was used for comparison with other state-ofthe-art sensors. The FoM defined in [13] was used for comparison.

FoM =
Power × Noise Frame rate × Total pixel number × 2 ADCbit (4) Table 2 shows the performance of the proposed CIS and other state-of-the-art technologies. The techniques [7][8][9] refer to the comparison. Except for the proposed CIS, the A-CDS method was used.

Conclusions
This paper illustrated the concept and structure of an SS-ADC with in-circuit bidirectional gamma correction achieving D-CDS with a hold-and-go counter. The captured images demonstrated that different bi-directional gamma curvatures can be applied by selecting the digital control signal. The measurement results of the prototype sensor demonstrated that the image visibility and the SNR were improved, especially in lowillumination areas. We expect the sensor to have various WDR applications such as computer vision, object detection, etc.
Author Contributions: J.C., H.C. and S.L. conceived and designed the circuits. S.Y., G.K. and S.K. performed the experiments and analyzed the data. All authors have read and agreed to the published version of the manuscript. Data Availability Statement: The datasets generated from the current study are available from the corresponding author upon reasonable request.