Analog Encoding Voltage—A Key to Ultra-Wide Dynamic Range and Low Power CMOS Image Sensor

Abstract

Usually Wide Dynamic Range (WDR) sensors that autonomously adjust their integration time to fit intra-scene illumination levels use a separate digital memory unit. This memory contains the data needed for the dynamic range. Motivated by the demands for low power and chip area reduction, we propose a different implementation of the aforementioned WDR algorithm by replacing the external digital memory with an analog in-pixel memory. This memory holds the effective integration time represented by analog encoding voltage (AEV). In addition, we present a “ranging” scheme of configuring the pixel integration time in which the effective integration time is configured at the first half of the frame. This enables a substantial simplification of the pixel control during the rest of the frame and thus allows for a significantly more remarkable DR extension. Furthermore, we present the implementation of “ranging” and AEV concepts on two different designs, which are targeted to reach five and eight decades of DR, respectively. We describe in detail the operation of both systems and provide the post-layout simulation results for the second solution. The simulations show that the second design reaches DR up to 170 dBs. We also provide a comparative analysis in terms of the number of operations per pixel required by our solution and by other widespread WDR algorithms. Based on the calculated results, we conclude that the proposed two designs, using “ranging” and AEV concepts, are attractive, since they obtain a wide dynamic range at high operation speed and low power consumption.

1. Introduction

Extending the dynamic range (DR) of a CMOS image sensor’s (CIS) remains one of the challenges to be faced in designing an effective versatile sensor. Improvement of image capture capability can be done either by reducing the noise floor (NF) of the sensor [1,2,3] or by extending its saturation toward higher light intensities. Most of the cases that are reported in the literature give solutions which focus on extending the DR toward high light intensities. There are numerous solutions that have been proposed throughout the years [4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27]. However, the majority of the proposed WDR solutions have used an external memory, which significantly increased their area and power consumption and, as a result, their cost.

Examples of using a digital memory can be found in a large variety of solutions [8] such as (a) multimode sensors that have a linear and a logarithmic response at dark and bright illumination levels, respectively [9]; (b) clipping sensors, in which a well capacity adjustment method is applied [10]; (c) frequency-based sensors, in which the sensor output is converted into a pulse frequency [12]; (d) time-to-first spike (TFS) sensors, in which the image is processed according to the time the pixel was detected as saturated [‎9]; (e) sensors with global control over the integration time, in which the pixel integrates for a set of exposures regardless of its input light intensity [15]; and (f) sensors with autonomous control over the integration time (multiple resets), in which each pixel has control over its own exposure period [16,17,18]. In the aforementioned designs the memories were used to store the possible pixel outputs or the DR extension data. The reason for using a memory is that the data processing becomes more straightforward, i.e., it is easy to split the processing into several independent stages and to store the intermediate results. However, in such a case, the overall processing time and power increase.

Nevertheless, we can find memory free WDR solutions as well. For example, in [19] a TFS algorithm implementation is described, where, instead of using a digital code to represent the saturation time, the authors paired each time slot with a certain voltage value that was subsequently converted into corresponding digital pixel value. This solution unfortunately suffers from the reduced sensitivity throughout the whole DR [8].

It is especially interesting to look into the free memory solutions in [20,21], which are based upon the global control over the integration time, since this category is the most widely spread in the industry. The memory can be omitted if there are two exposures as will be shown later on. However, using two exposures causes a substantial signal to noise ratio (SNR) drop in the mid-tones, which can necessitate subsequent image processing.

With autonomous control over the integration time category, the memory emerges as the main cause for the increase in both the complexity and the cost of the solution. Attempts to dump the memory in this category have been undertaken as well. In [22] a successful memory free solution was presented. That solution was based upon a very simple pixel structure, in which the pixel integration time value was encoded in voltage and stored on the intra-pixel capacitance. But, since the same capacitance was also used for readout of the intermediate pixel values, the stored integration times were continuously overwritten and refreshed over and over again slowing the sensor operation.

In this work, we present two novel concepts in “ranging” and analog encoding voltage (AEV) for design of low power, high speed, memory free WDR sensors with autonomous control over integration time. According to the “ranging” concept, the pixel signal is “coarsely” quantized at the beginning of the frame, and the final “fine” quantization takes place at the end of the frame. Generated data in the “coarse” quantization is stored as analog encoding voltage (AEV) inside each pixel on a separate capacitance, and this controls the pixel integration throughout the frame. Since we use different capacitances for a photo-induced signal and for an (AEV), the data processing becomes very simple and fluent. At the end of the frame, both of the two analog values, the photo-generated charge and the AEV, are merged through an analog to digital A/D conversion to a final digitized pixel value.

In addition, we present two designs which implement the “ranging” and AEV concepts. The designs are thoroughly described from the schematic to the layout. Moreover, we present the post-layout simulation results of the second design, showing successful DR extension up to 170 dBs. To illustrate the effectiveness of the proposed solutions, we perform a qualitative comparison with other published solutions and show that eventually our solutions are optimized for low power WDR imaging.

The presented work is organized as follows: Section 2 introduces our solutions, Section 3 presents the comparison of a number of operations in several types of WDR solutions, and Section 4 concludes the study. Following the last section, there are two Appendixes, namely A and B. They include the detailed considerations of pixel swing segmentation and signal to noise (SNR) considerations, respectively. In this way, the overall concept becomes clearer and no important design consideration is omitted.

2. WDR Solution Based upon Ranging and AEV

2.1. Problem Definition and Proposed Solution

The solution we have used for the DR extension is called the autonomous control over the integration time (multiple resets). None of the pixels in this solution are classic 3T’s, because this enables individual pixel reset via an additional transistor [8]. During the frame the outputs of a selected row are read nondestructively through the regular output chain. Then they are compared with an appropriate threshold at certain time points by comparators, which are found at a separate unit outside the pixel array. If a pixel value exceeds the threshold, a reset is given at that time point to that pixel. The binary information concerning the reset being applied or not is saved in a digital storage, in order to enable proper scaling of the value read. This enables the pixel value to be described as a floating-point representation, wherein the exponent will describe the scaling factor for the actual integration time, while the mantissa will be the regular A/D output. In this way the actual pixel value would be:

(1)

where V_pixel is the actual pixel value, M_pixel is the analog or digitized output value that has been read out at the end of the frame, X is a chosen constant, 2, for instance. EXP is the exponent value, which describes the scaling factor, i.e., the specific part of the integration time the pixel actually integrated without being saturated.

The bottleneck of the basic solution is that bits representing the EXP are generated during the integration sequentially; hence, after the generation of the subsequent bit, a digital memory should be refreshed. As the number of pixel rows increase, the time that can be allocated to EXP bits’ generation becomes more limited. The restrictions on the DR extension vary according to the mode of sensor operation: rolling or global shutter. This is more obvious in the rolling shutter, wherein the integration in the successive rows is skewed, so there is more time for DR processing. On the other hand, in the global shutter sensor, wherein the integration starts simultaneously in all rows, the processing times are substantially reduced. Therefore, the solution that we propose is targeted mainly for the global shutter sensor.

2.2. The Proposed Solution

Our solution is based upon “ranging” and AEV concepts. The purpose of this solution is to reduce DR processing times and to omit the memory unit. In addition, we have presented the solution architecture and the flow.

We have divided the WDR algorithm into three phases (Figure 1). During Phase 1, the EXP values are produced and memorized inside each pixel in a format of analog encoding voltage AEV. This assignment, which precedes the saturation checks, is called “ranging”, since it “coarsely” quantizes [23] the pixel signal. In fact, the AEV value produced in this quantization, contains the most significant bits of pixel signal Pix<M + L − 1:L> (Figure 2); whereas the Mantissa obtained at the end of the next phase contains the least significant bits Pix<L − 1:0>. In this way, each pixel is given its valid integration time for the current frame before the saturation checks start.

Figure 1. Three Phases of the proposed WDR algorithm.

Figure 1. Three Phases of the proposed WDR algorithm.

During Phase 2, the pixel integrates in accordance with its AEV. Since the integration time is already known, there is no need to retrieve the last reset information or to refresh information for the current check as was performed in our previous solutions; we merely compare the AEV to the global reference. In case the AEV is below the reference, the pixel will be reset; if not, the pixel will remain untouched till the next frame.

Figure 2. Digital Pixel Value: Mantissa and the EXP bits.

Figure 2. Digital Pixel Value: Mantissa and the EXP bits.

During Phase 3, both the AEV and Mantissa are digitized in a single slope analog to a digital conversion (ADC) (Figure 3). For this purpose, the AEV and Mantissa are read separately and converted with different resolutions. The AEV is digitized with a “coarse” resolution; whereas the Mantissa is converted by a ramp with a “fine” resolution.

Figure 3. The Coarse and Fine Quantization of the Pixel Signal.

Figure 3. The Coarse and Fine Quantization of the Pixel Signal.

In general, the three phases can overlap each other. However, in this study, we present the first version of the algorithm; therefore, we assume that each phase is completely separated from the others. In future works we will demonstrate how these phases can be overlaid on each other. We assessed the duration of each phase in accordance with further presented simulation results. In Figure 1, we present the division of a single frame (30 ms) into three phases as if the algorithm is applied on a 1000 × 1000 pixel array. In spite of all the phases being separate, it is important to note that the maximal integration time (Phase 2) still occupies the major part of the frame, whereas the overall integration times in the rest of the phases 1 and 3 are substantially shorter.

2.3. Phase 1: Assigning the Analog Encoding Voltage (AEV)

How are the values of AEV selected and paired with the corresponding EXP bits? The AEV values can be dispersed within the voltage domain in several ways: logarithmically or linearly (Figure 4). In this study we chose, for the sake of convenience, a linear quantization, according to which the all the AEV values are spaced at the same distance of 2ΔV_min. We also decided that the AEV corresponding to the highest light intensity would be represented by the lowest available voltage, since this simplifies all the subsequent conversions. The parameter ΔV_min is derived from the properties of the comparator that are to be employed for assigning the AEV values. The requirements for ΔV_min are intuitive and straightforward:

(2)

in which V_{out_comparator}, G, V_offset are the comparator’s output swing, gain, and offset, respectively.

Figure 4. Analog encoding voltage (AEV) concept: each dynamic range (DR) bit has its own Analog Encoding Voltage.

Figure 4. Analog encoding voltage (AEV) concept: each dynamic range (DR) bit has its own Analog Encoding Voltage.

After the pixel array has been reset globally, Phase 1 begins. The pixels are exposed to light and accumulate the photo-generated charge. At certain predetermined time points, each pixel row is sampled nondestructively and spanned by a piecewise increasing ramp. In case the pixel signal intersects with the ramp, the comparator flips and the corresponding AEV is written into the in-pixel dynamic memory. Detailed considerations of the AEV assignment are provided in Appendix A.

2.4. Phase 2 Performing the Conditional Resets

Phase 2 performs the conditional resets to the pixel array. Since the base of the DR extension is 2, the integration times are ordered in exactly the same way as in our previous solutions [24], i.e., in geometric descending sequence:

(3)

Nonetheless, in the current solution, the pixel reset cycle can be reduced by a factor of 2 at least, since the number of operations before the conditional reset is halved. All that has to be done is to compare the individual AEV with the reference, which corresponds to the current reset cycle. This phase can be performed either in a traditional fashion, i.e., row by row scan, or instantly within the whole array. We will discuss in further detail two different designs implementing both of the options, namely TYPE I and TYPE II.

2.5. Phase 3: Coarse, Fine ADC & Readout

Phase 3 concludes the frame. At this stage, the analog pixel value (Mantissa) and the AEV are converted by the same comparators, which had been used in the two preceding phases. In fact, we performed a single slope ADC to obtain the digitized values of Mantissa and AEV (Figure 2). The conversion is performed row by row: first, the AEV is spanned with the “coarse” resolution, and then the pixel swing is spanned by the ramp with “fine” resolution. Of course, after the span, both the AEV and pixel capacitances are reset to their initial predetermined values.

2.6. TYPE I Pixel

For most applications the required dynamic range (DR) does not exceed 120 dBs, so the pixel structure can be straightforward. In our case, all we need is to accommodate the AEV and the pixel Mantissa on two separate capacitors and to add a readout chain and conditional access circuitry (Figure 5) to each of them. Since the pixel structure here is not complex and the DR extension is typical, this design is named TYPE I.

Figure 5. Schematic of TYPE I Pixel.

Figure 5. Schematic of TYPE I Pixel.

The functionality of the TYPE I pixel is implemented with 11 transistors only. They can be divided into two groups: the first processes the data received from the pinned photodiode (PPD) M₁–M₆; and the second is responsible for the AEV handling M₇–M₁₁.

The charge transfer from the PPD is bidirectional, similar to [25]. In this way, we can deliberately dump the generated charge to drain the photo-diode through M₁ or we can transfer the charge for further processing to C_s through M₂. We use a simple conditional reset scheme, implemented by M₃ and M₄ transistors, to implement the multiple resets algorithm. By activating the Row_Reset signal, the required pixel row is chosen and then, by means of the C_Reset signal, driven by the column-wise comparators, is conditionally reset to V_rst. Transistors M₅, M₆ form a traditional source follower to handle the C_S readout [‎26].

AEV processing inside the pixel is very simple as well. The AEV value is fed to the pixel from the column-wise bus AEV_In. The write operation of the input AEV into the pixel dynamic memory is conditional and is implemented by using a trivial stacked scheme M₇ and M₈. By activating the Wr_AEV signal, a required row is accessed; while the Logic Decision signal, driven by column-wise comparators, enables the MOS capacitor C_AEV to sample the AEV_In bus. It is important to note that aside from its simplicity, this stacked scheme also minimizes the possible leakage from the AEV capacitor, thus maintaining the data integrity till the readout. The readout of the AEV is performed through a separate source follower, implemented by M₁₀ and M₁₁. This relatively simple pixel was implemented in layout using 0.18 µm CMOS technology, with 14 µm pitch and 40% fill factor (FF) (Figure 6).

Figure 6. Layout of TYPE I Pixel.

Figure 6. Layout of TYPE I Pixel.

Figure 7. Phase 1 TYPE I pixel.

Figure 7. Phase 1 TYPE I pixel.

Figure 7 depicts Phase 1 for the TYPE I pixel. At each span, the pixel (Pix value) is being accessed by election of a specific row and is being compared to the reference signal Ref.

After Phase 1 is completed, Phase 2 begins. During this phase, each pixel integrates in accordance to the assigned AEV. After a certain integration slot (3) elapses, the AEV’s within each row are selected and compared with the Ref, which now reflects the reference values of integration times (Figure 8).

Figure 8. Phase 2 TYPE I pixel.

Figure 8. Phase 2 TYPE I pixel.

In our solution, the pixel is reset as long as its AEV is lower than the reference value.

In a TYPE I solution, the conditional resets are applied to each pixel row sequentially. The processing time, which takes to apply the conditional reset a single pixel row T_{row_reset}, is given by:

(4)

where T_{AEV_read} and T_compare are the time to sample the AEV and to decide to reset or not the referred pixel, respectively. Since two saturation checks cannot overlap each other, the time it takes to complete a single one sets the minimal integration time T_{min_TYPE I}:

(5)

Given the minimal integration time, we can easily calculate the maximal possible DR extension factor (DRF) [8]. Through the performed simulations, we learned that the minimal integration time T_{min_TYPE I} is 112 µs. Assuming the maximal integration time is 30 ms, we have obtained a DR extension factor equal to 256 (48 dB). The expected intrinsic DR i.e., before the extension is 60 dBs, consequently the total DR equals 108 dBs.

In the last phase of the frame (Figure 9), the accumulated data are accessed for the final ADC. By activating two separate select signals Sel_Pix and Sel_AEV, the pixel and the AEV, respectively, are digitized using the same comparators as in Phase 1

The TYPE I design was successfully tested in various post-layout simulations, which proved its feasibility. To reduce the length of this work, we present only the post-layout simulation results of the TYPE II design because it is much more complex. To conclude, CMOS sensor based upon the presented TYPE I pixel is capable of providing a DR of almost 5 decades operating at video frame rate and providing an image with a decent resolution.

Figure 9. Phase 3 TYPE I pixel.

Figure 9. Phase 3 TYPE I pixel.

2.7. Ultra WDR (TYPE II) Pixel

The main difference between the Ultra WDR (TYPE II) pixel and the previous TYPE I solution lies in its accommodating a differential stage (D) within the pixel (Figure 10).

Figure 10. Schematic of TYPE II Pixel.

Figure 10. Schematic of TYPE II Pixel.

The inputs to this stage are: the pixel integration time represented by AEV (M₁₄); the photo-generated signal (M₈); and the global reference signal Ref (M₉). These inputs are sampled from C_AEV, C_S, and Ref, respectively. By means of Sel_AEV (M₁₅) or Sel_C_s (M₁₀), the differential stage compares the AEV or the photo-generated signal, to the Ref. the transistor M₁₁, controlled by Sel_Ref signal, was added to ensure an optimal matching within the differential amplifier. The bias point is controlled by an analog signal V_b (M₁₃), which sets the current magnitude throughout the amplifier branches. To facilitate the power reduction, we inserted an additional transistor M₁₂ in the series to the current source in order to shut down the entire stage, when no comparison is needed. The load of the D stage is implemented by two PMOS transistors: M₆ and M₇. The drain of the latter is connected to the drain of M₄, forming a self-reset structure.

The self-reset feature of the D amplifier enables the application of the conditional reset operation during a saturation check simultaneously to every pixel within the array rather than row by row. At each saturation check, the Row Reset (M₄) is raised globally, connecting the output of the D stage to the reset transistor M₅. Obviously, when the amplifier’s output is low, the C_s capacitance is reset to V_rst, otherwise C_s remains without a change. To stop the reset cycle, regardless of the differential stage output, we use End Reset signal (M₃), which forces AVDD on the gate of the reset transistor M_5.

AEV assignments are done row by row as will be explained further. Before the assignment operation, the first AEV is asserted onto the AEV_In bus. Then, the signal Wr_AEV (M₁₇) selects the appropriate pixels’ row. The Logic Decision signal (M₁₆), which is driven by the column-wise common source amplifiers, enables the corresponding AEV to be sampled by the C_AEV capacitor. The charge flow from the PPD is the same as in the TYPE I design throughout the whole frame.

At the end of the frame, both the AEV and the photo-generated charge are sampled nondestructively through the source follower (SF) (M₁₈ and M₁₉) and converted by column-wise amplifiers, as will be explained later.

Figure 11. Layout of the TYPE II Pixel.

Figure 11. Layout of the TYPE II Pixel.

The TYPE II pixel was implemented in a 0.18 CMOS process (Figure 11). All the control signals were divided into two groups: 7 analog signals, and 10 logic signals. To reduce the coupling between the control lines, we used extensive orthogonal patterning, so that part of the signals was routed along the X axis, whereas another part was laid along the Y axis. The analog signals were implemented in Metal1 and placed along the Y axis to the left and to the right of the PPD. Such an arrangement reduced the coupling between the analog lines themselves and created a substantial separation from the rest logic signals, which were laid along the X axis. Moreover, all the signals, which were running horizontally, were implemented in Metal 3, which reduced the coupling between the pixel analog and logic lines even further. Due to the dense layout, we successfully implemented the described pixel with 17 µm pitch and 25% fill factor FF (Figure 11).

The three phases in the TYPE II solution are described in detail below. The AEV assignment that occurs in Phase 1 requires a high gain comparator. Since the gain of the in-pixel differential amplifier is not high enough, we added an external common source (CS) amplifier (Figure 12). In this way, we obtained the three stage high gain comparator: (1) D; (2) SF; (3) CS, respectively. The Pix Value is compared to Ref inside each pixel of the selected row. The final amplification is performed by CS, which drives the Logic Decision signal (Figure 12). This signal stamps the appropriate AEV value onto C_AEV capacitance inside each pixel. Figure 13 illustrates the flow of the AEV assignment to a pixel matrix of 2 × 2 at the first span. Each pixel within the matrix has its own coordinates: the first denotes the row and the second denotes the column. We performed the simulation as if each pixel within this matrix receives a different incoming signal. Therefore, we stimulated the pixels with different current sources, imitating different discharge rates. The AEV values were generated by a piecewise ascending ramp voltage ranging from 0.9 V up to 2.6 V, corresponding to 18 different values distanced by 0.1 V from each other. The first span occurs after 100ns. Then, the photo-generated charge is transferred to C_S simultaneously within the whole matrix and is compared row by row with the reference, thus causing the corresponding AEV to be written onto C_AEV capacitance (Figure 10). Capacitances C_AEV<0,0> and C_AEV<0,1>, found at the row <0>, are processed first, whereas row <1> capacitances are programmed after the completion of row <0> programming.

Figure 12. Phase 1 of TYPE II Pixel.

Figure 12. Phase 1 of TYPE II Pixel.

Figure 13. Post-Layout Simulation of Phase 1 of TYPE II Pixel.

Figure 13. Post-Layout Simulation of Phase 1 of TYPE II Pixel.

In the illustrated case, there are four different photo-generated signals, falling within four different segments; therefore every pixel is assigned a different AEV from 0.9 V to 1.2 V. The lowest AEV is assigned to the most illuminated pixel, i.e., to C_AEV<0,1>, whereas the rest of the AEV’s: 1 V, 1.1 V, and 1.2 V are assigned to less illuminated pixels C_AEV<1,1>, C_AEV<0,0>, C_AEV<1,0>, respectively. It is important to note that the adjacent AEV values differ from each other by 2∆V_min, which is 0.1 V in this case, and this difference stays intact after the AEV’s were written to C_AEV. In such a case, decoding the analog data at the end of the frame will be easy; otherwise the final pixel value will be erroneous due to a possible shift of the stored AEV values.

It is important to understand that the time of the first span is shorter than the minimal integration time, since, during all the spans, the available signal to be accumulated is lower than the pixel swing by 2ΔV_min. However, during the next phase the entire pixel swing is available, thus the real integration period will be somewhat higher. In the specific example we have discussed, the minimal integration time will be not 100 ns, but rather 112 ns.

Phase 2 in a TYPE II pixel is ultra-fast. Due to its self-reset ability, the reset decision inside each pixel is autonomously fed from the differential stage through M₄ to the gate of M₅ (Figure 10). Consequently, each pixel is reset independently upon the comparison of its own AEV with the global reference Ref (Figure 14). Therefore, the minimal integration time for this design is:

(6)

Figure 14. Phase 2 of TYPE II Pixel.

Figure 14. Phase 2 of TYPE II Pixel.

Equation (6), which describes the minimal pixel conditional reset time, sets the minimal integration time and, as such, defines the available DR extension. In this case, the minimal integration times can be set as low as 112ns, which enables the DR to be extended by over five decades.

Figure 15a,b illustrates the flow of Phase 2 on a matrix of 2 × 2 pixels. There are four different illuminated pixels, each with its own AEV in Phase 1. At Phase 2, every pixel is reset in accordance with its AEV (Figure 15a). The bottom pixel, C_S<0,1>, has the lowest AEV and therefore it is reset for five times; whereas the pixel C_S<1,0>with the highest AEV is reset only twice. A close–up of the typical reset sequence is presented in Figure 15b. First, the differential stage inside the pixel is activated by raising the PD signal; then, the AEV is compared to a reference by raising Sel_AEV. After this, the bias point of the amplifier stabilizes, the Row Reset signal is activated, which enables the decision to reach the reset transistor. In the specific case we have presented, the reset decision is positive and therefore the capacitance Cs<0,1> is charged up. The whole sequence lasts for 112 ns; thus, by assuming a typical frame time of 30 ms, we obtain DR extension of 109 dB [‎8]. Adding to this the intrinsic DR of 60 dBs (10 bits), we obtain overall DR of 170 dB approximately.

Figure 15. Post-Layout simulation of: (a) 4 different pixel signals during Phase 1; (b) a close up of a single self-reset cycle.

Figure 15. Post-Layout simulation of: (a) 4 different pixel signals during Phase 1; (b) a close up of a single self-reset cycle.

The final A/D conversion occurs at Phase 3. We use the same comparator configuration as depicted in Figure 12 to perform a single slope conversion to a photo-generated signal (Mantissa) and to an AEV. Figure 16 depicts the process of A/D conversion to the stored AEV and the pixel Mantissa. First, the pixel AEV is converted using a ramp with reduced (“coarse”) resolution. It can easily be observed that the Ref signal spans the AEV domain in a reduced number of steps, which corresponds to the “coarse” conversion; whereas, the pixel Mantissa is spanned by the reference with a much higher (“fine”) resolution. As a matter of fact, after the Mantissa is converted, the capacitance C_S storing it is being reset for the next frame. The capacitance holding the AEV is not necessary to be reset, since it will be assigned the new AEV in Phase 1 of the subsequent frame in any case.

Figure 16. Post-Layout Simulation of Phase 3 of TYPE II pixel.

Figure 16. Post-Layout Simulation of Phase 3 of TYPE II pixel.

In conclusion, the relatively complex structure of the TYPE II pixel has enabled a very large DR extension and has substantially simplified the second and the third phases of the proposed algorithm. Figure 15, Figure 16 prove the feasibility of the TYPE II design and indicate that sensor based upon a TYPE II pixel can be successfully implemented in a silicon process.

2.8. A Power Profile of TYPE I and TYPE II Designs

Another important factor is the examination of the power performance of the TYPE I and TYPE II designs, since, if they were power hungry, it would be pointless using them. We made both of the designs optimized for low power using the following methods: the two stacked transistors scheme, and multiple power supplies. The result was that the leakage within the pixels was successfully minimized, and the power supplies could be lowered easily without degrading the anticipated sensor's performance.

In analyzing the TYPE I power per pixel (PPX), we have concluded that most of the power is consumed during the AEV assignment (Phase 1) and the A/D conversion (Phase 3), because these two operations require activating the comparators for a longer period of time than in Phase 2 (Figure 17a).

In TYPE II, the relative power contribution of Phase 2 is also the lowest, in comparison with the two other phases, as its overall effective duration is the shortest one (Figure 17b). Most of the power is consumed during Phase 1, which contains an increased number of spans and, thereby, in aggregate, lasts longer than Phase 3. Phase 3’s contribution to PPX, is almost the same as in the TYPE I case due to similar flow of the data conversion.

Based on these simulations, we have concluded that the two presented designs consume power of 2.5 nW and 5.4 nW, respectively. Taking into account the anticipated extraordinary DR extension they provide, we have found that the power budget is low and definitely appropriate to the state-of-the art CMOS image sensors [25].

Figure 17. Power per Pixel Distribution of TYPE I and TYPE II Designs. (a) PPX TYPE I = 2.5 nW; (b) PPX TYPE I = 5.4 nW.

Figure 17. Power per Pixel Distribution of TYPE I and TYPE II Designs. (a) PPX TYPE I = 2.5 nW; (b) PPX TYPE I = 5.4 nW.

To summarize the anticipated performance of the proposed designs, we provide Table 1, where different state-of-the art WDR solutions are compared with respect to several key attributes such as fabrication technology, WDR technique (WDR T.), pixel size, fill factor (FF), DR, SNR, power per pixel (PPX), and frame rate (FR).

Table 1. Comparison between state-of-the art WDR image sensors.

Parameter	[11]	[5]	[12]	[14]	[21]	TYPE I	TYPE II
Technology	0.18 µm	0.09 µm	0.18 µm	0.35 µm	0.18 µm	0.18 µm	0.18 µm
WDR T.	Well Cap.Adjustment	Well Cap.Adjustment	Frequency	TFS	Multiple Capt.	Multiple Resets	Multiple Resets
Pixel Size	3 µm × 3 µm	5.86 µm × 5.86 µm	23 µm × 23 µm	81.5 µm × 76.5 µm	5.6 µm × 5.6 µm	14 µm × 14 µm	17 µm × 17 µm
FF	-	-	25%	2%	45%	40%	25%
DR	100 dB	83 dB	130 dB	100 dB	99d B	108 dB	170 dB
SNR	48 dB*	48 dB *	-	-	-	48 dB *	48 dB *
PPX	-	400 nW **	250 nW	≤6.4 µW **	10 nW **	2.5 nW	5.4 nW
FR	-	30	-	-	15	33	33

(^*): The SNR was assessed based upon the well capacity of the photo-diode; (^**): The PPX was calculated by normalization of total power by the number of pixels within the array.

Table 1. Comparison between state-of-the art WDR image sensors.

Parameter	[11]	[5]	[12]	[14]	[21]	TYPE I	TYPE II
Technology	0.18 µm	0.09 µm	0.18 µm	0.35 µm	0.18 µm	0.18 µm	0.18 µm
WDR T.	Well Cap.Adjustment	Well Cap.Adjustment	Frequency	TFS	Multiple Capt.	Multiple Resets	Multiple Resets
Pixel Size	3 µm × 3 µm	5.86 µm × 5.86 µm	23 µm × 23 µm	81.5 µm × 76.5 µm	5.6 µm × 5.6 µm	14 µm × 14 µm	17 µm × 17 µm
FF	-	-	25%	2%	45%	40%	25%
DR	100 dB	83 dB	130 dB	100 dB	99d B	108 dB	170 dB
SNR	48 dB*	48 dB *	-	-	-	48 dB *	48 dB *
PPX	-	400 nW **	250 nW	≤6.4 µW **	10 nW **	2.5 nW	5.4 nW
FR	-	30	-	-	15	33	33

(^*): The SNR was assessed based upon the well capacity of the photo-diode; (^**): The PPX was calculated by normalization of total power by the number of pixels within the array.

From Table 1, we can understand that the pixel sizes of pixels presented in our work are mediocre. They are substantially larger than those associated with well capacity adjustment and multiple captures, but much smaller than frequency and TFS based sensors. Important to note that both of our designs maintain a decent FF relatively other listed solutions due to area effective layout. The DR of proposed herein designs emphasizes the extraordinary ability of multiple resets algorithm to extend the pixel dynamic range. TYPE I provides remarkable DR, whilst TYPE II brings it to extreme values. Moreover, such DR is obtained, maintaining excellent SNR (see Appendix B). Not less important to note that both of proposed designs operate at video frame rate and present the best power performance.

3. A Comparative Analysis of Multiple Captures & Multiple Resets Algorithms

When a new WDR solution is proposed, it is always interesting to compare it to the existing ones and to realize the added value the new solution contributes. In the present discussion, we perform a comparison between the two proposed solutions to the following algorithms: the multiple captures [15] and the former multiple resets solutions [24]. We chose to compare the number of operations needed to obtain a single digitized WDR pixel value; because, in our opinion, the most significant contribution of TYPE I and TYPE II solutions is their effectiveness and their ability to substantially reduce the processing time.

First, the multiple captures algorithm must be considered. In this solution, the pixel integrates for several periods of time, regardless of the intensity of the incoming light. After each integration, the intermediate pixel value is synthesized out of both the newly generated and previously stored samples, respectively. At this stage we assume that all the captures are performed in a rolling shutter mode to comply with the minimal memory requirements for this solution. In case multiple captures are performed in a global shutter mode, the memory requirements will be much higher. For a rolling shutter operation mode, the number of memory bits for each pixel equals its digital resolution: N_resolution. The total number of operations that are required to obtain a final digitized result for a single pixel after N_captures exposures is:

(7)

where 1_convert, 1_read, and 1_write stand for ADC, memory read, and write cycles, respectively. Please note, that in this case there are two captures only: the memory actually becomes a single row register where the first sample is kept, in order to be compared with the second.

On the other hand, both the previous and newly proposed multiple resets solutions are applied on a global shutter sensor to obtain the maximal possible number of operations. According to the snapshot mode, the number of memory bits for a single pixel equals Rep_EXP, which is the representation of the exponent bits N_EXP. Obviously, in case of linear representation, Rep_EXP equals N_EXP. If logarithmic representation is applied, the Rep_EXP is reduced to log₂N_EXP + 1 [27]. For example, if there are 6 exponent bits, then in the first case, Rep_EXP is 6, whereas in the second it is 4. The overall number of operations in previous solutions is:

(8)

The first term in (8), relates to the sequential generation of DR bits, which consists of memory read, pixel conversion by the comparator, and memory write cycles. The second term relates to the digital readout of the memory contents, and the last term denotes the A/D conversion of the pixel Mantissa at the end of the frame.

In the newly presented TYPE I and TYPE II solutions, the total number of operations coincides with:

(9)

The term l stands for the number of spans performed during Phase 1; N_EXP is for Phase 2, and the last term of 2 denotes the number of conversions during Phase 3.

Next, we will demonstrate the meaning of Equations (7–9) by substituting the possible values for each parameter. We have presented the results in Table 2, where each row is allocated for a different N_captures and N_EXP.

For the previous solutions of multiple resets, we chose the logarithmical representation as in [27], because it minimizes the required external memory size. We also have assumed that the number of the exponent bits N_EXP equals the number of captures: N_captures. This assumption not only simplifies the calculations, but it also ensures that the SNR dips, which exist throughout the extended DR in all of the solutions, are equal.

The advantage of the multiple captures algorithm is that most of it is implemented in a digital domain and the pixel structure is rather simple. The clear disadvantage is that this solution requires an extensive processing to obtain the final result. From Equation (7), it is shown that the number of operations increases linearly with the number of captures. This fact is clearly demonstrated by Table 2, where the multiple captures solution obviously possesses the highest number of operations per pixel. It is possible, though, to limit the number of captures to two or three; however this will cause large SNR dips at the boundary light intensities [8]. Circumventing these SNR artifacts by adding more integration slots will certainly slow the sensor operation speed. Consequently, the multiple captures algorithm is best suited to photograph slow changing scenes with a very high spatial resolution.

Table 2. The Comparison of a number of operations for “Multiple Captures”, “Multiple Resets Previous”, and “Multiple Resets New” algorithms.

NEXP = Ncaptures
Ncaptures = NEXP	Multiple Captures [15]	Multiple Resets Previous [27]	Multiple Resets New (This Work)
2	21	9	5
3	42	13	6
4	63	16	7
6	105	23	10
18	357	61	25

Table 2. The Comparison of a number of operations for “Multiple Captures”, “Multiple Resets Previous”, and “Multiple Resets New” algorithms.

NEXP = Ncaptures
Ncaptures = NEXP	Multiple Captures [15]	Multiple Resets Previous [27]	Multiple Resets New (This Work)
2	21	9	5
3	42	13	6
4	63	16	7
6	105	23	10
18	357	61	25

In the multiple resets algorithm, although the pixel structure and the pixel control are more complex than in the first case, the overall processing is much faster, since the pixel adjusts the integration time autonomously during the frame. Thus, there is no need for an extensive post-integration processing—only to combine the data of the Mantissa and the EXP bits. From Table 2, we realize that even the previous configurations of multiple resets have a reduced number of operations, relative to the multiple captures algorithm. The intuitive explanation of this is found within a sequential processing of the pixel information, which allows reading only the last produced bit from the memory before generating the subsequent one. The logarithmical representation of the EXP bits, used in the previous designs, decreases the operations number even further. Therefore, the previous solutions of multiple resets were perfectly suited to capture fast changing WDR scenes with a decent resolution.

The two solutions, proposed within the current study, bring the number of operations to its minimum, since they completely eliminate the operations related to a digital memory unit. This feature enables the proposed solutions to be the fastest among the three, which have been analyzed in this discussion. The clear disadvantage of the pixel using AEV is a relatively complex analog signal processing inside each pixel, which is especially pronounced in TYPE II design. Therefore, the newly proposed solutions can be candidates for implementation for capturing a very fast changing scene with wide or ultra-wide dynamic range.

4. Summary

We have presented two designs: TYPE I and TYPE II featuring the “ranging” and the AEV concepts. We have shown that it is possible to perform effective encoding of the DR bits using analog voltage, which can be easily stored within the pixel and processed further through the readout chain. Moreover, we have condensed the AEV assignment to a single phase in a process called “ranging”, which has led to another improvement: a substantial simplification of the conditional reset. In the past, each cycle consisted of three or even more different operations; now the number of operations has been nearly halved. The AEV solution has also granted a possibility to save chip area due to reuse of high gain comparators as a main block in a single slope ADC. The post-layout simulations and power calculations included in our study have shown that both the TYPE I and the TYPE II designs are feasible and can be successfully implemented in a CMOS process. The presented designs have reached a remarkable DR of up to 170 dB, while only consuming power of up to 5.4 nW. A further comparison between this work and the multiple captures and the former multiple resets solutions has clearly proven that the AEV concept effectively reduces the number of operations required to produce a pixel value and, as such, can be considered as a key to design a fast, low-power and ultra WDR CMOS sensor.