MDPI - Publisher of Open Access Journals

16 pages, 1447 KB

Open AccessArticle

Noise Suppressed Image Reconstruction for Quanta Image Sensors Based on Transformer Neural Networks

by Guanjie Wang and Zhiyuan Gao

J. Imaging 2025, 11(5), 160; https://doi.org/10.3390/jimaging11050160 - 17 May 2025

Cited by 2 | Viewed by 1823

The photon detection capability of quanta image sensors make them an optimal choice for low-light imaging. To address Possion noise in QIS reconstruction caused by spatio-temporal oversampling characteristic, a deep learning-based noise suppression reconstruction method is proposed in this paper. The proposed neural [...] Read more.

The photon detection capability of quanta image sensors make them an optimal choice for low-light imaging. To address Possion noise in QIS reconstruction caused by spatio-temporal oversampling characteristic, a deep learning-based noise suppression reconstruction method is proposed in this paper. The proposed neural network integrates convolutional neural networks and Transformers. Its architecture combines the Anscombe transformation with serial and parallel modules to enhance denoising performance and adaptability across various scenarios. Experimental results demonstrate that the proposed method effectively suppresses noise in QIS image reconstruction. Compared with representative methods such as TD-BM3D, QIS-Net and DPIR, our approach achieves up to 1.2 dB improvement in PSNR, demonstrating superior reconstruction quality. Full article

(This article belongs to the Section Image and Video Processing)

► Show Figures

Figure 1

12 pages, 5438 KB

Open AccessArticle

Threshold Uniformity Improvement in 1b Quanta Image Sensor Readout Circuit

by Zhaoyang Yin, Jiaju Ma, Saleh Masoodian and Eric R. Fossum

Sensors 2022, 22(7), 2578; https://doi.org/10.3390/s22072578 - 28 Mar 2022

Cited by 6 | Viewed by 3243

Abstract

A new readout architecture for single-bit quanta image sensor (QIS) consisting of a capacitive transimpedance amplifier (CTIA) before a 1-bit quantizer to improve the threshold uniformity of the readout cluster is proposed in this paper. The 1-bit quantizer in the previous single-bit QIS [...] Read more.

A new readout architecture for single-bit quanta image sensor (QIS) consisting of a capacitive transimpedance amplifier (CTIA) before a 1-bit quantizer to improve the threshold uniformity of the readout cluster is proposed in this paper. The 1-bit quantizer in the previous single-bit QIS had significant threshold non-uniformity likely caused by the fluctuation of the common-mode voltage of the jot output. To guarantee the stability of the common-mode voltage of input signals fed to the 1-bit quantizer, the CTIA is added before the 1-bit quantizer. A pipeline operation mode is also proposed so the CTIA and 1-bit ADC can work at the same time, thereby reducing the CTIA power consumption. A 2048 × 1024 high-speed test chip was implemented with 45 nm/65 nm stacked backside illuminated (BSI) CMOS image sensor (CIS) process and tested. According to the measured D-log-H results, a good threshold uniformity in the range of 0.3 to 0.8 e− for all readout clusters is demonstrated at 500 frame per second (fps) equivalent timing with 68 mW power consumption. Full article

(This article belongs to the Section Sensing and Imaging)

► Show Figures

Figure 1

16 pages, 6644 KB

Open AccessFeature PaperArticle

1/f Noise Modelling and Characterization for CMOS Quanta Image Sensors

by Wei Deng and Eric R. Fossum

Sensors 2019, 19(24), 5459; https://doi.org/10.3390/s19245459 - 11 Dec 2019

Cited by 16 | Viewed by 9110

Abstract

This work fits the measured in-pixel source-follower noise in a CMOS Quanta Image Sensor (QIS) prototype chip using physics-based 1/f noise models, rather than the widely-used fitting model for analog designers. This paper discusses the different origins of 1/f noise in [...] Read more.

This work fits the measured in-pixel source-follower noise in a CMOS Quanta Image Sensor (QIS) prototype chip using physics-based 1/f noise models, rather than the widely-used fitting model for analog designers. This paper discusses the different origins of 1/f noise in QIS devices and includes correlated double sampling (CDS). The modelling results based on the Hooge mobility fluctuation, which uses one adjustable parameter, match the experimental measurements, including the variation in noise from room temperature to –70 °C. This work provides useful information for the implementation of QIS in scientific applications and suggests that even lower read noise is attainable by further cooling and may be applicable to other CMOS analog circuits and CMOS image sensors. Full article

(This article belongs to the Special Issue Special issue on the 2019 International Image Sensor Workshop (IISW2019))

► Show Figures

Figure 1

13 pages, 4342 KB

Open AccessArticle

Effect of the Transition Points Mismatch on Quanta Image Sensors

by Jiangtao Xu, Xiyang Zhao, Liqiang Han, Kaiming Nie, Liang Xu and Jianguo Ma

Sensors 2018, 18(12), 4357; https://doi.org/10.3390/s18124357 - 10 Dec 2018

Cited by 4 | Viewed by 3039

Abstract

Mathematical models and imaging models that show the relationship between the transition points mismatch of analog-to-digital converters (ADCs) and the bit error rate (BER) in single-bit and multi-bit quanta image sensors (QISs) are established. The mathematical models suggest that when the root-mean-square (r.m.s.) [...] Read more.

Mathematical models and imaging models that show the relationship between the transition points mismatch of analog-to-digital converters (ADCs) and the bit error rate (BER) in single-bit and multi-bit quanta image sensors (QISs) are established. The mathematical models suggest that when the root-mean-square (r.m.s.) of the read noise in jots is 0.15e⁻, the standard deviation of the transition points should be less than 0.15e⁻ to ensure that the BER is lower than 1% in the single-bit QIS, and 0.21e⁻ to ensure that the BER is lower than 5% in the multi-bit QIS. Based on the mathematical models, the imaging models prove that the fixed-pattern noise (FPN) increases with a stronger transition point mismatch. The imaging models also compare the imaging quality in the case of different spatial oversampling factors and bit depths. The grayscale similarity index (GSI) is 3.31 LSB and 1.74 LSB when the spatial oversampling factors are 256 and 4096, respectively, in the single-bit QIS. The GSI is 1.93 LSB and 1.13 LSB when the bit depth is 3 and 4, respectively, in the multi-bit QIS. It indicates that a higher bit depth and a larger spatial oversampling factor could reduce the effect of the transition points mismatch of1-bit or n-bit ADCs. Full article

(This article belongs to the Section Physical Sensors)

► Show Figures

Figure 1

16 pages, 26646 KB

Open AccessArticle

High Dynamic Range Imaging at the Quantum Limit with Single Photon Avalanche Diode-Based Image Sensors

by Neale A.W. Dutton, Tarek Al Abbas, Istvan Gyongy, Francescopaolo Mattioli Della Rocca and Robert K. Henderson

Sensors 2018, 18(4), 1166; https://doi.org/10.3390/s18041166 - 11 Apr 2018

Cited by 42 | Viewed by 11436

Abstract

This paper examines methods to best exploit the High Dynamic Range (HDR) of the single photon avalanche diode (SPAD) in a high fill-factor HDR photon counting pixel that is scalable to megapixel arrays. The proposed method combines multi-exposure HDR with temporal oversampling in-pixel. [...] Read more.

This paper examines methods to best exploit the High Dynamic Range (HDR) of the single photon avalanche diode (SPAD) in a high fill-factor HDR photon counting pixel that is scalable to megapixel arrays. The proposed method combines multi-exposure HDR with temporal oversampling in-pixel. We present a silicon demonstration IC with 96 × 40 array of 8.25 µm pitch 66% fill-factor SPAD-based pixels achieving >100 dB dynamic range with 3 back-to-back exposures (short, mid, long). Each pixel sums 15 bit-planes or binary field images internally to constitute one frame providing 3.75× data compression, hence the 1k frames per second (FPS) output off-chip represents 45,000 individual field images per second on chip. Two future projections of this work are described: scaling SPAD-based image sensors to HDR 1 MPixel formats and shrinking the pixel pitch to 1–3 µm. Full article

(This article belongs to the Special Issue Special Issue on the 2017 International Image Sensor Workshop (IISW))

► Show Figures

Figure 1

21 pages, 2815 KB

Open AccessArticle

Images from Bits: Non-Iterative Image Reconstruction for Quanta Image Sensors

by Stanley H. Chan, Omar A. Elgendy and Xiran Wang

Sensors 2016, 16(11), 1961; https://doi.org/10.3390/s16111961 - 22 Nov 2016

Cited by 57 | Viewed by 13161

Abstract

A quanta image sensor (QIS) is a class of single-photon imaging devices that measure light intensity using oversampled binary observations. Because of the stochastic nature of the photon arrivals, data acquired by QIS is a massive stream of random binary bits. The goal [...] Read more.

A quanta image sensor (QIS) is a class of single-photon imaging devices that measure light intensity using oversampled binary observations. Because of the stochastic nature of the photon arrivals, data acquired by QIS is a massive stream of random binary bits. The goal of image reconstruction is to recover the underlying image from these bits. In this paper, we present a non-iterative image reconstruction algorithm for QIS. Unlike existing reconstruction methods that formulate the problem from an optimization perspective, the new algorithm directly recovers the images through a pair of nonlinear transformations and an off-the-shelf image denoising algorithm. By skipping the usual optimization procedure, we achieve orders of magnitude improvement in speed and even better image reconstruction quality. We validate the new algorithm on synthetic datasets, as well as real videos collected by one-bit single-photon avalanche diode (SPAD) cameras. Full article

(This article belongs to the Special Issue Photon-Counting Image Sensors)

► Show Figures

Graphical abstract

25 pages, 7848 KB

Open AccessReview

The Quanta Image Sensor: Every Photon Counts

by Eric R. Fossum, Jiaju Ma, Saleh Masoodian, Leo Anzagira and Rachel Zizza

Sensors 2016, 16(8), 1260; https://doi.org/10.3390/s16081260 - 10 Aug 2016

Cited by 112 | Viewed by 27941

Abstract

The Quanta Image Sensor (QIS) was conceived when contemplating shrinking pixel sizes and storage capacities, and the steady increase in digital processing power. In the single-bit QIS, the output of each field is a binary bit plane, where each bit represents the presence [...] Read more.

The Quanta Image Sensor (QIS) was conceived when contemplating shrinking pixel sizes and storage capacities, and the steady increase in digital processing power. In the single-bit QIS, the output of each field is a binary bit plane, where each bit represents the presence or absence of at least one photoelectron in a photodetector. A series of bit planes is generated through high-speed readout, and a kernel or “cubicle” of bits (x, y, t) is used to create a single output image pixel. The size of the cubicle can be adjusted post-acquisition to optimize image quality. The specialized sub-diffraction-limit photodetectors in the QIS are referred to as “jots” and a QIS may have a gigajot or more, read out at 1000 fps, for a data rate exceeding 1 Tb/s. Basically, we are trying to count photons as they arrive at the sensor. This paper reviews the QIS concept and its imaging characteristics. Recent progress towards realizing the QIS for commercial and scientific purposes is discussed. This includes implementation of a pump-gate jot device in a 65 nm CIS BSI process yielding read noise as low as 0.22 e− r.m.s. and conversion gain as high as 420 µV/e−, power efficient readout electronics, currently as low as 0.4 pJ/b in the same process, creating high dynamic range images from jot data, and understanding the imaging characteristics of single-bit and multi-bit QIS devices. The QIS represents a possible major paradigm shift in image capture. Full article

(This article belongs to the Special Issue Photon-Counting Image Sensors)

► Show Figures

Graphical abstract

17 pages, 3125 KB

Open AccessArticle

Single Photon Counting Performance and Noise Analysis of CMOS SPAD-Based Image Sensors

by Neale A. W. Dutton, Istvan Gyongy, Luca Parmesan and Robert K. Henderson

Sensors 2016, 16(7), 1122; https://doi.org/10.3390/s16071122 - 20 Jul 2016

Cited by 34 | Viewed by 17636

Abstract

SPAD-based solid state CMOS image sensors utilising analogue integrators have attained deep sub-electron read noise (DSERN) permitting single photon counting (SPC) imaging. A new method is proposed to determine the read noise in DSERN image sensors by evaluating the peak separation and width [...] Read more.

SPAD-based solid state CMOS image sensors utilising analogue integrators have attained deep sub-electron read noise (DSERN) permitting single photon counting (SPC) imaging. A new method is proposed to determine the read noise in DSERN image sensors by evaluating the peak separation and width (PSW) of single photon peaks in a photon counting histogram (PCH). The technique is used to identify and analyse cumulative noise in analogue integrating SPC SPAD-based pixels. The DSERN of our SPAD image sensor is exploited to confirm recent multi-photon threshold quanta image sensor (QIS) theory. Finally, various single and multiple photon spatio-temporal oversampling techniques are reviewed. Full article

(This article belongs to the Special Issue Photon-Counting Image Sensors)

► Show Figures

Graphical abstract

8 pages, 1260 KB

Open AccessArticle

Quantum Random Number Generation Using a Quanta Image Sensor

by Emna Amri, Yacine Felk, Damien Stucki, Jiaju Ma and Eric R. Fossum

Sensors 2016, 16(7), 1002; https://doi.org/10.3390/s16071002 - 29 Jun 2016

Cited by 7 | Viewed by 11695

Abstract

A new quantum random number generation method is proposed. The method is based on the randomness of the photon emission process and the single photon counting capability of the Quanta Image Sensor (QIS). It has the potential to generate high-quality random numbers with [...] Read more.

A new quantum random number generation method is proposed. The method is based on the randomness of the photon emission process and the single photon counting capability of the Quanta Image Sensor (QIS). It has the potential to generate high-quality random numbers with remarkable data output rate. In this paper, the principle of photon statistics and theory of entropy are discussed. Sample data were collected with QIS jot device, and its randomness quality was analyzed. The randomness assessment method and results are discussed. Full article

(This article belongs to the Special Issue Photon-Counting Image Sensors)

► Show Figures

Graphical abstract

Search Results (9)

Further Information

Guidelines

MDPI Initiatives

Follow MDPI

Saved Queries

Search Filter Reset All

Years

Feature Papers

Subjects

Journals

Article Types

Countries / Regions

Search Results (9)

Further Information

Guidelines

MDPI Initiatives

Follow MDPI