A 19 ps Precision and 170 M Samples/s Time-to-Digital Converter Implemented in FPGA with Online Calibration

Abstract

This paper presents a 19 ps precision and 170 M samples/s time-to-digital converter (TDC) in FPGA. Through the direct count method and tapped delay line method, the coarse count and fine count can be extracted, respectively. The direct count is realized by the 350 M clock and the tapped delay line is constructed by the CARRY4 block. The ones-counter encoder is used to convert the thermometer code with bubble errors into binary code, which is applicable to all the FPGA chips. This work not only explains the schematic of the ones-counter encoder, but also shows how to configure it. Owing to the inconsistency of delay elements caused by process, bin-by-bin calibration is utilized to improve the differential nonlinearities (DNL) and integral nonlinearities (INL) of the TDC. A novel method was developed to compensate the influence of voltage and temperature. As the delay elements vary with voltage and temperature, a frequency counter is used to extrapolate and compensate its effect on the delay line. All of the above strategies use online calibration and improve the precision and sampling rate of TDC. The experimental results show the least significant bit (LSB) achieves 17.4 ps, the DNL is within [−0.90, 1.67] LSB, and the INL is in the range of [−1.90, 3.31] LSB.

1. Introduction

Time-to-digital converters (TDCs) are applied to various areas of research such as laser distance meters, high energy physics, and medical application [1,2,3,4,5]. In positron emission tomography (PET), TDCs are indirectly used to reduce the position of positron emissions [6]. To meet the particle identification requirements of high counting rate and high time resolution in the PET, TDCs must provide high measurement speeds and high measurement precision [7].

For decades, compared with TDCs based on Application Specific Integrated Circuits (ASICs), TDCs based on Field Programmability Gate Arrays (FPGAs) have been developed rapidly due to the advantages of shorter development time and higher flexibility. The common technique in an FPGA-based TDC consists of the coarse counter and the time interpolator [8,9]. Generally, the coarse counter is obtained by the direct count of a system clock. Recent studies have demonstrated that there are three main methods to implement the time interpolator, such as Vernier delay line (VDL) [10,11], multiple clock phases (MCPs) [12,13], and a tapped delay line (TDL) [14,15,16]. The VDL-TDC utilizes two similar clock frequencies to measure the time interval. However, it is not suitable for high speed measurements to be used with a long dead time. In 2019, a VDL-TDC achieved 28 ps precision and 602 ns dead time through two ring oscillators [10]. The guideline of the MCP-TDC is that the hit is sampled by N channels of clocks with different phase shifts. Nevertheless, it is hard to obtain a higher resolution because the number of clock phase shifts does not exceed 16. An MCP-TDC structure with 256 channels, 56.2 ps precision, and 4.3 ns dead time was designed in the paper [17]. VDL-TDCs and MCP-TDCs are not suitable for high speed and high precision measurements because of their long dead time and poor precision measurements. The TDL employs the delay elements of the FPGA as a time interpolator. With the advancement of FPGA manufacturing technology, the intrinsic propagation delay becomes smaller and the TDL-TDC becomes the best choice for a high-speed and high-precision time interpolator. The critical issue in the TDL-TDC is the severe bubble errors on the FPGA fabricated with advanced process technology [18]. To solve this problem, one technology that can be used is to reorder the taps according to the static timing analysis (STA) [19]. The real sequence is not consistent with the STA due to the clock slew, thus the bubble problem is not completely solved by reordering the taps. Another improvement is using bin realignment to reorder the taps [20], but it is quite complicated because it requires at least two cycles for the FPGA synthesis. The non-uniformity of delay elements caused by process variation and mismatch in the TDL-TDC is another considerable problem. The scale and linearity of the delay elements in the TDL determine the time precision. The intrinsic propagation delay reflects scale, whereas the differential nonlinearities (DNL) and integral nonlinearities (INL) represent the linearity. The average delay method [21] and the bin-by-bin calibration methodology [22,23,24] were proposed to improve linearity. Recent studies have found that the bin-by-bin calibration is the most suitable approach for a TDL-TDC. In 2021, a 26.04 ps precision and 125 M samples/s TDC was design by using bin-by-bin calibration [25]. However, that manuscript did not introduce many details about the calibration, and it was not clear that the bin-by-bin calibration was carried out using software or hardware. Moreover, it was not verified by experiments that the sample rate reached 125 M. To improve the precision and sample rate, the bin-by-bin calibration needs to be online and automatic in the hardware.

In this work, the thermometer code with bubble errors is converted to the binary code through the ones-counter encoder, as it could be naturally used for global bubbling error correction. The ones-counter encoder can work at 350 MHz while maintaining 170 M samples/s, owing to the pipelined adder tree structure. Furthermore, due to the inconsistency of delay elements caused by the process, the bin-by-bin calibration is used to increase their linearity. There are two calibration tables (histogram/lookup table) in this design, which are used to calibrate automatically and online in the hardware. As the delay elements vary with voltage and temperature, a frequency counter is used to extrapolate their effects on the delay line and compensate them. The contribution of this work makes the TDL-TDC accessible to time measurement with high precision and speed related applications easily. For example, if the bandwidth of a silicon photomultiplier (sipm) detector is 7 M, it is expected that such a TDC can be utilized for a PET.

This paper is organized as follows: Section 2 describes the architecture of the TDC design, which employs a ones-counter encoder and online calibration. Section 3 contains the assessment platform, characterization of the TDC performance, and comparison to state-of-the-art studies. Finally, Section 4 comprises the summary and conclusion.

2. TDC Architecture

Figure 1 shows the architecture of the proposed TDL-TDC. The single-channel consists of a mixed-mode clock manager (MMCM), a coarse counter, a filter stage, a tapped delay line, a ring oscillator, a frequency counter, a ones-counter encoder, a histogram and a look up table (made by dual port RAM). The MMCM creates a 350 MHz clock from the 100 MHz system clock. The coarse counter is 24 bits at the time range of 47.9 ms. The tapped delay line is implemented with a cascading CARRY4 block. The ones-counter encoder module converts the thermometer code with bubble errors into binary code. The AXI BRAM controller module is used to calibrate the influence of the process and temperature on the tapped delay line. The main abbreviations applied in this paper are shown in

Table 1. Description of abbreviations.

Abbreviations	Description
TDC	Time-to-digital converter
FPGA	Field Programmable Gate Array
CARRY4	Slice primitive: Fast Carry Logic with Look Ahead
LSB	least significant bit
PET	positron emission tomograph
ASIC	Application Specific Integrated Circuit
VDL	Vernier delay line
MCP	multiple clock phases
TDL	tapped delay line
STA	static timing analysis
DNL	differential nonlinearities
INL	integral nonlinearities
MMCM	mixed-mode clock manager
LDO	low dropout regulator
IDELAYCTRL	Xilinx primitive: IDELAYE2/ODELAYE2 Tap Delay Value Control
IDELAYE2	Xilinx primitive: Input Fixed or Variable Delay Element

.

The TDL self-regulates and requires no additional reset logic. By utilizing the filter stage to filter the hit, the width of the hit becomes the time interval between the rising edge of the signal and the next rising edge of the clock, as shown in Figure 1. In the “FF1” stage, if the input of “D” is 1 and “CLR” is 0, the rising edge of the hit propagates along the TDL. When the hit transmitters conduct the first tap of the TDL, the “FF2” stage clears the “FF1” stage at the rising edge of the next clock and prevents the hit from entering the TDL. Once the entire TDL is cleared, the ”CLR” is set to 0 again and the TDL is ready to receive a new pulse [25].

2.1. Tapped Delay Line

The tapped delay line is the main module of the TDL-TDC in charge of the fine count. In fact, it can be formed by cascading any logic gates. For the purpose of high precision measurement, it is necessary to select a gate with the smallest delay to build a delay line in the FPGA. Previous research has shown that the CARRY4 block can be used as a delay element [26,27]. As shown in Figure 2, there are four multiplexers (MUXCYs) in the CARRY4 block, which can be perfectly used as the delay element. The hit signal propagates from the bottom to the top, which means that adjacent slices can be connected into carry chains in the vertical direction. In this paper, each CARRY4 module leads out CO0-CO3 as four taps. The TDL states are logic zeros if there is no signal. When bringing a pulse to the input, the states of the TDL propagate from “0” to “1”. When the next clock comes, D flip-flops to capture the TDL states in the form of a thermometer code. This means that it can accurately measure the time interval between the rising edge of the input signal and the next rising edge of the clock. As the clock and signal are asynchronous, more pipelined registers are created to eliminate metastability.

All delay elements of the TDL must be modified and located in the same clock region to avoid additional mismatch. The STA shows the delay of a CARRY4 block in the ZYNQ-7020 is 100 ps. Considering the ones-counter encoder in this paper, the number of taps should be a multiple of 6. In this work, 48 CARRY4 blocks are used to ensure the total delay is longer than 2.857 ns (350 MHz clock).

2.2. One-Counter Encoder

Normally, the TDL state changes from “0” to “1” in the order of taps, as shown in Figure 3a. However, Figure 3b shows that there are bubble errors in the TDL when simulating the timing of the delay line. It attributes to process, skew and jitters of the clock, or metastability of the D flip-flops. For example, under normal conditions, the D flip-flops will capture “111100”, whereas under the conditions of a bubble problem, the D flip-flops will capture “111010”.

The ones-counter encoder has been proposed as a remedy for bubble errors in the flash ADC [28]. The example illustrated in Figure 3 demonstrates the principle of the ones-counter encoder to solve this problem, with the state of each tap indicated as T0, T1, …, and T5. The TDL output is a perfect thermometer code when the tap sequence matches the actual delay sequence, as shown in Figure 3a. In Figure 3b, the TDL output has bubbles (if the tap sequence does not match the sequence). In fact, the output codes of the two situations have the same number of ones. The output “1” indicates the number of taps covered by the hit. It is not difficult to predict the ones-counter encoder will generate the same binary code no matter if there is a bubble error or not. It simplifies and clarifies the TDC design. Unlike the flash ADC design, this paper focuses on which encoder can best balance resources, power consumption, and operating speed to accomplish high speed measurement in a reasonable manner.

Figure 4 shows the ones-counter encoder used in this experiment. In stage 1, three 6 look up tables (6-LUT is an FPGA primitive) connect to six taps in parallel, encoding the six thermometer codes into a 3-bit binary code. The other stages are inputted into the pipelined adder tree structure, which accepts the 3-bit binary array as input and creates the 8-bit output. Due to the dynamic range of the final result being 8-bit (192 taps in total), only the 8-bit binary code is fed into the following calibration.

The initialization parameter for the 6-LUT primitive is what gives the LUT its logical value. By default, this value is zero, thus driving the output to a zero regardless of the input values. However, a new initialization value needs to be determined in order to specify the logic function. In this paper, the initialization contents of three 6-LUTs are configured to x“6996966996696996”, x“8117177E177E7EE8”, and x“FEE8E880E8808000”. These three Hex numbers give the configuration of the 6-LUT initialization contents, which is applicable to all different Xilinx devices. The function of the three 6-LUTs is to convert the six thermometer codes with bubble errors into binary codes. For example, as shown in

Table 2. The initialization contents of three 6-LUTs.

Hex Numbers	Binary Numbers
6996966996696996	01101001100101101001011001101001 10010110011010010110100110010110
8117177E177E7EE8	10000001000101110001011101111110 00010111011111100111111011101000
FEE8E880E8808000	11111110111010001110100010000000 11101000100000001000000000000000

, the six taps inputted are “111010”, which corresponds to the 59-th bit of initialization value (made with green), therefore, the three 6-LUTs output are “100”.

2.3. Online Calibration

Online calibration is performed to track and compensate PVT variations. Due to process, the delay element is not as uniform as the dedicated delay line in the ASIC. Some delay elements may have a small delay, whereas some delay elements may have a large delay. If the delay of each delay element is consistent, the bin width of each tap is uniform. The bin width of each tap is obtained by a code density test [16,20,29,30]. To balance resources and a correction effect, 1,024,000 is chosen as the number of test N in this design. The bin width of each delay element is determined using the following equation:

$$W_i=\frac{N_i}{N}{T}_{clk}$$

(1)

where W_i is the bin width of i-th tap, N_i is the hit number of each tap, and T_clk is the clock period. The DNL and INL are usually used to represent the linearity of the TDC. The DNL is the difference between the actual bin width and ideal bin width. The INL is the cumulative value, which indicates the deviation from the ideal value. The DNL and INL of the i-th tap are defined as follows:

$$DN{L}_i=\frac{W_i-W_{LSB}}{W_{LSB}}$$

(2)

$$IN{L}_i={\displaystyle \sum _{k=1}^{i}DN{L}_k}$$

(3)

To make the TDC more linear, Wu explained the bin-by-bin calibration structure [31]. The DNL and INL are derived from the histogram in this method and the bin-by-bin calibration using the DNL values. Instead of the edge of the bins, Wu proposed that the center of the bins should be corrected. Therefore, the INL after this correction is determined from the following:

$$INL(k)={\displaystyle \sum _{i=1}^{k-1}DNL(i)}+\frac{DNL(k)}{2}$$

(4)

Figure 5 shows the timing diagram of the online calibration. To improve precision and reduce power consumption, the calibration signal uses the hit instead of the ring oscillator. Firstly, the fine count associated with each binary code is retrieved and stored in the histogram through N hit events, which estimates the width of each bin. This step is also known as a code density test. After the histogram is established, the data are sent to the lookup table for bin-by-bin calibration. After the lookup table is built completely, the system starts measuring the hit events and the following hit events are corrected with the lookup table.

When the temperature changes, some parameters of the transistor will change, such as leakage current and carrier migration speed. The threshold voltage of the transistor decreases linearly with the temperature. The influence of power fluctuation can be ignored due to the low dropout regulator (LDO) providing low noise power for the FPGA core. Therefore, in the actual circuit, the voltage has little impact on the delay time. However, it is difficult to reduce the impact of temperature on the FPGA in many applications. It is necessary to calibrate the TDC online without affecting normal operation. A novel method, which consists in using a frequency counter to measure the ring oscillator frequency, is adopted to extrapolate and compensate its effect on the delay line. In this work, a ring oscillator based on 31 NOT gates and an AND gate is placed adjacent to the tapped delay line as shown in Figure 6. While the system is working, the ring oscillator frequency is measured and compared with that acquired at the beginning of the measurements. In practice, the compensation can be performed by scaling the lookup table coefficients T(i)_on-line evaluated in the bin-by-bin calibration phase as follows:

$$T{(i)}_{online}=\frac{f_{bybline}}{f_{online}}\times T{(i)}_{bybline}$$

(5)

where f_bybline is the frequency counter of bin-by-bin calibration and f_online is the frequency counter of online calibration.

3. Results

The proposed TDC was synthesized, implemented, and generated bitstream using Xilinx Vivado 2018.3, implemented on the ZYNQ-7020 FPGA (xc7z020clg400-2) as Figure 7 shows. The controllable time intervals between channel 1 and channel 2 were generated by utilizing the IDELAYCTRL and IDELAY2 primitive. Owing to the two signals generated inside the ZYNQ-7020, measurement errors were minimized. The performances of the DNL, the INL, and the precision of the TDC and sample rate are evaluated in this section.

Subsequently, the code density test was performed with 1,024,000 hit signals, which was clear from the histogram that the delay elements were not uniform (Figure 8a). The bin width of some elements was almost zero, whereas others were very large (even close to 90 ps), but most were below 40 ps. However, the bin width is more uniform after bin-by-bin calibration, with the max value of bin width of 46 ps, and with many around 17 ps as shown in Figure 8b. Only 165 elements were valid; therefore, 165 elements correspond to a clock cycle. Therefore, the least significant bit (LSB) = 2.857 ns/165 = 17.4 ps.

The DNL reached [−0.97, 4.12] LSB before calibration, and reached [−0.90, 1.67] LSB after bin-by-bin calibration (see Figure 9a). The INL reached [−2.45, 9.54] LSB before calibration, and reached [−1.90, 3.31] LSB after bin-by-bin calibration (see Figure 9b).

To further evaluate the performance of the TDC precision, the FPGA is used for testing standard deviation of the repeated 102,400 measurements. Figure 10a shows that the precision is 58 ps before bin-by-bin calibration when choosing the time interval of 0 ns. The precision is 18 ps after calibration when choosing the time interval of 0 ns, as shown in Figure 10b. Given the different time intervals ranging from 0 to 4.8 ns, the precision is below 19 ps, as shown in Figure 11.

Figure 12 shows the change of frequency count with temperature. It can be seen that the frequency count value decreases linearly with the increase of temperature, about 4 °C. In order to estimate the change of RMS precision with temperature, the TDC online temperature compensation circuit is tested from 30 °C to 70 °C, as shown in Figure 13. Figure 13 shows that the TDC RMS precision without calibration increases linearly with the increase of temperature (about 0.6 ps/°C), whereas the TDC with calibration circuit remains below 19 ps.

Dead time is also an important parameter influencing the sample rate. Different frequency signals entered into the TDC for testing the dead time. The precision is greatly reduced once the frequency is greater than 170 MHz as shown in Figure 14. The safest time interval is 5.882 ns, meaning the dead time is less than 3 clock cycles. More than one clock cycle is utilized to propagate the signal along the delay line, and then another clock cycle is used to sample the fine count by the D flip-flops.

Table 3. The timing of the TDC system.

Component	Worst Negative Slack
Setup	0.348 ns
Hold	0.068 ns

shows the timing of the TDC system. These data are obtained from the implementation timing report. The worst negative slack of setup is 0.348 ns and the worst negative slack of hold is 0.068 ns. The slack of both the hold and setup is positive, which means that the TDC meets the timing requirements.

Table 4. The resource utilization of the TDC system.

Module	LUTs (53200)	FFs (106400)	BRAMs (140)
TDC core	522	1080	4
One channel TDC system	1551	2313	4
Two channel TDC system	2513	4095	8

shows the resource utilization of the TDC system. These data are obtained from the implementation utilization report. The total numbers of LUTs are 53,200, FFs are 106,400, and BRAMs are 140 in the ZYNQ-7020 chip. The TDC core requires 522 LUTs, 1080 FFs, and 4 BRAMs. This TDC core needs many resources because it works at 350 MHz.

Table 5. The power of the TDC system.

Component	Power
Total dynamic	1.601 W
Clocks dynamic	1.399 W
Logic dynamic	0.015 W
BRAM dynamic	0.014 W
MMCM dynamic	0.123 W
PS	1.399 W
Total static	0.138 W

shows the power of the TDC system. These data are obtained from the implementation power report. The total dynamic power is 1.601 W and the main dynamic power comes from the PS; therefore, the dynamic power of the TDC is 0.202 W. The total static power is 0.138 W, which comes from the TDC. In other words, the total power consumption of the TDC is 0.340 W.

Table 6. Comparison with the FPGA-based TDL-TDCs.

Ref	Used Method	FPGA	LSB [ps]	Precision [ps]	DNL [LSB]	INL [LSB]	Sample Rate [Ms/s]	Resources	Power
2018 [30]	TDL	Cyclone-IV	45	18	[−0.5, 0.13]	[−0.48, 0.37]	75	-	-
2018 [3]	TDL	Kintex-7	3	5.76	-	[−9, 9]	45.5	-	-
2018 [22]	TDL	Spartan-6	25.6	37	[−0.90, 1.23]	[−0.0.43, 2.96]	115	415 Slices	0.131 W
2021 [25]	TDL	Artix-7	22.2	26.04	[−0.95, 1.19]	[−2.75, 1.24]	125	216 LUTs	0.164 W
This work	TDL	ZYNQ-7020	17.4	19	[−0.90, 1.67]	[−1.90, 3.31]	170	522 LUTs	0.340 W

provides a comparison with other TDCs based on similar principles. In terms of the precision, this paper is better than reference 22 and 25. For the sample rate, this paper is better than all references: a 19 ps precision and 170 M samples/s TDC implemented and evaluated in Zynq-7000 (programmable logic is equivalent to a 28 nm Artix-7 FPGA). The proposed TDC offers higher precision and a higher sample rate than that TDC from reference 25. In addition, the sample rate is verified by the experiment.

4. Conclusions

In summary, a high precision and high speed TDC implemented with online calibration based on the ZYNQ-7020 was designed and tested. A ones-counter encoder was employed to convert the thermometer code with bubble errors into binary code. The dead time was further decreased by taking advantage of the pipelined adder tree structure. Bin-by-bin calibration was implemented in the FPGA to minimize the DNL values. In addition, an innovative online calibration based on a frequency counter for temperature compensation is proposed. Experimental results demonstrated the sampling rate was 170 M and the LSB was 17.4 ps. The DNL reached [−0.97, 4.12] LSB before calibration, and reached [−0.90, 1.67] LSB after bin-by-bin calibration. The INL reached [−2.45, 9.54] LSB before calibration, and reached [−1.90, 3.31] LSB after bin-by-bin calibration. Finally, the online calibration based on a ring oscillator is tested at different temperatures, showing the precision of the TDC can still be maintained at 19 ps from 30 °C to 70 °C. Compared with previously published works, the sample rate and precision has been significantly improved thanks to our ones-counter encoder circuit and online calibration circuit (shown in Table 6). These outstanding performances demonstrate the potential for application in a high counting rate and a high time resolution detector.