Estimating Word Lengths for Fixed-Point DSP Implementations Using Polynomial Chaos Expansions

Abstract

Efficient custom hardware motivates the use of fixed-point arithmetic in the implementation of digital signal-processing (DSP) algorithms. This conversion to finite precision arithmetic introduces quantization noise in the system, which affects the system’s performance. As a result, characterizing quantization noise and its effects within a DSP system is a challenge that must be addressed to avoid over-allocating hardware resources during implementation. Polynomial chaos expansion (PCE) is a method used to model uncertainty in engineering systems. Although it has been employed to analyze quantization effects in DSP systems, previous investigations have been limited in scope and scale. This paper introduces new techniques that allow the application of PCE to be scaled up to larger DSP blocks with many noise sources, as needed for building blocks in software-defined radios (SDRs). Design space exploration algorithms that leverage the accuracy of PCE to estimate bit widths for fixed-point implementations of DSP blocks in an SDR system are explored, and their advantages will be presented.

1. Introduction

Digital signal-processing (DSP) systems often require fixed-point implementations of signal-processing algorithms to satisfy increasingly stringent power, performance, and area requirements [1]. However, the conversion to fixed-point introduces quantization noise, which degrades performance. As a result, a significant portion of the development time for DSP systems is spent on fixed-point conversion and managing this quantization noise [2,3].

Software-defined radio (SDRs) is an emerging communications platform that serves as a good testbed for fixed-point DSP implementations, given the strict performance requirements imposed by modern communication protocols [4]. In this paper, we explore a method to estimate the word lengths of signals within an SDR system using polynomial chaos expansions (PCEs) [5].

Advances in digital computing have resulted in many traditional signal-processing operations (performed using analog components) being moved into the realm of digital operations in a digital system. These digital systems are referred to as SDRs. Additionally, new algorithms have been developed that exploit the nature of discrete digital signals in order to extract significantly more information from a received signal [6]. However, the wide variety of wireless communication standards available has given rise to a new kind of challenge. Each wireless system requires its own digital system architecture with a highly specific mix of analog and digital components required to implement it. With the gradual slowing of Moore’s Law [7], it is becoming harder to justify single-use dedicated digital systems. Additionally, developing digital radio integrated circuits for every new wireless standard is vastly expensive and time-consuming [8]. Therefore, digital systems that are flexible and reconfigurable are being sought after.

An SDR is a digital system capable of reconfiguring its hardware resources to perform a different set of signal-processing operations from those for which it was initially configured. In doing so, the same SDR hardware is capable of supporting multiple sets of wireless standards. HackRF is an example of an amateur SDR. With a simple software update, HackRF can change its operation from being a GPS receiver [9] to an FM receiver [10].

Additionally, a commercial grade SDR, such as the Universal Software Radio Peripheral (USRP) B210, can operate as an entire base transceiver station (BTS) through the OpenBTS standard [11]. This significantly reduces the cost of bringing Global System for Mobile Communication (GSM) cellular networks to remote regions and underdeveloped regions. In [12], such an experiment was performed to successfully bring GSM access to rural Zambia, with some performance and logistic constraints. Another experiment was performed in [13], where early warning systems were introduced to a rural Colombian coffee plantation by using a USRP device along with OpenBTS to provide support for existing early warning systems.

This paper demonstrates how SDR implementations can be more efficient by converting floating-point designs [14] to fixed-point designs [15]. In particular, we present methods that characterize these quantization errors, as well as algorithms and techniques that demonstrate a good trade-off between quantization effects and resource usage. In Section 2, related work on the importance of modeling quantization effects is presented as well as current methods to do so. In Section 3, a review of the fundamentals of PCE is presented for the sake of completeness. Section 4 describes the methods used for applying PCE to model DSP systems. Section 6 presents some design space exploration algorithms that achieve a good trade-off between quantization effects and resource usage. Lastly, Section 8 quantitatively shows how PCE outperforms currently used methods as well as results on estimating word lengths using PCE.

2. Related Work

Field programmable gate arrays (FPGAs) and DSP processors can be utilized to meet latency and throughput requirements. The difference between FPGAs and DSP processors mainly comes down to the level of flexibility required to implement a design. FPGAs give the designer an unparalleled level of flexibility to fine-tune the design, but there will be some significant development time. To alleviate this issue, tools such as the HDL Coder by MathWorks exist to generate synthesizable hardware description language (HDL) code from MATLAB functions and Simulink models [16].

In addition to the platform itself, a designer needs to decide between using floating-point and fixed-point arithmetic [15]. In order to represent the infinite precision of real numbers using the finite precision of digital values, floating-point or fixed-point numbers need to be used. Fixed-point numbers have the radix point set at a fixed location; in other words, they have a set number of decimal points. Floating-point numbers set the radix point according to the number itself. For example, representing a large number using a floating-point format would require more bits to be allocated to the integer part of the number instead of the fractional part. On the other hand, representing a small number with the same number of bits would allocate more bits to the fractional part available. As a result, floating-point numbers represent large numbers with low precision and small numbers with high precision. Due to a variable radix point in floating-point numbers, the arithmetic requires more hardware resources and computation time. In [17], the authors showed a performance improvement of 27.17% when using a fixed-point unit, even with a hardened floating-point unit on an Intel Arria 10 FPGA. As a result, a trade-off exists where a designer needs to decide between being able to represent calculations accurately or quickly.

2.1. Quantization Effects

Quantization effects arise when attempting to represent infinite precision real-world signals with finite precision representations in the digital domain. The quantization noise is the difference between the real signal and the quantized signal. This noise depends on how the quantization itself is performed. To quantize a real signal, the signal is multiplied by a scale factor and then either rounded or truncated to a set number of bits. This truncation can occur by simply discarding the fractional portion after multiplication, or by considering the fractional portion and rounding the number to the closest representable value [18]. Rounding quantizers generate less quantization noise than truncating quantizers, but this comes at the cost of increased resource requirements.

Quantization noise and round-off noise propagate through a signal-processing system and significantly affect the maximum possible system performance [19]. As a result, these quantization effects need to be modeled as they propagate through an SDR signal-processing chain to ensure the radio system can operate according to its specifications.

2.2. Importance of Modeling Quantization Effects

Quantization effects in linear time-invariant (LTI) systems, such as finite impulse response (FIR) filters, Fourier transforms, etc., have been part of an active and long-standing area of research. This is largely motivated by the fact that implementing a design within a fixed-point implementation is reported to take up to 25% to 50% of the total design time [2,3]. A lot of this research has been in modeling or reducing the quantization effects for a particular block in a DSP system.

Due to the abundance of digital filters in a DSP system, quantization noise modeling for filters is a recurring topic [20,21,22,23,24]. Within a fixed-point implementation of a digital filter, the three sources of quantization error are due to the following:

• Errors arising from representing infinite-precision filter coefficients with finite-precision coefficients.
• Round-off errors due to the multiplication of two fixed-point numbers being quantized into fewer bits than required.
• The quantization of the input signal into discrete levels.

In [25], the effects of coefficient quantization are shown. A 44th-order low-pass FIR filter was designed using the Parks–McClellan algorithm [26] with a cutoff of $\pi /4$. The resulting double-precision floating-point numbers are quantized to 16 and 12 bits. The original filter had a stop band attenuation of −92 dB. After quantizing into 16 bits, the resulting filter had an attenuation of −83 dB, showing a 9 dB loss in attenuation. When the original filter was quantized into 12 bits, the effect was even more pronounced, and the attenuation was about −60 dB, showing a 32 dB loss in attenuation. From this, the importance of choosing the correct fixed-point configuration for all the signals in the DSP system is highlighted, motivating the need for a framework that can provide an accurate estimation of quantization effects.

2.3. Current Methods in Modeling Quantization Effects

Quantization noise modeling can be performed using a simulation-based approach, an analytical approach, or a combination of both [27]. Traditionally quantization noise effects have been modeled by a very large number of Monte Carlo (MC) simulations. These simulations compare the difference in statistics between an unquantized floating-point signal and a quantized signal. Such an approach is prohibitively expensive both from a time and computational perspective for large SDR systems [19]. Additionally, this approach does not allow for optimizing bit widths to minimize the round-off effects.

The analytical approach involves developing a mathematical model to understand how signal-processing operations impact the statistics of round-off noise at each node in the signal-processing chain. In [28], a noise analysis of a phase-locked loop (PLL) was demonstrated using stochastic differential equations in the presence of white noise sources. It was shown that this approach yields results that cannot be estimated by a traditional linear analysis. The downside to such an approach is that it is a tedious process that is highly problem-specific.

In [29], a computational method based on the satisfiability modulo theory (SMT) was presented, which captures a calculation as a set of constraints. In doing so, an SMT solver can present meaningful bounds on the calculation. This work can be applied here by setting a noise threshold at the nodes in the signal-processing graph and then using the SMT solver to check if an equation for the noise at that node exceeds the set threshold. Results from the solver could be a part of an optimization process to find a set of bit widths that minimize quantization effects. However, this approach is not applicable when there are feedback loops involved. Due to the number of simulation time steps required for the feedback loops to converge, the state space of the SMT solver would blow up to a scale that cannot be solved in a reasonable amount of time.

Another approach is to bind the limits of quantization noise and propagate the bounds through a data flow graph (DFG). Interval arithmetic (IA) can be used to write an expression for the noise range [30]. As an example, consider a measurement that has a value of 21 and an uncertainty in measurements of ±3. In the IA notation, this value would be represented as [18, 24]. The issue with using IA is the so-called dependency problem. To illustrate this issue, consider that there exists a value x = [−1, 1]. We would like to define a new value $y=x-x$. According to the rules of IA, $[a,b]-[c,d]=[a-d,b-c]$. This implies that $y=[-1,1]-[-1,1]=[-2,+2]$. Although the answer is obviously $y=0$, IA has a bound that is twice that of the input bounds. This overestimation can propagate through computations until the final error bounds are so large they lose all information about the dependencies in the graph.

To resolve the dependency problem, affine arithmetic (AA) is proposed as a solution. In AA, a quantity, x, is represented as $\widehat{x}=x_0+x_1{ϵ}_1+\dots +x_n{ϵ}_n,$ with ${ϵ}_i\in [-1,+1]$. Here, $x_0$ denotes the central value and $x_i,\dots ,x_n$ denote partial deviations, and ${ϵ}_i,\dots ,{ϵ}_n$ denote noise symbols [30]. A measurement of 21 with an uncertainty of ±3 would be represented in AA as $21+3{ϵ}_1$. Under the rules of AA, the dependency problem is resolved, and $y=x-x$ yields $y=0$. AA is often used for bit width estimation, such as in [31,32,33].

Reference [27] provided a starting point for this paper. Here, an approach was proposed where round-off noise was modeled using AA and propagated through a DFG. Each quantizer in the DFG was modeled as a uniform noise source with a specific mean and variance. The bounds of the uniform distribution were determined from the mean and variance, and, an AA representation of the bounds was propagated through the DFG.

Most of the current methods used in estimating bit widths have used either AA, IA, or MC methods to model the quantization noise. Due to the inability of AA and IA to represent non-linear operations accurately, this results in a significant overestimation of the bit width required. On the other hand, with MC techniques, it is possible to accurately represent quantization noise, but this comes at a significant cost in the number of operations required to run a simulation. In Section 8, it is shown that PCE can provide a comparable level of accuracy to MC, but at a much lower number of operations. While PCE has been used in the literature to explore the trade-off between the signal-to-noise ratio (SNR) and bit widths [34], it has been done from the perspective of modeling overflow errors. In this paper, we focus on modeling quantization noise as it arises from round-off errors and its propagation in fixed-point arithmetic.

3. Elementary Operations Using PCE

Converting floating-point architecture into fixed-point architecture introduces quantization noise; properties of the quantization noise can be captured using statistical distributions. PCE is then used to represent the distributions and model the interactions between different quantization noise sources within a system. In this section, a review of the fundamentals behind PCE is presented, which is also available in the literature and replicated here for the sake of completeness.

3.1. Probability Distribution of Quantization Noise

Quantization noise occurs whenever a signal requiring a certain level of precision is represented using a number system with lower precision. This can happen when a real-world input signal with infinite precision is quantized into a fixed-point representation with finite precision. Additionally, quantization noise can also take the form of round-off noise, where the result of an arithmetic operation is rounded off to the closest representable value. For example, round-off noise can arise when the multiplication of two n-bit signals is quantized and stored in another n-bit register instead of in a $2n$-bit register.

In [27], it was shown that quantization noise can be modeled using a random variable with a uniform probability distribution. The statistics of the uniform noise source depend on the type of quantizer used. Quantizers can either use rounding or truncation to approximate a signal. Assume that x is a real-valued input that is quantized to Q bits to produce the fixed-point number $x_Q$. A rounding quantizer uses Equation (1) to round the input to the nearest integer, while a truncation quantizer uses Equation (2) to truncate the input to an integer. Note that $⌊·⌋$ represents the floor function. The truncation quantizer is the simplest to implement, however, it introduces a bias to the result, whereas the rounding quantizer requires more hardware resources but introduces no bias.

$$\begin{array}{c}{x}_Q=\mathrm{round}(x·2^Q)\end{array}$$

(1)

$$\begin{array}{c}{x}_Q=⌊x·2^Q⌋\end{array}$$

(2)

According to [27], the mean and variance of the quantization noise for a truncation quantizer are given by Equations (3) and (4), while those for a rounding quantizer are given by Equations (5) and (6). In these equations, $f_w$ denotes the number of bits required to represent a signal without any loss of precision, and $f_{w_Q}$ denotes the number of bits chosen to represent the signal, as follows:

$$\begin{array}{c}{m}_{\mathrm{trunc}}=-{\displaystyle \frac{1}{2}}\left(2^{-f_{w_Q}}-2^{-f_w}\right)\end{array}$$

(3)

$$\begin{array}{c}{{\sigma }^2}_{\mathrm{trunc}}={\displaystyle \frac{1}{12}}\left(2^{-2f_{w_Q}}-2^{-2f_w}\right)\end{array}$$

(4)

$$\begin{array}{c}{m}_{round}=0\end{array}$$

(5)

$$\begin{array}{c}{{\sigma }^2}_{\mathrm{trunc}}={\displaystyle \frac{1}{12}}\left(2^{-2f_{w_Q}}-2^{-2f_w+1}\right)\end{array}$$

(6)

Using the mean and the variance, it is possible to find the interval for the uniform distribution $[a,b]$ according to Equations (7) and (8). Note that for Equations (7) and (8), $m_{\mathrm{trunc}}$ or $m_{round}$ is used in place of m, and ${{\sigma }^2}_{trunc}$ or ${{\sigma }^2}_{round}$ is used in place of ${\sigma }^2$, depending on whether a rounding or truncation quantizer is used:

$$\begin{array}{c}a=m-\sqrt{3{\sigma }^2}\end{array}$$

(7)

$$\begin{array}{c}b=m+\sqrt{3{\sigma }^2}\end{array}$$

(8)

As an example, consider 100,000 samples taken from a Uniform[0, 1) distribution, also denoted as a $\mathcal{U}[0,1)$ distribution. Here, the floating-point samples can be considered infinite-precision; thus, $f_w=\infty$. This signal is then quantized into four bits. If a truncation quantizer is used, then according to Equations (3), (4), (7), and (8), the noise distribution is $\mathcal{U}[-0.0625,0]$. The results are shown in Figure 1. The red line represents the predicted distribution while the blue bars show the histogram from the MC simulation.

3.2. Polynomial Chaos Expansion (PCE)

PCE is a spectral projection method in which a random variable is decomposed into a sum of orthogonal polynomial basis functions of additional random variables. The maximum degree of a polynomial within the basis set is denoted by d. One way to think about PCE is as a Fourier analysis for random variables [35]. If the random variable, z, is square-integrable, i.e., $E\left[\right|z{|}^2]$ is finite, then the random variable can be decomposed via PCE over a set of basis functions [34]. Also, if the random variable being decomposed is smooth, the error in the expansion exponentially decays to zero with the addition of more terms in the decomposed function [5]:

$$z=\sum _{i=1}^{\infty }{z}_i{P}_i\left({\zeta }_1,{\zeta }_2,\dots ,{\zeta }_n\right)$$

(9)

The idea behind PCE is shown using Equation (9). Here, the random variable z is decomposed into PCE coefficients, $z_i$, and the orthogonal PCE basis functions, $P_i\left({\zeta }_1,\dots ,{\zeta }_n\right)$. Here, ${\zeta }_1$ through ${\zeta }_n$ is an independent standard random variable. The distribution choice for ${\zeta }_n$ depends on how z is distributed. If z is uniformly distributed, it would make sense for ${\zeta }_n$ to also denote a standard uniform distribution. On the other hand, z might be a random variable modeling a more complicated distribution, in which case, it might work better to have each ${\zeta }_n$ have different distributions. The $\zeta$ distribution choice influences how the polynomials, $P_i$, are constructed:

$$z\approx \sum _{i=1}^m{z}_i{P}_i\left({\zeta }_1,{\zeta }_2,\dots ,{\zeta }_n\right)$$

(10)

$$m=d+1$$

(11)

Equation (9) shows that in order to decompose a random variable, an infinite set of polynomial basis functions needs to be constructed. The goal is to simulate a system using PCE via a computer program; thus, an infinite set of coefficients and polynomials is not possible to keep track of. As a result, the maximum number of polynomials is truncated to m in Equation (10). This comes at a cost of a loss in accuracy between the original distribution z, and its PCE representation. The number of polynomials, m, can be related to the degree of the polynomial, d, according to Equation (11). The accuracy of the system model increases as d increases:

$$z=\sum _{i=1}^{\infty }{z}_i{P}_i\left(\zeta \right)$$

(12)

As an example, assume that z is a uniform distribution, i.e., z∼$\mathcal{U}[a,b]$. Thus, z can be decomposed into one random variable with a uniform distribution, i.e., $\zeta$∼$\mathcal{U}[-1,1]$. This is represented in Equation (12). Now, a basis set of orthogonal polynomials is required to support $\zeta$. Many systems of orthogonal polynomials exist, and the issue arises on which set to choose. In [36], polynomial chaos expansions were generalized to accommodate various types of probability distributions, and it was proven which orthogonal polynomial systems correspond to specific random variables. This is reproduced in

Table 1. Distributions and their associated polynomials [36].

Distribution	Basis Functions for PCE
Uniform	Legendre polynomials
Gaussian	Hermite polynomials
Beta	Jacobi polynomials
Gamma	Laguerre polynomials
Poisson	Charlier polynomials
Negative binomial	Meixner polynomials
Binomial	Krawtchouk polynomials
Hypergeometric	Hahn polynomials

. For this simple example, Table 1 shows that the Legendre system of orthogonal polynomials should be selected in order to support $\zeta$ since $\zeta$ is uniformly distributed.

3.3. Non-Standard Distributions

So far, all the distributions are assumed to be standard, i.e., $\mathcal{U}[-1,1]$ or $\mathcal{N}(0,1)$. As described in Section 3.1, the uniform distribution modeling the statistics of quantization noise has a support that is defined using Equations (7) and (8), rather than over the interval $[-1,1]$. The construction of Legendre polynomials ensures orthogonality over the support $[-1,1]$, but not over a smaller subset as required for the quantization noise model. Similarly, Hermite polynomials are orthogonal for a standard normal distribution, but not for distributions that have a different mean and/or variance. As a result, the construction of polynomials needs to be modified to represent non-standard distributions.

According to [5], a random variable, such as x∼$\mathcal{U}[a,b]$, can be mapped to a standard uniform random variable, $\zeta$, using Equation (13):

$$x={\displaystyle \frac{a+b}{2}}+{\displaystyle \frac{b-a}{2}}\zeta$$

(13)

Equation (13) implies that a non-standard uniform random variable, x∼$\mathcal{U}[a,b]$, can still be supported by a basis set of standard uniform random variables via a change in PCE coefficients, as shown in Equation (14).

$$x=\sum _{i=1}^{\infty }{x}_i{P}_i\left(\zeta \right),x_i=\left\{\begin{array}{cc}{\displaystyle \frac{a+b}{2}},\hfill & \mathrm{if}i=0\hfill \\ {\displaystyle \frac{b-a}{2}},\hfill & \mathrm{if}i=1\hfill \\ 0\hfill & \mathrm{otherwise}\hfill \end{array}\right.$$

(14)

Similarly, the authors of [5] show that a non-standard normal random variable, x∼$\mathcal{N}(\mu ,{\sigma }^2)$, can be mapped to a standard normal random variable, $\zeta$, using Equation (15):

$$x=\mu +\sigma \zeta$$

(15)

Like in the case of the non-standard uniform distribution, a non-standard normal random variable, x∼$\mathcal{N}(0,1)$, can still be supported by a basis set of standard normal random variables via a change in PCE coefficients, as shown in Equation (16):

$$x=\sum _{i=1}^{\infty }{x}_i{P}_i\left(\zeta \right),x_i=\left\{\begin{array}{cc}\mu ,\hfill & \mathrm{if}i=0\hfill \\ \sqrt{{\sigma }^2},\hfill & \mathrm{if}i=1\hfill \\ 0\hfill & \mathrm{otherwise}\hfill \end{array}\right.$$

(16)

3.4. Multivariate Basis Polynomials

To decompose an input random variable (z) into multiple random variables (${\zeta }_1,{\zeta }_2,\dots$), a multivariate basis set of polynomials needs to be constructed. This is done by taking the tensor product of the individual polynomial sets. Each random variable is supported by its own basis set of polynomials up to a degree, d. The tensor product is obtained by conducting an element-by-element multiplication of the two sets. Note that the sizes of the two basis sets do not have to be equal. In order words, one random variable can be supported using a polynomial basis set of up to degree 3, while another random variable can be supported using a set of up to degree 4, and the multivariate basis set can still be calculated.

As an example, consider two standard uniform random variables: ${\zeta }_1$ and ${\zeta }_2$. Each random variable is represented using its polynomial basis set, limited up to degree $d=2$. For ${\zeta }_1$, this is set A, as shown in Equation (17), and for ${\zeta }_2$, this is set B, as shown in Equation (18).

$$\begin{array}{c}A=\left\{1,{\zeta }_1,{\displaystyle \frac{1}{2}}\left(3{{\zeta }_1}^2-1\right)\right\}\end{array}$$

(17)

$$\begin{array}{c}B=\left\{1,{\zeta }_2,{\displaystyle \frac{1}{2}}\left(3{{\zeta }_2}^2-1\right)\right\}\end{array}$$

(18)

To capture the interactions between these two random variables, the multivariate Legendre polynomials basis set $\mathbf{P}$ is constructed according to Equation (19). Here, ⊗ represents the tensor product operator, as follows:

$$\mathbf{P}=A\otimes B$$

(19)

The resulting polynomials are as follows:

$$\begin{gathered} P_0=1·1 \\ {P}_1=1·{\zeta }_2 \\ {P}_2=1·{\displaystyle \frac{1}{2}}\left(3{{\zeta }_2}^2-1\right) \\ {P}_3={\zeta }_1·1 \\ {P}_4={\zeta }_1·{\zeta }_2 \\ {P}_5={\zeta }_1·{\displaystyle \frac{1}{2}}\left(3{{\zeta }_2}^2-1\right) \\ {P}_6={\displaystyle \frac{1}{2}}\left(3{{\zeta }_1}^2-1\right)·1 \\ {P}_7={\displaystyle \frac{1}{2}}\left(3{{\zeta }_1}^2-1\right)·{\zeta }_2 \\ {P}_8={\displaystyle \frac{1}{2}}\left(3{{\zeta }_1}^2-1\right)·{\displaystyle \frac{1}{2}}\left(3{{\zeta }_2}^2-1\right) \end{gathered}$$

In this example, the maximum degree was chosen to be 2. As a result, $P_5$, $P_7$, and $P_8$ are discarded. The final basis polynomial set is given in Equation (20), as follows:

$$\mathbf{P}=\left\{\begin{array}{c}{P}_0({\zeta }_1,{\zeta }_2)=1,\\ P_1({\zeta }_1,{\zeta }_2)={\zeta }_2,\\ P_2({\zeta }_1,{\zeta }_2)={\displaystyle \frac{1}{2}}\left(3{{\zeta }_2}^2-1\right),\\ P_3({\zeta }_1,{\zeta }_2)={\zeta }_1,\\ P_4({\zeta }_1,{\zeta }_2)={\zeta }_1·{\zeta }_2,\\ P_5({\zeta }_1,{\zeta }_2)={\displaystyle \frac{1}{2}}\left(3{{\zeta }_1}^2-1\right)\end{array}\right\}$$

(20)

Constraining the maximum degree to a fixed value allows for a finite polynomial basis set. The error in the expansion decays exponentially with more basis functions [5] when the underlying distribution is smooth. However, the computational complexity increases as the number of polynomial basis functions increases. For the systems analyzed in this work, a second-order expansion is sufficient.

The total number of polynomials for n random variables represented by polynomials up to degree d is given by Equation (21). Note that the m in Equation (21) refers to the m in Equation (10). Lastly, Equation (21) shows one of the main drawbacks of this method, in that, the method suffers from the curse of dimensionality as the number of random variables increases:

$$m={\displaystyle \frac{(n+d)!}{n!·d!}}-1$$

(21)

This method also extends to cases where the random variables follow different distributions. For example, consider the two random variables: ${\zeta }_1$∼$\mathcal{U}[-1,1]$ and ${\zeta }_2$∼$\mathcal{N}(0,1)$. The resulting basis polynomial set would be the tensor product of the respective Legendre and Hermite polynomial sets.

3.5. Elementary Operations on PCE

Polynomial chaos expansions offer a way to decompose a random variable into a set of coefficients and an associated basis polynomial set. The goal now is to understand how quantization noise, modeled as a probability distribution, is affected by elementary signal-processing operations. Therefore, it is now necessary to define how arithmetic operations are carried out on PCEs as they propagate through the elementary operations in a DSP system. The elementary operations that comprise DSP systems are listed as follows:

• Scaling.
• Addition/subtraction.
• Multiplication.
• Time delay.

For all the operations below, $P_i$ is a polynomial in $\mathbf{P}$, and $\mathbf{P}$ is constructed as the tensor product between all the individual sets of basis polynomials, as described in the previous section.

3.5.1. Scaling

Figure 2 shows a scaling (multiplication by a constant) operation. Here, $\alpha$ is a constant, and x is given as follows:

$$x=\sum _{i=1}^m{x}_i·P_i$$

(22)

The output y is calculated as follows:

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill y& =\sum _{i=1}^m{y}_i·P_i\hfill \\ \hfill & =\alpha \sum _{i=1}^m{x}_i·P_i\hfill \\ \hfill & =\sum _{i=1}^m\alpha x_i·P_i\hfill \end{array}\end{array}$$

(23)

3.5.2. Addition/Subtraction

Figure 3 shows an addition/subtraction operation. Here, a and b are inputs to the node and are given by Equations (24) and (25), respectively.

$$a=\sum _{i=1}^m{a}_i·P_i$$

(24)

$$b=\sum _{i=1}^m{b}_i·P_i$$

(25)

The output y is then given by the following:

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill y& =\sum _{i=1}^m{y}_i·P_i\hfill \\ \hfill & =\sum _{i=1}^m\left(a_i·P_i\right)\pm \sum _{i=1}^m\left(b_i·P_i\right)\hfill \\ \hfill & =\sum _{i=1}^m\left(a_i\pm b_i\right)·P_i\hfill \end{array}\end{array}$$

(26)

3.5.3. Multiplication

Multiplication of PCEs is more ‘involved’ compared to the other operations. Figure 4 shows a multiplication operation. Here, the objective is to find the coefficients $y_i$ of output y such that we have the following:

$$\sum _{i=1}^m{y}_i{P}_i=\left(\sum _{j=1}^m{a}_j{P}_j\right)·\left(\sum _{k=1}^m{b}_k{P}_k\right)$$

(27)

According to [34], the coefficients of $y_i$ are given by Equation (28).

$$y_i={\displaystyle \frac{1}{E\left[{P_i}^2\right]}}\sum _{j,k}{a}_j{b}_{k}E\left[P_i·P_j·P_k\right]$$

(28)

Since all the polynomials $(P_i,P_j,P_k)$ are functions of independent random variables, the expectations in Equation (28) can be calculated quite easily. For a standard uniform random variable, $\zeta$∼$\mathcal{U}[-1,1]$, the n-th raw moment is defined in Equation (29). Additionally, for a standard normal random variable, $\zeta$∼$\mathcal{U}[-1,1]$, the n-th raw moment is defined in Equation (30) as follows:

$$E\left[{\zeta }^n\right]={\displaystyle \frac{{(-1)}^n+1}{2(n+1)}}$$

(29)

$$E\left[{\zeta }^n\right]=\left\{\begin{array}{cc}0\hfill & \mathrm{if}n\mathrm{is}\mathrm{odd}\hfill \\ {\displaystyle \frac{n!}{2^{\frac{n}{2}}n!}}\hfill & \mathrm{if}n\mathrm{is}\mathrm{even}\hfill \end{array}\right.$$

(30)

Due to the orthogonality of the Legendre polynomials, matrices containing these three-way expectations are quite sparse. The goal is to eventually be able to simulate larger systems using PCE. Once these expectation matrices are calculated, they can be reused for multiple simulations of the same system, as will be elaborated on in Section 5.

3.5.4. Time Delay

Time delay nodes (Figure 5) are found in many DSP systems, so it is necessary to show how PCE can be applied to a time delay operation.

For systems with time delays, PCEs can be calculated per simulation time step. The input x is written as follows:

$$x\left[n\right]=\sum _{i=1}^m{x}_i\left[n\right]·P_i$$

(31)

For a delay of k steps, the output y is given as follows:

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill y\left[n\right]& =x[n-k]\hfill \\ \hfill & =\sum _{i=1}^m{x}_i[n-k]·P_i\hfill \end{array}\end{array}$$

(32)

4. PCE for DSP Blocks

DSP systems are composed of signal-processing blocks, which are, in turn, made of numerous elementary signal-processing operations. Previously, it was shown that PCE can be applied to capture how elementary signal-processing operations affect probability distributions of quantization noise. This section demonstrates how PCE can be applied to DSP systems using the following steps:

• Represent the DSP block as a DFG.
• Model the input signal to the DSP block using an array of PCE coefficients.
• Propagate the PCE coefficient array through DFG.
• Add quantization noise sources to DFG corresponding to a particular bit width configuration.
• Propagate the signal-and-noise PCE coefficient arrays through DFG.
• Remove the signal distribution at the output to obtain statistics on the noise distribution.

A Simple Example

The purpose of this example is to show how applications of PCE can be extended from elementary signal operations to larger signal-processing blocks. Consider the DFG representation of a simple DSP block in Figure 6. The inputs to the system are signal a, which has a uniform distribution, and b, which has a normal distribution. Additionally, there is also a constant, c0, with a value of 0.2. The output is the signal out, and there are three intermediate signals (add, mult0, and mult1).

There are two inputs; the simulation was conducted with a max polynomial degree of 2. Equation (21) with $n=2$ and $d=2$, shows that six polynomials are required to form the basis set for this system. Calculating the tensor product of the individual basis polynomial sets and discarding the polynomials that exceed the maximum polynomial degree of 2, the basis set in Equation (33) is obtained:

$$\mathbf{P}=\left\{\begin{array}{c}{P}_0({\zeta }_1,{\zeta }_2)=1,\\ P_1({\zeta }_1,{\zeta }_2)={\zeta }_1,\\ P_2({\zeta }_1,{\zeta }_2)={\zeta }_2,\\ P_3({\zeta }_1,{\zeta }_2)={\displaystyle \frac{1}{2}}\left(3{{\zeta }_1}^2-1\right),\\ P_4({\zeta }_1,{\zeta }_2)={\zeta }_1{\zeta }_2,\\ P_5({\zeta }_1,{\zeta }_2)={{\zeta }_2}^2-1\end{array}\right\}$$

(33)

Equation (14) is used to calculate the PCE coefficients for the two input signals, namely, a and b (shown in Equation (34) and (35)). (Note that, although the polynomial chaos expansion is only exact for an expansion with infinite terms, the “equals” sign (=) is still used instead of the approximation (≈) in all subsequent expansions for notations).

$$\begin{array}{cc}\hfill \mathtt{a}& =\sum _{i=0}^5{a}_i{P}_i\hfill \\ \hfill & =\left(1\right)P_0+\left(4\right)P_1+\left(0\right)P_2+\left(0\right)P_3+\left(0\right)P_4+\left(0\right)P_5\hfill \end{array}$$

(34)

$$\begin{array}{cc}\hfill \mathtt{b}& =\sum _{i=0}^5{b}_i{P}_i\hfill \\ \hfill & =\left(0\right)P_0+\left(0\right)P_1+\left(1\right)P_2+\left(0\right)P_3+\left(0\right)P_4+\left(0\right)P_5\hfill \end{array}$$

(35)

These PCE coefficients can be represented as an array of values, as shown in (36) and (37), respectively.

$$\mathtt{a}=\left[\begin{array}{cccccc}1& 4& 0& 0& 0& 0\end{array}\right]$$

(36)

$$\mathtt{b}=\left[\begin{array}{cccccc}0& 0& 1& 0& 0& 0\end{array}\right]$$

(37)

In the array form, the i-th element is the PCE coefficient for the i-th basis polynomial. Henceforth, the polynomial chaos expansion of a signal will be represented as an array of coefficients. The PCE representation of a signal is the dot product between the vector containing its PCE coefficients and the vector containing the basis functions.

With the basis set and the PCE of the two input signals, the distributions at each of the nodes within the DFG can be calculated according to Section 3.5. The PCE coefficients of each of the nodes are given in

Table 2. PCE coefficients at each node of the DFG.

Node Label	PCE Coefficient
a	$\left[\begin{array}{cccccc}1& 4& 0& 0& 0& 0\end{array}\right]$
b	$\left[\begin{array}{cccccc}0& 0& 1& 0& 0& 0\end{array}\right]$
add	$\left[\begin{array}{cccccc}1& 4& 1& 0& 0& 0\end{array}\right]$
mult0	$\left[\begin{array}{cccccc}0.2& 0.8& 0.2& 0& 0& 0\end{array}\right]$
mult1	$\left[\begin{array}{cccccc}0.2& 0& 0.2& 0& 0.8& 0.2\end{array}\right]$
out	$\left[\begin{array}{cccccc}-0.8& -4& 0.2& 0& 0.8& 0.2\end{array}\right]$

.

The second moment of the output, out, is calculated as shown in Equation (38). This value is the signal power at the output of the DFG.

$$\begin{array}{cc}\hfill out& ={\left[\begin{array}{cccccc}-0.8& -4& 0.2& 0& 0.8& 0.2\end{array}\right]}^2·\left[\begin{array}{cccccc}E\left[{P_0}^2\right]& E\left[{P_1}^2\right]& E\left[{P_2}^2\right]& E\left[{P_3}^2\right]& E\left[{P_4}^2\right]& E\left[{P_5}^2\right]\end{array}\right]\hfill \end{array}$$

(38)

$$\begin{array}{cc}\hfill & ={\displaystyle \frac{473}{75}}\approx 6.3067\hfill \end{array}$$

(39)

The signal distribution described by the calculated output PCE coefficients is compared to an MC simulation of the system. Moreover, 100,000 samples are generated from their respective distributions of the inputs a and b and are propagated through the DFG in Figure 6. Additionally, 100,000 samples are generated from the calculated PCE representation of the output. The results are illustrated in Figure 7.

Obtaining the PCE coefficients of the quantization noise signal is done by adding noise nodes to the DFG, propagating the coefficients, and finally subtracting the signal distribution from the total distribution at the output. The new DFG with the noise nodes added (highlighted in gray) is shown in Figure 8.

Each noise injection node added to the DFG is another input that needs to be represented with its own random variable that captures the quantization effects at their respective nodes. The DFG now has six random variables, one for each of the inputs, and four for the noise sources. Adding four random variables for the noise nodes implies that the quantization noise induced at each node is uncorrelated with any other node. The polynomial basis set needs to be expanded to support these new random variables. With a maximum polynomial degree of 2, like before, according to Equation (21), the set of basis polynomials now contains 28 polynomials. The PCE coefficient arrays for each noise input can be calculated, similar to how the arrays for inputs a and b were calculated.

The new PCE representations are propagated through the DFG. The signal PCE coefficient array is subtracted from the resulting PCE array (representing the signal plus noise contribution). Taking the difference between the two arrays yields the noise distribution.

Note that the size of the basis set for the signal plus noise DFG is 28 while the signal PCE coefficient array has a size of 6. As a result, before subtracting the two arrays, the signal PCE coefficient array with size 6 needs to be remapped onto a size 28 coefficient array. The new basis set is a tensor product of the old basis set and the new one, and, assuming that both the basis sets are truncated to the same maximum degree, all the polynomials of the old basis set will appear in the new basis set. The remapping is conducted by placing the old coefficients into their new indices within the new basis set.

The resulting noise distributions from an MC simulation of the system and the PCE estimation of the noise distribution are shown in Figure 9. Due to the truncation of the PCE’s maximum polynomial degree, higher-order moments of the noise distribution, as seen by the MC simulation, are not captured by the PCE approximation. This is particularly evident in the peak at the center and the sharper drop-off at the edges in the MC simulation of the actual output noise distribution. The trade-off between accuracy and computational costs needs to be considered when conducting PCE simulations.

For this example, with the bit widths given in

Table 3. Signal bit width configuration.

Signal	Bit Width
a	5
b	5
mult0	3
mult1	2

, the PCE simulation estimates the quantization noise power to be 0.0063. Combined with the signal power, determined in Equation (38) to be 6.3067, the PCE simulation estimates the signal-to-noise ratio to be 29.78 dB. The MC simulation estimates the SNR to be 30.05 dB.

5. Design Space Exploration

The simple example presented in Section 4 shows how quantization noise effects for a given bit width can be analyzed via PCE. This flow is summarized in Algorithm 1.

Algorithm 1 PCE simulation for a given bit width configuration

Algorithm 1 returns an SNR estimate for a given bit width configuration. The goal is to try various signal bit widths and choose a configuration that gives the desired SNR. An exhaustive search through all possible configurations is not possible due to the large search space. Instead, the simulated annealing (SA) metaheuristic [37] is used to obtain solutions.

5.1. Using Simulated Annealing to Estimate Bit Widths

When applying SA, the goal is to identify a bit width configuration that introduces just enough quantization noise to achieve the desired SNR. Given the current bit width configuration, a better configuration would be one whose noise power is less than the maximum acceptable noise power, but, more than the noise power of the current best solution. The SA algorithm primarily tests a given bit width configuration by applying Algorithm 1, decides on whether to accept the trial configuration or not, and finally, identifies a new candidate configuration to test. This is shown in Algorithm 2.

Within Algorithm 2, the target SNR is given as SNR_tgt. Also, an initial solution for which the quantization noise exceeds SNR_tgt is given as sol_initial. This initial solution can be set to the size of a common standard data type as a starting point. The total number of simulated annealing iterations is given as SA_iters. Lastly, the DFG and the maximum polynomial degree are given as dfg and maxPolyDeg, respectively.

Algorithm 2 Simulated annealing

If a bit width configuration yields a higher noise power than the current best, the configuration is chosen since it implies a lower total number of bits. Otherwise, the solution is accepted with a probability, given the Probability() function. This function uses the current temperature (updated by a call to Temperature()) to initially accept worse solutions with the motivation that it might lead to a better solution down the line. The equation for the Probability() function is Equation (40); the cooling curve for a given initial temperature ($t_0$) and a total number of iterations ($\mathtt{SA}\mathtt{iters}$) is given in Equation (41):

$$P({\mathtt{SNR}}_{\mathtt{prop}},{\mathtt{SNR}}_{\mathtt{curr}},\mathtt{temp})=\exp \left(-{\displaystyle \frac{{\mathtt{SNR}}_{\mathtt{prop}}-{\mathtt{SNR}}_{\mathtt{curr}}}{\mathtt{temp}}}\right)$$

(40)

$$y=\left({\displaystyle \frac{t_0}{{{\mathtt{SA}}_{\mathtt{iters}}}^2}}\right)x^2-2\left({\displaystyle \frac{t_0}{{\mathtt{SA}}_{\mathtt{iters}}}}\right)x+t_0$$

(41)

Given a bit width configuration, new candidates to test for are given by the Neighbor() function. The set of neighbors is defined as the set of bit width configurations within a certain distance, $\Delta$, of the current solution. For example, if $\Delta$ is 1, then the set of neighbors includes all bit-width configurations where each signal’s bit width is at most one bit higher or lower than the current set of bit widths.

5.2. Improved Design Space Exploration Algorithm

After the first call to Algorithm 1 (line 3), repeated calls do not need to reinitialize the basis polynomials, modify the DFG, or reorder the signal PCE coefficients. Subsequent calls (line 8) can be made to TrySolution instead, as given in Algorithm 3.

Algorithm 3 TrySolution

5.3. Neighbor Selection

The Neighbor() function determines the set of potential solutions that can be explored from a given solution. Aggressive and non-aggressive neighbor selection algorithms are presented. The aggressive neighbor selection algorithm (Algorithm 4) always considers candidate configurations that have a lower total bit width than the current configuration. The non-aggressive neighbor selection algorithm (Algorithm 5) allows for an increase in the bit width 80% of the time. The rate (80%) is parameterizable and can be changed depending on the application.

Algorithms 4 and 5 are both heuristic methods used to obtain a solution. Choosing between the two algorithms depends on the trade-off between the resulting quality and the computation time available. When a single simulation takes a long time, it might not be possible to iterate over many undesirable solutions due to time constraints. In this scenario, Algorithm 4 may be preferred to attain an acceptable solution, even though it might over-allocate bits, or only come close to (but not achieve) the SNR target.

Algorithm 4 An algorithm for aggressive neighbor selection

Algorithm 5 A non-greedy neighbor selection algorithm

6. Results

A common method for bit width estimation involves using AA. However, AA overestimates the necessary bit width due to its approximation of non-linear functions. The next section demonstrates some of the shortcomings of AA in estimating bit widths when compared to PCE.

6.1. PCE vs. AA

Consider the simple DSP block in Figure 6, with two inputs: a∼$\mathcal{U}[-3,5]$ and b∼$\mathcal{N}(0,1)$. The first limitation of AA is evident from the fact that input b cannot be represented using AA since it is a normal distribution. For the sake of comparison, input b is changed to a $\mathcal{U}$∼$[-1,1]$ distribution.

Input signals a and b are bounded over $[-3,5]$ and $[-1,1]$ respectively, and so, their AA representations are given in Equations (42) and (43), as follows:

$$\widehat{a}=a_0+\sum _{i=1}^{n_a}{a}_i{\epsilon }_i=1+4{\epsilon }_1$$

(42)

$$\widehat{b}=b_0+\sum _{i=1}^{n_b}{b}_i{\epsilon }_i=1{\epsilon }_2$$

(43)

Following the AA propagation rules, the AA representations of the signals are shown as follows:

$$\begin{array}{cc}\hfill \mathtt{add}& =1+4{\epsilon }_1+1{\epsilon }_2\hfill \\ \hfill \mathtt{mult}\mathtt{0}& =0.2+0.8{\epsilon }_1+0.2{\epsilon }_2\hfill \\ \hfill \mathtt{mult}\mathtt{1}& =0.3{\epsilon }_2+1.2{\epsilon }_3\hfill \\ \hfill \mathtt{out}& =-1+(-4){\epsilon }_1+0.2{\epsilon }_2+1.2{\epsilon }_3\hfill \end{array}$$

Finally, the signal power is calculated to be 6.826.

Next, the noise nodes are added to the DFG as shown in Figure 8. With a rounding quantizer, the AA representations of the noise inputs are shown as follows:

$$n_3=\left({\displaystyle \raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$64$}\right.}\right){\epsilon }_4{n}_4=\left({\displaystyle \raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$64$}\right.}\right){\epsilon }_5{n}_5=\left({\displaystyle \raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$16$}\right.}\right){\epsilon }_6{n}_6=\left({\displaystyle \raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$8$}\right.}\right){\epsilon }_7$$

Propagating these coefficients through the DFG, the AA representation of the output signal out, is shown in Equation (44):

$$\mathtt{out}=-1+(-4){\epsilon }_1+0.2{\epsilon }_2+{\displaystyle \frac{0.2}{64}}{\epsilon }_5+\left(1.2+{\displaystyle \frac{0.4}{64}}+{\displaystyle \frac{1}{16}}\right)\left(1+{\displaystyle \frac{1}{64}}\right){\epsilon }_7$$

(44)

The signal plus noise power is calculated to be 6.8921. Finally, the SNR of this bit width configuration, as estimated by AA, is shown in Equation (45):

$$SNR=10{log}_{10}{\displaystyle \frac{6.826}{6.90-6.826}}=19.68\mathrm{dB}$$

(45)

A comparison of the AA estimation with the PCE estimation along with an MC simulation is presented in

Table 4. Accuracy of AA and PCE compared to MC.

Simulation Method	SNR
MC (10,000,000 samples)	30.71 dB
MC (100,000 samples)	30.78 dB
AA	19.68 dB
PCE with maximum polynomial degree = 1	30.70 dB

.

Estimating bit widths using affine arithmetic overestimates the hardware resources; this can be observed in Table 4. For a given bit width, AA estimates the SNR to be much lower than it is, in this case, about 11 dB less than the Monte Carlo estimate. As a result, when evaluating potential bit widths within a search space, an AA-based estimation would produce a solution that is a significant overestimate of resources. PCE simulations, on the other hand, can estimate the SNR significantly better. Table 4 shows that even when the maximum polynomial degree of the basis set in PCE is set to 1, it still yields a good estimate of the SNR predicted by the MC simulations.

The AA simulation claims that for the bit width configuration {a,b = 5, mult0 = 3, mult1 = 2}, the SNR is estimated to be about 19.68 dB. As described earlier, this is an over-allocation of resources. To obtain a measure of how much of an over-allocation it is, Algorithm 2 can be used to estimate a bit width configuration for a target SNR of 20 dB. Doing so gives the configuration shown in

Table 5. Bit widths for a target SNR of 20 dB.

${\mathit{SNR}}_{\mathbf{tgt}}$ (dB)	Signal Bit Width (Bits)				${\mathit{SNR}}_{\mathbf{MC}}$ (dB)
	a	b	mult0	mult1
20	1	1	1	1	20.93

.

If all the signals are quantized to 1-bit using a rounding quantizer, the 20 dB SNR target can be met. Compared to the AA solution, mult0 and mult1 can be 1-bit multipliers instead of 3-bit and 2-bit multipliers, respectively. Hardware resource requirements scale quadratically with the bit width of a multiplier and linearly with the bit width of an adder.

6.2. PCE vs. MC

This section demonstrates that for the DSP block in Figure 6, the PCE SNR estimation is comparable to an MC simulation with 1,000,000 samples. Figure 10 shows the SNR estimate converging to around 30.715 as the number of MC samples increases from 1000 to 50,000,000.

The PCE representation used to analyze the quantization noise for the DSP block in Figure 6 has six random variables. With a first-degree expansion, Equation (21) shows that the basis polynomial set has six polynomials. To propagate N PCE coefficients through the DSP block, the add, subtract and constant multiply nodes require N operations each. The non-linear multiply requires $N^3$ operations. Therefore, with 6 coefficients, it requires 234 operations to estimate the SNR.

An MC simulation with N samples requires $4N$ operations to propagate the samples to the output of the block. Figure 10 shows that the SNR estimate converges at around 1,000,000 samples, which requires 4,000,000 operations. It can be observed that while PCE suffers from the curse of dimensionality, a low order expansion is enough to estimate the bit widths accurately when compared to MC simulations with high sample numbers.

MC simulations can be parallelized very well, reducing the total simulation runtime. However, this advantage cannot always be exploited due to the significantly lower number of operations required by PCE for primitive DSP blocks. For example, with the DSP block shown in Figure 6, a PCE simulation requires 234 operations while MC requires around 4,000,000 operations. Reducing the runtime to around that of PCE requires a parallelization speed-up by a factor of more than 17,000. This requires significantly more computational resources.

6.3. Third-Order Taylor Series Sine/Cosine Expansion Around $x_0=2.0$

Figure 11 shows a DSP block, which calculates the Taylor series expansion around $x=x_0$. The input signal in this example varies from 0 to $2\pi$ uniformly and its PCE coefficients are given in Equation (46), as follows:

$$\mathbf{x}=\left[\begin{array}{ccc}\pi & \pi & 0\end{array}\right]$$

(46)

This DFG has seven random variables (one input and six noise sources), requiring thirty-six basis polynomials.

The results from 10,000 iterations of the simulated annealing algorithm, using the aggressive neighbor selection (Algorithm 4) as the Neighbor() function and an initial temperature of 1.0, are shown in

Table 6. Bit widths for the DFG in Figure 11 with $x_0=2$.

${\mathit{SNR}}_{\mathbf{tgt}}$ (dB)	Signal Bit Width (Bits)						${\mathit{SNR}}_{\mathbf{MC}}$ (dB)
	$\mathit{x}$	${\mathit{x}}^{\mathbf{2}}$	${\mathit{x}}^{\mathbf{3}}$	top	mid	bot
15	1	1	1	1	1	1	15.93
20	2	2	1	3	1	3	20.37
25	5	1	1	4	2	4	26.00
30	5	3	2	3	6	3	30.63
35	4	7	5	4	4	8	35.18
40	5	6	1	5	6	6	40.18
45	6	5	2	6	7	7	45.79
50	8	7	4	9	7	6	50.26
60	11	11	4	10	11	8	60.24
70	12	8	9	12	12	10	70.17
80	11	11	9	12	13	13	82.25
90	14	12	9	16	14	16	90.07
100	16	15	12	15	15	15	100.22

. It can be seen that even with second-degree polynomials and an aggressive design space exploration algorithm, it is possible to achieve a fine level of control over the desired quantization noise power at the output, even though the signal-processing chain is non-linear.

Figure 12 shows the shape of the noise distribution at the output obtained by an MC simulation (blue) and a PCE simulation (orange). The actual noise distribution is not smooth and has many peaks, signifying that a high maximum polynomial degree for the basis functions should be used to accurately capture all the higher-order moments of the distribution. However, since only the second-order moments are of interest when determining noise powers, the histogram and Table 6 show that truncating the maximum polynomial degree to two is sufficient to provide accurate results.

6.4. FM Demodulator

Figure 13 shows the DFG for an FM demodulator. The architecture used for the demodulator was elaborated on in [38]. Due to the presence of three time-delay elements, each input signal preceding the time-delay blocks in Figure 13 requires three input variables. The noise sources from the two multipliers after the time delays require only one random variable each. This results in the system implementation requiring a total of 14 random variables and 120 basis polynomials.

Analyzing Equation (28) shows that the number of operations required by a multiplier to compute its output is given by $m^3$, where m denotes the size of the basis set. As a result, with 120 basis polynomials, and a PCE simulation of three time steps, the total number of operations performed by a single multiplier is $3\times {\left(120\right)}^3=$ 5,184,000, which takes about 70 milliseconds (ms) per multiplier per time step for each PCE simulation. If the simulated annealing algorithm, as shown in Algorithm 2, is run for 10,000 iterations, then this simulation would require the following:

$$\begin{array}{cc}\hfill & 70{\displaystyle \frac{\mathrm{ms}}{\mathrm{multiplier}}}\times 2{\displaystyle \frac{\mathrm{multiplier}}{\mathrm{time}\mathrm{step}}}\times 3{\displaystyle \frac{\mathrm{time}\mathrm{step}}{\mathrm{simulation}}}\times \mathrm{10,000}\mathrm{simulation}\hfill \\ \hfill & =\mathrm{4,200,000}\mathrm{ms}\hfill \\ \hfill & =70\min \hfill \end{array}$$

To improve this runtime, sparse matrix multiplication libraries can be used to improve the milliseconds-per-multiplier factor, or the number of iterations of the simulated annealing algorithm can be reduced. The latter approach is taken here and the number of simulations is reduced from 10,000 to only 100. This puts more of an emphasis on the Neighbor().

The results from 100 iterations of the simulated annealing algorithm using Algorithm 4 as the Neighbor selection function are shown in

Table 7. FM demodulator bit widths.

${\mathit{SNR}}_{\mathbf{tgt}}$ (dB)	Signal Bit Width (Bits)				${\mathit{SNR}}_{\mathbf{MC}}$ (dB)
	I	Q	mult0	mult1
15	3	3	2	2	15.04
20	3	6	3	3	20.18
25	4	4	4	5	25.46
30	8	4	7	8	30.02
35	5	8	7	6	35.17
40	6	7	7	7	40.03
45	8	7	7	9	45.05
50	9	8	9	8	50.80
60	10	12	9	12	60.08
70	11	12	12	12	70.13
80	15	13	13	13	80.18
90	17	15	14	16	90.08
100	18	18	16	16	100.27

. It can be seen that all SNR targets are met with a high level of accuracy even with significantly reduced simulated annealing iterations. In this example, due to the reduced number of total simulated annealing iterations, there was not enough time to perform any form of hill climbing to attain a better solution. Thus, when the number of total iterations is low, it is better to use a greedy neighbor selection algorithm, like Algorithm 4, to obtain solutions.

Additionally, due to the presence of only two multipliers, this DSP system is well characterized by basis functions with a maximum polynomial degree of 2. This can be seen in the close match between the MC histogram, and the PCE-predicted histogram in Figure 14.

6.5. Single Pole IIR Filter

Figure 15 shows the DFG for an IIR filter. An IIR filter uses all past inputs to calculate its current input. As a result, to simulate an IIR filter using PCE, a new random variable must be introduced at each simulation time step.

Figure 16 shows that the estimate of the second moment, i.e., the signal power, converges to a final value after about four time steps. As a result, at least four random variables need to be injected per input of the system.

The simulated annealing algorithm is run for 10,000 iterations with an initial temperature of 1 and using Algorithm 4 for neighbor selection. The results are shown in

Table 8. IIR filter bit widths using Algorithm 4.

${\mathit{SNR}}_{\mathbf{tgt}}$ (dB)	Signal Bit Width (bits)			${\mathit{SNR}}_{\mathbf{MC}}$ (dB)
	x	x_c	y_dc
15	2	3	4	17.15
20	3	4	4	20.08
25	4	5	5	26.14
30	4	7	6	29.12
35	5	7	7	35.15
40	6	7	9	41.89
45	7	8	9	47.15
50	8	9	9	50.24
60	11	11	12	63.22
70	12	13	12	70.82
80	13	14	14	80.32
90	14	16	17	90.06
100	16	18	17	97.62

. Two of the SNR targets (30 dB and 100 dB) are not reached. This is primarily due to the aggressive neighbor selection algorithm used for the simulated annealing algorithm.

In order to meet all the SNR targets, the non-greedy neighbor selection algorithm (Algorithm 5) can be used instead. The results from this neighbor selection algorithm are shown in

Table 9. IIR filter bit widths using Algorithm 5.

${\mathit{SNR}}_{\mathbf{tgt}}$ (dB)	Signal Bit Width (Bits)			${\mathit{SNR}}_{\mathbf{MC}}$ (dB)
	x	x_c	y_dc
15	2	4	5	17.79
20	3	5	5	23.09
25	4	6	6	29.17
30	6	7	6	34.69
35	6	8	7	38.23
40	7	8	8	44.17
45	7	9	10	47.92
50	8	10	10	53.21
60	10	13	11	62.28
70	12	13	13	74.27
80	13	15	15	83.32
90	16	16	17	93.34
100	17	18	18	104.44

.

It can be observed that the 30 dB and 100 dB SNR targets are reached, as well as an overall trend toward solutions that slightly over-allocate resources. Often, not achieving a target SNR can significantly affect the reliability of the signal-processing operation itself. As a result, it is better to over-allocate resources to ensure all target SNRs are achieved, rather than minimize resource usage at the risk of failing to meet an SNR target.

6.6. FIR Filters

The FIR filter, shown in Figure 17, is a basic building block within modern DSP systems. The target signals for which the bit widths need to be determined are the input and the branches themselves. It can be assumed that all the branches are of the same bit width.

Using 10,000 iterations of the simulated annealing algorithm with the aggressive neighbor selection algorithm (Algorithm 4) and an initial temperature of 1, the following bit widths were obtained for different SNR targets, as shown in

Table 10. Bit widths of a 101-tap low-pass FIR filter using Algorithm 4.

${\mathit{SNR}}_{\mathbf{tgt}}$ (dB)	Signal Bit Width (Bits)		${\mathit{SNR}}_{\mathbf{MC}}$ (dB)
	x	branch_bitwidth
15	4	7	18.95
20	5	8	24.59
25	4	10	29.08
30	4	16	30.13
35	6	11	39.01
40	9	11	41.69
45	9	12	47.40
50	10	13	53.34
60	11	15	64.85
70	14	16	71.49
80	15	18	83.36
90	16	20	94.82
100	19	21	101.65

. The FIR filter for this table is a 101-tap low-pass filter with a cutoff of 100 kHz and a sampling rate of 2.4 MHz.

As another example, the results from using Algorithm 5, SA_iters=10,000, with an initial temperature of 1 for a 25-tap FIR filter (cutoff = 2 kHz and sampling rate = 8 kHz), are shown in

Table 11. Bit widths of a 25-tap low-pass FIR filter using Algorithm 5.

${\mathit{SNR}}_{\mathbf{tgt}}$ (dB)	Signal Bit Width (Bits)		${\mathit{SNR}}_{\mathbf{MC}}$ (dB)
	x	branch_bitwidth
15	3	5	20.75
20	3	7	23.74
25	5	7	32.26
30	5	8	34.87
35	6	9	40.75
40	7	10	46.69
45	8	10	49.96
50	8	12	53.66
60	10	13	64.67
70	13	14	75.30
80	14	16	85.95
90	15	18	94.71
100	18	19	105.31

.

Lastly, the results from using Algorithm 5, SA_iters = 100, with an initial temperature of 0.1 for a 251-tap FIR filter (cutoff = 3 kHz and sampling rate = 8 kHz), are shown in

Table 12. Bit widths of a 251-tap low-pass FIR filter using Algorithm 4.

${\mathit{SNR}}_{\mathbf{tgt}}$ (dB)	Signal Bit Width (Bits)		${\mathit{SNR}}_{\mathbf{MC}}$ (dB)
	x	branch_bitwidth
15	3	7	22.41
20	5	7	26.85
25	5	8	30.84
30	5	10	35.34
35	6	10	39.66
40	9	10	43.15
45	9	11	48.59
50	10	12	54.51
60	10	15	65.32
70	14	15	72.42
80	15	17	84.19
90	16	19	95.43
100	19	20	102.42

.

6.7. Phase-Locked Loop (PLL)

Figure 18 shows the DFG for a phase-locked loop (PLL). The PLL is stable when it tracks a sinusoid within its dynamic operational limits; as a result, the PLL is stabilized for PCE analysis by superimposing a sinusoidal wave on the 0-th index PCE coefficient. A consequence of superimposing this sinusoid is an oscillatory pattern still present even as the signal power PCE estimate converges (as seen in Figure 19).

The PLL has two accumulating structures within it. As a result, a new variable needs to be injected for each PCE simulation time step to accurately model the system. Additionally, like in the IIR filter example, the PCE simulation must be run for a sufficient number of time steps for the output to converge to a final value. Experimentally, it has been observed that this convergence takes about 100 simulation time steps (Figure 19). Since the size of the basis polynomial set increases factorially with the number of random variables (Equation (21)), and the runtime of a multiplier exhibits a cubic dependency on the size of the basis set (Equation (28)), it is clear that 100 random variables cannot be supported. To address this issue, only one distribution with a time-varying mean is propagated. However, this is a big approximation, and the effects of such an estimation will be seen in later results.

The target signals whose bit widths need to be determined are the signals within the loop filter (kp_ed and ki_ed). Using 1000 iterations of the simulated annealing algorithm with Algorithm 4 and an initial temperature of 0, the results in

Table 13. PLL signal bit widths with a greedy algorithm.

${\mathit{SNR}}_{\mathbf{tgt}}$ (dB)	Signal Bit Width (Bits)				${\mathit{SNR}}_{\mathbf{MC}}$ (dB)
	pll_in	e_d	kp_ed	ki_ed
15	3	4	4	9	27.34
20	3	4	9	13	32.75
25	6	5	6	10	36.30
30	10	5	10	13	45.57
35	7	7	7	12	44.79
40	8	7	8	14	51.67
45	10	9	11	13	60.34
50	9	11	10	14	59.14
60	10	11	13	16	78.24
70	12	13	13	18	79.47
80	14	16	14	20	87.03
90	17	15	18	23	103.23
100	18	17	19	23	111.16

can be obtained. Note that using an initial temperature of zero in Equation (41) results in a cooling curve that is always zero. Combining this with the neighbor selection algorithm in Algorithm 4 converts the design space exploration algorithm from a simulated annealing-based approach to a purely greedy algorithm.

More results are presented in

Table 14. PLL signal bit widths with simulated annealing.

${\mathit{SNR}}_{\mathbf{tgt}}$ (dB)	Signal Bit Width (Bits)				${\mathit{SNR}}_{\mathbf{MC}}$ (dB)
	pll_in	e_d	kp_ed	ki_ed
15	3	4	4	9	27.34
20	4	4	5	10	33.06
25	4	5	6	11	37.41
30	6	6	6	12	37.82
35	6	8	7	12	44.70
40	8	7	8	14	51.76
45	9	8	9	14	55.77
50	9	9	10	15	58.93
60	10	11	13	16	70.86
70	12	13	13	16	82.17
80	14	14	15	20	89.14
90	16	17	16	21	98.78
100	15	17	16	22	110.84

. These results look at the effects of using the simulated annealing algorithm with a non-greedy neighbor selection algorithm (Algorithm 5) and an initial temperature of 10.0.

Table 15. Savings in bit widths using a non-greedy neighbor selection vs. a greedy neighbor selection.

${\mathit{SNR}}_{\mathbf{tgt}}$ (dB)	Total Number of Bits		Bit Savings
	Greedy Algorithm	Non-Greedy Algorithm
15	20	20	0
20	29	23	6
25	27	26	1
30	38	30	8
35	33	33	0
40	37	37	0
45	43	40	3
50	44	43	1
60	50	50	0
70	56	54	2
80	64	63	1
90	73	70	3
100	77	70	7

shows that the simulated annealing algorithm either outperforms the greedy algorithm at every target SNR or, at worst, delivers a solution of equivalent quality. The metric used to assess the two solution sets is the overall total bit-width cost required to implement the system. For example, for the 20 dB target with the greedy algorithm, 29 bits were required to attain the 20 dB SNR target. However, by using simulated annealing, which can accept some bad solutions as valid, a solution with 23 total bits can be obtained. Regarding the 20 dB target—by allowing the pll_in signal to be represented with 4 bits instead of 3, the kp_ed and ki_ed signals can have lower bit widths.

7. Discussion

Current methods in bit width estimation such as AA and MC offer either fast or accurate estimates for the round-off noise in a DSP system, but not both. In [31,32,33], AA is used to model the quantization error as it propagates through a DFG. While AA methods are fast, they significantly overestimate the bit widths required. Reference [33] limited the application of AA to LTI and differentiable systems only. As a result the overestimation, while still present, was reduced. In [24], a unique bit width optimization method was presented; however, the authors also stated that expanding this approach to non-linear systems may result in unrealistic error estimates.

While PCE was applied to estimate bit widths in [39], the focus was on overflow error. That paper also highlighted that this approach may be extended to model round-off noise. In this work, we extended PCE to model round-off noise propagation through various blocks used in DSP systems. In addition, we provided algorithms that effectively searched through the design space and obtained bit widths for a desired SNR. Lastly, we showed that while PCE suffers from the curse of dimensionality, by constraining the maximum polynomial degree of the basis set, it is possible to obtain SNR estimates that are comparable to a high number of MC simulations while conducting a fraction of the number of calculations, as would be required in MC.

8. Conclusions

This paper shows that although PCE suffers from the curse of dimensionality, it can still be applied to DSP blocks (often with low dimensionality) to obtain accurate bit widths for its signals within a reasonable runtime. For all the DSP blocks analyzed, PCE bit width estimation was able to obtain accurate estimates of the required bit widths. Compared to affine arithmetic, a significant increase in accuracy is also demonstrated. With respect to MC simulations, a significant increase in runtime even under parallelization is shown. The runtime advantage and the accuracy of the methods presented in this work make them particularly valuable in the early stages of designing fixed-point DSP systems when it is too computationally expensive to carry out MC simulations.

Future Work

Numerous potential future avenues of research are rooted in the application of PCE for estimating bit widths in DSP systems. Similar to other methods used in estimating bit widths, PCE faces challenges in scaling to complex DSP systems. However, we believe that PCE methods have reached a level of maturity such that they can be applied to state-of-the-art FM receivers and DSP systems of similar complexity.

Currently, much of the work used in estimating bit widths can be categorized into analytical methods with simplistic noise models, simulation-based approaches that require significant computing resources, or a combination of the two. Adopting relatively new uncertainty quantification techniques [5,40] for bit width estimation remains unexplored. As a result, exploring how these new techniques can be adapted to this problem would be beneficial.

Additionally, this work has investigated how PCE can be applied to DSP blocks. Increasing the scope and applying PCE to full DSP systems would be beneficial. Due to the curse of dimensionality in PCE, the trade-off between simulation time and simulation accuracy would need to be explored when modeling entire receivers. The Gauss–Legendre quadrature methods for PCE coefficient estimations [5] would be particularly effective to explore.

Lastly, control systems utilizing feedback, such as PLLs, are frequently used in practical DSP systems. However, modeling their behavior using PCE has been a challenge. Extending the approach presented in this work to account for the temporal correlation in the input distributions would help in obtaining better bit width estimates in feedback systems.