# Efficient Stochastic Computing FIR Filtering Using Sigma-Delta Modulated Signals

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Stochastic Computing and Sigma-Delta Modulation Notation and Principle Operation

#### 2.1. Stochastic Number Generation & Properties

#### 2.2. Mathematical Properties of Logic Gates in Stochastic Computing

- NOT GateIn unipolar format, the output of the NOT gate, ${H}_{n}=\mathrm{NOT}\left({X}_{n}\right)$, complements the probability of the input,$$\begin{array}{c}\hfill H=P({H}_{n}=1)=P({X}_{n}=0)=1-P({X}_{n}=1)=1-X,\end{array}$$$$\begin{array}{c}\hfill H=P({H}_{n}=1)=P({X}_{n}=0)=1-P({X}_{n}=1)=-X.\end{array}$$
- AND Gate:The AND gate in unipolar format, ${H}_{n}=\mathrm{AND}({X}_{n},{Y}_{n})$, performs multiplication.$$\begin{array}{c}\hfill H=P({H}_{n}=1)=P({X}_{n}=1,{Y}_{n}=1)=P({X}_{n}=1)P({Y}_{n}=1)=XY.\end{array}$$
- XNOR Gate:The XNOR gate in bipolar format, ${H}_{n}=\mathrm{XNOR}({X}_{n},{Y}_{n})$, performs multiplication.$$\begin{array}{cc}\hfill H& =P({H}_{n}=1)=P({X}_{n}=1,{Y}_{n}=1)+P({X}_{n}=0,{Y}_{n}=0)\hfill \\ \hfill \phantom{\rule{1.em}{0ex}}& =2P({X}_{n}=1)P({Y}_{n}=1)-P({X}_{n}=1)-P({Y}_{n}=1)+1\hfill \\ \hfill \phantom{\rule{1.em}{0ex}}& =XY.\hfill \end{array}$$
- MultiplexerAssuming an an IID control sequence ${\left\{{C}_{n}\right\}}_{n=1}^{N}$, the multiplexer (MUX), ${H}_{n}=\mathrm{MUX}({X}_{n},{Y}_{n};{C}_{n})$, is the standard way to perform scaled addition between two SN, regardless of the format used, and is given as$$\begin{array}{cc}\hfill H& =P({H}_{n}=1)=P({X}_{n}=1,{C}_{n}=1)+P({Y}_{n}=1,{C}_{n}=0)\hfill \\ \hfill \phantom{\rule{1.em}{0ex}}& =P({X}_{n}=1)P({C}_{n}=1)+P({Y}_{n}=1)P({C}_{n}=0)\hfill \\ \hfill \phantom{\rule{4pt}{0ex}}\phantom{\rule{1.em}{0ex}}& =XC+Y\overline{C}.\hfill \end{array}$$Furthermore, if $P({C}_{n}=1)=1/2$, the MUX operates as a scaling adder, i.e.,$$\begin{array}{cc}\hfill H& =P({H}_{n}=1)=\frac{P({X}_{n}=1)+P({Y}_{n}=1)}{2}=\frac{X+Y}{2}.\hfill \end{array}$$Stochastic subtraction, on the other hand, can only be realized in the bipolar format, using a NOT gate in one of the two inputs as$$\begin{array}{cc}\hfill H& =P({H}_{n}=1)=\frac{P({X}_{n}=1)+P({Y}_{n}=0)}{2}=\frac{X-Y}{2}.\hfill \end{array}$$

#### 2.3. Correlation in Stochastic Computing

#### 2.4. The First-Order Sigma-Delta Modulator

## 3. Prior Work in Stochastic Computing FIR Filters

## 4. Proposed SDM-SC Processing Scheme

#### 4.1. SDM Encoding

#### 4.2. Stochastic FIR Filter

#### 4.3. Stochastic Coefficient Generation

## 5. Experimental Results

#### 5.1. Performance of the Proposed SDM-SC Architecture in the Spectral Domain

#### 5.2. Signal-to-Noise Ratio Comparisons

#### 5.3. FPGA Synthesis Results and Comparison

#### 5.4. Hardware Resources Comparison in a 45 nm Technology

## 6. Conclusions

## Author Contributions

## Funding

## Institutional Review Board Statement

## Informed Consent Statement

## Data Availability Statement

## Acknowledgments

## Conflicts of Interest

## Abbreviations

FIR | Finite impulse response |

FPGA | Field-programmable gate-array |

LFSR | Linear-feedback shift register |

LUT | Look-up table |

MUX | Multiplexer |

OSR | Oversampling ratio |

PSD | Power Spectral Density |

SC | Stochastic computing |

SDM | Sigma-delta modulator |

SDM-SC | Sigma-delta modulator-stochastic computing |

SN | Stochastic number |

SNG | Stochastic number generator |

SNR | Signal-to-noise ratio |

XNOR | Exclusive-NOR |

XOR | Exclusive-OR |

## References

- Mounica, Y.; Kumar, K.N.; Veeramachaneni, S.; Mahammad, N. Energy efficient signed and unsigned radix 16 booth multiplier design. Comput. Electr. Eng.
**2021**, 90, 106892. [Google Scholar] [CrossRef] - Leon, V.; Paparouni, T.; Petrongonas, E.; Soudris, D.; Pekmestzi, K. Improving Power of DSP and CNN Hardware Accelerators Using Approximate Floating-point Multipliers. ACM Trans. Embed. Comput. Syst.
**2021**, 20, 1–21. [Google Scholar] [CrossRef] - Alaghi, A.; Qian, W.; Hayes, J.P. The Promise and Challenge of Stochastic Computing. IEEE Trans. Comput.-Aided Des. Integr. Circuits Syst.
**2018**, 37, 1515–1531. [Google Scholar] [CrossRef] - Gross, W.J.; Gaudet, V.C. Stochastic Computing: Techniques and Applications; Springer, International Publishing, Springer Nature Switzerland AG: Cham, Switzerland, 2019. [Google Scholar]
- Alaghi, A.; Hayes, J.P. Survey of Stochastic Computing. ACM Trans. Embed. Comput. Syst.
**2013**, 12, 1–19. [Google Scholar] [CrossRef] - Metku, P.; Seva, R.; Choi, M. Energy-Performance Scalability Analysis of a Novel Quasi-Stochastic Computing Approach. MDPI J. Low Power Electron. Appl.
**2019**, 9, 30. [Google Scholar] [CrossRef] [Green Version] - Gaines, B.R. Stochastic Computing Systems; Springer: Boston, MA, USA, 1967. [Google Scholar]
- Najafi, M.H.; Jenson, D.; Lilja, D.J.; Riedel, M.D. Performing Stochastic Computation Deterministically. IEEE Trans. Very Large Scale Integr. (VLSI) Syst.
**2019**, 27, 2925–2938. [Google Scholar] [CrossRef] - Temenos, N.; Sotiriadis, P.P. Deterministic Finite State Machines for Stochastic Division in Unipolar Format. In Proceedings of the IEEE International Symposium on Circuits and Systems (ISCAS), Seville, Spain, 12–14 October 2020. [Google Scholar]
- Lee, V.T.; Alaghi, A.; Hayes, J.P.; Sathe, V.; Ceze, L. Energy-efficient hybrid stochastic-binary neural networks for near-sensor computing. In ACM Proceedings of the Conference on Design, Automation & Test in Europe, Laussane, Switzerland, 27–31 March 2017. [Google Scholar]
- Yuan, B.; Wang, Y.; Wang, Z. Area-Efficient Scaling-Free DFT/FFT Design Using Stochastic Computing. IEEE Trans. Circuits Syst. II Express Briefs
**2016**, 63, 1131–1135. [Google Scholar] [CrossRef] - Ting, P.; Hayes, J.P. Eliminating a hidden error source in stochastic circuits. In Proceedings of the IEEE International Symposium on Defect and Fault Tolerance in VLSI and Nanotechnology Systems (DFT), Cambridge, UK, 23–25 October 2017. [Google Scholar]
- Morro, A.; Canals, V.; Oliver, A.; Alomar, M.L.; Galán-Prado, F.; Ballester, P.J.; Rosselló, J.L. A Stochastic Spiking Neural Network for Virtual Screening. IEEE Trans. Neural Netw. Learn. Syst.
**2018**, 29, 1371–1375. [Google Scholar] [CrossRef] [PubMed] - Brown, B.D.; Card, H.C. Stochastic Neural Computation I: Computational Elements. IEEE Trans. Comput.
**2002**, 50, 891–905. [Google Scholar] [CrossRef] - Brown, B.D.; Card, H.C. Stochastic Neural Computation II: Soft Competitive Learning. IEEE Trans. Comput.
**2002**, 50, 906–920. [Google Scholar] [CrossRef] - Ardakani, A.; Leduc-Primeau, F.; Onizawa, N.; Hanyu, T.; Gross, W.J. VLSI Implementation of Deep Neural Network Using Integral Stochastic Computing. IEEE Trans. Very Large Scale Integr. (VLSI) Syst.
**2017**, 25, 2688–2699. [Google Scholar] [CrossRef] [Green Version] - Liu, S.; Jiang, H.; Liu, L.; Han, J. Gradient Descent Using Stochastic Circuits for Efficient Training of Learning Machines. IEEE Trans. Comput.-Aided Des. Integr. Circuits Syst.
**2018**, 37, 2530–2541. [Google Scholar] [CrossRef] - Liu, Y.; Liu, L.; Lombardi, F.; Han, J. An Energy-Efficient and Noise-Tolerant Recurrent Neural Network Using Stochastic Computing. IEEE Trans. Very Large Scale Integr. (VLSI) Syst.
**2019**, 27, 2213–2221. [Google Scholar] [CrossRef] - Liu, Y.; Liu, S.; Wang, Y.; Lombardi, F.; Han, J. A Survey of Stochastic Computing Neural Networks for Machine Learning Applications. IEEE Trans. Neural Netw. Learn. Syst.
**2020**, 32, 2809–2824. [Google Scholar] [CrossRef] - Temenos, N.; Sotiriadis, P.P. Nonscaling Adders and Subtracters for Stochastic Computing Using Markov Chains. IEEE Trans. Very Large Scale Integr. Syst. (VLSI)
**2021**, 29, 1612–1623. [Google Scholar] [CrossRef] - Li, P.; Lilja, D.J. Using Stochastic Computing to Implement Digital Image Processing Algorithms. In Proceedings of the IEEE 29th International Conference on Computer Design (ICCD), Amherst, MA, USA, 9–12 October 2011. [Google Scholar]
- Li, P.; Lilja, D.J.; Qian, W.; Bazargan, K.; Riedel, M.D. Computation on Stochastic Bit Streams Digital Image Processing Case Studies. IEEE Trans. Very Large Scale Integr. Syst. (VLSI)
**2014**, 2, 449–462. [Google Scholar] [CrossRef] - Alaghi, A.; Li, C.; Hayes, J.P. Stochastic circuits for real-time image-processing applications. In Proceedings of the IEEE 50th ACM/EDAC/IEEE Design Automation Conference (DAC), Austin, TX, USA, 29 May–7 June 2013. [Google Scholar]
- Temenos, N.; Sotiriadis, P.P. Stochastic Computing Max & Min Architectures Using Markov Chains: Design, Analysis, and Implementation. IEEE Trans. Very Large Scale Integr. Syst. (VLSI)
**2021**, 29, 1813–1823. [Google Scholar] - Alawad, M.; Lin, M. Fir filter based on stochastic computing with reconfigurable digital fabric. In Proceedings of the IEEE 23rd Annual International Symposium on Field-Programmable Custom Computing Machines, Vancouver, BC, Canada, 2–6 May 2015. [Google Scholar]
- Vlachos, A.; Temenos, N.; Sotiriadis, P.P. Exploring the Effectiveness of Sigma-Delta Modulators in Stochastic Computing-Based FIR Filtering. In Proceedings of the IEEE 10th International Conference on Modern Circuits and Systems Technologies (MOCAST), Thessaloniki, Greece, 5–7 July 2021. [Google Scholar]
- Saraf, N.; Bazargan, K.; Lilja, D.J.; Riedel, M.D. IIR filters using stochastic arithmetic. In Proceedings of the IEEE Design, Automation & Test in Europe Conference & Exhibition (DATE), Dresden, Germany, 24–28 March 2014. [Google Scholar]
- Ahmed, K.J.; Yuan, B.; Lee, M.J. High-Accuracy Stochastic Computing-Based FIR Filter Design. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Calgary, AB, Canada, 15–20 April 2018. [Google Scholar]
- Ichihara, H.; Sugino, T.; Ishii, S.; Iwagaki, T.; Inoue, T. Compact and Accurate Digital Filters Based on Stochastic Computing. IEEE Trans. Emerg. Top. Comput.
**2019**, 7, 31–43. [Google Scholar] [CrossRef] - Liu, Y.; Parhi, K.K. Architectures for Recursive Digital Filters Using Stochastic Computing. IEEE Trans. Signal Process.
**2016**, 64, 3705–3718. [Google Scholar] [CrossRef] - Tehrani, S.S.; Gross, W.J.; Mannor, S. Stochastic decoding of LDPC codes. IEEE Commun. Lett.
**2006**, 10, 716–718. [Google Scholar] [CrossRef] - Basetas, C.; Temenos, N.; Sotiriadis, P.P. Comparison of Recently Developed Single-Bit All-Digital Frequency Synthesizers in Terms of Hardware Complexity and Performance. In Proceedings of the IEEE International Symposium on Circuits and Systems (ISCAS), Florence, Italy, 27–30 May 2018. [Google Scholar]
- Basetas, C.; Temenos, N.; Sotiriadis, P.P. An Efficient Hardware Architecture for the Implementation of Multi-Step Look-Ahead Sigma-Delta Modulators. In Proceedings of the IEEE International Symposium on Circuits and Systems (ISCAS), Florence, Italy, 27–30 May 2018. [Google Scholar]
- Schreier, R.; Temes, G.C. Understanding Delta-Sigma Data Converters; John Wiley & Sons, Inc.: Hoboken, NJ, USA, 2005. [Google Scholar]
- Liu, S.; Han, J. Toward Energy-Efficient Stochastic Circuits Using Parallel Sobol Sequences. IEEE Trans. Very Large Scale Integr. (VLSI) Syst.
**2018**, 26, 1326–1339. [Google Scholar] [CrossRef] - Qian, W.; Li, X.; Riedel, M.D.; Bazargan, K.; Lilja, D. An Architecture for Fault-Tolerant Computation with Stochastic Logic. IEEE Trans. Comput.
**2011**, 60, 93–105. [Google Scholar] [CrossRef] - Camps, O.; Stavrinides, S.; Picos, R. Stochastic Computing Implementation of Chaotic Systems. Mathematics
**2021**, 9, 375. [Google Scholar] [CrossRef] - Lee, V.T.; Alaghi, A.; Ceze, L. Correlation manipulating circuits for stochastic computing. In Proceedings of the IEEE Design, Automation & Test in Europe Conference & Exhibition (DATE), Dresden, Germany, 19–23 March 2018. [Google Scholar]
- Alaghi, A.; Hayes, J.P. Exploiting correlation in stochastic circuit design. In Proceedings of the IEEE 31st International Conference on Computer Design (ICCD), Asheville, NC, USA, 6–9 October 2013. [Google Scholar]
- Liu, Y.; Parhi, M.; Riedel, M.D.; Parhi, K.K. Synthesis of Correlated Bit Streams for Stochastic Computing. In Proceedings of the IEEE 50th Asilomar Conference on Signals, Systems and Computers, Pacific Grove, CA, USA, 6–9 November 2016. [Google Scholar]
- Chang, Y.; Parhi, K.K. Architectures for digital filters using stochastic computing. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, Vancouver, BC, Canada, 26–31 May 2013. [Google Scholar]
- Meyer-Baese, U. Digital Signal Processing with Field Programmable Gate Arrays, 4th ed.; Springer: Berlin/Heidelberg, Germany, 2014. [Google Scholar]
- Ren, A.; Li, Z.; Ding, C.; Qiu, Q.; Wang, Y.; Li, K.; Qian, X.; Yuan, B. SC-DCNN: Highly-Scalable Deep Convolutional Neural Network using Stochastic Computing. In Proceedings of the ACM 22nd International Conference on Architectural Support for Programming Languages and Operating Systems(ASPLOS), Xi’an, China, 8–12 April 2017. [Google Scholar]
- Ichihara, H.; Ishii, S.; Sunamori, D.; Iwagaki, T.; Inoue, T. Compact and Accurate Stochastic Circuits with Shared Random Number Sources. In Proceedings of the IEEE 32nd International Conference on Computer Design (ICCD), Seoul, Korea, 19–22 October 2014. [Google Scholar]
- Stine, J.E.; Castellanos, I.; Wood, M.; Henson, J.; Love, F.; Davis, W.R.; Franzon, P.D.; Bucher, M.; Basavarajaiah, S.; Oh, J.; et al. FreePDK: An Open-Source Variation-Aware Design Kit. In Proceedings of the IEEE International Conference on Microelectronic Systems Education, San Diego, CA, USA, 3–4 June 2007. [Google Scholar]

**Figure 1.**Stochastic sequence generation: (

**a**) Stochastic number generator (SNG) block; (

**b**) Equivalent block.

**Figure 2.**Stochastic computing fundamental operations: (

**a**) Unipolar multiplication, (

**b**) bipolar multiplication, (

**c**) unipolar/bipolar addition.

**Figure 3.**The first-order single-bit sigma-delta modulator. A dithering sequence can optionally be used.

**Figure 5.**Stochastic implementation of a five-tap FIR filter based on the inner-product block of Figure 4.

**Figure 6.**Proposed SDM-SC architecture. The first-order SDM encodes a multi-bit input signal into a single-bit one, carrying the information in $0,1$ representation. The sequence is then processed by an SC-based M-tap FIR filter. A dithering sequence can optionally be used.

**Figure 8.**Power Spectral Density (

**top**) using pwelch with ${10}^{6}$ samples and frequency response (

**bottom**) of the SDM-SC architecture for a 5-tap FIR filter. Weights’ values are cited in Table 1.

**Figure 9.**Power Spectral Density (

**top**) using pwelch with ${10}^{6}$ samples and frequency response (

**bottom**) of the SDM-SC architecture for a 7-tap FIR filter. Weights’ values are cited in Table 1.

**Figure 10.**Power Spectral Density of the SDM’s output calculated using pwelch with ${10}^{6}$ samples.

Parameter Name | Parameter Value |
---|---|

Input signal ${U}_{n}$ | $sin\left(2\pi {f}_{B}n\right)$ |

5-tap FIR filter weights | ${w}_{0}={w}_{4}=0.7,\phantom{\rule{4pt}{0ex}}{w}_{1}={w}_{3}=0.6,\phantom{\rule{4pt}{0ex}}{w}_{2}=0.9$ |

7-tap FIR filter weights | ${w}_{0}={w}_{6}=0.6,\phantom{\rule{4pt}{0ex}}{w}_{1}={w}_{5}=0.4,\phantom{\rule{4pt}{0ex}}{w}_{2}={w}_{4}=0.3,\phantom{\rule{4pt}{0ex}}{w}_{3}=0.9$ |

5-Tap FIR Filter | 7-Tap FIR Filter | |||||
---|---|---|---|---|---|---|

SDM-SC | Inner-Product Adder-Tree [29,30,41] | Conv. Binary | SDM-SC | Inner-Product Adder-Tree [29,30,41] | Conv. Binary | |

SNR (dB) | 47.01 | 31.41 | 97.21 | 42.94 | 29.93 | 91.31 |

5-Tap FIR Filter | |||
---|---|---|---|

SDM-SC Filter | Inner-Product Adder-Tree [29,30,41] | Conv. Binary | |

Max Operating Frequency (MHz) | 667 | ||

Slice LUTs [Used/Util.] | 29/0.01% | 35/0.01% | 698/0.34% |

Slice Registers [Used/Util.] | 35/0.01% | 178/0.05% | 60/0.02% |

7-tap FIR filter | |||

SDM-SC Filter | Inner-Product Adder-Tree [29,30,41] | Conv. Binary | |

Max Operating Frequency (MHz) | 667 | ||

Slice LUTs [Used/Util.] | 30/0.01% | 42/0.01% | 907/0.45% |

Slice Registers [Used/Util.] | 37/0.01% | 238/0.06% | 90/0.02% |

**Table 4.**Comparison of hardware resources for the realization of two FIR filters with 5 & 7 taps in area $\left(\mathsf{\mu}{\mathrm{m}}^{2}\right)$, power (mW), delay (ns), and energy (pJ).

5-Tap FIR Filter | |||
---|---|---|---|

SDM-SC Filter | Inner-Product Adder-Tree [29,30,41] | Conv. Binary | |

Area $\left(\mathsf{\mu}{\mathrm{m}}^{2}\right)$ | 617.12 | 3425.88 | 9853.89 |

Power (mW) | 0.75 | 2.97 | 5.19 |

Delay (ns) | $1.5$ | ||

Energy (pJ) | 1.13 | 4.46 | 7.78 |

7-tap FIR filter | |||

SDM-SC Filter | Inner-Product Adder-Tree [29,30,41] | Conv. Binary | |

Area $\left(\mathsf{\mu}{\mathrm{m}}^{2}\right)$ | 700.66 | 4935.67 | 13,428.55 |

Power (mW) | 0.83 | 4.39 | 7.12 |

Delay (ns) | $1.5$ | ||

Energy (pJ) | 1.24 | 6.51 | 10.58 |

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Temenos, N.; Vlachos, A.; Sotiriadis, P.P.
Efficient Stochastic Computing FIR Filtering Using Sigma-Delta Modulated Signals. *Technologies* **2022**, *10*, 14.
https://doi.org/10.3390/technologies10010014

**AMA Style**

Temenos N, Vlachos A, Sotiriadis PP.
Efficient Stochastic Computing FIR Filtering Using Sigma-Delta Modulated Signals. *Technologies*. 2022; 10(1):14.
https://doi.org/10.3390/technologies10010014

**Chicago/Turabian Style**

Temenos, Nikos, Anastasis Vlachos, and Paul P. Sotiriadis.
2022. "Efficient Stochastic Computing FIR Filtering Using Sigma-Delta Modulated Signals" *Technologies* 10, no. 1: 14.
https://doi.org/10.3390/technologies10010014