Approximation of General Functions Using Stochastic Computing

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

This work presents “Approximation of General Functions using Stochastic Computing”. Authors are advised to consider the following comments.

1. Section 1:
   1. The notation followed throughout the manuscript is way off for the context of Stochastic Computing (SC). Specifically, the definition of Stochastic Numbers is missing. Typically, the value of a LFSR, a digital block producing random numbers for a fixed number of clock cycles, followed by a comparison with a binary number generates on each clock cycle a random variable assumed to follow a Bernoulli distribution (see References [1,4,5] cited below). Its expected value (also average in this case) converges to the binary number, where the larger the sequence length, the better the approximation. This is critical.
   2. Equations (1) and (2) do not serve any purpose as they are not used for rest of the manuscript.
   3. Authors are suggested to remove subsections from the introduction. Moreover, their contribution is absent and how their work is different from the state-of-the-art is also missing
   4. Stochastic Finite-State Machines analysis has been explored in [1,2,3].
   5. The Bipolar SC information should be moved close to the unipolar one
2. Section 2:
   1. Derivation of Equation 6 is not explained, which is also connected to the fact that authors do not make clear of the hardware architecture corresponding to their method. Essentially, here logic gates are used? Counters and registers? Both? This is critical
   2. In the algorithm, what is pi? What does it correspond to? If authors consider Markov Chain Models this should be made clear. Right now, this is missing (see also comment 1-d)
   3. Considering comments 2-a,b, Section 2 is very hard to follow
3. Section 3:
   1. By taking into account previous samples, as in equation 19, introduces correlation. This is not considered and not clear of how it works using the SDP
4. Section 4:
   1. The experimental setup and results have flaws here. In particular, what is the sequence length used for each point in figure 3? How can the numbers represented exceed the range of SC numbers? For each point shown in Figure 3, is there any averaging over a number of iterations? Figure 3 also has very poor quality
   2. The same questions as in comment 4-a apply to Section 4.2
   3. In figure 6, black dots appear only after 0.0 for cases b,c,d. Why?
   4. Proper comparison with Bernstein approximation is not correct here as hardware results are not included and/or shown
   5. Comparison with other SC dot product approaches [6,7] or state-of-the-art adders capable of forming dot products [5, 8, 9] are missing
   6. Hardware implementation (either FPGA, or ASIC) are not shown here. Considering the nature of SC, this is critical

 

 

References

1. D. Brown and H. C. Card, “Stochastic neural computation i: Computational elements,” IEEE Transactions on Computers, vol. 50, no. 9, pp. 891–905, Sep. 2002.
2. Li, D. J. Lilja, W. Qian, K. Bazargan, and M. D. Riedel, “Computation on stochastic bit streams digital image processing case studies,” IEEE Transactions on Very Large Scale Integration Systems (VLSI), vol. 2, no. 3, pp. 449–462, Apr. 2014
3. Temenos, N., & Sotiriadis, P. P. (2022). A Markov chain framework for modeling the statistical properties of stochastic computing finite-state machines. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 42(6), 1965-1977.
4. Alaghi, W. Qian, and J. P. Hayes, “The promise and challenge of stochastic computing,” IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, vol. 37, no. 8, pp. 1515 – 1531, Aug. 2018
5. Temenos, N., & Sotiriadis, P. P. (2021). Nonscaling adders and subtracters for stochastic computing using Markov chains. IEEE Transactions on Very Large Scale Integration (VLSI) Systems, 29(9), 1612-1623.
6. Temenos, N., & Sotiriadis, P. P. (2023). A stochastic computing sigma-delta adder architecture for efficient neural network design. IEEE Journal on Emerging and Selected Topics in Circuits and Systems, 13(1), 285-294.
7. Parhami and C.-H. Yeh, “Accumulative parallel counters,” in Proc. Conf. Rec. 29h Asilomar Conf. Signals, Syst. Comput., 1995, pp. 966–970.
8. Canals, A. Morro, A. Oliver, M. L. Alomar, and J. L. Rossellè, “A new stochastic computing methodology for efficient neural network implementation,” IEEE Trans. Neural Netw. Learn. Syst., vol. 27, no. 3, pp. 551–564, Mar. 2016.
9. Liu, S. Liu, Y. Wang, F. Lombardi, and J. Han, “A stochastic computational multi-layer perceptron with backward propagation,” IEEE Trans. Comput., vol. 67, no. 9, pp. 1273–1286, Sep. 2018

Author Response

Thank you for the detailed comments and suggestions. Please see the attached document for responses.

Author Response File: Author Response.pdf

Reviewer 2 Report

Comments and Suggestions for Authors

This work presents the use of stochastic computing and a partial sum of Chebyshev polynomials to approximate arbitrary functions on [−1,1]. The paper is well-organized and well-written. The reviewer’s comments are listed below:

• The notation of symbols is inconsistent, which may lead to misunderstandings. For example, in Equation (1), an​ represents coefficients, whereas in Section 1.2.4, it is used for a Bernoulli(an ​) process. Please clarify the relationship between different cases where the same symbol is used.
• The symbol σ is undefined, and it is not the standard deviation. This may cause confusion.
• The introduction of the Clenshaw Algorithm in Section 3.1 is insufficient. Please provide a detailed explanation of why N+1 cascaded circuit stages are required.
• In Section 4, 10^6 bits are utilized. Is this the bit length of each stochastic bitstream? If so, please explain why 10^6 is chosen and whether it can be reduced. In hardware implementation, 10^6 is a very large value. What happens if the bitstream length is reduced to 256 bits? What is the optimal bit length? Additionally, please discuss the tradeoff between bit length and the sample period M.
• In Section 4.2, the authors mention that for optimal performance, L should be set as small as possible and can be computed using the Clenshaw algorithm. Adding details on the calculation of L would improve the practical applicability of this work.

Author Response

Thank you for the detailed comments and suggestions. Please see the attached document for responses.

Author Response File: Author Response.pdf

Reviewer 3 Report

Comments and Suggestions for Authors

This paper proposes an efficient computing scheme to implement a novel algorithm that approximates arbitrary functions using stochastic circuit. The proposed Clenshaw algorithm has recursive elements that can be computed by repeatedly stacking stochastic dot product circuit, which makes it simple to scale up. The paper provides thorough background information on stochastic logic and clear presentation of the proposed algorithm using mathematical expressions. However, the paper lacks evaluation of the hardware cost (energy/latency/area) and quantitative results of approximation errors.

Here are some specific concerns about this paper:

• The proposed stochastic dot product algorithm is novel but does not take the advantages of SC because its core computing logic is a binary adder. Vector c is an integer vector and the step σ = σ+c_k requires a normal binary logic. The conventional binary dot product circuit can use the same adder circuit and compute in a bit-serial fashion to get accurate results.
• There is no mention of the length of the stochastic bit stream used to represent Vector x during simulation. The paper only mentions M, which is the sample period. The author should provide how many sample periods are used in simulation to give readers a sense of the simulation precision.
• Section 2 says M=2 for examples in section 4. Section 5 says M=1024 for the complicated example. Please clarify.
• The results in section 4 did not mention approximation errors. It only provides graphs showing qualitative comparison. I suggest: 1) in each example, provide an error percentage number that is averaged across all input points. 2) provide the error percentage number if the algorithm is computed using binary arithmetic and the same precision equivalent to the stochastic bit stream length.
• Can you provide an intuitive explanation of why the proposed method seems to be better than Bernstein approximation in section 4.3? How does the comparison look like for more common functions like the ones you used in section 4.2?

Author Response

Thank you for the detailed comments and suggestions. Please see the attached document for responses.

Author Response File: Author Response.pdf

Reviewer 4 Report

Comments and Suggestions for Authors

Dear author, the manuscript is clear and coincise. The structure of the paper also helps the reader in understanding the method applied and following the algorithm implementation. 

I do not have many adjunctive requests related to this article. One thing is just to make more quantitative considerations about the required memory space necessary to implement such algorithm with realistic values for N and M in order to achieve sufficient performances to represent complex functions. 

Except for this, the manuscript is complete in my opinion. 

Author Response

Thank you for the feedback and comments. Please take a look at the revised paper to see if it satisfies your suggestion.

Reviewer 5 Report

Comments and Suggestions for Authors

This manuscript presents an approximation of general functions by using stochastic computing. Some comments are listed as follows.

 

1. The motivation of this manuscript is not clear. The authors should well explain the weaknesses of current research bout general function approximation and the necessity of using stochastic computing.
2. Is there any research on general function approximation? What is the background of the research?
3. The authors are encouraged to provide some real cases for verification.
4. Please give more comparison between the proposed method and other existing reports.

Author Response

Thank you for the detailed comments and suggestions. Please see the attached document for responses.

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

Comments and Suggestions for Authors

This is the second revision of the work.

Comments to the authors
1. In abstract authors claim "A SC circuit that approximates arbitrary functions 3 on [−1, 1] using a partial sum of Chebyshev polynomials is presented.". No circuit is presented in this work, only an algorithm. This is misleading.
2. Notation was not improved. Authors answered to the comment "Most of this is covered more generally by simply invoking a Bernoulli process.". Readers not familiar with the concept cannot keep up and this is the reason for defining clearly what is used in the manuscript
3. Authors responded to the comment with the LFSR "A digital source block as described is just one possible implementation; ideally one could exploit some natural entropy source having true randomness." Correct LFSR is one implementation. However, authors a) propose a SC circuit implying a digital implementation (see comment 1) and b) each random number generator considered has its own mathematical properties. LFSR was suggested considering that authors assumed bernoulli process.
4. Authors did not provde a hardware implementation (e.g., FPGA, ASIC) to have common comparison method with the rest of the literature. Computational accuracy is not the only metric for consideration in SC.
5. Authors did not explore different sequence lengths to further stress their work. In combination with the above comment, comparison is missing.

Author Response

See attached file.

Author Response File: Author Response.pdf

Reviewer 2 Report

Comments and Suggestions for Authors

The author has addressed the reviewer's questions well, and there are no further questions.

Author Response

-

Reviewer 3 Report

Comments and Suggestions for Authors

The author has addressed my concerns.

Author Response

-

Reviewer 5 Report

Comments and Suggestions for Authors

The authors have well addressed all my comments, and it could be accepted now.

Author Response

-