Homomorphic ReLU with Full-Domain Bootstrapping

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

This paper presents an improved method for evaluating a homomorphic ReLU function over larger integers using full-domain bootstrapping. To this end, the authors propose a signed-digit decomposition algorithm bulit upon three components: LeftShift, HomMod, and CipherClean. Based on this construction, they develop an enhanced homomorphic ReLU algorithm. The paper concludes with implementation results demonstrating the advantages of the proposed approach.

 

While the core ideas of the paper are not entirely novel, they are interesting and are appropriately combined to achieve the stated objective. Moreover, the results obtained from the proposed algorithm appear to be meaningful. However, the current version of the paper contains several issues that require improvement or clarification.

 

* Lack of formal correctness analysis: The theoretical correctness of the proposed algorithms is unclear. The authors should provide a formal analysis, including precise theorems and rigorous proofs, to justify the correctness of their constructions.

 

* Numerous errors and inconsistencies: There are several errata scattered throughout the paper.

1. In the Preliminaries, the authors state that vectors are denoted by boldface letters, but this convention is not consistently followed throughout the paper.
2. Several equations and algorithm descriptions appear to be incorrect or ambiguous. For example,

   - In Line 3 of Algorithm 2, the variable i is used without being defined.

   - In Algorithm 3, it is unclear whether the variable num can take the value 5. If num=5, then 2^{4-i} is not an integer.

   - The equation should read $out = \frac{x}{2^{32-num_{out}}}$, but it is written incorrectly in the paper.

The above list regarding 1 and 2 is not exhaustive. The authors are strongly encouraged to conduct thorough revision of the manuscript to correct these issues and improve the overall clarity and rigor of the presentation.

Author Response

Comment 1: "Lack of formal correctness analysis: The theoretical correctness of the proposed algorithms is unclear. The authors should provide a formal analysis, including precise theorems and rigorous proofs, to justify the correctness of their constructions."
Response 1: Thank you for this critical suggestion. We fully agree on the importance of formal correctness analysis. In response, we have added two formal theorems (Theorem 4.1 and Theorem 4.2) with rigorous proofs in Section 4.5. Theorem 4.1 formally states and proves the correctness of our core SignedDecomp algorithm, demonstrating that it decomposes a large integer ciphertext into signed 6-bit segments without arithmetic error, with a quantified success probability of at least 95.2%. Theorem 4.2 builds upon this to prove the correctness of the complete LargeIntReLU algorithm, showing it correctly computes the ReLU function over the decomposed segments. The proofs detail the behavior of each atomic operation (LeftShift, HomMod, CipherClean) and their composition, linking the overall correctness to the established error bounds of the underlying TFHE integer bootstrapping.

Comment 2: "Numerous errors and inconsistencies: There are several errata scattered throughout the paper... The authors are strongly encouraged to conduct thorough revision of the manuscript to correct these issues and improve the overall clarity and rigor of the presentation."
Response 2: We sincerely thank the reviewer for the meticulous reading and for pointing out these errors. We have conducted a thorough revision of the entire manuscript to correct inconsistencies and improve clarity. Specific corrections include:

1. Vector Notation: We have ensured that boldface notation for vectors is used consistently throughout the paper.
2. Algorithm 2, Line 3: The variable i is now properly defined as the index iterating from 1 to n(the LWE dimension) in the loop structure.
3. Algorithm 3 and the num variable: We have clarified that the precision parameter num in CipherClean is within the range [1,5] for 6-bit segments (as stated in its Require line). The case num=5 is valid, and the term 2^{4-i} is an integer for i from 1 to 5. The logic has been double-checked and explained in the surrounding text.
4. Incorrect Equation: The equation for the output of CipherClean has been corrected from its erroneous form to out = x / 2^{32-num}s (Ensure line of Algorithm 3).
5. General Proofreading: We have carefully proofread the manuscript to fix typographical errors, ambiguous phrasing, and inconsistent formatting.

Reviewer 2 Report

Comments and Suggestions for Authors

This paper makes valuable innovative contributions to the field of homomorphic ReLU evaluation for large integers, with research outcomes that hold significant theoretical significance and application potential. The overall quality of the paper is high, and there is room for improvement as follows:

1.The paper mentions that LargeIntReLU does not merge the decomposed result ciphertexts. Although it explains that "the merged ciphertext cannot participate in subsequent homomorphic computations under high precision," it fails to clearly illustrate how to complete the full neural network inference process based on the segmented results in practical applications. It is recommended to supplement the processing logic in specific application scenarios.

2.The description of parameter adaptation details for the SignedDecomp algorithm when decomposing different precisions (e.g., 7-bit and 11-bit) is insufficient. It is recommended to add specific examples of the number of segments and offset calculation under different precisions.

3.The experiments only conducted actual measurement comparisons with LargeBitReLU, but no direct experimental verification with the Loris scheme. It is recommended to supplement the specific reasons why the Loris scheme cannot be tested experimentally (such as no public implementation, hardware resource limitations, etc.), or attempt to conduct simulation comparisons based on its public parameters.

4.The paper does not involve end-to-end performance testing of the algorithm on actual neural network models (e.g., CNN, MLP). It is recommended to supplement indicators such as inference accuracy and latency on typical models to further verify the practical application effect of the scheme.

5.The correctness of the algorithm relies on fixed parameter settings (e.g., LWE dimension n=500). It is recommended to analyze the impact of parameter adjustments on the algorithm's accuracy, security, and efficiency, and discuss the adaptive optimization strategy of parameters.

Author Response

Comment 1: "The paper mentions that LargeIntReLU does not merge the decomposed result ciphertexts... it fails to clearly illustrate how to complete the full neural network inference process based on the segmented results in practical applications. It is recommended to supplement the processing logic in specific application scenarios."
Response 1: We agree that clarifying the application workflow is important. We have expanded the discussion in Section 4.3. The key point is that for privacy-preserving neural network inference, subsequent layers (e.g., linear layers) would also need to operate on the segmented ciphertext representation. The homomorphic matrix multiplication can be adapted to process segmented inputs by applying the same decomposition to weights and performing batched inner products on corresponding segments, followed by a recombination step (which is a plaintext operation if done by the result owner). We have added a sentence to this effect: "In a full inference pipeline, subsequent homomorphic linear layers would be adapted to process the segmented ciphertext format, for instance, by decomposing weight matrices accordingly and performing segment-wise operations."

Comment 2: "The description of parameter adaptation details for the SignedDecomp algorithm when decomposing different precisions (e.g., 7-bit and 11-bit) is insufficient. It is recommended to add specific examples of the number of segments and offset calculation under different precisions."
Response 2: Thank you for this valuable suggestion. We have added a new subsection, 3.4.3. Parameter Adaptation Examples for Different Precisions, and a corresponding Table 2. This table provides concrete examples for precisions Q=7 to 11, showing the number of segments (n), the dynamic shift amounts (5n+1-Q, 5n-2-Q) used in Algorithm 4, and the effective data bits (num) for each segment, which directly guide the CipherClean step. This clearly illustrates how the algorithm parameters adapt to the input precision.

Comment 3: "The experiments only conducted actual measurement comparisons with LargeBitReLU, but no direct experimental verification with the Loris scheme. It is recommended to supplement the specific reasons why the Loris scheme cannot be tested experimentally... or attempt to conduct simulation comparisons based on its public parameters."
Response 3: We appreciate this comment. We added an explanation in Section 5.3: "Due to the unavailability of a practical, open-source implementation of the Loris scheme [14], direct experimental comparison was not feasible." Furthermore, to provide a meaningful theoretical comparison, we have enhanced Section 5.2 (Theoretical Complexity Analysis) and Table 5. We now derive and present the estimated total bootstrapping counts for a ciphertext-level Loris-based ReLU evaluation, which would require decomposition overhead plus its core bootstrapping. This analysis shows our LargeIntReLU maintains a theoretical advantage, reducing the number of bootstrappings from $6\ceil{\frac{n-1}{5}}+n-7$ to $4\ceil{\frac{n-1}{5}}+n-5$.

Comment 4: "The paper does not involve end-to-end performance testing of the algorithm on actual neural network models (e.g., CNN, MLP). It is recommended to supplement indicators such as inference accuracy and latency on typical models to further verify the practical application effect of the scheme."

Response 4: Thank you for your insightful suggestion regarding end‑to‑end evaluation on neural networks. We fully agree that such evaluation is essential for demonstrating practical applicability. In an effort to address this, we attempted to integrate our $\LargeIntReLU$ algorithm into the REDsec framework for encrypted MNIST inference. However, we encountered a fundamental obstacle: the ReLU module in REDsec introduces ciphertext‑specific errors that were not accounted for during plaintext model training. Consequently, while the plaintext model achieved ~99% accuracy, its encrypted inference accuracy dropped to approximately 10%—equivalent to random guessing. This confirms that models trained solely on plaintext cannot be directly deployed in homomorphic inference pipelines; cryptographic noise must be incorporated during training (e.g., via noise‑aware training).

Resolving this issue involves objective difficulties beyond time constraints. Accurately modeling REDsec's error distribution requires detailed knowledge of its internal bootstrapping parameters and encoding schemes—information not available in public documentation. Modifying the core codebase to enable noise injection would require maintainer privileges and risks introducing vulnerabilities. Moreover, REDsec offers no APIs for integrating training‑time noise simulation. We have contacted the original developers through multiple channels for guidance, but as of the submission deadline, we have not received a response. Without their assistance, we cannot safely adapt the library for noise‑aware training.

We have documented this limitation in Section 5.3 and remain committed to pursuing collaborative opportunities or alternative frameworks for comprehensive end‑to‑end evaluation in future work. We hope the theoretical contributions and component‑level validations presented here remain valuable at this stage.

Comment 5: "The correctness of the algorithm relies on fixed parameter settings (e.g., LWE dimension n=500). It is recommended to analyze the impact of parameter adjustments on the algorithm's accuracy, security, and efficiency, and discuss the adaptive optimization strategy of parameters."

Response 5: Thank you for raising this point. Our parameter set (detailed in Table 3) is aligned with the standardized, widely-adopted configuration of the TFHE framework [17] and prior work [13], ensuring security, comparability, and reproducibility. The achieved correctness probability (95.2% for 6-bit bootstrapping) is a direct outcome of these established parameters via Eq. 6. Adjusting core parameters (e.g., n, α_bk) involves navigating the well-known TFHE trade-offs among security, correctness, and efficiency. Our work benchmarks the algorithm's performance under this common setting. We have added a note in Section 4.4: "The chosen parameter set provides a concrete security level and correctness guarantee. Adapting parameters would follow the TFHE framework's well-established trade-offs between these properties."

Reviewer 3 Report

Comments and Suggestions for Authors

This paper proposes a homomorphic ReLU algorithm for large integer global bootstrapping, named LargeIntReLU. It addresses the efficiency bottleneck of the ReLU activation function on large integers in the traditional TFHE scheme by designing a signed number decomposition algorithm, SignedDecomp. This decomposition algorithm divides large integer ciphertexts into signed 6-bit segments through three atomic operations: LeftShift, HomMod, and CipherClean, avoiding carry propagation across segments. Combined with an optimized small integer bootstrapping scheme, it enables parallel processing. Experiments show that compared with existing methods, this algorithm has good performance in reducing bootstrapping costs, maintaining accuracy, and execution time, providing a feasible solution for large integer ReLU computations in privacy-preserving machine learning.

 

However, the following problems remain:

 

1. In the abstract section, the core differences from existing methods should be more clearly stated.
2. It is suggested to add captions or textual explanations to Figures 3 and 4 to clearly show the differences in slot allocation under different precisions.
3. In the experimental section, only comparisons with LargeBitReLU are made, without actual experimental comparisons with recent integer methods such as Loris. It is recommended to add relevant experimental comparisons to increase persuasiveness.
4. Algorithm 5 does not handle the merging step of the results of each segment after decomposition, only returning segmented ciphertexts. How to restore the complete result in practical applications? The authors mentioned that the results are not merged because the success probability decreases. It is necessary to supplement the discussion on the impact on subsequent computational architectures.
5. In the "6-bit integer accuracy 95.2%" experiment condition in Table 3, it is suggested to add explanations of the number of test samples, noise distribution, and other experimental information.
6. It is recommended to provide a proof to show that each operation during the decomposition process does not introduce additional information leakage.
7. It is recommended to supplement the implementation details of LargeBitReLU, explaining whether optimizations are used to ensure consistent comparison conditions.
8. In the future work of the article, it is mentioned to "extend to other activation functions", but no specific technical path or challenges are given. It is suggested to propose more specific research directions.

Author Response

Comment 1: "In the abstract section, the core differences from existing methods should be more clearly stated."
Response 1: We have revised the abstract to more sharply highlight our core innovations. Specifically, we now state: "Central to our approach is a signed digit decomposition algorithm, SignedDecomp, that partitions a large integer ciphertext into signed 6-bit segments using three new low-level primitives: LeftShift, HomMod, and CipherClean. This decomposition preserves arithmetic consistency, avoids cross-segment carry propagation, and allows parallelized bootstrapping."

Comment 2: "It is suggested to add captions or textual explanations to Figures 3 and 4 to clearly show the differences in slot allocation under different precisions."
Response 2: Thank you for the suggestion. We have significantly enhanced the captions for both Figures 3 and 4. The captions now explicitly describe the problem (inconsistent slot allocation causing errors) and the solution (modified coefficient assignment rule ensuring uniform allocation), guiding the reader through the key insight of Section 4.2.

Comment 3: "In the experimental section, only comparisons with LargeBitReLU are made, without actual experimental comparisons with recent integer methods such as Loris. It is recommended to add relevant experimental comparisons to increase persuasiveness."
Response 3: We have given an explanation in Section 5.3 regarding the lack of a public Loris implementation. More importantly, we have strengthened the theoretical comparison in Section 5.2 and Table 5, where we analytically derive and compare the bootstrapping operation counts required by LargeBitReLU, a ciphertext-adapted Loris approach, and our LargeIntReLU. This demonstrates our method's theoretical efficiency gain over a wider range of relevant schemes.

Comment 4: "Algorithm 5 does not handle the merging step of the results of each segment after decomposition, only returning segmented ciphertexts. How to restore the complete result in practical applications?... It is necessary to supplement the discussion on the impact on subsequent computational architectures."
Response 4: This is a crucial point. We have expanded the discussion at the end of Section 4.3. We clarify that merging is intentionally omitted because under current parameters, a merged high-precision ciphertext would have a very low success probability in subsequent homomorphic operations, rendering it useless. Therefore, the meaningful output is the set of segments. For practical application, the subsequent computational architecture (e.g., the next neural network layer) must be designed or adapted to process inputs in this segmented format, as briefly outlined in our response to Reviewer 2, Comment 1.

Comment 5: "In the '6-bit integer accuracy 95.2%' experiment condition in Table 3, it is suggested to add explanations of the number of test samples, noise distribution, and other experimental information."
Response 5: We have added a new paragraph in Section 4.6 which includes the requested experimental details: "The correctness probability of 95.2% reported in Table 4 is obtained under the following experimental conditions: Test samples: 10,000 randomly generated integers uniformly distributed over the full range of the 6-bit signed domain [-32,31]. Noise distribution: The error terms in LWE ciphertexts follow a discrete Gaussian distribution with standard deviation σ = 4.05×10^{-3}... Hardware/Software: Experiments were conducted on an Alibaba Cloud instance..."

Comment 6: "It is recommended to provide a proof to show that each operation during the decomposition process does not introduce additional information leakage."
Response 6: We have added a dedicated subsection on Information-Leakage Analysis in Section 4.6. We argue that all operations in SignedDecomp (LeftShift, HomMod, CipherClean) are public functions applied to the ciphertext. All bootstrapping uses public keys. Therefore, the entire process can be simulated given only the ciphertext and public parameters, satisfying the standard notion of circuit privacy for FHE and ensuring no additional information about the plaintext is leaked.

Comment 7: "It is recommended to supplement the implementation details of LargeBitReLU, explaining whether optimizations are used to ensure consistent comparison conditions."
Response 7: We have added clarifying details in Section 5.2. We state that for a fair ciphertext-level comparison, our baseline LargeBitReLU is constructed by composing our own BitDecomp algorithm (Algorithm 6) with the BitReLU algorithm from [13], both implemented using the same TFHE library [6] and the same set of cryptographic parameters (Table 3) as our LargeIntReLU. This ensures a consistent and optimized comparison framework.

Comment 8: "In the future work of the article, it is mentioned to 'extend to other activation functions', but no specific technical path or challenges are given. It is suggested to propose more specific research directions."
Response 8: We have expanded the future work section with more concrete details: "One immediate direction is to extend the signed decomposition framework to support other non-linear activation functions essential for deep learning, such as Sigmoid and GeLU. The technical path involves designing corresponding sign-preserving decomposition algorithms and approximating these functions via low-degree polynomials within each 6-bit segment, similar to our ReLU approach. However, challenges include managing the increased approximation error and ensuring that the decomposed segments remain within the bootstrappable precision limit."

Round 2

Reviewer 3 Report

Comments and Suggestions for Authors

THe authors have solved the comments, it can be accepted.