An Approach for Controlled Random Tests with a Given Hamming Distance Generation
Round 1
Reviewer 1 Report
Comments and Suggestions for AuthorsIn the Introduction, the authors fail to effectively articulate and emphasize the motivations, innovations, and contributions of this paper. A substantial revision of the Introduction section is necessary to clearly convey these aspects. Additionally, the final paragraph of this section should provide a concise overview of the structure and organization of the remainder of the paper.
On line 56, the notation for the set should be revised to adhere to standard mathematical conventions.
In Section 2, the related work and literature review are disorganized and lack a coherent structure. The authors should reorganize these sections to present the literature in a more logical and rational manner. Moreover, the literature review is inadequate and requires substantial expansion to comprehensively cover relevant prior work.
The paragraph spanning lines 232–244 is unclear and difficult to follow. The authors should thoroughly rewrite this section to enhance clarity and coherence.
The authors should verify the accuracy of the term “b256” on line 274.
Author Response
Dear Reviewer,
Thank you very much for your careful review and constructive comments on our manuscript. We have carefully revised the paper according to your suggestions and prepared a detailed point-by-point response, which is attached as a PDF file. We believe that the revisions have significantly improved the clarity and quality of the manuscript.
Sincerely,
The Authors
Author Response File: 
Author Response.pdf
Reviewer 2 Report
Comments and Suggestions for AuthorsThe authors' work has several strengths:
- Previous research on Controlled Random Tests (CRT) typically relied on a single scalar Hamming distance (or Euclidean) as the measure of diversity between test patterns. This paper presents a vector-based dissimilarity (MD), which computes Hamming distances after mapping each binary pattern into multiple numeral systems (quaternary, octal, etc.). This allows more nuanced discrimination between test patterns, capturing diverse structural and spatial pattern relationships.
- Traditional CRT and Adaptive Random Testing (ART) methods require computationally intensive candidate enumeration and selection based on maximizing diversity (distance) metrics, resulting in increased complexity as the test set size and required minimum distance grow. This paper introduces an algebraic, block-based partitioning algorithm: test sets are generated by dividing patterns into non-overlapping blocks and assigning distinct codes to each block, mathematically ensuring a minimum Hamming distance without exhaustive search. The authors show that this approach reduces the computational burden, making large or highly diverse test sets practical.
- Using Hamming and Plotkin bounds, the authors demonstrate that increasing the minimum Hamming distance inevitably decreases the number of patterns that can meet that criterion. The authors offer a formalism showing the trade-offs between diversity and test set size, providing practical guidance for testers on balancing these factors for reliability and cost.
- By not restricting characterization to binary vectors and allowing representation in arbitrary alphabets and numeral systems, the authors demonstrate that their method supports broader applications, including complex pattern-sensitive fault models in memory and embedded systems not effectively addressed by previous single-metric CRT methods.
- Through simulations and experiments, the authors show that the proposed tests provide greater statistical pattern diversity and higher fault coverage, especially for multicell (pattern-sensitive) faults, when compared to both standard random tests and prior CRTs such as antirandom and ART approaches. Performance gains were shown to be particularly pronounced with limited test iterations and for small fault group sizes.
- The authors propose a procedure where candidate test patterns are selected not just by highest scalar difference but by maximizing the earliest non-matching component in the dissimilarity vector. This strategy was shown to break ties and achieves finer discrimination between patterns, overcoming limitations of classical ART algorithms.
- Prior work used scalar Hamming/Euclidean distance or simple diversity heuristics, suffered from computational complexity and limited formal trade-off strategies, and focused largely on black-box coverage and simple fault models. In contrast, the current work presents an extensible, multi-component dissimilarity metric, establishes a low-overhead algorithm that guarantees diversity, provides theoretical and empirical validation for broader and deeper fault coverage, and extends the applicability of CRT to modern, pattern-sensitive computing device testing where classical methods underperform.
There are several areas where the writing can be improved. Here is a list of key areas:
In the Abstract section:
"This paper addresses the challenges of testing computing systems and their
components."" -> vague: “components” could specify type.
"It highlights the limitations of traditional random testing, which often fails
to utilize available information about the system under test and previously generated test
patterns." -> long sentence; convert into two or more shorter sentences.
"...introduces a novel measure of dissimilarity between test patterns, based on the calculation of Hamming distances for binary patterns represented in various numerical systems." -> "various..." vague; be more precise.
" A method for generating controlled random tests with a fixed Hamming distance is proposed. The core idea relies on representing binary patterns as symbols from non-binary numerical systems, where ensuring a specific Hamming distance in the symbolic domain guarantees the same distance in binary representation." -> a long sentence. Please convert to two or more shorter sentences.
"The effectiveness of the proposed method is evaluated through simulation, particularly in the context of memory testing and the detection of multicell faults." -> briefly define "multicell faults"
"Keywords: test, test pattern, computing systems testing, random test, controlled random test, Hamming distance, pattern sensitive faults, march tests" -> capitalize "March tests"
In the "Introduction section:
"Despite decades of efforts to develop alternative technologies, testing remains the primary method for verifying the quality of computer system software, hardware, memory devices, and applications. Nevertheless, it remains a labor-intensive, time-consuming, and imperfect process. " -> remove the repeat of "remains".
"Probabilistic testing (Random Testing) is conceptually simple, easy to implement, and effective in identifying faults in computing systems, including their software, hardware, and memory components. This testing approach has been widely used as an indepen-dent method and also forms the core of many other testing methodologies." -> "also" is unnecessary/repetitive.
Consequently, various methods have been developed in which the random factor plays a subordinate role. Among these, Adaptive Random Testing (ART), also referred to as Controlled Ran-dom Testing (CRT) holds a prominent position -> ART ≠ CRT; clarify terminology.
"This type of testing and its numerous modifications rely on calculating specific char-acteristics for the controlled generation of the next random test pattern." -> repeats what was said in the last sentence in the previous paragraph.
"Most known approaches to generating adaptive random tests utilize the Hamming distance as a key characteristic for determining the selection of the next test pattern [9–11]." -> Reference #9 discusses Black-box testing, rather than Antirandom testing.
"Table 3. Example of the first modification for Hamming distance calculation." -> Table heading not descriptive; could clarify context, e.g., "Hamming Distance Computation in Multiple Alphabets for n=…"
In the References section
" In Proceedings of the Proceedings of the ...." -> Repeat of "Proceedings of the...".
"Duran, J.W.; Ntafos, S.C. An Evaluation of Random Testing. IEEE Transaction on Software Engineering. 1984, 10, 438–444." -> "Transaction" should be "Transactions".
"Peterson, W. Error-correcting codes. Cambridge, MA: MIT Press google schola 1972, 2, 208–213. -> "google schola 1972" should be corrected.
In the "Author Contributions" section
" writing—original draft preparation, V.Y. and. I.M" -> "and." should be "and"
In the "Funding" section: "Ministry of Science and Higer Education, Poland. " -> "Higer" should be "Higher"
General comments:
- Use active, rather than passive, voice to make the paper more engaging, and whereever possible, make sentences more concise.
- Check for inconsistent use of spaces after commas and full-stops/periods.
Author Response
Dear Reviewer,
Thank you very much for your careful review and constructive comments on our manuscript. We have carefully revised the paper according to your suggestions and prepared a detailed point-by-point response, which is attached as a PDF file. We believe that the revisions have significantly improved the clarity and quality of the manuscript.
Sincerely,
The Authors
Author Response File: Author Response.pdf
Reviewer 3 Report
Comments and Suggestions for Authors1. The authors claim that “The paper introduces a novel measure of dissimilarity between test patterns, based on the calculation of Hamming distances for binary patterns represented in various numerical systems.” However, upon closer inspection, the proposed “novel measure” does not appear to offer substantial innovation. In computer science, binary numerical systems are widely used precisely because of their simplicity and practicality. The authors’ modification seems minor and arguably unnecessary.
2. Traditional random testing is also known for its simplicity and ease of implementation. The paper does not provide sufficient evidence to demonstrate that the proposed CRT method consistently outperforms traditional random testing in terms of test effectiveness. The differences shown in Table 10 and Table 11 are relatively small and may not be statistically significant. A more extensive set of empirical evaluations would be needed to validate such claims.
3. The paper does not include a derivation or conclusion regarding the computational complexity of Algorithm 1. This is a critical omission. I strongly recommend that the authors address this point. If the algorithm is computationally expensive, the proposed method may be impractical despite its theoretical appeal.
4. On line 56, the authors write: “we consider a set T_i = \{t_{i,0}, t_{i,1}, ..., t_{i,n-1}\}.” However, T_i is not a set but rather an ordered n-tuple, since the order of elements clearly matters in the context. The terminology should be corrected to reflect this.
Author Response
Dear Reviewer,
Thank you very much for your careful review and constructive comments on our manuscript. We have carefully revised the paper according to your suggestions and prepared a detailed point-by-point response, which is attached as a PDF file. We believe that the revisions have significantly improved the clarity and quality of the manuscript.
Sincerely,
The Authors
Author Response File: Author Response.pdf
Reviewer 4 Report
Comments and Suggestions for AuthorsThis paper proposes a method for generating controlled random tests with a specified Hamming distance to enhance the diversity and effectiveness of test sets for computing systems, particularly memory devices. The proposed algorithm (Algorithm 1) can directly generate a test set with a preset minimum Hamming distance without screening a large number of candidate patterns, significantly reducing the computational complexity. The following comments for the authors to consider.
- The current method mainly targets binary systems. Although non-binary representation is mentioned, the actual generation is still based on binary blocks. It would be necessary to generate real non-binary symbols and applied in the experiments.
- The paper utilizesrandom padding to handle the remaining bits, which may introduce uncertainty. The authors need to justify how to ensure the distance constraint and reduce uncertainty.
- The experimental analysis and comparison needs to be further consolidated.
Overall, this paper addresses an interesting issue in the field of controllable random test generation, proposing a computationally efficient, theoretically rigorous, and experimentally validated method. It is particularly suitable for scenarios such as memory testing and embedded systems that require high test diversity and efficiency. The method is versatile and scalable, providing a solid foundation for subsequent research.
We recommend acceptance for publication after addressing the above-mentioned issue. We encourage the authors to further explore extensions to non-binary systems and comparisons with other modern test generation methods in future work.
Author Response
Dear Reviewer,
Thank you very much for your careful review and constructive comments on our manuscript. We have carefully revised the paper according to your suggestions and prepared a detailed point-by-point response, which is attached as a PDF file. We believe that the revisions have significantly improved the clarity and quality of the manuscript.
Sincerely,
The Authors
Author Response File: Author Response.pdf
Round 2
Reviewer 1 Report
Comments and Suggestions for AuthorsNone.
Reviewer 3 Report
Comments and Suggestions for AuthorsThe author has carefully addressed all my comments, and I am satisfied with the responses. The revised version demonstrates significant improvement in both writing quality and the clarification of factual details. I recommend accepting this paper.
