Empirical Comparison of Higher-Order Mutation Testing and Data-Flow Testing of C# with the Aid of Genetic Algorithm

Round 1

Reviewer 1 Report

The article discusses an empirical comparison between data-flow testing and higher-order mutation testing in detecting faults in C# programs. The study uses search-based algorithms to generate test sets and evaluates the subsumption relation between the testing criteria. The study concludes that second-order mutation testing is "ProbBeter" than all du-pairs data flow testing, with a significantly higher failure detection efficiency. The paper introduces the concept of "ProbBeter" and "ProbSubsumes" relationships between testing criteria and proposes a phased approach that includes a program analyzer, mutant generator, and test data generator module. The experimental results demonstrate the effectiveness of the proposed approach in generating test cases that satisfy different coverage criteria for the given program or its mutant.

There are some areas of improvement that can be suggested for the article. Firstly, the study is limited to only two testing techniques - data-flow testing and higher-order mutation testing. Including other techniques and comparing them with the existing ones could provide more insight into their effectiveness and help make a more informed decision about which testing technique to use for a given program.

Secondly, the study acknowledges that the approach proposed in the paper needs to be tested and validated further. It would be beneficial to have more experiment results from other settings to obtain a better understanding of the proposed approach and its potential applications.

Lastly, the paper could benefit from a more comprehensive discussion about the limitations of the proposed approach and how these limitations are addressed. A detailed analysis of the potential drawbacks and drawbacks of the approach, as well as its benefits and benefits for certain scenarios, could help readers better understand the rigor and limitations of the study.

Author Response

Dear Reviewer,

Please check the attached PDF file, which includes the authors' replies on your comments.

Thanks

Author Response File: Author Response.pdf

Reviewer 2 Report

The quality of the paper is poor, and the authors need to carefully read whether their paper is correct before submitting it, such as:

s. finally, the

In Error! Reference source not found.

. Error! Ref- 240

erence source not found. gives the p e. Error! Reference source not found. show

be adequate with respect to c2"..

program to be tested .

I can not find the Figure 2.

Incomplete display of title and content in Figure 1

 

2. There are a lot of Syntax errors in the paper, and the author needs polishing

 

3. Need to list the core experimental parameters of GA

4. Need to list the objective function

5. Unified writing 

Data-Flow 

Higher-Order Mutation

 

There are a lot of Syntax errors in the paper, and the author needs polishing

Author Response

Dear Reviewer,

Please check the attached PDF file, which includes the authors' replies on your comments.

Thanks

Author Response File: Author Response.pdf

Reviewer 3 Report

REVIEW COMMENTS

Applied Sciences

 

Manuscript ID: applsci-2463396

 

Title:    Empirical Comparison of Higher-Order Mutation Testing and Data-Flow Testing of

C# with the Aid of Genetic Algorithm

 

·         In Abstract, line 11, It is suggested to brief the motivation behind the work “to compare data flow and higher-order mutation.”

·         How did the authors develop the test data could be explained in Abstract with one or two lines.

(“test data developed for each. Line Number 13)

·      In Table 1, Proper citation is required.

·      Figure 1, must show all the letters clearly, Similarly in Fig.1 Caption, “Overall chart of the proposed ap-”……..Looks like incomplete….

·      “Figure. 2 shows the architecture of the proposed approach. In the following, (in line155, 219) but Figure 2 is missing in the paper.

·      In Table 2, “LOC” must be expanded when it is use for first time in the paper.

·      All equations must follow the Journal guidelines and be legible to read.

·      In GA parameters setup (section 4.3. GA parameters setup line-230), The authors are expected to explain how these parameters were considered for their study.

·      Line number 243 “number of 2nd-order mutants (in column”…………and line number 244  are to be in same line

      “4), the number of killable 2nd-order mutants (in column 5), the number of killable”

·      The authors are suggested to explain in “Comparison hypotheses (in Section 4.4)” that the reason for choosing these hypothesis.

·      The Table Numbers must be thoroughly checked throughout the paper, since In section 4.3. the Table 1, Table 2, repeats.

·      In 4.5. Experimental Results, Why the author has used the sub section numbers as (1. i,  2. ii, 3. iii)

·       In Conclusion,(line 341) It is expected to infer the results thoroughly (not simply by showing what the values they got out of the research, but why and how the change of parameter setting will influence these results…… must be stated clearly)..

Comments for author File: Comments.docx

Author Response

Dear Reviewer,

Please check the attached PDF file, which includes the authors' replies on your comments.

Thanks

Author Response File: Author Response.pdf

Reviewer 4 Report

The topic is interesting, but I have important observations that must be addressed in order for the article to be publishable.

- Error: Page 3, reference to figure 1 is “Error! Reference source not found” . Correct it. Same thing on page 7, page 8, page 9, page 10 with references to other figures or tables.

- Wrong figure label: On the page 3 and on the page 4 are different figures but with the same label Figure 1. Re-number figures labels.

- Figure problem: Figure 1, page 4 the text is not aligned and cannot be read. Align the text in the figure !

- Title wrong placed: Page 5. Move the title of section 4 to the next page!

- Clarification request: What is the objective function of the genetic algorithm? Present it and explain it (in section 3.3). That function is used to compute fitness value for each individual – it is an important link between the two testing methods and final conclusions and results.

- Clarification request: What coding schema did you apply? Put in a figure the structure of the chromosome and explain it.

- Clarification request: It is not clear to me what you determine with the genetic algorithm - I consider the lack of explanations in this regard to be a major lacuna in the paper. Probably the answer to the two previous observations will clarify the problem.

Author Response

Dear Reviewer,

Please check the attached PDF file, which includes the authors' replies on your comments.

Thanks

Author Response File: Author Response.pdf

Round 2

Reviewer 2 Report

. finally, the conclusion  -> Finally

The number of Figure 2 is incorrect

 

edit by professional language

Author Response

Dear Reviewer,

Thanks so much for your comments that improved our manuscript.

Please, check the attached PDF file which includes the reply on your comments.

 

Author Response File: Author Response.pdf

Reviewer 4 Report

The authors did not respond to my comments, or if they did, they did not include the responses in revision 2 of the paper. I have attached a file in which I have pointed out exactly what to do.

Comments for author File: Comments.pdf

Author Response

Dear Reviewer,

Thanks so much for your comments that improved our manuscript.

Please, check the attached PDF which includes the replying on your requested observations.

 

Author Response File: Author Response.pdf