Improving Variabilty Analysis through Scenario-Based Incompatibility Detection

Round 1

Reviewer 1 Report

This paper proposes an extension of the authors' approach, named SeVaTax, for analyzing variability models involving a well-defined variability analysis process. This process includes analyzing variability using a comprehensive set of scenarios, which allows a designer to detect (and even correct in some cases) different incompatibilities.

General Comments

Figure 1: It is a good idea to place the figure close to the text snippet that presents it.

What is the level of completeness of the validation scenarios that exist today in the query component?

More than that, is there problem domain or application independence (i.e., finance, healthcare, retail)? If not, what are the cost and steps (process, method, methodology?) to add the "infrastructure" (or control mechanisms) needed to support more problem domains?

Table 2 is not a table, right?

The experiments were focused on the verification of the main characteristics of the proposal. However, apart from a check, I think that for this issue at hand - CASE tools to support variability analysis in SPL development - it is very important and relevant that the experiments be focused on validation as well.

In other words, from the point of view of Software Engineering, we practice some experiments to validate the behaviors, costs, and journey (or experience) of the main user in using the proposed tool.

Acceptance testing, systems testing, end-to-end testing all collaborate on this. But understanding the journey cost of using the proposed tool is perhaps more important than having a high accuracy rate, for example, because the cost (effort, time, etc.) of configuring and running the tool in a real case can end up being prohibitive.

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 2 Report

The manuscript proposes a method to improve variability analysis in the software product line development. Several scenarios in software product line development are identified, and the validation support (e.g., method, algorithm, and tools) is designed and implemented to detect redundancies, anomalies, and inconsistencies correspondingly. Two evaluations, including accuracy and coverage, are also performed to show the efficacy of the proposed method.

Overall, the manuscript is well-organized and well-written. Readers can get insight into the issues in software product line development and the validation method for variability proposed by the manuscript.

Some suggestions are listed as the following.

1. Please check "Wang2 etal." in line 181.

2. Please check "SevaTax" in line 204.

3. It is suggested that the authors can elaborate on the definitions of global variation point and specific variation point. Thus, readers can get the insight of them without foraging previous studies.

4. Please check and refine the CNF Translation and CNF Numeric Translation in Table 3. For instance, there is a symbol 'a' in CNF Translation.

5. Please check and refine the scope of code segment in all the pseudo code. Proper indentation or curly braces can be used to improve the readability.

6. Please check "Figure 10" in line 617. It should be "Figure 14".

Author Response

Please see the attachment.

Author Response File: Author Response.pdf