BAG-DSM: A Method for Generating Alternatives for Hierarchical Multi-Attribute Decision Models Using Bayesian Optimization

Round 1

Reviewer 1 Report

the paper presents the method BAG-DSM (Bayesian Alternative Generator for Decision Support Models),  a model for generating alternatives for qualitative multi-attribute decision models. BAG-DSM aids the search through the decision space of DEX decision support models. 

The paper is well written and sound. The experiments are very interesting. They add value to the work and demonstrate the effectiveness of the approach.

minor spell check is required, proofread the paper.

Overall the work deserves to be accepted

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 2 Report

Reviewer comments

The author proposed a method to generate alternative solutions to a problem and assist with the decision support model. The authors mention the generation of neighbouring sets of attributes with the alternative generator that avoids generating known alternatives. The reviewer would like to know how this will affect the correlations between different attributes (inputs), i.e., inputs A and B should be both increased or decreased at the same time or one is increased while the other is decreased, to the objective function.

The reviewer would like to know how the weighting factors w_i are decided?

The reviewer would like the author to elaborate on the section where the neighbourhood relation is defined and the alternatives x1 and x2 are considered. The reviewer would like to further understand how the proposed model decides/sets the boundaries or limits for the neighbouring alternatives.

In the experiments section, the authors generated 23 alternatives to start with. How can one make sure that the global optimal solution falls within these alternatives? If the objective functions of these alternatives are too far from the optimal value then how can one be sure that searching around the neighbouring alternatives can improve the objective function?

The BAG-DSM method seems to rely on the initial alternatives. This may limit the outcomes to the local maxima or minima. Can the author please elaborate on this point?

Other comments:

·      On line 95, the authors mention that ‘the number of possible alternatives is close to 3.5 billion’ and in the brackets it is three raised to the power of twenty (3^20). These are totally different. Can the author please explain?

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 3 Report

The article is devoted to solving  topical issues in the framework of a general probem of multi-attribute decision making where decision alternatives are evaluatedby multi-criteria models. The approach offered by the authors (named as BAG-DSM- Bayesian Alternative Generator for Decision Support Models) allows us to generate new alternatives that demand least change in the current alternative to obtain a desirable outcome. The advantages of this approach are illustrated by the way of comparison to baseline method on 42 different benchmark models and one real-life model. The results of numerical experiments show that BAG-DSM outperforms the baseline by such important indices as: time to obtain the first appropriate alternative, number of generated alternatives, number of attribute changes required to reach the generated alternatives.

 In my opinion, the offered method is sufficiently attractive and useful in many applications of multi-attribute decision making support.

Some remarks:

1. Usually Introduction contains a short description of  the article structure (i.e. enumeration of its Sections).

2. The Figures 11, 12, 13 (pages16, 17) are not completely understandable. What do the blue lines mean ? How are the corresponding alternatives compared ? 

3. When you evaluate the distance between the generated and current alternatives,  you answer the question: which decision (alternative) is better. Butthe question remains: what is the desirable outcome (i.e. the best alternative) ?

4/ There are some typos in the text ( Line 197- it should be: Figure 3; Line 481- should be: negative change; Line 676- should be: better). 

Author Response

Please see the attachment.

Author Response File: Author Response.pdf