A Measuring Method Based on Graph Structure for Decision-Making Complexity in Major Science and Technology Projects

Round 1

Reviewer 1 Report

This paper explores how to characterize, describe and measure the complexity of decision analysis and decision management in large science and technology projects, and introduces complexity theory to the study of decision-making in large science and technology projects. The authors propose a novel graph structure-based decision complexity measurement method, aiming to provide a theoretical basis for decision-makers of large science and technology projects to implement decision complexity management. However, it still needs further improvement. I provide some suggestions below for the revised version:

1.       In Page 2, it is suggested to introduce the decision complexity measurement method proposed in this paper in more detail to show the principle and application scenarios of the method more visually.

2.       In Page 2, in ‘accurate information of nonlinear organizations, Managers can only make decisions’, ‘Managers’ should be ‘managers’.

3.       In Page 2, ‘that were not considered as engineering project phenomena’ should be ‘that may not be traditionally regarded as typical engineering project phenomena’ to increase rigor.

4.       In Page 2, suggest changing ‘make the system complex’ to ‘characterize the complexity of the system’, to more accurately express the meaning.

5.       In Page 3, in ‘typical characteristics of complex system’, ‘complex system’ should be ‘complex systems’.

6.       In Page 3, suggest changing ‘of cross region, cross department, cross discipline and cross industry’ to ‘from diverse regions, departments, disciplines, and industries’, to be more in line with the rules of English grammar.

7.       In Page 4, Figure 1 is so complex that it is difficult to fully grasp the meaning. And at the far right, what is ‘subject iveautonomy’?

8.       In Page 5, ‘consensus on the consensus’ should be ‘agreement on decision-making options’ to clarify the meaning more precisely.

9.       In Page 6, ‘the combination of nodes and connected edges’ should be ‘a combination of nodes and connecting edges’, to be more in line with the rules of English grammar.

10.     It is recommended to provide some numerical examples to show the good nature of the model.

11.     It is suggested to further explain how the findings of this paper can be applied to real-life large-scale science and technology projects for better understanding by the reader.

12.     Some current related works are suggested to discuss, e.g., A complex weighted discounting multisource information fusion with its application in pattern classification; Generalized divergence-based decision making method with an application to pattern classification; EFMCDM: Evidential fuzzy multicriteria decision making based on belief entropy; GEJS: A generalized evidential divergence measure for multisource information fusion.

This work is interesting. I recommend accepting this paper after addressing the above revisions.

Author Response

Response to Reviewer 1 Comments

Point 1: In Page 2, it is suggested to introduce the decision complexity measurement method proposed in this paper in more detail to show the principle and application scenarios of the method more visually.

Response 1: According to the reviewer's comments, we have systematically summarized literature review of the decision complexity measurement method to ensure that the principles and application scenarios of this method is fully presented.

Point 2: In Page 2, in ‘accurate information of nonlinear organizations, Managers can only make decisions’, ‘Managers’ should be ‘managers’.

Response 2: According to the reviewer's comments, we have made modifications.

Point 3: In Page 2, ‘that were not considered as engineering project phenomena’ should be ‘that may not be traditionally regarded as typical engineering project phenomena’ to increase rigor.

Response 3: According to the reviewer's comments, we have made modifications.

Point 4: In Page 2, suggest changing ‘make the system complex’ to ‘characterize the complexity of the system’, to more accurately express the meaning.

Response 4: According to the reviewer's comments, we have made modifications.

Point 5: In Page 3, in ‘typical characteristics of complex system’, ‘complex system’ should be ‘complex systems’.

Response 5: According to the reviewer's comments, we have made modifications.

Point 6: In Page 3, in ‘typical characteristics of complex system’, ‘complex system’ should be ‘complex systems’.

Response 6: According to the reviewer's comments, we have made modifications.

Point 7: In Page 4, Figure 1 is so complex that it is difficult to fully grasp the meaning. And at the far right, what is ‘subject iveautonomy’?

Response 7: Due to the limited space of the block diagram in Figure 1, we only provided some key words, which made it difficult to understand. The text below Figure 1 provides a detailed explanation. At the bottom right corner of the graph, 'subject iveautonomy' should be 'subject autonomy'.

Point 8: In Page 5, ‘consensus on the consensus’ should be ‘agreement on decision-making options’ to clarify the meaning more precisely.

Response 8: According to the reviewer's comments, we have made modifications.

Point 9: In Page 6, ‘the combination of nodes and connected edges’ should be ‘a combination of nodes and connecting edges’, to be more in line with the rules of English grammar.

Response 9: According to the reviewer's comments, we have made modifications.

Point 10: It is recommended to provide some numerical examples to show the good nature of the model.

Response 10: According to the reviewer's comments, we Added typical case analysis to verify the operability of the model through numerical calculations.

Point 11: It is suggested to further explain how the findings of this paper can be applied to real-life large-scale science and technology projects for better understanding by the reader.

Response 11: According to the reviewer's comments, we have used the the space-ground integrated information network, sourced from one of the major science and technology projects in China's technological innovation 2030, as a typical case to verify the utility of the decision complexity measurement method through numerical calculations.

Point 12: Some current related works are suggested to discuss, e.g., A complex weighted discounting multisource information fusion with its application in pattern classification; Generalized divergence-based decision making method with an application to pattern classification; EFMCDM: Evidential fuzzy multicriteria decision making based on belief entropy; GEJS: A generalized evidential divergence measure for multisource information fusion.

Response 12: According to the reviewer's comments, we have conducted sufficient literature collection and found that the research on decision complexity is still in the preliminary exploration stage, and related research results are relatively scarce. The research on complexity measurement theory and methods mainly focuses on organization complexity, management complexity, and project complexity. In view of this, we have rewritten the literature review for these four aspects.

Reviewer 2 Report

1) abstract is a one super long oddly worded sentence. I’ve never seen such a long sentence. Break it down into at least five smaller sentences. Furthermore, in this one sentence abstract the “science and technology project” is repeated five times without it being described. So explain the context of your paper “decisions in science and technology projects” a bit better in the abstract. 

2) There are some bold but inappropriate sentences such as “The construction of major science and technology projects is not accomplished overnight.” Which lack substance and can be deleted.

3) The literature review is nowhere near enough. Consider notions and works on “system complexity”, and “task complexity” to uncover more related papers to decision complexity.

4) Section 5 is very unclear rather incomplete, and does not connect well with Section 4 which is pretty clear. For example it sates that “each node in the tree diagram in Figure 1 represents a node in the diagram structure, and the branch in Figure 1 represents the connected edge in the diagram structure, which can build the structure model of project decision complexity diagram.” Does this include the node “decision complexity” as well? What is n? is it the number of nodes? If so it is a know factor?!! This section needs to be rewritten neatly and as it is feels out of the paper’s context.

5) Please add a case study to demonstrate the utility of your measure. 

Author Response

Response to Reviewer 2 Comments

Point 1: Abstract is a one super long oddly worded sentence. I’ve never seen such a long sentence. Break it down into at least five smaller sentences. Furthermore, in this one sentence abstract the “science and technology project” is repeated five times without it being described. So explain the context of your paper “decisions in science and technology projects” a bit better in the abstract.

Response 1: According to the reviewer's comments, we have made new revisions to the abstract.

Point 2: There are some bold but inappropriate sentences such as “The construction of major science and technology projects is not accomplished overnight.” Which lack substance and can be deleted.

Response 2: According to the reviewer's comments, we have made modifications.

Point 3: The literature review is nowhere near enough. Consider notions and works on “system complexity”, and “task complexity” to uncover more related papers to decision complexity.

Response 3: According to the reviewer's comments, we have conducted sufficient literature collection and found that the research on decision complexity is still in the preliminary exploration stage, and related research results are relatively scarce. The research on complexity measurement theory and methods mainly focuses on organization complexity, management complexity, and project complexity. In view of this, we have rewritten the literature review for these four aspects.

Point 4: Section 5 is very unclear rather incomplete, and does not connect well with Section 4 which is pretty clear. For example it sates that “each node in the tree diagram in Figure 1 represents a node in the diagram structure, and the branch in Figure 1 represents the connected edge in the diagram structure, which can build the structure model of project decision complexity diagram.” Does this include the node “decision complexity” as well? What is n? is it the number of nodes? If so it is a know factor?!! This section needs to be rewritten neatly and as it is feels out of the paper’s context.

Response 4: In response to the questions raised by the reviewer, we have rewritten Section 5 to ensure that the revision requirements are met.

Point 5: In Page 3, in ‘typical characteristics of complex system’, ‘complex system’ should be ‘complex systems’.

Response 5: According to the reviewer's comments, we have made modifications.

Point 6: Please add a case study to demonstrate the utility of your measure.

Response 6: According to the reviewer's comments, we have used the the space-ground integrated information network, sourced from one of the major science and technology projects in China's technological innovation 2030, as a typical case to verify the utility of the decision complexity measurement method through numerical calculations.

Round 2

Reviewer 2 Report

1) Still problems with abstract’s English, you can use this if it helps:

Unlike general large-scale projects, major science and technology projects (MSTPs) are strategically positioned to meet national needs, reflecting the forward-looking direction of science and technology development. The correctness of decision-making for MSTPs is crucial for the long-term development and strategic interests of the country. To measure decision complexity accurately, we propose a graph-based approach that utilizes information entropy theory. This approach provides decision-makers with a theoretical foundation for managing decision complexity effectively.

2) For Figure 1 consider categorising the nodes into perspectives (top and middle nodes) and measures (lower nodes). Some of the combinations of the top nodes don’t make sense for example what are “decision objective” + “agents”? Furthermore some of the measures only apply to specific perspectives. Please use the framework here to distinguish between measures and perspectives: Efatmaneshnik, Mahmoud, and Michael J. Ryan. "A general framework for measuring system complexity." Complexity21.S1 (2016): 533-546.

After that provide a list of perspectives that make sense and for each perspective identify the measures that are applicable to the perspective.

3) Check the English in “Due to the neglect of the characteristics  of diversity, multidimensional, hierarchical, and heterogeneity, the factors and causal variables of  complexity source are simplified.”

“As shown in the following Figure 2. The characteristics of major science and technology projects depend on both the elements at various  levels and the connection relationships between them.”

Proofread your paper before submission. 

4) All of a sudden in Figure 4 MSTPs are characterised using systems thinking!!! Create an introduction in the early part of your work on systems thinking and how you are going to use it. 

5) For the case study example can you provide a scenario and a set of actions that can reduce the complexity index?

6) Compare your measure. to other measures if possible in other papers you have reviewed. 

Author Response

Response to Reviewer 2 Comments

Point 1: Still problems with abstract’s English, you can use this if it helps:

Unlike general large-scale projects, major science and technology projects (MSTPs) are strategically positioned to meet national needs, reflecting the forward-looking direction of science and technology development. The correctness of decision-making for MSTPs is crucial for the long-term development and strategic interests of the country. To measure decision complexity accurately, we propose a graph-based approach that utilizes information entropy theory. This approach provides decision-makers with a theoretical foundation for managing decision complexity effectively..

Response 1: According to the reviewer's comments, we have made modifications to the abstract.

Point 2: For Figure 1 consider categorising the nodes into perspectives (top and middle nodes) and measures (lower nodes). Some of the combinations of the top nodes don’t make sense for example what are “decision objective” + “agents”? Furthermore some of the measures only apply to specific perspectives. Please use the framework here to distinguish between measures and perspectives: Efatmaneshnik, Mahmoud, and Michael J. Ryan. "A general framework for measuring system complexity." Complexity21.S1 (2016): 533-546.

After that provide a list of perspectives that make sense and for each perspective identify the measures that are applicable to the perspective.

Response 2: According to the reviewer's comments, we have some modifications. Studying decision complexity mainly involves identifying the sources of complexity. The sources of complexity should firstly be analyzed from the constituent elements of the decision-making system, and secondly from the perspective of decision related engineering management. Based on this, our two research perspectives are formed. We first constructed the top nodes from the perspective of the main components of decision-making systems, such as decision objectives, decision makers, population structure, group consensus, and decision process. Next, we constructed middle nodes from the perspective of engineering management, such as demand, mass, agent, cost, environment, time, information, and risk. Finally, we refined and decomposed the middle nodes from measures, and constructed the lower nodes.

Regarding the construction of relationships between nodes in graph structures, especially for some meaningless combinations of top nodes for example what are “decision objective” + “agents”, we consider that major science and technology projects are complex systems that covers multiple scales, levels, involving system level, subsystem level, device level, module level, and component level. The decision objectives of each level are different and mainly depend on the interests and values of the agents at each level. Therefore, there is a correlation between decision objective and agents, and the project decision mode follows the Stackelberg master slave game theory. The specific project decision mode will be discussed in other papers in the future.

Regarding the use of recommended references: Efatmaneshnik, Mahmoud, and Michael J. Ryan. Framework A. Complexity 21. S1（2016）:533-546. We have thoroughly studied the recommended paper and learned that it provides a general framework for measuring complexity from a lifecycle perspective of systems engineering or engineered systems. The perspective in the recommended paper such as design, manufacturing/assembly, integration, maintenance, or operation, or from the perspective of any individuals or groups involved in engineering processes. Unlike recommending papers from the engineering development lifecycle, as shown above, we mainly construct top and middle layer nodes from two perspectives: one is from the main components of the decision-making system such as decision objectives, decision makers, population structure, group consensus, and decision process, and the other is from project management such as demand, mass, agent, cost, environment, time, information, and risk. Therefore, our paper is motivated by exploring the sources of decision complexity, while the recommended paper is based on the development lifecycle of systems engineering.

Table 1 A list of perspectives of decision complexity

Point 3: Check the English in “Due to the neglect of the characteristics  of diversity, multidimensional, hierarchical, and heterogeneity, the factors and causal variables of  complexity source are simplified.”

“As shown in the following Figure 2. The characteristics of major science and technology projects depend on both the elements at various  levels and the connection relationships between them.”

Proofread your paper before submission.

Response 3: According to the reviewer's comments, we have made modifications.

Point 4: All of a sudden in Figure 4 MSTPs are characterised using systems thinking!!! Create an introduction in the early part of your work on systems thinking and how you are going to use it..

Response 4: According to the reviewer's comments, we have rewritten the paragraph above Figure 2 in Section 5.

Point 5: For the case study example can you provide a scenario and a set of actions that can reduce the complexity index?

Response 5: According to the reviewer's comments, we have made modifications. In the final paragraph of the case study, we further discussed the work of reducing decision complexity and analyzed the risk control scenario as an example, supplementing relevant content.

Point 6: Compare your measure. to other measures if possible in other papers you have reviewed.

Response 6: There is still no consensus on how to measure complexity, and research on decision complexity is even rarer. The main reason for this phenomenon is that the theory of complexity science has not yet achieved breakthroughs and cannot truly solve complex problems such as organization, system engineering, and decision-making. Even in the paper recommended by the reviewer, the author only analyzed and compared various existing complexity measurement methods based on the monotonic increasing nature of the engineering system complexity measurement function during the engineering lifecycle stage. Due to different research backgrounds and perspectives, it is difficult to compare and evaluate various measurement methods. We will continue to pay attention to and follow up on this issue in the future.

Author Response File: Author Response.pdf

Round 3

Reviewer 2 Report

The authors have addressed the comments in a satisfactory manner.