Operation Loop-Based Optimization Model for Resource Allocation to Military Countermeasures versus Probabilistic Threat

Weapons development planning is an unstructured and complex multi-criteria decisionmaking problem, especially in antagonistic environments. In this paper, the defender’s decision was modelled as a high complexity non-linear optimization problem with limited resources. An operation loop with realistic link rules was first proposed to model the cooperation relationships among weapons in the defense system. The system dynamics principle was used to characterize the dynamic behavior of the nodes in a complex weapons network. Then, we used cumulative threat and development risk to measure different planning solutions by considering the opponent and uncertainties in the development process. Next, an improved Differential Evolution (DE) and Non-Dominated Sorting Differential Evolution (NSDE) were designed to determine the optimal planning solutions for a single objective and multi-objective. The compromise solution, based on the Technique for Order Preference by Similarity to an Ideal Solution (TOPSIS), was used to evaluate the Pareto solution set of the multi-objective. Finally, an illustrative case was studied to verify the feasibility and validity of the proposed model.


Introduction
After the 11 September 2001 terrorist attacks on the World Trade Center and the Pentagon in the United States, interest in counter-terrorism methods with limited resources has rapidly increased.However, allocating scarce defensive resources to optimize an objective function is one of the classic problems in Operations Research [1], being an important and difficult task [2].For the defense agencies in many countries, weapon system of systems (WSOS) development planning is a critical process used to develop countermeasures (CMs) to confront the threats posed by other hostile countries [3].The planning solution focuses on how to allocate resources to different weapons to effectively eliminate the threat from hostile countries.
In prior studies on weapons development planning, a decision maker (DM) seeks to maximize a single objective function, such as capability, profit, and effectiveness.For example, Gu et al. [4] established a WSOS planning model by maximizing the expected effectiveness as the optimization target.Håkenstad [5] compared the long-term defense planning systems of seven Nordic countries and found that most countries aimed to improve their own profit.Zhang et al. [6] proposed an optimization model to minimize the capability gaps to find the best development scheme by comparing different combinations of weapons, based on the given capability requirements.Throughout the literature research, however, did not mention the change of the opponent.In contrast, this paper considered minimizing the threat caused by the enemy as the objective function.Herein, we addressed an arms

Problem Formulation
Throughout history, two or more countries are often in hostilities, such as the United States and the Soviet Union during the Cold War, India and Pakistan, and so on.Therefore, WSOS development planning transforms from a capacity to a military demand, as seen as an arms race.The R&D of national military weapons occurs in a situation requiring rapidity of development.Each country will set the other country or countries as the "reference" or "opponent" and change their own R&D strategies following the change of the R&D strategies of the opponent.Herein, we considered the dynamic competition between two parties, denoted as the attacker (R) and the defender (B).Generally speaking, R is a military power with advanced military technology and in a leading position in the military.R will independently develop a set of new weapons early on to attack B to gain an advantage at time t R1 .The weapons developed by R are targets that B must counter.If all information about R is obtained by the intelligence department of B completely and opportunely at time t R1 , then the initial damage (d 1 ) caused by R, typically measured in casualties and economic damages [15], to B is known.As shown in Figure 1a, to defend the homeland from being attacked by R, B must develop new weapons (B n ) as CMs to mitigate the damage.However, as shown in Figure 1b, if B cannot intercept information about R's new weapon or B's CMs fail, the threat will remain.
Appl.Sci.2018, 8, x FOR PEER REVIEW 3 of 25 information or energy, and certain nodes could only be arranged in descending or ascending order.This phenomenon is common in real-life operational battles.In summary, this problem can be modelled as a constrained nonlinear optimization problem, which is known as NP-hard.The complexity of this type of problem depends on the complex relationships among weapons.Consequently, we argue that the analytical solution to the problem is not feasible, and a heuristic-based optimization algorithm would be effective.In this article, we used Differential Evolution (DE) and Non-Dominated Sorting based Genetic Algorithm-II (NSGA-II) for our optimization tasks.
The main contributions of this paper are as follows: (1) When assessing the threat posed by the enemy, we not only considered the cooperation among different nodes based on the traditional ideology of the operation loop, but also the real link rules of these nodes.Meanwhile, the system dynamics principle was used to characterize the dynamic behavior of the nodes in the complex weapons network; (2) According to the characteristics of the weapon life cycle curve, we considered some realistic constraint conditions, such as the annual weapons budget, the total weapons budget, and the weapon planning cycle; (3) A complex multi-objective planning problem and the related solving approach were proposed, considering the development risks and probabilistic threat from the enemy.In this paper, we considered practical settings in realistic dynamic competition.
This paper was organized as follows.Section 2 introduced the modeling method based on operation loop and the threat assessment process.Section 3 elaborated upon the single objective optimization model with time and investment constraints.In addition, the algorithm to solve the problem was outlined.Section 4 extended the objective functions to an actual weapons R&D process.Section 5 demonstrated the usefulness of the models by presenting the results of a numerical experiment.Finally, Section 6 concluded the paper and discussed possible future research.

Problem Formulation
Throughout history, two or more countries are often in hostilities, such as the United States and the Soviet Union during the Cold War, India and Pakistan, and so on.Therefore, WSOS development planning transforms from a capacity to a military demand, as seen as an arms race.The R&D of national military weapons occurs in a situation requiring rapidity of development.Each country will set the other country or countries as the "reference" or "opponent" and change their own R&D strategies following the change of the R&D strategies of the opponent.Herein, we considered the dynamic competition between two parties, denoted as the attacker (R) and the defender (B).Generally speaking, R is a military power with advanced military technology and in a leading position in the military.R will independently develop a set of new weapons early on to attack B to gain an advantage at time tR1.The weapons developed by R are targets that B must counter.If all information about R is obtained by the intelligence department of B completely and opportunely at time tR1, then the initial damage (d1) caused by R, typically measured in casualties and economic damages [15], to B is known.As shown in Figure 1a, to defend the homeland from being attacked by R, B must develop new weapons (Bn) as CMs to mitigate the damage.However, as shown in Figure 1b, if B cannot intercept information about R's new weapon or B's CMs fail, the threat will remain.The collection of different weapons is called WSOS.Modern weapons are complex, integrated, and diverse.The connections among weapons also vary.Therefore, we not only considered interdependency among the current weapons, but also relationships among all current and future weapons.In this paper, the concept of meta-functional nodes and meta-functional edges, based on the operation loop, was used to model the cooperation relationships among weapons.

Meta-Functional Node Modeling
A meta-functional node can be represented as a three tuple, containing a node identity, node type, and node threat vector, as shown below: The node identification indicates the position of the meta-functional node in combat operation process, such as red or blue.The node type refers to the classification of the meta-functional node, which can be divided into different categories from different angles.Cares [19] divided nodes into sensor, decider, influencer, and target according to the different roles on the battlefield.In this study, meta-functional nodes were divided into four categories: scout, command-and-control, influence, and target, which are expressed as follows: where S, D, I and T represent the meta scout functional node, meta command-and-control functional node, meta influence functional node, and meta target functional node, respectively.Node S refers to the weapon, equipment, entity, or system that performs basic surveillance, reconnaissance, and warning in a battle, such as radar and infrared detection systems.Node D is a system that performs basic command and control tasks during a combat operation.Node I is a device, entity, or system that performs basic fire strikes or electromagnetic interference during a battle, such as artillery, missiles, and torpedoes.Different nodes are included in both the same and different weapons.The collection of different weapons is called WSOS.Modern weapons are complex, integrated, and diverse.The connections among weapons also vary.Therefore, we not only considered interdependency among the current weapons, but also relationships among all current and future weapons.In this paper, the concept of meta-functional nodes and meta-functional edges, based on the operation loop, was used to model the cooperation relationships among weapons.

Meta-Functional Node Modeling
A meta-functional node can be represented as a three tuple, containing a node identity, node type, and node threat vector, as shown below: The node identification indicates the position of the meta-functional node in combat operation process, such as red or blue.The node type refers to the classification of the meta-functional node, which can be divided into different categories from different angles.Cares [19] divided nodes into sensor, decider, influencer, and target according to the different roles on the battlefield.In this study, meta-functional nodes were divided into four categories: scout, command-and-control, influence, and target, which are expressed as follows: where S, D, I and T represent the meta scout functional node, meta command-and-control functional node, meta influence functional node, and meta target functional node, respectively.Node S refers to the weapon, equipment, entity, or system that performs basic surveillance, reconnaissance, and warning in a battle, such as radar and infrared detection systems.Node D is a system that performs basic command and control tasks during a combat operation.Node I is a device, entity, or system that performs basic fire strikes or electromagnetic interference during a battle, such as artillery, missiles, and torpedoes.Different nodes are included in both the same and different weapons.For example, an armored observation and command vehicle can be characterized as node S and node D.    Considering the antagonism between the two sides, B will set all of R's nodes as the target.Similarly, R will set all of B's nodes as the target.Thus, 36 (node number × node number) different combinations of the three kinds of meta-functional nodes exist.Some types of meta-functional edges, however, do not exist in practical operation.Based on the following assumptions, we filtered the 36 edge types: (1) the node S cannot act independently; it can only share information with other S or D nodes; (2) the node D cannot attack the enemy target directly; and (3) the node I will not interfere or attack its own weapons.After applying these filters, 20 edge relationships remained in the WSOS network under the antagonistic situation, as shown in Table 1.

Operation Loop Modeling
Modern warfare theory, which is also called Observe Orient Decide Act (OODA) theory [20], argues that a combat operation is a cyclic process.First is target discovery (T→S), then information is delivered to the node D (S→D).Next, node D analyzes the battlefield situation and charges node I (D→I) to attack node T (I→T).The close loop (T→S→D→I→T) is called the operation loop [21].During the operation, both parties build their own operation loops, and treat all the opponent's nodes as targets, as shown in Figure 3.Although only four classes of meta-functional nodes exist in one operation loop, each type of node can contain more than one weapon.Different weapons cooperate with others through loops.The number and technical performance of loops directly impact the combat effectiveness.The greater the number of loops, the stronger the threat capability.The fewer the nodes and edges in a loop, the better the threat effect.

Meta-Functional Edge Modeling
Considering the antagonism between the two sides, B will set all of R's nodes as the target.Similarly, R will set all of B's nodes as the target.Thus, 36 (node number × node number) different combinations of the three kinds of meta-functional nodes exist.Some types of meta-functional edges, however, do not exist in practical operation.Based on the following assumptions, we filtered the 36 edge types: (1) the node S cannot act independently; it can only share information with other S or D nodes; (2) the node D cannot attack the enemy target directly; and (3) the node I will not interfere or attack its own weapons.After applying these filters, 20 edge relationships remained in the WSOS network under the antagonistic situation, as shown in Table 1.Modern warfare theory, which is also called Observe Orient Decide Act (OODA) theory [20], argues that a combat operation is a cyclic process.First is target discovery (T→S), then information is delivered to the node D (S→D).Next, node D analyzes the battlefield situation and charges node I (D→I) to attack node T (I→T).The close loop (T→S→D→I→T) is called the operation loop [21].During the operation, both parties build their own operation loops, and treat all the opponent's nodes as targets, as shown in Figure 3.Although only four classes of meta-functional nodes exist in one operation loop, each type of node can contain more than one weapon.Different weapons cooperate with others through loops.The number and technical performance of loops directly impact the combat effectiveness.The greater the number of loops, the stronger the threat capability.The fewer the nodes and edges in a loop, the better the threat effect.

Node Threat Vector
In this study, we applied the concept of accumulative damage presented by Golany [15], which is an objective function that measures the threat posed by the opponent.To assess the threat of the

Node Threat Vector
In this study, we applied the concept of accumulative damage presented by Golany [15], which is an objective function that measures the threat posed by the opponent.To assess the threat of the whole WSOS, we first analyzed the node threat vector.Different types of nodes have different functions, and the corresponding threat vectors vary considerably.Based on prior studies [22][23][24], we improved the index system of different nodes.The threat vector of the node S is: where Radius, Accuracy, RPG and Mobility indicate the operational coverage radius, the hitting accuracy, the ammunition quantity, and the maneuvering speed of the node I, respectively.Λ means that the vector can be extended to different weapons.

Tactical and Technical Index Normalization
The different indicators have different dimensions.For example, the unit of the operational coverage radius is km, and the unit of the maneuvering speed is km/h.Therefore, different index values have to be standardized.The quantitative indexes are divided into two categories: the benefit type and the cost type.The greater is the value of the benefit index, the greater is the threat.The greater is the value of the cost index, the lesser is the threat.For example, the higher is the hitting precision of the node S, the os higher the threat.The shorter is the decision time of the node D, the sooner the attack can be implemented, resulting in a greater threat.The processing methods of the two quantitative indicators are as follows: where Equation ( 6) is the standardized function of the dimensionless benefit index and Equation ( 7) is the standardized function of the dimensionless cost index.I is the original index.I max and I min are the worldwide maximum and minimum values of the current index, respectively.As for the benefit index, if the value of I reaches I max , the score of the index is 1.Otherwise, the score is 0. The same logic applies to the function symbol of the cost index.

Threat Assessment Based on Operation Loop
The basic goal of the WSOS is to strike the opponent targets and reduce the opponent combat capability, until that capability is completely lost.Hence, threat assessment should emphasize the influence on enemy targets.Each operation loop in the WSOS represents a method to strike the enemy target.As a result of the differences in the performance of the weapons within the operation loop, the threat to enemy targets varies.After normalization of the index value, we classified and aggregated the threat value of the various nodes.The weighted sum method was used in this paper, as it is the most commonly used index aggregation method.The method assigns weight w j to each index, and then weights the sum to obtain the threat value d j of one node.
An operation loop contains the node set S, the node set D, and the node set I. The threat to the opponent target of the entire operation loop is where d s , d d , and d i represent the threat vector aggregation value of node S, node D, and node I, respectively.The total threat to the opponent is the sum of all loops.

Initial Threat Rate and Threat Accumulation
The goal of the development of new weapons is to reduce the threat from the opponent.The threat that changes over time is determined by the performance of one's own weapons.For example, Figure 4 shows the dynamic change in the threat from R during the development process of B. The horizontal axis represents time and the vertical axis represents the threat.The red squares represent the deployment of R's weapons and the blue dots represent the deployment of B's weapons.The threat from R increases after R's new weapon R 1 is deployed.Then, the threat accumulates for a period of time.Once B has developed new weapons against R, R's threat level decreases.Then, R develops new weapons to increase the threat again.That is to say, the threat changes with the deployment of new weapons, and accumulates over time.B selects B 1 , B 2 , and B 3 from the optional weapons collection.B 1 and B 2 form operation loops that cover R with other weapons, and the threat from R decreases.Although B develops B 3 , it cannot form an operation loop that covers R. Therefore, the threat from R remains unchanged.
In summary, B must choose the most suitable weapons against R, and rationally plan the development.In addition, weapons must be deployed as soon as possible to neutralize the threat that accumulate with time.
Appl.Sci.2018, 8, x FOR PEER REVIEW 7 of 25 most commonly used index aggregation method.The method assigns weight wj to each index, and then weights the sum to obtain the threat value dj of one node.
An operation loop contains the node set S, the node set D, and the node set I. The threat to the opponent target of the entire operation loop is where ds, dd, and di represent the threat vector aggregation value of node S, node D, and node I, respectively.The total threat to the opponent is the sum of all loops.

Initial Threat Rate and Threat Accumulation
The goal of the development of new weapons is to reduce the threat from the opponent.The threat that changes over time is determined by the performance of one's own weapons.For example, Figure 4 shows the dynamic change in the threat from R during the development process of B. The horizontal axis represents time and the vertical axis represents the threat.The red squares represent the deployment of R's weapons and the blue dots represent the deployment of B's weapons.The threat from R increases after R's new weapon R1 is deployed.Then, the threat accumulates for a period of time.Once B has developed new weapons against R, R's threat level decreases.Then, R develops new weapons to increase the threat again.That is to say, the threat changes with the deployment of new weapons, and accumulates over time.B selects B1, B2, and B3 from the optional weapons collection.B1 and B2 form operation loops that cover R with other weapons, and the threat from R decreases.Although B develops B3, it cannot form an operation loop that covers R. Therefore, the threat from R remains unchanged.
In summary, B must choose the most suitable weapons against R, and rationally plan the development.In addition, weapons must be deployed as soon as possible to neutralize the threat that accumulate with time.

Evaluation of Threat Reduction Effects
According to the real-life military operations, B develops new weapons to form operation loops to mitigate the threat from R. The number of operation loops represents the number of ways in which the opponent targets can be attacked.The higher the number of operation loops, the more ways are

Evaluation of Threat Reduction Effects
According to the real-life military operations, B develops new weapons to form operation loops to mitigate the threat from R. The number of operation loops represents the number of ways in which the opponent targets can be attacked.The higher the number of operation loops, the more ways are available to cover the opponent target nodes, and the better the threat reduction effect.
The WSOS network is complex dynamical network system.To research this kind of problem, the dynamical theory is usually introduced into the nodes of the complex network.In a complex network, the dynamic behavior of each node is governed by two factors: the original dynamic behavior mechanism of the node itself, and the influence of the nodes to which they are connected.Considering a complex dynamical network system that is coupled to N nodes, the general expression is [25]: where x i (t) denotes the state of node v i at time t; f (x i (t)) denotes the primitive dynamic behavior of node v i ; c is the coupling coefficient; g(x j (t)) denotes one coupling function through the coupling relationships among nodes; and A = (a ij ) N×N is the topological structure or adjacency matrix of the complex network, and every element is positive.If a ij = 0, no connection exists between node i and node j.Conversely, if a ij = 0, a relationship between node i and node j exists.Next, we combined the actual nature of the WSOS to establish a dynamic WSOS network model based on the general model.
From the analysis of the operational process outlined in Section 2.2.4,we knew that the formation of the operation loop of B reduced the threat from R. As a result, the coupling coefficient c was −1.
The coupling function g(x j (t)) was calculated using Equations ( 8) and ( 9).We assumed that the number of new operation loops was m during B's weapons development process, and only the operation loops that covered R's threat nodes reduced the threat from R. Thus, not all m operation loops reduced the threat from R. The value of a ij has the following two forms: 1, if B s new operation loops can cover R s threat nodes 0, if B s new operation loops can not cover R s threat nodes (11) In summary, the cumulative threat (D B ) expression is: where ∧ x(t) denotes the threat from the entire WSOS of R at time t; f (x(t)) denotes the initial threat caused by R, namely, the situation where B's confrontation is not considered; and denotes a measure of threat reduction.

Single Objective Modeling and Solving
Golany [7] viewed an arms race as a process between two asymmetrical groups.The advantage established by one side is temporary, and the advantage disappears when the other side exceeds it.Therefore, the issue faced by B is how to use limited resources to develop effective CMs to mitigate the threat from R as much as possible.Thus, the cumulative threat is used as the optimization objective function.To simplify the computation, we only considered the operation loops that cover the threat nodes of R. Consequently, a ij is removed from the expression.Then, all new m loops work to counter R. The initial threat from R is set as d 1 .The entire duration of the confrontation is expressed as T. The number of B's operation loops that cover R is m, and the moment of forming of the jth loop is t j .Then, the cumulative threat expression is:

Constraints
In the WSOS development planning process, B must consider the following two constraints: time and money.

Time
The cumulative threat posed by R gradually increases over time.Supposing B chooses to develop 10 new weapons.If all investments are made at time 0, then the optimal choice is that the development of the 10 new weapons is initiated at time 0. Therefore, B can deploy new weapons to counter R more rapidly.If not all investments are made simultaneously, but in batches due to the limited investments amount for each batch, B must consider the sequence of these new weapons.Therefore, the starting time and R&D time of the weapons directly impact the threat reduction.Herein, time is considered as a constraint, including the beginning and throughout the duration of the operation.
The starting time (st Bj ) of each weapon should satisfy the following relationship: The upper limit of the R&D duration should be less than the weapon development planning; otherwise, this development planning would be meaningless.The lower limit of the R&D duration depends on the technical maturity of R&D.Therefore, the R&D duration (t Bj ) should be between the shortest and longest development time length:

Investment
To date, the research on the weapons funding allocation problem is still prominent, focusing on limited budgets and how to allocate resources to different weapons.Therefore, R&D investments should be considered as a constraint, including the single weapon budget, the annual budget, and the total budget.
The upper and lower limits of R&D investment usually depend on defense funds.If the investment is too small, the weapon will fail to develop.Similarly, the investment has an upper limit that cannot be more than the assigned national defense funds.Therefore, the R&D investment (c Bj ) of each weapon should be between the minimum and maximum investments: The total investment for all new weapons should be lower than the total budget (C): Although the total investment was confirmed prior to creating the development plan, the R&D funds are often used in batches because of a long development period of many years or even decades.This article focused on a five-year planning problem.Thus, the total investment C was divided into average usage over five years.A strict capital flow restriction was placed on the annual budget.Moreover, the unused budget for one year could not be carried over to the next year.We assumed that the funds are continuously and evenly invested in the development of one weapon.As multiple weapons were being developed in one year, the annual investment in the Hth year should not exceed C/5:

Modeling the Relationship between Time and Investment
In reality, a negative correlation exists between the R&D time and investment.To ensure the smooth development of weapons, the investment of each weapon cannot be lower than a certain threshold c min , otherwise it will not be successfully developed.The lowest investment amount corresponds to the longest development time t max .As the investment gradually increases, the development time correspondingly gradually shortens.However, as the marginal effect of investment decreases, the development time declining rate slows.In addition, due to technical restrictions, the development time cannot continually decrease.No matter the amount of the investment, the development time will not be less than the shortest time t min .The corresponding investment is now c max .Therefore, for the relationship between time and investment, the inverse scaling function model was designed as shown in Figure 5. Herein, a Bj generally represents the technical difficulty in the development process.

Modeling the Relationship between Time and Investment
In reality, a negative correlation exists between the R&D time and investment.To ensure the smooth development of weapons, the investment of each weapon cannot be lower than a certain threshold cmin, otherwise it will not be successfully developed.The lowest investment amount corresponds to the longest development time tmax.As the investment gradually increases, the development time correspondingly gradually shortens.However, as the marginal effect of investment decreases, the development time declining rate slows.In addition, due to technical restrictions, the development time cannot continually decrease.No matter the amount of the investment, the development time will not be less than the shortest time tmin.The corresponding investment is now cmax.Therefore, for the relationship between time and investment, the inverse scaling function model was designed as shown in Figure 5. Herein, aBj generally represents the technical difficulty in the development process.The relationship above is simple.However, we argue that a realistic relationship between development time and the resource investments can only be empirically determined from experts or statistically from data.This problem is neither a major concern in this paper, nor a key component of our model.
In Golany's model [15], tBj was used to determine the development time of each weapon.tBj was determined by the level of intensity, which is a discrete variable; therefore, tBj was also a discrete variable.As opposed to Golany's model, tBj = aBj/cBj is used to describe the development time, determined by the allocated investment, which is quite realistic in defense contracting.The continuous variable cBj implies continuity of tBj, resulting in a wider range of possible solutions.Hence, the problem faced by B is to decide which weapon to develop, when to start development, and what proportion of the available resources to allocate to each weapon based on the current weapons.The objective is to minimize the cumulative damage over the time horizon, subject to budgetary constraints.The relationship above is simple.However, we argue that a realistic relationship between development time and the resource investments can only be empirically determined from experts or statistically from data.This problem is neither a major concern in this paper, nor a key component of our model.

Algorithm Design
In Golany's model [15], t Bj was used to determine the development time of each weapon.t Bj was determined by the level of intensity, which is a discrete variable; therefore, t Bj was also a discrete variable.As opposed to Golany's model, t Bj = a Bj /c Bj is used to describe the development time, determined by the allocated investment, which is quite realistic in defense contracting.The continuous variable c Bj implies continuity of t Bj , resulting in a wider range of possible solutions.Hence, the problem faced by B is to decide which weapon to develop, when to start development, and what proportion of the available resources to allocate to each weapon based on the current weapons.The objective is to minimize the cumulative damage over the time horizon, subject to budgetary constraints.

Algorithm Design
The decision variables of the problem are development time and investment amount.The unit of development time is usually years.The unit of investment is usually millions or an order of magnitude higher.For easy figures, all variables take integers.We defined the solution space as S, and the corresponding scale as: The scale of S increases exponentially with an increase in n.The computational complexity of this problem is at least NP-hard.Therefore, using an intelligent optimization algorithm for solving this problem is reasonable.
Differential Evolution (DE) is an evolutionary optimization methods proposed by Storn and Price [26].A simple mutation operation and a one-on-one competitive survival strategy reduce the complexity of a genetic operation.DE is robust and has a strong global convergence ability, suitable for solving certain optimization problems that conventional mathematical programming methods cannot solve in complex environments [27].Given the advantages of DE and the characteristics of this model, we designed the following solving algorithm.
Step 1: Initialize the population.Using decimal integers to encode all variables according to the type of decision variables is appropriate for this problem.A total of J pending weapons exist, so an individual of 2J is generated.The encoding form of the individual is shown in Figure 6.The scale of S increases exponentially with an increase in n.The computational complexity of this problem is at least NP-hard.Therefore, using an intelligent optimization algorithm for solving this problem is reasonable.
Differential Evolution (DE) is an evolutionary optimization methods proposed by Storn and Price [26].A simple mutation operation and a one-on-one competitive survival strategy reduce the complexity of a genetic operation.DE is robust and has a strong global convergence ability, suitable for solving certain optimization problems that conventional mathematical programming methods cannot solve in complex environments [27].Given the advantages of DE and the characteristics of this model, we designed the following solving algorithm.
Step 1: Initialize the population.Using decimal integers to encode all variables according to the type of decision variables is appropriate for this problem.A total of J pending weapons exist, so an individual of 2J is generated.The encoding form of the individual is shown in Figure 6.The upper boundary for time in terms of the variable is five, whereas the lower boundary is one, which respectively indicates that the weapon is developed in the fifth or the first year.As for the investment, the upper boundary is c max and the lower boundary is c min .
Step 2: The fitness function and penalty function.Evolutionary algorithms have been widely used to solve optimization problems.However, when the optimization problem has many nonlinear, linear, inequality, and equality constraints, the solving process is more complex.Therefore, designing a constraint processing technology with better performance is essential.Michalewicz [28] and Coellol [29] conducted extensive investigations into the constraint handling techniques and classified the techniques into five classes: penalty functions, special representations and operators, repair algorithms, separation of constraints and objectives, and hybrid methods.Among these constraint handling approaches, we applied the penalty functions to address the constraints.We set the development investments, so that the planning each year that exceeded the annual budget is the penalty function.The fitness function is defined as the objective function.The smaller is the fitness value, the better is the individual.The penalty function f is: where each part in Equation (20) denotes the extent to which the corresponding limit is exceeded.
Taking the first item max( , 0)  The upper boundary for time in terms of the variable is five, whereas the lower boundary is one, which respectively indicates that the weapon is developed in the fifth or the first year.As for the investment, the upper boundary is c max and the lower boundary is c min .
Step 2: The fitness function and penalty function.Evolutionary algorithms have been widely used to solve optimization problems.However, when the optimization problem has many nonlinear, linear, inequality, and equality constraints, the solving process is more complex.Therefore, designing a constraint processing technology with better performance is essential.Michalewicz [28] and Coellol [29] conducted extensive investigations into the constraint handling techniques and classified the techniques into five classes: penalty functions, special representations and operators, repair algorithms, separation of constraints and objectives, and hybrid methods.Among these constraint handling approaches, we applied the penalty functions to address the constraints.We set the development investments, so that the planning each year that exceeded the annual budget is the penalty function.The fitness function is defined as the objective function.The smaller is the fitness value, the better is the individual.The penalty function f is: where each part in Equation (20) denotes the extent to which the corresponding limit is exceeded.
Taking the first item max( ∑ j∈H c B j − C H , 0) as an example, when the annual investment exceeds the That is, no violation of the constraint is triggered.As for the time, st Bj should take an integer between 1 and 5.If the starting time of a weapon does not meet this requirement, the excess is computed in the penalty function max(1 − st B j , 0) + max(st B j − 5, 0).The same logic is applied to the other constraints.
Step 3: Initial population evaluation.The fitness value and the degree by which each individual in the population violates the constraints are calculated.When no individual violates the constraints in the population, the penalty function is 0.Then, the individual with the best fitness value is the current global optimal solution.When all individuals in the population violate the constraints, the penalty functions of all individuals are greater than 0.Then, the individual with the least penalty function value is chosen as the current global optimal solution.
Step 4: Mutation operation.Equation ( 21) is used to mutate the individual. ( where X best is the best target individual; x r1 (G), x r2 (G), and x r3 (G) are random target individuals; v i (G + 1) is the mutant individual, and F ∈ [0, 1] indicates the mutant scale factor, recommended as 0.5 [30].
Step 5: Crossover operation.To increase the diversity of the population, the crossover operation is performed between the temporary individual and the parent individual.Substitute the parent individuals for temporary individuals with a probability of CR = 0.2.
Step 6: Cross-border processing.The classical DE does not consider how to address the constraints handling.The randomness of the general cross-border processing strategy is too strong to preserve the historical optimal information accumulated through evolution.We used the symmetric mapping processing strategy as previously reported [31].As shown in Figure 7, range left and range right are the left and right boundaries of each variable in the individual, respectively.When the extent to which a variable violates the constraint is less than the absolute value of the variable range, the boundary point is taken as the center to map this variable equidistantly onto the range of value.
Step 7: Competitive survival operation.We used the idea of competition survival to compare the fitness and penalty function of the corresponding individual in the parent population and temporary population, and selected the preferable individual to form a new population.The specific rules are: (1) if both individuals do not violate the constraints, we judge them by comparing their fitness values; (2) if a solution satisfies the constraints, whereas the other one does not, then the solution meeting the constraints is better; and (3) if both do not meet the constraints, the solution with the lower penalty function is better.
Step 5: Crossover operation.To increase the diversity of the population, the crossover operation is performed between the temporary individual and the parent individual.Substitute the parent individuals for temporary individuals with a probability of CR = 0.2.
Step 6: Cross-border processing.The classical DE does not consider how to address the constraints handling.The randomness of the general cross-border processing strategy is too strong to preserve the historical optimal information accumulated through evolution.We used the symmetric mapping processing strategy as previously reported [31].As shown in Figure 7, rangeleft and rangeright are the left and right boundaries of each variable in the individual, respectively.When the extent to which a variable violates the constraint is less than the absolute value of the variable range, the boundary point is taken as the center to map this variable equidistantly onto the range of value.
Step 7: Competitive survival operation.We used the idea of competition survival to compare the fitness and penalty function of the corresponding individual in the parent population and temporary population, and selected the preferable individual to form a new population.The specific rules are: (1) if both individuals do not violate the constraints, we judge them by comparing their fitness values; (2) if a solution satisfies the constraints, whereas the other one does not, then the solution meeting the constraints is better; and (3) if both do not meet the constraints, the solution with the lower penalty function is better.Step 8: Evolution and iteration.We determined whether the scheduled iterations are reached.If so, then the operation is stopped; otherwise, proceed to step 3.If the desired iterations are reached, but the satisfactory solution has not been found, the population size and the iterations should be adjusted.

Multi-Objective Modeling and Solving
Many uncertainties exist in the weapons development process.One uncertainty is the development speed of military technology.Another uncertainty is the development process and Step 8: Evolution and iteration.We determined whether the scheduled iterations are reached.If so, then the operation is stopped; otherwise, proceed to step 3.If the desired iterations are reached, but the satisfactory solution has not been found, the population size and the iterations should be adjusted.

Multi-Objective Modeling and Solving
Many uncertainties exist in the weapons development process.One uncertainty is the development speed of military technology.Another uncertainty is the development process and conditions, such as funds, talents, and management [32].These uncertainties result in risk during the development of weapons, meaning that the weapons may not attain the initially planned threat value.Therefore, DMs should consider the risk in weapons development so as to minimize the risk and meet future needs.

Development Risk Modeling
Weapons development risk refers to the possibility that weapons may not attain the tactical and technical index given the constraints imposed by limited resources, which negatively impacts the overall performance of combat [33].Therefore, we first reasonably modelled the risk.In general, risk is defined as the consequence of failing to reach the expected purpose, multiplied by the probability of that consequence [34].According to the typical risk model, which is called Expected Downside Risk (EDR) [35]: where π is the target value, x is the actual value, and α is the risk sensitive exponent.The model considers the difference between the actual value and the target value as the consequence, and calculates the sum of the probability of all consequences.We used this model to define the gap between the actual threat reduction value and the expected value as the consequence, and then totaled all gaps with the corresponding probabilities to measure the risk.
Assuming that I possibilities of threat reduction value of each weapon exist, then the ith possible threat reduction value is: where A i is the conversion coefficient of the ith possible threat reduction value, T denotes the expected maximum value of the threat reduction without considering any constraints, and η k denotes the risk sensitive coefficient of the expected maximum threat reduction value.The probability of T is modelled as: where P i is the probability conversion coefficient of the ith possible threat reduction value and t is the development duration of the weapon.For the weapons that can counter the same threat value, the longer is the development duration, the greater is the probability that the weapon will achieve the expected effect, and the smaller is the risk, and vice versa.
Based on the above model, the total weapons development risk of B is defined as:

Optimization Model
According to the threat gap planning model outlined in Section 3 and Equation ( 25), we built a multi-objective optimization model, which allows DMs to balance the two objectives of threat and risk, and obtains the best compromise solution to create a combined weapons planning scheme.
Therefore, the problem is treated as a typical Multi-Objective Optimization Problem (MOOP).The objectives that need to be optimized can be represented by an objective function set and certain equality or inequality constraints, and conflicts often exist among these objectives.It is impossible to simultaneously optimize all sub-objective functions; instead, we can obtain a compromise solution to these objectives, which is the Pareto optimality problem, first proposed by Pareto [36].
A global optimal solution to the single-objective optimization problem must exist.Conversely, it is possible that no definite optimal solution exists, but an acceptable solution set to MOOP may be possible.When all objectives are considered, the acceptable solution set is the best solution within the entire search space.However, the solution may not be optimal from the perspective of any of the sub-objectives.This solution set is called a Pareto optimal set, or non-dominated set, and the other solution set is called a dominated set.Because any of the solutions in the non-dominated set are not better than the other solutions, then the key to solve MOOP lies in how to find the Pareto optimal set, and determine the most appropriate solution in the Pareto optimal set according to the preference of the DMs.

Non-Dominated Sorting Differential Evolution Algorithm
The NSGA-II is a popular optimization method used to solve MOOP [37].In 1994, NSGA was proposed by Srinivas and Deb [38].Based on the shortcomings of NSGA, Deb et al. introduced NSGA-II in 2000 [39], which has gradually become the mainstream algorithm used for solving MOOP.It uses non-dominated sorting and shared variable methods to effectively maintain the diversity of Pareto frontiers.However, NSGA-II does not perform well in processing MOOP with a complex Pareto front.Given these shortcomings of NSGA-II, we designed an improved algorithm, which we have called the non-dominated sorting differential evolution (NSDE) algorithm.NSDE introduces a differential evolution operator to replace the crossover operation in NSGA-II and adaptively adds the global optimal solution information in the new generations to improve the local search accuracy and diversity.Considering the characteristics of the model, we combined NSGA-II with DE, and designed the following solving algorithm.
Step 1 Define the parameters of the algorithm.Define population size, iterations, and the crossover probability parameter, and the counter is set to 0. Step 2 Initialize the population.The initial population is randomly generated and the individual encoding structure is outlined in Section 3. Step 5 Evaluate the temporary population.The temporary population is composed of the present population P and the offspring population Q, and non-dominated sorting of the temporary population is performed by comparing the rank and crowding distance of the individuals.
Step 6 Generate a new population.Select the best individuals from the temporary population to generate the new population.
Step 7 Evolution and iteration.If the number of iterations is reached, the Pareto optimal solution is output.Otherwise, the counter increases, and we continue back to Step 3.

Compromise Solution Based on TOPSIS
After obtaining the Pareto optimal set, evaluating the solution in the Pareto optimal set and selecting the compromise solution according to the preference of the DM are necessary.To date, many methods can be used to obtain a compromise solution from the Pareto optimal set.We used the classical TOPSIS method proposed by Hwang and Yoon [40] to obtain the compromise solution.TOPSIS essentially compares the proximity between different schemes and the ideal scheme.The main steps are as follows: Step 1: Establish a decision matrix.Establish and normalize the decision matrix A = (a ij ) p×2 .p is the number of solutions, and 2 is the number of evaluation criteria, which, in this paper, are the threat gap and weapons development risk.
Step 2: Standardization the Decision matrix.Establish the weighted standardized decision matrix R = (r ij ) p×2 , r ij = x ij × q j , where q j is the weight value of the jth criteria.
Step 3: Define the ideal solution.Define the positive ideal solution f + j and the negative ideal solution f − j .Because the threat gap and weapons development risk are both cost-type indexes: Step 4: Calculate the Euclidean distance between each scheme and the positive and negative ideal solutions.
Step 5: Sort all solutions.Calculate the relative approach degree C i between each scheme and the ideal point.
Finally, sort C i in descending order.The solution with the largest C i value is the optimal solution in the Pareto solution set.

Validation Demonstration
To clearly illustrate the application and procedure of the proposed approach and to demonstrate its feasibility and effectiveness, a case study is presented in this section.

Parameters Settings
Two countries, denoted as the attacker (R) and the defender (B), are in conflict with each other.R will deploy a sort of weapons in the next five years.As a result, B should develop a defense system consisting of new weapons as CMs.
As shown in Table 2, R 1 , R 2 , R 3 , R 4 , R 5 , R 6 , R 7 and R 8 are R's current weapons; B 1 , B 2 , B 3 , B 4 , B 5 , B 6 , B 7 , B 8 , B 9 and B 10 are new weapons that B may develop.Each weapon can be characterized as one or several meta-functional nodes.For example, R 1 is abstracted into two meta-functional nodes.B 1 is abstracted into three meta-functional nodes.In addition to the functional descriptions of the nodes, the threat vectors that can counter R must be computed.Part of the data is obtained from prior studies on [41,42].Tactical and technical index values of B's weapons are shown in Table 3.The reference values of the tactical and technical indexes are provided in Table 4, and the normalization result is calculated in Table 5 according to the normalization method mentioned in Section 2.2.2.Next, we classified and aggregated the threat value of various nodes.The result was shown in Table 6.The weights were set to be the same.To facilitate the calculation, we set all threats from R as 40.Some restrictions exist based on real-life operational situations: (1) if more than one S node exists in one loop, they can only be arranged based on their values of Radius in descending order.For example, the Radius of S B3 and S B2 are 10 and 5, respectively.Then, S B3 →S B2 is logical, S B2 →S B3 is unreasonable; (2) if more than one D node exists in one loop, they can only be arranged in descending order in terms of Efficiency.For example, the Efficiency of D B2 and D B9 are 15 and 10, respectively.Then, D B2 →D B9 is logical, D B9 →D B2 is unreasonable; (3) for the connection from node S to node D, due to the limitation of communication links, D B1 , D B2 , D B3 , and D B9 can only receive information from S B1 , S B2 , S B3 , and S B9 , respectively.S B5 and S B7 can only pass information to D B2 .S B8 and S B10 can only pass information to D B9 ; (4) for node I, due to the limitation of communication links, I B1 , I B2 , and I B3 can only receive information from D B1 , D B2 , and D B3 , respectively.I B4 , I B6 , and I B7 can only receive a strike command from D B2 .I B10 can only receive a strike command from D B9 ; and (5) for the same functional nodes, more than two nodes cannot exist simultaneously in one loop.Based on the loops finding rules, a total of 115 loops covering the opponent nodes are found.The distribution of loops is shown in Table 7.The structure and threat values of all loops are shown in Table A1.
The related parameters of the weapons to be developed are shown in

Numerical Results
Table 9 depicts the parameters used in this study associated with DE.Herein, two other algorithms, Genetic Algorithm (GA) [43] and Particle Swarm Optimization (PSO) [44], were also applied independently under the same conditions.We ran the above three algorithms 10 times independently.Figure 8 shows the best evolutionary curves corresponding to the three algorithms, obtained with the evolutionary generations of the algorithm as abscissa and the optimization objective function D B in Equation ( 26) as the ordinate.The smaller is the threat gap, the better is the corresponding planning.In particular, the final objective function values obtained by DE and PSO are considerably better than that of GA.Comparing DE with PSO, although the two final objective function values are basically the same, DE converges to a satisfactory solution at a faster rate, in about 60 generations, whereas PSO required about 280 generations.The minimum DE threat gap is 20.6594 with the best individual as [4, 1, 1, 2, 3, 1, 1, 5, 1, 1, 21,8,14,7,27,6,28,38,33,40].We statistically analyzed the results, as shown in Table 10.The optimal value, the worst value, the mean value, the middle value, and the variance of the function in the 10 running results were tabulated.The optimal and worst value illustrate the merits of the scheme.The average value reflects the overall quality of the algorithm.The mid-value explains the convergence speed of the algorithm, to a certain extent.The variance is the overall stability of the algorithm.All DE indexes are better than those of GA and PSO.Although the optimal solution found by PSO is close to the optimal solution calculated by DE, the mid-value and variance corresponding to the PSO result are inferior to those of DE.In summary, DE has better efficiency and stability for this model.
As for multi-objective optimization, NSDE and NSGA-II were used to optimize the weapons development scheme to obtain the Pareto optimal set, and then the TOPSIS method was used to find the compromise solution.Similarly, we ran the two algorithms 10 times independently.Each algorithm obtained 10 Pareto sets.Then, these solutions were sorted.The best solutions were selected to form the Pareto solution set of the two algorithms, as shown in Figure 9, which is plotted with the optimization objective function R B in Equation ( 26) as the ordinate, against the optimization objective function D B in Equation ( 26) as the abscissa.The coordinate axis is the same in Figure 10.
development scheme to obtain the Pareto optimal set, and then the TOPSIS method was used to find the compromise solution.Similarly, we ran the two algorithms 10 times independently.Each algorithm obtained 10 Pareto sets.Then, these solutions were sorted.The best solutions were selected to form the Pareto solution set of the two algorithms, as shown in Figure 9, which is plotted with the optimization objective function RB in Equation ( 26) as the ordinate, against the optimization objective function DB in Equation ( 26) as the abscissa.The coordinate axis is the same in Figure 10.

Results Analysis
In this section, we analyze the model based on the numerical results.

Results Analysis
In this section, we analyze the model based on the numerical results.

Comparison of a Single Objective and Multi-Objective
The best planning solution of the single objective problem is [4, 1, 1, 2, 3, 1, 1, 5, 1, 1, 21, 8, 14, 7, 27,6,28,38,33,40], and the compromise solution of the multi-objective problem is [1, 1, 1, 1, 1, 1, 1, 5, 5, 1, 8, 7, 12,6,23,5,26,30,21,40].As for the investment, the maximum difference between these two results is related to the first and eighth weapons, which means that their threat reduction effects are the best operation loops to counter the enemy.DMs can focus on developing these two weapons by increasing the amount of investment or reducing uncertain risks.As for time, the difference between these two results is related to the first and ninth weapons, indicating that their R&D risks are very high.In the development process of these two weapons, DMs should ensure the risks are controlled.

Different Weights of Multi-Objective
Different DMs may have different preferences for the above-mentioned two objectives.Consequently, we used different weighted values for the threat gap and the risk objectives.The optimal results are shown in Table 11.If the DMs are risk-seeking, they can set the weight of the development risk objective lower.On the contrary, if the DMs are risk-averse, they can set the weight of the development risk objective higher.We also found that the risk value does not increase even when the DM is generally displaying a risk-seeking tendency when making choices.

Decision Preference
During the actual decision-making process, DMs must artificially adjust and optimize the planning to coordinate the development of weapons.Consider a scenario with an m-period and M weapons to be developed: (1) if the ith weapon must be developed in the jth year, then, the time encoding of the ith weapon is set to j when initializing the population; and (2) if the development duration of the ith weapon must satisfy k years, then, the investment encoding of the ith weapon is set to a/k when initializing the population.The logic is applied when other similar situations occur.The model and the corresponding solving algorithm allow users to set specific encoding and adjust the development planning based on individual preferences.The model is practical for actual application.

Conclusions
This paper proposed a new approach to solve the problem of allocating limited resources when developing military CMs versus probabilistic threat, with emphasis on representing the cooperation relationships among different weapons.The competition between hostile countries was transformed into an operation loop-based optimization model.We applied the system dynamics principle to characterize the dynamic behavior of the nodes in a complex weapons network.The intrinsic properties of a weapon were converted into certain constraints.To address the development of CMs with limited resources to counter the enemy, the threat gap was set as the single objective function.In addition, considering the uncertainties in the development process, the R&D risks of weapons were used to optimize the planning.Ultimately, certain intelligent optimization algorithms were used to obtain a numerical solution.
The numerical results showed that more investments result in a more effective design, and more time leads to lower risk.The key weapons that can effectively reduce threat and could potentially encounter technical difficulties during R&D should be developed as soon as possible.With a fixed investment amount, the results also demonstrate that the system effectiveness can be significantly improved by increasing the amount allocated to the weapon that is found in more operation loops.
Future research directions include allowing for uncertainty in players' preferences, having a dynamic resource distribution strategy over a certain time horizon, and enabling multiple players or multi-objective in a sequential game.These challenges will contribute to the expansion of this methodology for tackling a broader range of problems.
For example, an armored observation and command vehicle can be characterized as node S and node D. A self-propelled antiaircraft gun can only be characterized as node I.An illustrative example of different nodes in WSOS is provided in Figure 2. Different parties are described with corresponding color.R has five existing nodes (S 1 , D 1 , D 2 , D 3 and I 2 ) and one new weapon (S 2 ) awaiting development.S 1 , S 2 and I 2 can be detected by B and become B's targets.The same applies to the other party.Appl.Sci.2018, 8, x FOR PEER REVIEW 4 of 25 A self-propelled antiaircraft gun can only be characterized as node I.An illustrative example of different nodes in WSOS is provided in Figure 2. Different parties are described with corresponding color.R has five existing nodes (S1, D1, D2, D3 and I2) and one new weapon (S2) awaiting development.S1, S2 and I2 can be detected by B and become B's targets.The same applies to the other party.

Figure 2 .
Figure 2. Weapon system of systems nodes location in battle.

Figure 2 .
Figure 2. Weapon system of systems nodes location in battle.

Figure 3 .
Figure 3. Schematic of an operation loop in battle.

Figure 3 .
Figure 3. Schematic of an operation loop in battle.

Figure 5 .
Figure 5. Relationship modeling between time and investment.

Figure 5 .
Figure 5. Relationship modeling between time and investment.

.
as an example, when the annual investment exceeds the budget, the first item is Herein, the part of the excess budget is computed as part of the penalty function.Conversely, if max( penalty function is 0. That is, no violation of the constraint is triggered.As for the time, stBj should take an integer between 1 and 5.If the starting time of a weapon does not meet this requirement, the excess is computed in the penalty
first item is ∑ j∈H c B j − C H . Herein, the part of the excess budget is computed as part of the penalty function.Conversely, if max( ∑ j∈H c B j − C H , 0) = 0, the corresponding penalty function is 0.

3 . Step 3
Sort the population.According to the objective function and constraints, evaluate the individuals.Use non-dominated sorting on the initial population to obtain the Pareto solution set.Then individuals are given ranks and crowding distance values.The binary tournament selection operation is implemented on the population.Step 4 Mutation and crossover.The mutation operator based on DE was used to mutate the population.Simultaneously, a new population Q is obtained by using crossover operation on each of the individuals with a certain probability.
better is the corresponding planning.In particular, the final objective function values obtained by DE and PSO are considerably better than that of GA.Comparing DE with PSO, although the two final objective function values are basically the same, DE converges to a satisfactory solution at a faster rate, in about 60 generations, whereas PSO required about 280 generations.The minimum DE threat gap is 20.6594 with the best individual as [4, 1, 1, 2, 3, 1, 1, 5, 1, 1,21,8,14,7,27,6,28,38,33,40].

Figure 8 .
Figure 8. Results of the three algorithms.

Figure 8 .
Figure 8. Results of the three algorithms.

Figure 9 .
Figure 9. Pareto sets of two different algorithms.

Figure 9 .
Figure 9. Pareto sets of two different algorithms.NSDE has a longer evolutionary curve than the traditional NSGA-II.Comparing the results of the two algorithms, the NSDE Pareto solutions are superior to those of NSGA-II in terms of diversity and convergence.We set the weighted values of the threat gap and the risk objectives as 0.6 and 0.4, respectively.The compromise solution calculated by the TOPSIS method is shown in Figure 10, which is [1, 1, 1, 1, 1, 1, 1, 5, 5, 1, 8, 7, 12, 6, 23, 5, 26, 30, 21, 40].Appl.Sci.2018, 8, x FOR PEER REVIEW 20 of 25

Figure 10 .
Figure 10.Compromise solution of the Pareto set.

Figure 10 .
Figure 10.Compromise solution of the Pareto set.Uncertainty exists in the weapons R&D process and in all model relationships.A sensitivity analysis was performed for certain variables to examine the uncertainties of the model.Herein, the total budget was set as 200 and 230.The compromise solutions calculated by NSDE and TOPSIS are shown in Figure 11, which are [1, 1, 1, 1, 1, 1, 1, 5, 1, 5, 8, 7, 12, 6, 19, 5, 21, 30, 21, 32] and [1, 1, 1, 1, 1, 1, 1, 5, 1, 1, 8, 7, 12, 6, 21, 5, 24, 30, 27, 40].Compared to the original settings, the maximum difference between these programs is related to the ninth and tenth weapons.The model is affected less by uncertainty.The results indicate that their R&D risks may be high.In the development process of these two weapons, DMs should focus on the technical progress and upgrading to reduce risk.

Figure 11 .
Figure 11.(a) Compromise solution with a total budget of 200; and (b) compromise solution with a total budget of 230.

Figure 11 .
Figure 11.(a) Compromise solution with a total budget of 200; and (b) compromise solution with a total budget of 230.

Table 3 .
Tactical and technical index values.

Table 4 .
The reference values of the tactical and technical indexes.

Table 5 .
Tactical and technical index normalization.

Table 6 .
Threat vector aggregation of nodes.

Table 8 .
Parameters of the weapons.

Table 10 .
Comparison of algorithm results.

Table 11 .
Comparison of algorithm results.