Model and Data-Driven System Portfolio Selection Based on Value and Risk

Featured Application

The work is potential to be applied to the weapon system portfolio selection and other project portfolio selection on the context of research and development.

Abstract

System portfolio selection is a kind of tradeoff analysis and decision-making on multiple systems as a whole to fulfill the overall performance on the perspective of System of Systems (SoS). To avoid the subjectivity of traditional expert experience-dependent models, a model and data-driven approach is proposed to make an advance on the system portfolio selection. Two criteria of value and risk are used to indicate the quality of system portfolios. A capability gap model is employed to determine the value of system portfolios, with the weight information determined by correlation analysis. Then, the risk is represented by the remaining useful life (RUL), which is predicted by analyzing time series of system operational data. Next, based on the value and risk, an optimization model is proposed. Finally, a case with 100 candidate systems is studied under the scenario of anti-missile. By utilizing the Non-dominated Sorting Differential Evolution (NSDE) algorithm, a Pareto set with 200 individuals is obtained. Some characters of the Pareto set are analyzed by discussing the frequency of being selected and the association rules. Through the conclusion of the whole procedures, it can be proved that the proposed model and data-driven approach is feasible and effective for system portfolio selection.

1. Introduction

Joint operations have become the main trend of modern warfare. The construction of “system of systems (SoS)” is not only a goal but also a basic guideline on the long-term weapon/equipment development. System portfolio selection is a widely used concept of weapon SoS construction, where a key step is evaluation [1]. Traditional evaluation models rely too much on subjective awareness, making assessment results inaccurate and unconvincing to some extent. With the rise of data science, an effective method to compensate for the low accuracy and implementing difficulty of relying on expert experience is making decisions according to real data. Therefore, the combination of data-driven methods and model-based approaches is a new trend to solving system portfolio selection problems.

Markowitz first proposed the portfolio theory in 1952, opening a new era of utilizing mathematical approaches in resource allocation problems [2,3]. In the field of management science and operation research, the portfolio theory is widely used in project research and development (R&D) [4], supplier selection [5], material selection etc. With the development of SoS science, portfolio theory shows increasing popularity in the field of weapon SoS construction, where the optimal system portfolio will be selected by evaluating system portfolio candidates through model-based methods. So far, there has been little research that measures weapon system portfolios without subjective criteria. Typical measurements in most literature, such as benefit-risk analysis [6], cost-efficiency analysis [7], and requirement-satisfaction analysis [8] are inaccurate and unconvincing to some extent because they usually require too much expert experience.

Motivated by solving issues mentioned above, data-driven methods are combined with traditional model-based approaches to improve the accuracy and credibility of evaluation results on system portfolios, by reducing dependence on artificial expertise. In addition, data-driven methods are complements of model-based approaches, instead of substitutions, because pure data without models cannot construct the bridge connecting scheme variable inputs and evaluation outputs.

In civilian fields, the portfolio selection theory has been mainly studied and applied on project portfolio problems [9]. From the perspective of modeling, the scenario-based models are frequently used to describe the boundary of possible cases, based on which, decision-makers evaluate and select the well-matched optimal system portfolio [10,11,12,13]. Robust models are also widely studied and applied in project portfolio problems to solve the difficulties in determining probabilities of future scenarios, aiming to select an ideal system portfolio that performances well at almost all possible situations [14,15,16]. As for the evaluation and trade-off of project portfolios, variant methods are proposed and studied, such as risk analysis methods [17], value evaluation methods [18], cost-efficiency methods [19], fuzzy assessment methods [4], preference-based methods [20], game theory, interactive decision methods [21], etc. A common ground of those methods is determining the value and risk of a system portfolio to abstractly indicate what decision-makers expect or not expect. As regard to portfolio planning and optimization, the goal is to select the optimal project portfolio by analyzing and comparing candidate project portfolios. The mixed integer model [22], multi-objective optimization [23], hybrid and dynamic planning are the most popular optimization methods. In addition, genetic algorithms [23], Monte Carlo simulation [24] and Lagrangian relaxation methods are also widely used in the solving process, when facing a large solution scale and specific constraints.

In military fields, most methods in system portfolio selection are based on specific evaluation models, where the most commonly investigated techniques include multiple objective analysis, multiple criteria analysis [25], value analysis [26], cost-efficiency analysis [27], expert judgment [27], Monte Carlo technique, risk analysis and etc. In detail, Yang et al. [26] formulize the weapon system portfolio problem with a mixed integer non-linear optimization model and solve the problem with an adaptive immune genetic algorithm. Greiner et al. [28] conclude challenges of the Department of Defense (DoD) in determining weapon system value during portfolio selection processes. Cheng et al. [29] use combat network and operation loop to analyze strategies of the weapon system portfolio selection problem, where the operational capability evaluation indexes of weapon systems are constructed. Zhou et al. [30] deal with weapon system portfolio selection problems based on fuzzy clustering, with the maximum deviation methods applied to rank all the candidates by calculating the weight of each weapon system. Kangaspunta J et al. [27] use the cost-efficiency method to decide the acquisition and maintenance of military equipment, aiming to build long-term capabilities in future military conflicts. Li et al. [31] adopt a network-based method to formulate and analyze weapon system portfolio architecting problem by embedding different types of systems into a network. Zhou et al. [32] study the evolving capability requirement-oriented portfolio planning problem with a capability-based approach from the perspective of operational research. Huang et al. [33] regard the weapon system portfolio as a constrained combinatorial optimization problem and use a self-adaptive memetic algorithm-based decision-making method to maximize the expected damage of hostile targets.

Whatever in civilian fields and military fields, the model-based portfolio selection methods have been elaborately studied. With the increase of requirements for more accurate and valid approaches, the data-driven idea is appropriate to be applied to the portfolio selection. In the paper, we focus mainly on system portfolio evaluation, where a key part is determining criteria that influence the evaluation result of an object. Herein, two criteria of value and risk are used to evaluate system portfolios, where the value criterion is decided according to capability gaps of system portfolios and the risk criterion is decided by the remaining useful life (RUL) of systems. Based on the two criteria, the optimization is to obtain the system portfolio with the maximal value and minimal risk, within the limitation of a certain cost. To increase the credibility and practicability, the weight information in value evaluation and the RUL are all decided according to simulation data, instead of expert experience.

The remaining parts of the paper are structured as follows. In the second section, the capability gap-based value decision method and the RUL-based risk decision method are studied. In the third section, a case is examined to verify the utility and effectiveness of the proposed methods and models. Then, the results are discussed by analyzing the frequency of being selected and the association rules.

2. Materials and Methods

2.1. Capability Gap Based Value Decision

2.1.1. Weight Decision Based on Correlation Analysis

Weight information determines the criteria importance, and thus influences the final evaluation results of systems portfolios. Typically, criteria weights tend to be determined according to expert experience, which is criticized for its subjectivity and infeasibility. Correlation analysis is a quantitative method to measure the correlations between independent and dependent variables, and therefore can be used to indicate the weights of independent variables.

The maximal information coefficient (MIC) is used to measure correlations among all kinds of data. Compared to other measurements, it is acknowledged that the MIC is more sensible to identify correlations among variables, whatever for linear relations or non-linear relations (cubic, exponential, sinusoidal, parabolic, etc.)

The correlation of variable X and Y can be indicated by the mutual information, as Equation (1) shows.

$$I\left(X,Y\right)={\displaystyle \sum _{x,y}p\left(x,y\right)\log \frac{p\left(x,y\right)}{p\left(x\right)p\left(y\right)}},$$

(1)

Because the information entropy of discrete random variables is denoted as $H\left(p\right)=-{\displaystyle {\sum }_{i=1}^n{p}_i\log p_i}$, where ${\displaystyle {\sum }_{i=1}^n{p}_i=1}$. Therefore, the $I\left(X,Y\right)$ can be proved to be equivalent to Equation (2).

$$\begin{array}{ll}I\left(X,Y\right)& =H\left(X\right)+H\left(Y\right)-H\left(X,Y\right)\\ & =H\left(X\right)-H\left(X|Y\right)\\ & =H\left(Y\right)-H\left(Y|X\right)\end{array},$$

(2)

With respect to MIC, it is assumed that the $D=X\times Y\subset {ℝ}^2$ is the variable space. Dividing the D into a grid G of $x\times y$. The distribution of data D in grid G is represented by $D{|}_G$. Executing multiple divisions on D and calculating the $I(D{|}_G)$ under different divisions. Then, the MIC can be obtained by calculating the max $I(D{|}_G)$ value from all possible division schemes.

Here, a grid of one row and one column is taken for example as Figure 1 shows. All data points are divided into four areas: top left, top right, bottom left, and bottom right. The numbers of data points belonging to each area are 1, 4, 4, and 1. Then, the normalized numbers of data point frequencies in the four regions are 0.1, 0.4, 0.4, and 0.1. Herein, X has two values: left and right, and Y has two values: upper and lower. The joint probabilities of the data in four regions can be calculated as $p\left(X=left,Y=up\right)=0.1$, $P\left(X=right,Y=up\right)=0.4$, $P\left(X=left,Y=down\right)=0.4$, $P\left(X=right,Y=down\right)=0.1$. Therefore, the point frequencies in X, Y are $P\left(X=left\right)=0.5$, $P\left(X=right\right)=0.5$, $P\left(Y=upper\right)=0.5$ and $P\left(Y=lower\right)=0.5$. According to the mutual information calculation formula introduced above, the mutual information of X and Y is obtained as Equation (3).

$$\begin{array}{l}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}p\left(X=left,Y=up\right)\log \left(\frac{p\left(X=left,Y=up\right)}{p\left(X=left\right)p\left(Y=up\right)}\right)\\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}+p\left(X=left,Y=up\right)\log \left(\frac{p\left(X=left,Y=up\right)}{p\left(X=left\right)p\left(Y=up\right)}\right)\\ I\left(X,Y\right)=+p\left(X=left,Y=up\right)\log \left(\frac{p\left(X=left,Y=up\right)}{p\left(X=left\right)p\left(Y=up\right)}\right)\\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}+p\left(X=left,Y=up\right)\log \left(\frac{p\left(X=left,Y=up\right)}{p\left(X=left\right)p\left(Y=up\right)}\right)\\ \hspace{1em}\hspace{1em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}=0.1\times \log \left(\frac{0.1}{0.5\times 0.5}\right)+0.4\times \log \left(\frac{0.4}{0.5\times 0.5}\right)+0.4\times \log \left(\frac{0.4}{0.5\times 0.5}\right)+0.1\times \log \left(\frac{0.1}{0.5\times 0.5}\right)\end{array},$$

(3)

By using the traversal algorithm, we can find the maximal value from all possible division schemes as $I^{\ast }\left(D,\hspace{0.17em}x,\hspace{0.17em}y\right)$ $=\max I(D{|}_G)$. Then, the characteristic matrix $M{(D)}_{x,y}$ is constructed through normalization operation on $I^{\ast }\left(D,\hspace{0.17em}x,\hspace{0.17em}y\right)$ as $M{(D)}_{x,y}=I^{\ast }\left(D,\hspace{0.17em}x,\hspace{0.17em}y\right)/\log \min \{x,\hspace{0.17em}y\}$. Whereupon the MIC can be obtained according to $MIC(D)=\underset{x,y<B(n)}{\max }\{M{(D)}_{x,y}\}$.

Finally, for the relation of $y=R\left(x\right)$, $x=\left[x_1,x_{2,},\dots ,x_n\right]$, the weight information of elements in x can be obtained by calculating the MIC between each independent variable of $\left[x_1,x_{2,},\dots ,x_n\right]$ and dependent variable y.

2.1.2. Value Model Construction

Value is a measurement to denote the importance degree of an object. In the military field, a frequently used measurement for denoting value is the capability gap, meaning the gap between system capabilities and capability requirements. Before calculating the capability gap, it’s necessary to obtain system portfolio capabilities, which are the combination of those of component systems.

Firstly, let $CR=\left\{c{r}_1,\dots ,c{r}_k\right\}$ be the set of capabilities requirements, proposed by stakeholders, where $c{r}_j$ denotes the value of the jth capability requirement. The $C(s_i)=\left(c_{i1},\dots ,c_{ik}\right),C(s_i)\in {\{0,1\}}^m$ represents the capability value of the system $s_i$, where $c_{ij}$ denotes the capability value of $s_i$ on the jth capability. The $PC(x_i)=\left(p{c}_1(x_i),\dots ,p{c}_k(x_i)\right)$ denotes the capabilities values of the system portfolio corresponding to scheme $x_i$, where $p{c}_j(x_i)$ is the jth capability value.

In the combination process, one case must be considered is that certain systems in a system portfolio may have the same capabilities. It’s a key procedure to deal with the combination of this kind of capabilities. Hereon, four rules are introduced as follows to support this combination.

Assume there are n systems $s_i,i=1,\dots ,n$ in a system portfolio, providing the same capability $t$, with capability values:$c_{it},i=1,\dots ,n$.

(1)

Additive rule: the combined capability value is ${\displaystyle \sum c_{it}},i=1,\dots n$. E.g., Assuming that 3 transportation systems operate at the same time, the freight volume are 5t, 6t and 7t respectively, then the portfolio of the 3 systems can provide a freight capability of 18t.

(2)

Maximal rule: the combined capability value is $max\left\{c_{it}\right\},i=1,\dots ,n$. E.g., Assuming that there are three bridges over a river, and each bridge can bear the weight of 100t, 120t and 130t respectively, then only object less than 130t can pass the river, because one object can only pass over one bridge at the same time.

(3)

Minimal rule: the combined capability value is $min\left\{c_{it}\right\},i=1,\dots ,n$. E.g., Assuming that there are 3 tandem oil pipelines with oil flow of 5 t/hour, 6 t/hour and 3 t/hour, then the max oil flow of the 3 pipeline is 3 t/hour.

(4)

Average rule: the combined capability value is ${\displaystyle \sum c_{it}}/n,i=1,\dots n$. E.g., Assuming there are 3 forecast systems, with correctly predicting probability 50%, 60%, and 70%, then the overall correctly predicting probability is 60%.

In addition, the capability can be classified into benefit and cost types. Firstly, the relation between a single capability of a solution and corresponding capability requirement is defined based on three premises: (1) When a capability of a solution is deficient or worse than the inferior value of the corresponding capability requirement interval, which means the solution absolutely cannot meet the capability requirement. Its capability gap is supposed to be 1. (2) When a solution can provide a capability, with its value falling in the interval of corresponding capability requirement, its capability gap should be a number between $\left[0,1\right]$. Especially when the solution’s capability value exactly falls in the middle of the capability requirement interval, the capability gap is supposed to be 0.5, which means the capability can meet the capability requirement by half. (3) When a capability of a solution equals to or better than the superior value of the corresponding capability requirement interval, it means that the capability can absolutely meet the corresponding capability requirement, and the capability gap is supposed to be 0.

A concrete case is given in Figure 2, reflecting the relation between a capability and corresponding capability requirement interval $\left[l,u\right]$. Four special points discussed above are marked by circle points. To mathematically model the relations with universal forms, a formula is defined to fit the linear line above, shown in Equation (4).

$$G\left(c_i,c{r}_i\right)=\{\begin{array}{l}\{\begin{array}{l}-c_i/\left(2a\right)+1,\hspace{1em}0\le c_i<a\\ \left(-c_i+u\right)/\left(2b\right),\hspace{1em}a\le c_i<u\\ 0,\hspace{1em}u<c_i\end{array}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em};benefit\hspace{0.17em}type\\ \{\begin{array}{l}1,\hspace{1em}{c}_i=0\\ 0,\hspace{1em}0<c_i<l\\ \left(c_i-l\right)/\left(2b\right),\hspace{1em}l\le c_i<a\\ \left(c_i+worst-2a\right)/\left(2\times \left(worst-a\right)\right),a\le c_i<worst\end{array};\hspace{0.17em}Cost\hspace{0.17em}type\end{array},$$

(4)

where $G\left(c_i,c{r}_i\right)$ represents the capability gap between the ith capability and corresponding capability requirement interval $\left[l,u\right]$. Two special notation a and b are defined to simplify the formula, where a = (l + u)/2, b = (u − l)/2. The notation worst denotes the requirement threshold of cost-type capability that the capability gap will be judged to be 1 if a cost-type capability value exceeds worst.

Then, the capability gaps of all capabilities should be aggregated to compute the total capability gap of a system portfolio as ${\displaystyle {\sum }_{i=1}^k{w}_{i}G\left(c_i,c{r}_i\right)}$, where the $w_i$ denote the weight of the ith capability and is determined according to the method in Section 2.1.1. Then, according to capability gaps, the value is defined as Equation (5), which indicates that the bigger the total capability gap, the smaller the value of a system portfolio.

$$V\left(x_j\right)=\frac{{\displaystyle \sum _{i=1}^k{w}_i\left(1-G\left(c_i,c{r}_i\right)\right)}}{k},{\displaystyle \sum _{i=1}^k{w}_i}=1,$$

(5)

Different to general “capability-based” evaluation methods, which are usually based on hierarchy structures such as a tree structure that the capabilities of all systems in bottom level are up-aggregated to an integrated value, capability gap is a criterion denoting the gap between capabilities and capability requirements. For general methods, an inevitable defect is that an extremely high capability will pull up the integrated capability value, which is obviously unreasonable. Whereas for the capability gap, it can mitigate the effects of extreme capability values. For example, even an infinitely high benefit-type capability can only result in corresponding capability gap of 0, instead of unreasonably extreme value.

2.2. Risk Decision Based on RUL Prediction

General risk assessment methods follow the steps of risk factor identification, risk analysis, risk assessment and risk management. However, for weapon systems, decision-makers mainly focus on the availability and stability of systems in the operation process. Therefore, the RUL, which denotes when a system is predicted to fault, is used as a weapon system risk criterion. The longer the RUL, the smaller the probability a risk will happen in the operational process. In addition, the RUL prediction method based on similarities of degradation characteristics is proved to perform better than other classical methods when there are sufficient historical samples. Therefore, the paper adopts degradation characteristics similarities to predict the RUL to indicate system portfolio risks.

The key resource for predicting RUL is the operational data, which is typically obtained from embedded sensors with time series. In specific, the degradation data is analyzed to support the construction of the RUL prediction model and the main steps can be concluded as degradation track phase space construction, track matching, and RUL prediction. Detailed procedures are given in Figure 3.

2.2.1. Feature extraction based on variation coefficient

When multiple sensors are used to monitor the health of a system, a key step for predicting health states is selecting features from the multidimensional time series data, because only parameters without commonality can be treated as attributes features. Therefore, parameters screening is the prerequisite to ensure the non-commonality of parameters. Intuitively, practitioners should firstly deal with correlations of parameters to make the transformed parameters independent.

In this paper, the statistic of variation coefficients is used to select the parameters features. Similar to the concept of standard deviation and variance in statistics, the variation coefficient is described as the dispersion degree of observations. The variation coefficient is calculated by the ratio of the standard deviation to the mean, as Equation (6) shows. Therefore, it is dimensionless.

$$VC=\left(StandardDeviation/Mean\right)\times 100\%.$$

(6)

Variables with larger variation coefficients show more obvious features and therefore are more suitable for describing characteristics of data.

2.2.2. Reconstruction of Degradation Track Phase Space

According to Taken’s theorem, the potential dynamics laws of a system can be studied by constructing the phase space that preserves the topological properties of the original system. Coupled with the nonlinear characteristics of degradation, the time delay embedding theorem is often applied to construct high-dimensional phase space.

Assume that the time series is $X=(x_1,\hspace{0.17em}{x}_2,\hspace{0.17em}\cdots ,\hspace{0.17em}{x}_N)$, then the points in the phase space can be expressed as the following form of row vector.

$$X_i=(x_{i-(d-1)\tau },\hspace{0.17em}{x}_{i-(d-2)\tau },\hspace{0.17em}\cdots ,\hspace{0.17em}{x}_{i-\tau },\hspace{0.17em}{x}_i),\hspace{0.17em}i=1+(d-1)\tau ,\hspace{0.17em}2+(d-1)\tau ,\hspace{0.17em}\cdots ,\hspace{0.17em}N,$$

(7)

where d is the embedded dimension of phase space and $\tau$ is the time delay.

Embedded dimension d is the key parameter in phase space construction, whose value will be determined by the non-subjective algorithm. $X_i(d)$ is a point in the d dimension phase space. If the point $X_{n(i,d)}$ nearest to $X_i(d)$ exists, the following relation can be deduced.

$$\Vert X_i(d)-X_{n(i,d)}(d)\Vert =\underset{j=1+(d-1)\tau ,\cdots ,N,j\ne i}{\min }{\Vert X_i(d)-X_j(d)\Vert }_{\infty },$$

(8)

For all points in the d and d + 1 dimensional phase spaces, the definitions for $a(i,d),E(d)$ $E^{\ast }(d),E_1(d)$, and $E_2(d)$ are given respectively.

$$\begin{array}{c}a(i\hspace{0.17em},\hspace{0.17em}d)=\frac{\Vert X_i(d+1)-X_{n(i,d+1)}(d+1)\Vert }{\Vert X_i(d)-X_{n(i,d)}(d)\Vert }\\ E(d)=\frac{1}{N-d\tau }{\displaystyle \sum _{i=1+(d-1)\tau }^{N-\tau }a(i,d)}\\ E_1(d)=\frac{E(d+1)}{E(d)}\\ E^{\ast }(d)=\frac{1}{N-d\tau }{\displaystyle \sum _{i=1}^{N-\tau }\left|x_{i+d\tau }-x_{n(i,d)+d\tau }\right|}\\ E_2(d)=\frac{E^{\ast }(d+1)}{E^{\ast }(d)}\end{array},$$

(9)

Then the value of the embedded dimension d can be determined by finding the smallest spatial dimension that makes the d and d + 1 dimensional phase spaces topologically equivalent. The topological equivalence here refers to that the nearest neighbors in the d dimensional space still remain the closest in the d + 1 dimensional space, which means that the $E_1(d)$ tends to be a stable value when the d and d + 1 dimensional phase spaces are topologically equivalent. However, in reality, it is difficult to find the smallest d that makes $E_1(d)$ a stable value. That’s why the $E^{\ast }(d)$ and $E_2(d)$ are defined. If x is a random time series, then $E_2(d)$ will vary with d, or $E_2(d)$ will approach 1. Then by combining $E_1(d)$ and $E_1(d)$, the embedded dimension of the degradation track phase space can be determined. After the construction of the phase space, the time series can be transformed into the form of special track. By referring to the degradation data transformed from historical data and comparing the similarity of the new track with the reference track in phase space, the RUL can be predicted.

2.2.3. System Portfolio Risk Determination Based on RUL

Assume that $Z=\{Z_1,\hspace{0.17em}{Z}_2,\hspace{0.17em}\cdots ,\hspace{0.17em}{Z}_{l_1}\}$ represents the degradation reference track and $Y=\{Y_1,\hspace{0.17em}{Y}_2,\hspace{0.17em}\cdots ,Y_{l_2}\}$, $l_1>l_2$, is the current degradation track. Then, the normalized cross correlation (NCC) is introduced to measure the similarity between tracks. The specific definition of NCC is given as follows:

$$s_{YZ}(i)=\frac{(Y-\overline{Y})\times (Z_i-{\overline{Z}}_i{)}^{\prime }}{{\Vert Y-\overline{Y}\Vert }_2{\Vert Z_i-{\overline{Z}}_i\Vert }_2},$$

(10)

where $Z_i=\{Z_i,\hspace{0.17em}{Z}_{i+1},\hspace{0.17em}\cdots ,\hspace{0.17em}{Z}_{i+l_2-1}\}$ and $Y$ are of the same length. $\overline{Y}$ and ${\overline{Z}}_i$ are the mean vectors of $Y$ and $Z_i$ respectively.

Taking the jth time series $x^j=\{x_1^j,\hspace{0.17em}{x}_2^j,\hspace{0.17em}\cdots ,x_{N_j}^j\}$ as an example, the time points of the time series are denoted as $t^j=\{t_1^j,\hspace{0.17em}{t}_2^j,\hspace{0.17em}\cdots ,\hspace{0.17em}{t}_{N_j}^j\}$. Apparently, $t_{N_j}^j$ is the running life of the j th degradation process. Taking time delay $\tau$ as 1 and considering embedded dimension as d for all degradation track phase spaces (d is no less than the maximal embedded dimension of all phase spaces). Then the original time series of degradation process is transformed into the track of phase space. Denoting the reference track for the jth process as $Z_j={[X_d^j,\hspace{0.17em}{X}_{d+1}^j,\hspace{0.17em}\cdots ,\hspace{0.17em}{X}_{N_j}^j]}^T$, the points in phase space are expressed as $X_i^j=[x_{i-(d-1)}^j,\hspace{0.17em}{x}_{i-(d-1)}^j,\hspace{0.17em}\cdots x_i^j]$.

Assuming the current degradation time series is $y=\{y_1,\hspace{0.17em}{y}_2,\hspace{0.17em}\cdots ,\hspace{0.17em}{y}_c\}$. Similarly, it is transformed into an incomplete degradation track form denoted as $Y=[Y_d,\hspace{0.17em}{Y}_{1+d},\hspace{0.17em}\cdots ,\hspace{0.17em}{Y}_c]$, $Y_i=[y_{i-(d-1)},\hspace{0.17em}{y}_{i-(d-2)},\cdots ,\hspace{0.17em}{y}_{i-1},\hspace{0.17em}{y}_i]$.

When performing track matching, a track subset $Y^k=[Y_{k-l+1},\hspace{0.17em}\cdots ,\hspace{0.17em}{Y}_k]$ is selected with a time window length of l in the current phase space. Then the NCC between the current track $Y^k$ and the reference track $Z_j$ should be modified as follows.

$$s_k^j=s_{YZ}+(1-\frac{\left|t_k-t_k^j\right|}{t_k}),$$

(11)

If the $s_k^j$ reaches the maximum at the time point $T_k^j$, then sub-tracks at time $T_k^j$ are regarded as the best matching results in terms of degradation reference track $Z_j$. This result also reflects the influence of the track shape to the degradation stage. Therefore, the RUL of the most similar part to the jth degradation reference track can be estimated as $L_k^j=t_{N_j}^j-T_k^j$.

On the basis of the above result of the best track matching, the weight of the remaining life calculated according to the jth degradation reference track is given as $w_k^j=s_k^j(T_k^j)/\left({\displaystyle {\sum }_{j=1}^M{s}_k^j(T_k^j)}\right)$, where M is the total number of the degradation processes. Then the remaining life of the system k is estimated as Equation (12).

$$RU{L}_k={\displaystyle \sum _{j=1}^M{\omega }_k^j{L}_k^j},$$

(12)

Then, the RUL should be transformed into risk according to the joint probability. It is identified as a risk event when any system in a system portfolio breaks. Therefore, for a system portfolio $x_i=\left(s_1,s_2,\dots ,s_n\right)$, the risk is calculated as Equation (13) shows.

$$R\left(x_i\right)=1-{\displaystyle \prod _{j=1}^n\left[{\left(1-\frac{1}{RU{L}_j}\right)}^{x_{ij}}\right]},$$

(13)

In conclusion, the RUL is an important criterion for the weapon system research because longer RUL can reduce the failure risk of systems in combat. The availability of each component system is the basis to guarantee the normal operation of systems portfolios. An operation activity will be broken or even unsuccessful if any system of a system portfolio malfunctions in the operation process. This induces the demand for long RUL of systems.

2.3. System Portfolio Optimization

As discussed above, the paper tries to solve the system portfolio problem considering system values and risks. The corresponding notation is as follows. Let $S=\left\{s_1,\dots ,s_m\right\}$ denotes the set of alternative systems, indicating m systems can be selected for a system portfolio. $x_i=\left(x_{i1},\dots ,x_{im}\right),x_i\in {\{0,1\}}^m$ is one of system portfolio schemes, where $x_{ij}=0\backslash 1$, with $x_{ij}$ = 0 denoting the jth system in S is not selected in scheme $x_i$, and $x_{ij}$ = 1 denoting the jth system in S is selected in scheme $x_i$. All possible schemes compose the solution space X. Let $E=\left(e_1,\dots ,e_m\right)$ represents the cost of systems, where $e_i$ is the cost of system $s_i$, and B denotes the total budget.

Then, the system portfolio optimization model can be formulated as follows.

$$\begin{array}{l}\begin{array}{cc}\hspace{1em}max& V\left(x_j\right)=\frac{{\displaystyle \sum _{i=1}^k{w}_i\left(1-G\left(c_i,c{r}_i\right)\right)}}{k}\end{array}\\ \begin{array}{cc}\hspace{1em}min& R\left(x_j\right)=1-{\displaystyle \prod _{i=1}^k\left[{\left(1-\frac{1}{RU{L}_i}\right)}^{x_{ji}}\right]}\end{array}\\ s.t.\{\begin{array}{l}{\displaystyle \sum _{i=1}^m{x}_{ji}\times e_i}\le B\\ x_j=\left(x_{j1},\dots ,x_{jm}\right),x_j\in {\{0,1\}}^m\end{array}\end{array},$$

(14)

where $V(x_j)$ and $R(x_j)$ are the value and risk of the system portfolio scheme $x_j$. $G\left(c_i,c{r}_i\right)$ represents the capability gap on the ith capability. In addition, the cost of scheme $x_j$ must within the budget limitation.

It can be seen that the optimization is a multi-objective problem and the 2 objectives are conflict with each other. Therefore, an optimal system portfolio with the best performance in both 2 objectives does not exist. The target is to obtain the Pareto optimal solutions of 2 objectives, also known as non-dominated solutions.

3. Results

3.1. Background Description

It is hypothesized that the problem aims to select an optimal system portfolio under the anti-missile scenario, where an object will suffer saturated missiles attacks. The objective is selecting a system portfolio from 100 alternative systems ($2^{100}-1$ candidate system portfolios in total) under the budget limitation to maximize the system portfolio value and simultaneously minimize the whole risk.

According to the operation process of OODA, capabilities discussed in the paper are $c1$ detection range, $c2$ communication range, $c3$ striking range and $c4$ decision time, where the former three capabilities are beneficial type and the last one is cost type. In a specific operation scenario, the capability requirements and combination rules are shown in

Table 1. Capabilities requirements.

Capabilities	$\mathit{c}1 \left(\mathbf{Km}\right)$	$\mathit{c}2 \left(\mathbf{Km}\right)$	$\mathit{c}3 \left(\mathbf{Km}\right)$	$\mathit{c}4 \left(\mathbf{Second}\right)$
CR	[100,200]	[100,250]	[150,300]	[10,30]
Rules	Maximal	Addition	Addition	Average

.

In addition, for capabilities of systems, they are generated by executing a Monte Carlo simulation method according to truncated normal distribution functions. The histogram of generated data is shown in Figure 4. The worst value of cost-type capability decision time is 34.6853, which will be used in value calculation according to Section 2.1.2.

3.2. Value Calculation

3.2.1. Weight Determination

Based on simulations on the “Command: Modern Air/Naval Operations”, an ultimate military simulator for modern military conflicts, the weight information can be deduced by the correlation analysis.

The independent variables are four capabilities, that is detection range, communication range, striking range and decision time. The dependent variable is the intercepted missile number. By auto-simulating for 10,000 times, 10,000 sets of data are generated. Through the MIC algorithm, the corresponding results can be obtained, as

Table 2. Obtained maximal information coefficient (MIC) of input and output.

MIC	Intercepted Missile Number
Detection range	0.854
Communication range	0.772
Striking range	0.537
Decision time	0.928

shows. Through normalization, the weight of four capabilities are determined as 0.276, 0.250, 0.174, and 0.300.

3.2.2. Value Calculation

Because it is impossible to calculate all values of $2^{100}-1$ candidate system portfolios, an example of the value calculation process is introduced. Assuming a system portfolio SP1 have 5 component systems of S1, S2, S3, S4, and S5, with capability information shown in

Table 3. Capabilities of 5 systems.

Systems	Detection Range	Communication Range	Striking Range	Decision Time
S1	128.8502018	45.00221559	41.93626171	22.55909356
S2	205.1604693	36.35948504	50.01541323	16.53053513
S3	156.9600287	27.11329846	43.28357399	16.46184426
S4	152.7120943	26.19071935	53.57115127	21.31900886
S5	111.5170147	15.40553804	29.95161145	15.33123079
SP1	205.1604693	150.0713	218.758	18.44034

.

According to capability combination rules in Table 1, the combined capabilities of system portfolio SP1 are shown in the last column in Table 3.

Then, according to Equation (4), the capability gaps of four combined capabilities are calculated as Equation (15) shows. The value of any system portfolio can be calculated based on the same steps.

$$\begin{array}{l}G\left(c1,Cr1\right)=0\\ G\left(c1,Cr1\right)=\left(\frac{-150.0713}{100+250}\right)+1=0.571\\ G\left(c1,Cr1\right)=\left(\frac{-218.758}{150+300}\right)+1=0.514\\ G\left(c1,Cr1\right)=\left(\frac{18.44034-10}{30-10}\right)=0.422\end{array},$$

(15)

3.3. Risk Determination Based on RUL Prediction

In the case study, the key component of weapon systems, the turbine engine, is taken as an example for analyzing risks. The data is derived from the experiment conducted by a commercial modular simulation software C-MAPSS as shown in Figure 5.

The C-MAPSS simulates the operation of a turbine engine with 900,000-pound thrust and records monitoring signals. Based on the principle of thermodynamics, two failure modes are designed: high-pressure compressor degradation and fan degradation. The main functional modules and connections are shown in Figure 6. The simulation runs in the following settings:

(1)

The simulation experiment data contains time series of 21 variables. It can be further divided into a training set and a testing set. Each multivariate time series corresponds to a specific engine, meaning that the data can be considered to be generated by engines of different systems.

(2)

The initial wear condition of each engine might not be identical and there are manufacturing variations, which are considered reasonable and not treated as reasons of engine failures.

(3)

There are 3 operational setting parameters that have a substantial impact on an engine’s performance.

(4)

There are noises in the data.

(5)

The engines operate normally at the initial moment and begin to degrade at some points in time series. In the training set, the cumulative degradation quantity continues to grow until it reaches or exceeds the preset threshold. In the testing set, the time series will terminate when engines fail.

As a result, 100 degradation tracks are obtained in the training set and 100 tracks before failure in the testing set. The training data is used to establish the RUL prediction model of engines, and the testing set is used to test the feasibility of the model.

The monitoring data is shown in the scatter plots in Figure 7. Each plot visualizes the 100 degradation tracks of one variable in the training set. The engine code, the operation cycle and the 3 operational setting parameters are not shown in the figure.

Due to the fact that the constant variable is unable to reflect the evolution of engine degradations, variable 1, 5, 6, 10, 16, 18, and 19 are not regarded as feature variables. What’s more, the tracks exhibit different trends in terms of variable 9 and 14, so variable 9 and 14 are inadequate to describe the degradation process.

Then, the variation coefficients of the rest 12 variables are calculated based on the degradation data and the results are shown in

Table 4. Variation coefficients (VC) of the rest 12 variables.

Variable	Sensor 2	Sensor 3	Sensor 4	Sensor 7	Sensor 8	Sensor 11	Sensor 12	Sensor 13	Sensor 15	Sensor 17	Sensor 20	Sensor 21
VC	0.0567	0.2760	0.5110	0.1239	0.0020	0.4573	0.1117	0.0020	0.3408	0.2907	0.3519	0.3514

. According to the rule of eliminating variables with small variation coefficients, variable 3, 4, 11, 15, 17, 20 and 21 are chosen as the base variables that represent engine degradation characters.

According to the RUL prediction method, the remaining life of the engines in the testing set can be estimated by matching the testing data with the reference tracks. Then, the risks can be obtained. The results are shown in

Table 5. The prediction results of the remaining life of different weapon systems.

S	RUL	Risk	S	RUL	Risk	S	RUL	Risk	S	RUL	Risk
S1	112	0.008929	S26	197	0.005076	S51	114	0.008772	S76	76	0.013158
S2	104	0.009615	S27	162	0.006173	S52	72	0.013889	S77	78	0.012821
S3	97	0.010309	S28	82	0.012195	S53	78	0.012821	S78	199	0.005025
S4	92	0.01087	S29	75	0.013333	S54	187	0.005348	S79	167	0.005988
S5	133	0.007519	S30	83	0.012048	S55	189	0.005291	S80	109	0.009174
S6	105	0.009524	S31	75	0.013333	S56	75	0.013333	S81	75	0.013333
S7	81	0.012346	S32	150	0.006667	S57	76	0.013158	S82	75	0.013333
S8	61	0.016393	S33	121	0.008264	S58	75	0.013333	S83	202	0.00495
S9	137	0.007299	S34	75	0.013333	S59	190	0.005263	S84	74	0.013514
S10	66	0.015152	S35	74	0.013514	S60	87	0.011494	S85	162	0.006173
S11	107	0.009346	S36	75	0.013333	S61	72	0.013889	S86	156	0.00641
S12	107	0.009346	S37	99	0.010101	S62	68	0.014706	S87	203	0.004926
S13	79	0.012658	S38	76	0.013158	S63	67	0.014925	S88	126	0.007937
S14	136	0.007353	S39	208	0.004808	S64	74	0.013514	S89	79	0.012658
S15	197	0.005076	S40	74	0.013514	S65	200	0.005	S90	79	0.012658
S16	188	0.005319	S41	94	0.010638	S66	74	0.013514	S91	72	0.013889
S17	79	0.012658	S42	73	0.013699	S67	200	0.005	S92	75	0.013333
S18	101	0.009901	S43	79	0.012658	S68	75	0.013333	S93	69	0.014493
S19	171	0.005848	S44	185	0.005405	S69	197	0.005076	S94	79	0.012658
S20	75	0.013333	S45	78	0.012821	S70	87	0.011494	S95	187	0.005348
S21	156	0.00641	S46	82	0.012195	S71	202	0.00495	S96	189	0.005291
S22	182	0.005495	S47	119	0.008403	S72	130	0.007692	S97	176	0.005682
S23	182	0.005495	S48	163	0.006135	S73	190	0.005263	S98	121	0.008264
S24	75	0.013333	S49	28	0.035714	S74	117	0.008547	S99	189	0.005291
S25	203	0.004926	S50	195	0.005128	S75	195	0.005128	S100	73	0.013699

.

3.4. Portfolio Selection Results Analysis

In total, there are $2^{100}-1$ possible schemes, which is a huge number. Thus, a heuristic algorithm is necessary to be applied to the solving process. Considering the value and risk factors, the objectives are maximizing the value and minimize the risk of system portfolios. Therefore, a multi-objective algorithm is employed to solving the optimization problem. The non-dominated sorting generic algorithm (NSGA) is a kind of widely used multi-objective algorithm, which exhibits a good performance for retaining elites in offspring. On the other side, the differential evolution (DE) is a nice genetic operator, which plays really well on keeping population diversity. Thus, the paper uses the non-dominated differential evolution (NSDE) algorithm, which fuses the two advantages of NSGA and DE, to solve the system portfolio optimization problem. The corresponding parameters are set as follows. The population size is Pop = 100, the number of iterations is Gen = 1000, the mutation probability is 0.01 and the crossover probability is 0.2.

Due to the certain randomization of all genetic algorithms, The NSDE also generates results with certain fluctuant. A typical method to guarantee the optimality of generated result is running the algorithm for multiple times, and then select the best individuals by comparing the corresponding multiple results. In the case, the program is iterated for 10 times to generate 10 Pareto results with each containing 200 individuals, shown in (a) of Figure 8. Then, the 10 sets of Pareto results are combined together to obtain the best 200 individuals among them, as shown in (b) of Figure 8.

In detail, the 200 individuals of the best Pareto set are shown in the system option diagram in Figure 9. The rectangular area is divided into 100 × 200 rectangles according to the number of system candidates and the dimension of the non-dominated weapon system portfolios. Each rectangle represents whether a system candidate is selected in the Pareto set. If a weapon system i is selected by the j th non-dominated system portfolio, the i th row and j th column rectangular block will be colored black, otherwise, it is left blank.

From Figure 10, it can be seen that some systems are frequently selected in the Pareto set. However, some systems are seldom selected or even never selected. To compare the importance degree of different systems, the frequencies for all systems of being selected in the Pareto set is counted. Systems of S6, S9, S15, S16, S22, S25, S32, S39, S50, S52, S65, S69, S70, S71, S72, S73, S75, S79, S83, S90, S98, and S99 are selected by at least one system portfolio in the Pareto set. In addition, the systems S9, S25, S50 are quite important according to the high selected numbers in the Pareto set. Further, the rank of systems according to selected numbers is: S25 > S9 > S50 > S39 > S69 > S75 > S83 > S22 > S71 > S32 > S73 > S16 > S98 > S72 > S15 > S6 > S65 > S99 > S70 > S79 > S52 > S90, which to some extend indicates the importance degrees of selected systems. As regarding to the rest systems, they can be directly neglected in the system portfolio selection process.

By deeper analysis, it can be discovered that some systems tend to always be selected together. Therefore, a frequent item set mining algorithm of Apriori is applied to identify the association rules, shown in

Table 6. The association rules mined by frequent item set mining.

Ranking by Support			Ranking by Confidence
Association Rules	Support	Confidence	Association Rules	Support	Confidence
9$\to$25	0.615	0.8723	75$\to$25	0.255	1.0000
25$\to$9	0.615	0.7593	[9,39]$\to$25	0.28	0.9032
50$\to$25	0.57	0.8906	[9,50]$\to$25	0.465	0.9029
25$\to$50	0.57	0.7037	50$\to$25	0.57	0.8906
50$\to$9	0.515	0.8047	9$\to$25	0.615	0.8723
9$\to$50	0.515	0.7305	[25,50]$\to$9	0.465	0.8158
[9,50]$\to$25	0.465	0.9029	50$\to$9	0.515	0.8047
[25,50]$\to$9	0.465	0.8158	[25,39]$\to$9	0.28	0.7887
[9,25]$\to$50	0.465	0.7561	25$\to$9	0.615	0.7593
50$\to$[9,25]	0.465	0.7266	[9,25]$\to$50	0.465	0.7561
9$\to$[25,50]	0.465	0.6596	69$\to$50	0.25	0.7353
[9,39]$\to$25	0.28	0.9032	9$\to$50	0.515	0.7305
[25,39]$\to$9	0.28	0.7887	50$\to$[9,25]	0.465	0.7266
39$\to$[9,25]	0.28	0.6222	25$\to$50	0.57	0.7037
75$\to$25	0.255	1.0000	9$\to$[25,50]	0.465	0.6596
69$\to$50	0.25	0.7353	39$\to$[9,25]	0.28	0.6222

. The support parameter indicates the ratio between the simultaneously appearing frequency and all items, which means the probability of appearing simultaneously. The confidence of the rule of “A$\to$B” represents the ratio of $support\left(A\cup B\right)/support\left(A\right)$, which means the probability of $A\cup B$ when $A$ appears.

In Table 5, the association rules are ranked by the value of support and confidence respectively. Firstly, according to the ranking by support, it can be elicited that the “S9$\to$S25” is the most frequent rule, which means they tend to be selected together. In addition, when system S75 is selected, the system S25 must also be selected according to the first rule in the ranking by confidence. Referring the association rules, decision-makers can have a deeper understanding of the significance of system portfolios.

4. Discussion

The paper shows the feasibility of replacing expert subjective expertise with knowledge obtained from data. Firstly, the weight information of capabilities is determined by analyzing correlations between capabilities and the intercepted missile numbers, based on operation simulation data. Then, as regards the risk criterion, the paper tries to determine the risk by mining information from system operation data. The data-driven methods are only components of the model-based approaches, aiming to increase the accuracy and credibility of results.

In the case study, 100 system candidates are provided to be optimized on the scenario of anti-missile. By automatically simulating the operation scenario for 10,000 times, 10,000 simulation results are generated, according to which, the maximal information coefficients between four capabilities and the variable of intercepted missile quantity are calculated as the weight of capabilities. It quantitatively indicates that the capability of decision time has the biggest impact on the interpreted missile quantity. In addition, by running the simulator of C-MAPSS for 200 times, 200 groups of system operation data are generated, according to which, systems risks are obtained through prediction of RUL.

In the system portfolio optimization, considering the great number of $2^{100}-1$ candidate system portfolios, the NSDE algorithm is applied to solving the optimization problem. To guarantee the optimality of the result as far as possible, 10 Pareto sets are obtained by running the NSDE for 10 times. 200 non-dominated individuals are reserved by comparing the 10 Pareto sets. However, it can be not proved the best Pareto set, due to the almost infinity of candidate system portfolios and the randomness of genetic algorithms. By further analyzing the characters of generated Pareto set, 22 systems are selected at least one time, and 16 association rules are mined. These characters can play an assistant rule for decision-makers to make a deeper understanding of the system portfolios.

In conclusion, the system portfolio selection is the mainstream trend of future equipment development. Compared to other traditional system portfolio decision and optimization methods, the proposed model and data-driven approach provide a solution to avoid the excessive dependence on subjective expert experience in the evaluation and decision process. Traditionally, determining these parameters requires cumbersome processes of organizing experts, collecting expert opinions, analyzing expert scores etc., which are time and effort-consuming and more likely to be questioned. The model and data-driven methods can make use of models that have been proved to be effective on one hand, and on the other hand, it can determine the required parameter values in the model through data analysis. Therefore, it supports more efficient, more credible, and more practical evaluation and decisions in system portfolio selection and other fields applications.