Probabilistic Modelling of System Capabilities in Operations

Abstract

We present a probabilistic method for simplifying the complexities in evaluating high-level capabilities. The method is demonstrated by using questionnaire data from a technology forecasting project. Our model defines capability as the probability of a successful operation. The model maps the actual observable capabilities to system capabilities. These theoretical quantities are used in calculating observable capability values for system combinations that are not directly evaluated as explanatory variables in the model. From the users’ point of view, the method is easy to use because the number of evaluated parameter values is minimal compared with many other methods. The model is most suited to applications where the resilience and effectiveness of systems are central factors in design and operations. Resilience is achieved by using alternative systems that produce similar capabilities. We study the limited use of alternative systems and their capabilities. We present experiments of model variations and discuss how to perform the model’s built-in consistency checks. The proposed method can be used in various applications such as comparing military system capabilities, technological investments, medical treatments and public education. Our method can add a novel view for understanding and identifying interrelations between systems’ operations.

1. Introduction

We present a probabilistic model with the primary goal of supporting understanding, analysis, communication and decision making. We demonstrate the model in three military capability areas of protection, awareness and engagement, a subset of military capabilities [1,2,3] at the highest conceptual hierarchy level. The basic model has a simple structure of necessary and alternative systems that produce capabilities on a specified hierarchy level. In this study, capability is defined as the success probability of operation in an existing or planned scenario [4]. Despite the origin of the model [4,5,6,7,8,9,10] being in military applications, it can be used in other domains where the resilience and effectiveness of systems are central factors in design and operations.

Capability-based planning (CBP) takes place under uncertainty, the results of which are definitions of capabilities in a wide range of different requirements and environments, considering economic factors and possible alternatives [11,12,13,14,15]. One definition of capability is the ability to perform a specified set of tasks [13,15]. To achieve this, writing scenarios is an important method for describing various operations and operating environments [4]. A conceptual model for defining total capability areas is used by the defence forces in many countries. Military capability areas are defined as statistically independent sets of functionalities. These functionalities form hierarchies [14,15], with the help of which the exact capability areas are defined [2]. Modelling can be performed at different levels of hierarchy and levels of detail [16,17]. The modeller decides to which level of detail the model is designed based on the requirements of the problem [2]. Throughout this study, we use the term capability area because our numerical examples in Section 3.3, Section 3.4, Section 5.1 and Section 5.2 are built on this concept.

Our probabilistic model [5,7,8,9,10] describes the capabilities of a system of systems (SoS) [1,13,15,18,19,20,21,22]. Under the general concept of capability, we consider high-level capability areas, subcapabilities, capabilities of a system of systems and individual system capabilities [7,8]. We use two types of concepts: observable capability for an actual system of systems and system capability for a theoretical decomposition of a system of systems. This is a practical approach in a case where detailed system characteristics are not available, for example, when forecasting long-term technological progress [10,23,24]. In addition, these quantities are easier for a subject expert to evaluate than the more technical individual systems’ capabilities. In the model, numerical values of capabilities are estimated probabilities of successful operations. Therefore, capabilities depend on the scenario and operation at hand. In our numerical demonstrations, we use the capability values of high-level capability areas and subcapabilities of two systems, Unmanned Aerial Vehicles (UAVs) and satellites, from a questionnaire as input data. While we have presented the model in our earlier articles, new features of the model are discussed in the context of using subsets of available systems.

We show how the method can be applied in modelling the concurrent use of subsets of available systems. Two different ways of using the model are adding (or removing) systems from an existing configuration and deploying existing systems only for a limited set of capabilities. An example of the latter is when protection and awareness capabilities are fixed for a system, for example, UAVs can only utilise their own combined protection and awareness capabilities. On a higher level, this does not exclude the possibility of deploying UAVs and satellites for producing the same capabilities.

Several extensions of the basic model show how complex interrelations can be considered. In the military context, one important application is describing systems’ self-protection capabilities. Self-protection of an alternative system is a particular component not included in the common supporting systems or in the model’s cross-terms of alternative systems. Self-protection components of alternative systems are treated as a part of the protection capability of the systems themselves.

The research question of this study is to develop a mathematical model for simplifying the evaluation of high-level capabilities in operations. The main applications are in the military domain or other critical environments where the redundancy of systems is a central means to increase the resilience [25] of a system of systems. Our goal is to develop a method for analysing trends from questionnaire data or other uncertain information. One application is technology forecasting, where technological developments are not known accurately.

Current methods, such as the Delphi method or traditional linear regression methods, require the evaluation or fitting of many variables in the corresponding models. In our model, the number of estimated numerical values is not increasing as a function of different combinations but linearly as a function of the number of alternative systems. This is achieved by modelling individual systems as independent entities that can be used as ’building blocks’ in composing different combinations of necessary and alternative systems. The proposed method is easy to use, and it is possible to visualise modelling results in real time during the evaluation process. The model is designed for describing the high-level capabilities or subcapabilities of a limited set of system functionalities. Other system engineering techniques [1,16,26] or trade-off methods [27] may be more appropriate in modelling functional system requirements or operations given that the detailed information is available. In this respect, the proposed method has limitations but it fills a specific gap in the existing literature of system engineering methods. As the concept of military utility [28] is closely related to the concept of military capability, we also present some principles of military utility in Section 2. In the literature review, we also discuss some other general topics related to our work, such as trade-off analysis [27] and technology forecasting [7,17,29].

In this study, we demonstrate the model in the military context. It is easy to find applications in many other domains. The model can be used, for example, in comparing alternative options of training and education programs [30], public procurement [31], energy use and sources [32], means of transportation [33], medical treatments [34] and technological intensity and innovation [35].

In all applications, it is worth noticing that the use of the model is based on the probabilistic definition of capability. Typically, in each application, many alternative definitions are possible. For example, in the case of evaluating training programs’ future earnings, future lifetime, meaningful work, etc., can be considered as targets for evaluating the probability of success. Moreover, what we mean by a successful event or operation must be defined. For example, USD 100,000 or above as one year’s salary is one choice for the target level. It is possible to define a discrete or continuous variable for different earning levels. Even a combination of different performance or quality levels for each evaluation target can be studied. Even multiple targets of evaluation can be considered simultaneously with different numerical levels for each target of evaluation. If needed, more than one scenario can be used in defining the situational environment.

2. Literature Review

In the literature, few mathematical models have been proposed for describing high-level capabilities. However, there is an extensive volume of literature in related fields of system engineering [20,21,22,27] and systems of systems [18]. General methodologies such as Multiple-Criteria Decision Analysis (MCDA) [36], Structural Equation Modelling (SEM) [37] and Analytic Hierarchy Process (AHP) [38] are related to our work, but they are typically used for more detailed modelling problems. A nonparametric method called Technology Forecasting with Data Envelopment Analysis (TFDEA) [29] has been used successfully in the literature. In the TFDEA method, the system is not modelled in detail, but nevertheless, the correct selection of input data is essential to obtain correct predictions.

Different trade-off methods [14,27] are often used in the decision-making process where alternatives are examined in uncertain circumstances. These decisions can vary from choosing a management style, hiring personnel and choosing the optimal systems for operations. The processes of trade-off analysis [27] consist of the following steps: understand the problem, find and define the alternatives, define the criteria, set the weights for criteria, determine the scoring, analyse the results and make decisions. These steps should be largely carried out before the modelling work, depending on the functional and technical requirements of the research question at hand. Similar phases also exist when implementing the proposed method of this study.

The concept of military utility has been proposed for the evaluation of technological systems in military operations [28]. It considers the selection of military technologies and how those technologies are used. The concept is useful in military decision-making processes such as technology foresight, operational planning, development and use of defence systems. It affects performance on the battlefield and the ability to keep up over time. Military utility is a measure of military effectiveness, affordability and suitability in specified circumstances [28]. It is derived using concept analysis [39] and is based on similar concepts in social and system sciences. With the help of the concept of military utility, it is possible to study what military capabilities are made of, the effects of the development of technologies and their use and what the effects of different military doctrines are.

In technology forecasting [17,29], the goal is to find out changes in performance and reveal the sources that improve capabilities that arise from the introduction of new systems and improvements to existing systems. Structural system modelling and probabilistic description of capabilities can be realistic forecasting methods when compared with standard mathematical methods, such as linear regression, extrapolation, factor analysis and principal component analysis.

3. Basic Iterative Model

We construct the model for a system of systems from parallel and serial subcapabilities, modelling their respective alternative and necessary capabilities. In detailed analyses, capabilities can be modelled using nested structures of parallel and serial components. In our probabilistic model, we calculate the total effect of necessary (serial) capabilities by taking the product of capability values. However, dealing with alternative (parallel) capabilities is more involved. Particularly, if more than two alternative systems are considered, the calculation is tedious without using the iterative procedure in Section 3.2.

First, we present the basic case where all systems can be described by their corresponding system capabilities that are used in the probabilistic model for calculating the capability of a capability area. These methods can be used in cases where the modelling of subsystems is difficult or there is not enough information for more detailed modelling. A practical iterative procedure is helpful in such calculations. After that, we present methods and applications for describing more complex system interrelationships.

3.1. Preliminaries

In the following, we present an iterative method for adding (or removing) new systems to a system of systems. With the iterative procedure, calculating capabilities for individual system capabilities, or for any combinations of systems, is effortless and easy. The iterative method works in a basic case when there are no complex interrelations between systems. Another use case of the concept of system capabilities is to calculate the observable capabilities of a system of systems in different scenarios or future technological progress for lower or higher system capabilities. Using the basic model, the simplest method to estimate the effects of changing system capability values is to scale the system capability values and calculate relevant observable capability values for system configurations in the operation [10].

After presenting the mathematical method for calculating system capabilities for the basic case, we present two complex variations of the model where the mathematical formulas can be solved for two or three added systems. Numerical methods can be used to solve the polynomial formulas for more than three additional systems. The order of the polynomial increases with the number of additional systems. In practice, numerical methods for three systems may be more practical, even for three new systems. The two complex cases’ internal capabilities and external system capabilities are presented in Section 5.1 and Section 5.2, respectively.

Next, we show how parallel capabilities can be computed iteratively. We denote existing capabilities by M and K, where M describes the necessary system of systems of existing capabilities and K describes the system of systems of existing alternative capabilities. This is the baseline, at time $T=0$, of the analysis. New systems $\{S_1,\dots ,S_d\}$ are introduced in addition to the existing system of systems $\{K,M\}$. This configuration is shown in Figure 1. The system capabilities of systems $M,K,S_i,i=1,\dots ,d$ are denoted by $X_m,X_k,X_i,i=1,\dots ,d$.

The capability value of the combined system of systems $\{K,M\}$ is denoted by $p_0$. Because the two systems are serial, we have $p_0=X_m{X}_k$. The value of $p_0$ is approximated by a constant because new systems’ functionalities can compensate for existing capabilities. This approximation is consistent with our earlier results [6,7,9], where the quantity $X_m{X}_k$ is a constant with an increasing value of $X_m$ and a decreasing value of $X_k$ in most cases. This is interpreted as increasing the capability of necessary system capabilities and decreasing the capability of alternative system capabilities of existing partially compensated systems. In more detailed modelling, the assumption of a constant $p_0$ is not made, and $p_0$ is replaced by $p_T$, which allows changes as a function of time T.

In the context of this study, where a subset of the system of systems $\{K,M\}$ is in operation, the capability values of $X_k$ and $X_m$, or the combined effect $X_m{X}_k$, may depend on the particular subset of deployed systems or other decisions made of the systems’ use. In these cases, corresponding effects should be evaluated and included in the model. One such example is the increased or decreased system capability of a system when deployed in different scenarios. Two methods for dealing with internal and external system capabilities are presented in Section 5.1 and Section 5.2, respectively. In the following, we assume that the value of $p_0$ is a constant function of time, and it is not dependent on using a subset of systems or the restrictions resulting from decision making.

3.2. The Model

We denote by $p_{1\dots i}$ the change in an observable capability value with i additional systems in a capability area. These values are obtained from questionnaire results or are evaluated through modelling or simulation methods. First, we add a new system $S_1$ with the capability value of $X_1$ in parallel with the system of systems K. According to the basic probability theory, specifically, the theorem of nonmutually exclusive events [40], the capability value of these three systems $\{K,M,X_1\}$ is

$$p_0+p_1=X_m(X_1+X_k-X_1{X}_k)=X_m{X}_1+p_0-X_1{p}_0.$$

(1)

From Equation (1) $X_1$ can be solved as

$$X_1=\frac{p_1}{X_m-p_0}.$$

In Equation (1), the left side represents the observable capability and the right side describes the system capabilities of our model. For one added system, the capability value of $X_m$ (or $X_k$) is required. Next, we add a second system with the capability value of $X_2$. Here, we assume that $\{K,M,S_1\}$ is not affected when system $S_2$ is added. The system of systems capability value of the four systems $\{K,M,S_1,S_2\}$ is

$$p_0+p_{12}=X_m(X_1+X_2+X_k-X_1{X}_2-X_1{X}_k-X_2{X}_k+X_1{X}_2{X}_k)=p_0+(X_m-p_0)A_{12},$$

where $A_{12}=1-(1-X_1)(1-X_2)$.

The value of $X_2$ can be solved as the value of $X_1$ is known:

$$X_2=\frac{1}{1-X_1}\left(\frac{p_{12}}{X_m-p_0}-X_1\right).$$

(2)

From the above, we know that $X_m-p_0$ is $p_1{X}_1^{-1}$, which is independent of the initial capability area’s capability value $p_0$. This feature of the model is desirable because the initial values are the predicted values from the questionnaire. It is possible to adjust the values of $p_0$ without affecting the system capability values $X_i,i=1,\dots ,d$. On the other hand, the system capability values of the original defence system $\{K,M\}$ depend on $p_0$. We presented and discussed an example of adjusting, or re-evaluating, the value of $p_0$ in a technological forecasting application in [6] Section 2.4. In the model, this is achieved simply by replacing $p_0$ by $a_m{p}_0$ with a new parameter $a_m$ describing the relative change.

In general, the following iterative formula for system capabilities holds:

$$X_i=\frac{1}{1-A_{i-1}}\left(\frac{p_{1\dots i}}{X_m-p_0}-A_{i-1}\right),$$

(3)

where

$$A_i=1-\prod _{s=1}^i(1-X_s),i=2,\dots ,d;A_0=0,A_1=X_1.$$

(4)

For two parallel systems, the value of $X_m-p_0$ can be solved with the help of the known value of $p_{12}$:

$$X_m-p_0=\frac{p_1{p}_2}{p_1+p_2-p_{12}}.$$

(5)

The iterative procedure works when new parallel systems are added to the existing system of systems. The interpretation is that the capability values of an increasing set of a system of systems $\{M,K,S_1,\dots ,S_i\}$ are known for $i=0,1,\dots ,d$. Alternatively, the same number of capability values of different combinations of systems are known. With the help of the model, different capability values for individual systems or combinations of systems can be calculated using the probabilistic model.

On the system level, our model makes use of the system of systems principles. In our numerical demonstrations in Section 3.3, Section 3.4, Section 5.1 and Section 5.2, two systems are assembled in parallel or series with other systems. This idea can be compared with the ideas in Martino’s more heuristic model [17]. It has some resemblance with the proposed model of this study regarding its intended use and mathematical form as a quotient. However, Martino’s model is not derived from the probability theory, and it is not specifically designed for modelling high-level capabilities or for investigating the resilience of a system of systems. The model does not involve the concept of system capability as defined in this study. As a result, all system combinations of a system of systems cannot be calculated from the same input data, as is the case in our model.

3.3. Limited Use of Systems

System capabilities may be used in a limited manner for a variety of reasons: (a) all potential capabilities are not available, (b) all potential capabilities are decided not to be used, or (c) there is not enough time or other resources to deploy all available capabilities. Respondents in the questionnaire were not informed about any restrictions on using different combinations of system capabilities. Thus, the system capability values calculated from the questionnaire data correspond to the general case where system capabilities can be used interchangeably. For example, the protection capability produced by satellites can be used to protect UAVs.

When modelling the use of subsets of potential systems, a specific question is how to take into account the reduced requirements caused by not using all of the theoretically possible systems. One important example is the absence of protection for systems not deployed in operation: If UAVs are not in operation, they do not need protection. This means that the protection capability provided to UAVs by satellites or other operative systems needs to be removed from calculations (i.e., not included).

In Figure 2, we provide an example where UAVs are not used for maintaining situational awareness, yet UAVs alone are used for engagement operations. In the example of Figure 2, systems $\{X_{p,m},X_{p,k}\},\{X_{a,m},X_{a,k},X_1\}$ and $\{X_{e,e},X_2\}$ are used for protection, awareness and engagement, respectively. Note that the system capability values for necessary systems $X_m$ and alternative systems $X_k$ usually have different values for protection, awareness and engagement capability areas. In this particular case, the protection capability provided to UAVs’ surveillance operations by the Original Defence System (ODS) and satellites is not in effect. If this result is important, this part of the protection capability should be eliminated from the calculations. This can be carried out just by scaling the corresponding system capability value by a suitable factor or using the methods in [8,10] depending on the situation.

Removing (and adding) systems from a set of a system of systems is a built-in feature of the model—a system is removed just by setting the system capability value of the system to zero. This is straightforward when removing or adding systems and it does not change any other systems’ capability values. In many cases, this is true or, at least, a good approximation for small and moderate changes in the system composition.

In

Table 1. Average initial values over three scenarios for three capability areas.

Prot	Awa	Eng
0.567	0.600	0.533

, we provide the average initial observable capability values $p_{p,0}$, $p_{a,0}$ and $p_{e,0}$ for the protection, awareness and engagement capability areas, respectively. Notice, that these values are kept constant during the evaluation horizons of 1 year, 10 years and 20 years. The results of this study are conditional on this assumption. In

Table 2. Average values over three scenarios of the questionnaire results for three capability areas, two systems and three forecasting periods. In the questionnaire, the number of system units was not fixed. System of systems K is included in all of these evaluations.

Capability Area	Systems	1 Year	10 Years	20 Years
Prot	Sat	0.037	0.071	0.082
	UAV	0.040	0.087	0.143
	Sat and UAV	0.065	0.124	0.171
Awa	Sat	0.048	0.058	0.072
	Uav	0.073	0.081	0.099
	Sat and UAV	0.091	0.101	0.125
Eng	Sat	0.018	0.022	0.043
	Uav	0.037	0.043	0.079
	Sat and UAV	0.040	0.046	0.090

, we provide the observable capability values from the questionnaire for the three capability areas in the three evaluation horizons for systems $\{S_1,K\}$ and $\{S_2,K\}$, as well as the combined use of systems $\{S_1,S_2,K\}$. Notice that, in the questionnaire, the auxiliary system of systems K is also included in the evaluations. In

Table 3. System capability values calculated from the model. The three rows in the table correspond to the three time horizons of the valuations 1 year, 10 years and 20 years.

Prot				Awa				Eng
${\mathit{X}}_{\mathbf{1}}$	${\mathit{X}}_{\mathbf{2}}$	${\mathit{X}}_{\mathit{m}}$	${\mathit{X}}_{\mathit{k}}$	${\mathit{X}}_{\mathbf{1}}$	${\mathit{X}}_{\mathbf{2}}$	${\mathit{X}}_{\mathit{m}}$	${\mathit{X}}_{\mathit{k}}$	${\mathit{X}}_{\mathbf{1}}$	${\mathit{X}}_{\mathbf{2}}$	${\mathit{X}}_{\mathit{m}}$	${\mathit{X}}_{\mathit{k}}$
0.317	0.339	0.685	0.828	0.413	0.625	0.716	0.838	0.382	0.792	0.580	0.920
0.398	0.486	0.746	0.760	0.463	0.644	0.725	0.827	0.438	0.836	0.584	0.913
0.380	0.659	0.784	0.723	0.465	0.636	0.756	0.794	0.407	0.738	0.640	0.834

, system capability values $X_1$, $X_2$, $X_m$ and $X_k$ calculated from the model of Section 3 are provided for the three capability areas and time horizons of 1 year, 10 years and 20 years.

In Figure 3, the left-hand side figures show observable capability results from the questionnaire data and the right-hand side figures show observable capability results calculated from the model in Section 3. Here, we do not perform the full analysis, but some observations and conclusions can be made from the results. The calculated results (right) can reveal phenomena that are not easy to see from the original questionnaire data (left). The curves for systems $\{M,S_2\}$ highlight the important role of the system $S_2$ (UAVs) in producing the protection capability in the long run. In addition, system $S_2$ is expected to produce engagement capability both in short and long time horizons.

3.4. Limited Use of System Capabilities

Next, we assume that the use of system capabilities in an operation is not utilised with full power: We assume that a parallel system can use only its own protection and awareness capabilities in order to produce a combined capability for its two capabilities. Yet, the combined system capability, comprised of protection and awareness, can be utilised interchangeably between alternative systems. This example is particularly relevant because the protection and awareness capabilities are highly related and they support each other. Equations for the combined protection and awareness capabilities of systems $\{M,S_1,S_2\},\{M,K,S_1\},\{M,K,S_2\}$ and $\{M,K,S_1,S_2\}$ are the following:

$$\begin{array}{cc}\hfill & P_{12}=X_{p,m}{X}_{a,m}(X_{p,1}{X}_{a,1}+X_{p,2}{X}_{a,2}-X_{p,1}{X}_{a,1}{X}_{p,2}{X}_{a,2}),\hfill \\ \hfill & P_{k1}=X_{p,m}{X}_{a,m}(X_{p,k}{X}_{a,k}+X_{p,1}{X}_{a,1}-X_{p,k}{X}_{a,k}{X}_{p,1}{X}_{a,1}),\hfill \\ \hfill & P_{k2}=X_{p,m}{X}_{a,m}(X_{p,k}{X}_{a,k}+X_{p,2}{X}_{a,2}-X_{p,k}{X}_{a,k}{X}_{p,2}{X}_{a,2}),\hfill \\ \hfill & P_{k12}=X_{p,m}{X}_{a,m}(X_{p,k}{X}_{a,k}+X_{p,1}{X}_{a,1}+X_{p,2}{X}_{a,2}-\hfill \\ \hfill & X_{p,k}{X}_{a,k}{X}_{p,1}{X}_{a,1}-X_{p,k}{X}_{a,k}{X}_{p,2}{X}_{a,2}-X_{p,1}{X}_{a,1}{X}_{p,2}{X}_{a,2}+\hfill \\ \hfill & X_{p,k}{X}_{a,k}{X}_{p,1}{X}_{a,1}{X}_{p,2}{X}_{a,2)}),\hfill \end{array}$$

(6)

where subscripts p and a denote protection and awareness. Note that system capability values $X_{p,i},X_{a,i},i=1,2,k,m$ are calculated from the basic model where restrictions were not applied. By default, the necessary system of systems M is in use, and we do not indicate that on the left-hand side of the formulas. The system capability $X_m$ is a multiplicative factor on the right side of the formulas. If the necessary system of systems M is not functioning, all the capability values are zero in Equation (6).

In Figure 4, the combined observable protection and awareness capability values for the limited use of systems, as defined in this section, are compared with the basic results of Section 3. The dashed lines show that the effect of limited-use systems has an effect of about five per cent for systems $\{M,S_1,S_2\}$ and about three per cent for systems $\{M,K,S_1\}$, $\{M,K,S_2\}$ and $\{M,K,S_1,S_2\}$.

4. Interrelations between Systems

In theory, different complex interrelations between systems can be modelled by modifying the system capability equations. Usually, only correcting effects can be taken into account using these approaches. In the following, three different approaches with potential applications are discussed. Another application is how to adjust a system’s capability value in different scenarios or according to future technological developments [10]. In this case, all occurrences of the system capability value are multiplied by the same adjusting factor. If changing a system capability value has no effect on other system capabilities, exact results are obtained by this method.

Modelling the higher or lower capability of the concurrent use of two systems is presented in [8]. In this study, we present two additional variants of modifying the system capability equations. They describe external and internal system capabilities. External capabilities could describe auxiliary or supporting systems that are not included in the detailed model.

It may be useful to model internal systems because, for example, self-protection can be modelled by modifying the system equations with this method. Self-protection is the protection provided by a system to itself. Improved manoeuvring or camouflage are examples of self-protection. Here, the concept of self-protecting includes only the system capability $(1-a_1)X_1$ of the corresponding parallel system capability, not the multiplicative system capability $X_m$:

$$p_0+p_1=X_m(a_1{X}_1+(1-a_1)X_1+X_k-a_1{X}_1{X}_k)=X_m(X_1+X_k)-a_1{X}_1{p}_0.$$

(7)

The protection capability value of system $X_1$ is separated into two parts: self-protection $(1-a_1)X_1$ and the remaining portion $a_1{X}_1$ of the capability. The cross-term has only the term $a_1{X}_1$, describing protection without the self-protection component. In case system K also has self-protection capability, the system equation is modelled accordingly: the cross-term is multiplied by a factor $a_{1k}=a_1{a}_k$.

5. Experiments with Variations of the Model

In this section, we present two different parameterisations of the model. They can approximate internal and external use capabilities as a supplementary part of modelled system capabilities. In Section 3.4, the basic model is modified by extra factors in the cross-terms of Equation (8). In Section 5.2, the basic model is modified by extra factors in the system capabilities of Equation (11).

On the grounds of these formulas, the variations can have distinguishing interpretations as real-world changes in the success probabilities of operations. Depending on the parameter $a_1$ value, the formulas have different relative weights for the system capability terms and the cross-terms describing the redundancy of the systems. One interpretation is to consider the two variants as internal and external capabilities that are not explicitly included in the model.

These variations of the model can also serve as a ‘What if’ analysis or as a way of testing the allowed range of parameter values in the model. We used the model’s self-consistency features to determine the parameter $a_1$ range in Figure 5 and Figure 6.

5.1. Internal System Capabilities

Self-protection is one of the commonly used concepts to consider protection capabilities produced by the system for its own protection. Internal capabilities can be modelled by decreasing corresponding cross-terms in the formulas. After using the relationship $p_0=X_m{X}_k$, we obtain from Equation (7) the following system equations for modified capability values:

$$\begin{array}{cc}\hfill & p_1=X_m{X}_1-a_1{X}_1{p}_o,\hfill \\ \hfill & p_2=X_m{X}_2-a_2{X}_2{p}_o,\hfill \\ \hfill & p_{12}=X_m{X}_1+X_m{X}_2-a_{12}{X}_m{X}_1{X}_2-a_{1k}{X}_1{p}_0-a_{2k}{X}_2{p}_0+a_{12k}{X}_1{X}_2{p}_0.\hfill \end{array}$$

(8)

Observable capability values for a capability area are $p_0+p_i,i=1,2,12$, as before. Equation (8) can be expressed as a second-order equation

$$c_1{X_1}^2+c_2{X}_1+c_3=0,$$

where the coefficients $c_1,c_2$ and $c_3$ are

$$\begin{array}{cc}\hfill & c_1=p_2{p}_0(a_{12k}-a_{1k}{a}_{12}),\hfill \\ \hfill & c_2=p_0(a_{1k}-a_{2k})p_1-a_{12}{p}_1{p}_2+a_{1k}{p}_0{p}_2-a_{2k}{p}_0{p}_2-p_{12},\hfill \\ \hfill & c_3={p_1}^2-p_1{p}_2-p_{12}{p}_1.\hfill \end{array}$$

(9)

Now, we can use the textbook formula of second-order polynomials for $X_1$ and solve the quantities $X_2,X_m$ and $X_k$:

$$\begin{array}{cc}\hfill & X_2=\frac{X_1{p}_2}{p_1{X}_1{p}_0(a_{1k}-a_{2k})},\hfill \\ \hfill & X_m=\frac{p_1+p_0{a}_{1k}{X}_1}{X_1},\hfill \\ \hfill & X_k=\frac{p_0}{X_m}.\hfill \end{array}$$

(10)

Figure 5 shows that internal capabilities have effects on observable capabilities for different time horizons. We assume that the initial capability $p_0=X_m{X}_k$ remains constant. The observable capability values of systems $\{M,S_1\}$, $\{M,S_2\}$ and $\{M,S_1,S_2\}$ are decreasing, but the observable capability value of $\{M,S_1,S_2,K\}$ is increasing as a function of the parameter value $a_1$. This indicates the increasing role of the existing alternative system of systems K as a function of $a_1$. The parameter value $a_1=1$ corresponds to the basic case of Section 3 and Figure 3.

5.2. External System Capabilities

Complex interrelations with external systems can be modelled by adjusting corresponding terms in the equations. External systems should be modelled in detail, but approximate methods may be sufficient for describing some external capability effects. The system capability equations and their solutions are provided in Equations (11) and (13). Practical applications are presented in Figure 6. For systems $\left\{S_1,K\right\}$, $\left\{S_2,K\right\}$ and $\left\{S_1,S_2,K\right\}$, the capabilities are

$$\begin{array}{cc}\hfill & p_0+p_1=X_m(a_1{X}_1+a_k{X}_k-X_1{X}_k)=X_m{a}_1{X}_1+p_o-X_1{p}_o,\hfill \\ \hfill & p_0+p_2=X_m(a_2{X}_2+a_k{X}_k-X_2{X}_k)=X_m{a}_2{X}_2+p_o-X_2{p}_o,\hfill \\ \hfill & p_0+p_{12}=X_m(a_1{X}_1+a_2{X}_2+a_k{X}_k-X_1{X}_2-X_1{X}_k-X_2{X}_k+X_1{X}_2{X}_k).\hfill \end{array}$$

(11)

where $a_1\ge 0.9$ and $a_k\ge 0.9$ describe the changed capabilities of systems $S_1$ and K due to their effects via external capabilities. Notice that the consistency of the model allows also a slight decrease caused by the external system capabilities (see Section 7.2). The equation for the initial capability $p_0=X_k{X}_m$ is replaced by $p_0=a_k{X}_k{X}_m$. From Equation (11), the four equations and the unknown variables $X_1,X_2$ and $X_k$, as well as $X_m$, can be solved. The following quadratic equation follows for $X_1$, which usual textbook methods can solve

$$p_0{p}_2(a_1-1)X_1^2+(p_0(p_{12}-p_1-p_2)(a_1-a_2)-p_1{p}_2{a}_k)X_1-(p_{12}-p_1-p_2)p_1{a}_2{a}_k=0.$$

(12)

The system capability value $X_2$ for system $S_2$ has the symmetric formulation where $X_1$ and $a_1$ are exchanged with $X_2$ and $a_2$. On the other hand, after $X_1$ is known, $X_2,X_m$ and $X_k$ can be solved as

$$\begin{array}{cc}\hfill & X_2=\frac{a_1{a}_k{p}_2{X}_1}{a_2{a}_k+p_0(a_2-a_1)X_1},\hfill \\ \hfill & X_m=\frac{p_1{a}_k+p_1{X}_1}{a_1{a}_k{X}_1},\hfill \\ \hfill & x_k=\frac{p_0}{a_k{X}_m}.\hfill \end{array}$$

(13)

Figure 6 shows that external capabilities have an effect of decreasing observable capability values as a function of the parameter value $a_1$. The observable capability values of systems $\{M,S_1\}$, $\{M,S_2\}$, $\{M,S_1,S_2\}$ and $\{M,S_1,S_2,K\}$ are all decreasing as a function of $a_1$. The parameter value $a_1=1$ corresponds to the basic case of Section 3 and Figure 3.

6. Results

In this study, we presented a method for modelling system capabilities in operations. The method is based on describing the success probability of operations with different systems in use. Necessary and alternative systems are modelled with the basic probability theory. In our demonstrations, we used questionnaire data from two systems, satellites and UAVs, together with auxiliary necessary and alternative systems. General formulas for more than two alternative systems can be expressed as an iterative procedure, as explained in Section 3.2.

We use the concept of system capability for the derived quantities of the model $X_i,i=1,\dots ,d$, where the number of alternative systems is denoted by d. These quantities can be regarded approximately as independent of other systems in the scenario, that is $X_i$ is statistically independent of quantities $X_j,j\ne i$. This assumption enables us to calculate all combinations of system capabilities, in addition to the ones that are direct consequences of the model input data, which in our example are the questionnaire data. It is noteworthy that we need not evaluate (or estimate) all different system combinations: it is sufficient to evaluate only $2+d$ values. For example, if there are two alternative systems, evaluating the four capability values $p_0,p_1,p_2,p_{12}$ is sufficient. In Section 4, we showed how to calculate the capabilities of three additional system combinations (right in Figure 3). In case there are more systems, this property of the model is even more important. A side effect is that we could have evaluated any four of the eight possible combinations of the capability (see Figure 2, where each capability area has eight combinations, including $X_m$). Certainly, one should choose the systems which can be evaluated the most easily and reliably.

Moreover, we can analyse the limited use of alternative systems such that only independent use of a system or system combination is feasible. This kind of situation can arise as a result of technical failures or operational restrictions. In Section 3.3, we demonstrated the limited use of alternative systems based on the same questionnaire data as in Section 3. This analysis also provides a lower bound for the system and observable capabilities (see the dashed lines in Figure 4).

7. Discussion

In this section, we discuss the role of observable and system capabilities. Then, we introduce the model’s unique characteristics that can be used for checking the consistency and validation of the model’s results.

7.1. Observable Capabilities and System Capabilities

In this study, we demonstrated our probabilistic modelling method by analysing an empirical questionnaire data set. The main idea of the method is to map the operation-level success probability evaluations to system-level capability values. All quantities of the model are expressed as probabilities that constitute a basis for consistency checks for the answers provided by the respondents and also for the model’s derived results. If the model yields inconsistent results, or probability values outside the range $[0,1]$, the model does not apply, the model structure is not detailed enough, or the model parameter values are incorrect. This kind of validation test is not a common property of all system engineering models [26,27], not to mention any other models.

Formulas for more than two necessary systems are expressed as multiplicative factors for each system or a set of systems. In theory, an arbitrary configuration of a system composed of several (sub)systems can be modelled with our method. However, in practice, it may be difficult to differentiate and define all necessary and alternative functions of the analysed systems and to represent the relations of the systems as a diagram to help write the equations.

In Figure 3, Figure 4, Figure 5 and Figure 6, we chose to present the observable capability values and not the theoretical system capability values, because the former are more relevant in practice as actual quantities. System capability results can be found in our earlier studies [5,6,7,8], or they can be calculated from the formulas of these studies.

Mathematically, it is possible to evaluate the system capability values instead of the initial capability value and observable capability increment values (in our example, $p_0,p_1,p_2$ and $p_{12}$). In any case, the observable capability values expressed as success probabilities should be shown in real time to the participants of a questionnaire. Vice versa, it would also be helpful to show the calculated system capability values to the participants when the actual capability values are evaluated. This would be another consistency check because the system capability values are comparable with each other, while the evaluated capability values of different or aggregated system combinations may not be so informative.

In the proposed model, our definition of system capabilities enables calculating the observable capabilities in different combinations by using the probabilistic formulas in Section 3. This procedure is novel and original when compared with other models in the literature. For example, the heuristic model proposed by Martino [17] or any other general methods reviewed in Section 2 require the evaluation of more numerical values of model variables, typically for each combination of systems. We discussed other related literature at the end of Section 1.

On the other hand, if the combined use of systems is complicated, that is, it does not obey the rule of nonmutually exclusive events of the probability theory [40], systems should be modelled in a lower level of subsystems. This may not be possible in all cases, and then other methods are more suitable, for example, the system engineering methods mentioned in Section 2. Even though, in theory, it is possible to model a system on a detailed level, it may be difficult to conceptually define the success probabilities in the model or to describe the interrelations between systems with basic probabilistic formulas. This can be a limitation of the proposed model. In this study, we also discussed how these kinds of complexities could be modelled approximately in some cases by introducing additional phenomenological parameters in the model.

7.2. Consistency and Validation of the Results

Because the concept of capability is defined as a probability, the model has a built-in consistency check for the modelling results. Formulas that map the observable capability values to the theoretical system capabilities can produce system capability values that are negative or higher than 1. If we hold on to the probability interpretation, output variables must be in the range $[0,1]$.

In our original questionnaire, for example, the $X_m$ value for the protection capability for a 10 year forecasting period in Scenario 1 had the value $X_m=1.12$. We have discussed possible causes for this nonphysical result in [5] and in Section 2.4 in [6]. Note that the overflow does not come up in this work, as we analyse average results over the three scenarios of the original questionnaire compensating for individual outliers in the calculations. Outliers are possible because there were only 10 respondents in the questionnaire, which is a low number with which to obtain statistically accurate results. In summary, there are several possible explanations for the discrepancy: (a) too low a number of participants in the questionnaire, (b) a bias in a significant portion of input data [10], (c) protection capability as a multifaceted concept that is difficult to figure out and evaluate [5], (d) the model is too simple to describe the protection capability, etc.

The model for a system of systems consisting of d parallel systems in addition to the auxiliary parallel system of systems K needs $2+d$ explanatory variables for auxiliary necessary systems, auxiliary alternative systems and d alternative systems. The model has degrees of freedom in how to choose the evaluated explanatory variables. For example, in the case of two alternative systems, we have eight possible variables. We can use the four leftover combinations of systems for checking purposes, or we can evaluate more than the required number of observable capability values. It would be informative to visualise all response variables of the model, both observable capabilities and theoretical system capabilities.

Moreover, the system capability values can be used for consistency checks because they are expressed as separate quantities which can be compared with each other, unlike the cumulative capability values in the iterative process explained in Section 3. The iterative algorithm is based on adding new systems into the system of systems one by one. As we have defined capability as the probability of a successful operation, added alternative systems produce less additional observable capability because redundancy is increasing. This emphasises the importance of monitoring the system capability values during the evaluation process and tuning evaluated values when needed. Notice that the formulas in Section 3 apply only to the cumulative method of iteratively adding a system to an existing set of systems. If the observable capabilities are handled in some other order, the formulas must be derived for that case.

8. Conclusions

Our proposed probabilistic models can assist in simplifying subjective complexities and providing descriptions that can be used as heuristics to understand and manage systems and their capabilities. Although the approach has some limitations in describing detailed complicated structures of systems, it can be useful in many applications, for example, in forecasting long-term technological progress, where detailed information about systems’ characteristics and performance is not available. On the other hand, the method can be used as a tool to analyse questionnaire results where the opinions or subjective views of the participants are surveyed.

The novelty of our work is in presenting a probabilistic method for simplifying the complexities in evaluating the high-level capabilities defined as success probabilities in operations. This is accomplished by using the basic probability theory and introducing the concept of system capability. In the model, the system capabilities are approximately independent of other system capabilities in the scenario. In other words, the model maps the actual observable capabilities to independent system capabilities that can be used in calculating capability values for new system combinations that have not been evaluated in the original questionnaire or estimated in a submodel. The new system combinations can be either a combination of several theoretical system capabilities or an observable system of systems capabilities. This enables calculating capabilities of different system combinations from the same input data without evaluating all different combinations, for example, contrary to the Delphi method [24].

The method is easy to use from the user’s point of view. The number of different numerical values to be evaluated is small compared with many other models in the literature. Yet, the model provides capability values for system configurations that have not been directly included in the evaluation process. The model can be used to present these modelling results and visualisations simultaneously with the evaluation work. These features of the model help to check and validate the evaluations.

We demonstrated the use of system capabilities in three different ways. We calculated the capability values of system combinations that were not directly considered in the questionnaire, and we calculated the effects of the limited use of systems as alternatives in producing capabilities in operations. Thirdly, we experimented with variants of the model by using the same questionnaire data as input. These calculations can be used to test the boundaries of the model’s parameter range and to examine how sensitive the model results are to variations of the model.

We discussed how the consistency of the results can be checked and how the modelling results can be validated by monitoring the calculated system capabilities of individual systems and combinations of systems. We used consistency tests in selecting the viable parameter ranges of our model variants of internal and external capabilities.

There are many practical applications where our method can be used in modelling the success probabilities of operations or as a complementary instrument to other system engineering methods. The method can be used in almost all branches of activities from technological investments to medical treatments and public education. Our method can add a novel view for understanding and identifying interrelations between systems’ operations that can be useful in validating other models being used and their parameter values.