Decisions in Risk and Reliability: An Explanatory Perspective

Abstract

The paper discusses issues that surround decisions in risk and reliability, with a major emphasis on quantitative methods. We start with a brief history of quantitative methods in risk and reliability from the 17th century onwards. Then, we look at the principal concepts and methods in decision theory. Finally, we give several examples of their application to a wide variety of risk and reliability problems: software testing, preventive maintenance, portfolio selection, adversarial testing, and the defend-attack problem. These illustrate how the general framework of game and decision theory plays a relevant part in risk and reliability.

1. Introduction

Decision (and game) theory is becoming more relevant to addressing reliability and risk issues under uncertainty. A recent contribution was made by Ríos Insua et al. [1]. For many applications, quantifying risk and reliability measures is a means to an end; to be useful, those values must be used to derive optimal policies to minimise risk, to decide on maintenance strategies, or to make some other decision. The methods of decision theory provide coherent mathematical approaches to doing this. When decisions are made in an environment where there are competing players, game theory is the relevant mathematical approach; examples include adversarial commercial situations or minimising risks from criminal or terrorist threats.

In this paper, we address these topics with illustrative examples taken mainly (but not exclusively) from our work. Some of them are worked out analytically, while others require numerical computations; however, the reasoning behind each example is explained, and the reader is referred to the relevant references for a complete discussion. The target of the paper is twofold. On one hand, it is addressed to people who have no or limited knowledge about methods and models in decision theory applied to reliability and risk problems. On the other hand, this paper illustrate recent approaches, such as adversarial risk analysis, or possible uses of older but less known methods such as Bayesian robustness; therefore, this paper could also interest readers with more in-depth knowledge. There are so many methods, models, and application fields that could be used to achieve the above goals but they are limited to a few in this paper. The choice has fallen on a few topics in reliability (although also related to risk) and a particular risk, from finance. In reliability, we identified three relevant areas (software testing, maintenance, and acceptance sampling) while portfolio selection has been discussed in the context of financial risk. We were compelled to leave out many other works, for example, on natural disasters (flooding and earthquakes), or other papers of ours such as Ebert et al. [2] on queues at immigration control in airports and Trucco et al. [3] on human and organisational factors affecting safety in maritime transport. The examples chosen not only are representative of some areas in the field but also allow us to introduce approaches that could be of interest for readers such as influence diagrams, decision trees, Bayesian robustness, and adversarial risk analysis.

In Section 2, we give a historical account of how the development of risk and reliability methods is connected to the progress in probability theory and utility theory for normative decision making. In Section 3.1, we introduce the basic concept of utility-based decision-making. Decision problems are best framed with the help of graphical representations, so we consider decision trees and influence diagrams in Section 3.2. Examples of applications of decision trees for replacement in maintenance optimisation follow in Section 3.3. In Section 4, we include other application examples where a single decision maker deals with software testing, preventive maintenance, and portfolio selection. In Section 5, we examine problems in risk and reliability analysis that involve two or more actors with competing interests in the adversarial risk analysis framework. Section 6 concludes the paper.

2. A Brief History of Quantitative Risk and Reliability

The section is not intended to provide a complete history of how risk and reliability became the object of quantitative study and then of interest in decision theory. Rather it mentions some works that have had an impact on this process. It shows how the current research in decision analysis, as applied to risk and reliability, is the result of centuries of work, which have grown independently until they have found common ground. More extensive illustrations can be found in papers about history of statistics such as Singpurwalla and Wilson [4].

The history of quantitative risk methods is tied closely to the history of probability theory, both of which have roots in insurance. Christiaan Huygens (1629–1695) was one of the earliest scientists to think mathematically about risk, motivated by problems in annuities widely due, at that time, to states and towns borrowing money. Huygens wrote up the solution, due to Fermat and Pascal, to a gambling problem called “The Problem of Points”, where the question is to determine the fair bet for a game where each player has an equal chance of winning and the bet is won as soon as either player wins the game a predetermined number of times. Huygens stated an axiom on the value of a fair game, which is considered the first approach to the notion of expectation. Huygens [5] is thus credited with publishing the first book on probability theory.

The idea of a fair price was linked to probability by Jacob Bernoulli (1654–1705) in a book [6] published posthumously by his nephew Nicholas as the first substantial treatment of probability. Jacob Bernoulli changed the focus from expected values, which were tediously calculated by a recursive approach, towards probabilities using his Law of Large Numbers. Expected value then became a derived concept, and the calculation of probabilities was direct and faster as it did not require the recursion that Huygens used. The connection between fair prices and probability is the basis for insurance pricing and came about during a period of rapid development of the insurance market, driven by the growth of maritime commerce in the 17th and 18th centuries. Soon after the publication of Bernoulli’s work, corporations began to engage in insurance. They were first chartered in England in 1720, and in 1735, the first insurance company in the American colonies was founded at Charleston, S.C. Therefore, by 1750, the idea of probability used in risk quantification, such as probability distributions, expected values, and fair price, and mortality were in use in insurance.

Later in the 18th century, a notable name is that of Thomas Bayes (1702–1761) and his famous essay on inverse probability [7,8]. Decision theory makes extensive use of Bayes’s Law as a way to express the uncertainty about factors that affect the consequence of a decision from expert opinion and data. Another important aspect of decision theory is the idea of utility to quantify preferences of consequences of adverse events, an important component in the process of managing risks in a coherent manner. The idea of utility arose through Daniel Bernoulli in 1738 and utilitarian philosophers such as Bentham (1748–1832). Around this time, the industrial revolution meant that manufacturing and transport carried far graver risks than before, and we do see the first attempts at risk management through regulation. In the United Kingdom, the Factory Act of 1802 (named the “Health and Morals of Apprentices Act”) started a sequence of such acts that attempted to improve health and safety at work.

The foundations of modern utility theory, from which a prescription for normative decision making comes about that is the basis of decision theory, had to wait until the early 20th century with the work by Ramsey [9]. The mathematical basis of today’s quantitative risk analysis is indeed normative decision theory. The impetus for a formal approach to utility came from von Neumann and Morgenstern [10] with their interest in rational choice, game theory, and the modelling of preferences.

Although Pearson [11] names the exponential distribution for the first time, it was only in the 1950s that the field of statistical reliability emerged and we began to see some of the methods that are in common use today. Here, we see Weibull’s (1887–1961) advocacy of the Weibull distribution [12,13], the statistical analysis of failure data by Davis [14], the proposal of Epstein and Sobel [15] that the exponential distribution should be used as a basic tool for reliability analysis, and the approach of Kaplan and Meier [16] for estimating the survival function under censoring. We begin to see the idea of system reliability emerging at this time as well. Drenick [17] looked at the failure characteristics of a complex system with the replacement of failed units; then, Birnbaum et al. [18] investigated the structural representation of systems of components. Fault trees also appeared in this decade; see the work by Watson [19]. Much of this work is summarised in the two books of Barlow and Proschan [20,21].

The last fifty years have been marked by many developments and the quantity of publications in the literature is rather overwhelming. From a historical perspective, perhaps the most important trend is the availability of computation to facilitate more complex risk and reliability analyses. However, the fundamental link between quantitative risk and reliability methods to probability and decision making has remained.

3. Basic Concepts of Decision Theory and Graphical Methods to Describe Decision Problems

3.1. Basic Concepts

The final goal of a risk analysis and a reliability study is, in general, a decision that reduces the social, economic, environmental, etc. negative consequences (losses) for an individual or a group, or increases their utility. A plethora of work has been published on decision theory and decision analysis, and we refer to the works of Wald [22], DeGroot [23], Berger [24], and French and Ríos Insua [25] for a thorough illustration. We consider decision-making under uncertainty and we present a Bayesian approach through definitions, properties, and a simple example related to risk. There are many critical aspects in the stages of a decision process: problem structuring, belief modelling, preference modelling, optimisation, and sensitivity analysis. Here, we are interested in illustrating some aspects, mostly related to sensitivity, arising when considering belief and preference modelling. In a Bayesian framework, beliefs over possible states of nature are modelled through a (prior) probability distribution, which in the presence of additional information, is updated via Bayes’s theorem, whereas preferences over consequences are modelled using utility functions (or, more commonly in the statistical community, loss functions). The consequences are the result of actions chosen within a feasible set combined with states of nature, and the goal is to find an optimal action, namely the one maximising the expected utility. The assessment of beliefs and preferences is a difficult task, especially when there are several decision makers and/or experts. Sensitivity analysis (often called Bayesian robustness in this framework) deals with the uncertainty in specifying prior distributions and utility/loss functions; see the book by Ríos Insua and Ruggeri [26] for a thorough survey.

More formally, we assume that the decision maker has to choose among a set $\mathbb{A}$ of feasible actions a. Prior beliefs on the state variable $\theta \in \Theta$ are assessed through a prior distribution with density $\pi$, and they are updated, via Bayes’s theorem, into a posterior distribution, with density $\pi (·|x)$, where x is the result of an experiment with likelihood $l_x\left(\theta \right)$ over a sample space X. A consequence $c\in \mathbb{C}$ is associated to each pair $(a,\theta )$, and preferences over the consequences $c(a,\theta )$ are modelled with a utility function $u\left(c\right(a,\theta \left)\right)$, which we simply denote by $u(a,\theta )$. We associate with each action a its posterior expected utility:

$${\displaystyle T(u,\pi ,a)=\frac{{\displaystyle {\int }_{\Theta }u(a,\theta )l_x\left(\theta \right)d\pi \left(\theta \right)}}{{\displaystyle {\int }_{\Theta }{l}_x\left(\theta \right)d\pi \left(\theta \right)}}={\int }_{\Theta }u(a,\theta )d\pi \left(\theta \right|x).}$$

According to the Maximum Expected Utility Principle, we look for the optimal action $a^{*}$, which maximises $T(u,\pi ,a)$.

However, the assessment of u and $\pi$, and the choice of the model $l_x$, are performed with limited knowledge and some degree of arbitrariness. Such uncertainty in the specification has an impact on the optimal action and its expected utility, the model output in our case. A thorough review of the literature on how to handle such issues is provided in the book by Ríos Insua and Ruggeri [26]. Here, we present a simple, no data example adapted from Ruggeri et al. [27], which should make clear the importance of investigating the consequences of assessing probabilities and utilities imprecisely.

A football team is interested in signing a new player on a one-year contract. Two players, a and b, are available, and the team expects to have a (utility) gain in signing one of them, which depends on the possibility of qualifying (${\theta }_1$) or not (${\theta }_2$) for the European Champions League the next year. The probability of qualification is $\pi \left({\theta }_1\right)=p_1$. The team managers believe that the monetary consequence of signing a and not being qualified is equivalent to the one obtained with signing b and being qualified. The other two possible consequences are, in general, different. We could represent the consequences in

Table 1. Monetary consequences of signing players.

	${\mathit{\theta }}_1$	${\mathit{\theta }}_2$
a	$c_1$	$c_2$
b	$c_2$	$c_3$

:

In the team, there is uncertainty, maybe due to different opinions among the managers, about both the probability of being qualified and the monetary consequences. As a result, ranges are obtained for all of them: $0.4\le p_1\le 0.6$, $0.5\le u\left(c_1\right)\le 1$, $0.25\le u\left(c_2\right)\le 0.75$, and $0\le u\left(c_3\right)\le 0.5$, where utilities are in million euros. We consider (

Table 2. Expected utilities of signing players.

${\mathit{p}}_1$	$\mathit{u}\left({\mathit{c}}_1\right)$	$\mathit{u}\left({\mathit{c}}_2\right)$	$\mathit{u}\left({\mathit{c}}_3\right)$	$\mathit{T}(\mathit{u},\mathit{\pi },\mathit{a})$	$\mathit{T}(\mathit{u},\mathit{\pi },\mathit{b})$
0.4	1	0.75	0.5	0.85	0.6
0.4	0.5	0.25	0	0.35	0.1
0.6	1	0.75	0.5	0.9	0.65
0.6	0.5	0.25	0	0.4	0.15

) the following four utility–probability pairs associated with the bounds on utilities and probabilities:

Since in all four cases, $T(u,\pi ,a)>T(u,\pi ,b)$, the team managers might decide to hire a but such a choice would not necessarily be optimal, e.g., considering $p_1=0.6,$ $u\left(c_1\right)=0.5,$ $u\left(c_2\right)=0.75,$ and $u\left(c_3\right)=0.5$, then it would be $T(u,\pi ,a)=0.6$ and $T(u,\pi ,b)=0.65$, raising doubts about the preference of a over b.

Martin et al. [28] addressed the issue of the choice of actions when the priors and the utilities (losses in their work) are in the classes $\Gamma$ and $\mathbb{U}$, respectively, as is done here. They proposed to consider non-dominated actions, i.e., the actions $a\in \mathbb{A}$ such that there exists no other action $b\in \mathbb{A}$ such that $T(u,\pi ,b)\ge T(u,\pi ,a)$, for all $u\in \mathbb{U}$, $\pi \in \Gamma$, with strict inequality for one pair $(u,\pi )$.

3.2. Graphical Representation of Decision Problems

3.2.1. Decision Trees

Decision problems can be represented graphically using decision trees, where decision nodes are typically represented by a rectangle; chance (or random) nodes are represented by a circle; and outcomes, which are functions of decision actions and uncertainties, lie at the terminal points of the branches of the tree. Figure 1 shows the decision tree representation of a single-period decision problem. In the tree, the decision node is denoted by $D_1$ and the chance node is denoted by $R_1$. The branches of the decision node $D_1$ represent the different decision actions a whereas the branches of $R_1$ represent possible values of the random quantity $\theta$, which is commonly referred to as the state of nature.

The decision tree is a chronological representation of the events in a decision problem. In Figure 1, first, a decision action a is taken at the decision node $D_1$ and then a possible value of the state of nature $\theta$ is observed at the chance node $R_1$ where uncertainty about $\theta$ is described by the probability distribution $\pi \left(\theta \right)$. Each combination of the $(a,\theta )$ pair implies an outcome or utility value $u(a,\theta )$ at the terminal point of a specific path of the tree.

The solution to the single-period decision problem in Figure 1 is obtained by folding back the tree (see Lindley [29]) starting at the terminal nodes. This is done by working backwards, taking the expectation of $u(a,\theta )$ at a random node such as $R_1$,

$${\displaystyle E\left[u(a,\theta )\right]={\int }_{\theta }u(a,\theta )d\pi \left(\theta \right),}$$

and by maximising $E\left[u\right(a,\theta \left)\right]$ with respect to action a at a decision node such as $D_1$. The optimal action $a^{*}$ obtained at $D_1$ maximises the expected utility, as discussed in Section 3.1.

Later, we consider an example of a single-period decision tree that arises in the development of replacement strategies for components/systems in reliability analysis.

3.2.2. Influence Diagrams

An influence diagram (ID) is a graphical representation of a decision problem that contains the same information encoded by an equivalent decision tree. It is possible to convert an ID into a decision tree and vice versa. An ID is a directed acyclic graph containing three types of nodes: chance nodes that are represented by circles, decision nodes that are represented by squares, and deterministic nodes that are represented by squares with rounded edges. If an ID contains only chance nodes, then it is called a belief or Bayes network. A deterministic node is so-called because its value is a function of those taken by its parent nodes. A special type of deterministic node is a value node, which contains the value taken by utility as a function of its parent nodes. The advantage of an ID over decision tree is that the former provides a more compact and high-level representation of the decision problem because its size does not increase exponentially as additional decisions or uncertainties are added to an analysis. In fact, it displays the dependence among variables and the state of information under which decisions are made but it does not show the possible values associated with each decision or chance variable. Therefore, it is often the case that a problem is first represented as an ID and, afterwards, is converted to a decision tree for computation of the solution, which means finding the expected utility of all possible decisions, to be maximised over the decision space. The single-period replacement problem in Section 3.3.1 provides a simple example of the more compact representation of an ID, as shown in Figure 2, where there are no multiple arcs from the nodes. The decision node contains the value of $\tau$, the chance node contains $N\left(\tau \right)$, the value node contains $C(\tau ,N(\tau \left)\right)$, the decision node influences the chance node, and the value node is influenced by both of them. The problem representation is completed by information on the conditional distributions of chance nodes given the parent nodes, that is, the conditional distribution of $N\left(\tau \right)$ given $\tau$ for the specific example.

The ID representation of the stopping problem tree in Section 3.3.2 is not a simplification because decision nodes in the multi-stage decision are binary, each stage is represented in the same way in a repetitive fashion, and the tree always has a terminal leaf after a stop branch.

In Figure 3, we show a simple example with a nonrepetitive structure adapted from Bedford and Cooke [30], which also shows that an ID enables the derivation of conditional independence relationships among nodes. The chance nodes in the ID are described as follows: fault can take values of yes or no, repair can take values of repairman is sent or repairman is not sent, and both primary alarm and secondary alarm can take values of alarm signals a fault or no signal. The decision node is self explanatory and the cost node can take two values if no secondary alarm is installed: one is the cost of sending a repairman after a false alarm; the second one adds the actual repair cost if a fault has really occurred. With the secondary alarm, the two previous cost values are increased by the installation cost. It is assumed that the alarms never fail to signal a fault, but false alarms can occur. Thus, unnecessary repairs can be reduced by installing a secondary alarm and by calling the repairman only if both alarms signal a fault. If the decision is not to install the secondary alarm, the value of its node is always alarm signals a fault. The repair node is in fact a deterministic node because its state is entirely known given the state of the alarms; however, it is regarded as a chance node with a degenerate distribution for later use. The problem description is completed by assigning probabilities to the following events: a fault occurs, the primary alarm signals a fault given that there is no fault, and the secondary alarm signals a fault given that there is no fault.

From the ID, we can find out conditional independence relationships using the global Markov property on any belief network determined by assigning a value to decision nodes. Suppose that the decision is to install a secondary alarm and to consider the repair node. Take the set of all its ancestors, which includes the two alarm nodes and the fault node (not the decision node because its value has been assigned) and remove all the remaining nodes (the cost node). Now moralise this subgraph by joining all unmarried parents using an undirected arc (the two alarm nodes in this case), and change all the directed arcs to undirected: the repair node is independent from the fault node given the pair of alarm nodes because all paths between them always go through the set of alarm nodes. With the same method, we can discover that the two alarm nodes are independent conditionally upon the fault node. The two moralised subgraphs are shown in Figure 4a,b.

The decision tree structure for this problem is more complex. It starts with the decision node with two outgoing branches, each terminating with the fault node, from which two more branches come out and terminate with an alarm node. From this node, either four or two branches come out, depending on which subtree we are in. If we decide to install the secondary alarm, there are four branches, one for every pair of states for the two alarms. The terminal nodes are the costs associated with every path in the tree.

To complete this illustration, let $\pi$ be the probability of a fault and let p be the probability of a false alarm using any of the two alarms, which are assumed to be devices of the same type. Let also $c_S$ be the cost of sending a repairman, $c_R$ be the cost of an actual repair, and $c_I$ be the cost of a secondary alarm installation. Then, the costs of the decision of installing and not installing a secondary alarm are given by

$$c_I+c_S{A}_1{A}_2+c_{R}F\mathrm{and}{c}_S{A}_1+c_{R}F$$

respectively, where $A_1$, $A_2$, and F belong to $\{0,1\}$ and are random variables representing the state of the two alarms and the presence of a fault. The expected costs are easily derived using the conditional independence properties derived earlier, and it is found that it is convenient to install a secondary alarm if

$$c_I<c_{S}p(1-p)(1-\pi ).$$

Banks et al. [31] remarked that the graphical structure of an ID has the property that there is a directed path containing all decision nodes that specifies the order in which decisions are made by the decision maker. The authors pointed out also a drawback of IDs, i.e., their impossibility of representing problems in which there is no predetermined order for the decisions, for example, in medical diagnosis, where the decision about subsequent tests depends on the results of previous ones. In those cases, asymmetric decision trees are better graphical tools, but they might have too many branches as a consequence of having to consider as many subtrees as all possible test orders. The authors suggested, as a possible alternative, to consider multiple IDs, one for each possible test order, and to compare the solutions.

A simple introduction to IDs can be found in Bedford and Cooke [30], whereas a more advanced introduction is given by Banks et al. [31].

3.3. Single and Multi-Period Decisions

3.3.1. Single-Period Replacement Problem

Systems and components experience ageing or wear as a function of time and/or usage. For such systems, planned replacement strategies are used to prevent in-service failures that may be very costly relative to the cost associated with a planned replacement/repair. Mazzuchi and Soyer [32] proposed a Bayesian decision theoretic approach to develop optimal replacement strategies using age and block replacement protocols. In what follows, we consider a single-period block replacement problem under the assumption of minimal repair.

Typically, under the block replacement protocol, a planned replacement is made at time epochs $\tau$, $2\tau$, …, irrespective of the age of the system and an in-service replacement is made whenever the system fails (the “good as new” scenario, since the replacement brings the reliability back to the initial one, assumed “good”). Another block replacement scenario was considered by Barlow and Hunter [33], where the system is minimally repaired upon failure but replaced at times $\tau$, $2\tau$, …. This is known as block replacement with minimal repair where the item can be repaired so that its failure characteristics are restored to the state just prior to the failure (the “bad as old” scenario).

We let $c_P$ denote the cost of a planned replacement; $c_R$ be the cost of minimal repair, such that $c_P<c_R$; and $N\left(t\right)$ be the number of failures in a time interval of length t. Then, in a planned replacement cycle of length $\tau$, the cost per unit time is given by

$$C(\tau ,N\left(\tau \right))=\frac{c_P+c_{R}N\left(\tau \right)}{\tau }.$$

The decision problem is to find $\tau$ such that the expected value of $C(\tau ,N(\tau \left)\right)$ is minimised. The decision tree for the problem is shown in Figure 5, where the random $R_1$ represents $N\left(\tau \right)$, an unknown number of repairs in a replacement cycle of length $\tau$.

Following Barlow and Hunter [33], we model $N\left(t\right)$ as a non-homogeneous Poisson process (NHPP) with intensity function $\lambda \left(t\right)$. For a system subject to ageing or wear, a commonly used model is the power law form for the intensity function, that is,

$$\lambda \left(t\right)=\alpha \beta t^{\beta -1}$$

where $\alpha >0,\beta >0$, and values of $\beta >1$ imply ageing over time. The mean value function $\Lambda \left(t\right)$ for the NHPP is the cumulative intensity:

$${\displaystyle E\left[N\left(t\right)\right]=\Lambda \left(t\right)={\int }_0^t\lambda \left(u\right)du=\alpha t^{\beta }.}$$

The common approach in the literature is to assume that $\alpha$ and $\beta$ are given (or estimated based on past data) and to select the optimal replacement interval, ${\tau }^{*}$, at the decision node $D_1$ of the tree by minimising

$$E\left[C(\tau ,N\left(\tau \right))\right]=\frac{c_P+c_R\left(\alpha {\tau }^{\beta }\right)}{\tau }$$

with respect to $\tau$. For $\beta >1$, it can be shown that the optimal interval is given by

$${\tau }^{*}={\left(\frac{c_P}{c_R\alpha (\beta -1)}\right)}^{1/\beta }.$$

Recent work in optimal replacement includes semiparametric policies considered by Merrick and Soyer [34], who considered block replacement for rail sections.

3.3.2. Multi-Period Stopping Problem

Decision trees are typically used in representing multi-period decision problems where a sequence of decisions are made and uncertainties are updated dynamically. Such problems arise in risk and reliability analysis in the context of the design of systems, life testing, optimal stopping, portfolio selection, etc. In this section, we present an example of an optimal stopping problem in software testing. The solution of sequential decision problems of this type relies on pre-posterior analysis and can become quite challenging to solve. These problems also arise in life testing; see, for example, van Dorp et al. [35], and Erkanli and Soyer [36].

Morali and Soyer [37] considered an optimal stopping problem in software testing and presented a Bayesian decision theoretic setup. In what follows, we use their notation and formulation of the multi-period problem. During the development phase, a new software goes through several stages of testing and, after each stage of testing, modifications are made to the software to fix the faults (or bugs). This process is known as debugging.

Let $X_i,i=1,2,\dots$ denote the life-length of the software during the ith stage of testing after the $(i-1)$th modification made to it. Morali and Soyer [37] assumed that the failure rate ${\theta }_i$ during the ith stage of testing is constant and, thus, that $X_i$ is exponentially distributed with rate ${\theta }_i$. The special feature of their model is that the failure rate ${\theta }_i$ changes from one testing stage to another as a result of corrections made to the software.

The authors assumed that, at the end of each stage, following modifications made to the software, a decision must be made whether to terminate the debugging process. Thus, after completion of i stages of testing, the decision of whether to stop testing is based on $X^{\left(i\right)}=(X^{\left(0\right)},x_1,x_2,\dots ,x_i)$, where $X^{\left(0\right)}$ represents the available information prior to any testing. Morali and Soyer [37] considered a loss function that reflects the tradeoff between additional testing versus releasing an unreliable piece of software. More specifically, they defined the loss associated with stopping and releasing the software after the ith stage of testing stage as

$$L_i(X^{\left(i\right)},{\theta }_{i+1})=\sum _{j=1}^i{L}_T\left(X_j\right)+L_S\left({\theta }_{i+1}\right),$$

where $L_{\mathrm{T}}$(·) represents the loss due to testing for one stage and $L_{\mathrm{S}}$(·) relates the loss associated with stopping and releasing the software. Note that, in this paper, we use both utilities and losses, choosing the notion more suitable for the topic at hand.

The stopping problem can be represented as a sequential decision problem as given by the m-stage decision tree in Figure 6 and can be solved using dynamic programming. The solution of the tree proceeds in the usual way by taking the expectation at random nodes and by minimising the expected loss at the decision nodes. At decision node i, the additional expected loss associated with the stop and the test decisions are given by the terms $E\left[L_S\left({\theta }_{i+1}\right)\right|X^{\left(i\right)}]$ and $E\left[L_T\left(X_{i+1}\right)\right|X^{\left(i\right)}]+L_{i+1}^{*}$, respectively, where

$$L_i^{*}=min\left\{E[L_S\left({\theta }_{i+1}\right)|X^{\left(i\right)}],E[L_T\left(X_{i+1}\right)|X^{\left(i\right)}]+L_{i+1}^{*}\right\}$$

for $i=0,1,\dots$, and the optimal decision at decision node i then is the one associated with $L_i^{*}$.

Multi-period decision problems also arise in the design of life tests such as accelerated life tests. Some of the recent work in this area include Zhang and Meeker [38,39], Meeker et al. [40], and Polson and Soyer [41].

4. Examples of Decision Problems

4.1. One-Stage Software Testing

Figure 1 showed the structure of the simplest decision problem, that of a single-period problem where a decision is made, a state of nature is revealed, and an outcome follows. There is no opportunity to learn from data about the state of nature $\theta$, and therefore, one relies entirely on the prior $\pi \left(\theta \right)$.

Such a type of decision problem can be illustrated through its application to software testing. Here, we mention the paper by McDaid and Wilson [42] and the more recent book by Kenett et al. [43]. When developing software, bugs are often introduced and they cause it to fail, producing a result different from the specification. Developers are therefore interested in testing the software to discover and remove bugs before its release. Of course, they should be very careful to prevent the introduction of new bugs. There is an issue of the quality of testing and about the length of the test phase. There are conflicting aspects about costs: on one side, there could be excessive costs due to a very long test but, on the other side, early release might imply a less reliable software. There are other aspects such as the possible obsolescence of the software caused by a very delayed release, the loss of reputation due to a poor software, and the need to market the software before the release of similar ones by competitors. Therefore, it is important to determine an optimal release time, taking into account especially costs for testing and fixing software, with the latter strongly dependent on the number of bugs left in the software at its release. Such an optimal time could be easily found through a one-stage test.

In Figure 7, we present the decision tree for one-stage testing. In terms of the notation of this paper, the set of feasible actions are times a that one could test, $\mathbb{A}=\left\{a\right|a\ge 0\}$, and the state of nature is the number of bugs discovered during and after testing, denoted $N\left(a\right)$ and $\overline{N}\left(a\right)$, respectively; hence, $\theta =(N\left(a\right),\overline{N}\left(a\right))$. The only deviation from Figure 1 is that $\theta$ is described by a probability model $\pi \left(\theta \right|\psi )$ with parameters $\psi$ that are themselves unknown, and the prior distribution $\pi \left(\psi \right)$ is directly specified on $\psi$. However, the Partition Law of probability gives the prior on $\theta$ directly by integrating out $\psi$:

$$\pi (\theta )=\int \pi (\theta |\psi )\pi (\psi )d\psi .$$

(1)

Many models have been proposed for $N\left(a\right)$; see, e.g., Singpurwalla and Wilson [44]). In this work a popular model of Goel and Okumoto [45] is used. The model assumes that $N\left(a\right)$ is a Poisson process with mean function $\Lambda \left(a\right)={\psi }_1(1-e^{-{\psi }_{2}a})$, for parameters ${\psi }_1$ and ${\psi }_2$ that represent the expected total number of bugs to be discovered eventually in the software (at $a=\infty$) and the rate of discovery, respectively. Thus, $N\left(a\right)$ is Poisson distributed with an expected value $\Lambda \left(a\right)$. The motivation from this model comes from the fact that $\Lambda \left(a\right)$ satisfies the differential equation:

$$\frac{d}{da}\Lambda \left(a\right)={\psi }_2({\psi }_1-\Lambda \left(a\right)),$$

so that the rate of bug discovery is proportional to ${\psi }_1-\Lambda \left(a\right)$, the expected number of bugs remaining to be discovered at time a.

Our knowledge about $N\left(a\right)$ is quantified by the values of ${\psi }_1$ and ${\psi }_2$. Expert opinion can be used to quantify this knowledge in the form of a probability distribution for ${\psi }_1$ and ${\psi }_2$. In this approach, gamma distributions are used as they have a relatively simple form and can be defined to have an arbitrary positive mean and variance. The gamma distribution is defined by two parameters, a scale $\alpha$ and a shape $\beta$, and its density function has the form $f\left(x\right)={\alpha }^{\beta }{x}^{\beta -1}{e}^{-\alpha x}/\Gamma \left(\beta \right)$, where $\Gamma (·)$ is the gamma function. The important thing to note here is that the mean and standard deviation are $\beta /\alpha$ and $\sqrt{\beta }/\alpha$, so that $\alpha$ and $\beta$ can be uniquely determined if one has an opinion on the mean and standard deviation. McDaid and Wilson [42] described an elicitation process for these parameters that uses these relationships. Here, we assume that such an elicitation process has led to specifying a scale ${\alpha }_1$ and a shape ${\beta }_1$ for ${\psi }_1$ and a scale ${\alpha }_2$ and a shape ${\beta }_2$ for ${\psi }_2$.

Given ${\psi }_1$ and ${\psi }_2$, $N\left(a\right)$ and $\overline{N}\left(a\right)$ are Poisson distributed. McDaid and Wilson [42] derived the unconditional prior $\pi \left(\theta \right)=\pi (N\left(a\right),\overline{N}\left(a\right))$ and showed that the expected values are as follows:

$$\begin{array}{ccc}\hfill E\left[N\right(a\left)\right]& =& \frac{{\beta }_1}{{\alpha }_1}\left[1-{\left(\frac{{\alpha }_2}{{\alpha }_2+a}\right)}^{{\beta }_2}\right]\hfill \end{array}$$

(2)

$$\begin{array}{ccc}\hfill E\left[\overline{N}\left(a\right)\right]& =& \frac{{\beta }_1}{{\alpha }_1}{\left(\frac{{\alpha }_2}{{\alpha }_2+a}\right)}^{{\beta }_2}.\hfill \end{array}$$

(3)

With regard to utility function $u(a,\theta )$, the simple form for the utility of testing until time a and then releasing is as follows:

$$u(a,\theta =(N\left(a\right),\overline{N}\left(a\right)))=A-CN\left(a\right)-D\overline{N}\left(a\right)-Fa,$$

(4)

where A is the profit from releasing the software without any testing, C is the cost of fixing a bug discovered in testing, D is the cost of fixing a bug post-release, and F is the cost per unit time of testing, that includes both the testing costs as well as lost sales and market opportunity. In practice, D should be considerably larger than C.

Now, all the components of the decision problem have been specified. Solving the simple tree in Figure 7 involves taking the expectation with respect to the unknown states of nature $N\left(a\right)$ and $\overline{N}\left(a\right)$ (making use of their expected values as in Equations (2) and (3)) and then maximising the resulting expected utility with respect to a to find the optimal testing time. Plugging Equations (2) and (3) into Equation (4) gives the expected utility of testing for a time a,

$$T(u,\pi ,a)=A-C\frac{{\beta }_1}{{\alpha }_1}\left[1-{\left(\frac{{\alpha }_2}{{\alpha }_2+a}\right)}^{{\beta }_2}\right]-D\frac{{\beta }_1}{{\alpha }_1}{\left(\frac{{\alpha }_2}{{\alpha }_2+a}\right)}^{{\beta }_2}-Fa,$$

and the value of a that maximises this function, and therefore, the optimal time to test, is as follows:

$$a^{*}={\alpha }_2\left[{\left(\frac{{\beta }_1{\beta }_2(D-C)}{{\alpha }_1{\alpha }_{2}F}\right)}^{1/({\beta }_2+1)}-1\right].$$

(5)

This is not valid if $D<C$, a case that we do not anticipate in practice and would imply that the optimal strategy is not to test and just repair all bugs post-release.

Figure 8 presents the expected utility and optimal testing time when the prior mean on ${\psi }_1$ is 100 (so that about 100 bugs are expected in the code), whereas the prior mean on ${\psi }_2$ is 0.01. For such a purpose, we take ${\alpha }_1=0.01, {\beta }_1=1, {\alpha }_2=100$ and ${\beta }_2=1$. As a consequence, the two standard deviations coincide with the prior means, i.e., 100 and 0.01, respectively. The utility parameters are chosen as the following values: $A=2000$, $C=1$, $D=20$, and $F=0.5$. The left plot shows $T(u,\pi ,a)$ as a function of a and identifies the optimal release time $a^{*}=516.4$ for an expected utility of 1333.6. The right plot shows how $a^{*}$ changes as a function of D, the cost of fixing a bug post-release; this shows how the testing time should increase as the relative cost of fixing bugs after testing rises.

In this solution, the software is released regardless of the results of the testing. As a consequence, there is no opportunity to learn about its reliability from those results. In McDaid and Wilson [42], moving beyond this simplest case is also considered involving more than one stage of testing and in which learning about $\theta$ takes place.

A recent review of decision models for software testing including adversarial issues can be found in Ruggeri and Soyer [46].

4.2. Preventive Maintenance of Water Pumps

Christen et al. [47] analysed the operation data of a Worthington water pump, operating 24 h a day at the PEMEX Salamanca refinery in Guanajuato, Mexico. Data were recorded about operation hours before either maintenance or failure, whichever came first. The authors considered a random sign censoring model to describe the maintenance and failure processes and, based on a maximum expected utility approach, they proposed a maintenance policy that improved upon the existing one. Here, we take a different approach, further elaborating on the ideas about sensitivity issues in decision analysis introduced in Section 3.1.

There are $n=34$ observations, and they are presented in

Table 3. Pump data: failure and maintenance times.

Type
Failure	14 30 48 7  3  2  7 6  11 7  8 24 21 5 1
	7  3  92 13 10 4 10 49 89 3 28 23 22
Maintenance	48 12 8 3 3 3

, split into 28 failure and 6 maintenance times.

We consider independent and identically distributed exponential failure times with parameter $\theta$, and we denote the observed times by $X_i$, $i=1,\dots ,34$. The likelihood function is given by

$${\displaystyle l_x\left(\theta \right)={\theta }^{28}{e}^{-\theta {\sum }_{i=1}^{34}{X}_i},}$$

(6)

based on the density $\theta e^{-\theta x}$ at a failure time and the survival probability $e^{-\theta x}$ at a maintenance time.

We assume that interventions are made at fixed times, even when failures occur, and we consider two possible actions: $a=$ {intervention after 10 h} and $b=$ {intervention after 20 h}. We consider the following loss functions:

$$\begin{array}{cc}\hfill L(a,\theta )& =C+2{\int }_0^{10}x(10-x)\theta e^{-\theta x}dx=C+\frac{20}{\theta }+\frac{20e^{-10\theta }}{\theta }-\frac{4}{{\theta }^2}+\frac{4e^{-10\theta }}{{\theta }^2},\hfill \\ \hfill L(b,\theta )& ={\int }_0^{20}x(20-x)\theta e^{-\theta x}dx=\frac{20}{\theta }+\frac{20e^{-20\theta }}{\theta }-\frac{2}{{\theta }^2}+\frac{2e^{-20\theta }}{{\theta }^2}.\hfill \end{array}$$

The integral component of the loss function is related to the cost incurred when the pump fails at time x and does not operate until an intervention occurs. We compare the two actions over a 20 h period so that we add the factor 2 in the first loss since failures can occur over two 10 h periods. The term C accounts for the cost of the extra intervention. A cost should multiply the integral part, but it is simpler to remove it and to consider the losses apart from a multiplicative constant.

If we consider a gamma distribution prior on $\theta$, then we get a gamma distribution posterior $\pi$, which we denote $\mathcal{G}(\alpha ,\beta )$. It is possible to prove that, under such a distribution, the posterior expected losses are   

$$T(L,\pi ,a)=C+\frac{20\beta }{\alpha -1}+\frac{20}{\alpha -1}\frac{{\beta }^{\alpha }}{{(\beta +10)}^{\alpha -1}}-\frac{4{\beta }^2}{(\alpha -1)(\alpha -2)}+\frac{4}{(\alpha -1)(\alpha -2)}\frac{{\beta }^{\alpha }}{{(\beta +10)}^{\alpha -2}},$$

$$T(L,\pi ,b)=\frac{20\beta }{\alpha -1}+\frac{20}{\alpha -1}\frac{{\beta }^{\alpha }}{{(\beta +20)}^{\alpha -1}}-\frac{2{\beta }^2}{(\alpha -1)(\alpha -2)}+\frac{2}{(\alpha -1)(\alpha -2)}\frac{{\beta }^{\alpha }}{{(\beta +20)}^{\alpha -2}}.$$

Therefore, it holds that

$$\begin{array}{c}\hfill {\displaystyle T(L,\pi ,a)-T(L,\pi ,b)=C+\frac{20{\beta }^{\alpha }}{\alpha -1}\times \left\{\frac{1}{{(\beta +10)}^{\alpha -1}}-\frac{1}{{(\beta +20)}^{\alpha -1}}\right\}-}\\ \hfill -\frac{2{\beta }^2}{(\alpha -1)(\alpha -2)}+\frac{2{\beta }^{\alpha }}{(\alpha -1)(\alpha -2)}\times \left\{\frac{2}{{(\beta +10)}^{\alpha -2}}-\frac{1}{{(\beta +20)}^{\alpha -2}}\right\}.\end{array}$$

We now suppose that there is uncertainty about both beliefs and preferences, namely on the prior distribution of $\theta$ and the value of C. In the former case, there could be different opinions on the expected failure time of the pump, given by $1/\theta$; in the latter case, there might be variability in the cost of the extra repair. The classes we entertain are very simplistic but useful for illustrative purpose. More sophisticated classes are presented in Ríos Insua and Ruggeri [26]. We consider the classes:

$$\mathcal{L}=\left\{L\right(·,\theta ):L(a,\theta \left)\mathrm{and}L\right(b,\theta )\mathrm{as}\mathrm{above}\mathrm{and}20\le C\le 30\},$$

$$\Gamma =\left\{\mathcal{G}\right(\alpha ,\beta ):1\le \alpha \le 2,1\le \beta \le 2\}.$$

Given the likelihood (6), then the posterior distribution belongs to the class

$${\Gamma }^{*}=\{\mathcal{G}(\alpha ,\beta ):29\le \alpha \le 30,625\le \beta \le 626\}.$$

If we take $\alpha =30$, $\beta =625$, then the choice of $C=20$ or $C=30$ leads to opposite conclusions. In the former case, ${\displaystyle T(L,\pi ,a)-T(L,\pi ,b)=-7.52}$ implies that the intervention after 10 h is the optimal action whereas the intervention after 20 h is the optimal decision in the latter case since we get ${\displaystyle T(L,\pi ,a)-T(L,\pi ,b)=2.48}$. This is a typical situation in which the decision maker needs extra effort to specify beliefs and preferences or to honestly report that there is no clear-cut decision.

Bayesian analysis of repairable systems is considered in Pievatolo and Ruggeri [48], and minimal repair models for train systems are discussed in Pievatolo and Ruggeri [49]. Maintenance strategies for machine tools are presented in Merrick et al. [50], and maintenance practices for railroads are discussed in Merrick et al. [51]. Recent developments in maintenance optimisation can be found in Damien et al. [52] and Belyi et al. [53]. Degradation-based maintenance policies are considered in Zhang et al. [54].

4.3. Portfolio Selection

Markowitz [55] considered the single-period portfolio selection problem where an investor has to allocate a sum of money among K securities. A Bayesian decision theoretic setup to the problem was introduced by Winkler and Barry in [56], and the multi-period problem was discussed. In this section, we present a setup considered by Soyer and Tanyeri [57], who follow the formulation of Winkler and Barry [56].

We let $W_t$ denote the wealth of the investor at the end of time period t and $W_0$ denote the initial wealth of the investor. We define $r_t^k$ to be the return from security k during time period t and assume that there are no transaction costs. If $W_t^k$ is the amount invested in security k at the beginning of time period $t+1$, then the wealth of the investor at the end of time period $(t+1)$ can be written as

$$W_{t+1}=\sum _{k=1}^K(1+r_{t+1}^k)W_t^k=W_t+\sum _{k=1}^K{r}_{t+1}^k{W}_t^k.$$

Following Winkler and Barry [56], we assume that the investor’s objective is to maximise $u\left(W_T\right)$, the utility of wealth at the end of a finite time period T (with dependence on $\theta$ omitted). In the multi-period problem with a finite horizon T, the investor maximises the utility $u\left(W_T\right)$ by sequentially choosing the decision variables $W_t^k,t=0,\dots ,T-1$ and $k=1,\dots ,K$ at different points in time based on the available information. That is, the optimal allocation is revised as the random quantities $r_t^k$, $t=1,\dots ,T$ are observed over time. The decision tree for the multi-period portfolio selection problem is shown in Figure 9, where the decision nodes at time periods $t=0,\dots ,T-1$ are denoted by $D_0,D_1,\dots ,D_{T-1}$ and the random (observation) nodes are denoted by $O_1,O_2,\dots ,O_T$.

Given the initial wealth $W_0$ at $D_0$, the investor determines ${\mathit{W}}_0^{\mathit{K}}=(W_0^1$,$\dots$,$W_0^K)$, the amounts invested in K securities. The random node $O_1$ denotes, for time period $t=1$, the observed returns from K securities, ${\mathit{r}}_1={(r_1^1,r_1^2,\dots ,r_1^K)}^{\prime }$. Given ${\mathit{r}}_1$, the investor determines ${\mathit{W}}_{\mathbf{1}}^{\mathit{K}}=(W_1^1$,$\dots$,$W_1^K{)}^{\prime }$ at decision node $D_1$, and this process is repeated at subsequent nodes. The solution of the problem involves dynamic programming formulation and backward induction by taking the expectation at random nodes and by maximising the expected utility at the decision nodes, as in Winkler and Barry [56].

As pointed out by Soyer and Tanyeri [57], at decision node $(T-1)$, given the observed returns from the first $(T-1)$ periods, the decision variables ${\mathit{W}}_{T-1}^{\mathit{K}}={(W_{T-1}^1,W_{T-1}^2,\dots ,W_{T-1}^K)}^{\prime }$ are chosen by maximising $E_{T-1}\left[u\left(W_T\right)\right]$, where $E_{T-1}$ denotes the expectation conditional on the returns from the first $T-1$ periods. We denote the expected utility corresponding to the optimal allocation ${\mathit{W}}_{T-1}^{\mathit{K}\mathbf{*}}=(W_{T-1}^{1*},W_{T-1}^{2*},\dots ,W_{T-1}^{K*})$ by $u_{T-1}^{*}\left(W_T\right)$ at time $(T-1)$. Then, at decision node $T-2$, the optimal allocation is obtained by maximising $E_{T-2}\left[u_{T-1}^{*}\left(W_T\right)\right]$, where $u_{T-1}^{*}\left(W_T\right)=E_{T-1}\left[u\left(W_{T-1}+{\sum }_{k=1}^K{r}_T^k{W}_{T-1}^{k*}\right)\right]$. Continuing in this manner at time 0, calculation of the optimal allocation for the first investment period involves implicit computation of expectations and maximisations at each time period. This can become quite cumbersome if the underlying parameters of the return distributions are unknown and a Bayesian approach is used for inference, as discussed in Winkler and Barry [56].

For the case of the single-period problem, that is, $T=1$, one can obtain the solution analytically in some simple cases or use Monte Carlo-based methods. For example, if we consider a quadratic utility function for $W_1$ as

$$u\left(W_1\right)=W_1-{\displaystyle \frac{B}{2}}{W}_1^2,$$

where $B>0$, we can obtain an analytical solution for the optimal allocation at $t=0$ and if we assume that, at time period 1, the K dimensional return vector ${\mathit{r}}_1$ is normally distributed with mean vector $\mathit{\mu }$ and covariance matrix $\mathbf{\Sigma }$, denoted as $\left({\mathit{r}}_1\right|\mathit{\mu },\mathbf{\Sigma })\sim \mathit{N}(\mathit{\mu },\mathbf{\Sigma })$ where $\mathit{\mu }$ and $\mathbf{\Sigma }$ are known, we can write

$$u\left(W_1\right)=u(W_0+\sum _{k=1}^K{r}_1^k{W}_0^k)=u(W_0+{\mathit{r}}_1^{\prime }{\mathit{W}}_{\mathbf{0}}^{\mathit{K}})$$

and, using the quadratic utility function, obtain

$$u\left(W_1\right)=(W_0+{\mathit{r}}_1^{\prime }{\mathit{W}}_{\mathbf{0}}^{\mathit{K}})-{\displaystyle \frac{B}{2}}{(W_0+{\mathit{r}}_1^{\prime }{\mathit{W}}_{\mathbf{0}}^{\mathit{K}})}^2.$$

Then, it can be easily shown that the optimal allocation for investment period 1 is given by

$${\mathit{W}}_0^{K*}={\displaystyle \frac{2(1-B{W}_0)}{B}}{\mathit{R}}^{-1}\mathit{\mu },$$

where $\mathit{R}=\mathit{\mu }{\mathit{\mu }}^{\mathbf{\prime }}+\mathbf{\Sigma }$.

If we consider a two-period problem with a quadratic utility function for $W_2$, that is, $u\left(W_2\right)=W_2-{\displaystyle \frac{B}{2}}{W}_2^2$, then the dynamic programming solution can be obtained analytically under the assumption of independence of the return vectors. More specifically, if $\left({\mathit{r}}_t\right|\mathit{\mu },\mathbf{\Sigma })\sim \mathit{N}(\mathit{\mu },\mathbf{\Sigma })$ with $\mathit{\mu }$ and $\mathbf{\Sigma }$ known and ${\mathit{r}}_t$s independent, then we can show that the optimal allocations for investment periods 1 and 2 are given by

$${\mathit{W}}_0^{K*}={\displaystyle \frac{2(1-B{W}_0)}{B}}{\mathit{R}}^{-1}\mathit{\mu }$$

and

$${\mathit{W}}_1^{K*}={\displaystyle \frac{2(1-B{W}_1)}{B}}{\mathit{R}}^{-1}\mathit{\mu }.$$

Furthermore, it can be shown by induction that, for the T period problem, the optimal allocations are given by

$${\mathit{W}}_t^{K*}={\displaystyle \frac{2(1-B{W}_t)}{B}}{\mathit{R}}^{-1}\mathit{\mu }.$$

for $t=0,1,\dots ,(T-1)$.

When the parameters of the return vectors are not known, the solution of the dynamic program becomes quite cumbersome even when ${\mathit{r}}_t$ is independent. Winkler and Barry [56] considered the case of unknown mean vector $\mathit{\mu }$ for the two-period problem with quadratic utility function and noted that the solution can be obtained using numerical methods. More recently, Soyer and Tanyeri [57] considered two-period portfolio selection problems with stochastic volatility and illustrated Bayesian solutions using Monte Carlo-based methods.

5. Adversarial Issues in Reliability Analysis

5.1. Basic Concepts of Adversarial Decision Problems

The decision problems presented in the previous sections involved a single decision maker and were solved using decision analysis methods. There are problems in risk and reliability analysis that may involve two or more actors with competing interests. These problems with adversarial components can be set up as games that can be solved using game theory methods; see, for example, Luce and Raiffa [58]. Examples of adversarial situations in reliability analysis can be found in areas such as acceptance sampling (see Lindley and Singpurwalla [59]), life testing (see Lindley and Singpurwalla [60]), reliability demonstration (see Rufo et al. [61]), and warranty analysis (see Singpurwalla and Wilson [62]). A recent review of adversarial issues in reliability and survival analysis can be found in Singpurwalla et al. [63].

Lindley and Singpurwalla [59] considered an adversarial situation with two actors: a manufacturer M and a consumer C. The manufacturer M tries to sell a batch of items to C, who may either accept ($\mathcal{A}$) or reject ($\mathcal{R}$) the batch provided by M based on his utility function $u_C$. The decision by C depends on the “evidence” provided by M to C based on a sample from an inspection or a life test that M may perform. The decision M faces is whether to offer a sample to C and, if so, the size of the sample based on his utility function $u_M$. It is assumed that both M and C are expected utility maximisers.

Lindley and Singpurwalla [59] presented their setup in the context of acceptance sampling where M tries to convince C about the quality of his product implied by the unknown quantity $\theta$. We can think of $\theta$ as the percent of defective items in the batch in the quality control setting or as the failure rate (or mean time to failure) in the life testing context considered by Lindley and Singpurwalla [60]. The outcome of the inspection/test is denoted by D.

Figure 10 shows the game tree associated with the problem. The decision node M in the game tree represents the manufacturer’s decision about n, the sample size, that is offered to the consumer. The decision $n=0$ implies that offering a sample to C is not beneficial to M, and in this case, the game is concluded. The random node D denotes the data, that is, the outcome of the inspection/test. In their development, the authors assume that both M and C agree about the probability model generating the data. Let $p\left(D\right|\theta )$ denote the common probability model for the data-generating process. The decision node C represents the consumer’s decision to accept ($\mathcal{A}$) or reject ($\mathcal{R}$) the batch. Note that this decision is based on the observed sample data D, which is used by C for revising uncertainty about $\theta$. C updates his prior ${\pi }_C\left(\theta \right)$ to posterior distribution using ${\pi }_C\left(\theta \right|D)\propto p\left(D\right|\theta ){\pi }_C\left(\theta \right)$. Lindley and Singpurwalla [59] specified the utility function of C as $u_C=u_C(\mathcal{A},\theta )$ or $u_C=u_C(\mathcal{R},\theta )$. The utility function of M is specified as $u_M=u_M(\mathcal{A},n,\theta )$ or $u_M=u_M(\mathcal{R},n,\theta )$, implying that M’s utility is a function of C’s decision as well as n and $\theta$.

Lindley and Singpurwalla [59] developed a solution for the game tree assuming that M knows C’s decision criterion and the prior ${\pi }_C\left(\theta \right)$. Thus, for a given $(n,D)$, M is able to infer if

$$\int u_C(\mathcal{A},\theta ){\pi }_C\left(\theta \right|D)d\theta >(<)\int u_C(\mathcal{R},\theta ){\pi }_C\left(\theta \right|D)d\theta$$

at decision node C. The authors denote the sets of Ds implying the acceptance and rejection regions for C using $\mathcal{A}\left(n\right)$ and $\mathcal{R}\left(n\right)$. Once the sets $\mathcal{A}\left(n\right)$ and $\mathcal{R}\left(n\right)$ are known, M chooses the optimal value of n at decision node M by maximising

$${\int }_{D\in \mathcal{A}\left(n\right)}\int u_M(\mathcal{A},n,\theta )p\left(D\right|\theta ){\pi }_M\left(\theta \right)d\theta dD+{\int }_{D\in \mathcal{R}\left(n\right)}\int u_M(\mathcal{R},n,\theta )p\left(D\right|\theta ){\pi }_M\left(\theta \right)d\theta dD,$$

where ${\pi }_M\left(\theta \right)$ denotes M’s prior for $\theta$. The authors developed the optimal strategy for M for Bernoulli, Poisson, and Gaussian sampling using numerical methods. An implementation of the proposed approach to the case of exponentially distributed life times is considered in Lindley and Singpurwalla [60]. More recently, Rufo et al. [61] extended the Lindley–Singpurwalla approach for life testing by introducing a Bayesian negotiation model.

The proposed framework by Lindley and Singpurwalla [59,60] for adversarial testing problems is based on the assumption that the manufacturer knows both the decision criterion for C as well as C’s prior for $\theta$. In many cases, it may not be possible to obtain such information. The Bayesian approach to games involves a decision maker’s assessment of probabilities of the opponent’s actions. In their discussion of the Bayesian game theoretic approach, Kadane and Larkey [64] stated that

“…a decision maker has a subjective probability opinion with respect to all of the unknown contingencies affecting his payoffs. In particular in a simultaneous-move two-person game, the player whom we are advising is assumed to have an opinion about the major contingency faced, namely what the opposing player is likely to do”.

Furthermore, the authors point out that “infinite regress” is not a problem for the Bayesian decision maker since

“… all aspects of his opinion except his opinion about his opponent’s behaviour are irrelevant, and can be ignored in the analysis by integrating them out of the joint opinion”.

These and other criticisms of classical game theory have motivated the search for alternative solutions for decision problems with multiple actors. One such alternative is the adversarial risk analysis (ARA) approach recently proposed by Ríos Insua et al. [65]. ARA builds on the Kadane–Larkey approach by developing a model of the opponent’s strategies. This is done by incorporating uncertainty via subjective probabilities of the decision maker. As noted by Banks et al. [31], p. 1, the ARA model provides a probability distribution over the possible actions of the decision maker’s opponent and, using this, the optimal action is chosen by maximising the expected utility of the decision maker. We present the ARA formulation of Lindley–Singpurwalla’s adversarial life-testing problem in the next section.

5.2. Life Testing with Adversarial Modeling

Following Lindley and Singpurwalla [59,60], we consider the game tree of Figure 10 and analyse the manufacturer M’s decision process. The manufacturer/consumer game of Figure 10 is a sequential game similar to the “defend-attack” models discussed in Rios and Ríos Insua [66]. As before, we have the priors ${\pi }_M\left(\theta \right)$ and ${\pi }_C\left(\theta \right)$ and the utility functions $u_M=u_M(\mathcal{C},n,\theta$) and $u_C=u_C(\mathcal{C},\theta )$ for M and C, respectively. $\mathcal{C}=(\mathcal{A},\mathcal{R})$ denotes the consumer’s decision actions at decision node C.

The first step in the ARA solution is converting the game tree in Figure 10 to the decision tree of the manufacturer M shown in Figure 11. This is achieved by converting the decision node C to a random node for the manufacturer. Given the manufacturer’s decision n at node M and the outcome observed at random node D, the manufacturer needs to assess the probability of a consumer’s actions. These are the manufacturer’s subjective probabilities and we denote this discrete probability distribution by ${\pi }_M\left(\mathcal{C}\right|n,D)$. The main issue is how to assess ${\pi }_M\left(\mathcal{C}\right|n,D)$. This can be done directly as suggested in Kadane and Larkey [64] or by using the ARA approach, which takes into account the manufacturer’s perception of the consumer’s decision problem.

The decision tree in Figure 12 represents the consumer’s decision problem as seen by the manufacturer. It is important to note that the consumer’s decision tree is obtained by converting the manufacturer’s decision node M to a random node. As before, the manufacturer analyses the consumer’s decision problem by assuming that he is an expected utility maximiser. The analysis of the tree is used to estimate ${\pi }_M\left(\mathcal{C}\right|n,D)$. To achieve this, the manufacturer needs to specify $F_M({\pi }_C,u_C)$, his probability distribution of the consumer’s prior ${\pi }_C={\pi }_C\left(\theta \right)$, and utilities $u_C$. Once $F_M({\pi }_C,u_C)$ is specified, we can estimate ${\pi }_M\left(\mathcal{C}\right|n,D)$ via Monte Carlo simulation. More specifically, we can simulate realisations of $({\pi }_C,u_C)$ from $F_M({\pi }_C,u_C)$, and for each realisation, we can solve the tree for given values of $(n,D)$ and obtain ${\pi }_M\left(\mathcal{C}\right|n,D)$. In the final step, we use the manufacturer’s decision tree in Figure 11, where the probabilities are given by ${\pi }_M\left(\mathcal{C}\right|n,D)$ at the random node C. The optimal strategy for the manufacturer is computed by maximising expected utility.

Implementation of the ARA approach may become complicated depending on the distributional assumptions and the form of the utility functions. In the Bernoulli sampling example of Lindley and Singpurwalla [59], the authors assumed a beta prior with parameters $({\alpha }_C,{\beta }_C)$ for the unknown proportion defective $\theta$. The utility function of the consumer is specified as $u_C(\mathcal{A},\theta )=a_1+a_2\theta$ and $u_C(\mathcal{R},\theta )=a_3$, where $a_1>a_3>(a_1+a_2),a_2<0$. In the ARA setup, we can assume that the form of the probability distribution and the utility function are known but the respective coefficients are unknown. In other words, the manufacturer may know that the consumer’s prior distribution is a beta density with unknown parameters $({\alpha }_C,{\beta }_C)$. Similarly, in the utility function $u_C$, the coefficients $a_1,a_2$, and $a_3$ are unknown. Thus, the manufacturer specifies his distribution $F_M({\pi }_C,U_C)=F_M({\alpha }_C,{\beta }_C,a_1,a_2,a_3)$, which is used in the solution of the consumer’s decision tree shown in Figure 12. The same approach can be used in specifying $F_M({\pi }_C,U_C)$ for Poisson and Gaussian sampling cases as well as the adversarial life-testing example discussed in Lindley and Singpurwalla [60]. The approach above is thoroug y illustrated in Gonzalez-Ortega et al. [67] and we refer the interested reader to it. Other implementation issues in ARA and its applications in risk analysis can be found in Banks et al. [31], Gonzalez-Ortega et al. [68], and Naveiro et al. [69].

5.3. Defend-Attack Problems in an Adversarial Setting

As pointed out in Section 5.2, the model employed for the specific life-testing problem belongs to the category of ARA defend-attack models. Ríos Insua et al. [70] provided a general description of the approach through the use of IDs. In this setting, there are two decision makers: the defender and the attacker. The attacker, after observing decision $d\in \mathcal{D}$ taken by the defender, selects an attack $a\in \mathcal{A}$. After both parties have made their choice, there is a random outcome $S\in \mathcal{S}$ of the attack, with the conditional distribution depending on the choices. The actions and the outcome will produce utilities $u_D(d,s)$ and $u_A(a,s)$ for the defender and the attacker, respectively. This problem can be represented through a Multi-Agent Influence Diagram (MAID), in which some nodes are owned by the defender, some are owned by the attacker, and some are shared. The MAID, displayed in Figure 13 with one chance node, can be viewed as the superposition of two IDs, in which chance nodes are usually shared and assigned different (conditional) probability distributions by the two decision makers.

The defender must choose an action to maximise her expected utility without knowing the attacker’s action. Therefore she expresses her uncertainty by placing a probability distribution on the set $\mathcal{A}$, $p_D\left(a\right|d)$. The attacker’s utility itself is no concern for the defender; therefore, from her point of view, the ID of the defend-attack model becomes as represented in Figure 14.

The defender is now able to solve her ID: she computes her expected utility conditional on the attacker’s action first and then marginalises it with respect to $p_D\left(a\right|d)$. Letting $p_D\left(s\right|d,a)$ denote the defender’s conditional distribution of the outcome, we obtain

$$\begin{array}{cc}\hfill {\psi }_D\left(d\right|a)& =\int u_D(d,s)p_D\left(s\right|d,a)ds\hfill \\ \hfill {\psi }_D\left(d\right)& =\int {\psi }_D\left(d\right|a)p_D\left(a\right|d)da\hfill \end{array}$$

and the best decision $d^{*}$ is the one that maximises ${\psi }_D\left(d\right)$.

In order to assess $p_D\left(a\right|d)$, the defender has to make a guess on the attacker’s utility and conditional distribution of the outcome, assuming he also is a utility maximiser, so that $p_D\left(a\right|d)$ is a probability distribution of the best action chosen by him. The defender’s guess can be expressed as a probability distribution on the pair $\{u_A(a,s),p_A\left(s\right|d,a)\}$, so that the random optimal action of the attacker is $A^{*}\left(d\right)$, maximising

$${\Psi }_A\left(a\right|d)=\int U_A(a,s)P_A\left(s\right|d,a)ds.$$

where the capitalised $U_A$ and $P_A$ emphasise that they are regarded as random quantities (and consequently also ${\Psi }_A$). The defender’s distribution over $(U_A,P_A)$ can be considerably simplified using a parametric form. The cumulative distribution function of the defender over $\mathcal{A}$ is now found as $p_D(A\le a|d)=Pr(A^{*}\left(d\right)\le a)$; it is often approximated via Monte Carlo simulations of $(U_A,P_A)$, as shown in Algorithm 1.

Algorithm 1 Approximation of $p_D\left(a\right|d)$.

for  $d\in \mathcal{D}$ do

for $k=1$ to K do

sample  $(U_A^k,P_A^k)$

for $a\in \mathcal{A}$ do

compute ${\Psi }_A^k(a,d)$

end for

find $A^{k,*}\left(d\right)={max}_a{\Psi }_A^k(a,d)$

end for

compute the empirical distribution function of ${\left\{A^{k,*}\left(d\right)\right\}}_{k=1}^K$

end for

A worked-out case study in this framework is presented in Ríos Insua et al. [70] concerning a facility operator who wants to defend from a group of organised fare evaders (fare colluders).

6. Conclusions

In the paper, we presented many aspects of the use of decision theory methods in risk and reliability. Although advanced notions are introduced in the paper, we tried to present ideas and applications in a way useful for the learned readers and not only for researchers specialised on the topics so that they will be able to frame their problem in a rigorous and coherent way and to find pointers to specific analytic and computational techniques for their solution. The range of application areas that we included reflect our research interests, but the general principles of our work remain valid in other areas. Our experience also affected the approach (i.e., Bayesian) that we followed in the examples. Reliability has been one of the first applied areas in which Bayesian methods have been valued as very important due to the possible use of experts’ opinions about very reliable systems. The first comprehensive Bayesian book in reliability was by Martz and Waller [71] and was quite mathematical, and applications were scarce because of limitations in statistical and computing power in the 1980s. The most recent book by Hamada et al. [72] provides a different perspective, mostly due to the development of powerful computational methods, such as Markov Chain Monte Carlo, and increased interest in stochastic processes; it addresses many issues in reliability, including some that would fit very well with the scope of the current paper. In particular, Chapter 9 presents methods for planning the optimal collection of reliability data using genetic algorithms as computational tools. Chapter 10 discusses assurance testing to ensure that a reliability-related quantity of interest meets the given requirements; different risk criteria are considered to determine the plans. Accelerated life tests, mentioned in Chapter 7, offer opportunities to use decision analysis methods in an optimal way.

Two relevant books addressing both risk and reliability from a Bayesian viewpoint are the already cited one by Singpurwalla and Wilson [44] and one further by Singpurwalla [73]. The first book is about software testing, which was considered in Section 4. The second book is a must-read since it provides a rigourous and extensive illustration of decision theory applied to reliability, survival analysis, econometrics, and finance.

As mentioned earlier, risk is a notion related not only to reliability and finance but also to many other applied fields. A few examples are Varis and Kuikka [74] about environmental and natural resources management, Veneziano et al. [75] on seismic design of building, and Palomo et al. [76] on project management under disruptive events. For a discussion of approaches different from ours, we recommend Bedford and Cooke [30], Ben-Haim [77], Borgonovo et al. [78], and de Almeida et al. [79]. These sources are also valuable for the references therein.