Sequential Interval Reliability for Discrete-Time Homogeneous Semi-Markov Repairable Systems

: In this paper, a new reliability measure, named sequential interval reliability, is introduced for homogeneous semi-Markov repairable systems in discrete time. This measure is the probability that the system is working in a given sequence of non-overlapping time intervals. Many reliability measures are particular cases of this new reliability measure that we propose; this is the case for the interval reliability, the reliability function and the availability function. A recurrent-type formula is established for the calculation in the transient case and an asymptotic result determines its limiting behaviour. The results are illustrated by means of a numerical example which illustrates the possible application of the measure to real systems. by the behaviour on singleton events (the probability mass function deﬁnes the distribution).


Introduction
This paper is concerned with reliability indicators for semi-Markov systems. As it is well known (see, e.g., [1][2][3][4][5][6]), semi-Markov processes represent an important modelling tool for practical problems in reliability, survival analysis, financial mathematics, and manpower planning, among other applied domains. The attractiveness of these processes comes from the fact that the sojourn time in a state can be arbitrarily distributed, as compared to Markov processes, where the sojourn time in a state is constrained to be geometrically or exponentially distributed.
Several researchers have investigated the reliability measures of semi-Markov processes. Examples of discrete-time semi-Markov processes with the associated reliability measures and statistical topics can be found in, e.g., [7][8][9][10], who proposed a semi-Markov chain usage model in discrete time and provided analytical formulas for the mean and variance of the single-use reliability of the system. The evaluation of reliability indicators for continuous-time semi-Markov processes and statistical inference can be found in [11][12][13][14][15]. The readers interested in solving numerically continuous-time semi-Markov processes by using discrete-time semi-Markov processes for solving continuous ones are referred to [16][17][18][19].
In the present work, we propose a new measure for analysing the performance of a system, called the sequential interval reliability (SIR). This generalises the notion of interval reliability, as it is introduced in [20] for discrete-time semi-Markov processes and further studied in [21,22]. In line with the work of [23], we are also interested in a general definition that takes into account the dependence on what is called the final backward. It is worth mentioning that interval reliability was first introduced and studied for continuoustime semi-Markov systems in [24,25]. In those contributions, the interval reliability was expressed in terms of a system of integral equation.
This measure computes the probability that a system is in a working state during a sequence of non-overlapping intervals. This type of measure is of importance in several is the discrete-time counting process of the number of jumps in [0, k] ⊂ N.
then Z = (Z k ) k is called a semi-Markov chain (SMC) and (J, T) = (J n , T n ) n is called a Markov renewal chain (MRC).
Throughout this paper, we assume that the MRC or SMC are homogeneous with respect to the time in the sense that Equation (2) is independent of n. Thus, we will work under the following assumption: Assumption 1. The SMC (or, equivalently, the MRC) is assumed to be homogeneous in time.
It is clear that, if (J, T) is a MRC, then J = (J n ) n∈N is a Markov chain with state space E, called the embedded Markov chain of the MRC (J, T) (or of the SMC Z).
Note that:

Remark 1.
We would like to draw the attention to a specific terminological matter that we encountered in the literature of discrete-time semi-Markov processes with finite or countable state space and that may lead to terminological confusion. Some authors use the term "semi-Markov kernel" of discrete-time SM processes for P(J n+1 = j, T n+1 − T n = k|J n = i) (see, e.g., [1,7,8,10,19,20,22]). Other authors use the term "semi-Markov kernel" of discrete-time SM processes for P(J n+1 = j, T n+1 − T n ≤ k|J n = i) (see, e.g., [21,23,[26][27][28]), while the quantity P(J n+1 = j, T n+1 − T n = k|J n = i) can have several names, for instance semi-Markov core matrix. In this article, we used this second terminology.
In the authors' opinion, this terminological confusion stems from the following reasons: 1.
On the one hand, when working in discrete time and calling P(J n+1 = j, T n+1 − T n ≤ k|J n = i), k ∈ N, the (discrete-time) semi-Markov kernel, this is achieved by using exactly the same term as in continuous time, where P(J n+1 = j, T n+1 − T n ≤ t|J n = i), t ∈ R represents the (continuous-time) semi-Markov kernel.

2.
On the other hand, when working in discrete time and calling P(J n+1 = j, T n+1 − T n = k|J n = i), k ∈ N, the (discrete-time) semi-Markov kernel, this is done by analogy with continuous time, since for a discrete-time finite/countable state-space, a Markov or sub-Markov kernel is determined by the behaviour on singleton events (the probability mass function defines the distribution).
In any case, all this discussion is only a matter of notational convenience.
Clearly, a semi-Markov chain is uniquely determined a.s. by an initial distribution (µ i ) i∈E and a semi-Markov core matrix (q ij (k)) i,j∈E,k∈N or, equivalently, by an initial distribution (µ i ) i∈E , a Markov transition matrix (p ij ) i,j∈E and conditional sojourn time distributions ( f ij (k)) i,j∈E,k∈N .
Our work will be carried out under the following assumptions: Transitions to the same state are not allowed, i.e., p ii ≡ 0 for all i ∈ E.
Clearly, Assumption 2 is equivalent to q ii (k) = 0 for all i ∈ E, k ∈ N, and Assumption 3 is equivalent to f ij (0) ≡ 0 for all i, j ∈ E; note that this implies that T is a strictly increasing sequence.
For the conditional sojourn time distribution and sojourn time distribution in a state, one can consider the associated cumulative distribution functions defined by For any distribution function F(·), we can consider the associated survival/reliability function defined by Consequently, we have: To investigate the reliability behaviour of a semi-Markov system, we split the space E into two subsets: U for the up-states and D for the down-states, with E = U ∪ D and E = U ∩ D = ∅. For simplicity, we consider U = {1, . . . , s 1 } and D = {s 1 + 1, . . . , s}.
Two important reliability measures of a system are the reliability (or survival) function at time k ∈ N, denoted by R(k), and the (instantaneous) availability function at time k ∈ N, denoted by A(k), defined, respectively, by If ever we condition on the initial state, we obtain the corresponding conditional reliability (or survival) function at time k ∈ N given that {Z 0 = i}, i ∈ U, denoted by R i (k), and the conditional (instantaneous) availability function at time k ∈ N given that Let us now define the interval reliability, introduced in [20] as the probability that the system is in up-states during a time interval. Definition 3 (Interval reliability, conditional interval reliability, cf. [20]). For k, p ∈ N and i ∈ E, the interval reliability IR(k, p) and conditional interval reliability IR i (k, p) given the event {Z 0 = i} are, respectively, defined by IR(k, p) := P(Z l ∈ U, l ∈ [k, k + p]); (3) For k, p ∈ N and i ∈ E, it is clear that we have the following properties of the interval reliability and conditional interval reliability (cf. Proposition 1 and Remark 1 of [20]):

Sequential Interval Reliability
Let us consider a repairable system. In this section, we introduce a new reliability measure that we will call sequential interval reliability. This will generalise the notion of interval reliability presented before, in the sense that we are looking at the probability that the system is in working mode during two or several non-overlapping intervals.
More precisely, let us consider t := (t i ) i=1,...,N and p := (p i ) i=1,...,N two time sequences such that: p i ≥ 0 for all i = 1, . . . , N; It is clear that, in this case, {[t i , t i + p i ]} i=1,...,N is a sequence of non-overlapping real intervals.
Definition 4 (Sequential interval reliability). Let (Z k ) k∈N be a discrete time semi-Markov system and let t := (t i ) i=1,...,N and p := (p i ) i=1,...,N , N ∈ N * , be two time sequences such that {[t i , t i + p i ]} i=1,...,N is a sequence of non-overlapping real intervals. We assume that Assumptions 1-3 hold true.

1.
We define the sequential interval reliability, SIR (N) (t, p), as the probability that the system is in the up-states U during the time intervals ..,N , meaning that:

2.
For v ∈ N and k ∈ E, we define the conditional sequential interval reliability, SIR k (v; t, p), as the conditional probability that the system is in the up-states U during the time intervals where B t := t − T N(t) is the backward time process associated to the semi-Markov process.
Note that we have the obvious relationship between the sequential interval reliability and the conditional sequential interval reliability: For notational convenience, we will set: Remark 2. Under the previous notation, we have: If p i = 0 for all i = 1, . . . , N, and v = 0, then SIR k (t) can be called the sequential availability function; 4.
If there exists a j ∈ {1, . . . , N} such that t i = 0 for i < j and t h = t j for h ≥ j, p i = 0 for all i = 1, . . . , N, and v = 0, then SIR , the availability function is computed in t j .

Transient Analysis
We will now investigate the recursive formula for computing the sequential interval reliability of a discrete-time semi-Markov system. Proposition 1. Let (Z k ) k∈N be a discrete time semi-Markov system, assuming Assumptions 1-3 hold true and let t := (t i ) i=1,...,N and p := (p i ) i=1,...,N , N ∈ N * , be two time sequences such that ..,N is a sequence of non-overlapping real intervals. Let v ∈ N be the value of the backward process at time t = 0 and k ∈ E be the initial state. Then, the conditional sequential interval reliability, SIR (N) k (v; t, p), satisfies the following equation: where 1 1:N is a vector of 1s of length N, and g where 1 {k∈U} is the indicator function of the event {k ∈ U} and R b ij (v; k) is the reliability with final backward defined by Proof. Before proceeding with the proof, let us introduce the notation: From the definition of the SIR, it is clear that Let us consider now the r.v. T 1 and observe that the events } are mutually exclusive. Consequently, we can write (14) as follows: We need to compute the four terms of the right-hand side of (15); let us denote them by RT 1 , RT 2 , RT 3 and RT 4 , respectively.
First, through a straightforward computation, we obtain Second, using the double expectation formula and conditioning with respect to (J 1 , T 1 ), we immediately obtained the second term given by Third, using the double expectation formula, conditioning with respect to (J 1 , T 1 ), summing over all the possible values of T 1 and splitting the computation according to the interval to which T 1 belongs, a quite long computation yields: Furthermore, fourth, using the double expectation formula, conditioning with respect to (J 1 , T 1 ) and summing over all the possible values of T 1 , we obtain: Substituting these four terms in (15), we obtain the recurrence formula given in (11) and (12).
If no initial backward is considered, taking v = 0 in Equation (11), we immediately obtain the following recursive formula for the sequential interval reliability of a discretetime semi-Markov system, given the initial state. Corollary 1. Let (Z k ) k∈N be a discrete time semi-Markov system, assuming Assumptions 1-3 hold true and let t : ..,N is a sequence of non-overlapping real intervals. Then, the sequential interval reliability, SIR (N) (t, p), satisfies the following equation: where we have set g The next result provides a formula for computing the reliability with the final backward R b ij (v; k) defined in Equation (13).
Lemma 1. For a discrete time semi-Markov system (Z k ) k∈N , under Assumptions 1-3, let us define the entrance probabilities e ij (n), i, j ∈ E, n ∈ N by e ij (n) = the probability that the system that entered state i at time 0 will enter state j at time n. Under the previous notations, the reliability with the final backward R b ij (v; k) defined in Equation (13) is given by where e q ij (k − v) represents the entrance probabilities for the semi-Markov system (cf. [27]) associated with the semi-Markov core matrix: with q UU (k) and q UD (k) being the partitions of the matrix q(k) according to U × U and U × D.
Proof. First, it can be easily seen that: Second, note that P(Z k−v = j, Z k−v−1 = j, Z s ∈ U, s = 0, 1, . . . , k − v | Z 0 = i), i, j ∈ E, represent the entrance probabilities for the semi-Markov system associated to the semi-Markov core matrix q(k). See also Proposition 5.1 of [1] for the use of the semi-Markov system associated to the semi-Markov core matrix q(k) in reliability computation.
Third, in order to compute the entrance probabilities, one can use the recurrence formulas (see [27]): where Looking at the recurrence relationship given in Proposition 1 for computing the conditional sequential interval reliability with initial backward SIR (N) k (v; t, p), and taking into account that, for N = 1, we obtain the interval reliability with the initial backward, and see that we need a formula for computing the interval reliability with the initial backward, denoted by IR k (v; t, p) and defined by The next result provides a formula for computing this quantity.

Lemma 2.
Under the previous notations, the interval reliability with initial backward IR k (v; t, p) is given by Proof. The proof is a quite straightforward adaptation of a more general result presented in [21].
The next result provides a series of inequalities between sequential interval reliability, sequential availability, conditional reliability and conditional availability. Proposition 2. Let (Z k ) k∈N be a discrete time semi-Markov system, assuming Assumptions 1-3 hold true, and let k ∈ E and v ∈ N. 1.
For any t 1:N and p 1: ..,N is a sequence of nonoverlapping real intervals, and for s ≤ N we have:

2.
For any t 1:N , a 1:N and b 1:

3.
For any t 1:N , p 1:N , ..,N are two sequences of non-overlapping real intervals such that t 1:N + p 1:N = x 1:N + w 1:N , and t 1:N ≥ x 1:N (element-wise), then we have: Proof. For any k ∈ E and v ∈ N, let us define the set Ω (k,v) by The first point is obtained noticing that, for s ≤ N, we have: Applying the probability on this chain of inequalities and taking into account the definitions of reliability, sequential reliability, availability and sequential availability, we obtain the inequalities given in (25).
In order to prove the second point, we first observe that a i ≤ b i , i = 1, . . . , N, implies that the two sequences of non-overlapping real intervals k (v; t 1:N , a 1:N ), so we obtain (26). To prove the last point, since t 1:N ≥ x 1:N and t 1:N + p 1:N = x 1:N + w 1:N , we have that [t i , t i + p i ] ⊆ [x i , x i + w i ], i = 1, . . . , N. Consequently, we have: k (v; t 1:N , p 1:N ), so we obtain (27).

Asymptotic Analysis
We can investigate the asymptotic analysis of sequential interval reliability where (ν(i)) i∈E represents the stationary distribution of the embedded Markov chain (J n ) n∈N .
Before giving the proof of this result, we first need some preliminary notions and results. First, let us recall some definitions related to the matrix convolution product. Let us denote by M E the set of real matrices on E × E and by M E (N) the set of matrix-valued functions defined on N, with values in M E . For A ∈ M E (N), we write A = (A(k); k ∈ N), where, for k ∈ N fixed, A(k) = (A ij (k); i, j ∈ E) ∈ M E . Let I ∈ M E be the identity matrix and 0 ∈ M E be the null matrix. Let us also define I := (I(k); k ∈ N) as the constant matrix-valued function whose value for any nonnegative integer k is the identity matrix, that is, I(k) := I for any k ∈ N. Similarly, we set 0 := (0(k); k ∈ N), with 0(k) := 0 for any k ∈ N.
Let A, B ∈ M E (N) be two matrix-valued functions. The matrix convolution product A * B is the matrix-valued function C ∈ M E (N) defined by (29) or, in matrix form: It can be easily checked whether the identity element for the matrix convolution product in discrete time exists, and whether it is unique and given by δI or, in matrix form: The power in the sense of convolution is straightforwardly defined using the previous definition of the matrix convolution product given in (29). For A ∈ M E (N), a matrixvalued function and n ∈ N, the n-fold convolution A (n) is the matrix-valued function recursively defined by Second, let us introduce two sets of functions that will be useful for our study. Thus, let us define: and, for l ∈ N N−1 , p ∈ N N : where, by writing f (t 1 ; l, p), we mean that the function f is a function of the variable t 1 , while l, p are some parameters. Let us consider a map between the two sets, Φ : A → B l,p defined by where The map Φ allows to represent a function f (t, p) ∈ A as an element of the set B l,p , that is to say as a parametric function of one variable, namely t 1 . One can easily check that Φ is bijective and linear.
The last point before giving the proof of Theorem 1 will be to introduce a new matrix convolution product, important for our framework, and to see the relationship with the classical matrix convolution product. a matrix-valued function and let b = (b 1 , . . . , b s ) be a vectorvalued function such that every component b r ∈ A, r ∈ E. The matrix convolution product * is defined by or, in matrix form: The next result will give a relationship between this new introduced matrix convolution product (cf. Definition 5) and the classical one defined in (29). Proof. From the additivity of the map Φ, we have: Proof of Theorem 1. First of all, it is important to notice that we have: provided that this limit exists. Consequently, since our interest now is in a limiting result, in order to investigate the asymptotic behaviour of SIR (N) k (v; t 1:N , p 1:N ) as t 1 goes to ∞ we can consider the initial backward v = 0. Thus, the expression of sequential interval reliability that we will take into account in the next computations will be SIR (N) (t 1:N , p 1:N ), that is recurrently obtained through Relation (20).
The main idea of this proof is to consider SIR (N) k as a function of the variable t 1 and also on other additional parameters; then, we will apply the Markov renewal theory (cf. [1]) to this function of t 1 . Using Proposition 3 and applying the function Φ defined in (32) to the left and right hand sides of Equation (20), we obtain:
It is clear that Equation (33) is an ordinary Markov renewal equation (MRE) in variable t 1 , with parameters (l 2:N , p). The solution of this MRE is well known (cf. [1]) and it is given by or element-wise, using Proposition 3: where the matrix-valued function ψ = (ψ(k); k ∈ N) is given by Since SIR (N) k (t, p), we obtain: Consequently: Let us now compute the second limit using the key Markov renewal Theorem (cf. [1]). First, we observe that: g k (t 1 ; l 2:N , p) = g k (t, p) = P (k,0) (Z l ∈ U, for all l ∈ [t i , t i + p i ], i = 1, . . . , N, T 1 > t 1 ) Using this result, we have: where T D is the lifetime of the system. Thus, we are under the hypotheses of the key Markov renewal theorem and we obtain: where µ jj is the mean recurrence time to state j for the semi-Markov chain.

A Numerical Example
In this section, we will present a numerical example considering a semi-Markov model that governs a repairable system. The setting is as follows: the state space is E = {1, 2, 3}, the operational states are the first two, U = {1, 2}, and the non-working state is the last one, D = {3}.
The transitions of the repairable semi-Markov model are given in the flowgraph of Figure 1. Now, let X ij be the conditional sojourn time of the SMC Z in state i given that the next state is j (j = i). The conditional sojourn times are given as follows: In the following figures, we investigate the semi-Markov repairable system in terms of the proposed reliability measures studied in Section 3. Figure 2 illustrates the conditional sequential interval reliability for two time intervals moving equally through the time with the same length (one time unit). That is the probability that the system will be operational in the time intervals (k, k + 1) and (k + 2, k + 3) for k ∈ {1, 2, 3, 4, 5, 6, 7, 8}. We have to note that, as the time k passes, then the system tends to converge to the asymptotic sequential interval reliability SIR (2) (0; t, p) = 0.6603, as given in Theorem 1. Furthermore, the sequential interval reliability SIR (2) (0; (k, k + 2), (1, 1)) is equal to the conditional one SIR 1 (0; (k, k + 2), (1, 1)) due to the fact that the only possible initial state is the first one. The point here is to study the probability of a system being operational during different time periods with the same working duration and a fixed time distance between them, equal to one time unit. Then, Figure 3 examines the probability that the system is working during two time periods, with same working duration (equal to one time unit) and increasing the time distance between them. We considered the first interval to be fixed equal to (1,2) and the other moving apart with step 1 each time and be (k + 2, k + 3) for k ∈ {1, 2, 3, 4, 5, 6, 7, 8}. It can be easily seen that the probability of the system will be still working in both time intervals and is a decreasing function of time k, which means that the two time intervals are far enough apart for the system to be operational. Note that the probability of the system starting from the non-working state 3 to be in the up-states is sufficiently small.
As in all the previous cases, the concept of the initial backward did not play a role in the simulations (Figures 2 and 3), and in Figure 4, the sequential interval reliability with the initial backward v = 10 is examined. The first time interval is considered as fixed, (1,2), and the other one is moving apart with step 1 each time and it is (k + 3, k + 4) for k ∈ {1, 2, 3, 4, 5, 6, 7, 8}.
From the point of view of real applications, the proposed reliability measures can be applied to a huge variety of physical phenomena which they characterised from time dependence. In the literature, a lot of research works are presented for modelling a variety of such phenomena via semi-Markov processes, from financial [23] to power demand [29]. D'Amico et al. ( [23,26]) proposed a semi-Markov model and associated reliability measures for constructing a credit risk model that solves problems arising from the non-Markovianity nature of the phenomenon. Further developments and recent contributions on this aspect are provided in [28,30,31].
The characteristics of semi-Markov chains which allow for no-memoryless sojourn time distributions, permit considering the duration problem in an effective way. Indeed, it is possible to define and compute different probabilities of changing state, default probabilities included, taking into account the permanence of time in a rating class. This aspect is crucial in credit rating studies because the duration dependence of transition probabilities naturally translates in many financial indicators, which change their values according to the time elapsed in the last rating class, see, e.g., [32]. them. We considered the first interval to be fixed equal to (1,2) and the other moving apart with step 1 each time and be (k + 2, k + 3) for k ∈ {1, 2, 3, 4, 5, 6, 7, 8}. It can be easily seen that the probability of the system will be still working in both time intervals is a decreasing function of time k, which means that the two time intervals are far enough for the system being operational. Note that the probability that the system starting from the non-working state 3 to be in the up states is sufficiently small. As in all the previous cases the concept of the initial backward did not play a role   In the case of financial modelling (see, e.g., [33]), the presented results could be applied in order to create advanced credit scoring models. A financial asset, similarly to government bond, usually takes a "grade" based on the reliability of the country to pay the debt (also known as creditworthiness). These grades strongly affect the interest rates of the country's debt and they are clearly separated. Let us consider the following set of states, from a simple point of view, as the ratings: If the country bond receives a rating in the set U = {A, B}, this means that it is thought to be creditworthy and can borrow money from the markets with reasonable interest rates. On the contrary, if the rating is within the set D = {C}, then the country cannot borrow money from the markets due to very high interest rates caused from its problems for repaying the debt. D'Amico et al. [23] proposed flexible reliability measures based on semi-Markov processes for constructing a credit risk model that solves the problems arising from the non-Markovianity nature of the phenomenon. Following that work, the measures proposed in the present paper can be applied in the same way as follows: • The conditional sequential interval reliability with final backward SIR (N) k (v; t, p) gives the probability that the bond remains creditworthy in a sequence of different time intervals {[t i , t i + p i ]} i=1,...,N given its current rating and assuming a secondary process B t . This measure allows us to estimate the credit risk of the asset for different time periods and at the same time by knowing the complete trajectory of the system due to the backward process B t . • The sequential interval reliability SIR (N) (v; t, p) which gives the probability that the bond remains creditworthy in a sequence of different time intervals {[t i , t i + p i ]} i=1,...,N without taking into account the current rating. To implement this measure, we have to know the initial probability of the system being in each state.
It became clear that these measures have applications in a variety of stochastic phenomena due to their flexibility and their ability to significantly extend our knowledge about the process evolution. They can solve problems and provide answers for systems that can shift from failure to operational states, from a probabilistic point of view. Finally, they can be considered as generalisations of the classical measures of reliability for semi-Markov processes.

Concluding Remarks
This paper presents a new reliability indicator, called sequential interval reliability (SIR), which is evaluated for a discrete-time homogeneous semi-Markov repairable system. This indicator includes as particular cases several functions that are frequently used in reliability studies, such as the reliability and availability functions, as well as the interval reliability function.
The paper contains new theoretical results on both the transient and the asymptotic cases. More precisely, a recurrent-type equation is established for the calculation of the SIR function in the transient case and a limit theorem establishes its asymptotic behaviour. These results generalise corresponding known results for standard reliability indicators. The possibility to apply our results to real systems is shown by implementing a numerical example where the theoretical results are illustrated from a practical point of view. The paper leaves unresolved several aspects among which an important role is played by the application of the proposed indicator in different applied problems involving the use of real data.