Homogeneous Continuous-Time, Finite-State Hidden Semi-Markov Modeling for Enhancing Empirical Classification System Diagnostics of Industrial Components

: This work presents a method to improve the diagnostic performance of empirical classification system (ECS), which is used to estimate the degradation state of components based on measured signals. The ECS is embedded in a homogenous continuous-time, finite-state semi-Markov model (HCTFSSMM), which adjusts diagnoses based on the past history of components. The combination gives rise to a homogeneous continuous-time finite-state hidden semi-Markov model (HCTFSHSMM). In an application involving the degradation of bearings in automotive machines, the proposed method is shown to be superior in classification performance compared to the single-stage ECS.


Introduction
Multistate degradation modeling has been receiving considerable attention for supporting dynamic maintenance paradigms based on condition monitoring (CM), such as condition-based maintenance and predictive maintenance (examples in References [1][2][3][4]). In fact, multistate models describe the degradation evolution more realistically than binary models, as the evolution of many degradation processes proceed in successive phases characterized by different physical degradation mechanisms. For example, the crack growth degradation mechanism of a mechanical component subject to cyclic loads is typically modeled as a four-state degradation process in which different degradation mechanisms take place [5,6].
A further reason for the growing interest in multistate degradation models is their proximity to the maintenance data acquired in the field from operating systems. For example, operators typically assign a qualitative tag to equipment health during periodic inspections. Multistate modeling has also been adopted to describe the evolution of degradation in membranes of pumps operated in nuclear power plants [7], in turbine nozzles for the oil and gas industry [8], in components of the electrical industry [9,10], liners of marine diesel engine cylinders [2,11], and piping of nuclear power plants [12].
Finally, a strong advantage of multistate degradation models is that they exploit sound, wellestablished mathematical techniques for their quantification, such as the Markov and semi-Markov models. In particular, when CM data are available, the hidden semi-Markov process (HSMP) theory offers a full-fledged framework for building a multistate degradation model while handling the two uncertainties involved: (a) the observation process, which relates the CM data to the actual, but hidden, degradation state of the component; and (b) the multistate degradation process of the component.
In the literature, many authors have tackled the problem of assessing the degradation state of aging multistate components by resorting to hidden Markov models (HMMs) or hidden semi-Markov models (HSMMs). References [3,[13][14][15] provide some examples. However, all these works disregarded the engineering issue of building a classifier from CM data, which is the aim of this work. Rather, they sought to estimate the performance of this classifier as inferred from available sequences of observation of degradation trajectories. In this way, they assumed that there is a one-to-one correspondence between component degradation and a single feature extracted from the data, which is a priori known. However, this situation may be not be typical in several practical cases where a direct measure of component degradation is not available, and a feature selection from monitored signals [16] is required to identify features that are relevant for building an effective classifier of the equipment degradation state. For example, when dealing with bearings mounted on the powertrain of machinery [17,18], the available condition monitored data are vibrational signals from which a large number of features can be extracted, and a combination of them is required to infer the observed degradation state [19].
The diagnostic issue of building a classifier based on features to be embedded in a HSMM is considered in References [20,21]. To do this, the authors used a mixture of log-concave, elliptical, or symmetric continuous probability density functions (pdfs) (e.g., mixture of Gaussians) to describe the uncertainty of having a given vector of features and the actual degradation state of the component. The limitations of this approach stem from the large number of parameters that need to be estimated. For example, if the semi-Markov degradation process has = 4 states and the optimal combination of features is made of = 10 elements, then = 10 mean values and ( + 1)/2 = 55 values of the covariance matrix needs to be estimated for each of the = 4 states, i.e., a total of 55 × 4 = 220 parameters. This is a computationally burdensome approach that requires a very large dataset to infer the parameter values. Moreover, as mentioned above, some restrictions on the form of the model pdfs hold.
The main contribution of this paper is to develop a novel hybrid scheme for diagnosing the degradation state of monitored components, which integrates a robust, largely applicable empirical classification system (ECS) with the homogeneous continuous-time finite-state hidden semi-Markov model (HCTFSHSMM) framework. This combination gives rise to a new diagnostic tool that improves the current estimation of ECS using the observed path of component degradation. The proposed approach is detailed in the next subsection.

Methodology Snapshot
We assume that the information available for the development of the hybrid diagnostic approach is constituted by a dataset containing K-tested similar components of the pairs signal measurementscorresponding degradation state, collected during the entire degradation trajectory of every tested component, from the beginning of its life to the failure. These pairs are used to build an ECS (see Section 2.1). However, the limitation of ECSs for fault diagnostics is that their classification outputs are static, i.e., the class assignment process is independent of the actual stochastic dynamics of the degradation process whereby the information content about the component degradation history is lost.
To overcome this limitation, we propose to combine ECS with a homogeneous continuous-time finite-state semi-Markov model (HCTFSSMM), thus obtaining a HCTFSHSMM. The underlying degradation process is modeled as HCTFSSMM, which is called "hidden" because the information about the component degradation state is not directly known. The ECS is framed as the observation process of the HCTFSHSMM. The classification performance of ECS on a set of pairs signal measurements-corresponding degradation state not previously employed to build the classifier forms the observation matrix, containing the probabilities that a component in a given state is assigned by the ECS to the degradation state , for any possible and ∈ , where is the finite set of degradation states.
Then, maximum likelihood estimation (MLE) is applied to infer the parameters of the underlying HCTFSHSMM from the sequences of classification labels provided by the ECS on a given set of training trajectories. This way, the proposed procedure uses the output of the built ECS as input data for the estimation of the remaining parameters of the HCTFSHSMM.
The obtained HCTFSHSMM-based diagnostic system estimates the degradation state of a test component as that with the largest probability of being occupied. Therefore, the HCTFSHSMM works like a corrector of the classifications provided by the ECS because it also takes into account previous observations and parameters of the stochastic degradation process. Therefore, the degradation state provided by the developed semi-Markov model is more accurate than that provided by the ECS.
This solution, which is the main contribution of the present work, has an additional computational advantage. The model frames the classification performance of ECS as the entries of the observation matrix avoid developing an algorithm for estimating the value of its entries, as proposed by Moghaddass and Zuo [3]. This reduces the number of parameters that need to be estimated within the HCTFSHSSM framework and hence the data and the computational effort required to set the HCTFSHSSM.
The proposed methodology is shown using the example of degradation of bearings mounted on electrical machines. This is based on our previous work [22].
The remainder of the paper is organized as follows: Section 2 introduces the hybrid diagnostic system for degradation state assessment; Section 3 illustrates techniques to estimate hidden model parameters in the specific context in which the observation matrix is known. Section 4 shows the method to assess on-line the current degradation state of the equipment. In Section 5, a case study and the application of the proposed method are presented. The limitations of the proposed methodology and its possible developments are discussed in Section 6. Finally, Section 7 outlines the conclusions of the work.

The Hybrid Diagnostic Approach
In this section, we describe the hybrid diagnostic approach. In Section 2.1, a general framework to develop an ECS within the feature-driven approach is presented. Section 2.2 discusses multistate semi-Markov processes for degradation modeling. Finally, in Sections 2.3 and 2.4, the HCTFSHSMMwhich combines the ECS with multistate semi-Markov processes for degradation state assessmentis presented.

Development of the ECS within the Feature-Driven Approach
We assume to have available a training set containing measurements performed on similar components in different degradation states. These data are used to develop an ECS for identification of the equipment degradation state, i.e., to establish the relationship between measured signals and degradation state. The classification algorithm is developed within a feature-driven framework [23] through the following general procedure [22]: (1) The extraction of features from the raw measurements [24]. To do this, we rely on the strategy discussed by Cannarile et al. [25]. (2) The selection of an optimal subset of relevant features to be used for the classification [26] through the scheme proposed in our earlier work [22], i.e., the feature selector behaves as a wrapper around the specific learning algorithm used to construct the classifier [16]. The objective functions used for evaluating and comparing the feature subsets during the search are the recognition rate achieved by the ECS (to be maximized) and the number of features forming the subsets (to be minimized). This way, the feature selection problem is formulated as a multiobjective optimization problem [27]. As the large number of extracted features makes it infeasible for an exhaustive search, we use a binary differential evolution (BDE) [28], which has been shown to explore the decision space more efficiently compared to other multiobjective evolutionary algorithms [29] such as non-dominated sorting genetic algorithm II (NSGA-II) [30], strength Pareto evolutionary algorithm (SPEA2) [31], and indicator-based evolutionary algorithm (IBEA) [32]. (3) The development of the empirical classifier. A k-nearest neighbors (KNN) is used as the classification algorithm [33,34]. This choice is justified by the following advantages: (1) KNN requires setting few parameters and (2) it does not require the classes to be linearly separable in the input space.
The proposed feature-driven diagnostic system is shown in Figure 1a.

Degradation Modeling Based on Homogeneous Continuous-Time Finite State Semi-Markov Processes
We assume that the degradation states are regeneration states, i.e., when the ith state is reached, the previous history of the process can be neglected as it has no influence on the future degradation evolution. This assumption is made by several models of degradation phenomena, such as the Paris-Erdogan model [35]. This class of processes is called homogenous processes and is usually further divided into two main categories: • Markov models, in which the sojourn times in the states are exponentially distributed and the transition rates are constant.

•
Semi-Markov models, where the transition rates depend on the time spent in the current state.
In semi-Markov models, the transition occurrences depend on the sojourn times, which can follow arbitrary distributions. In this work, we assume that they are Weibull distributions, as these are the probability distributions most commonly used to describe the degradation processes of industrial components [3,36].
As anticipated above, we assume that the real degradation state of the equipment is not directly observable on-line and that the available information comes from the ECS described in Section 2.1, which classifies the equipment degradation state. Furthermore, the classification performance of the ECS is assumed to be known. Hence, the resulting degradation model is framed as a hidden semi-Markov process [4,[37][38][39][40] with known misclassification probabilities.
Finally, the process is assumed to be a left-to-right process, that is, at time = 0 the equipment is always in an 'as good as new' (AGAN) condition (i.e., not degraded). Once it starts degrading, it is modeled by jumps from less degraded states to more degraded states, with the last state being equipment failure. More details on semi-Markov model are provided in Appendix A.

Hidden Semi-Markov Model for Degradation Assessment
The idea behind the use of a hidden model lies in the fact that we are interested in the evolution of the process { , ≥ 0}, which is hidden, i.e., it cannot be directly observed. Formally, where Ω is the sample space and is the state space. Indeed, we can observe the evolution of another process-the observed process, { , ≥ 0} : which is related to in that the state in which the process is observed depends on the state of the hidden process [40].
This work builds on the work of Moghaddass and Zuo [3] to propose a hybrid method for the identification of the degradation state (Figure 1b), namely, HCTFSHSMM embeds the ECS diagnostic system, which is framed as the observation process that links the actual degradation state with its corresponding on-line estimation based on sensor signals. This way, the proposed procedure uses the output of an ECS as input data for the HCTFSHSMM. Notice that without loss of generality, we assume that the number of observed states is equal to that of the actual state (i.e., ≡ ).

HCTFSHSMM for Degradation Modeling
The objective of this subsection is to use the HCTFSHSMM modeling approach to improve the classification performance of an ECS. To do this, we address the following issues: 1. Estimation of the model parameters, i.e., the parameters of the transition rate functions. This issue arises during the development of the hybrid diagnostic model. 2. Estimation of the most likely degradation state at the current time , given the sequence of observations. This issue arises when the developed diagnostic model is used for assessing the degradation state of a test component of interest.
The following assumptions hold: The data available to estimate the model parameters are sequences of observations { 1 , 2 , … , }, which represent the outcomes of the KNN classifier taken at times { 1 , 2 , … , }, where the superscript refers to the sequence of observations, = 1, … , . III.
The acquisition time period Δ = +1 -is constant, = 1, … , and = 1, … , . For simplicity, but with no loss of generality, we suppose Δ = 1; then, the equations presented in the next sections can be easily extended to the more general case. V.
Just one transition can occur in the time interval Δ. VI.
The last observation of each time series coincides with the failure of the component, which is directly observed. VII.
No maintenance and repair operations are considered; transitions only go left-to-right across the states. VIII.
The ECS presented in Section 2.1 has been developed to assess the degradation state of the equipment. The misclassification probabilities , , , ∈ {1, … , } of the KNN classification algorithm are estimated by testing the performance of the diagnostic system on degradation patterns for which the actual degradation state is known. The values of the , probabilities are then considered as the entries of the observation matrix .

Maximum Likelihood Estimation of the Transition Function Rates
Suppose we have independent sample paths of observations = { 1 , . . . , }. Then, the likelihood of gathering them is given by: where ∈ are the unknown model parameters, and ℙ( | ) is the probability of observing the sequence , given the value of . The MLE is given by: Maximizing likelihood (Equation (4)) is equivalent to maximizing the following log-likelihood function: Since Equation (6) cannot be written in closed form, the procedure proposed by Rabiner [14] and adapted by Moghaddass and Zuo [3] is typically adopted to treat the nonhomogeneous semi-Markov processes. The probabilities ℙ( | ) are written in terms of the forward probabilities: where 1: = { 1 , … , }. From Equation (7) and More details on forward probabilities computation are provided in References [3,40,41]. To maximize Equation (6), we use the differential evolution (DE) algorithm [42]. According to Moghaddass and Zuo [3], the transition times, which are continuous variables, are discretized into time channels corresponding to the observation intervals [3]. We assume that these time intervals are short enough so that assumption V in Section 2.4 is not violated. The most significant consequence of the discretization concerns the estimation of the parameters. To see this, assume that we can perfectly observe the degradation process , = 1, … , , = 1, … , . If the transition from state to state + 1 occurs just after the ( → +1 )-th inspection time, the event will be recorded only at time instant where ( , 1: ) = ℙ( = , 1 , … , ) is the joint probability of being in state at time and having observed the sequence { 1 , … , } (for more details see References [3,40]). It is important to observe that maximizing Finally, notice that when applying the proposed approach in an on-line setting, gathering a new observation +1 allows us not only to estimate the current state ̂+ 1, +1 , but also re-estimate the previous states ̂, +1 . This gives the possibility to correct the previous estimations up to the current time + 1. The estimation of the current, ̂+ 1, +1 , and the previous, ̂, +1 , degradation states are fundamental to the definition of appropriate condition-based maintenance (CBM) interventions.

Case Study
In this section, the diagnostic method discussed above is applied to the condition monitoring of bearings. In recent years, bearing monitoring has received considerable attention due to the criticality of bearings faults [22,[45][46][47][48][49][50]. In particular, some works proposed in the literature have already modeled the degradation of bearings as a hidden process. For example, Dong and He [21] used a five-state HSMM with Gaussian sojourn time to describe bearing degradation. Tobon-Mejia et al. developed [51] a three-state mixture of Gaussians hidden Markov models (MoG-HSMMs) with Gaussian sojourn times. Soualhi et al. developed [52] a four-state Markov model where sojourn-times are exponentially distributed. Notice, however, that the assumptions made by the authors of References [21,51]-that sojourn time is normally distributed and that transition time is independent of the time spent in the actual state-are not completely realistic for modeling this degradation process [53].
In this work, we applied our hybrid diagnostic approach to the Case Western Reserve University (CWRU) bearing dataset [54], which contains the results of 72 experiments consisting the measurement of three acceleration signals on healthy and degraded bearings. The acceleration signals were measured using three accelerometers placed at the 12 o'clock position at the drive end (DrE), at the fan end (FE) of the motor housing, and on the motor supporting base plate. Data were collected at frequencies of 12,000 samples per second for time lengths of about 10 s. Two ball bearings were installed at the DrE and at the FE of the motor. For both bearings, three degradation modes were considered, affecting the inner race, the outer race, and the ball. For each failure mode, 12 experiments were performed, considering all the possible combinations of three different degradation states (i.e., 7, 14, 21, mils (mil inches) long defects) and four different operation conditions (motor loads from 0 to 3 horsepower). Bearings in normal conditions were also tested at the same four loads considered for the degraded bearings.
Without loss of generality, our work focused on the failure mode affecting the balls of the bearing installed at the DrE of the motor and we modeled its degradation process as a four-state HCTFSSMM with Weibull distributed sojourn times [55]. Thus, we set the number of degradation states to four and we referred to them as = {1,2,3,4}, as shown in Figure 2. We considered the KNN-based ECS that we developed in our earlier work [22] for the classification of the bearing degradation state applied to the features set selected using the wrapper approach discussed in Section 2.1. Table 1 reports the selected feature set. The performance of the obtained ECS was verified by computing the confusion matrix [56], which indicated the fractions of patterns correctly classified or assigned to the other classes for each true class of the validation patterns. For the assessment of the degradation state of DrE bearing affected by balls failure mode, Table 2 reports the classification performance on the real CWRU bearing dataset as developed in our earlier work [22]. The overall misclassification rate of the developed KNN classifier, assuming a balanced validation set (i.e., same number of patterns for each degradation state), is given by , of the confusion matrix indicates the probability that a validation pattern of true class is assigned to class . For example, entry (1,1) indicates that the probability of observing = 1 when = 1 is 0.887, whereas this probability decreases to 0.082 if the actual state is 2. As mentioned above, this confusion matrix will be used as the observation matrix within the HCTFSHSMM framework.

Degradation Trajectory Simulation
The CWRU bearing dataset did not contain the vibration signals measured during complete degradation trajectories-from normal condition to failure; it only included the vibration behavior for different bearings at different states of degradation. Thus, in order to include time dependence into the available dataset, we artificially generated Without loss of generality, we arbitrarily set the value of parameters , +1 and . +1 , which are reported in Table 3.    Table 2, which refers to the performance of the KNN on the CWRU bearing dataset. For example, if we consider a bearing whose true degradation state is 1, the correct classification is sampled with a probability of 0.887, in states 2, 3, and 4 are 0.082, 0.031, and 0.000, respectively. Figure 4 shows the classifications of the degradation state obtained during the degradation trajectory of Figure 3. This is the input to the proposed hybrid ECS + HCTFSHSMM for on-line diagnostics of the bearing degradation state.

Results
The results of the application of the procedure introduced in Sections 3, 4, and 5 to the bearings case study are presented and discussed in this subsection.

Parameter Optimization
To estimate the = 6 parameters of the model, = ( 1,2 , 2,3 , 3,4 , 1,2 , 2,3 , 3,4 ), the MLEbased approach was applied. The objective function to be maximized is the log-likelihood in Equation (6), where the observation matrix is given in Table 2. Figure 5 shows the value of the log-likelihood of the best candidate solution in the current population at each iteration (also referred to as generation). After 36 iterations, there was no substantial improvement in the fitness function. This suggested that the algorithm had converged. The values of the best candidate solutions ̂ are reported in Table 4. With respect to the parameters values estimation, Table 5 reports the relative percentage errors. We found that the smaller the expected sojourn time in a state (see Table 3), the larger was the error in the parameter estimate. This can be attributed to the fact that the relevance of the censoring mechanism introduced by time discretization increases (see Section 3). Notice that increasing the number of observation sequences used to estimate MLEs of the parameters of the transition time distributions decreases the estimation error, according to the consistency property of MLE [43].

Assessment of the Degradation State
The MAP estimation introduced in Section 4 was applied to 200 other observation trajectories sampled by applying the same procedure described in Section 5.1. Figure 6 shows an example of observed trajectory (squares) and the corresponding estimated degradation state sequence (triangles) provided by the proposed methodology when it was applied at the end of the degradation trajectory (i.e., at = 119 months). It clearly emerges that this allows: • Correction of the misclassifications of the KNN classifier. In particular, the decoded sequences are always increasing in monotone due to the fact that the underlying semi-Markov process allows only left-to-right transitions. • Estimation, in an unambiguous way, of the time instant that the bearing entered for the first time in a given state. With reference to Figure 6, we can state that the bearing entered for the first time in states 2 and 3 at time instants 65 and 102 (months), respectively. Notice that this information cannot be retrieved by the KNN-based ECS only as it is pointwise static. Table 6 reports the average misclassification rate value, which is defined as where , is the true state of the bearing at time for the -th sequence of observations, ̂, the corresponding state estimated by the hybrid method at time , using the observation sequence 1: . For comparison, the misclassification rate in Equation (12) was also computed considering the KNN classification algorithm, which is indicated by , instead of ̂, in (12) ( Table 6).   Table 6 shows that the use of the hybrid method to estimate the degradation state significantly decreased the misclassification rate of the KNN classifier. Notice that errors in the estimation of the degradation state mainly occurred in correspondence to the jumping times from one state to the next degraded state. For example, consider Figure 7, which refers to the estimation of the sequence of states ̂, . Here, at time = 44, i.e., in the proximity of the real jumping time from state 1 towards state 2, 43 < ≤ 44, the hybrid method misclassifies the state as 1. As time passes and new observations are collected, the hybrid method makes new estimates of the previous component state, i.e., at time > it is possible to obtain a new classification ̂, that considers the observation sequence 1: , and it is expected to be more accurate than ̂, , which is based on more information. For example, Figure 8 shows the estimate of the degradation state at time = 44 performed using the observation collected until time = = 44 (top), = 45 (middle), and = 46 (bottom). Notice that at time = 45, the hybrid method is able to correctly classify the component state. This is a useful piece of information because the next transition depends on the sojourn time in this state. By contrast, the KNN does not correct the degradation state estimate based on the current knowledge of the degradation process. . Notice that at time = 45, the hybrid method is able to correctly classify the component state.

Discussion and Outlooks
In this work, we considered a multistate degradation model that allows only left-to-right transitions to the neighbor state. However, different transition types can be considered in practice, such as direct transitions to the failure state (e.g., with respect to Figure 2, transitions from state 2 to state 4) or right-to-left transition due to preventive maintenance actions. The generalization of the proposed approach requires modeling more complex likelihood functions, although it does not impact the underlying structure of the approach.
A main limitation of our approach is that it assumes the availability of signal data for which the corresponding degradation state is known. Indeed, this information may not be available in many industrial applications. Future works will investigate the solution proposed by Cannarile et al. [19] to build an unsupervised clustering approach to identify the degradation state of the component. After this step, an ECS can be developed and combined with an HSMM as discussed in this work.
Finally, the developed approach estimates the parameters of the HSMM through an analytical solution to the problem of correcting the ECS output, which makes our hybrid method preferable to any other heuristic approach. However, this analytic approach is not applicable when the amount of data available is poor and the system is complex. In these cases, heuristics may be necessary for the approach to be applicable to real industrial settings. For example, the degradation state predicted at the previous time-step can be added as an additional feature to the diagnostic classifier. The integration of the proposed approach with heuristics will be addressed in future research work.

Conclusions
In this work, we have presented a novel diagnostic method for assessing the degradation state of industrial components in situations where the degradation process is not directly observable, but there is a classifier that provides an estimation of the actual state based on sensor signals. To improve the diagnostic performance, the classifier is integrated with a HCTFSSMM capturing the dynamic evolution of the degradation process. The resulting HCTFSHSMM parameters (transition function rates) are estimated using a method based on advanced MLE combined with the DE optimization technique.
The HCTFSHSMM adjusts the classifications provided by the empirical classifier ECS, by exploiting the knowledge of the sequence of observations collected from the monitored component. Thus, the final classification performance outperforms that of a single stage empirical classifier. This is shown by the considered case study of bearing degradation described by Weibull-distributed state sojourn times. Furthermore, the proposed approach can be used to fully characterize the underlying semi-Markov process as we can estimate both the current state and the time instant the system entered it, which is not possible using only the ECS.

Appendix A. Homogenous Semi-Markov Model
The Markov renewal theory [57] is embraced to analyze the described degradation model. In particular, consider the pair of random variables ( , ) defined on the probability space (Ω, ℱ, ℙ) [57]: where Ω is the sample space, ℱ a sigma-algebra on Ω , and ℙ a probability measure on the measurable space (Ω, ℱ). represents the state reached by the process at the ℎ transition, ∈ ℕ; = {1, … , } is the state space of , is the time corresponding to the ℎ transition. Then, the bivariate stochastic process {( , ), ≥ 0} is a homogeneous renewal Markov process (RMP) [57] whose kernel is given as follows: , ( ) = ℙ( +1 = , +1 − ≤ | = ) , ∈ , ≠ ; ∈ ℝ + (A2) From this, we can derive the probability of leaving state to reach state and the probability of remaining in state , respectively [57] , = lim In the case where there is just one transition out from a state, then ( ) = ,• ( ). Notice that the main difference between continuous-time Markov processes (CTMPs) and continuous-time semi-Markov processes (CTSMPs) lies in the distribution functions ( ). In the former, ( ) is distributed according to an exponential distribution; in the latter, it can follow any arbitrary distribution.
Finally, the HCTFSSM process { , ≥ 0} can be introduced as the process that tracks the state occupied over time: Figure A1 shows a typical sample path of a semi-Markov model in which the first jump from 1 to 2 occurs at time 1 , whereas the next jump occurs at time 2 to reach state 3 . Figure A1. Sample path of a semi-Markov process.
The probabilistic properties of a semi-Markov model can also be described in terms of transition function rates [19]. We define the transition rate between state and at time as [43]: Notice that if the transition rates are constant, the semi-Markov model reduces to a Markov model.