Bayesian Reliability Estimation for Deteriorating Systems with Limited Samples Using the Maximum Entropy Approach

In this paper the combinations of maximum entropy method and Bayesian inference for reliability assessment of deteriorating system is proposed. Due to various uncertainties, less data and incomplete information, system parameters usually cannot be determined precisely. These uncertainty parameters can be modeled by fuzzy sets theory and the Bayesian inference which have been proved to be useful for deteriorating systems under small sample sizes. The maximum entropy approach can be used to calculate the maximum entropy density function of uncertainty parameters more accurately for it does not need any additional information and assumptions. Finally, two optimization models are presented which can be used to determine the lower and upper bounds of systems probability of failure under vague environment conditions. Two numerical examples are investigated to demonstrate the proposed method. 5493 Acronyms and Abbreviations RV Random variable PDF Probability density function CDF Cumulative distribution function MM Method of moments MLE Maximum likelihood estimation MEE Maximum entropy estimation S. Integral domain  Lagrange multipliers A fuzzy number Membership function The -level set of Triangular fuzzy number Prior distribution Maximum entropy density function Posterior PDF D i (t) Degradation at time t of unit D f Critical threshold y. Given samples


Introduction
Degradation is a common cause of failure of many products.In structural reliability analysis, traditionally, the uncertain deterioration and degradation are usually modeled using lifetime distributions [1], and the information of systems can be analyzed using degradation data.Generally, system reliability assessment using degradation data has many advantages because the analysts can use less collected degradation data to assess system reliability, which is useful for applications [2][3][4][5].
In engineering practice, time-dependent reliability analysis for modeling deterioration systems is necessary because the performance of many products is a deterioration process [6].Traditionally, the deteriorations of structures usually are modeled by using random variables models (such as the deterioration rate), cumulative damage models and gamma processes [6][7][8][9], respectively.The random variable (RV) models (also called degradation curve models) are used widely in engineering applications for assessing the reliability of deteriorating components [10].Examples of the RV models for reliability estimation can be found in references [11][12][13][14][15]. Another widely used method for modeling deteriorating systems is the gamma process, which belongs to a general class of stochastic processes [6].The gamma process can be used to model gradual damage monotonically accumulating over time, such as wear, fatigue, corrosion, erosion, crack growth, consumption, creep, swell, etc., and these failure models are common causes of failure of many engineering components [8,16].
In engineering practice, it is well known that data sometimes cannot be recorded or collected precisely due to various uncertainties [17,18] such as human errors, machine errors, incomplete ith information or some other unexpected situations [19].In this case, the Bayesian inference has shown to be useful for modeling uncertainty.A lot of researchers have paid more attention to Bayesian reliability analysis under vague information conditions.For example, Bayesian reliability estimations under fuzzy environments constraints were proposed by Wu [19][20][21], Taheri and Zarei [22], respectively, in recent years.Despite many efforts, reliability assessment for deteriorating systems using traditional RV models also has some limitations.In the traditional RV models, the distributions of random variables (such as the deterioration rate) usually are assumed known and precisely determined, such as normal distribution and Weibull distribution.However, determining lifetime distributions requires sufficient data which is often impossible to acquire in engineering practice, especially for some reliable and long lifetime products such as satellites, spaceships and so on.Furthermore, these assumptions are not reasonable in the case of less data and incomplete information.For example, we may face the problem that the distribution of one random variable can be viewed as a normal or Weibull distribution under small sample sizes.In order to solve the problem and consider the advantages of both the maximum entropy method and the Bayesian inference for systems under small sample sizes and incomplete information constraints, combinations of the two methods are considered for the deteriorating systems, and the fuzzy Bayesian reliability assessment for deteriorating components is proposed in this paper.
The remainder of the paper is organized as follows: in Section 2, Bayesian inference and fuzzy theory are introduced.The maximum entropy method is given in Section 3. The proposed fuzzy Bayesian reliability assessment for deteriorating components under small sample sizes is given in Section 4. Two numerical examples are presented in Section 5. Finally, Section 6 presents brief discussions and our conclusions.

Bayesian Inference
Bayesian inference is based on the subjective view of probability.Let be a random variable with a probability density function (PDF) , which is indexed by a parameter vector .Bayesian inference with observations can be given by [23][24][25]: ( where is the prior PDF, is the posterior PDF of , and is the sampling PDF of the observations, respectively.To apply Equation (1), the prior PDF is needed.Generally, the prior PDF plays an important role in reliability analysis.The more accurate of the prior PDF is, the more accurate of the posterior PDF will be.There are five methods to formulate a prior PDF based on the past data [24]: (I) the method of moments (MM); (II) maximum likelihood estimation (MLE); (III) maximum entropy estimation (MEE); (IV) two-state updating of a non-informative "pre-prior"; and (V) credible interval-based method.Furthermore, non-informative prior PDF is used widely in engineering under the case of no prior available information.More information about the non-informative prior PDF can be found in references [25,26].

Fuzzy Theory
In engineering practice, the value or distribution of a parameter usually cannot be determined precisely under small sample sizes or vague information conditions.For example, we often state that the critical threshold is "about 10" or "about 9-11", and the statements "about 10" and "about 9-11" can be viewed as fuzzy numbers.Therefore, the fuzzy sets theory provides an appropriate tool for modeling the situations where some parameters are fuzzy numbers.Let be a fuzzy real number with membership function . Its -level set is defined by for all .
Generally, is convex, closed and bounded in real number [20], and is a closed interval which can be expressed as .
There are many kinds of membership functions that can be used in engineering applications.For illustration purposes, a special kind of fuzzy real number is introduced in the paper.We say that is a triangular fuzzy real number if its membership function is given by [22]: (2) Triangular fuzzy number can be denoted by . The -level set of triangular fuzzy real number is calculated by = for all .In this paper, is also denoted by . The -level set of , shown in Figure 1, is used widely in system reliability analysis.
Figure 1.The -level set of the fuzzy real number a  .

Maximum Entropy Methods
In order to improve the accuracy in systems reliability assessment, additional assumptions should be avoided.The maximum entropy method [27][28][29] is a flexible and powerful tool for density approximation as it does not need any additional information or assumptions.The algorithms for calculating maximum entropy density can be found in [29][30][31].The maximum entropy density function (also called FDF of sample in order to keep the naming consistent with PDF) is calculated by maximizing Shannon's entropy [29]: with the constraints: (4) (5) where is the maximum entropy density function, is the integral domain, and is the known ith moment.
From the Lagrange's method, the p(x) can be expressed as [30]: where  i is the Lagrangian multiplier for the ith moment constraint, and: To solve for the Lagrange multipliers , a feasible method, named the Newton's iterative algorithm, can be adopted [29,31].Some studies, such as references [32,33], have been shown that the first three or four moments are sufficient to describe a wide range of distribution types.

The Degradation Curve of Deteriorating Components
In order to assess the reliability of deteriorating components using RV models, the degradation curves of components should be determined firstly.Suppose that a linear degradation curve exists and its initial value is D 0 , the degradation at time t for the ith unit is given by (8) where θ is a random variable and θ i is a unit specific parameter.When the unit's degradation reaches a critical threshold D f , the unit is considered to fail.From Equation ( 8), the pseudo lifetime of the ith unit is .Suppose the measurement error is a normally distributed random variable with distribution .The observed degradation at is denoted by , and the degradation of the unit at is denoted by .The relationships between and under considering measurement errors can be expressed as: In reality, the degradation curves are more complicated than linear degradation curves such as power laws and exponential curves.In these situations, these complicated curves can be transformed by logarithms.For example, we can transform an exponential curve to a linear curve by logarithmic arithmetic, which is given by: (10) In the situation of considering measurement errors, Equation ( 10) can be rewritten as: Where .Let , , and , Equation ( 11) can be rewritten as: For simplicity, we let .Consider the situation of , where , which is a very complicated regression model and is called non-linear regression model [34].The information of non-linear regression model is omitted here, but can be found in some related books.The following degradation curves are used widely for deterioration system reliability assessment [35], which can be expressed as: The different kinds of degradation curves are shown in Figure 2.

Bayesian Reliability Assessment for Deteriorating Components under Small Sample Sizes
From the aforementioned discussions, many commonly used degradation curves can be transformed into a linear curve by using some corresponding transformations.For simplicity and illustration purposes, the linear degradation curve is considered in this paper.Suppose that ( 30 n  ) units' degradation data of a component is shown in Table 1 [9], the degradation curve is given by , and the critical threshold is D f .θ is a random variable, ln ln and specific parameters can be determined by the least squares method for the ith unit degradation data.Three cases are considered in order to assess the component reliability, respectively.
Case 1: There is no prior information for random variables θ and T.
According to Equation ( 13), the pseudo lifetime T i of the corresponding ith unit can be calculated as .Then we can obtain n pseudo lifetimes T i .The maximum entropy density (PDF of sample) p(t) of the random variable T can be given by: ( From Equation ( 16), the cumulative distribution function (CDF) of is given by: ( From Equation (17), the probability of failure under becomes: Generally, Equation (17) has no analytical and close form solutions.We can solve it by using numerical integration methods such as the Simpson and Romberg algorithms [36].
Case 2: There is prior information for random variable T.
Suppose that the prior PDF of T is .From Equations ( 1) and ( 16), the posterior PDF is given by: ( The integration is a constant which can be denoted by . Therefore, Equation ( 19) can be rewritten as: From Equation (20), the CDF can be expressed as: (21) From Equation ( 21), the probability of failure under becomes: (

22)
Case 3: There is prior information for the random variable θ .
Suppose that the prior PDF of θ is .According to Equations ( 1) and ( 16), the posterior PDF is given by: ( where = is a constant.
For example, if the prior PDF of θ is normally distributed such as , according to Equations ( 1) and ( 23), the posterior PDF of θ becomes: where = .
Similarly, if the prior PDF of θ is a Weibull distribution with = , from Equations ( 1) and ( 23), the posterior PDF of θ is given by: ( Where  is the shape parameter,  is the scale parameter and = , respectively.
Considering a random variable X with PDF , , is a monotonically differentiable function of x with a unique inverse function.We have is a continuous random variable, and its PDF is given by [34]: , , Entropy 2013, 15 5500 where , , and is unique inverse function of , respectively.
According to Equation ( 26), the posterior PDF of is calculated by: ( From Equation ( 27), the CDF is given by: ( From Equation ( 28), the probability of failure under becomes: For example, let , when the prior PDF of θ is normally distributed such as , from Equations ( 24) and ( 27), the PDF is given by: ( From Equation (30), the CDF of becomes: Similarly, when the prior PDF of θ is a Weibull distribution with the shape parameter  and the scale parameter , from Equations ( 24) and ( 27), the posterior PDF and CDF are respectively given by: (32) and:

Fuzzy Bayesian Reliability Assessment for Deteriorating Components
In Section 4.2, we described how to combine Bayesian inference and the maximum entropy method to assess component reliability in details.However, due to various uncertainties and lacking of data, it is impossible to determine all parameters precisely.For example, the critical threshold cannot be determined precisely but can be modeled by the fuzzy number .The prior distribution of the random variable θ is , that is, the mean value is a fuzzy number .In this section, we will discuss how to assess reliability when fuzzy parameters exist in the systems.

Case 1:
The critical threshold is a fuzzy number of the corresponding ith unit under the -level set can be obtained by . Then n lower and upper bounds pseudo lifetimes       under the set can be obtained, respectively.According to Equation ( 16), the maximum entropy density (PDF of sample) under       can be expressed as: (34) and: (35) respectively.
From Equations ( 34) and (35), the lower and upper bounds of probability of failure under the -level sets can be calculated as: (36) and: (37) respectively.
Case 2: Some parameters in the prior distribution are fuzzy numbers Suppose that the prior PDF of the random variable θ is , and the parameters in are fuzzy numbers.The lower and upper bounds of probability of failure under the -level set can be expressed as the following optimization model: With different -level, the membership function of can be determined.The minimum and maximum probability of failure can be calculated by: (39) For example, suppose that the prior PDF of is , where the mean value is a fuzzy number.The lower and upper bounds of probability of failure under the -level sets can be determined as: (41) respectively.

Illustrative Examples and Discussion
In this section, two examples are analyzed to demonstrate the proposed method.The first example is associated with only one linear degradation cure, and its critical threshold is a fuzzy number.The degradation curve of the second example is a non-linear curve, and some fuzzy numbers exist in the prior distribution.

Example 1
GaAs is a typical reliable and long lifetime laser which is widely used in military.Since the GaAs laser is highly reliable and very costly, there is little failure information because carrying out many experiments is impossible.Suppose the degradation data of five GaAs laser units is as shown in Table 2.The critical threshold is 10, that is, the laser is failure if its current increases more than the original value by 10% [9].The degradation curve of the GaAs lasers is a linear function.Since the degradation curve is a linear function, it can be expressed as , where can be determined by the least squares method from the degradation data of the ith unit.In order to assess system reliability, θ and T usually are assumed as known distribution such as normal distribution or Weibull distribution in the traditional methods, and the values of distribution parameters are determined by using the maximum likelihood estimation.In reality, the distributions information of θ and T cannot be known clearly under the small sample size conditions.In order to avoid personal assumptions, the maximum entropy method can be used since it does not need any additional assumptions and information.The degradation data of each unit and its fitted curve are shown in Figure 3. From Table 2 and Equation ( 16), the PDFs of the random variable T are shown in Figure 4, where k = 3 and k = 4 denote the results calculated using the first three and four moments, respectively.Generally, the first four moments are sufficient to describe a wide range of distribution types.From Equation (17), the CDFs of T are shown in Figure 5.
In reality, critical thresholds sometimes cannot be determined precisely due to various uncertainties.The more appropriate way to describe the thresholds is that D f is a fuzzy number around 10.The phrase "around 10" should be regarded as a fuzzy number .Suppose that the critical threshold D f in this example is a triangular fuzzy number (9,10,11).System reliability under T equal to 4000 hours is considered.From Equations ( 36) and (37), the probability of failure under different -levels are shown in Figure 6, which shows that the probability of failure is a fuzzy number rather than precise value.From Figure 6 we also know that a small variation of the critical threshold may lead to a big change of the probability of failure.

Example 2
Suppose that the degradation curve of a time-depending deteriorating system is , and the critical threshold D f = 10.θ is a random variable with fuzzy normally distributed prior distribution , that is, the value of μ θ is a triangular fuzzy number (1.1, 1.2, 1.3).The values of θ for the six units are given in Table 3.In engineering practice, the available information, such as the past data and expert opinions, is valuable information for us.In this example, we consider the situation where there is prior information for the random variable θ.However, the mean value of the prior distribution is a triangular fuzzy number.Suppose that μ θ is precisely determined, that is, μ θ = 1.2.From Equation ( 23) and the Bayesian inferences, the posterior distributions (also called updated distributions) of θ are shown in Figure 7.However, if μ θ is a fuzzy number, the posterior distribution are family distributions rather single ones.From Equations ( 27), ( 28), ( 38) and (39), the probability of failure under = 4.5 year are given in Tables 4, 5 and Figure 8, respectively.From Figure 8, we know that the probability of failure is a fuzzy number.When , it denotes that is precisely determined, and the probability of failure is a precise value.Furthermore, the probability of failure under different values is shown in Figure 9.

Conclusions
In this paper the combination of Bayesian inference and the maximum entropy method have been applied to the problem of deteriorating system reliability assessment.Due to various uncertainties and incomplete information, precise determination of all parameters is impossible in engineering applications.In this case, fuzzy sets theory and Bayesian inference as well as the maximum entropy method have shown to be useful for the case of vague environments and less data.The maximum entropy method is robust under limited data constraints because it does not need any additional assumptions.The numerical examples have shown that the probability of failure is a fuzzy number in the case that fuzzy parameters exist in systems.Furthermore, the memberships can be determined by the optimization models proposed in the paper.From the discussions and the illustrated examples, we know that the proposed method requires neither the additional assumptions nor the large sample sizes, which are needed in other commonly used methods.It should be noted that there are some limitations to the proposed method.The RV deterioration model cannot capture temporal variability associated with evolution of degradation.The extension of the method for handling multiple failure modes will be the subject of future work in our research.

Figure 3 .
Figure 3. Degradation data of each unit and its fitted curve.

Figure 4 .
Figure 4. PDFs of T calculated by using the first three and four moments.

Figure 5 .
Figure 5. CDFs calculated by using the first three and four moments.

Figure 6 .
Figure 6.Probability of failure under different -levels.

Figure 9 .
Figure 9. Probability of failure for different values

Table 3 .
The values of θ from experiments.

Table 4 .
Probability of failure bounds under different -levels when k = 3.

Table 5 .
Probability of failure bounds under different -levels when k = 4. Probability of failure under is a fuzzy number.