A Penalized Likelihood Approach to Parameter Estimation with Integral Reliability Constraints

Abstract

Stress-strength reliability problems arise frequently in applied statistics and related fields. Often they involve two independent and possibly small samples of measurements on strength and breakdown pressures (stress). The goal of the researcher is to use the measurements to obtain inference on reliability, which is the probability that stress will exceed strength. This paper addresses the case where reliability is expressed in terms of an integral which has no closed form solution and where the number of observed values on stress and strength is small. We find that the Lagrange approach to estimating constrained likelihood, necessary for inference, often performs poorly. We introduce a penalized likelihood method and it appears to always work well. We use third order likelihood methods to partially offset the issue of small samples. The proposed method is applied to draw inferences on reliability in stress-strength problems with independent exponentiated exponential distributions. Simulation studies are carried out to assess the accuracy of the proposed method and to compare it with some standard asymptotic methods.

1. Introduction

We consider a stress-strength reliability problem, R = P (Y < X), where X and Y are independently distributed as exponentiated exponential random variables. In this case, the parameter of interest, R, is an integral with no closed form solution. Likelihood-based inference for this problem involves, in part, numerical constrained optimization. The likelihood ratio test, for example, uses constrained maximum likelihood parameter estimates where the log-likelihood function is maximized subject to the constraint that the parameter of interest takes on the null-hypothesized value.

A standard approach to solving (equality) constrained problems is to apply the Lagrange method. Detailed discussions can be found in many reference works in optimization, for example, see [1]. Using the Lagrange technique, this problem requires that, in each iteration, one parameter has to be used to guarantee that the integral equality constraint is satisfied up to a given level of numerical accuracy. Satisfying the constraint involves simultaneous numerical integration and the numerical solution of a nonlinear equation for the correct parameter value. Moreover, all of this needs to be repeated for each value of the remaining parameters.

We found that the Lagrange approach led to problems of numerical accuracy and slow convergence. We also discovered that the integral function was homogeneous of degree 0 in two of the parameters of the model. The integral is therefore insensitive to equi-proportionate changes in the two parameters and this complicated optimization.

Penalty function methods are also often used to solve equality and inequality-constrained optimization problems. See Byrne [2] and the extensive list of references therein. The basic idea is to solve a constrained optimization problem by solving a sequence of unconstrained problems where constraint violation is penalized in a successively harsher manner. The sequence of unconstrained optima converges to the constrained optimum. Smith et al. (2014) [3] demonstrates the accuracy of the penalty method in a constrained optimization problem. When implementing this approach to the problem considered in this paper, we found it was numerically stable and rapid in the sense that parameter estimates from successively more harshly penalized models quickly converged to optimal values and became effectively insensitive to further penalization.

The remainder of the paper is organized as follows. Section 2 begins with a brief description of our stress-strength reliability problem. We then examine some of the properties of the unconstrained likelihood function. This seems necessary because the likelihood function is not concave and generally cannot be optimized by an algorithm that uses the exact Hessian matrix. We then introduce the integral constraint and some of its properties together with a numerical example to illustrate how the Lagrange approach failed. The section concludes with a discussion of some properties and an implementation of the penalty function approach. Section 3 briefly reviews the likelihood-based third-order method for inference concerning a scalar parameter of interest when samples are small. Section 4 applies the approaches developed in Sections 2 and 3 to study inference for the stress-strength reliability problem with independent exponentiated exponential distributions. Results from simulation studies are presented to illustrate the extreme accuracy of our suggested approach. Some concluding remarks are given in Section 5. Technical details related to some issues raised in Section 2 are presented in the Appendix.

2. Computational Issues and Penalized Likelihood

2.1. Problem

We consider statistical inference for the reliability, R = P (Y < X) where X and Y are independently distributed with exponentiated exponential distributions. The estimation of R is important in the statistical literature and has been widely studied in other areas, such as radiology, mechanical engineering and materials science. In the context of the stress-strength model, reliability refers to the probability that the unit’s strength Y is less than the stress X. Following the notation in [4], the cumulative distribution function of the two-parameter exponentiated exponential distribution, EE(α, β), is

$$F(x;\alpha ,\beta )={(1-e^{-\beta x})}^{\alpha };\alpha >0,\beta >0,x>0$$

(1)

where α is the shape parameter and β is the scale parameter. The EE(α, β) distribution is also known as the generalized exponential distribution. It is equivalent to the exponentiated Weibull (κ = 1, α, σ) distribution as introduced in [5], where κ is the first shape parameter, α is the second shape parameter and σ is the scale parameter. If α = 1, the distribution reduces to the one parameter exponential (β) model. The EE(α, β) distribution is recognized as a useful model for lifetime data or skewed data. It has a monotone increasing hazard function when α > 1, decreasing when α < 1 and constant when α = 1. It also has a unimodal and right-skewed density function. For a fixed scale parameter value, the skewness gradually decreases as the shape parameter increases.

Kundu and Gupta [6] and Raqab et al. [7] considered maximum likelihood estimation of R when X and Y are independent EE(α₁, β₁) and EE(α₂, β₂) distributed random variables and where β₁ and β₂ are assumed to be the same. Then $R=\frac{{\alpha }_1}{{\alpha }_1+{\alpha }_2}$. Likelihood-based inference for R can then be obtained easily. However, without the assumption that β₁ = β₂, then

$$R=P(Y<X)={\displaystyle {\int }_0^{\infty }{\alpha }_1{\beta }_1{(1-e^{-{\beta }_{1}x})}^{{\alpha }_1-1}{e}^{-{\beta }_{1}x}{(1-e^{-{\beta }_{2}x})}^{{\alpha }_2}}dx$$

(2)

and this integral does not have a known closed form. To obtain likelihood-based inference for R in this case, it is necessary to solve an optimization problem with an integral constraint. We found that standard macros, functions and subroutines in popular software packages such as R or Matlab experienced problems in converging to constrained maximum likelihood estimates in a Lagrange setting. As noted in the Introduction, a penalized likelihood method is proposed to provide a solution for this constrained maximization problem.

Once the constrained maximum likelihood estimates have been obtained, likelihood methods can be used to draw inferences about R. In this paper, the third-order method discussed in Fraser and Reid [8] is applied. This is known to be very accurate even when the sample sizes are small. See, for example, Chang and Wong [9] and She et al. [10] where the accuracy of third order methods has been assessed.

2.2. Unconstrained Likelihood and Its Properties

Let x = (x₁,…, x_n)′ and y = (y₁,…, y_m)′ be independent random samples from EE(α₁, β₁) and EE(α₂, β₂) respectively. Then the log-likelihood function is:

$$\begin{array}{l}l(\theta ;x,y)=l({\alpha }_1,{\beta }_1,{\alpha }_2,{\beta }_2;x,y)\\ =n\log {\alpha }_1+n\log {\beta }_1+({\alpha }_1-1){\displaystyle \sum _{i=1}^n\log (1-e^{-{\beta }_1{x}_i})-{\beta }_1{\displaystyle \sum _{i=1}^n{x}_i}}\\ +m\log {\alpha }_2+m\log {\beta }_2+({\alpha }_2-1){\displaystyle \sum _{j=1}^m\log (1-e^{-{\beta }_2{y}_j})-{\beta }_2{\displaystyle \sum _{j=1}^m{y}_j}}\end{array}$$

(3)

where θ = (α₁, α₂, β₁, β₂)′.

The log-likelihood function is infinitely differentiable and has some interesting properties. In the first place it is not concave. Nor is it quasiconcave or pseudoconcave or a member of any of the other classes of generalized concave functions that appear regularly in the literature on optimization. However, the function is strictly concave in each of its parameters individually. For fixed values of any 3 parameters, the function reaches a unique maximum with respect to the fourth. The determinant of the Hessian matrix changes sign throughout the domain of the function. But, the gradient of the log-likelihood function vanishes at a unique vector of parameter values and at that point and in a neighbourhood around it the Hessian matrix is negative definite. All of these results are derived in the Appendix. So, when the function is restricted to an open region about the global optimum, the function is concave. These results have some important implications regarding how to maximize log-likelihood for any given sample. In particular, all parameters cannot typically be estimated simultaneously using any algorithm that uses exact Hessian information. Quasi–Newton techniques (sometimes called variable metric) work by building an increasingly accurate quadratic approximation to the log-likelihood surface. These will work in this setting as long as exact Hessians are not used building the surface approximation and updating the parameters. Hessian matrices from the exponentiated exponential model usually won’t be negative definite and will lead to inappropriate updating. The Broyden–Fletcher–Goldfarb–Shanno (BFGS) algorithm can work because some versions of it start with negative definite and symmetric “approximate” Hessians and update them until they converge to the true Hessian at the optimum. Given the independence of X and Y it is also possible to concentrate the log-likelihood using the derivative information on α₁ and α₂ and then apply simple line searches to find the β parameters. The Appendix shows why this latter, perhaps more cumbersome, approach will work but only in the unconstrained case.

In summary, obtaining the unconstrained parameter estimates is possible if done carefully. As well, we found it is often helpful to restrict large changes in parameters during estimation. We recommend box constraints be placed on parameters during estimation. Preset upper and lower bounds on the parameters require that the parameters must be chosen from a four dimensional hyper-cube. A version of BFGS that implants constraints of this type is available in the statistical package, R.

2.3. The Integral Reliability Constraint

In this paper, the parameter of interest is stress-strength reliability, R = P (Y < X), where X and Y are independently distributed as EE(α₁, β₁) and EE(α₂, β₂) distributions respectively. The constraint function, R, is given in (2) and has no known closed form solution. Straightforward differentiation shows that R is an increasing function of α₁ and β₂ and a decreasing function of α₂ and β₁. Introducing the change of variables: z = β₁x, the integral constraint can be expressed equivalently as:

$$R={\displaystyle {\int }_0^{\infty }{\alpha }_1{(1-e^{-z})}^{{\alpha }_1-1}{e}^{-z}}{(1-e^{-\frac{{\beta }_2}{{\beta }_1}z})}^{{\alpha }_2}dz$$

This establishes that R is homogeneous of degree 0 in (β₁, β₂). The contours of R are all constant along the line in parameter space satisfying β₂ = cβ₁. This can complicate the numerical optimization process. It also provides a good reason to introduce the box constraints on the parameters as discussed in Section 2.2.

2.4. Constrained Optimization: Lagrange

We need to find the constrained maximum likelihood parameter estimates ${\widehat{\theta }}_{\psi }={({\tilde{\alpha }}_1,{\tilde{\alpha }}_2,{\tilde{\beta }}_1,{\tilde{\beta }}_2)}^{\prime }$ for a given ψ₀. This requires that we maximize the log-likelihood function in (3) subject to the constraint R = ψ(θ) = ψ₀ in (2). Using the Lagrange approach, for given values of ψ₀ and three of the likelihood parameters, the constraint R = ψ(θ) = ψ₀ can be numerically integrated and the fourth parameter simultaneously chosen so that the integral constraint holds. The value of the fourth parameter and ψ₀ are held fixed while the log-likelihood function is then maximized with respect to the three “free” likelihood parameters. This process is iterated until convergence.

There is much that can go wrong in the above iterative process. The likelihood function is not concave, the equality constraint is highly nonlinear in the parameters, the constraint integral must be evaluated numerically and then solved numerically and the integral R is homogeneous of degree 0 with respect to two of the parameters.

We discovered in simulation that this iterative Lagrange method sometimes gave relatively satisfactory results for some values of ψ₀ but it often went astray. Typically it took a long time for the estimation process to converge. Performance degraded quickly when R was constrained to be closer to the boundary values of 0 or 1.

Table 1. Dataset1: Two Simulated Datasets.

Data Set	Observations					Sample Size
X	0.4977 0.6414	0.0781 0.2669	0.3827 0.1978	0.2694 0.1968	0.4125 0.2397	10
Y	1.7057 1.9481	1.0191 2.1290	0.5899 0.8109	0.9031 1.6463	0.9207 1.9842	10

records one of these situations for a simulated data set from EE(2, 3) and EE(5, 1.6015) distributions each with a sample size of 10 and ψ₀ was set equal to 0.1. Initial attempts to program the search algorithm in Matlab resulted in floating point overflow and division by zero errors.

We next tried Matlab’s built-in macro, “fminsearch”, which uses a simplex search method. This macro provided the correct unconstrained parameter estimates but failed when constraints were introduced. The unconstrained parameter estimates are recorded in

Table 2. Parameter Estimates for Dataset1.

Method	α1	α2	β1	β2	R	Loglikelihood
Unconstrained	4.4239	8.5227	6.7793	2.0262	0.0142	−3.3191
Constrained: Penalty	3.6028	3.2018	5.2707	1.5700	0.0989	−5.1659

. Not surprisingly, these unconstrained parameter estimates do not solve the constrained optimization problem. They yield a constraint value of 0.0141 as opposed 0.1. Correct results were found using a penalized likelihood method (appearing as Penalty in the table). This method is introduced below.

We encountered similar problems with small samples and when sample sizes were unequal. We reprogrammed the Lagrange constrained optimization algorithm in the statistical package R. In addition to slow, inaccurate and sometimes failed convergence, we also encountered cases of singular matrices during the optimization process.

2.5. The Penalized Likelihood Approach

As noted in the previous section, we encountered a wide variety of serious numerical problems when we tried to implement the classical Lagrange approach to our integral equality constrained optimization problem. We now present a penalized likelihood method, which was also discussed in Smith et al. (2014) [3], that reliably solves the constrained optimization problems addressed in this paper. We begin by defining the penalized likelihood function, PL, as:

$$PL(\theta ,{\psi }_0;x,y)=l(\theta ;x,y)-k{(R(\theta )-{\psi }_0)}^2,k>0$$

(4)

The function consists of the log-likelihood function l(θ; x, y) and a positive function, k(R(θ) – ψ₀)², called the penalty function, that is subtracted from the the likelihood function. Here we consider a penalty function that is the square of the divergence of R from the constraint value ψ₀. The greater is k, the more the likelihood is penalized (decreased) for given divergences of R from ψ₀. Next, consider the unconstrained problem of maximizing PL (θ, ψ₀; x, y) with respect to θ for a given value of k. Formally, we obtain an optimal vector ${\theta }_k^{*}$. When k = 0, we obtain the parameter vector that maximizes the unconstrained log-likelihood. As k increases and we successively solve the optimization problems, we obtain a sequence of optimal parameter vectors that converges to the solution of the constrained problem of choosing θ to maximize l(θ; x, y) subject to R(θ) = ψ₀. In contrast to the Lagrange approach, we now have a sequence of unconstrained optimization problems instead of one constrained problem. This means the penalty approach may be theoretically more complicated. But, in practice, we found it is faster and more reliable than the Lagrange approach to our problem. First, the penalty approach does not require that the equality R(θ) = ψ₀ hold at any stage of the optimization process. Instead, divergence of R(θ) from ψ₀ is (increasingly) discouraged as k increases. Many of the numerical problems we encountered in applying the Lagrange technique arose from trying to impose R(θ) = ψ₀ exactly. Second, in practice as k increases we found that the estimated parameters quickly converged to optimal values and it was not necessary to continue to increase k and solve additional optimization problems. Finally, the Lagrange and Penalty approaches are formally dual so that the Lagrange multiplier can be recovered as the limiting value of the partial derivative of P L with respect to the parameter ψ₀.

A recent analysis of the rigorous foundations of the penalty (and other) approaches to constrained optimization can be found in [2] and the discussion that follows draws upon Byrne’s presentation. We need very few technical conditions to hold in order for the sequence of P L-maximizing vectors, ${\theta }_k^{*}$, to converge to the solution of the constrained optimization problem. First, we need the permissible parameter values to be drawn from a compact set. We satisfy this condition by imposing box constraints on the four likelihood parameters. Second, there is a straightforward restriction that l and R are continuous functions of the parameters. These conditions are satisfied. Finally, we require that the sequence of optimal vectors, ${\theta }_k^{*}$, correspond to the global maximum of penalized likelihood for each k. Our experience is that, for each k, there is only ever one optimum and that occurs where the gradient of PL vanishes. In all cases the Hessian matrix at the optimum was negative definite.

When implementing the penalty approach we found that when 0.1 < ψ₀ < 0.9, there was little gain from increasing k beyond 10,000. For the extreme values of ψ₀, k did not have to exceed 80,000. As well, short sequences were sufficient. It was not necessary solve separate optimization problems for all possible values of k.

We obtained parameter estimates from unconstrained optimization of P L using the optim function in the statistical software package, R. Within optim, we adopted the L-BFGS-B algorithm. L-BFGS-B was developed by Byrd et al. [11] and is a quasi-Newton optimization variant of the Broydon-Fletcher-Goldfarb-Shanno (BFGS) algorithm. L-BFGS-B conveniently allows for simple upper and lower search bounds (box constraints) on the parameters. As noted earlier, L-BFGS-B does not use an exact Hessian. Instead it updates an approximate Hessian that is guaranteed to be negative definite and symmetric. The approximate Hessian converges to the true one. Our code is available upon request.

3. Likelihood-based Inference for Any Scalar Parameter of Interest

In this section, we provide a brief review of likelihood-based inference for any scalar parameter of interest. Let y = (y₁,…, y_n)′ be a random sample from a distribution with log-likelihood function ℓ(θ) = ℓ(θ; y), where θ is a vector parameter with dim(θ) = p. Also let ψ = ψ(θ) be a scalar parameter of interest. Denote $\widehat{\theta }$ to be the overall maximum likelihood estimate, which satisfies

$${{\mathit{\ell }}_{\theta }(\widehat{\theta })=\frac{\partial \mathit{\ell }(\theta )}{\partial \theta }|}_{\theta =\widehat{\theta }}=0$$

Moreover, denote ${\widehat{\theta }}_{\psi }$ as the constrained maximum likelihood estimate of θ for a given ψ(θ) = ψ₀. $\widehat{\theta }$ and ${\widehat{\theta }}_{\psi }$ can be obtained from the penalized likelihood method discussed in Section 2. Moreover, with ${\widehat{\theta }}_{\psi }$, the corresponding Lagrange multiplier, $\widehat{K}$, can be obtained. Define the tilted log-likelihood function as

$$\tilde{\mathit{\ell }}(\theta )=\mathit{\ell }(\theta )+\widehat{K}[\psi (\theta )-{\psi }_0]$$

(5)

which has the property $\tilde{\mathit{\ell }}({\widehat{\theta }}_{\psi })=\mathit{\ell }({\widehat{\theta }}_{\psi })$.

Two widely used likelihood-based methods for obtaining asymptotic confidence interval for ψ are based on the maximum likelihood estimator of θ and the signed log-likelihood ratio statistic. It is well-known that ${(\widehat{\theta }-\theta )}^{\prime }{[\mathit{\upsilon ar}(\widehat{\theta })]}^{-1}(\widehat{\theta }-\theta )$ is asymptotically distributed as χ²-distribution with p degrees of freedom, and $\mathit{\upsilon ar}(\widehat{\theta })$ can be estimated by the inverse of either the expected Fisher information matrix or the observed information matrix evaluated at $\widehat{\theta }$. In practice, the latter,

$$\widehat{\mathit{\upsilon ar}}(\widehat{\theta })\approx j_{\theta {\theta }^{\prime }}^{-1}(\widehat{\theta })={\left[-{\mathit{\ell }}_{\theta {\theta }^{\prime }}(\widehat{\theta })\right]}^{-1}={\left[-\frac{{\partial }^2\mathit{\ell }(\theta )}{\partial \theta \partial {\theta }^{\prime }}\right]}_{\theta =\widehat{\theta }}^{-1}$$

is preferred because of the simplicity in calculation. By applying the Delta method, we have

$$\widehat{\mathit{\upsilon ar}}(\widehat{\psi })=\widehat{\mathit{\upsilon ar}}(\psi (\widehat{\theta }))\approx {{\psi }^{\prime }}_{\theta }(\widehat{\theta })\widehat{\mathit{\upsilon ar}}(\widehat{\theta }){\psi }_{\theta }(\widehat{\theta })$$

(6)

where ${{\psi }_{\theta }(\widehat{\theta })=\frac{\partial \psi (\theta )}{\partial \theta }|}_{\theta =\widehat{\theta }}$. Hence $\frac{\widehat{\psi }-\psi }{\sqrt{\widehat{\mathit{\upsilon ar}}(\widehat{\psi })}}$ is asymptotically distributed as N(0; 1). Thus a (1−γ)100% confidence interval of ψ based on the maximum likelihood estimator is

$$\left(\widehat{\psi }-z_{\gamma /2}\sqrt{\widehat{\mathit{\upsilon ar}}(\widehat{\psi })},\widehat{\psi }+z_{y/2}\sqrt{\widehat{\mathit{\upsilon ar}}(\widehat{\psi })}\right)$$

(7)

where z_γ/2 is the (1 − γ/2)100^th percentile of the standard normal distribution.

Alternatively, with regularity conditions stated in [12], the signed log-likelihood ratio statistic is

$$\begin{array}{l}r(\psi )=\mathrm{sgn}(\widehat{\psi }-\psi )\{2[\mathit{\ell }(\widehat{\theta })-\mathit{\ell }(\widehat{\theta }\psi )]{\}}^{1/2}\\ =\mathrm{sgn}(\widehat{\psi }-\psi )\{2[\mathit{\ell }(\widehat{\theta })-\tilde{\mathit{\ell }}(\widehat{\theta }\psi )]{\}}^{1/2}\end{array}$$

(8)

is asymptotically distributed as N(0, 1). Therefore, a (1 − γ)100% confidence interval of ψ based on the signed log-likelihood ratio statistic is

$$\{\psi :|r(\psi )|\le z_{\gamma /}{}_2\}$$

(9)

It should be noted that both methods have rates of convergence only O(n^−1/2). While the maximum likelihood estimator based interval is often preferred because of the simplicity in calculation, the signed log-likelihood ratio method is invariant to reparametrization and the maximum likelihood estimator based method is not. Doganaksoy and Schmee [13], showed that the signed log-likelihood ratio statistic based interval has better coverage property than the maximum likelihood estimator based interval in cases they considered.

In recent years, various adjustments to the signed log-likelihood ratio statistic have been proposed to improve the accuracy of the signed log-likelihood ratio statistic. Reid [14] and Severeni [15] gave detail overview of this development. In this paper, we consider the modified signed log-likelihood ratio statistic, which was introduced by [16,17] and has the form

$$r*(\psi )=r(\psi )+r{(\psi )}^{-1}\log \left\{\frac{Q(\psi )}{r(\psi )}\right\}$$

(10)

where r(ψ) is the signed log-likelihood ratio statistic, and Q(ψ) is a statistic that based on the log-likelihood function and an ancillary statistic. Barndorff-Nielsen [16,17] showed that r^*(ψ) is asymptotically distributed as N(0, 1) with rate of convergence O(n^−3/2). Hence a 100(1 − γ)% confidence interval for ψ(θ) is given by

$$\{\psi :|r^{*}(\psi )|\le z_{\gamma /}{}_2\}$$

(11)

However, for a general model, an ancillary statistic might not exist, and even when it does, it might not be unique. Fraser and Reid [8] showed that Q(ψ) is a standardized maximum likelihood departure in the canonical parameter scale and Fraser et al. [18] derived the general formula for obtaining the statistic Q(ψ). More specifically, Fraser et al. [18] obtained the locally defined canonical parameter

$$\phi (\theta )=\frac{\partial \mathit{\ell }(\theta )}{\partial y}V$$

(12)

where

$${V=-{\left(\frac{\partial z(y;\theta )}{\partial y}\right)}^{-1}\left(\frac{\partial z(y;\theta )}{\partial \theta }\right)|}_{\theta =\widehat{\theta }}$$

(13)

is the ancillary direction and z(y; θ) = (z₁(y; θ),⋯, z_n(y; θ))′ is a vector of pivotal quantities. Then the standardized maximum likelihood departure in φ(θ) scale takes the form:

$$Q(\psi )=\mathrm{sgn}(\widehat{\psi }-\psi )\frac{|\chi (\widehat{\theta })-\chi ({\widehat{\theta }}_{\psi })|}{\sqrt{\widehat{\mathit{\upsilon ar}}(\chi (\widehat{\theta })-\chi ({\widehat{\theta }}_{\psi }))}}$$

(14)

where

$$\chi (\theta )={\psi }_{\theta }({\widehat{\theta }}_{\psi }){\phi }_{\theta }^{-1}({\widehat{\theta }}_{\psi })\phi (\theta )$$

(15)

is the recalibrated parameter of interest in the φ(θ) scale, and $|\chi (\widehat{\theta })-\chi ({\widehat{\theta }}_{\psi })|$ is a measure of maximum likelihood departure of $|\widehat{\psi }-\psi |$ in φ(θ) scale,

$${\psi }_{\theta }(\theta )=\frac{\partial \psi (\theta )}{\partial \theta }\text{and}{\phi }_{\theta }(\theta )=\frac{\partial \phi (\theta )}{\partial \theta }$$

Since the exact $\mathit{\upsilon ar}(\chi (\widehat{\theta })-\chi ({\widehat{\theta }}_{\psi }))$ is difficult to obtain, Fraser et al. [18] showed that it can be approximated by

$$\widehat{\mathit{\upsilon ar}}(\chi (\widehat{\theta }-\chi ({\widehat{\theta }}_{\psi }))\approx {\psi }_{\theta }({\widehat{\theta }}_{\psi }){\tilde{j}}_{\theta \theta }^{-1}({\widehat{\theta }}_{\psi }){{\psi }^{\prime }}_{\theta }({\widehat{\theta }}_{\psi })\frac{|{\tilde{j}}_{(\theta \theta )}({\widehat{\theta }}_{\psi })|}{|j_{(\theta \theta )}(\widehat{\theta })|}$$

16

where

$$\begin{array}{ccc}|j_{(\theta \theta )}(\widehat{\theta })|=|j_{\theta \theta }(\widehat{\theta })||{\phi }_{\theta }(\widehat{\theta }){|}^{-2}& \text{and}& |{\tilde{j}}_{(\theta \theta )}(\widehat{\theta })|=|{\tilde{j}}_{\theta \theta }({\widehat{\theta }}_{\psi })||{\phi }_{\theta }({\widehat{\theta }}_{\psi }){|}^{-2}\end{array}$$

with j_θθ(θ) and ${\tilde{j}}_{\theta \theta }(\theta )$ being the observed information matrix obtained from the log-likelihood function and the tilted log-likelihood function respectively. Hence, the confidence interval of ψ based on r^*(ψ) can be obtained from Equation (11).

4. Application to Stress-Strength Reliability with Independent EE Distributions

In Section 2 the EE distribution was introduced as was the reliability constraint. In Section 4.1, we consider the case where the scale parameters are different. The proposed penalized likelihood method is used to obtain the constrained maximum likelihood estimate. The likelihood-based third order method is then applied to obtain inference for R = P (Y < X). Section 4.2 presents results from some numerical studies. The special case where the scale parameters are assumed to be equal, and hence, R = P (Y < X) has a closed-form, is also examined.

4.1. Stress-Strength Reliability with Unequal Scale Parameters

Let X and Y be independently distributed as EE(α₁, β₁) and EE(α₂, β₂) respectively. R(θ) is given in (2) which does not have a closed form when β₁ ≠ β₂. Following the steps in Section 3, we first obtain the overall maximum likelihood estimate $\widehat{\theta }={({\widehat{\alpha }}_1,{\widehat{\alpha }}_2,{\widehat{\beta }}_1,{\widehat{\beta }}_2)}^{\prime }$ and the constrained maximum likelihood estimate ${\widehat{\theta }}_{\psi }=({\tilde{\alpha }}_1,{\tilde{\alpha }}_2,{\tilde{\beta }}_1,{\tilde{\beta }}_2)\prime$ by the penalized likelihood method discussed in Section 2.5. Then observed information matrix $j_{\theta \theta }(\widehat{\theta })$ can be obtained accordingly as follows.

$$\begin{array}{l}{j}_{\theta \theta }(\theta )=-{\mathit{\ell }}_{\theta \theta }(\widehat{\theta })\\ =\left(\begin{array}{cccc}\frac{n}{{\widehat{\alpha }}_1^2}& -{\displaystyle \sum _{i=1}^n\frac{x_i{e}^{-{\widehat{\beta }}_1{x}_i}}{1-e^{-{\widehat{\beta }}_1{x}_i}}}& 0& 0\\ -{\displaystyle \sum _{i=1}^n\frac{x_i{e}^{-{\widehat{\beta }}_1{x}_i}}{1-e^{-{\widehat{\beta }}_1{x}_i}}}& \frac{n}{{\widehat{\beta }}_1^2}+A& 0& 0\\ 0& 0& \frac{m}{{\widehat{\alpha }}_2^2}& -{\displaystyle \sum _{j=1}^m\frac{y_j{e}^{-{\widehat{\beta }}_2{y}_j}}{1-e^{-{\widehat{\beta }}_2{y}_j}}}\\ 0& 0& -{\displaystyle \sum _{j=1}^m\frac{y_j{e}^{-{\widehat{\beta }}_2{y}_j}}{1-e^{-{\widehat{\beta }}_2{y}_j}}}& \frac{m}{{\widehat{\beta }}_2^2}+B\end{array}\right)\end{array}$$

where $A=({\widehat{\alpha }}_1-1){\displaystyle \sum _{i=1}^n\frac{x_i^2{e}^{-{\widehat{\beta }}_1{x}_i}}{{(1-e^{-{\widehat{\beta }}_1{x}_i})}^2}}$ and $B=({\widehat{\alpha }}_2-1){\displaystyle \sum _{j=1}^m\frac{y_j^2{e}^{-{\widehat{\beta }}_2{y}_j}}{{(1-e^{-{\widehat{\beta }}_2{y}_j})}^2}}$.

The tilted log-likelihood function $\tilde{l}(\theta )$ is defined as

$$\tilde{l}(\theta )=l(x,y;{\alpha }_1,{\alpha }_2,{\beta }_1,{\beta }_2)+\widehat{K}\left[\psi (\theta )-\psi \right]$$

where ψ(θ) = R defined by (2). Similarly, we can obtain the constrained maximum likelihood estimate ${\widehat{\theta }}_{\psi }={({\tilde{\alpha }}_1,{\tilde{\alpha }}_2,{\tilde{\beta }}_1,{\tilde{\beta }}_2)}^{\prime }$ by the penalized likelihood method and $\widehat{K}$ is the Lagrange multiplier, then constrained observed information matrix ${\widehat{j}}_{\theta \theta }({\theta }_{\psi })$ can be written as

$${\tilde{j}}_{\theta \theta }({\widehat{\theta }}_{\psi })=-{\tilde{l}}_{\theta \theta }({\widehat{\theta }}_{\psi })=\left(\begin{array}{llll}{\tilde{j}}_{{\alpha }_1{\alpha }_1}({\widehat{\theta }}_{\psi })\hfill & {\tilde{j}}_{{\alpha }_1{\alpha }_2}({\widehat{\theta }}_{\psi })\hfill & {\tilde{j}}_{{\alpha }_1{\beta }_1}({\widehat{\theta }}_{\psi })\hfill & {\tilde{j}}_{{\alpha }_1{\beta }_2}({\widehat{\theta }}_{\psi })\hfill \\ {\tilde{j}}_{{\beta }_1{\alpha }_1}({\widehat{\theta }}_{\psi })\hfill & {\tilde{j}}_{{\beta }_1{\alpha }_2}({\widehat{\theta }}_{\psi })\hfill & {\tilde{j}}_{{\beta }_1{\beta }_1}({\widehat{\theta }}_{\psi })\hfill & {\tilde{j}}_{{\beta }_1{\beta }_2}({\widehat{\theta }}_{\psi })\hfill \\ {\tilde{j}}_{{\alpha }_2{\alpha }_1}({\widehat{\theta }}_{\psi })\hfill & {\tilde{j}}_{{\alpha }_2{\alpha }_2}({\widehat{\theta }}_{\psi })\hfill & {\tilde{j}}_{{\alpha }_2{\beta }_1}({\widehat{\theta }}_{\psi })\hfill & {\tilde{j}}_{{\alpha }_2{\beta }_2}({\widehat{\theta }}_{\psi })\hfill \\ {\tilde{j}}_{{\beta }_2{\alpha }_1}({\widehat{\theta }}_{\psi })\hfill & {\tilde{j}}_{{\beta }_2{\alpha }_2}({\widehat{\theta }}_{\psi })\hfill & {\tilde{j}}_{{\beta }_2{\beta }_1}({\widehat{\theta }}_{\psi })\hfill & {\tilde{j}}_{{\beta }_2{\beta }_2}({\widehat{\theta }}_{\psi })\hfill \end{array}\right)$$

where

• ${\tilde{j}}_{{\alpha }_1{\alpha }_1}({\widehat{\theta }}_{\psi })=\frac{n}{{\tilde{\alpha }}_1^2}-\widehat{K}{R}_{{\alpha }_1{\alpha }_1}({\widehat{\theta }}_{\psi })$, where $R_{{\alpha }_1{\alpha }_1}({\widehat{\theta }}_{\psi })={\frac{{\partial }^{2}R(\theta )}{\partial {\alpha }_1^2}|}_{\theta ={\widehat{\theta }}_{\psi }}$.
• ${\tilde{j}}_{{\alpha }_1{\alpha }_2}({\widehat{\theta }}_{\psi })=-\widehat{K}{R}_{{\alpha }_1{\alpha }_2}({\widehat{\theta }}_{\psi })$, where $R_{{\alpha }_1{\alpha }_2}({\widehat{\theta }}_{\psi })={\frac{{\partial }^{2}R(\theta )}{\partial {\alpha }_1\partial {\alpha }_2}|}_{\theta ={\widehat{\theta }}_{\psi }}$.
• ${\tilde{j}}_{{\alpha }_1{\beta }_1}({\widehat{\theta }}_{\psi })=-{\displaystyle \sum _{i=1}^n\frac{x_i{e}^{-{\tilde{\beta }}_1{x}_i}}{1-e^{-{\tilde{\beta }}_1{x}_i}}}-\widehat{K}{R}_{{\alpha }_1{\beta }_1}({\widehat{\theta }}_{\psi })$, where $R_{{\alpha }_1{\beta }_1}({\widehat{\theta }}_{\psi })={\frac{{\partial }^{2}R(\theta )}{\partial {\alpha }_1\partial {\beta }_1}|}_{\theta ={\widehat{\theta }}_{\psi }}$.
• ${\tilde{j}}_{{\alpha }_1{\beta }_2}({\widehat{\theta }}_{\psi })=-\widehat{K}{R}_{{\alpha }_1{\beta }_2}({\widehat{\theta }}_{\psi })$, where $R_{{\alpha }_1{\beta }_2}({\widehat{\theta }}_{\psi })={\frac{{\partial }^{2}R(\theta )}{\partial {\alpha }_1\partial {\beta }_2}|}_{\theta ={\widehat{\theta }}_{\psi }}$.
• ${\tilde{j}}_{{\alpha }_2{\alpha }_2}({\widehat{\theta }}_{\psi })=\frac{m}{{\tilde{\alpha }}_2^2}-\widehat{K}{R}_{{\alpha }_2{\alpha }_2}({\widehat{\theta }}_{\psi })$, where $R_{{\alpha }_2{\alpha }_2}({\widehat{\theta }}_{\psi })={\frac{{\partial }^{2}R(\theta )}{\partial {\alpha }_2^2}|}_{\theta ={\widehat{\theta }}_{\psi }}$.
• ${\tilde{j}}_{{\alpha }_2{\beta }_1}({\widehat{\theta }}_{\psi })=-\widehat{K}{R}_{{\alpha }_2{\beta }_1}({\widehat{\theta }}_{\psi })$, where $R_{{\alpha }_2{\beta }_1}({\widehat{\theta }}_{\psi })={\frac{{\partial }^{2}R(\theta )}{\partial {\alpha }_2\partial {\beta }_1}|}_{\theta ={\widehat{\theta }}_{\psi }}$.
• ${\tilde{j}}_{{\alpha }_2{\beta }_2}({\widehat{\theta }}_{\psi })=-{\displaystyle \sum _{j=1}^m\frac{y_j{e}^{-{\tilde{\beta }}_2{y}_j}}{1-e^{-{\tilde{\beta }}_2{y}_j}}}-\widehat{K}{R}_{{\alpha }_2{\beta }_2}({\widehat{\theta }}_{\psi })$, where $R_{{\alpha }_2{\beta }_2}({\widehat{\theta }}_{\psi })={\frac{{\partial }^{2}R(\theta )}{\partial {\alpha }_2\partial {\beta }_2}|}_{\theta ={\widehat{\theta }}_{\psi }}$.
• ${\tilde{j}}_{{\beta }_1{\beta }_1}({\widehat{\theta }}_{\psi })=\frac{m}{{\tilde{\beta }}_1^2}+({\tilde{\alpha }}_1-1){\displaystyle \sum _{i=1}^n\frac{x_i^2{e}^{-{\tilde{\beta }}_1{x}_i}}{{(1-e^{-{\tilde{\beta }}_1{x}_i})}^2}}-\widehat{K}{R}_{{\beta }_1{\beta }_1}({\widehat{\theta }}_{\psi })$, where $R_{{\beta }_1{\beta }_1}({\widehat{\theta }}_{\psi })={\frac{{\partial }^{2}R(\theta )}{\partial {\beta }_1^2}|}_{\theta ={\widehat{\theta }}_{\psi }}$.
• ${\tilde{j}}_{{\beta }_1{\beta }_2}({\widehat{\theta }}_{\psi })=-\widehat{K}{R}_{{\beta }_1{\beta }_2}({\widehat{\theta }}_{\psi })$, where $R_{{\beta }_1{\beta }_2}({\widehat{\theta }}_{\psi })={\frac{{\partial }^{2}R(\theta )}{\partial {\beta }_1\partial {\beta }_2}|}_{\theta ={\widehat{\theta }}_{\psi }}$.
• ${\tilde{j}}_{{\beta }_2{\beta }_2}({\widehat{\theta }}_{\psi })=\frac{m}{{\tilde{\beta }}_2^2}+({\tilde{\alpha }}_2-1){\displaystyle \sum _{j=1}^m\frac{y_j^2{e}^{-{\tilde{\beta }}_2{y}_j}}{{(1-e^{-{\tilde{\beta }}_2{y}_j})}^2}}-\widehat{K}{R}_{{\beta }_2{\beta }_2}({\widehat{\theta }}_{\psi })$, where $R_{{\beta }_2{\beta }_2}({\widehat{\theta }}_{\psi })={\frac{{\partial }^{2}R(\theta )}{\partial {\beta }_2^2}|}_{\theta ={\widehat{\theta }}_{\psi }}$.

Thus r(ψ) can be obtained accordingly.

For the problem we are considering, the natural choice of the pivotal quantity is

$$\begin{array}{l}z=(z_1,\dots ,z_n,z_n{}_{+1},\dots ,z_n{}_{+m})\prime \\ =(\log F(x_1;{\alpha }_1,\beta ),\dots ,\log F(x_n;{\alpha }_1,\beta ),\log F(y_1;{\alpha }_2,\beta ),\dots ,\log F(y_m;{\alpha }_2,\beta ))\prime .\end{array}$$

Hence the ancillary direction V is

$$\begin{array}{l}V=(V_1,V_2,V_3,V_4)\\ =\left(\begin{array}{cccc}-\log (1-e^{-{\widehat{\beta }}_1{x}_1})\frac{1-e^{-{\widehat{\beta }}_1{x}_1}}{{\widehat{\alpha }}_1{\widehat{\beta }}_1{e}^{-{\widehat{\beta }}_1{x}_1}}& -\frac{x_1}{{\widehat{\beta }}_1}& 0& 0\\ ⋮& ⋮& ⋮& \\ -\log (1-e^{-{\widehat{\beta }}_1{x}_n})\frac{1-e^{-{\widehat{\beta }}_1{x}_n}}{{\widehat{\alpha }}_1{\widehat{\beta }}_1{e}^{-{\widehat{\beta }}_1{x}_n}}& -\frac{x_n}{{\widehat{\beta }}_1}& 0& 0\\ 0& 0& -\log (1-e^{-{\widehat{\beta }}_2{y}_1})\frac{1-e^{-{\widehat{\beta }}_2{y}_1}}{{\widehat{\alpha }}_2{\widehat{\beta }}_2{e}^{-{\widehat{\beta }}_2{y}_1}}& -\frac{y_1}{{\widehat{\beta }}_1}\\ ⋮& ⋮& ⋮& ⋮\\ 0& 0& -\log (1-e^{-{\widehat{\beta }}_2{y}_m})\frac{1-e^{-{\widehat{\beta }}_2{y}_m}}{{\widehat{\alpha }}_2{\widehat{\beta }}_2{e}^{-{\widehat{\beta }}_2{y}_m}}& -\frac{y_m}{{\widehat{\beta }}_2}\end{array}\right)\end{array}$$

Then we can calculate the locally defined canonical parameter φ(θ) as

$$\phi (\theta )={\left({\displaystyle \sum _{i=1}^{n+m}\frac{\partial l(\theta )}{\partial w_i}}{V}_{1i},{\displaystyle \sum _{i=1}^{n+m}\frac{\partial l(\theta )}{\partial w_i}}{V}_{2i},{\displaystyle \sum _{i=1}^{n+m}\frac{\partial l(\theta )}{\partial w_i}}{V}_{3i},{\displaystyle \sum _{i=1}^{n+m}\frac{\partial l(\theta )}{\partial w_i}}{V}_{4i}\right)}^{\prime }$$

where w = (x₁,…, x_n, y₁,…, y_m)′ be the observed data. Hence, we also have φ_θ(θ). Therefore, for this unequal scale parameter case, χ(θ), $\widehat{\mathit{\upsilon ar}}\left(\chi (\widehat{\theta })-\chi ({\widehat{\theta }}_{\psi })\right)$, Q(ψ) and r^*(ψ) can be obtained accordingly. Hence, the (1 − γ)100% confidence interval can be obtained from the modified signed log-likelihood ratio statistics.

4.2. Numerical Examples

To illustrate the proposed third-order method for interval estimation, the following two data sets with sample size of 11 and 9 were used: x = (2.1828, 0.5911, 1.0711, 0.9007, 1.7814, 1.3616, 0.8629, 0.2301, 1.5183, 0.8481, 1.0845) and y = (0.8874, 1.1482, 0.8227, 0.4086, 0.5596, 1.1978, 1.1324, 0.5625, 1.0679). By assumption, X and Y are independently distributed as EE(α₁, β₁) and EE(α₂, β₂) distributions respectively. We are interested in testing H₀ : R = 0.5 vs. H₁ : R > 0.5, where R = P (Y < X) is given in (2).

Table 3. Interval Estimates of ψ for Example.

	90% Confidence Interval		95% Confidence Interval
MLE	(0.4223,	0.8179)	(0.3843,	0.8557)
r	(0.4151,	0.7966)	(0.3767,	0.8241)
Proposed	(0.4080,	0.7910)	(0.3698,	0.8188)

present the 90% and 95% confidence intervals (CI) for R based on the maximum likelihood method (MLE), the signed log-likelihood ratio statistic method (r) and the proposed third-order method (Proposed). In examining Table 3, we observe that all three methods give different interval estimates.

When the scale parameters is the same, the value of R can be expressed in closed form as $\frac{{\alpha }_1}{{\alpha }_1+{\alpha }_2}$. This simplifies the analysis. Following the same procedures introduced in Section 3, $\widehat{\mathit{\upsilon ar}}\left(\chi (\widehat{\theta })-\chi ({\widehat{\theta }}_{\psi })\right)$, Q(ψ) and r^*(ψ) can be obtained.

To compare the accuracy of the three methods discussed in this paper, Monte Carlo simulation studies were conducted. The cases of unequal and equal scale parameters are both examined. For each parameter configuration and for each sample size, we generate 10,000 random samples from the EE distributions by using the following transformation:

$$T=-\frac{1}{\beta }\log (1-U^{1/\alpha })$$

where U is a uniform variate between 0 and 1.

For each simulated setting, we report the proportion of ψ that fall outside the lower bound of the confidence interval (lower error), the proportion of ψ that fall outside the upper bound of the confidence interval (upper error), the proportion of ψ that fall within the confidence interval (central coverage), and the average bias (Average Bias), which is defined as

$$\text{Average Bias}=\frac{|\text{lower error}-0.025|+|\text{upper error}-0.025|}{2}$$

The nominal values for the lower and the upper errors, the central coverage and the average bias are 0.025, 0.025, 0.95 and 0 respectively. These values reflect the desired properties of the accuracy and symmetry of the interval estimates of ψ.

Table 4. β₁ ≠ β₂, α₁ = 2, α₂ = 5, β₁ = 3 and β₂ is obtained by Equation (2), (n, m) = (10, 10).

R	Method	Lower Error	Upper Error	Central Coverage	Average Bias
	MLE	0.1602	0.0033	0.8365	0.07845
0.1	r	0.0401	0.0177	0.9422	0.01120
	Proposed	0.0207	0.0252	0.9541	0.00255
	MLE	0.1138	0.0125	0.8737	0.05065
0.2	r	0.0388	0.0235	0.9377	0.00765
	Proposed	0.0218	0.0258	0.9524	0.00200
	MLE	0.0857	0.0225	0.8918	0.03160
0.3	r	0.0372	0.0262	0.9366	0.00670
	Proposed	0.0230	0.0259	0.9527	0.00135
	MLE	0.0656	0.0362	0.8982	0.02590
0.4	r	0.0352	0.0294	0.9354	0.00730
	Proposed	0.0244	0.0259	0.9497	0.00075
	MLE	0.0505	0.0506	0.8989	0.02555
0.5	r	0.0317	0.0328	0.9355	0.00725
	Proposed	0.0249	0.0255	0.9496	0.00030
	MLE	0.0353	0.0670	0.8977	0.02615
0.6	r	0.0290	0.0359	0.9351	0.00745
	Proposed	0.0244	0.0246	0.9510	0.00050
	MLE	0.0235	0.0900	0.8865	0.03325
0.7	r	0.0257	0.0394	0.9349	0.00755
	Proposed	0.0246	0.0238	0.9516	0.00080
	MLE	0.0142	0.1234	0.8624	0.05460
0.8	r	0.0239	0.0419	0.9342	0.00900
	Proposed	0.0261	0.0239	0.9500	0.00110
	MLE	0.0035	0.1763	0.8202	0.08640
0.9	r	0.0198	0.0465	0.9337	0.01335
	Proposed	0.0262	0.0240	0.9498	0.00110

Table 5. β₁ ≠ β₂, α₁ = 2, α₂ = 5, β₁ = 3 and β₂ is obtained by Equation (2), (n, m) = (10, 50).

R	Method	Lower Error	Upper Error	Central Coverage	Average Bias
	MLE	0.1341	0.0046	0.8613	0.06475
0.1	r	0.0399	0.0186	0.9415	0.01065
	Proposed	0.0231	0.0246	0.9523	0.00115
	MLE	0.1007	0.0131	0.8862	0.04380
0.2	r	0.0370	0.0243	0.9387	0.00635
	Proposed	0.0239	0.0251	0.9510	0.00060
	MLE	0.0774	0.0249	0.8977	0.02625
0.3	r	0.0349	0.0289	0.9362	0.00690
	Proposed	0.0228	0.0270	0.9502	0.00210
	MLE	0.0615	0.0353	0.9032	0.02340
0.4	r	0.0327	0.0311	0.9362	0.00690
	Proposed	0.0220	0.0260	0.9520	0.00200
	MLE	0.0475	0.0488	0.9037	0.02315
0.5	r	0.0294	0.0323	0.9383	0.00585
	Proposed	0.0229	0.0240	0.9531	0.00155
	MLE	0.0351	0.0682	0.8967	0.02665
0.6	r	0.0278	0.0348	0.9374	0.00630
	Proposed	0.0222	0.0222	0.9556	0.00280
	MLE	0.0225	0.0962	0.8813	0.03685
0.7	r	0.0256	0.0388	0.9356	0.00720
	Proposed	0.0226	0.0227	0.9547	0.00235
	MLE	0.0126	0.1249	0.8625	0.05615
0.8	r	0.0217	0.0431	0.9352	0.01070
	Proposed	0.0224	0.0242	0.9534	0.00170
	MLE	0.0063	0.1779	0.8158	0.08580
0.9	r	0.0168	0.0473	0.9359	0.01525
	Proposed	0.0215	0.0238	0.9547	0.00235

–

Table 6. β₁ ≠ β₂, α₁ = 2, α₂ = 5, β₁ = 3 and β₂ is obtained by Equation (2), (n, m) = (50, 10).

R	Method	Lower Error	Upper Error	Central Coverage	Average Bias
	MLE	0.1257	0.0046	0.8697	0.06055
0.1	r	0.0392	0.0167	0.944115	0.01125
	Proposed	0.0207	0.0203	0.9590	0.00450
	MLE	0.0885	0.0154	0.8961	0.03655
0.2	r	0.0347	0.0223	0.9430	0.00620
	Proposed	0.0226	0.0226	0.9548	0.00240
	MLE	0.0657	0.0234	0.9109	0.02115
0.3	r	0.0332	0.0252	0.9416	0.00420
	Proposed	0.0221	0.0233	0.9546	0.002130
	MLE	0.0497	0.0335	0.9168	0.01660
0.4	r	0.0317	0.0286	0.9397	0.00515
	Proposed	0.0228	0.0241	0.9531	0.00155
	MLE	0.0368	0.0428	0.9204	0.01480
0.5	r	0.0285	0.0309	0.9405	0.00475
	Proposed	0.0222	0.0236	0.9542	0.00210
	MLE	0.0264	0.0499	0.9237	0.01315
0.6	r	0.0248	0.034830	0.9422	0.00410
	Proposed	0.0215	0.0231	0.9554	0.00270
	MLE	0.0184	0.0595	0.9221	0.02055
0.7	r	0.0222	0.0332	0.9446	0.00550
	Proposed	0.0206	0.0235	0.9559	0.00295
	MLE	0.0112	0.0715	0.9173	0.03015
0.8	r	0.0186	0.0324	0.9490	0.00690
	Proposed	0.0196	0.0212	0.9592	0.00460
	MLE	0.0055	0.0927	0.9018	0.04360
0.9	r	0.0149	0.0291	0.9560	0.00710
	Proposed	0.0183	0.0209	0.9608	0.00540

present simulation results for the unequal scale parameters case, i.e., α₁ = 2, α₂ = 5, β₁ = 3, and R = 0.1(0.1)0.9 with (n, m) = (10, 10), (10, 50) and (50, 10). Note that, we fixed R and β₂ is determined uniquely by Equation (2). It is clear that the coverage probabilities for R are poor and the two-tail error probabilities are extremely asymmetric from the MLE method. Results from the signed log-likelihood method are not satisfactory especially when the two sample sizes are small or sample sizes are unequal, and it also shows some evidence of asymmetry of two-tail error probabilities. However, the proposed method gives not only an almost exact coverage probability but also it has symmetric two-tail error probabilities even for small or uneven sample sizes.

Note that if we had used the built-in macros / subroutines from Matlab or R, our simulation study could not have been completed because of difficulties in obtaining the constrained maximum likelihood estimates. Fortunately, we did not encounter any difficulties when the penalized likelihood method was used.

Table 7. α₁ = 4, β = 8 and α₂ satisfies R = α₁/(α₁ + α₂), (n, m) = (10, 10).

R	Method	Lower Error	Upper Error	Central Coverage	Average Bias
	MLE	0.1608	0.0024	0.8368	0.07920
0.1	r	0.0526	0.0189	0.9285	0.01685
	Proposed	0.0236	0.0267	0.9497	0.00155
	MLE	0.1195	0.0121	0.8684	0.05370
0.2	r	0.0502	0.0236	0.9262	0.01330
	Proposed	0.0269	0.0264	0.9467	0.00165
	MLE	0.0944	0.0224	0.8832	0.03600
0.3	r	0.0428	0.0256	0.9316	0.00920
	Proposed	0.0225	0.0234	0.9541	0.00205
	MLE	0.0724	0.0347	0.8929	0.02855
0.4	r	0.0387	0.0277	0.9336	0.00820
	Proposed	0.0256	0.0224	0.9520	0.00160
	MLE	0.0523	0.0517	0.8960	0.02700
0.5	r	0.0335	0.0328	0.9337	0.00815
	Proposed	0.0244	0.0230	0.9526	0.00013
	MLE	0.0378	0.0753	0.8869	0.03155
0.6	r	0.0295	0.0400	0.9305	0.00975
	Proposed	0.0234	0.0260	0.9506	0.00130
	MLE	0.0245	0.0944	0.8811	0.03495
0.7	r	0.0282	0.0454	0.9264	0.01180
	Proposed	0.0261	0.0262	0.9477	0.00115
	MLE	0.0109	0.1187	0.8704	0.05390
0.8	r	0.0226	0.0467	0.9307	0.01205
	Proposed	0.0239	0.0262	0.9499	0.00115
	MLE	0.0027	0.1464	0.8509	0.07185
0.9	r	0.0211	0.0499	0.9290	0.01440
	Proposed	0.0268	0.0252	0.9480	0.00100

Table 8. α₁ = 4, β = 8 and α₂ satisfies R = α₁/(α₁ + α₂), (n, m) = (10, 50).

R	Method	Lower Error	Upper Error	Central Coverage	Average Bias
	MLE	0.0722	0.0093	0.9185	0.03145
0.1	r	0.0302	0.0273	0.9425	0.00375
	Proposed	0.0250	0.0261	0.9489	0.00055
	MLE	0.0577	0.0206	0.9217	0.01855
0.2	r	0.0287	0.0268	0.9445	0.00275
	Proposed	0.0260	0.0244	0.9496	0.00080
	MLE	0.0431	0.0327	0.9242	0.01290
0.3	r	0.0276	0.0306	0.9418	0.00410
	Proposed	0.0261	0.0257	0.9482	0.00090
	MLE	0.0299	0.0412	0.9289	0.01055
0.4	r	0.0216	0.0290	0.9494	0.00370
	Proposed	0.0218	0.0227	0.9555	0.00275
	MLE	0.0475	0.0488	0.9037	0.02315
0.5	r	0.0234	0.0358	0.9408	0.00620
	Proposed	0.0243	0.0269	0.9488	0.00130
	MLE	0.0166	0.0688	0.9146	0.02610
0.6	r	0.0221	0.0350	0.9429	0.00645
	Proposed	0.0249	0.0254	0.9497	0.00250
	MLE	0.0105	0.0789	0.9106	0.03420
0.7	r	0.0206	0.0366	0.9428	0.00800
	Proposed	0.0276	0.0229	0.9495	0.00235
	MLE	0.0059	0.1021	0.8920	0.04810
0.8	r	0.0195	0.0426	0.9379	0.01155
	Proposed	0.0259	0.0269	0.9472	0.00140
	MLE	0.0016	0.1210	0.8774	0.05970
0.9	r	0.0160	0.0456	0.9384	0.01480
	Proposed	0.0239	0.0249	0.9512	0.00060

–

Table 9. α₁ = 4, β = 8 and α₂ satisfies R = α₁/(α₁ + α₂), (n, m) = (50, 10).

R	Method	Lower Error	Upper Error	Central Coverage	Average Bias
	MLE	0.1120	0.0008	0.8872	0.05560
0.1	r	0.0412	0.0161	0.9427	0.01255
	Proposed	0.0230	0.0247	0.9523	0.00115
	MLE	0.1011	0.0048	0.8941	0.04815
0.2	r	0.0407	0.0202	0.9319	0.01025
	Proposed	0.0261	0.0260	0.9479	0.00105
	MLE	0.0805	0.0108	0.9087	0.03485
0.3	r	0.0366	0.0232	0.9402	0.00670
	Proposed	0.0248	0.0275	0.9477	0.00135
	MLE	0.0648	0.0140	0.9212	0.02540
0.4	r	0.0338	0.0215	0.9447	0.00615
	Proposed	0.0238	0.0247	0.9515	0.00075
	MLE	0.0551	0.0258	0.9191	0.01545
0.5	r	0.0317	0.0256	0.9427	0.00365
	Proposed	0.0242	0.0276	0.9482	0.00170
	MLE	0.0437	0.0296	0.9267	0.01165
0.6	r	0.0329	0.0228	0.9443	0.00505
	Proposed	0.0247	0.0229	0.9524	0.00120
	MLE	0.0332	0.0400	0.9268	0.01160
0.7	r	0.0307	0.0247	0.9446	0.00300
	Proposed	0.0257	0.0241	0.9502	0.00080
	MLE	0.0228	0.0548	0.9224	0.01600
0.8	r	0.0294	0.0266	0.9440	0.00300
	Proposed	0.0256	0.0242	0.9502	0.00070
	MLE	0.0098	0.0670	0.9232	0.02860
0.9	r	0.0247	0.0299	0.9454	0.00260
	Proposed	0.0234	0.0261	0.9505	0.00135

present simulation results for the equal scale parameter case, i.e., α₁ = 4, β = 8, and R = 0.1(0.1)0.9 with (n, m) = (10, 10), (10, 50) and (50, 10). In estimation, we fixed R and α₂ is then determined uniquely by R = α₁/(α₁ + α₂). Again, the proposed method outperformed the other two methods even when the sample sizes were small. Other simulation results, though not reported here, essentially corroborate those of Tables 7–9, and are available upon request. In the equal scale parameter case, when sample sizes were small, or when R was close to the boundary values, we used the penalized likelihood method to obtain the constrained maximum likelihood estimates. More simulations have been performed with the same pattern of results. They are not reported here, but are available on request.

5. Conclusions

A penalized likelihood method is proposed to overcome numerical difficulties that arose in using the Lagrange method to solve an optimization problem with an integral constraint. The proposed method is used to obtain constrained maximum likelihood estimates for the stress-strength reliability with independent exponentiated exponential distributions. In turn, these estimates are used to obtain inference for the stress-strength reliability via a likelihood-based third order asymptotic method. In our simulation studies, the penalized likelihood method did not encounter any numerical difficulties in obtaining the constrained maximum likelihood estimates. Moreover, the likelihood-based third order asymptotic method gives very accurate results even when the sample sizes are small. Although the paper is restricted to independent exponentiated exponential distributions, it can easily be extended to other distributions.