Software Reliability Model with Dependent Failures and SPRT

Abstract

Software reliability and quality are crucial in several fields. Related studies have focused on software reliability growth models (SRGMs). Herein, we propose a new SRGM that assumes interdependent software failures. We conduct experiments on real-world datasets to compare the goodness-of-fit of the proposed model with the results of previous nonhomogeneous Poisson process SRGMs using several evaluation criteria. In addition, we determine software reliability using Wald’s sequential probability ratio test (SPRT), which is more efficient than the classical hypothesis test (the latter requires substantially more data and time because the test is performed only after data collection is completed). The experimental results demonstrate the superiority of the proposed model and the effectiveness of the SPRT.

1. Introduction

Software performance and reliability are essential in various fields, such as the Internet of Things. If software reliability is not guaranteed, economic and human losses may occur. Hence, various studies have been conducted to improve software reliability and prevent loss.

Software reliability growth models (SRGMs) are tools that estimate the quality and reliability of software products. SRGMs not only provide information regarding software reliability for developers and consumers, but also help establish an optimal release policy.

Most SRGMs assume that the mean value function, m(t), is a nonhomogeneous Poisson process (NHPP). The m(t) of each model is a single function that reflects the failure intensity, failure detection, number of remaining failures, and the environment. In particular, assumptions regarding the test and development environments are essential; furthermore, the parameters and the form of $\mathrm{m}\left(\mathrm{t}\right)$ depend on the assumptions regarding the environment.

Previous studies have considered the types of fault detection functions and the shape of m(t). The Goel–Okumoto (GO) model [1] is a basic stochastic NHPP SRGM that has inspired several researchers. Ohba et al. [2] proposed an inflection S-shaped curve model. Yamada [3] developed a similar delayed model. Pham et al. [4] reported a nondecreasing inflection S-shaped model with an error introduction rate as an exponential function; the model is known as the Pham–Zhang (PZ) model. Moreover, Pham et al. [5] proposed a generalized NHPP software model that reflected the testing coverage. Subsequently, researchers became interested in environmental factors. In particular, many studies have focused on uncertain operating or testing environments [6,7,8,9,10,11,12,13]. Some recent research has been studied on SRGMs with considerations of time-dependent fault detection [14,15,16,17,18,19], random operating environments [20,21], and multi release software [22,23,24].

Recent studies combined deep learning and machine learning with software reliability [25,26,27,28]. Wang et al. discussed an optimal release policy and the selection of the best software reliability model [28]. Lee et al. proposed an SRGM to consider the actual test time instead of the designed test time [29]. Minamino et al. [30] discussed software reliability and release policies that considered the change-points of data based on the theory of multiproperty utilities. Rani et al. [31] developed a hazard rate model that introduced an imperfect debugging parameter and a single change-point parameter.

Some studies suggest statistical techniques to predict software reliability. Typically, parameters of reliability models are estimated using the least-square estimator (LSE) method. Additionally, the maximum likelihood estimator (MLE) and Bayesian methods have been used [32,33]. However, software reliability models have complex structures and numerous parameters, rendering it difficult to apply the MLE and Bayesian methods; hence, we used the LSE to estimate the parameters in this study.

We applied the sequential probability ratio test (SPRT), a statistical test technique, to efficiently determine software reliability. The SPRT was designed for military and naval equipment development by Wald [34]. The main advantage of a sequential test is that it requires a shorter test time compared with that required by the classical test, which requires a fixed sample size. In addition, the SPRT can instantly determine software reliability whenever new faults occur. Hence, reliability can be assessed based on the change-points of data. This procedure is described in Section 2.

Software reliability studies using Wald’s SPRT procedure have been conducted continually. Stieber applied the SPRT to an NHPP SRGM for the first time [35]. This method has been applied to various SRGMs in many studies [36,37,38,39,40]. Furthermore, the authors of [38] performed the SPRT using order statistics.

Herein, we propose a new SRGM that assumes interdependent software failures. In Section 2, we discuss the efficiency of the SPRT. The proposed model is described in Section 3. In Section 4, we introduce the criteria and data used in the experiments. Subsequently, we discuss the results in Section 5. Finally, Section 6 concludes the paper.

2. Wald’s SPRT

Wald described the efficiency of the SPRT [34]. The probability ratio ${\mathrm{p}}_1/{\mathrm{p}}_0$ is used as a test statistic of the SPRT for testing the null hypothesis against an alternative hypothesis. The test is continued (until the next decision) if the following condition is satisfied:

$$\mathrm{B}<\frac{{\mathrm{p}}_1}{{\mathrm{p}}_0}<\mathrm{A},$$

(1)

where $\mathrm{A}$ and $\mathrm{B}$ are constants used to decide the acceptance and rejection of the null hypothesis ${\mathrm{H}}_0$, respectively. If ${\mathrm{p}}_1/{\mathrm{p}}_0\ge \mathrm{A}$, then ${\mathrm{H}}_0$ is rejected. If ${\mathrm{p}}_1/{\mathrm{p}}_0\le \mathrm{B}$, then ${\mathrm{H}}_0$ is accepted.

Moreover, A and B depend on α and β, as shown in Equations (2)–(5). Here, α and β are type-1 and -2 errors, respectively. In other words, α is the producer’s risk, whereas β is the consumer’s risk.

$$1-\mathsf{\beta }\ge \mathrm{A}\mathsf{\alpha }$$

(2)

$$\mathrm{A}\le \frac{1-\mathsf{\beta }}{\mathsf{\alpha }}$$

(3)

$$\mathsf{\beta }\le \left(1-\mathsf{\alpha }\right)\mathrm{B}$$

(4)

$$\mathrm{B}\ge \frac{\mathsf{\beta }}{1-\mathsf{\alpha }}$$

(5)

The SPRT can be expressed visually. Figure 1 shows the method to assess software reliability using the SPRT.

In Figure 1, ${\mathrm{N}}_{\mathrm{L}}\left(\mathrm{t}\right)$ and ${\mathrm{N}}_{\mathrm{U}}\left(\mathrm{t}\right)$ denote the lower and upper limits of the area in which the software is reliable, respectively. Here, $\mathrm{N}\left(\mathrm{t}\right)$ is the expected number of failures at time $\mathrm{t}$ for the NHPP. If $\mathrm{N}\left(\mathrm{t}\right)$ is in the reliable region, then the software is reliable.

$${\mathrm{N}}_{\mathrm{L}}\left(\mathrm{t}\right)=\mathrm{at}-{\mathrm{b}}_1,{\mathrm{N}}_{\mathrm{U}}\left(\mathrm{t}\right)=\mathrm{at}+{\mathrm{b}}_2,$$

(6)

where $\mathrm{a},{\mathrm{b}}_1$, and ${\mathrm{b}}_2$ are expressed as:

$$\mathrm{a}=\frac{{\mathsf{\lambda }}_1-{\mathsf{\lambda }}_0}{\ln \left(\frac{{\mathsf{\lambda }}_1}{{\mathsf{\lambda }}_0}\right)},$$

(7)

$${\mathrm{b}}_1=\frac{\ln \left(\frac{1-\mathsf{\alpha }}{\mathsf{\beta }}\right)}{\ln \left(\frac{{\mathsf{\lambda }}_1}{{\mathsf{\lambda }}_0}\right)},{\mathrm{b}}_2=\frac{\ln \left(\frac{1-\mathsf{\beta }}{\mathsf{\alpha }}\right)}{\ln \left(\frac{{\mathsf{\lambda }}_1}{{\mathsf{\lambda }}_0}\right)}.$$

(8)

Stieber applied the SPRT to an NHPP SRGM [35] with ${\mathsf{\lambda }}_0=\frac{\mathsf{\lambda }\ln \left(\mathrm{q}\right)}{\mathrm{q}-1}$ and ${\mathsf{\lambda }}_1=\mathrm{q}\frac{\mathsf{\lambda }\ln \left(\mathrm{q}\right)}{\mathrm{q}-1}$, where $\mathrm{q}={\mathsf{\lambda }}_1/{\mathsf{\lambda }}_0$. A general NHPP SRGM is expressed as:

$$\mathrm{Pr}\left\{\mathrm{N}\left(\mathrm{t}\right)=\mathrm{n}\right\}=\frac{{\left[\mathrm{m}\left(\mathrm{t}\right)\right]}^{\mathrm{n}}}{\mathrm{n}!}{\mathrm{e}}^{-\mathrm{m}\left(\mathrm{t}\right)},\mathrm{n}=0,1,2,\dots .$$

(9)

where m(t) indicates the expected number of detected failures until time t, which is described as:

$$\mathrm{m}\left(\mathrm{t}\right)={\int }_0^{\mathrm{t}}\mathsf{\lambda }\left(\mathrm{s}\right)\mathrm{ds}.$$

(10)

Stieber defined the probability ratio, which is used as a statistic of the SPRT for NHPP SRGMs, as in Equation (1).

$${\mathrm{p}}_0=\frac{{\mathrm{e}}^{-{\mathrm{m}}_0\left(\mathrm{t}\right)}{\left[{\mathrm{m}}_0\left(\mathrm{t}\right)\right]}^{\mathrm{N}\left(\mathrm{t}\right)}}{\mathrm{N}\left(\mathrm{t}\right)!},{\mathrm{p}}_1=\frac{{\mathrm{e}}^{-{\mathrm{m}}_1\left(\mathrm{t}\right)}{\left[{\mathrm{m}}_1\left(\mathrm{t}\right)\right]}^{\mathrm{N}\left(\mathrm{t}\right)}}{\mathrm{N}\left(\mathrm{t}\right)!}.$$

(11)

Using Equations (3), (5) and (11), we can rewrite Equation (1) as follows:

$$\frac{\ln \left(\frac{\mathsf{\beta }}{1-\mathsf{\alpha }}\right)+{\mathrm{m}}_1\left(\mathrm{t}\right)-{\mathrm{m}}_0\left(\mathrm{t}\right)}{\ln {\mathrm{m}}_1\left(\mathrm{t}\right)-\ln {\mathrm{m}}_0\left(\mathrm{t}\right)}<\mathrm{N}\left(\mathrm{t}\right)<\frac{\ln \left(\frac{1-\mathsf{\beta }}{\mathsf{\alpha }}\right)+{\mathrm{m}}_1\left(\mathrm{t}\right)-{\mathrm{m}}_0\left(\mathrm{t}\right)}{\ln {\mathrm{m}}_1\left(\mathrm{t}\right)-\ln {\mathrm{m}}_0\left(\mathrm{t}\right)}.$$

(12)

In Equation (12), the left term is constant B, and the right term is constant A. Moreover, $\mathrm{N}\left(\mathrm{t}\right)$ refers to the probability ratio, ${\mathrm{p}}_1/{\mathrm{p}}_0$.

3. New SRGM

A general SRGM that follows the NHPP, as shown in Equation (9), can be obtained by solving the differential equation shown in Equation (13). Function $\mathrm{b}\left(\mathrm{t}\right)$ varies according to each assumption as follows:

$$\frac{\mathrm{dm}\left(\mathrm{t}\right)}{\mathrm{dt}}=\left[\mathrm{b}\left(\mathrm{t}\right)\right]\left[\mathrm{N}-\mathrm{m}\left(\mathrm{t}\right)\right],$$

(13)

where $\mathrm{b}\left(\mathrm{t}\right)$ is the fault detection rate function. The model assumes independent failures.

However, the proposed model is different. For example, if a fault occurs because of a syntax error and a developer cannot fix it completely, new faults will be affected. Consequently, failures may depend on one another (Figure 2). Hence, we assume that the failures are dependent. The SRGM based on these assumptions can be obtained from the differential equation expressed in Equation (14).

$$\frac{\mathrm{dm}\left(\mathrm{t}\right)}{\mathrm{dt}}=\left[\mathrm{b}\left(\mathrm{t}\right)\right]\left[\mathrm{a}\left(\mathrm{t}\right)-\mathrm{m}\left(\mathrm{t}\right)\right]\mathrm{m}\left(\mathrm{t}\right),$$

(14)

where $\mathrm{a}\left(\mathrm{t}\right)$ and $\mathrm{b}\left(\mathrm{t}\right)$ are the functions of the total failure content rate and failure detection rate, respectively. In the proposed model, $\mathrm{a}\left(\mathrm{t}\right)$and $\mathrm{b}\left(\mathrm{t}\right)$ are expressed as:

$$\mathrm{a}\left(\mathrm{t}\right)=\mathrm{a},\mathrm{b}\left(\mathrm{t}\right)=\frac{\mathrm{b}}{1+{\mathrm{ce}}^{-\mathrm{bt}}}.$$

(15)

Therefore, we can calculate the $\mathrm{m}\left(\mathrm{t}\right)$ of the proposed model. The initial value is $\mathrm{m}\left(0\right)=\mathrm{h}$.

$$\mathrm{m}\left(\mathrm{t}\right)=\frac{\mathrm{a}}{1+\frac{\mathrm{a}}{\mathrm{h}}{\left(\frac{\mathrm{b}+\mathrm{c}}{\mathrm{c}+{\mathrm{be}}^{\mathrm{bt}}}\right)}^{\frac{\mathrm{a}}{\mathrm{b}}}}$$

(16)

Table 1. Mean value functions of software reliability growth models (SRGMs).

No.	Model	$m\left(t\right)$
1	Goel Okumoto (GO) [1]	$\mathrm{m}\left(\mathrm{t}\right)=\mathrm{a}\left(1-{\mathrm{e}}^{-\mathrm{bt}}\right)$
2	Delayed S-shaped (DS) [3]	$\mathrm{m}\left(\mathrm{t}\right)=\mathrm{a}\left(1-\left(1+\mathrm{bt}\right){\mathrm{e}}^{-\mathrm{bt}}\right)$
3	Inflection S-shaped (IS) [2]	$\mathrm{m}\left(\mathrm{t}\right)=\frac{\mathrm{a}\left(1-{\mathrm{e}}^{-\mathrm{bt}}\right)}{1+{\mathsf{\beta }\mathrm{e}}^{-\mathrm{bt}}}$
4	Yamada Imperfect Debugging (YID) [3]	$\mathrm{m}\left(\mathrm{t}\right)=\mathrm{a}\left(1-{\mathrm{e}}^{-\mathrm{bt}}\right)\left(1-\frac{\mathsf{\alpha }}{\mathrm{b}}\right)+\mathsf{\alpha }\mathrm{at}$
5	Pham–Nordmann–Zhang (PNZ) [41]	$\mathrm{m}\left(\mathrm{t}\right)=\frac{\mathrm{a}\left(1-{\mathrm{e}}^{-\mathrm{bt}}\right)\left[1-\frac{\mathsf{\alpha }}{\mathrm{b}}\right]+\mathsf{\alpha }\mathrm{at}}{1+{\mathsf{\beta }\mathrm{e}}^{-\mathrm{bt}}}$
6	Pham–Zhang (PZ) [4]	$\mathrm{m}\left(\mathrm{t}\right)=\frac{\left(\left(\mathrm{c}+\mathrm{a}\right)\left[1-{\mathrm{e}}^{-\mathrm{bt}}\right]-\left[\frac{\mathrm{ab}}{\mathrm{b}-\mathsf{\alpha }}\right]\left({\mathrm{e}}^{-\mathrm{at}}-{\mathrm{e}}^{-\mathrm{bt}}\right)\right)}{1+{\mathsf{\beta }\mathrm{e}}^{-\mathrm{bt}}}$
7	Testing Coverage (TC) [9]	$\mathrm{m}\left(\mathrm{t}\right)=\mathrm{N}\left[1-{\left(\frac{\mathsf{\beta }}{\mathsf{\beta }+{\left(\mathrm{at}\right)}^{\mathrm{b}}}\right)}^{\mathsf{\alpha }}\right]$
8	New model	$\mathrm{m}\left(\mathrm{t}\right)=\frac{\mathrm{a}}{1+\left(\frac{\mathrm{a}}{\mathrm{h}}\right){\left(\frac{\mathrm{b}+\mathrm{c}}{\mathrm{c}+{\mathrm{be}}^{\mathrm{bt}}}\right)}^{\frac{\mathrm{a}}{\mathrm{b}}}}$

summarizes the m(t) for existing NHPP SRGMs.

4. Experiments

4.1. Criteria

In this section, we elucidate the use of eight criteria to compare the NHPP SRGMs for two real-world datasets.

(1) The mean squared error (MSE) [42] measures the distance between the estimated and actual data, considering the number of observations and the number of parameters in the models. The MSE is defined as follows:

$$\mathrm{MSE}=\frac{{\sum }_{\mathrm{i}=1}^{\mathrm{n}}{\left(\hat{\mathrm{m}}\left({\mathrm{t}}_{\mathrm{i}}\right)-{\mathrm{y}}_{\mathrm{i}}\right)}^2}{\mathrm{n}-\mathrm{N}},$$

where $\mathrm{n}$ is the number of total cumulative failures, N the number of model parameters, and ${\mathrm{y}}_{\mathrm{i}}$ the number of cumulative failures at ${\mathrm{t}}_{\mathrm{i}}$, obtained from the dataset.

(2) The predictive ratio risk (PRR) [42] indicates the distance of model estimates from actual data with respect to the model estimates. It is calculated as:

$$\mathrm{PRR}={\displaystyle \sum }_{\mathrm{i}=1}^{\mathrm{n}}{\left(\frac{\hat{\mathrm{m}}\left({\mathrm{t}}_{\mathrm{i}}\right)-{\mathrm{y}}_{\mathrm{i}}}{\hat{\mathrm{m}}\left({\mathrm{t}}_{\mathrm{i}}\right)}\right)}^2.$$

(3) The predictive power (PP) [43] measures the distance of actual data from the estimate with regard to the actual data. It is defined as:

$$\mathrm{PP}={\displaystyle \sum }_{\mathrm{i}=1}^{\mathrm{n}}{\left(\frac{\hat{\mathrm{m}}\left({\mathrm{t}}_{\mathrm{i}}\right)-{\mathrm{y}}_{\mathrm{i}}}{{\mathrm{y}}_{\mathrm{i}}}\right)}^2.$$

(4) R-square (${\mathrm{R}}^2$) [44] is the correlation index of the regression curve equation. It is used to explain the fitting power of the SRGMs and is described as follows:

$${\mathrm{R}}^2=1-\frac{{\sum }_{\mathrm{i}=1}^{\mathrm{n}}{\left({\mathrm{y}}_{\mathrm{i}}-\hat{\mathrm{m}}\left({\mathrm{t}}_{\mathrm{i}}\right)\right)}^2}{{\sum }_{\mathrm{i}=1}^{\mathrm{n}}{\left({\mathrm{y}}_{\mathrm{i}}-\overline{\mathrm{y}}\right)}^2}.$$

(5) Akaike’s information criterion (AIC) [45] is used to maximize the likelihood function. It can be considered as an approximate distance from the true probability model, as follows:

$$\mathrm{AIC}=-2\mathrm{lnL}+2\mathrm{N},$$

where $\mathrm{N}$ is the degree of freedom. Here, $\mathrm{L}$ and $\mathrm{lnL}$ are written as follows:

$$\mathrm{L}={\displaystyle \prod }_{\mathrm{i}=1}^{\mathrm{n}}\frac{\left(\mathrm{m}({\mathrm{t}}_{\mathrm{i}}\right)-\mathrm{m}\left({\mathrm{t}}_{\mathrm{i}-1}\right){)}^{{\mathrm{y}}_{\mathrm{i}}-{\mathrm{y}}_{\mathrm{i}-1}}}{\left({\mathrm{y}}_{\mathrm{i}}-{\mathrm{y}}_{\mathrm{i}-1}\right)!}{\mathrm{e}}^{-\left(\mathrm{m}\left({\mathrm{t}}_{\mathrm{i}}\right)-\mathrm{m}\left({\mathrm{t}}_{\mathrm{i}-1}\right)\right)},$$

$$\mathrm{lnL}={\displaystyle \sum }_{\mathrm{i}=1}^{\mathrm{n}}\left\{\left({\mathrm{y}}_{\mathrm{i}}-{\mathrm{y}}_{\mathrm{i}-1}\right)\ln \left((\mathrm{m}\left({\mathrm{t}}_{\mathrm{i}}\right)-\mathrm{m}\left({\mathrm{t}}_{\mathrm{i}-1}\right)\right)-\left(\mathrm{m}\left({\mathrm{t}}_{\mathrm{i}}\right)-\mathrm{m}\left({\mathrm{t}}_{\mathrm{i}-1}\right)\right)-\ln \left(\left({\mathrm{y}}_{\mathrm{i}}-{\mathrm{y}}_{\mathrm{i}-1}\right)!\right)\right\}.$$

(6) The sum of absolute error (SAE) [11] measures the distance between the predicted number of failures and the observed data. It is defined as:

$$\mathrm{SAE}={\displaystyle \sum }_{\mathrm{i}=1}^{\mathrm{n}}\left|\hat{\mathrm{m}}\left({\mathrm{t}}_{\mathrm{i}}\right)-{\mathrm{y}}_{\mathrm{i}}\right|.$$

(7) The variation [46] is the standard deviation of the prediction bias. It is expressed as follows:

$$\mathrm{Variation}=\sqrt{\frac{{\sum }_{\mathrm{i}=1}^{\mathrm{n}}\left((\hat{\mathrm{m}}\left({\mathrm{t}}_{\mathrm{i}}\right)-{\mathrm{y}}_{\mathrm{i}}\right)-\mathrm{Bias}{)}^2}{\mathrm{n}-1}},$$

where bias is expressed as:

$$\mathrm{Bias}={\displaystyle \sum }_{\mathrm{i}=1}^{\mathrm{n}}\left[\frac{\hat{\mathrm{m}}\left({\mathrm{t}}_{\mathrm{i}}\right)-{\mathrm{y}}_{\mathrm{i}}}{\mathrm{n}}\right].$$

(8) The root-mean-square prediction error (RMSPE) [46] estimates the closeness of the predicted values with the actual observations.

$$\mathrm{RMSPE}=\sqrt{{\mathrm{Variation}}^2+{\mathrm{Bias}}^2}$$

The closer the value of ${\mathrm{R}}^2$ is to 1, the better the goodness-of-fit of the dataset. For other criteria, a smaller value indicates a better fit.

4.2. Datasets

Table 2. Datasets.

Dataset 1			Dataset 2
Time	Failure	Cum. Failure	Time	Failure	Cum. Failure
1	1	1	1	90	90
2	0	1	2	17	107
3	1	2	3	19	126
4	1	3	4	19	145
5	2	5	5	26	171
6	0	5	6	17	188
7	0	5	7	1	189
8	3	8	8	1	190
9	1	9	9	0	190
10	2	11	10	0	190
11	2	13	11	2	192
12	2	15	12	0	192
13	4	19	13	0	192
14	0	19	14	0	192
15	3	22	15	11	203
16	0	22	16	0	203
17	1	23	17	1	204
18	1	24
19	0	24
20	0	24
21	2	26	□	□	□

shows the two datasets used in the experiments. Dataset 1 was collected from a telecommunication system that manages the radio access of wireless systems on a weekly basis [42,47]. It is a test dataset corresponding to two releases of the software. For dataset 1, the number of cumulative failures is $\left\{1,1,\dots ,26\right\}$ at ${\mathrm{t}}_{\mathrm{i}}=1,2,\dots ,21$, and the number of total cumulative failures is 26. Dataset 2 includes one of the three releases of weekly medical record system test data that correspond to 188 software tools [42,48]. For dataset 2, the number of cumulative failures is $\left\{90,107,\dots ,204\right\}$ at ${\mathrm{t}}_{\mathrm{i}}=1,2,\dots ,17$.

5. Results

5.1. Results of Parameter Estimation and Goodness-of-Fit

Before comparing the fit and applying the SPRT, we estimated the parameters of all models listed in Table 1 using the LSE method.

Table 3. Estimation of parameters for datasets.

Model	Dataset 1	Dataset 2
GO	$\hat{\mathrm{a}}=254045,\hat{\mathrm{b}}=0.000005$	$\hat{\mathrm{a}}=197.387,\hat{\mathrm{b}}=0.399$
DS	$\hat{\mathrm{a}}=39.8212,\hat{\mathrm{b}}=0.11041$	$\hat{\mathrm{a}}=192.528,\hat{\mathrm{b}}=0.882$
IS	$\hat{\mathrm{a}}=26.693,\hat{\mathrm{b}}=0.2919,\hat{\mathsf{\beta }}=21.71$	$\hat{\mathrm{a}}=197.354,\hat{\mathrm{b}}=0.399,\hat{\mathsf{\beta }}=0.000001$
YID	$\hat{\mathrm{a}}=0.008,\hat{\mathrm{b}}=0.462,\hat{\mathsf{\beta }}=185.571$	$\hat{\mathrm{a}}=182.934,\hat{\mathrm{b}}=0.464,\hat{\mathsf{\beta }}=0.0071$
PNZ	$\hat{\mathrm{a}}=26.686,\hat{\mathrm{b}}=0.292,\hat{\mathsf{\beta }}=0.00001,$ $\hat{\mathsf{\beta }}=21.726$	$\hat{\mathrm{a}}=183.125,\hat{\mathrm{b}}=0.463,\hat{\mathsf{\alpha }}=0.007,$ $\hat{\mathsf{\beta }}=0.0001$
PZ	$\hat{\mathrm{a}}=0.0001,\hat{\mathrm{b}}=0.298,\hat{\mathsf{\alpha }}=2000,$ $\hat{\mathsf{\beta }}=22.987,\hat{\mathrm{c}}=26.536$	$\hat{\mathrm{a}}=195.99,\hat{\mathrm{b}}=0.3987,\hat{\mathsf{\alpha }}=1000,$ $\hat{\mathsf{\beta }}=0,\hat{\mathrm{c}}=1.39$
TC	$\hat{\mathrm{a}}=0.149,\hat{\mathrm{b}}=2.234,\hat{\mathsf{\alpha }}=1961.82,$ $\hat{\mathsf{\beta }}=8176.811,\hat{\mathrm{N}}=26.838$	$\hat{\mathrm{a}}=0.053,\hat{\mathrm{b}}=0.774,\hat{\mathsf{\alpha }}=181,$ $\hat{\mathsf{\beta }}=38.6,\hat{\mathrm{N}}=204.14$
New	$\hat{\mathrm{a}}=25.338,\hat{\mathrm{b}}=0.032,\hat{\mathrm{c}}=3.260,\hat{\mathrm{h}}=1.115$	$\hat{\mathrm{a}}=194.766,\hat{\mathrm{b}}=0.304,\hat{\mathrm{c}}=304.566,$ $\hat{\mathrm{h}}=135.464$

summarizes the estimated parameters for datasets.

Table 4. Comparison of all criteria for dataset 1.

Model	MSE	PRR	PP	${\mathit{R}}^2$	SAE	AIC	Variation	RMSPE
GO	3.8516	1.3319	4.8508	0.9546	33.9339	65.3611	1.8323	1.9091
DS	1.4938	12.0680	0.9676	0.9824	19.9967	63.9399	1.1908	1.1913
IS	0.6744	2.8509	0.6561	0.9925	12.9465	64.1779	0.7727	0.7788
YID	2.3842	5.7663	0.8579	0.9734	23.4627	67.1715	1.4649	1.4649
PNZ	0.7141	2.8584	0.9698	0.9925	12.9502	66.1785	0.7728	0.7788
PZ	0.7621	3.2272	0.7029	0.9924	13.1848	68.276	0.7722	0.78
TC	1.0939	102.1599	1.7468	0.9891	16.0532	70.5594	0.9198	0.9348
New	0.5503	0.4518	0.8020	0.9942	11.7685	66.8464	0.6839	0.6839

and

Table 5. Comparison of all criteria for dataset 2.

Model	MSE	PRR	PP	${\mathit{R}}^2$	SAE	AIC	Variation	RMSPE
GO	80.6779	0.1705	0.1013	0.9388	104.4025	184.3314	0.6839	0.6839
DS	232.6282	1.2915	0.3330	0.8234	142.5442	331.8567	8.6734	8.6955
IS	86.4395	0.1706	0.1013	0.9388	104.3703	186.3337	14.6423	14.7605
YID	78.8367	0.1276	0.0866	0.9442	100.6173	157.8252	8.6711	8.6953
PNZ	84.9077	0.1281	0.0867	0.9442	100.6045	159.8744	8.2915	8.3047
PZ	100.9894	0.1719	0.1017	0.9387	104.3539	190.3321	8.2915	8.3049
TC	72.2812	0.0521	0.0479	0.9561	103.1593	158.9319	8.6767	8.7014
New	26.8104	0.0096	0.0092	0.9824	63.9541	Nan	4.6673	4.6673

present the results for the abovementioned eight criteria of all SRGMs for each dataset. Moreover, Figure 3 and Figure 4 show the m(t).

As shown in Table 4, the proposed model achieved the best results for all criteria except for the PP and AIC. However, the PP and AIC of the proposed model were the third smallest and fifth smallest values, respectively. In general, the proposed model outperformed other models.

As shown in Table 5, the proposed model indicated the best results for most criteria (except for the AIC, variation, and RMSPE). Similarly, the variation and RMSPE of the proposed model were the second smallest values. In general, the proposed model exhibited better fitting than that shown by other models for dataset 2. However, because $\mathrm{m}\left(\mathrm{t}\right)$ converged for ${\mathrm{t}}_{\mathrm{i}}=\left\{14,15,16,17\right\}$, the AIC was calculated as not-a-number (NaN). If the AIC of the proposed model was calculated for the time before convergence, ${\mathrm{t}}_{\mathrm{i}}=\left\{1,2,\dots ,13\right\}$, then the value would be 83.51093 (which would be the smallest).

Table 6. Predicted values by proposed model for dataset 1.

Time	$\hat{\mathbf{m}}\left(\mathbf{t}\right)$	Time	$\hat{\mathbf{m}}\left(\mathbf{t}\right)$	Time	$\hat{\mathbf{m}}\left(t\right)$	Time	$\hat{\mathbf{m}}\left(t\right)$
1	1.35543	8	7.423002	15	21.09665	22	25.1102
2	1.727669	9	9.219	16	22.33409	23	25.19946
3	2.209491	10	11.24305	17	23.27083	24	25.2552
4	2.831189	11	13.411	18	23.95131	25	25.28936
5	3.628004	12	15.60229	19	24.42872
6	4.637697	13	17.68339	20	24.75394
7	5.895141	14	19.53903	21	24.96993

and

Table 7. Predicted values by proposed model for dataset 2.

Time	$\hat{\mathbf{m}}\left(t\right)$	Time	$\hat{\mathbf{m}}\left(t\right)$	Time	$\hat{\mathbf{m}}\left(t\right)$	Time	$\hat{\mathbf{m}}\left(t\right)$
1	90.76292	6	185.0622	11	194.766	16	194.766
2	105.7008	7	192.2739	12	194.766	17	194.766
3	125.2	8	194.3851	13	194.766	18	194.766
4	147.9813	9	194.7363	14	194.766	19	194.766
5	169.7534	10	194.765	15	194.766	20	194.766

and Figure 5 and Figure 6 show the predicted values at ${\mathrm{t}}_{\mathrm{i}}$ for each dataset. The predicted values mean the expected cumulative number of failures by the proposed model. In Table 6, the predicted values for future points are {25.1102, 25.19946, 25.2552, 25.28936} at $\mathrm{t}=\left\{22,23,24,25\right\}$. In Table 7, the predicted values for future points are {194.766, 194.766, 194.766} at $\mathrm{t}=\left\{18,19,20\right\}$.

5.2. Confidence Interval

The two-sided limit confidence interval [42] of NHPP SRGMs is defined as:

$$\hat{m}\left(\mathrm{t}\right)\pm {\mathrm{z}}_{\mathsf{\alpha }/2}\sqrt{\hat{m}\left(\mathrm{t}\right)},$$

(17)

where ${\mathrm{z}}_{\mathsf{\alpha }/2}$ is the 100(1$–\mathsf{\alpha }$) percentile of the standard normal distribution.

Table 8. Confidence intervals for datasets 1 and 2.

Dataset 1			Dataset 2
Time	LC	UC	Time	LC	UC
1	0.0	3.637278	1	72.09043	109.4354
2	0.0	4.303861	2	85.55025	125.8514
3	0.0	5.122851	3	103.2694	147.1306
4	0.0	6.129052	4	124.1388	171.8238
5	0.0	7.361210	5	144.2171	195.2896
6	0.416853	8.858541	6	158.3993	211.7251
7	1.136366	10.65392	7	165.0965	219.4513
8	2.083043	12.76296	8	167.0589	221.7113
9	3.267999	15.17000	9	167.3854	222.0871
10	4.671164	17.81494	10	167.4121	222.1180
1	6.233412	20.58859	11	167.4130	222.1190
2	7.860481	23.34409	12	167.4130	222.1190
3	9.441424	25.92536	13	167.4130	222.1190
14	10.87541	28.20265	14	167.4130	222.1190
15	12.09433	30.09898	15	167.4130	222.1190
16	13.0715	31.59667	16	167.4130	222.1190
17	13.81599	32.72567	17	167.4130	222.1190
18	14.35923	33.54339
19	14.74152	34.11592
20	15.00246	34.50541
21	15.176	34.76385

, Figure 7 and Figure 8 show the lower-limit confidence interval (LC) and the upper-limit confidence interval (UC) of the proposed model for datasets 1 and 2 at ${\mathrm{t}}_{\mathrm{i}}$.

5.3. Results of the SPRT for Datasets

We determined the software reliability by applying the SPRT based on the parameters of the proposed model. In particular, a sensitivity analysis showed that $\mathrm{a}$ and $\mathrm{b}$ were more sensitive than other parameters of the proposed model; therefore, we focused on $\mathrm{a}$ and $\mathrm{b}$. The related assumptions are listed in

Table 9. Case for applying the sequential probability ratio test (SPRT).

Case	$\mathsf{\delta }$	$\mathsf{\alpha } \left(\mathbf{Type}1\mathbf{Error}\right)$	$\mathsf{\beta } \left(\mathbf{Type}2\mathbf{Error}\right)$
Case 1 (for parameter $\mathrm{a}$)	0.9	0.1	0.1
Case 2 (for parameter $\mathrm{b}$)	0.03	0.1	0.1

.

Subsequently, we set ${\mathrm{a}}_0=\mathrm{a}-\mathsf{\delta }$, ${\mathrm{a}}_1=\mathrm{a}+\mathsf{\delta }$ for Case 1 and ${\mathrm{b}}_0=\mathrm{b}-\mathsf{\delta }$, ${\mathrm{b}}_1=\mathrm{b}+\mathsf{\delta }$ for Case 2. Substituting ${\mathrm{a}}_0$ and ${\mathrm{a}}_1$ for the $\mathrm{m}\left(\mathrm{t}\right)$ of the proposed model instead of $\mathrm{a}$, we obtained ${\mathrm{m}}_0\left(\mathrm{t}\right)$ and ${\mathrm{m}}_1\left(\mathrm{t}\right)$, respectively. Likewise,${\mathrm{m}}_0\left(\mathrm{t}\right)$ and ${\mathrm{m}}_1\left(\mathrm{t}\right)$ were obtained by substituting ${\mathrm{b}}_0$ and ${\mathrm{b}}_1$ into $\mathrm{m}\left(\mathrm{t}\right)$, respectively. Substituting ${\mathrm{m}}_0\left(\mathrm{t}\right)$ and ${\mathrm{m}}_1\left(\mathrm{t}\right)$, obtained in Equation (12), allows us to discuss the SPRT procedure.

Table 10. SPRT results for both datasets (Case 1, parameter $\mathrm{a}$).

Dataset 1					Dataset 2
T	N(t)	Acceptance Region	Rejection Region	Result	T	N(t)	Acceptance Region	Rejection Region	Result
1	1	−105.151	107.8621	continue	1	90	−314.127	495.6517	continue
2	1	−55.3275	58.78291	continue	2	107	−196.369	407.7699	continue
3	2	−36.7052	41.12448	continue	3	126	−116.805	367.2043	continue
4	3	−26.7731	32.43642	continue	4	145	−63.6092	359.5706	continue
5	5	−20.4124	27.67021	continue	5	171	−34.4734	373.9780	continue
6	5	−15.8077	25.08616	continue	6	188	−28.1345	398.2559	continue
7	5	−12.1513	23.94599	continue	7	189	−34.7014	419.2460	continue
8	8	−9.04072	23.89200	continue	8	190	−40.7869	429.5541	continue
9	9	−6.28005	24.72266	continue	9	190	−42.7149	432.1846	continue
10	11	−3.80494	26.29212	continue	10	190	−42.9682	432.4955	continue
11	13	−1.64563	28.46133	continue	11	192	−42.9805	432.5098	continue
12	15	0.107043	31.08023	continue	12	192	−42.9807	432.5099	continue
13	19	1.344969	33.99174	continue	13	192	−42.9807	432.5099	continue
14	19	1.991195	37.04500	continue	14	192	−42.9807	432.5099	continue
15	22	2.038062	40.10500	continue	15	203	−42.9807	432.5099	continue
16	22	1.560907	43.05311	continue	16	203	−42.9807	432.5099	continue
17	23	0.703089	45.78459	continue	17	204	−42.9807	432.5099	continue
18	24	−0.36003	48.21173	continue
19	24	−1.46272	50.27382	continue
20	24	−2.4802	51.94671	continue
21	26	−3.34086	53.24403	continue

shows the results of the SPRT for parameter $\mathrm{a}$ for both datasets. Judging the reliability using the proposed model, we conclude that software testing should continue because we cannot determine the software reliability yet.

Table 11. SPRT results for both datasets (Case 2, parameter $\mathrm{b}$).

Dataset 1					Dataset 2
T	N(t)	Acceptance Region	Rejection Region	Result	T	N(t)	Acceptance Region	Rejection Region	Result
1	1	−3.554070	6.287044	continue	1	90	11.45807	170.0957	continue
2	1	−0.636960	4.216873	continue	2	107	71.55002	139.9607	continue
3	2	0.776205	3.995391	continue	3	126	103.6526	146.8394	continue
4	3	1.977399	4.409981	continue	4	145	129.8891	165.5602	continue
5	5	3.205886	5.201452	continue	5	171	149.0555	188.4585	continue
6	5	4.434785	6.177075	continue	6	188	152.8588	214.1708	continue
7	5	5.509077	7.107788	accept	7	189	120.5342	261.5581	continue
					8	190	−56.62470	444.3444	continue
					9	190	−1258.160	1647.398	continue
					10	190	−14,878.80	15,268.35	continue
					11	192	−325,116	325,505.2	continue
					12	192	−1.8 × 107	18,130,355	continue
					13	192	−3.5 × 109	3.47 × 109	continue
					14	192	−3.3 × 1012	3.28 × 1012	continue
					15	203	-Inf	Inf	continue
					16	203	-Inf	Inf	continue
					17	204	-Inf	Inf	continue

shows the SPRT results for parameter $\mathrm{b}$ for both datasets. When t = 7 in dataset 1, the test was accepted. Using the proposed model for dataset 2, we conclude that software testing should continue because we cannot determine the software reliability yet. However, as converged for ${\mathrm{t}}_{\mathrm{i}}=\left\{14,15,16,17\right\}$, the result was “Inf”.

6. Conclusions and Remarks

Herein, we proposed a new SRGM. General SRGMs assume independent failures. However, in the proposed model, software failures depend on one another. We presented experiments on real-world datasets using eight evaluation criteria. The proposed model achieved the best goodness-of-fit on both datasets. In addition, we evaluated the software reliability using the SPRT, which is more efficient than the classical hypothesis test. As shown in Table 10, we demonstrated that the test based on parameter $\mathrm{a}$ should be continued for both datasets because no reliable judgment was obtained. The test based on parameter $\mathrm{b}$ was accepted, as shown in Table 11, when t = 7 for dataset 1. However, the test should be continued for dataset 2 because the judgment was unreliable.

In the future, experiments based on more recent data should be performed to further validate the superiority of the proposed model. In addition, after estimating the parameters of software reliability models using the MLE and Bayesian techniques, we plan to discuss software reliability by applying the SPRT.