Joint Statistical Inference for the Area under the ROC Curve and Youden Index under a Density Ratio Model

Abstract

The receiver operating characteristic (ROC) curve is a valuable statistical tool in medical research. It assesses a biomarker’s ability to distinguish between diseased and healthy individuals. The area under the ROC curve ($AUC$) and the Youden index (J) are common summary indices used to evaluate a biomarker’s diagnostic accuracy. Simultaneously examining $AUC$ and J offers a more comprehensive understanding of the ROC curve’s characteristics. In this paper, we utilize a semiparametric density ratio model to link the distributions of a biomarker for healthy and diseased individuals. Under this model, we establish the joint asymptotic normality of the maximum empirical likelihood estimator of $(AUC,J)$ and construct an asymptotically valid confidence region for $(AUC,J)$. Furthermore, we propose a new test to determine whether a biomarker simultaneously exceeds prespecified target values of $AU{C}_0$ and $J_0$ with the null hypothesis $H_0:AUC\le AU{C}_0$ or $J\le J_0$ against the alternative hypothesis $H_a:AUC>AU{C}_0$ and $J>J_0$. Simulation studies and a real data example on Duchenne Muscular Dystrophy are used to demonstrate the effectiveness of our proposed method and highlight its advantages over existing methods.

1. Introduction

The ROC curve is a valuable statistical tool in medical research for evaluating the performance of binary classifiers across different thresholds. It finds wide applications in fields like radiology, oncology, and genomics [1,2]. In medical studies, ROC curves are particularly useful when evaluating a continuous biomarker to classify individuals as diseased or healthy. Graphically, the ROC curve plots sensitivity (the proportion of true positive) versus one minus specificity (the proportion of false positive) at all possible biomarker thresholds. Extensive and in-depth research has delved into the intricate realm of statistical inferences associated with ROC curves, offering valuable insights and understanding of how these curves are used to evaluate the performance of classification models. For a detailed review, refer to [3,4,5,6].

Let $F_0$ denote the cumulative distribution function (CDF) of the healthy population and $F_1$ denote that of the diseased population. Without loss of generality, let us assume that biomarker values are higher in the diseased group than in the healthy group, and an individual is classified as diseased when their biomarker value exceeds a given threshold (x). Under this assumption, the sensitivity is $1-F_1\left(x\right)$, and the specificity is $F_0\left(x\right)$. The ROC curve is then given by

$$ROC\left(s\right)=1-F_1\left(F_0^{-1}(1-s)\right)$$

for $s\in (0,1)$, where $F_0^{-1}(1-s)=inf\{x\in \mathbb{R}:F_0\left(x\right)\ge 1-s\}$.

In ROC analysis, two common summary indices are used to assess a biomarker’s diagnostic accuracy: the $AUC$ [7,8] and the Youden index (J) [9,10,11]. They are defined mathematically as

$$AUC={\int }_0^{1}ROC\left(s\right)ds\mathrm{and}J=\{1-F_1\left(c\right)+F_0\left(c\right)-1\}=\{F_0\left(c\right)-F_1\left(c\right)\},$$

(1)

where $c=arg{max}_x\{F_0\left(x\right)-F_1\left(x\right)\}$ is the “optimal” threshold. By definition, the $AUC$ summarizes the overall performance of a classifier across all possible thresholds. While valuable, it does not directly provide an “optimal” threshold. On the other hand, J is the maximum value of the sensitivity plus the specificity minus 1. Not only does J quantify the biomarker’s effectiveness (with a value of $J=1$ indicating complete separation of the biomarker’s distributions for diseased and healthy populations, and $J=0$ indicating complete overlap), but it also offers a distinct advantage over $AUC$ by providing a criterion for selecting the “optimal” threshold c. However, J only measures the diagnostic accuracy at the “optimal” threshold c and not at other thresholds.

In practical scenarios where medical practitioners encounter multiple biomarkers, they often use the $AUC$ to choose the most diagnostically useful biomarker [12,13]. However, relying solely on $AUC$ has limitations. The biomarker with the highest $AUC$ might not have the best overall accuracy at the “optimal” threshold. Similarly, focusing only on the Youden index selects the biomarker with the highest total accuracy at the “optimal” threshold. But this "best" biomarker by the Youden index may not perform well overall. If the threshold changes, the biomarker may no longer maintain satisfactory diagnostic accuracy. For real examples and further discussions, refer to [1,14]. In summary, both the $AUC$ and the Youden index are valuable tools for evaluating a biomarker’s effectiveness, each emphasizing distinct aspects of its performance. Simultaneously examining $AUC$ and J, which provide complementary information, may help us make better decisions [1]. This motivates us to develop joint inference procedures for $AUC$ and J in this paper.

In the literature, ref. [1] considered both parametric and nonparametric methods for constructing confidence regions of $AUC$ and J. Later, ref. [2] proposed both parametric and nonparametric tests to determine if a biomarker exceeds predefined target values with hypotheses $H_0:AUC\le AU{C}_0\mathrm{or}J\le J_0$ versus $H_a:AUC>AU{C}_0\mathrm{and}J>J_0$. For the parametric inference procedures, it is assumed that the original biomarkers or the biomarkers after the Box–Cox transformation follow normal distributions in both the healthy and diseased groups. For the nonparametric inference procedures, the empirical CDF or kernel method is used to estimate $F_0$ and $F_1$.

Generally, parametric joint inference procedures are highly efficient when the underlying parametric models are correct. This means that the resulting confidence region for $(AUC,J)$ has a smaller area, and the joint testing procedure has greater power. However, these procedures may not be robust to misspecification of the models for $F_0$ and $F_1$. See Section 4 for more details. On the other hand, nonparametric methods are free from assumptions about the models of $F_0$ and $F_1$. In medical research, it has been observed that healthy and diseased populations often share certain common characteristics [4,15,16,17]. However, fully nonparametric methods ignore this information, potentially leading to inefficient inference procedures.

In this paper, we develop new semiparametric joint inference procedures for $(AUC,J)$ based on a semiparametric density ratio model (DRM; refs. [18,19,20]), which effectively utilizes information from both healthy and diseased populations. Let $d{F}_1$ and $d{F}_2$ be the probability density functions of $F_1$ and $F_2$, respectively. The DRM assumes that

$$d{F}_1\left(x\right)=exp\left\{\alpha +{\mathit{\beta }}^{\mathrm{T}}\mathbf{q}\left(x\right)\right\}d{F}_0\left(x\right)=exp\left\{{\mathit{\theta }}^{\mathrm{T}}\mathbf{Q}\left(x\right)\right\}d{F}_0\left(x\right),$$

(2)

where $\mathbf{q}\left(x\right)$ is a prespecified, p-variate vector-valued nontrivial function of x, $\mathbf{Q}\left(x\right)={(1,\mathbf{q}{\left(x\right)}^{\mathrm{T}})}^{\mathrm{T}}$, and $\mathit{\theta }={(\alpha ,{\mathit{\beta }}^{\mathrm{T}})}^{\mathrm{T}}$ are unknown parameters. The unspecified baseline distribution $F_0$ makes DRM a semiparametric model. This flexibility allows DRM to encompass many distributions commonly used in studying ROC curves [21]. For instance, if we set $\mathbf{q}\left(x\right)$ to $\log \left(x\right)$, the DRM encompasses the lognormal distributions (with equal variance on the log scale) and the beta distributions (sharing the same power parameter for $(1-x)$). Similarly, setting $\mathbf{q}\left(x\right)$ to x, it includes the normal distributions with the same variance and exponential distributions. The DRM has a close relationship with the logistic regression model. To illustrate this point, let us define D = 0 and 1 as indicators for individuals from the healthy and diseased populations, respectively. As shown by [18,19], the DRM is equivalent to the logistic regression model through the following equation:

$$P(D=1|x)=\frac{exp\{{\alpha }^{*}+{\mathit{\beta }}^{\mathrm{T}}\mathbf{q}\left(x\right)\}}{1+exp\{{\alpha }^{*}+{\mathit{\beta }}^{\mathrm{T}}\mathbf{q}\left(x\right)\}},$$

where ${\alpha }^{*}=\alpha +\log \{P(D=1)/P(D=0)\}$.

The DRM has proven itself as a valuable tool for inference on ROC curves and their summary indices [4,15,17,22]. Existing theoretical and numerical studies have shown that point estimators of $AUC$ and J under the DRM are more efficient than fully nonparametric estimators. However, as far as we are aware, semiparametric joint inference procedures for $(AUC,J)$, such as confidence regions and joint hypothesis testing procedures, remain uninvestigated under the DRM (2). This paper aims to fill this gap.

Our contributions are three-fold. First, we establish the joint asymptotic normality of the maximum empirical likelihood estimator (MELE) of $(AUC,J)$ under the DRM (2). This allows us to construct an asymptotically valid Wald-type confidence region for $(AUC,J)$. We further propose a nonparametric bootstrap procedure to improve the coverage accuracy of the Wald-type confidence region. Second, we develop a joint testing procedure for the null hypothesis: $H_0:AUC\le AU{C}_0\mathrm{or}J\le J_0$ versus the alternative hypothesis: $H_a:AUC>AU{C}_0\mathrm{and}J>J_0$. We introduce a novel bootstrap procedure to obtain its p-value. Finally, we evaluate the performance of our proposed methods through simulation studies and application to real data on Duchenne Muscular Dystrophy. The numerical studies demonstrate that the proposed method produces more precise confidence regions for $(AUC,J)$ with smaller areas. Additionally, the newly proposed joint testing procedure maintains controlled type-I error rates while achieving satisfactory power.

The rest of the paper is structured as follows. In Section 2, we introduce the maximum empirical likelihood estimator (MELE) of $(AUC,J)$ and prove its joint asymptotic normality. Section 3 details the proposed joint inference procedures. This includes constructing confidence regions and conducting joint hypothesis tests for $(AUC,J)$. Section 4 presents simulation results and Section 5 contains a real application. A summary and discussion of the findings are given in Section 6 and Section 7, respectively.

2. Methodology

Let $\left\{x_1,\dots ,x_{n_0}\right\}$ and $\left\{y_1,\dots ,y_{n_1}\right\}$ denote independent random samples from the healthy and diseased populations, respectively. We define a combined sample of size $n=n_0+n_1$ by setting $z_i=x_i$ for $1\le i\le n_0$ and $z_{n_0+i}=y_i$ for $1\le i\le n_1$.

2.1. Maximum Empirical Likelihood Estimators of AUC and J

We begin by developing the empirical likelihood (EL) function. By the EL principle [23] and under the DRM (2), the likelihood function based on the observed data is

$$\begin{array}{c}\hfill L_n=\prod _{i=1}^{n_0}\mathrm{d}{F}_0\left(x_i\right)\prod _{i=1}^{n_1}\mathrm{d}{F}_1\left(y_i\right)=\prod _{i=1}^n\mathrm{d}{F}_0\left(z_i\right)\prod _{i=1}^{n_1}exp\left\{{\mathit{\theta }}^{\mathrm{T}}\mathbf{Q}\left(y_i\right)\right\}=\prod _{i=1}^n{p}_i·\prod _{i=1}^{n_1}exp\left\{{\mathit{\theta }}^{\mathrm{T}}\mathbf{Q}\left(y_i\right)\right\},\end{array}$$

where $p_i=d{F}_0\left(z_i\right)$ for $i=1,\dots ,n$, and they satisfy

$$\begin{array}{c}\hfill p_i\ge 0,\sum _{i=1}^n{p}_i=1,\sum _{i=1}^n{p}_{i}exp\left\{{\mathit{\theta }}^{\mathrm{T}}\mathbf{Q}\left(z_i\right)\right\}=1.\end{array}$$

(3)

The MELEs of $(\mathit{\theta },p_1,\dots ,p_n)$, denoted as $(\widehat{\mathit{\theta }},{\widehat{p}}_1,\dots ,{\widehat{p}}_n)$, are defined as the maximizer of $L_n$ subject to the constraints in (3).

Let $\rho =n_1/n$. Following [19,24], we obtain the MELE of $\mathit{\theta }$ by

$$\widehat{\mathit{\theta }}=arg\underset{\mathit{\theta }}{max}{\ell }_n\left(\mathit{\theta }\right),$$

where

$${\ell }_n\left(\mathit{\theta }\right)=\sum _{i=1}^{n_1}\left\{{\mathit{\theta }}^{\mathrm{T}}\mathbf{Q}\left(y_i\right)\right\}-\sum _{i=1}^n\log \left[1-\rho +\rho exp\left\{{\mathit{\theta }}^{\mathrm{T}}\mathbf{Q}\left(z_i\right)\right\}\right]$$

(4)

is the dual empirical log-likelihood function.

Once we have $\widehat{\mathit{\theta }}$, we calculate the MELEs of $p_i$ as

$${\widehat{p}}_i=\frac{1}{n}\frac{1}{1-\rho +\rho exp\left\{{\widehat{\mathit{\theta }}}^{\mathrm{T}}\mathbf{Q}\left(z_i\right)\right\}},i=1,\cdots ,n.$$

Subsequently, the MELEs of $F_0$ and $F_1$ are given by

$$\begin{array}{c}\hfill {\widehat{F}}_0\left(x\right)=\sum _{i=1}^n{\widehat{p}}_i\mathbb{1}\left(z_i\le x\right)\mathrm{and}{\widehat{F}}_1\left(x\right)=\sum _{i=1}^n{\widehat{p}}_{i}exp\left\{{\widehat{\mathit{\theta }}}^{\mathrm{T}}\mathbf{Q}\left(z_i\right)\right\}\mathbb{1}\left(z_i\le x\right),\end{array}$$

where $\mathbb{1}(·)$ is the indicator function.

Recall the definition of $AUC$ in (1). It can be verified that

$$\begin{array}{c}\hfill AUC={\int }_{-\infty }^{\infty }{F}_0\left(x\right)\mathrm{d}{F}_1\left(x\right)={\int }_{-\infty }^{\infty }\{1-F_1\left(x\right)\}\mathrm{d}{F}_0\left(x\right).\end{array}$$

The MELE of $AUC$ is then given by

$$\begin{array}{ccc}\hfill \widehat{AUC}& =& \frac{1}{2}{\int }_{-\infty }^{\infty }{\widehat{F}}_0\left(x\right)\mathrm{d}{\widehat{F}}_1\left(x\right)+\frac{1}{2}{\int }_{-\infty }^{\infty }\{1-{\widehat{F}}_1\left(x\right)\}\mathrm{d}{\widehat{F}}_0\left(x\right)\hfill \\ & & =\frac{1}{2}\sum _{i=1}^n{\widehat{p}}_{i}exp\left\{{\widehat{\mathit{\theta }}}^{\mathrm{T}}\mathbf{Q}\left(z_i\right)\right\}{\widehat{F}}_0\left(z_i\right)+\frac{1}{2}\sum _{i=1}^n{\widehat{p}}_i\{1-{\widehat{F}}_1\left(z_i\right)\}.\hfill \end{array}$$

Again, recall the definition of the Youden index J in (1). Then, the optimal threshold c should satisfy $d{F}_1\left(c\right)=d{F}_0\left(c\right)$ [17]. This is equivalent to

$${\mathit{\theta }}^{\mathrm{T}}\mathbf{Q}\left(c\right)=0.$$

With $\widehat{\mathit{\theta }}$, the MELE $\widehat{c}$ of c solves

$${\widehat{\mathit{\theta }}}^{\mathrm{T}}\mathbf{Q}\left(\widehat{c}\right)=0.$$

The MELE of J is then defined as

$$\begin{array}{c}\hfill \widehat{J}={\widehat{F}}_0\left(\widehat{c}\right)-{\widehat{F}}_1\left(\widehat{c}\right).\end{array}$$

2.2. Joint Asymptotic Normality of $(\widehat{AUC},\widehat{J})$

In this section, we establish the joint asymptotic normality of $(\widehat{AUC},\widehat{J})$. We begin by introducing some notation. Let ${\mathit{\theta }}^{*}$ denote the true value of $\mathit{\theta }$,

$$\omega \left(x\right)=exp\left\{{\left({\mathit{\theta }}^{*}\right)}^{\mathrm{T}}\mathbf{Q}\left(x\right)\right\},h\left(x\right)=1-\rho +\rho \omega \left(x\right),h_1\left(x\right)=\frac{\rho \omega \left(x\right)}{h\left(x\right)},h_0\left(x\right)=1-h_1\left(x\right),$$

and

$${\mathbf{U}}_n=\left(\begin{array}{c}-n^{-1}\sum _{i=1}^n{h}_1\left(z_i\right)\mathbf{Q}\left(z_i\right)+n^{-1}\sum _{i=n_0+1}^n\mathbf{Q}\left(z_i\right)\\ {\left\{n(1-\rho )\right\}}^{-1}\sum _{i=1}^n{h}_0\left(z_i\right)F_1\left(z_i\right)-(1-AU{C}^{*})\\ {\left(n\rho \right)}^{-1}\sum _{i=1}^n{h}_1\left(z_i\right)F_0\left(z_i\right)-AU{C}^{*}\\ {\left\{n(1-\rho )\right\}}^{-1}\sum _{i=1}^n{h}_0\left(z_i\right)\mathbb{1}\left(z_i\le c^{*}\right)-F_0\left(c^{*}\right)\\ {\left(n\rho \right)}^{-1}\sum _{i=1}^n{h}_1\left(z_i\right)\mathbb{1}\left(z_i\le c^{*}\right)-F_1\left(c^{*}\right)\end{array}\right).$$

(5)

In Lemma A2 of Appendix A, we show that

$$E\left({\mathbf{U}}_n\right)=\mathbf{0}\mathrm{and}Var\left({\mathbf{U}}_n\right)=n^{-1}\mathbf{V},$$

where the detailed form of $\mathbf{V}$ is given in Lemma A2.

We denote the true values of $AUC$, Youden index J, and optimal threshold c as $AU{C}^{*},J^{*}$ and $c^{*}$, respectively. The asymptotic results in this section rely on the following regularity conditions.

C1.

$J_{ϵ}={sup}_{\left|x-c^{*}\right|>ϵ}\left\{F_0\left(x\right)-F_1\left(x\right)\right\}<J^{*}$ for any $ϵ>0$.

C2.

$F_0^{\prime }\left(x\right)$ and $F_1^{″}\left(x\right)$ are continuous in the neighborhood of $c^{*}$, with $F_0^{\prime }\left(c^{*}\right)-F_1^{\prime }\left(c^{*}\right)=0$ and $F_0^{″}\left(c^{*}\right)-F_1^{″}\left(c^{*}\right)<0$.

C3.

The total sample size $n=n_0+n_1\to \infty$, and $\rho =n_1/n$ remains constant.

C4.

The DRM (2) is satisfied by $F_0$ and $F_1$. Additionally, ${\int }_{-\infty }^{\infty }\mathbf{Q}\left(x\right)\mathbf{Q}{\left(x\right)}^{\mathrm{T}}d{F}_0\left(x\right)$ is positive definite, and for $\mathit{\theta }$ in a neighborhood of ${\mathit{\theta }}^{*}$,

$${\int }_{-\infty }^{\infty }exp\left\{{\mathit{\theta }}^{\mathrm{T}}\mathbf{Q}\left(x\right)\right\}d{F}_0\left(x\right)<\infty .$$

We note that Conditions C1 and C2 are from [25]. These conditions ensure the identifiability of $c^{*}$. Condition C4 ensures that the components of $\mathbf{q}\left(x\right)$ are linearly independent under both $F_0\left(x\right)$ and $F_1\left(x\right)$. Conditions C3 and C4 guarantee the asymptotic normality of $\widehat{\mathit{\theta }}$.

The following theorem establishes the joint asymptotic normality of $(\widehat{AUC},\widehat{J})$. The proof is provided in Appendix A.

Theorem 1. 

Suppose Conditions C1–C4 are satisfied. As the total sample size $n\to \infty$, we have

$$\begin{array}{c}\hfill \sqrt{n}\left(\begin{array}{c}\widehat{AUC}-AU{C}^{*}\\ \widehat{J}-J^{*}\end{array}\right)\to \mathrm{N}\left(\mathbf{0},\mathsf{\Sigma }\right)\end{array}$$

in distribution, where $\mathsf{\Sigma }=\mathbf{H}\mathbf{V}{\mathbf{H}}^{\mathrm{T}}$ with

$$\mathbf{H}=\left(\begin{array}{ccccc}\mathbf{B}{\mathbf{A}}^{-1}& -1& 1& 0& 0\\ \mathbf{C}{\mathbf{A}}^{-1}& 0& 0& 1& -1\end{array}\right),$$

where

$$\begin{array}{ccc}\hfill \mathbf{A}& =& {\int }_{-\infty }^{\infty }{h}_1\left(x\right)\mathbf{Q}\left(x\right)\mathbf{Q}{\left(x\right)}^{\mathrm{T}}d{F}_0\left(x\right),\hfill \\ \hfill {\mathbf{B}}^{\mathrm{T}}& =& {\int }_{-\infty }^{\infty }{h}_1\left(x\right)\left\{\frac{1}{\rho }{F}_0\left(x\right)+\frac{1}{1-\rho }{F}_1\left(x\right)\right\}\mathbf{Q}\left(x\right)d{F}_0\left(x\right),\hfill \\ \hfill {\mathbf{C}}^{\mathrm{T}}& =& -\frac{1}{\rho (1-\rho )}{\int }_{-\infty }^{c^{*}}{h}_1\left(x\right)\mathbf{Q}\left(x\right)d{F}_0\left(x\right).\hfill \end{array}$$

Remark 1. 

Our method relies on the DRM assumption (refer to Equation (2)). To assess the validity of this model assumption in practice, we can use goodness-of-fit test statistics proposed by [19]:

$${\Delta }_{n0}=\underset{x}{sup}|{\widehat{F}}_0\left(x\right)-{\tilde{F}}_0\left(x\right)|\mathrm{or}{\Delta }_{n1}=\underset{x}{sup}\left|{\widehat{F}}_1\left(x\right)-{\tilde{F}}_1\left(x\right)\right|,$$

and apply the bootstrap method to perform the test. Here,

$${\tilde{F}}_0\left(x\right)=\frac{1}{n_0}\sum _{i=1}^{n_0}\mathbb{1}\left(x_i\le x\right)and{\tilde{F}}_1\left(x\right)=\frac{1}{n_1}\sum _{i=1}^{n_1}\mathbb{1}\left(y_i\le x\right).$$

It can be shown that ${\Delta }_{n0}=\rho {\Delta }_{n1}/(1-\rho )$. Therefore, the test results based on ${\Delta }_{n0}$ and ${\Delta }_{n1}$ are equivalent to each other. Consequently, we only need to consider one statistic, for example ${\Delta }_{n0}$, for practical applications.

3. Joint Inference Procedures for $(\mathit{AUC},\mathit{J})$ under the DRM

3.1. Confidence Region of $(AUC,J)$

The variance–covariance matrix $\mathsf{\Sigma }$ in Theorem 1 depends on $\left({\mathit{\theta }}^{*},AU{C}^{*},J^{*},c^{*},F_0,F_1\right)$. Replacing these by their MELEs $\left(\widehat{\mathit{\theta }},\widehat{AUC},\widehat{J},\widehat{c},{\widehat{F}}_0,{\widehat{F}}_1\right)$ leads to a variance estimator

$$\widehat{\mathsf{\Sigma }}=\left(\begin{array}{cc}{\widehat{\sigma }}_{11}& {\widehat{\sigma }}_{12}\\ {\widehat{\sigma }}_{21}& {\widehat{\sigma }}_{22}\end{array}\right).$$

It can be easily shown that $\widehat{\mathsf{\Sigma }}$ is consistent with $\mathsf{\Sigma }$.

Theorem 2. 

Under the conditions of Theorem 1, it follows that $\widehat{\mathsf{\Sigma }}\to \mathsf{\Sigma }$ in probability as $n\to \infty$.

For notational convenience, let $\mathit{\eta }={(AUC,J)}^{\mathrm{T}}$, ${\mathit{\eta }}^{*}={(AU{C}^{*},J^{*})}^{\mathrm{T}}$, and $\widehat{\mathit{\eta }}={(\widehat{AUC},\widehat{J})}^{\mathrm{T}}$. Building upon the asymptotic results in Theorems 1 and 2, we conclude that

$$T_n\left({\mathit{\eta }}^{*}\right)=n{\left(\widehat{\mathit{\eta }}-{\mathit{\eta }}^{*}\right)}^{\mathrm{T}}{\widehat{\mathsf{\Sigma }}}^{-1}\left(\widehat{\mathit{\eta }}-{\mathit{\eta }}^{*}\right)\to {\chi }_2^2$$

in distribution as $n\to \infty$. Therefore, a $100(1-a)\%$th asymptotic Wald-type confidence region for $\mathit{\eta }$ is

$$\begin{array}{c}\hfill \left\{\mathit{\eta }:T_n\left(\mathit{\eta }\right)⩽{\chi }_{2,1-a}^2\right\},\end{array}$$

(6)

where ${\chi }_{2,1-a}^2$ represents the $100(1-a)\%$ quantile of the Chi-square distribution with two degrees of freedom. Our simulation results in Section 4 demonstrate that this approach yields liberal confidence regions when sample sizes are insufficiently large. To enhance coverage accuracy, we propose a bootstrap method. Throughout the subsequent discussion, we denote quantities derived from the l-th bootstrap sample with the subscript “$B,l$”.

Step 1.

Calculate $\widehat{\mathit{\eta }}$ and the corresponding $\widehat{\mathsf{\Sigma }}$ based on the observed data ${\left\{x_i\right\}}_{i=1}^{n_0}$ and ${\left\{y_i\right\}}_{i=1}^{n_1}$.

Step 2.

For $l=1,\dots ,L$, draw a bootstrap sample of size $n_1$ with replacements from ${\left\{y_i\right\}}_{i=1}^{n_1}$ and another bootstrap sample of size $n_0$ with replacements from ${\left\{x_i\right\}}_{i=1}^{n_0}$.

Step 3.

For $l=1,\dots ,L$, based on the lth bootstrap two-sample data in Step 2, calculate the estimate ${\widehat{\mathit{\eta }}}_{B,l}$ for $\mathit{\eta }$ and the corresponding ${\widehat{\mathsf{\Sigma }}}_{B,l}$ for $\mathsf{\Sigma }$, and compute

$$T_{n,B,l}=n{\left({\widehat{\mathit{\eta }}}_{B,l}-\widehat{\mathit{\eta }}\right)}^{\mathrm{T}}{\widehat{\mathsf{\Sigma }}}_{B,l}^{-1}\left({\widehat{\mathit{\eta }}}_{B,l}-\widehat{\mathit{\eta }}\right).$$

Step 4.

Obtain the $100(1-a)\%$th quantile of ${\left\{T_{n,B,l}\right\}}_{l=1}^L$, which is denoted as $q_{1-a,T}$.

Step 5.

The $100(1-a)\%$ bootstrap confidence region of $\mathit{\eta }$ is given by

$$\begin{array}{c}\hfill \left\{\mathit{\eta }:T_n\left(\mathit{\eta }\right)⩽q_{1-a,T}\right\}.\end{array}$$

(7)

In our simulation study, we set $L=500$. The resulting bootstrap confidence region for $\eta$ offers improved coverage accuracy, as will be discussed in Section 4.

3.2. Joint Hypothesis Testing on $(AUC,J)$

In this section, we examine a testing procedure to determine whether a biomarker simultaneously exceeds prespecified target values of $AU{C}_0$ and $J_0$. We define the hypotheses as follows:

$$\begin{array}{c}\hfill H_0:AUC\le AU{C}_0\mathrm{or}J\le J_0\mathrm{versus}{H}_a:AUC>AU{C}_0\mathrm{and}J>J_0.\end{array}$$

(8)

It is important to note that the null hypothesis represents a multivariate order-restrictive hypothesis within a non-convex space.

Our testing procedure is motivated by the results in Proposition 1. The proof is provided in Appendix B.

Proposition 1. 

Suppose that ${(X_1,X_2)}^{\mathrm{T}}$ is a bivariate normal random vector with unknown mean $\mathit{\mu }={({\mu }_1,{\mu }_2)}^{\mathrm{T}}$ and known variance–covariance matrix $\left(\begin{array}{cc}{\sigma }_1^2& \rho {\sigma }_1{\sigma }_2\\ \rho {\sigma }_1{\sigma }_2& {\sigma }_2^2\end{array}\right)$. Let

$${\mathsf{\Theta }}_0=\left\{\mathit{\mu }\right|{\mu }_1\le {\mu }_{10}\mathit{or}{\mu }_2\le {\mu }_{20}\}.$$

(a) 

The maximum likelihood estimator of μ in ${\mathsf{\Theta }}_0$ based on ${(X_1,X_2)}^{\mathrm{T}}$ is given by

$$\begin{array}{c}\hfill {\tilde{\mathit{\mu }}}^{\mathrm{T}}=\left\{\begin{array}{cc}(X_1,X_2),\hfill & \mathit{if}{X}_1\le {\mu }_{10}\mathit{or}{X}_2\le {\mu }_{20},\hfill \\ \left(X_1-\frac{\rho {\sigma }_1(X_2-{\mu }_{20})}{{\sigma }_2},{\mu }_{20}\right),\hfill & \mathit{if}{X}_1>{\mu }_{10},X_2>{\mu }_{20},\mathit{and}\frac{X_1-{\mu }_{10}}{{\sigma }_1}\ge \frac{X_2-{\mu }_{20}}{{\sigma }_2},\hfill \\ \left({\mu }_{10},X_2-\frac{\rho {\sigma }_2(X_1-{\mu }_{10})}{{\sigma }_1}\right),\hfill & \mathit{if}{X}_1>{\mu }_{10},X_2>{\mu }_{20},\mathit{and}\frac{X_1-{\mu }_{10}}{{\sigma }_1}<\frac{X_2-{\mu }_{20}}{{\sigma }_2}.\hfill \end{array}\right.\end{array}$$

(b) 

The likelihood ratio test statistic for testing $H_0:\mathit{\mu }\in {\mathsf{\Theta }}_0$ versus $H_a:\mathit{\mu }\notin {\mathsf{\Theta }}_0$ is

$$\begin{array}{c}\hfill {\left[min\left\{{\left(\frac{X_1-{\mu }_{10}}{{\sigma }_1}\right)}^{+},{\left(\frac{X_2-{\mu }_{20}}{{\sigma }_2}\right)}^{+}\right\}\right]}^2,\end{array}$$

where $x^{+}$ is the positive part of x.

Define

$$W_{n1}=\frac{\widehat{AUC}-AU{C}_0}{\sqrt{{\widehat{\sigma }}_{11}}}\mathrm{and}{W}_{n2}=\frac{\widehat{J}-J_0}{\sqrt{{\widehat{\sigma }}_{22}}}.$$

With the asymptotic results in Theorems 1 and 2, we propose, according to Proposition 1 (b), to test (8) based on the following test statistic

$$W_n={\left\{min\left(W_{n1}^{+},W_{n2}^{+}\right)\right\}}^2.$$

We reject the null hypothesis in (8) when $W_n$ exceeds a critical value.

The distribution of $W_n$ depends on the true null model, which is unknown. Motivated by Proposition 1, we suggest a bootstrap procedure based on $W_n$ for the hypothesis testing problem in (8). For ease of presentation, let ${\widehat{F}}_{0,{AUC}_0}\left(x\right)$ and ${\widehat{F}}_{1,{AUC}_0}\left(x\right)$ denote the MELEs of $F_0\left(x\right)$ and $F_1\left(x\right)$ subject to the constraint that the $AUC$ is fixed at $AU{C}_0$. Similarly, let ${\widehat{F}}_{0,J_0}\left(x\right)$ and ${\widehat{F}}_{1,J_0}\left(x\right)$ denote the MELEs of $F_0\left(x\right)$ and $F_1\left(x\right)$ subject to the constraint that the Youden index J is fixed at $J_0$. Their numerical calculations will be discussed in Appendix C. Let a be the significance level.

Step 1.

Calculate the test statistic $W_n$ based on the observed data ${\left\{x_i\right\}}_{i=1}^{n_0}$ and ${\left\{y_i\right\}}_{i=1}^{n_1}$.

Step 2.

For $l=1,\dots ,L$, generate l-th boostrap two-sample data as follows:

(a)

If $\widehat{AUC}\le AU{C}_0$ or $\widehat{J}\le J_0$, draw a bootstrap sample of size $n_1$ from ${\widehat{F}}_1\left(x\right)$ and another bootstrap sample of size $n_0$ from ${\widehat{F}}_0\left(x\right)$.

(b)

If $\widehat{AUC}>AU{C}_0$, $\widehat{J}>J_0$, and $W_{n1}>W_{n2}$, draw a sample of size $n_1$ from ${\widehat{F}}_{1,J_0}\left(x\right)$ and another sample of size $n_0$ from ${\widehat{F}}_{0,J_0}\left(x\right)$.

(c)

If $\widehat{AUC}>AU{C}_0$, $\widehat{J}>J_0$, and $W_{n1}<W_{n2}$, draw a sample of size $n_1$ from ${\widehat{F}}_{1,{AUC}_0}\left(x\right)$ and another sample of size $n_0$ from ${\widehat{F}}_{0,{AUC}_0}\left(x\right)$.

Step 3.

For $l=1,\dots ,L$, calculate the test statistic $W_{n,B,l}$ based on the l-th bootstrap two-sample data in Step 2 (using the same method as in Step 1).

Step 4.

Calculate the p-value of $W_n$ as

$$p-\mathrm{value}=\frac{1}{L}\sum _{i=1}^L\mathbb{1}\left(W_{n,B,l}>W_n\right).$$

Step 5.

Reject the null hypothesis in (8) at the significance level a if the p-value of $W_n$ is smaller than a.

We note that Cases (a)–(c) in Step 2 correspond to the three cases outlined in Proposition 1(a). For instance, consider Step 2(b), in which $\widehat{AUC}>AU{C}_0$, $\widehat{J}>J_0$, and $W_{n1}>W_{n2}$. By the second case of Proposition 1(a), we set the MELE of J to be $J_0$ under the null model in (8). Hence, we generate the bootstrap two-sample data from ${\widehat{F}}_{1,J_0}\left(x\right)$ and ${\widehat{F}}_{0,J_0}\left(x\right)$.

In our simulation study, we set $L=500$. The simulation results in Section 4 demonstrate that the bootstrap procedure effectively controls the type-I error.

4. Simulation Study

This section employs simulation examples to compare the finite-sample performances of our proposed joint inference procedures and some existing competitors.

4.1. Simulation Parameter Settings

We consider two distributional settings:

(1)

$f_0\sim LN({\mu }_0,{\sigma }_0^2)$ and $f_1\sim LN({\mu }_1,{\sigma }_1^2)$;

(2)

$f_0\sim Beta(a_0,b_0)$ and $f_1\sim Beta(a_1,b_1)$.

Here, $LN(\mu ,{\sigma }^2)$ denotes the lognormal distribution with mean $\mu$ and variance ${\sigma }^2$, both with respect to the log scale; $Beta(a,b)$ denotes the beta distribution with power parameters for x and $(1-x)$ being a and b, respectively.

We comment that Setting (1) corresponds to the case where the model assumption for the Box–Cox method is satisfied, while Setting (2) pertains to the case where this assumption is violated. In both cases, we consider three true values 0.3, 0.5, and 0.7 for the Youden index to cover low, moderate, and high levels of diagnostic accuracy. The details of the parameter settings are given in Table 1.

Table 1. Simulation settings.

Distribution	${\mathrm{AUC}}^{*}$	${\mathit{J}}^{*}$	${\mathit{\mu }}_0$	${\mathit{\sigma }}_0^2$	${\mathit{\mu }}_1$	${\mathit{\sigma }}_1^2$
lognormal	0.707	0.3	0	1	0.77	1
	0.830	0.5	0	1	1.35	1
	0.928	0.7	0	1	2.07	1
Distribution	${\mathit{AUC}}^{*}$	${\mathit{J}}^{*}$	${\mathit{a}}_{\mathbf{0}}$	${\mathit{b}}_{\mathbf{0}}$	${\mathit{a}}_{\mathbf{1}}$	${\mathit{b}}_{\mathbf{1}}$
beta	0.704	0.3	1.5	3	2.77	3
	0.824	0.5	1.5	3	4.25	3
	0.922	0.7	1.5	3	7.09	3

When specifying the parameter settings in Table 1, we first specify the parameter values (Columns 4 and 5) of $f_0$ and keep them fixed. Then, for each $J^{*}$ (Column 3), we choose the parameter values (Columns 6 and 7) of $f_1$ for the designated distribution (Column 1) such that the $J^{*}$ value is achieved. Finally, we calculate the corresponding $AU{C}^{*}$ (Column 2) under the pair of $f_0$ and $f_1$. This approach of selecting parameter values has been used in [1,17,21].

Throughout this section, our proposed joint inference procedures use the correctly specified $\mathbf{q}\left(x\right)=\log \left(x\right)$.

4.2. Simulation for Confidence Regions

We compare the proposed confidence regions for $(AUC,J)$ with four methods from [1] using simulation studies. The six methods are listed below:

• Empirical likelihood method (proposed in (6)), which is denoted as “EL”;
• Bootstrap empirical likelihood method with $L=500$ (proposed in (7)), which is denoted as “BEL”;
• Parametric Box–Cox asymptotic delta method, which is denoted as “AD”;
• Parametric Box–Cox generalized inference approach, which is denoted as “GPQ”;
• Nonparametric bootstrap confidence region, which is denoted as “BTI”;
• Nonparametric bootstrap confidence region with the arcsin-square-root transformation, which is denoted as “BTAT”.

We consider three different combinations of sample sizes: $(n_0,n_1)$ = (50, 50), (100, 100), and (150, 50). This results in nine combinations of parameters and sample sizes for each of the two distributional settings of $f_0$ and $f_1$. We repeat the simulation 2000 times for each combination.

We now examine the behavior of the 95% confidence region of $\mathit{\eta }={(AUC,J)}^{\mathrm{T}}$. The performance of a confidence region is evaluated by the coverage probability (CP) in percentage and area of the confidence region (ACR), which are computed as follows:

$$CP=\frac{1}{L}\sum _{l=1}^L\mathbb{1}({\mathit{\eta }}^{*}\in {\mathcal{I}}^{\left(l\right)})\times 100,ACR=\frac{1}{L}\sum _{l=1}^L{C}^{\left(l\right)},$$

where ${\mathcal{I}}^{\left(l\right)}$ is the confidence region of $\mathit{\eta }$ computed from the l-th simulation run, and $C^{\left(l\right)}$ is the area of the confidence region. The simulation results are presented in Table 2 and Table 3.

Table 2. Summary of CP (%) and ACR (×100, in parentheses) of 95% confidence regions for $(AUC,J)$ under lognormal distributions.

$({\mathit{n}}_0,{\mathit{n}}_1)$	${\mathit{AUC}}^{*}$	${\mathit{J}}^{*}$	EL	BEL	AD	GPQ	BTI	BTAT
(50, 50)	0.707	0.3	$\underset{\left(0.50\right)}{90.20}$	$\underset{\left(0.71\right)}{95.85}$	$\underset{\left(5.08\right)}{98.90}$	$\underset{\left(1.28\right)}{96.70}$	$\underset{\left(3.08\right)}{52.80}$	$\underset{\left(3.18\right)}{55.10}$
	0.830	0.5	$\underset{\left(0.60\right)}{90.80}$	$\underset{\left(0.81\right)}{94.45}$	$\underset{\left(3.00\right)}{97.70}$	$\underset{\left(0.79\right)}{96.40}$	$\underset{\left(2.29\right)}{87.90}$	$\underset{\left(2.35\right)}{89.05}$
	0.928	0.7	$\underset{\left(0.45\right)}{88.55}$	$\underset{\left(0.85\right)}{94.85}$	$\underset{\left(2.52\right)}{96.45}$	$\underset{\left(0.46\right)}{95.50}$	$\underset{\left(1.22\right)}{92.10}$	$\underset{\left(1.22\right)}{93.65}$
(100, 100)	0.707	0.3	$\underset{\left(0.24\right)}{93.40}$	$\underset{\left(0.29\right)}{95.50}$	$\underset{\left(1.94\right)}{99.30}$	$\underset{\left(0.48\right)}{98.20}$	$\underset{\left(1.67\right)}{53.30}$	$\underset{\left(1.71\right)}{54.05}$
	0.830	0.5	$\underset{\left(0.29\right)}{92.65}$	$\underset{\left(0.34\right)}{94.45}$	$\underset{\left(1.39\right)}{98.75}$	$\underset{\left(0.28\right)}{97.85}$	$\underset{\left(1.26\right)}{83.80}$	$\underset{\left(1.28\right)}{84.85}$
	0.928	0.7	$\underset{\left(0.22\right)}{91.35}$	$\underset{\left(0.29\right)}{95.05}$	$\underset{\left(2.67\right)}{97.30}$	$\underset{\left(0.17\right)}{95.70}$	$\underset{\left(0.68\right)}{91.20}$	$\underset{\left(0.69\right)}{92.60}$
(150, 50)	0.707	0.3	$\underset{\left(0.28\right)}{92.60}$	$\underset{\left(0.37\right)}{95.80}$	$\underset{\left(3.81\right)}{99.50}$	$\underset{\left(0.72\right)}{98.20}$	$\underset{\left(2.13\right)}{46.45}$	$\underset{\left(2.18\right)}{48.70}$
	0.830	0.5	$\underset{\left(0.34\right)}{93.10}$	$\underset{\left(0.43\right)}{95.90}$	$\underset{\left(2.58\right)}{99.35}$	$\underset{\left(0.43\right)}{98.05}$	$\underset{\left(1.58\right)}{83.00}$	$\underset{\left(1.61\right)}{84.15}$
	0.928	0.7	$\underset{\left(0.25\right)}{89.85}$	$\underset{\left(0.38\right)}{94.60}$	$\underset{\left(1.87\right)}{97.10}$	$\underset{\left(0.25\right)}{96.60}$	$\underset{\left(0.82\right)}{90.55}$	$\underset{\left(0.83\right)}{92.00}$

Table 3. Summary of CP (%) and ACR (×100, in parentheses) of 95% confidence regions for $(AUC,J)$ under beta distributions.

$({\mathit{n}}_0,{\mathit{n}}_1)$	${\mathit{AUC}}^{*}$	${\mathit{J}}^{*}$	EL	BEL	AD	GPQ	BTI	BTAT
(50, 50)	0.704	0.3	$\underset{\left(0.49\right)}{92.00}$	$\underset{\left(0.64\right)}{96.50}$	$\underset{\left(3.48\right)}{96.70}$	$\underset{\left(1.51\right)}{95.00}$	$\underset{\left(3.15\right)}{60.85}$	$\underset{\left(3.45\right)}{65.80}$
	0.824	0.5	$\underset{\left(0.62\right)}{91.25}$	$\underset{\left(0.80\right)}{94.60}$	$\underset{\left(1.67\right)}{89.85}$	$\underset{\left(1.21\right)}{87.20}$	$\underset{\left(2.44\right)}{87.55}$	$\underset{\left(2.50\right)}{88.65}$
	0.922	0.7	$\underset{\left(0.53\right)}{89.85}$	$\underset{\left(0.93\right)}{96.45}$	$\underset{\left(0.86\right)}{62.90}$	$\underset{\left(0.82\right)}{58.90}$	$\underset{\left(1.42\right)}{90.25}$	$\underset{\left(1.43\right)}{91.80}$
(100, 100)	0.704	0.3	$\underset{\left(0.24\right)}{92.80}$	$\underset{\left(0.27\right)}{94.70}$	$\underset{\left(0.96\right)}{96.30}$	$\underset{\left(0.67\right)}{94.70}$	$\underset{\left(1.72\right)}{59.55}$	$\underset{\left(1.76\right)}{60.10}$
	0.824	0.5	$\underset{\left(0.30\right)}{93.15}$	$\underset{\left(0.34\right)}{94.50}$	$\underset{\left(0.62\right)}{86.10}$	$\underset{\left(0.57\right)}{82.40}$	$\underset{\left(1.35\right)}{86.10}$	$\underset{\left(1.37\right)}{86.70}$
	0.922	0.7	$\underset{\left(0.26\right)}{92.65}$	$\underset{\left(0.32\right)}{95.50}$	$\underset{\left(0.38\right)}{44.40}$	$\underset{\left(0.38\right)}{39.05}$	$\underset{\left(0.81\right)}{92.10}$	$\underset{\left(0.81\right)}{93.90}$
(150, 50)	0.704	0.3	$\underset{\left(0.27\right)}{91.90}$	$\underset{\left(0.33\right)}{94.85}$	$\underset{\left(1.42\right)}{96.05}$	$\underset{\left(0.86\right)}{94.25}$	$\underset{\left(2.03\right)}{58.40}$	$\underset{\left(2.07\right)}{58.60}$
	0.824	0.5	$\underset{\left(0.33\right)}{94.20}$	$\underset{\left(0.39\right)}{96.05}$	$\underset{\left(0.82\right)}{85.90}$	$\underset{\left(0.70\right)}{81.85}$	$\underset{\left(1.50\right)}{85.10}$	$\underset{\left(1.52\right)}{85.75}$
	0.922	0.7	$\underset{\left(0.28\right)}{93.20}$	$\underset{\left(0.34\right)}{95.40}$	$\underset{\left(0.44\right)}{43.10}$	$\underset{\left(0.44\right)}{39.85}$	$\underset{\left(0.81\right)}{93.25}$	$\underset{\left(0.82\right)}{94.75}$

Table 2 presents simulation results at the nominal level of 95% under the lognormal distributional setting. The BEL method improves the coverage of the EL method at the cost of increasing the area size. Nonetheless, the proposed BEL method still demonstrates the most stable and comparable performance in almost all cases with CPs reasonably close to 95% in nearly all scenarios. As sample sizes become larger, the AD method generally overestimates the coverage probabilities, resulting in the largest area of confidence regions among the six methods. The GPQ method is generally comparable to the BEL method, maintaining satisfactory coverage probabilities with a relatively small area, especially when the true value of J is large. This is likely because the model assumption for the Box–Cox method is satisfied in this case. Both the BTI and BTAT methods underestimate the coverage probabilities, especially when the true value of J is 0.3, and they produce similar areas in many cases.

Table 3 presents simulation results for the beta distributional setting at the nominal level of 95%. The EL, BEL, BTI, and BTAT methods exhibit similar trends in the lognormal setting. However, the performance of the AD and GPQ methods is quite different. Both AD and GPQ methods struggle to accurately estimate the CPs when the true value of J is 0.5 or 0.7. The GPQ method exhibits particularly poor performance. For example, when the true value of J is 0.7 and $(n_0,n_1)=(100,100)$, the CP of the GPQ method is only 39.05%.

In summary, when $\mathbf{q}\left(x\right)$ is correctly specified, and the Box–Cox model is satisfied, the BEL and GPQ methods have comparable performance and are better than other methods. When the Box–Cox model is not satisfied, the BEL method performs better than the parametric and nonparametric methods.

4.3. Simulation for Joint Hypothesis Testing

This section presents a simulation study comparing the performance of the proposed bootstrap joint test procedure with $L=500$ (denoted as “BELT”), introduced in Section 3.2, with two recommended joint test methods from [2]:

• Parametric bootstrap joint test method, which is denoted as “PBA”;
• Nonparametric kernel-smoothed-based joint test method, which is denoted as “NKS”.

All tests were carried out at the significance level $a=0.05$. We consider three different combinations of sample sizes: $(n_0,n_1)$ = (50, 50), (75, 75), and (75, 50). For both lognormal and beta distributional settings, the model with $J=0.5$ is chosen as the null model and that with $J=0.7$ is chosen as the alternative model. The number of replications is 2000. The simulated type-I errors and powers of three tests are shown in Table 4 and Table 5.

Table 4. Simulated type-I errors and powers of three tests for testing $H_0:AUC\le 0.830$ or $J\le 0.5$ versus $H_a:AUC>0.830$ and $J>0.5$ under lognormal distributions at the 0.05 significance level.

$({\mathit{AUC}}^{*},{\mathit{J}}^{*})$	$({\mathit{n}}_0,{\mathit{n}}_1)$	BELT	PBA	NKS
(0.830, 0.5)	(50, 50)	5.60	2.65	0.25
	(75, 50)	5.60	2.40	0.10
	(75, 75)	5.50	2.35	0.05
(0.928, 0.7)	(50, 50)	90.85	58.35	24.45
	(75, 50)	96.35	66.25	23.10
	(75, 75)	98.25	70.75	37.25

Table 5. Simulated type-I errors and powers of three tests for testing $H_0:AUC\le 0.824$ or $J\le 0.5$ versus $H_a:AUC>0.824$ and $J>0.5$ under beta distributions at the 0.05 significance level.

$({\mathit{AUC}}^{*},{\mathit{J}}^{*})$	$({\mathit{n}}_0,{\mathit{n}}_1)$	BELT	PBA	NKS
(0.824, 0.5)	(50, 50)	5.65	7.30	3.05
	(75, 50)	5.40	8.75	2.30
	(75, 75)	5.05	7.90	2.45
(0.922, 0.7)	(50, 50)	91.00	94.00	81.55
	(75, 50)	95.70	97.60	90.25
	(75, 75)	97.70	99.05	93.90

The first block of Table 4 presents the simulated type-I errors of three tests under the lognormal distribution. We observe that all three methods maintain type-I error rates at 0.05, although the PBA and NKS methods tend to be conservative. The second block of Table 4 presents the simulated powers of the three tests. The proposed BELT method exhibits the largest power.

Table 5 presents the results for beta distributions. While BELT and NKS effectively control type-I errors, the PBA test’s errors significantly exceed 5%. When data come from the alternative model, BELT exhibits greater power compared to NKS.

In conclusion, the BELT method effectively controls type-I errors across both distributional settings (lognormal and beta) and demonstrates superior power compared to the nonparametric method.

5. Real Data Analysis

In this section, we evaluate the performance of the proposed methods using a dataset on Duchenne Muscular Dystrophy (DMD). DMD is a genetic disorder characterized by progressive muscle weakness and wasting. It is caused by mutations in the dystrophin gene, the largest human gene, located on the X chromosome (Xp21). DMD primarily affects males in early childhood. Interestingly, females with one copy of the mutated gene typically do not show symptoms. Therefore, identifying potential female carriers is crucial.

According to [26], individuals carrying the DMD gene mutation may not exhibit symptoms but often have elevated levels of specific biomarkers. The authors of [27] compiled a dataset encompassing four biomarkers: Creatine Kinase (CK), Hemopexin (H), Lactate Dehydrogenase (LD), and Pyruvate Kinase (PK). These biomarkers were measured in blood serum samples from a healthy control group ($n_0$ = 127) and a group of DMD carriers ($n_1$ = 67).

For illustration, we consider the biomarkers PK and H. We choose $\mathbf{q}\left(x\right)=\log \left(x\right)$ in the proposed methods for each biomarker. We perform the goodness-of-fit test suggested in Remark 1; the p-values based on 1000 bootstrap samples are 0.215 and 0.780 for PK and H, respectively. This suggests that the DRM in (2) with $\mathbf{q}\left(x\right)=\log \left(x\right)$ provides reasonable fits for both biomarkers PK and H.

Table 6 presents the point estimates (PEs) of $(AUC,J)$ and the ACRs for $(AUC,J)$ at the 95% confidence level based on the BEL, GPQ, and BTAT methods. We omit the results for the EL, AD, and BTI methods because, as shown in Section 4, the BEL method achieves better coverage than the EL method, the AD method has larger ACRs compared to the GPQ method, and the BTI method performs similarly to the BTAT method. Clearly, the BEL method gives the coverage region with the smallest area. As an illustration, we further plot the 95% confidence regions of $(AUC,J)$ for the biomarker H based on the BEL, GPQ, and BTAT methods in Figure 1, which demonstrates similar observations as in Table 6.

Table 6. Point estimates (PEs) of $(AUC,J)$ and the ACRs(×100) for $(AUC,J)$ at the 95% confidence level based on the BEL, GPQ, and BTAT methods.

Biomarker	$(\mathit{AUC},\mathit{J})$	BEL	GPQ	BTAT
PK	PE	$(0.824,0.483)$	$(0.803,0.481)$	$(0.814,0.511)$
	ACR	$0.442$	$0.836$	$1.743$
H	PE	$(0.753,0.371)$	$(0.748,0.374)$	$(0.760,0.421)$
	ACR	$0.332$	$0.573$	$1.772$

Figure 1. The 95% confidence regions of $(AUC,J)$ for biomarker H in the DMD dataset based on the BEL (black solid), GPQ (red dashed), and BTAT (blue dot–dashed) methods. The BEL point estimate of $(AUC,J)$ is $(0.753,0.371)$, which is indicated by the black hollow point at the center of the ellipse.

To illustrate the proposed joint test method BELT, we assess whether the biomarker PK simultaneously exceeds the prespecified target values of $AU{C}_0=0.753$ and $J_0=0.371$ simultaneously. These values represent the PE of $(AUC,J)$ based on the BEL method for biomarker H. Our BELT method with $L=500$ gives the p-value 0.022. This result provides strong evidence to reject the null hypothesis $H_0:AUC\le 0.753$ or $J\le 0.371$ at the 5% significance level. In contrast, applying both the PBA and NKS tests from [2] fails to reject the same null hypothesis. In conclusion, our BELT method provides stronger evidence against the null hypothesis $H_0:AUC\le 0.753$ or $J\le 0.371$. This implies that the biomarker PK has better discriminatory ability than biomarker H in terms of both $AUC$ and the Youden index J.

6. Summary

In this paper, we proposed a bootstrap confidence region for $(AUC,J)$ and a bootstrap joint testing procedure for the hypothesis testing problem in (8) based on the MELE of $(AUC,J)$. We conducted extensive simulations to evaluate the performance of our proposed semiparametric approaches. The results demonstrate that the BEL method accurately constructs confidence regions for $(AUC,J)$ with the desired coverage probability. Additionally, the proposed bootstrap testing method, BELT, consistently maintains the type-I error rate and exhibits satisfactory power compared to existing joint tests.

Theoretically, we established the joint asymptotic normality of the MELE of $(AUC,J)$, providing the theoretical foundation for the proposed confidence region and joint testing procedure for $(AUC,J)$. Practically, we developed R functions to implement the proposed methods, which are available in the Supplementary Materials.

To use the proposed methods, it is necessary to specify $\mathbf{q}\left(x\right)$ in (2). Common choices for $\mathbf{q}\left(x\right)$ include $\mathbf{q}\left(x\right)=\log \left(x\right)$ and $\mathbf{q}\left(x\right)=x$. We recommend that practitioners first use the goodness-of-fit test described in Remark 1 to assess the suitability of a prespecified choice of $\mathbf{q}\left(x\right)$. The R function for implementing the goodness-of-fit test for the DRM with commonly used $\mathbf{q}\left(x\right)$ is included in the Supplementary Materials. If practitioners do not have a suitable choice for $\mathbf{q}\left(x\right)$, the nonparametric method outlined in Section 4 may be preferable.

7. Discussion

We observe that the proposed methods have the potential to be applied to other research problems. For example, we may extend them to compare paired or multiple markers based on both $AUC$ and the Youden index J [28]. This paper considers two-sample/two-group data only. Multiple sample/group data are also commonly seen [29], and we may explore the proposed methods in this scenario. Furthermore, although widely used, the $AUC$ has its limitations. A major drawback is that it summarizes the entire ROC curve, including regions that may not be directly relevant to clinical applications. To address this issue while retaining some of the $AUC$’s beneficial properties, one can use the partial area under the ROC curve ($pAUC$). Considering a clinically relevant range of false-positive or true-positive rates, the $pAUC$ focuses on a specific portion of the curve [30,31,32,33,34]. We can extend the proposed method to study statistical inference for the $pAUC$. We leave these research problems for future investigation.

Appendix A.1. Some Preparation

Recall that $\omega \left(x\right)=exp\left\{{\left({\mathit{\theta }}^{*}\right)}^{\mathrm{T}}\mathbf{Q}\left(x\right)\right\}$ and $h\left(x\right)=1-\rho +\rho \omega \left(x\right)$. The following lemma is helpful for our subsequent calculation, in which we use $E_0$ to denote the expectation with respect to $F_0\left(x\right)$.

Lemma A1. 

Let $g_1\left(x\right)$ and $g_2\left(x\right)$ be arbitrary functions of x such that the expectations below are all finite. Then, we have

$$\begin{array}{ccc}\hfill E\left\{\sum _{i=1}^{n_1}{g}_1\left(y_i\right)\right\}& =& n\rho E_0\left\{\omega \left(Z\right)g_1\left(Z\right)\right\},\hfill \end{array}$$

(A1)

$$\begin{array}{ccc}\hfill E\left\{\sum _{i=1}^n{g}_1\left(z_i\right)\right\}& =& n{E}_0\left\{h\left(Z\right)g_1\left(Z\right)\right\},\hfill \end{array}$$

(A2)

$$\begin{array}{ccc}\hfill Cov\left(\sum _{i=1}^{n_1}{g}_1\left(y_i\right),\sum _{i=1}^{n_1}{g}_2\left(y_i\right)\right)& =& n\rho E_0\left\{\omega \left(Z\right)g_1\left(Z\right)g_2\left(Z\right)\right\}\hfill \\ & & -n\rho E_0\left\{\omega \left(Z\right)g_1\left(Z\right)\right\}{E}_0\left\{\omega \left(Z\right)g_2\left(Z\right)\right\},\hfill \end{array}$$

(A3)

$$\begin{array}{ccc}\hfill Cov\left(\sum _{i=1}^n{g}_1\left(z_i\right),\sum _{i=1}^n{g}_2\left(z_i\right)\right)& =& n{E}_0\left\{h\left(Z\right)g_1\left(Z\right)g_2\left(Z\right)\right\}\hfill \\ & & -n\rho E_0\left\{\omega \left(Z\right)g_1\left(Z\right)\right\}{E}_0\left\{\omega \left(Z\right)g_2\left(Z\right)\right\}\hfill \\ & & -n(1-\rho )E_0\left\{g_1\left(Z\right)\right\}{E}_0\left\{g_2\left(Z\right)\right\}.\hfill \end{array}$$

(A4)

Proof. 

We first consider (A1). Using the DRM (2), we have

$$\begin{array}{c}\hfill E\left\{\sum _{i=1}^{n_1}{g}_1\left(y_i\right)\right\}=n_{1}E\left\{g_1\left(y_1\right)\right\}=n_1{\int }_{-\infty }^{\infty }{g}_1\left(z\right)\omega \left(z\right)d{F}_0\left(z\right)=n\rho E_0\left\{\omega \left(Z\right)g_1\left(Z\right)\right\}.\end{array}$$

Equation (A2) can be similarly proved.

Next, we consider (A3). Note that

$$\begin{array}{ccc}\hfill Cov\left(\sum _{i=1}^{n_1}{g}_1\left(y_i\right),\sum _{i=1}^{n_1}{g}_2\left(y_i\right)\right)& =& n_{1}Cov(g_1\left(y_1\right),g_2\left(y_1\right))\hfill \\ & =& n\rho E_0\left\{\omega \left(Z\right)g_1\left(Z\right)g_2\left(Z\right)\right\}\hfill \\ & & -n\rho E_0\left\{\omega \left(Z\right)g_1\left(Z\right)\right\}{E}_0\left\{\omega \left(Z\right)g_2\left(Z\right)\right\}.\hfill \end{array}$$

Equation (A4) can be similarly proved. □

Next, we rewrite the ${\mathbf{U}}_n$ in (5) as

$${\mathbf{U}}_n=\left(\begin{array}{c}{\mathbf{U}}_{n1}\\ U_{n2}\\ U_{n3}\\ U_{n4}\\ U_{n5}\end{array}\right)=\left(\begin{array}{c}-n^{-1}\sum _{i=1}^n{h}_1\left(z_i\right)\mathbf{Q}\left(z_i\right)+n^{-1}\sum _{i=n_0+1}^n\mathbf{Q}\left(z_i\right)\\ {\left\{n(1-\rho )\right\}}^{-1}\sum _{i=1}^n{h}_0\left(z_i\right)F_1\left(z_i\right)-(1-AU{C}^{*})\\ {\left(n\rho \right)}^{-1}\sum _{i=1}^n{h}_1\left(z_i\right)F_0\left(z_i\right)-AU{C}^{*}\\ {\left\{n(1-\rho )\right\}}^{-1}\sum _{i=1}^n{h}_0\left(z_i\right)\mathbb{1}\left(z_i\le c^{*}\right)-F_0\left(c^{*}\right)\\ {\left(n\rho \right)}^{-1}\sum _{i=1}^n{h}_1\left(z_i\right)\mathbb{1}\left(z_i\le c^{*}\right)-F_1\left(c^{*}\right)\end{array}\right).$$

(A5)

The next lemma presents the expectation and variance of ${\mathbf{U}}_n$.

Lemma A2. 

For ${\mathbf{U}}_n$ defined in (A5), we have

$$E\left({\mathbf{U}}_n\right)=\mathbf{0}and\mathbb{V}\mathrm{ar}\left({\mathbf{U}}_n\right)=n^{-1}\mathbf{V},$$

where

$$\mathbf{V}=\left(\begin{array}{ccccc}{\mathbf{V}}_{11}& {\mathbf{V}}_{12}& {\mathbf{V}}_{13}& {\mathbf{V}}_{14}& {\mathbf{V}}_{15}\\ {\mathbf{V}}_{21}& V_{22}& V_{23}& V_{24}& V_{25}\\ {\mathbf{V}}_{31}& V_{32}& V_{33}& V_{34}& V_{35}\\ {\mathbf{V}}_{41}& V_{42}& V_{43}& V_{44}& V_{45}\\ {\mathbf{V}}_{51}& V_{52}& V_{53}& V_{54}& V_{55}\end{array}\right),$$

with

$$\begin{array}{ccc}\hfill {\mathbf{V}}_{11}& =& (1-\rho )E_0\left\{h_1\left(Z\right)\mathbf{Q}\left(Z\right){\mathbf{Q}}^{\mathrm{T}}\left(Z\right)\right\}-\frac{1-\rho }{\rho }{E}_0\left\{h_1\left(Z\right)\mathbf{Q}\left(Z\right)\right\}{E}_0\left\{h_1\left(Z\right){\mathbf{Q}}^{\mathrm{T}}\left(Z\right)\right\},\hfill \\ \hfill {\mathbf{V}}_{21}& =& {\mathbf{V}}_{12}^{\mathrm{T}}=E_0\left[\{1-\omega \left(Z\right)\}{h}_0\left(Z\right)F_1\left(Z\right)\right]E_0\left\{h_1\left(Z\right){\mathbf{Q}}^{\mathrm{T}}\left(Z\right)\right\},\hfill \\ \hfill V_{22}& =& {(1-\rho )}^{-1}{E}_0\left\{h_0\left(Z\right)F_1^2\left(Z\right)\right\}-{\rho }^{-1}{\left[E_0\left\{h_1\left(Z\right)F_1\left(Z\right)\right\}\right]}^2\hfill \\ & & -{(1-\rho )}^{-1}{\left[E_0\left\{h_0\left(Z\right)F_1\left(Z\right)\right\}\right]}^2,\hfill \\ \hfill {\mathbf{V}}_{31}& =& {\mathbf{V}}_{13}^{\mathrm{T}}=\frac{1-\rho }{\rho }{E}_0\left[\{1-\omega \left(Z\right)\}{h}_1\left(Z\right)F_0\left(Z\right)\right]E_0\left\{h_1\left(Z\right){\mathbf{Q}}^{\mathrm{T}}\left(Z\right)\right\},\hfill \\ \hfill V_{32}& =& V_{23}={\rho }^{-1}{E}_0\left\{h_1\left(Z\right)F_1\left(Z\right)F_0\left(Z\right)\right\}-{\rho }^{-1}{E}_0\left\{\omega \left(Z\right)h_1\left(Z\right)F_0\left(Z\right)\right\}{E}_0\left\{h_1\left(Z\right)F_1\left(Z\right)\right\}\hfill \\ & & -{\rho }^{-1}{E}_0\left\{h_1\left(Z\right)F_0\left(Z\right)\right\}{E}_0\left\{h_0\left(Z\right)F_1\left(Z\right)\right\},\hfill \\ \hfill V_{33}& =& {\rho }^{-1}{E}_0\left\{\omega \left(Z\right)h_1\left(Z\right)F_0^2\left(Z\right)\right\}-\frac{1-\rho }{{\rho }^2}{\left[E_0\left\{h_1\left(Z\right)F_0\left(Z\right)\right\}\right]}^2\hfill \\ & & -{\rho }^{-1}{\left[E_0\left\{\omega \left(Z\right)h_1\left(Z\right)F_0\left(Z\right)\right\}\right]}^2,\hfill \\ \hfill {\mathbf{V}}_{41}& =& {\mathbf{V}}_{14}^{\mathrm{T}}=E_0\left[\{1-\omega \left(Z\right)\}{h}_0\left(Z\right)\mathbb{1}\left(Z\le c^{*}\right)\right]E_0\left\{h_1\left(Z\right){\mathbf{Q}}^{\mathrm{T}}\left(Z\right)\right\},\hfill \\ \hfill V_{42}& =& V_{24}={(1-\rho )}^{-1}{E}_0\left\{h_0\left(Z\right)F_1\left(Z\right)\mathbb{1}\left(Z\le c^{*}\right)\right\}\hfill \\ & & -{\rho }^{-1}{E}_0\left\{h_1\left(Z\right)\mathbb{1}\left(Z\le c^{*}\right)\right\}{E}_0\left\{h_1\left(Z\right)F_1\left(Z\right)\right\}\hfill \\ & & -{(1-\rho )}^{-1}{E}_0\left\{h_0\left(Z\right)\mathbb{1}\left(Z\le c^{*}\right)\right\}{E}_0\left\{h_0\left(Z\right)F_1\left(Z\right)\right\},\hfill \\ \hfill V_{43}& =& V_{34}={\rho }^{-1}{E}_0\left\{h_1\left(Z\right)F_0\left(Z\right)\mathbb{1}\left(Z\le c^{*}\right)\right\}\hfill \\ & & -{\rho }^{-1}{E}_0\left\{\omega \left(Z\right)h_1\left(Z\right)F_0\left(Z\right)\right\}{E}_0\left\{h_1\left(Z\right)\mathbb{1}\left(Z\le c^{*}\right)\right\}\hfill \\ & & -{\rho }^{-1}{E}_0\left\{h_1\left(Z\right)F_0\left(Z\right)\right\}{E}_0\left\{h_0\left(Z\right)\mathbb{1}\left(Z\le c^{*}\right)\right\},\hfill \\ \hfill V_{44}& =& {(1-\rho )}^{-1}{E}_0\left\{h_0\left(Z\right)\mathbb{1}\left(Z\le c^{*}\right)\right\}-{\rho }^{-1}{\left[E_0\left\{h_1\left(Z\right)\mathbb{1}\left(Z\le c^{*}\right)\right\}\right]}^2\hfill \\ & & -{(1-\rho )}^{-1}{\left[E_0\left\{h_0\left(Z\right)\mathbb{1}\left(Z\le c^{*}\right)\right\}\right]}^2,\hfill \\ \hfill {\mathbf{V}}_{51}& =& {\mathbf{V}}_{15}^{\mathrm{T}}=\frac{1-\rho }{\rho }{E}_0\left[\{1-\omega \left(Z\right)\}{h}_1\left(Z\right)\left\{\mathbb{1}\left(Z\le c^{*}\right)\right\}\right]E_0\left\{h_1\left(Z\right){\mathbf{Q}}^{\mathrm{T}}\left(Z\right)\right\},\hfill \\ \hfill V_{52}& =& V_{25}={\rho }^{-1}{E}_0\left\{h_1\left(Z\right)F_1\left(Z\right)\mathbb{1}\left(Z\le c^{*}\right)\right\}\hfill \\ & & -{\rho }^{-1}{E}_0\left\{\omega \left(Z\right)h_1\left(Z\right)\mathbb{1}\left(Z\le c^{*}\right)\right\}{E}_0\left\{h_1\left(Z\right)F_1\left(Z\right)\right\}\hfill \\ & & -{\rho }^{-1}{E}_0\left\{h_1\left(Z\right)\mathbb{1}\left(Z\le c^{*}\right)\right\}{E}_0\left\{h_0\left(Z\right)F_1\left(Z\right)\right\},\hfill \\ \hfill V_{53}& =& V_{35}={\rho }^{-1}{E}_0\left\{\omega \left(Z\right)h_1\left(Z\right)F_0\left(Z\right)\mathbb{1}\left(Z\le c^{*}\right)\right\}\hfill \\ & & -\frac{1-\rho }{{\rho }^2}{E}_0\left\{h_1\left(Z\right)\mathbb{1}\left(Z\le c^{*}\right)\right\}{E}_0\left\{h_1\left(Z\right)F_0\left(Z\right)\right\}\hfill \\ & & -{\rho }^{-1}{E}_0\left\{\omega \left(Z\right)h_1\left(Z\right)\mathbb{1}\left(Z\le c^{*}\right)\right\}{E}_0\left\{\omega \left(Z\right)h_1\left(Z\right)F_0\left(Z\right)\right\},\hfill \\ \hfill V_{54}& =& V_{45}={\rho }^{-1}{E}_0\left\{h_1\left(Z\right)\mathbb{1}\left(Z\le c^{*}\right)\right\}\hfill \\ & & -{\rho }^{-1}{E}_0\left\{\omega \left(Z\right)h_1\left(Z\right)\mathbb{1}\left(Z\le c^{*}\right)\right\}{E}_0\left\{h_1\left(Z\right)\mathbb{1}\left(Z\le c^{*}\right)\right\}\hfill \\ & & -{\rho }^{-1}{E}_0\left\{h_1\left(Z\right)\mathbb{1}\left(Z\le c^{*}\right)\right\}{E}_0\left\{h_0\left(Z\right)\mathbb{1}\left(Z\le c^{*}\right)\right\},\hfill \\ \hfill V_{55}& =& {\rho }^{-1}{E}_0\left\{\omega \left(Z\right)h_1\left(Z\right)\mathbb{1}\left(Z\le c^{*}\right)\right\}-\frac{1-\rho }{{\rho }^2}{\left[E_0\left\{h_1\left(Z\right)\mathbb{1}\left(Z\le c^{*}\right)\right\}\right]}^2\hfill \\ & & -{\rho }^{-1}{\left[E_0\left\{\omega \left(Z\right)h_1\left(Z\right)\mathbb{1}\left(Z\le c^{*}\right)\right\}\right]}^2.\hfill \end{array}$$

Proof. 

For $E\left({\mathbf{U}}_n\right)$, we show only that $E\left(U_{n2}\right)=0$. The other parts can be verified using Lemma A1 directly. Recall that

$$U_{n2}={\left\{n(1-\rho )\right\}}^{-1}\sum _{i=1}^n{h}_0\left(z_i\right)F_1\left(z_i\right)-(1-AU{C}^{*}).$$

Using (A2), we have that

$$\begin{array}{ccc}\hfill E\left(U_{n2}\right)& =& \frac{n}{n(1-\rho )}{E}_0\left\{h\left(Z\right)h_0\left(Z\right)F_1\left(Z\right)\right\}-(1-AU{C}^{*})\hfill \\ & =& E_0\left\{F_1\left(Z\right)\right\}-(1-AU{C}^{*})=0.\hfill \end{array}$$

For $Var\left({\mathbf{U}}_n\right)$, we only verify

$$\begin{array}{c}\hfill {\mathbf{V}}_{31}=nCov(U_{n3},{\mathbf{U}}_{n1})=\frac{1-\rho }{\rho }{E}_0\left[\{1-\omega \left(Z\right)\}{h}_1\left(Z\right)F_0\left(Z\right)\right]E_0\left\{h_1\left(Z\right){\mathbf{Q}}^{\mathrm{T}}\left(Z\right)\right\}.\end{array}$$

The other parts, again, can be similarly checked.

Note that

$$\begin{array}{ccc}& & Cov(U_{n3},{\mathbf{U}}_{n1})\hfill \\ & & =Cov\left(\frac{1}{n\rho }\sum _{i=1}^n{h}_1\left(z_i\right)F_0\left(z_i\right),-\frac{1}{n}\sum _{i=1}^n{h}_1\left(z_i\right)\mathbf{Q}\left(z_i\right)+\frac{1}{n}\sum _{i=n_0+1}^n\mathbf{Q}\left(z_i\right)\right)\hfill \\ \hfill & & =-\frac{1}{n^2\rho }Cov\left(\sum _{i=1}^n{h}_1\left(z_i\right)F_0\left(z_i\right),\sum _{i=1}^n{h}_1\left(z_i\right)\mathbf{Q}\left(z_i\right)\right)\hfill \\ & & +\frac{1}{n^2\rho }Cov\left(\sum _{i=n_0+1}^n{h}_1\left(z_i\right)F_0\left(z_i\right),\sum _{i=n_0+1}^n\mathbf{Q}\left(z_i\right)\right).\hfill \end{array}$$

(A6)

By Lemma A1, we have

$$\begin{array}{ccc}\hfill & & Cov\left(\sum _{i=n_0+1}^n{h}_1\left(z_i\right)F_0\left(z_i\right),\sum _{i=n_0+1}^n\mathbf{Q}\left(z_i\right)\right)\hfill \\ & =& n\rho E_0\left\{\omega \left(Z\right)h_1\left(Z\right)F_0\left(Z\right)\mathbf{Q}{\left(Z\right)}^{\mathrm{T}}\right\}\hfill \\ & & -n\rho E_0\left\{\omega \left(Z\right)h_1\left(Z\right)F_0\left(Z\right)\right\}{E}_0\left\{\omega \left(Z\right)\mathbf{Q}{\left(Z\right)}^{\mathrm{T}}\right\}\hfill \end{array}$$

(A7)

and

$$\begin{array}{ccc}& & Cov\left(\sum _{i=1}^n{h}_1\left(z_i\right)F_0\left(z_i\right),\sum _{i=1}^n{h}_1\left(z_i\right)\mathbf{Q}\left(z_i\right)\right)\hfill \\ & =& n\rho E_0\left\{\omega \left(Z\right)h_1\left(Z\right)F_0\left(Z\right)\mathbf{Q}{\left(Z\right)}^{\mathrm{T}}\right\}\hfill \\ & & -n\rho E_0\left\{\omega \left(Z\right)h_1\left(Z\right)F_0\left(Z\right)\right\}{E}_0\left\{\omega \left(Z\right)h_1\left(Z\right)\mathbf{Q}{\left(Z\right)}^{\mathrm{T}}\right\}\hfill \\ & & -n(1-\rho )E_0\left\{h_1\left(Z\right)F_0\left(Z\right)\right\}{E}_0\left\{h_1\left(Z\right)\mathbf{Q}{\left(Z\right)}^{\mathrm{T}}\right\}.\hfill \end{array}$$

(A8)

Combining (A6)–(A8), after some algebraic manipulation, we obtain

$$Cov(U_{n3},{\mathbf{U}}_{n1})=\frac{1-\rho }{n\rho }{E}_0\left[\{1-\omega \left(Z\right)\}{h}_1\left(Z\right)F_0\left(Z\right)\right]E_0\left\{h_1\left(Z\right){\mathbf{Q}}^{\mathrm{T}}\left(Z\right)\right\}.$$

This finishes the proof. □

Appendix A.2. Proof of Theorem 1

Proof. 

Recall that the MELEs of $p_i$ are given by

$${\widehat{p}}_i=\frac{1}{n}\frac{1}{1-\rho +\rho exp\left\{{\widehat{\mathit{\theta }}}^{\mathrm{T}}\mathbf{Q}\left(z_i\right)\right\}},i=1,\cdots ,n.$$

Let

$${\widehat{q}}_i={\widehat{p}}_{i}exp\left\{{\widehat{\mathit{\theta }}}^{\mathrm{T}}\mathbf{Q}\left(z_i\right)\right\}.$$

It follows that the MELEs of $F_0$ and $F_1$ are given by

$$\begin{array}{c}\hfill {\widehat{F}}_0\left(x\right)=\sum _{i=1}^n{\widehat{p}}_i\mathbb{1}\left(z_i\le x\right)and{\widehat{F}}_1\left(x\right)=\sum _{i=1}^n{\widehat{q}}_i\mathbb{1}\left(z_i\le x\right).\end{array}$$

The joint asymptotic normality of $(\widehat{AUC},\widehat{J})$ relies on linear approximations of $(\widehat{AUC},\widehat{J})$. According to [4,17], we have

$$\begin{array}{ccc}\hfill \widehat{AUC}-AU{C}^{*}& =& \sum _{i=1}^n{\widehat{q}}_i{F}_0\left(z_i\right)+\sum _{i=1}^n{\widehat{p}}_i\{1-F_1\left(z_i\right)\}-2AU{C}^{*}+o_p\left(n^{-1/2}\right)\hfill \\ & =& \left\{\sum _{i=1}^n{\widehat{q}}_i{F}_0\left(z_i\right)-AU{C}^{*}\right\}\hfill \\ & & -\left\{\sum _{i=1}^n{\widehat{p}}_i{F}_1\left(z_i\right)-(1-AU{C}^{*})\right\}+o_p\left(n^{-1/2}\right)\hfill \end{array}$$

(A9)

and

$$\begin{array}{ccc}\hfill \widehat{J}-J^{*}& =& \{{\widehat{F}}_0\left(c^{*}\right)-F_0\left(c^{*}\right)\}-\{{\widehat{F}}_1\left(c^{*}\right)-F_1\left(c^{*}\right)\}+o_p\left(n^{-1/2}\right)\hfill \\ & =& \left\{\sum _{i=1}^n{\widehat{p}}_i\mathbb{1}\left(z_i\le c^{*}\right)-F_0\left(c^{*}\right)\right\}-\left\{\sum _{i=1}^n{\widehat{q}}_i\mathbb{1}\left(z_i\le c^{*}\right)-F_1\left(c^{*}\right)\right\}+o_p\left(n^{-1/2}\right).\hfill \end{array}$$

Recall the form of ${\mathbf{U}}_{\mathbf{n}}$ in (A5). Applying a first-order Taylor expansion to (A9), we have

$$\begin{array}{ccc}\hfill \widehat{AUC}-AU{C}^{*}& =& -U_{n2}+U_{n3}+\frac{1}{n\rho }\sum _{i=1}^n{h}_0\left(z_i\right)h_1\left(z_i\right)F_0\left(z_i\right)\mathbf{Q}{\left(z_i\right)}^{\mathrm{T}}\left(\widehat{\mathit{\theta }}-{\mathit{\theta }}^{*}\right)\hfill \\ & & +\frac{1}{n(1-\rho )}\sum _{i=1}^n{h}_0\left(z_i\right)h_1\left(z_i\right)F_1\left(z_i\right)\mathbf{Q}{\left(z_i\right)}^{\mathrm{T}}\left(\widehat{\mathit{\theta }}-{\mathit{\theta }}^{*}\right)\hfill \\ & & +o_p\left(n^{-1/2}\right).\hfill \end{array}$$

Using the weak law of large numbers and Lemma A1, we further obtain

$$\begin{array}{ccc}\hfill \widehat{AUC}-AU{C}^{*}& =& -U_{n2}+U_{n3}+\frac{1-\rho }{\rho }{E}_0{\left\{h_1\left(Z\right)F_0\left(Z\right)\mathbf{Q}\left(Z\right)\right\}}^{\mathrm{T}}\left(\widehat{\mathit{\theta }}-{\mathit{\theta }}^{*}\right)\hfill \\ & & +E_0{\left\{h_1\left(Z\right)F_1\left(Z\right)\mathbf{Q}\left(Z\right)\right\}}^{\mathrm{T}}\left(\widehat{\mathit{\theta }}-{\mathit{\theta }}^{*}\right)\hfill \\ & & +o_p\left(n^{-1/2}\right).\hfill \end{array}$$

(A10)

By [19], we have

$$\widehat{\mathit{\theta }}-{\mathit{\theta }}^{*}={(1-\rho )}^{-1}{\mathbf{A}}^{-1}{\mathbf{U}}_{n1}+o_p\left(n^{-1/2}\right),$$

which, together with (A10), implies that

$$\begin{array}{ccc}\hfill \widehat{AUC}-AU{C}^{*}& =& -U_{n2}+U_{n3}+\frac{1}{\rho }{E}_0{\left\{h_1\left(Z\right)F_0\left(Z\right)\mathbf{Q}\left(Z\right)\right\}}^{\mathrm{T}}{\mathbf{A}}^{-1}{\mathbf{U}}_{n1}\hfill \\ & & +\frac{1}{1-\rho }{E}_0{\left\{h_1\left(Z\right)F_1\left(Z\right)\mathbf{Q}\left(Z\right)\right\}}^{\mathrm{T}}{\mathbf{A}}^{-1}{\mathbf{U}}_{n1}+o_p\left(n^{-1/2}\right).\hfill \end{array}$$

(A11)

Similarly, we obtain

$$\begin{array}{ccc}\hfill & & \widehat{J}-J^{*}\hfill \\ \hfill & =& U_{n4}-U_{n5}-\frac{1}{\rho }{E}_0{\left\{h_1\left(Z\right)\mathbb{1}(z_i\le c^{*})\mathbf{Q}\left(Z\right)\right\}}^{\mathrm{T}}\left(\widehat{\mathit{\theta }}-{\mathit{\theta }}^{*}\right)+o_p\left(n^{-1/2}\right).\hfill \\ & =& U_{n4}-U_{n5}-\frac{1}{\rho (1-\rho )}{E}_0{\left\{h_1\left(Z\right)\mathbb{1}(z_i\le c^{*})\mathbf{Q}\left(Z\right)\right\}}^{\mathrm{T}}{\mathbf{A}}^{-1}{\mathbf{U}}_{n1}+o_p\left(n^{-1/2}\right).\hfill \end{array}$$

(A12)

Combining (A11) and (A12) leads to

$$\begin{array}{c}\hfill \sqrt{n}\left(\begin{array}{c}\widehat{AUC}-AU{C}^{*}\\ \widehat{J}-J^{*}\end{array}\right)=\sqrt{n}\mathbf{H}{\mathbf{U}}_n+o_p\left(1\right).\end{array}$$

Using the central limit theorem, Lemma A2, and Slutsky’s theorem, we have

$$\begin{array}{c}\hfill \sqrt{n}\left(\begin{array}{c}\widehat{AUC}-AU{C}^{*}\\ \widehat{J}-J^{*}\end{array}\right)\to \mathrm{N}\left(\mathbf{0},\mathsf{\Sigma }=\mathbf{H}\mathbf{V}{\mathbf{H}}^{\mathrm{T}}\right)\end{array}$$

in distribution, as claimed in Theorem 1. This finishes the proof. □