# Estimation of the Average Kappa Coefficient of a Binary Diagnostic Test in the Presence of Partial Verification

^{1}

^{2}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Weighted Kappa Coefficient and Average Kappa Coefficient

#### 2.1. Weighted Kappa Coefficient

#### 2.2. Average Kappa Coefficient

- If $Se=Sp=1$ then ${\kappa}_{1}={\kappa}_{2}=1$, and if $Se=1-Sp$ then ${\kappa}_{1}={\kappa}_{2}=0$. Therefore $0\le {\kappa}_{i}\le 1$, $i=1,2$.
- Coefficient ${\kappa}_{1}$ is greater than ${\kappa}_{2}$ if $p>Q$, and ${\kappa}_{1}$ is lower than ${\kappa}_{2}$ if $Q>p$.
- ${\kappa}_{1}$ minimizes the expression $2{\displaystyle {\int}_{0}^{0.5}{\left\{\kappa \left(c\right)-x\right\}}^{2}dc}$ and ${\kappa}_{2}$ minimizes the expression $2{\displaystyle {\int}_{0.5}^{1}{\left\{\kappa \left(c\right)-x\right\}}^{2}dc}$. Therefore, when $x={\kappa}_{1}$ $\left(x={\kappa}_{2}\right)$ the first (second) expression is the variance of the weighted kappa coefficients around ${\kappa}_{1}$ $\left({\kappa}_{2}\right)$.
- For fixed values of $\kappa \left(0\right)$ and $\kappa \left(1\right)$ (or Se, Sp and p), the weighted kappa coefficient is a function of $c$ which is continuous in the interval $\left[0,1\right]$. Therefore, the average kappa coefficient ${\kappa}_{i}$ coincides with a value of the weighted kappa coefficient in the interval $\left[0,1\right]$. This value of the weighted coefficient kappa has a value of weighting index $c$. So, as ${\kappa}_{i}=\kappa \left(c\right)$ for some value of $c$, from Equation (1) and for a specific sample it is possible to calculate a value of the weighting index $c$ associated to the estimated average kappa coefficient. Thus, the estimation of the average kappa coefficient allows us to estimate how much greater (or smaller) the loss due to the false negatives is than the loss due to the false positives.

## 3. Estimation with Complete Data

#### 3.1. Wald CI

#### 3.2. Logit CI

#### 3.3. Arcsine CI

## 4. Estimation in the Presence of Partial Verification

#### 4.1. Maximum Likelihood

#### 4.1.1. Wald CI

#### 4.1.2. Logit CI

#### 4.1.3. Arcsine CI

#### 4.2. Multiple Imputation

## 5. Simulation Experiments

- (a)
- With respect to ML, the verification probabilities do not have a clear effect on the coverage probabilities (CPs) of the CIs. With respect to the CIs, in general terms their CPs far exceed 95% when the sample size is small ($n=50$) or moderate ($n=100\u2013200$), fluctuating around 95% when the sample size is large ($n=500\u20131000$). The Wald CI has a CP that fluctuates around 95% when the sample size is moderate or large. The logit CI has a higher CP than that of the Wald CI, especially when the sample size is small or moderate. The arcsine CI can have a CP of less than 90% when the sample size is small and ${\kappa}_{1}$ is small (${\kappa}_{1}=0.2$) and fluctuates around 95% when the sample size is large. In general terms, the Wald CI is the interval with the best performance when the sample size is small or moderate, while all three CIs have a very similar asymptotic behavior when the sample size is large.
- (b)
- With respect to MICE, the verification probabilities do not have a clear effect on the CPs of the CIs. The Wald CI has a coverage probability that exceeds 95% when the sample size is small or moderate and the value of ${\kappa}_{1}$ is small (${\kappa}_{1}=0.2$), fluctuating around 95% in the other situations and sample sizes. The logit CI has a CP that is slightly higher than that of the Wald CI, especially when the sample size is small or moderate. The arcsine CI has a CP closer to 95% when the sample size is small, and in the rest of sample size its CP is slightly higher than that of the Wald CI.
- (c)
- Comparing the CIs obtained by ML and those obtained by MICE, MICE along with the Wald CI presents, in general terms, better fluctuations around 95% than any of the CIs obtained by ML; once MICE, along with the Wald CI, reaches a CP of 95%, it fluctuates very slightly around 95%. Furthermore, in general terms, MICE along with the Wald CI begins to fluctuate around 95% with a sample size smaller than the CIs by ML. Regarding the average lengths, the CIs obtained by ML have an average length slightly less than that of the CIs obtained by applying MICE when the sample size is small or moderate, although the latter show better fluctuations around 95%. The average lengths are very similar when the sample size is large.
- (d)
- Regarding the comparison of the estimators obtained by ML and MICE, relative biases are very similar. Difference (in absolute value) is small (less than 5%) when the sample size is small, and the difference is very small (less than 1%) when the sample size is large. Therefore, ML and MICE provide estimators of ${\kappa}_{1}$ that are, on average, very similar.

## 6. Function Eakcpv

## 7. Example

## 8. Discussion

## Supplementary Materials

## Author Contributions

## Funding

## Institutional Review Board Statement

## Informed Consent Statement

## Data Availability Statement

## Acknowledgments

## Conflicts of Interest

## Appendix A

## Appendix B

## References

- Kraemer, H.C. Evaluating Medical Tests. Objective and Quantitative Guidelines; Sage Publications: Newbury Park, CA, USA, 1992. [Google Scholar]
- Kraemer, H.C.; Periyakoil, V.S.; Noda, A. Kappa coefficients in medical research. Stat. Med.
**2002**, 21, 2109–2129. [Google Scholar] [CrossRef] - Roldán-Nofuentes, J.A.; Olvera-Porcel, C. Average kappa coefficient: A new measure to assess a binary test considering the losses associated with an erroneous classification. J. Stat. Comput. Simul.
**2015**, 85, 1601–1620. [Google Scholar] [CrossRef] - Begg, C.B.; Greenes, R.A. Assessment of diagnostic tests when disease verification is subject to selection bias. Biometrics
**1983**, 39, 207–215. [Google Scholar] [CrossRef] - Zhou, X.H. Maximum likelihood estimators of sensitivity and specificity corrected for verification bias. Comm. Statist. Theory Methods
**1993**, 22, 3177–3198. [Google Scholar] [CrossRef] - Harel, O.; Zhou, X.H. Multiple imputation for correcting verification bias. Stat. Med.
**2006**, 25, 3769–3786. [Google Scholar] [CrossRef] [PubMed] - Alonzo, T.A. Verification bias-impact and methods for correction when assessing accuracy of diagnostic tests. REVSTAT
**2014**, 12, 67–83. [Google Scholar] - Roldán-Nofuentes, J.A.; Luna del Castillo, J.D. Risk of error and the kappa coefficient of a binary diagnostic test in the presence of partial verification. J. Appl. Stat.
**2007**, 34, 887–898. [Google Scholar] [CrossRef] - Youden, W.J. Index for rating diagnostic tests. Cancer
**1950**, 3, 32–35. [Google Scholar] [CrossRef] - Landis, R.; Koch, G. The measurement of observer agreement for categorical data. Biometrics
**1977**, 33, 159–174. [Google Scholar] [CrossRef] [PubMed] [Green Version] - Cicchetti, D.V. The precision of reliability and validity estimates re-visited: Distinguishing between clinical and statistical significance of sample size requirements. J. Clin. Exp. Neuropsychol.
**2001**, 23, 695–700. [Google Scholar] [CrossRef] - Martín-Andrés, A.; Álvarez-Hernández, M. Two-tailed asymptotic inferences for a proportion. J. Appl. Stat.
**2014**, 41, 1516–1529. [Google Scholar] [CrossRef] - Rubin, D.B. Inference and missing data. Biometrika
**1976**, 4, 73–89. [Google Scholar] [CrossRef] - Rubin, D.B. Multiple Imputation for Nonresponse in Surveys; Wiley: New York, NY, USA, 1987. [Google Scholar]
- Rubin, D.B. Multiple Imputation after 18+ years. J. Am. Stat. Assoc.
**1996**, 91, 473–489. [Google Scholar] [CrossRef] - Schafer, J.L. Analysis of Incomplete Multivariate Data; Chapman and Hall: New York, NY, USA, 1997. [Google Scholar]
- Harel, O.; Zhou, X.H. Multiple imputation: Review of theory, implementation and software. Stat. Med.
**2007**, 26, 3057–3077. [Google Scholar] [CrossRef] [Green Version] - Montero-Alonso, M.A.; Roldán-Nofuentes, J.A. Approximate confidence intervals for the likelihood ratios of a binary diagnostic test in the presence of partial disease verification. J. Biopharm. Stat.
**2019**, 29, 56–81. [Google Scholar] [CrossRef] [PubMed] - White, I.R.; Royston, P.; Wood, A.M. Multiple imputation using chained equations: Issues and guidance for practice. Stat. Med.
**2011**, 30, 377–399. [Google Scholar] [CrossRef] [PubMed] - R Core Team. A Language and Environment for Statistical Computing; R Foundation for Statistical Computing: Vienna, Austria, 2016; Available online: https://www.R-project.org/ (accessed on 1 June 2021).
- van Buuren, S.; Groothuis-Oudshoorn, K. Mice: Multivariate imputation by chained equations in R. J. Stat. Softw.
**2011**, 45, 3. [Google Scholar] [CrossRef] [Green Version] - Drum, D.E.; Christacopoulos, J.S. Hepatic scintigraphy in clinical decision making. J. Nucl. Med.
**1972**, 13, 908–915. [Google Scholar] - Kosinski, A.S.; Barnhart, H.X. Accounting for nonignorable verification bias in assessment of diagnostic tests. Biometrics
**2003**, 59, 163–171. [Google Scholar] [CrossRef] - Kosinski, A.S.; Chenb, Y.; Lylesc, R.H. Sample size calculations for evaluating a diagnostic test when the gold standard is missing at random. Stat. Med.
**2010**, 29, 1572–1579. [Google Scholar] [CrossRef] - Zhou, X.H.; Obuchowski, N.; McClish, D. Statistical Methods in Diagnostic Medicine, 2nd ed.; Wiley: New York, NY, USA, 2011. [Google Scholar]
- Zhou, X.H. Effect of verification bias on positive and negative predictive values. Stat. Med.
**1994**, 13, 1737–1745. [Google Scholar] [CrossRef] [PubMed]

$\mathbf{Observed}\mathbf{Frequencies}\mathbf{of}\mathbf{the}2\times 2\mathbf{Table}$ | |||
---|---|---|---|

$T=1$ | $T=0$ | Total | |

$D=1$ | ${x}_{1}$ | ${x}_{0}$ | $x$ |

$D=0$ | ${y}_{1}$ | ${y}_{0}$ | $y$ |

Total | ${m}_{1}$ | ${m}_{0}$ | $m$ |

$\mathbf{Observed}\mathbf{Frequencies}\mathbf{of}\mathbf{the}3\times 2\mathbf{Table}$ | |||
---|---|---|---|

$T=1$ | $T=0$ | Total | |

$V=1$ | |||

$D=1$ | ${s}_{1}$ | ${s}_{0}$ | $s$ |

$D=0$ | ${r}_{1}$ | ${r}_{0}$ | $r$ |

$V=0$ | ${u}_{1}$ | ${u}_{0}$ | $u$ |

Total | ${n}_{1}$ | ${n}_{0}$ | $n$ |

**Table 3.**Coverage probabilities and average lengths of CIs for ${\kappa}_{1}=\left\{0.2,0.4\right\}$.

${\mathit{\kappa}}_{1}=0.2\phantom{\rule{0.5em}{0ex}}\mathit{S}\mathit{e}=0.7773\phantom{\rule{0.5em}{0ex}}\mathit{S}\mathit{p}=0.7308\phantom{\rule{0.5em}{0ex}}\mathit{p}=10\mathit{\%}$ | ||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|

${\lambda}_{1}=0.70\phantom{\rule{0.5em}{0ex}}{\lambda}_{0}=0.25$ | ||||||||||||||

Maximum Likelihood Method | MICE Method | |||||||||||||

n | Relative Bias (%) | Wald CI | Logit CI | Arcsine CI | Relative Bias (%) | Wald CI | Logit CI | Arcsine CI | ||||||

CP | AL | CP | AL | CP | AL | CP | AL | CP | AL | CP | AL | |||

50 | −13.6 | 0.998 | 0.492 | 0.994 | 0.721 | 0.875 | 0.527 | −16.7 | 0.998 | 0.481 | 0.999 | 0.831 | 0.946 | 0.577 |

100 | −11.4 | 0.982 | 0.372 | 0.989 | 0.486 | 0.973 | 0.394 | −14.3 | 0.982 | 0.372 | 0.996 | 0.649 | 0.979 | 0.437 |

200 | −7.5 | 0.960 | 0.276 | 0.988 | 0.302 | 0.985 | 0.277 | −10.6 | 0.954 | 0.293 | 0.996 | 0.413 | 0.993 | 0.315 |

500 | −3.5 | 0.942 | 0.173 | 0.972 | 0.174 | 0.957 | 0.172 | −6.1 | 0.948 | 0.187 | 0.978 | 0.199 | 0.965 | 0.189 |

1000 | −1.8 | 0.948 | 0.121 | 0.963 | 0.122 | 0.956 | 0.121 | −2.4 | 0.948 | 0.128 | 0.971 | 0.131 | 0.962 | 0.129 |

${\lambda}_{1}=0.95\phantom{\rule{0.5em}{0ex}}{\lambda}_{0}=0.40$ | ||||||||||||||

Maximum Likelihood Method | MICE Method | |||||||||||||

n | Relative Bias (%) | Wald CI | Logit CI | Arcsine CI | Relative Bias (%) | Wald CI | Logit CI | Arcsine CI | ||||||

CP | AL | CP | AL | CP | AL | CP | AL | CP | AL | CP | AL | |||

50 | −13.5 | 0.988 | 0.425 | 0.993 | 0.602 | 0.946 | 0.460 | −15.3 | 0.988 | 0.420 | 0.995 | 0.751 | 0.952 | 0.503 |

100 | −9.2 | 0.960 | 0.322 | 0.984 | 0.380 | 0.979 | 0.330 | −11.7 | 0.953 | 0.329 | 0.991 | 0.509 | 0.980 | 0.367 |

200 | −5.6 | 0.953 | 0.232 | 0.988 | 0.239 | 0.981 | 0.230 | −7.8 | 0.948 | 0.242 | 0.993 | 0.279 | 0.987 | 0.246 |

500 | −2.7 | 0.951 | 0.146 | 0.962 | 0.146 | 0.954 | 0.145 | −3.9 | 0.950 | 0.151 | 0.971 | 0.153 | 0.962 | 0.150 |

1000 | −0.5 | 0.947 | 0.102 | 0.956 | 0.102 | 0.953 | 0.102 | −1.1 | 0.951 | 0.104 | 0.956 | 0.105 | 0.953 | 0.104 |

${\kappa}_{1}=0.4\phantom{\rule{0.5em}{0ex}}Se=0.7413\phantom{\rule{0.5em}{0ex}}Sp=0.7441\phantom{\rule{0.5em}{0ex}}p=30\%$ | ||||||||||||||

${\lambda}_{1}=0.70\phantom{\rule{0.5em}{0ex}}{\lambda}_{0}=0.25$ | ||||||||||||||

Maximum Likelihood Method | MICE Method | |||||||||||||

n | Relative Bias (%) | Wald CI | Logit CI | Arcsine CI | Relative Bias (%) | Wald CI | Logit CI | Arcsine CI | ||||||

CP | AL | CP | AL | CP | AL | CP | AL | CP | AL | CP | AL | |||

50 | −18.1 | 0.979 | 0.630 | 0.996 | 0.655 | 0.973 | 0.618 | −20.9 | 0.979 | 0.624 | 0.997 | 0.740 | 0.965 | 0.653 |

100 | −10.1 | 0.963 | 0.476 | 0.994 | 0.470 | 0.988 | 0.464 | −13.8 | 0.952 | 0.499 | 0.995 | 0.554 | 0.973 | 0.509 |

200 | −4.5 | 0.961 | 0.340 | 0.989 | 0.330 | 0.977 | 0.333 | −6.2 | 0.947 | 0.365 | 0.984 | 0.373 | 0.970 | 0.364 |

500 | −1.5 | 0.952 | 0.213 | 0.961 | 0.210 | 0.956 | 0.211 | −2.6 | 0.949 | 0.225 | 0.960 | 0.224 | 0.955 | 0.224 |

1000 | −1.1 | 0.954 | 0.150 | 0.959 | 0.149 | 0.958 | 0.149 | −1.5 | 0.950 | 0.158 | 0.955 | 0.158 | 0.951 | 0.158 |

${\lambda}_{1}=0.95\phantom{\rule{0.5em}{0ex}}{\lambda}_{0}=0.40$ | ||||||||||||||

Maximum Likelihood Method | MICE Method | |||||||||||||

n | Relative Bias (%) | Wald CI | Logit CI | Arcsine CI | Relative Bias (%) | Wald CI | Logit CI | Arcsine CI | ||||||

CP | AL | CP | AL | CP | AL | CP | AL | CP | AL | CP | AL | |||

50 | −15.5 | 0.960 | 0.550 | 0.993 | 0.561 | 0.982 | 0.539 | −18.4 | 0.955 | 0.559 | 0.996 | 0.638 | 0.989 | 0.575 |

100 | −8.2 | 0.955 | 0.405 | 0.985 | 0.393 | 0.980 | 0.395 | −10.9 | 0.950 | 0.421 | 0.991 | 0.431 | 0.985 | 0.418 |

200 | −3.4 | 0.956 | 0.283 | 0.974 | 0.277 | 0.967 | 0.279 | −5.1 | 0.956 | 0.294 | 0.980 | 0.290 | 0.967 | 0.290 |

500 | −1.1 | 0.947 | 0.178 | 0.957 | 0.176 | 0.952 | 0.177 | −1.3 | 0.950 | 0.182 | 0.963 | 0.181 | 0.957 | 0.181 |

1000 | −0.6 | 0.955 | 0.125 | 0.958 | 0.125 | 0.957 | 0.125 | −0.7 | 0.951 | 0.128 | 0.958 | 0.128 | 0.955 | 0.128 |

**Table 4.**Coverage probabilities and average lengths of CIs for ${\kappa}_{1}=\left\{0.6,0.8\right\}$.

${\mathit{\kappa}}_{1}=0.6\phantom{\rule{0.5em}{0ex}}\mathit{S}\mathit{e}=0.6816\phantom{\rule{0.5em}{0ex}}\mathit{S}\mathit{p}=0.8624\phantom{\rule{0.5em}{0ex}}\mathit{p}=50\mathit{\%}$ | ||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|

${\lambda}_{1}=0.70\phantom{\rule{0.5em}{0ex}}{\lambda}_{0}=0.25$ | ||||||||||||||

Maximum Likelihood Method | MICE Method | |||||||||||||

n | Relative Bias (%) | Wald CI | Logit CI | Arcsine CI | Relative Bias (%) | Wald CI | Logit CI | Arcsine CI | ||||||

CP | AL | CP | AL | CP | AL | CP | AL | CP | AL | CP | AL | |||

50 | −17.4 | 0.984 | 0.701 | 1 | 0.638 | 0.997 | 0.658 | −20.3 | 0.989 | 0.714 | 1 | 0.694 | 0.977 | 0.692 |

100 | −8.9 | 0.969 | 0.508 | 0.994 | 0.471 | 0.987 | 0.485 | −11.5 | 0.963 | 0.539 | 0.993 | 0.514 | 0.981 | 0.521 |

200 | −4.9 | 0.963 | 0.358 | 0.974 | 0.343 | 0.968 | 0.349 | −6.3 | 0.955 | 0.384 | 0.973 | 0.371 | 0.964 | 0.376 |

500 | −2.0 | 0.946 | 0.224 | 0.952 | 0.221 | 0.950 | 0.222 | −2.7 | 0.950 | 0.238 | 0.956 | 0.235 | 0.954 | 0.237 |

1000 | −0.6 | 0.953 | 0.157 | 0.954 | 0.156 | 0.954 | 0.156 | −0.8 | 0.951 | 0.165 | 0.953 | 0.165 | 0.953 | 0.166 |

${\lambda}_{1}=0.95\phantom{\rule{0.5em}{0ex}}{\lambda}_{0}=0.40$ | ||||||||||||||

Maximum Likelihood Method | MICE Method | |||||||||||||

n | Relative Bias (%) | Wald CI | Logit CI | Arcsine CI | Relative Bias (%) | Wald CI | Logit CI | Arcsine CI | ||||||

CP | AL | CP | AL | CP | AL | CP | AL | CP | AL | CP | AL | |||

50 | −13.5 | 0.973 | 0.600 | 1 | 0.555 | 0.991 | 0.568 | −15.2 | 0.973 | 0.608 | 1 | 0.587 | 0.971 | 0.593 |

100 | −6.7 | 0.958 | 0.420 | 0.980 | 0.398 | 0.967 | 0.407 | −7.2 | 0.952 | 0.433 | 0.986 | 0.414 | 0.968 | 0.421 |

200 | −3.1 | 0.960 | 0.293 | 0.968 | 0.285 | 0.962 | 0.289 | −3.6 | 0.954 | 0.303 | 0.968 | 0.295 | 0.963 | 0.299 |

500 | −1.5 | 0.954 | 0.184 | 0.958 | 0.182 | 0.956 | 0.183 | −1.7 | 0.950 | 0.187 | 0.951 | 0.187 | 0.950 | 0.188 |

1000 | −0.4 | 0.952 | 0.130 | 0.953 | 0.130 | 0.953 | 0.130 | −0.5 | 0.950 | 0.133 | 0.953 | 0.133 | 0.953 | 0.133 |

${\kappa}_{1}=0.8\phantom{\rule{0.5em}{0ex}}Se=0.7969\phantom{\rule{0.5em}{0ex}}Sp=0.9707\phantom{\rule{0.5em}{0ex}}p=70\%$ | ||||||||||||||

${\lambda}_{1}=0.70\phantom{\rule{0.5em}{0ex}}{\lambda}_{0}=0.25$ | ||||||||||||||

Maximum Likelihood Method | MICE Method | |||||||||||||

n | Relative Bias (%) | Wald CI | Logit CI | Arcsine CI | Relative Bias (%) | Wald CI | Logit CI | Arcsine CI | ||||||

CP | AL | CP | AL | CP | AL | CP | AL | CP | AL | CP | AL | |||

50 | −17.9 | 0.990 | 0.646 | 0.987 | 0.596 | 0.994 | 0.612 | −20.2 | 0.987 | 0.682 | 0.978 | 0.640 | 0.976 | 0.652 |

100 | −9.2 | 0.979 | 0.434 | 0.948 | 0.418 | 0.970 | 0.417 | −10.7 | 0.978 | 0.471 | 0.947 | 0.455 | 0.964 | 0.453 |

200 | −4.7 | 0.969 | 0.291 | 0.949 | 0.288 | 0.959 | 0.285 | −5.8 | 0.971 | 0.322 | 0.952 | 0.316 | 0.961 | 0.313 |

500 | −1.8 | 0.961 | 0.179 | 0.964 | 0.180 | 0.964 | 0.180 | −2.1 | 0.961 | 0.186 | 0.954 | 0.187 | 0.957 | 0.187 |

1000 | −0.6 | 0.959 | 0.123 | 0.952 | 0.122 | 0.956 | 0.122 | −0.7 | 0.957 | 0.134 | 0.951 | 0.134 | 0.954 | 0.133 |

${\lambda}_{1}=0.95\phantom{\rule{0.5em}{0ex}}{\lambda}_{0}=0.40$ | ||||||||||||||

Maximum Likelihood Method | MICE Method | |||||||||||||

n | Relative Bias (%) | Wald CI | Logit CI | Arcsine CI | Relative Bias (%) | Wald CI | Logit CI | Arcsine CI | ||||||

CP | AL | CP | AL | CP | AL | CP | AL | CP | AL | CP | AL | |||

50 | −13.2 | 0.997 | 0.535 | 0.949 | 0.504 | 0.965 | 0.509 | −14.8 | 0.971 | 0.551 | 0.957 | 0.523 | 0.968 | 0.527 |

100 | −6.9 | 0.973 | 0.349 | 0.946 | 0.341 | 0.961 | 0.339 | −7.5 | 0.968 | 0.363 | 0.944 | 0.355 | 0.963 | 0.352 |

200 | −3.3 | 0.965 | 0.231 | 0.949 | 0.230 | 0.961 | 0.227 | −3.6 | 0.967 | 0.240 | 0.953 | 0.241 | 0.959 | 0.239 |

500 | −1.4 | 0.961 | 0.141 | 0.952 | 0.141 | 0.959 | 0.140 | −1.5 | 0.956 | 0.148 | 0.945 | 0.147 | 0.950 | 0.147 |

1000 | −0.6 | 0.953 | 0.099 | 0.951 | 0.099 | 0.952 | 0.099 | −0.6 | 0.950 | 0.102 | 0.945 | 0.102 | 0.946 | 0.102 |

**Table 5.**Coverage probabilities and average lengths of CIs for ${\kappa}_{2}=\left\{0.2,0.4\right\}$.

${\mathit{\kappa}}_{2}=0.2\phantom{\rule{0.5em}{0ex}}\mathit{S}\mathit{e}=0.5904\phantom{\rule{0.5em}{0ex}}\mathit{S}\mathit{p}=0.6901\phantom{\rule{0.5em}{0ex}}\mathit{p}=70\mathit{\%}$ | ||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|

${\lambda}_{1}=0.70\phantom{\rule{0.5em}{0ex}}{\lambda}_{0}=0.25$ | ||||||||||||||

Maximum Likelihood Method | MICE Method | |||||||||||||

n | Relative Bias (%) | Wald CI | Logit CI | Arcsine CI | Relative Bias (%) | Wald CI | Logit CI | Arcsine CI | ||||||

CP | AL | CP | AL | CP | AL | CP | AL | CP | AL | CP | AL | |||

50 | −15.8 | 0.984 | 0.597 | 0.976 | 0.744 | 0.883 | 0.610 | −19.4 | 0.981 | 0.616 | 0.978 | 0.795 | 0.947 | 0.662 |

100 | −7.3 | 0.985 | 0.465 | 0.971 | 0.583 | 0.943 | 0.486 | −10.7 | 0.971 | 0.490 | 0.976 | 0.663 | 0.972 | 0.529 |

200 | −2.9 | 0.958 | 0.357 | 0.960 | 0.418 | 0.967 | 0.365 | −5.1 | 0.953 | 0.381 | 0.967 | 0.500 | 0.967 | 0.401 |

500 | −1.7 | 0.945 | 0.241 | 0.960 | 0.248 | 0.963 | 0.239 | −1.9 | 0.949 | 0.260 | 0.963 | 0.281 | 0.962 | 0.261 |

1000 | −0.7 | 0.949 | 0.172 | 0.968 | 0.173 | 0.955 | 0.171 | −0.8 | 0.950 | 0.180 | 0.960 | 0.185 | 0.957 | 0.182 |

${\lambda}_{1}=0.95\phantom{\rule{0.5em}{0ex}}{\lambda}_{0}=0.40$ | ||||||||||||||

Maximum Likelihood Method | MICE Method | |||||||||||||

n | Relative Bias (%) | Wald CI | Logit CI | Arcsine CI | Relative Bias (%) | Wald CI | Logit CI | Arcsine CI | ||||||

CP | AL | CP | AL | CP | AL | CP | AL | CP | AL | CP | AL | |||

50 | −11.1 | 0.979 | 0.511 | 0.970 | 0.662 | 0.907 | 0.530 | −13.6 | 0.980 | 0.530 | 0.973 | 0.728 | 0.953 | 0.577 |

100 | −6.1 | 0.971 | 0.395 | 0.968 | 0.485 | 0.960 | 0.409 | −8.2 | 0.966 | 0.412 | 0.972 | 0.560 | 0.967 | 0.440 |

200 | −2.3 | 0.955 | 0.299 | 0.961 | 0.326 | 0.974 | 0.301 | −4.1 | 0.953 | 0.312 | 0.971 | 0.375 | 0.977 | 0.321 |

500 | −1.1 | 0.937 | 0.196 | 0.961 | 0.197 | 0.951 | 0.194 | −1.4 | 0.947 | 0.206 | 0.965 | 0.214 | 0.962 | 0.206 |

1000 | −0.5 | 0.956 | 0.138 | 0.962 | 0.139 | 0.959 | 0.138 | −0.6 | 0.951 | 0.147 | 0.959 | 0.148 | 0.957 | 0.146 |

${\kappa}_{2}=0.4\phantom{\rule{0.5em}{0ex}}Se=0.7773\phantom{\rule{0.5em}{0ex}}Sp=0.7308\phantom{\rule{0.5em}{0ex}}p=10\%$ | ||||||||||||||

${\lambda}_{1}=0.70\phantom{\rule{0.5em}{0ex}}{\lambda}_{0}=0.25$ | ||||||||||||||

Maximum Likelihood Method | MICE Method | |||||||||||||

n | Relative Bias (%) | Wald CI | Logit CI | Arcsine CI | Relative Bias (%) | Wald CI | Logit CI | Arcsine CI | ||||||

CP | AL | CP | AL | CP | AL | CP | AL | CP | AL | CP | AL | |||

50 | −36.5 | 0.999 | 0.743 | 1 | 0.842 | 0.891 | 0.701 | −38.8 | 0.999 | 0.683 | 1 | 0.892 | 0.967 | 0.738 |

100 | −26.1 | 0.981 | 0.630 | 1 | 0.680 | 0.983 | 0.631 | −29.3 | 0.963 | 0.598 | 1 | 0.778 | 0.991 | 0.649 |

200 | −16.3 | 0.964 | 0.496 | 0.998 | 0.493 | 0.995 | 0.486 | −19.2 | 0.943 | 0.516 | 0.995 | 0.597 | 0.997 | 0.534 |

500 | −7.2 | 0.954 | 0.320 | 0.984 | 0.312 | 0.969 | 0.315 | −9.5 | 0.949 | 0.356 | 0.989 | 0.361 | 0.968 | 0.354 |

1000 | −3.9 | 0.957 | 0.226 | 0.966 | 0.223 | 0.962 | 0.224 | −4.4 | 0.951 | 0.247 | 0.971 | 0.247 | 0.958 | 0.247 |

${\lambda}_{1}=0.95\phantom{\rule{0.5em}{0ex}}{\lambda}_{0}=0.40$ | ||||||||||||||

Maximum Likelihood Method | MICE Method | |||||||||||||

n | Relative Bias (%) | Wald CI | Logit CI | Arcsine CI | Relative Bias (%) | Wald CI | Logit CI | Arcsine CI | ||||||

CP | AL | CP | AL | CP | AL | CP | AL | CP | AL | CP | AL | |||

50 | −31.5 | 0.999 | 0.684 | 0.999 | 0.762 | 0.984 | 0.678 | −35.3 | 0.999 | 0.652 | 1.000 | 0.847 | 0.976 | 0.702 |

100 | −19.7 | 0.969 | 0.556 | 0.999 | 0.571 | 0.992 | 0.548 | −23.2 | 0.951 | 0.559 | 1.000 | 0.674 | 0.992 | 0.586 |

200 | −10.9 | 0.964 | 0.415 | 0.996 | 0.402 | 0.994 | 0.404 | −13.4 | 0.951 | 0.437 | 0.999 | 0.458 | 0.992 | 0.437 |

500 | −5.3 | 0.954 | 0.261 | 0.970 | 0.256 | 0.962 | 0.258 | −7.6 | 0.952 | 0.278 | 0.978 | 0.277 | 0.966 | 0.277 |

1000 | −1.9 | 0.945 | 0.184 | 0.954 | 0.182 | 0.949 | 0.183 | −2.3 | 0.951 | 0.193 | 0.961 | 0.191 | 0.959 | 0.193 |

**Table 6.**Coverage probabilities and average lengths of CIs for ${\kappa}_{2}=\left\{0.6,0.8\right\}$.

${\mathit{\kappa}}_{2}=0.6\phantom{\rule{0.5em}{0ex}}\mathit{S}\mathit{e}=0.8864\phantom{\rule{0.5em}{0ex}}\mathit{S}\mathit{p}=0.6746\phantom{\rule{0.5em}{0ex}}\mathit{p}=30\mathit{\%}$ | ||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|

${\lambda}_{1}=0.70\phantom{\rule{0.5em}{0ex}}{\lambda}_{0}=0.25$ | ||||||||||||||

Maximum likelihood method | MICE Method | |||||||||||||

n | Relative Bias (%) | Wald CI | logit CI | Arcsine CI | Relative Bias (%) | Wald CI | logit CI | Arcsine CI | ||||||

CP | AL | CP | AL | CP | AL | CP | AL | CP | AL | CP | AL | |||

50 | −32.9 | 0.97 | 0.799 | 1 | 0.74 | 0.987 | 0.742 | −35.6 | 0.973 | 0.767 | 1 | 0.794 | 0.968 | 0.762 |

100 | −18.1 | 0.971 | 0.61 | 1 | 0.555 | 0.996 | 0.576 | −21.9 | 0.944 | 0.649 | 0.997 | 0.629 | 0.972 | 0.629 |

200 | −9.8 | 0.970 | 0.417 | 0.984 | 0.394 | 0.976 | 0.404 | −12.3 | 0.955 | 0.470 | 0.981 | 0.450 | 0.966 | 0.458 |

500 | −3.8 | 0.960 | 0.254 | 0.966 | 0.248 | 0.964 | 0.251 | −4.9 | 0.956 | 0.278 | 0.960 | 0.278 | 0.958 | 0.281 |

1000 | −2.2 | 0.945 | 0.177 | 0.949 | 0.176 | 0.949 | 0.176 | −2.9 | 0.948 | 0.187 | 0.951 | 0.187 | 0.949 | 0.187 |

${\lambda}_{1}=0.95\phantom{\rule{0.5em}{0ex}}{\lambda}_{0}=0.40$ | ||||||||||||||

Maximum likelihood method | MICE Method | |||||||||||||

n | Relative Bias (%) | Wald CI | Logit CI | Arcsine CI | Relative Bias (%) | Wald CI | Logit CI | Arcsine CI | ||||||

CP | AL | CP | AL | CP | AL | CP | AL | CP | AL | CP | AL | |||

50 | −22.9 | 0.978 | 0.694 | 1 | 0.633 | 0.996 | 0.652 | −26.4 | 0.967 | 0.709 | 1 | 0.701 | 0.973 | 0.692 |

100 | −12.6 | 0.966 | 0.487 | 0.996 | 0.454 | 0.978 | 0.468 | −16.1 | 0.956 | 0.531 | 0.995 | 0.507 | 0.969 | 0.514 |

200 | −6.2 | 0.967 | 0.331 | 0.976 | 0.319 | 0.969 | 0.324 | −8.8 | 0.959 | 0.360 | 0.972 | 0.348 | 0.971 | 0.350 |

500 | −2.4 | 0.956 | 0.203 | 0.960 | 0.200 | 0.959 | 0.201 | −3.3 | 0.955 | 0.216 | 0.956 | 0.212 | 0.957 | 0.215 |

1000 | −1.3 | 0.960 | 0.142 | 0.957 | 0.142 | 0.956 | 0.142 | −1.5 | 0.956 | 0.150 | 0.957 | 0.150 | 0.957 | 0.150 |

${\kappa}_{2}=0.8\phantom{\rule{0.5em}{0ex}}Se=0.8644\phantom{\rule{0.5em}{0ex}}Sp=0.9817\phantom{\rule{0.5em}{0ex}}p=50\%$ | ||||||||||||||

${\lambda}_{1}=0.70\phantom{\rule{0.5em}{0ex}}{\lambda}_{0}=0.25$ | ||||||||||||||

Maximum likelihood method | MICE Method | |||||||||||||

n | Relative Bias (%) | Wald CI | Logit CI | Arcsine CI | Relative Bias (%) | Wald CI | Logit CI | Arcsine CI | ||||||

CP | AL | CP | AL | CP | AL | CP | AL | CP | AL | CP | AL | |||

50 | −20.6 | 0.938 | 0.688 | 0.933 | 0.644 | 0.931 | 0.662 | −23.8 | 0.935 | 0.711 | 0.933 | 0.672 | 0.942 | 0.695 |

100 | −10.6 | 0.956 | 0.495 | 0.912 | 0.492 | 0.942 | 0.485 | −13.1 | 0.949 | 0.531 | 0.922 | 0.525 | 0.941 | 0.519 |

200 | −5.5 | 0.958 | 0.356 | 0.936 | 0.356 | 0.954 | 0.348 | −7.2 | 0.951 | 0.392 | 0.942 | 0.391 | 0.954 | 0.382 |

500 | −2.3 | 0.960 | 0.228 | 0.954 | 0.228 | 0.959 | 0.228 | −3.0 | 0.953 | 0.235 | 0.946 | 0.234 | 0.950 | 0.233 |

1000 | −1.1 | 0.953 | 0.158 | 0.956 | 0.158 | 0.953 | 0.157 | −1.5 | 0.949 | 0.175 | 0.950 | 0.175 | 0.949 | 0.174 |

${\lambda}_{1}=0.95\phantom{\rule{0.5em}{0ex}}{\lambda}_{0}=0.40$ | ||||||||||||||

Maximum likelihood method | MICE Method | |||||||||||||

n | Relative Bias (%) | Wald CI | Logit CI | Arcsine CI | Relative Bias (%) | Wald CI | Logit CI | Arcsine CI | ||||||

CP | AL | CP | AL | CP | AL | CP | AL | CP | AL | CP | AL | |||

50 | −13.8 | 0.957 | 0.567 | 0.915 | 0.553 | 0.938 | 0.553 | −16.1 | 0.965 | 0.59 | 0.924 | 0.537 | 0.943 | 0.573 |

100 | −6.7 | 0.963 | 0.399 | 0.933 | 0.401 | 0.958 | 0.392 | −7.9 | 0.952 | 0.422 | 0.933 | 0.423 | 0.945 | 0.418 |

200 | −3.6 | 0.943 | 0.285 | 0.935 | 0.285 | 0.942 | 0.279 | −4.4 | 0.947 | 0.301 | 0.937 | 0.302 | 0.943 | 0.296 |

500 | −1.3 | 0.954 | 0.178 | 0.944 | 0.179 | 0.949 | 0.177 | −1.6 | 0.950 | 0.190 | 0.945 | 0.191 | 0.946 | 0.188 |

1000 | −0.7 | 0.949 | 0.126 | 0.949 | 0.126 | 0.947 | 0.126 | −0.8 | 0.950 | 0.132 | 0.953 | 0.132 | 0.950 | 0.132 |

Observed Frequencies of the Study of Drum and Christacopoulos | ||
---|---|---|

$T=1$ | $T=0$ | |

$V=1$ | ||

$D=1$ | 231 | 27 |

$D=0$ | 32 | 54 |

$V=0$ | 166 | 140 |

Total | 429 | 221 |

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Roldán-Nofuentes, J.A.; Regad, S.B.
Estimation of the Average Kappa Coefficient of a Binary Diagnostic Test in the Presence of Partial Verification. *Mathematics* **2021**, *9*, 1694.
https://doi.org/10.3390/math9141694

**AMA Style**

Roldán-Nofuentes JA, Regad SB.
Estimation of the Average Kappa Coefficient of a Binary Diagnostic Test in the Presence of Partial Verification. *Mathematics*. 2021; 9(14):1694.
https://doi.org/10.3390/math9141694

**Chicago/Turabian Style**

Roldán-Nofuentes, José Antonio, and Saad Bouh Regad.
2021. "Estimation of the Average Kappa Coefficient of a Binary Diagnostic Test in the Presence of Partial Verification" *Mathematics* 9, no. 14: 1694.
https://doi.org/10.3390/math9141694