# On the Convergence of the Benjamini–Hochberg Procedure

^{1}

^{2}

^{3}

^{*}

## Abstract

**:**

## 1. Introduction

**M**), the convergence is exponential and give explicit expression of the constants involved in the asymptotics. That condition allows the usage of suitable Beta distributions amongst others. The relationship between our condition and the non-criticality condition in [13] will be discussed below. Condition (

**M**) allows us to find a lower bound for the probability that for fixed parameters of the model the power is within given interval around that asymptotic limit. In this sense our work can be understood as a tool which can be used in the planning of the experiment and which complements [14], another article by the authors of [15], which deals with a variety of questions, such as estimation of the proportion of significant tests, the effect of the sample size to the quality of results, etc.

## 2. Main Results and Their Proofs

#### 2.1. Some Preliminary Notation

#### 2.2. General Inequalities and Information concerning the Distribution of ${R}_{m}$

**Proposition**

**1.**

**Remark**

**1.**

**Remark**

**2.**

**Remark**

**3.**

**Proof of Proposition**

**1.**

**Proposition**

**2.**

**Proof.**

#### 2.3. Condition (**M**)

**M**) and which is synthesized as follows.

**Definition**

**1.**

**M**) holds for fixed $\alpha \in \left(0,1\right)$ if and only if $H\left(t\right):=\frac{G\left(t\right)-t}{t}$ is decreasing on $\left(0,\alpha \right]$ with $\underset{t\to 0}{lim}H\left(t\right)=\infty $.

**Remark**

**4.**

**M**) also holds when $b<1$ and $a<(1-2\alpha )/(1-\alpha )$, although that case should not happen for well-behaved statistical procedures.

**Remark**

**5.**

**Remark**

**6.**

**M**) it is impossible that G has atoms on $\left(0,\alpha \right)$ as otherwise if $\mathbf{a}\in \left(0,\alpha \right)$ is an atom then

**M**). This also means that H is continuous on $\left(0,\alpha \right)$ and thus ${\underline{x}}_{\phantom{\rule{0.166667em}{0ex}}\alpha ,\gamma}^{\beta}={\overline{x}}_{\alpha ,\gamma}^{\beta}={x}_{\alpha ,\gamma}^{\beta}$.

**Lemma**

**1.**

**M**) is valid for fixed $\alpha \in \left(0,1\right)$ if G has on $\left(0,\alpha \right)$ a density $g={G}^{\prime}$ satisfying $\underset{x\to 0}{lim}g\left(x\right)=g\left(0\right)=\infty $ and $tg\left(t\right)-G\left(t\right)<0$ on $\left(0,\alpha \right)$. The latter is satisfied if, for example, ${g}^{\prime}\left(t\right)<0$ on $\left(0,\alpha \right)$.

**Proof.**

**M**) holds.

**Lemma**

**2.**

**M**) hold for G and ${m}_{1}=m(1-\gamma )$. Then we have that

**Remark**

**7.**

**M**) we have that $G\left(t\right)c\left(d\frac{t}{G\left(t\right)}\right)$ is non-decreasing on $\left(0,\alpha \right)$, for any $d>0$, since $\frac{t}{G\left(t\right)}$ is non-decreasing on $\left(0,\alpha \right)$ and c is increasing on $\left(0,\infty \right)$, see Proposition A1. Hence the function ${K}_{G}^{\mathbf{M}}\left(x,m,1-\gamma ,\u03f5,\alpha \right)$ is decreasing in x as long as the other parameters stay fixed. The same is valid for ${K}_{F}^{\mathbf{M}}$.

**Proof of Lemma**

**2.**

**M**) we have that the function $G\left(t\right)c\left(\frac{\u03f5}{1-\gamma}\frac{t}{G\left(t\right)}\right)$ is non-decreasing on $\left(0,\alpha \right)$, see Remark 7. We just note that by definition ${m}_{1}=m(1-\gamma )$ and we have employed this for k in (6) to derive (19). Finally, (20) follows from (7). □

#### 2.4. Theoretical Bounds on the Distribution of ${R}_{m}$ under Condition ( **M**)

**M**) holds we have the identity equivalent to (17)

**Theorem**

**1.**

**M**) hold for given G and fixed $\alpha \in \left(0,1\right)$ with $G\left(\alpha \right)<1$. Fix $\gamma \in \left(0,1\right)$. Recall from (18) that ${x}_{\alpha ,\gamma}={\underline{x}}_{\alpha ,\gamma}^{\alpha}={\overline{x}}_{\alpha ,\gamma}^{\alpha}$. Then $\underset{m\to \infty}{lim}\frac{{R}_{m}}{m}={x}_{\alpha ,\gamma}\in \left(0,1\right)$. Moreover, for any $x\in \left({x}_{\alpha ,\gamma},1\right)$, $\exists \u03f5\left(x\right)$ such that for any $0\le \u03f5\le \u03f5\left(x\right)$ the inequality $\left(1-\gamma \right)H\left(\alpha x\right)<\frac{1}{\alpha}-1-2\u03f5$ holds and then for any $m\ge {m}_{G}(x,\frac{1}{\alpha}-1-2\u03f5,\alpha ,\gamma )$ we have that

**Remark**

**8.**

**Remark**

**9.**

**Remark**

**10.**

**Proof of Theorem**

**1.**

**M**) we have that ${\underline{x}}_{\alpha ,\gamma}^{\alpha}={\overline{x}}_{\alpha ,\gamma}^{\alpha}:={x}_{\alpha ,\gamma}$, see (18). Clearly, ${x}_{\alpha ,\gamma}\in \left(0,1\right)$ since ${u}_{\alpha ,\gamma}\left(0\right)=\infty $ and ${u}_{\alpha ,\gamma}\left(1\right)=\frac{G\left(\alpha \right)-\alpha}{\alpha}<\frac{1}{\alpha}-1$, see (17), and by assumption $G\left(\alpha \right)<1$. The fact that $\underset{m\to \infty}{lim}\frac{{R}_{m}}{m}={x}_{\alpha ,\gamma}\in \left(0,1\right)$ follows from Theorem 3.1 in [15]. Henceforth, it remains to show (24) and (25). The former thanks to (16) and (19) follows for any $m\ge {m}_{G}(x,\frac{1}{\alpha}-1-2\u03f5,\alpha ,\gamma ),$ if ${m}_{G}(x,\frac{1}{\alpha}-1-2\u03f5,\alpha ,\gamma )<\infty $, since then

**Theorem**

**2.**

**Proof.**

**M**) holds, then

#### 2.5. Exponential Convergence to the Theoretical Limit of the Power of the Benjamini–Hochberg Procedure

**Theorem**

**3.**

**M**) be valid and assume the notation of Theorem 1. Let next ${x}_{\alpha ,\gamma}\in \left({x}_{1},{x}_{2}\right)\subset \left(0,1\right)$. Then there exists ${m}^{*}={m}^{*}({x}_{1},{x}_{2},\u03f5,\alpha ,\gamma )$ and ${\u03f5}^{*}=\u03f5({x}_{1},{x}_{2})$ such that for any $m\ge {m}^{*}$ and any $\u03f5\le {\u03f5}^{*}$

**Remark**

**11.**

**Proof of Theorem**

**3.**

**Theorem**

**4.**

**M**) be valid and assume the notation of Theorem 2. Let $m\in {\mathbb{N}}^{+}$ be fixed. Let next ${x}_{\alpha ,\gamma}\in \left({x}_{1},{x}_{2}\right)\subset \left(0,1\right)$ and ${x}_{1}$ is such that $\frac{\lceil m{x}_{1}\rceil}{m}<{x}_{\alpha ,\gamma}$. Finally, let $\eta \in \left(0,1\right)$ be fixed. Then with

**Remark**

**12.**

**Proof of Theorem**

**4.**

## 3. Workflow

**M**) is satisfied. In this case Lemma 1 is valid and condition (

**M**) holds since elementary calculations yield that ${g}_{a,b}^{\prime}<0$ on $(0,\alpha )$ and $\underset{x\to 0}{lim}{g}_{a,b}\left(x\right)=\infty $, where ${g}_{a,b}$ is the probability density function of $B(a,b)$.

- we calculate ${x}_{\alpha ,\gamma}$ by solving numerically Equation (22) wherein $G(\xb7)=G\left(\xb7,a,b\right)$;
- for given $l<G(\alpha {x}_{\alpha ,\gamma},a,b)$, any given $\eta $ and ${x}_{1}$ is such that $\frac{\lceil m{x}_{1}\rceil}{m}<{x}_{\alpha ,\gamma}$ we have an estimate based on (42), that is$$\begin{array}{cc}\hfill \phantom{\rule{1.em}{0ex}}& \mathbb{P}\left(\left(1-\eta \right)G\left(\alpha {x}_{1}\right)\le {S}_{m}^{*}\right)\hfill \\ \hfill \phantom{\rule{1.em}{0ex}}& \ge max\{1-{K}_{F}^{\mathbf{M}}({x}_{1},m,\gamma ,{\u03f5}_{1},\alpha )-{K}_{G}^{\mathbf{M}}({x}_{1},m,1-\gamma ,{\u03f5}_{1},\alpha )-\hfill \\ \hfill \phantom{\rule{1.em}{0ex}}& -{e}^{-mc\left(\eta \right)(1-\gamma )G\left(\alpha {x}_{1},a,b\right)};0\};\hfill \end{array}$$
- fixing $\eta $ we attempt to solve (numerically) in ${x}_{1}$ the equation$$l=\left(1-\eta \right)G\left(\alpha {x}_{1},a,b\right);$$
- if $\frac{\lceil m{x}_{1}\rceil}{m}<{x}_{\alpha ,\gamma}$ then (42) represents a valid theoretical lower bound, that is$$\begin{array}{cc}\hfill p\left(\eta \right)& =max\{1-{K}_{F}^{\mathbf{M}}({x}_{1},m,\gamma ,{\u03f5}_{1},\alpha )-{K}_{G}^{\mathbf{M}}({x}_{1},m,1-\gamma ,{\u03f5}_{1},\alpha )-\hfill \\ \hfill \phantom{\rule{1.em}{0ex}}& -{e}^{-mc\left(\eta \right)(1-\gamma )G\left(\alpha {x}_{1},a,b\right)};0\}\hfill \end{array}$$
- finally, we optimize $p\left(\eta \right)$ in $\eta $ and choose ${\eta}^{*}$ so that the estimated probability bound is the largest, that is $\mathbb{P}\left(l\le {S}_{m}^{*}\right)\ge {sup}_{\eta \in \left(0,1\right)}p\left(\eta \right)=p\left({\eta}^{*}\right).$

## 4. Results

## 5. Discussion

#### 5.1. Related Work

#### 5.2. Considerations about Our Work

## Supplementary Materials

## Funding

## Acknowledgments

## Conflicts of Interest

## Abbreviations

FDR | False Discovery Rate |

PET | Positron Emission Tomography |

CT | Computerized Tomography |

fMRI | Functional Magnetic Resonance Imaging |

## Appendix A. Petrov’s Inequality

**Proposition**

**A1.**

**Proof.**

## References

- Benjamini, Y.; Hochberg, Y. Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing. J. R. Stat. Soc. Ser. B (Methodol.)
**1995**, 57, 289–300. [Google Scholar] [CrossRef] - Love, M.; Huber, W.; Anders, S. Moderated estimation of fold change and dispersion for RNA-seq data with DESeq2. Genome Biol.
**2014**, 15, 550. [Google Scholar] [CrossRef] [Green Version] - Robinson, M.D.; McCarthy, D.J.; Smyth, G.K. edgeR: A Bioconductor package for differential expression analysis of digital gene expression data. Bioinformatics
**2010**, 26, 139–140. [Google Scholar] [CrossRef] [PubMed] [Green Version] - Trapnell, C.; Hendrickson, D.G.; Sauvageau, M.; Goff, L.; Rinn, J.L.; Pachter, L. Differential analysis of gene regulation at transcript resolution with RNA-seq. Nat. Biotechnol.
**2012**, 31, 46–53. [Google Scholar] [CrossRef] - Chalkidou, A.; O’Doherty, M.J.; Marsden, P.K. False Discovery Rates in PET and CT Studies with Texture Features: A Systematic Review. PLoS ONE
**2015**, 10, e0124165. [Google Scholar] [CrossRef] [Green Version] - Bennett, C.M.; Wolford, G.L.; Miller, M.B. The principled control of false positives in neuroimaging. Soc. Cogn. Affect. Neurosci.
**2009**, 4, 417–422. [Google Scholar] [CrossRef] - Miller, C.J.; Genovese, C.; Nichol, R.C.; Wasserman, L.; Connolly, A.; Reichart, D.; Hopkins, A.; Schneider, J.; Moore, A. Controlling the False-Discovery Rate in Astrophysical Data Analysis. Astron. J.
**2001**, 122, 3492. [Google Scholar] [CrossRef] [Green Version] - Benjamini, Y.; Yekutieli, D. The Control of the False Discovery Rate in Multiple Testing under Dependency. Ann. Stat.
**2001**, 29, 1165–1188. [Google Scholar] [CrossRef] - Storey, J.D.; Taylor, J.E.; Siegmund, D. Strong control, conservative point estimation and simultaneous conservative consistency of false discovery rates: A unified approach. J. R. Stat. Soc. Ser. B (Stat. Methodol.)
**2004**, 66, 187–205. [Google Scholar] [CrossRef] - Pollard, K.S.; van der Laan, M.J. Choice of a null distribution in resampling-based multiple testing. J. Stat. Plan. Inference
**2004**, 125, 85–100. [Google Scholar] [CrossRef] - Van Noorden, R.; Maher, B.; Nuzzo, R. The top 100 papers. Nature
**2014**, 514, 550–553. [Google Scholar] [CrossRef] [Green Version] - Farcomeni, A. Some Results on the Control of the False Discovery Rate under Dependence. Scand. J. Stat.
**2007**, 34, 275–297. [Google Scholar] [CrossRef] - Chi, Z. On the performance of FDR control: Constraints and a partial solution. Ann. Statist.
**2007**, 35, 1409–1431. [Google Scholar] [CrossRef] [Green Version] - Ferreira, J.A.; Zwinderman, A.H. Approximate Power and Sample Size Calculations with the Benjamini-Hochberg Method. Int. J. Biostat
**2006**, 2, 8. [Google Scholar] [CrossRef] - Ferreira, J.A.; Zwinderman, A.H. On the Benjamini—Hochberg method. Ann. Statist.
**2006**, 34, 1827–1849. [Google Scholar] [CrossRef] [Green Version] - Dvoretzky, A.; Kiefer, J.; Wolfowitz, J. Asymptotic Minimax Character of the Sample Distribution Function and of the Classical Multinomial Estimator. Ann. Math. Stat.
**1956**, 27, 642–669. [Google Scholar] [CrossRef] - Pounds, S.; Morris, S.W. Estimating the occurrence of false positives and false negatives in microarray studies by approximating and partitioning the empirical distribution of p-values. Bioinformatics
**2003**, 19, 1236–1242. [Google Scholar] [CrossRef] [PubMed] - Neuvial, P. Asymptotic properties of false discovery rate controlling procedures under independence. Electron. J. Stat.
**2008**, 2, 1065–1110. [Google Scholar] [CrossRef] - Finner, H.; Dickhaus, T.; Roters, M. On the false discovery rate and an asymptotically optimal rejection curve. Ann. Statist.
**2009**, 37, 596–618. [Google Scholar] [CrossRef] - Gontscharuk, V.; Finner, H. Asymptotic FDR control under weak dependence: A counterexample. Stat. Probab. Lett.
**2013**, 83, 1888–1893. [Google Scholar] [CrossRef] - Atanassov, E.; Gurov, T.; Karaivanova, A.; Ivanovska, S.; Durchova, M.; Dimitrov, D. On the parallelization approaches for Intel MIC architecture. AIP Conf. Proc.
**2016**, 1773, 070001. [Google Scholar] [CrossRef] - Petrov, V.V. Sums of Independent Random Variables; Brown, A.A., Translator; Ergebnisse der Mathematik und ihrer Grenzgebiete; Springer: Berlin/Heidelberg, Germany, 1975; Volume 82. [Google Scholar]

**Figure 1.**Empirical results when the p-values under the alternative hypothesis have $B(0.5,100)$ distribution and the proportion of alternative tests is $2\%$. The x-axis shows the common logarithm of the number of tests m. The y-axis shows the empirical power. The dots depict the mean of the empirical power, the vertical lines are of length one standard deviation below and above the mean. The dotted line shows the asymptotic power.

**Table 1.**Simulation results for B(0.1, 100), number genes = 20,000, proportion significant = 0.1, real power limit = 0.940359.

Lower Bound l | Empirical Probability | Theoretical Estimate $\mathit{p}\left({\mathit{\eta}}^{*}\right)$ | ${\mathit{\eta}}^{*}$ |
---|---|---|---|

0.750 | 1.000 | 1.000 | 0.098 |

0.760 | 1.000 | 1.000 | 0.097 |

0.770 | 1.000 | 1.000 | 0.096 |

0.780 | 1.000 | 1.000 | 0.096 |

0.790 | 1.000 | 1.000 | 0.095 |

0.800 | 1.000 | 1.000 | 0.095 |

0.810 | 1.000 | 1.000 | 0.094 |

0.820 | 1.000 | 1.000 | 0.094 |

0.830 | 1.000 | 1.000 | 0.093 |

0.840 | 1.000 | 1.000 | 0.093 |

0.850 | 1.000 | 0.999 | 0.085 |

0.860 | 1.000 | 0.996 | 0.078 |

0.870 | 1.000 | 0.985 | 0.068 |

0.880 | 1.000 | 0.952 | 0.058 |

0.890 | 1.000 | 0.873 | 0.048 |

0.900 | 1.000 | 0.714 | 0.037 |

0.910 | 1.000 | 0.476 | 0.027 |

0.920 | 1.000 | 0.204 | 0.016 |

0.930 | 0.967 | 0.018 | 0.005 |

0.940 | 0.550 | 0.000 |

**Table 2.**Simulation results for B(0.25, 2), number genes = 20,000, proportion significant = 0.1, real power limit = 0.233737.

Lower Bound l | Empirical Probability | Theoretical Estimate $\mathit{p}\left({\mathit{\eta}}^{*}\right)$ | ${\mathit{\eta}}^{*}$ |
---|---|---|---|

0.190 | 1.000 | 0.779 | 0.105 |

0.200 | 0.997 | 0.434 | 0.072 |

0.210 | 0.972 | 0.000 | 0.001 |

0.220 | 0.868 | 0.000 | 0.001 |

0.230 | 0.626 | 0.000 | 0.001 |

**Table 3.**Simulation results for B(0.1, 2), number genes = 20,000, proportion significant = 0.1, real power limit = 0.620014.

Lower Bound l | Empirical Probability | Theoretical Estimate $\mathit{p}\left({\mathit{\eta}}^{*}\right)$ | ${\mathit{\eta}}^{*}$ |
---|---|---|---|

0.500 | 1.000 | 1.000 | 0.119 |

0.510 | 1.000 | 1.000 | 0.117 |

0.520 | 1.000 | 1.000 | 0.116 |

0.530 | 1.000 | 1.000 | 0.115 |

0.540 | 1.000 | 0.999 | 0.106 |

0.550 | 1.000 | 0.996 | 0.097 |

0.560 | 1.000 | 0.979 | 0.081 |

0.570 | 1.000 | 0.926 | 0.067 |

0.580 | 1.000 | 0.786 | 0.052 |

0.590 | 0.994 | 0.521 | 0.036 |

0.600 | 0.953 | 0.189 | 0.02 |

0.610 | 0.803 | 0.000 | 0.001 |

0.620 | 0.511 | 0.000 |

**Table 4.**Simulation results for B(0.1, 10), number genes = 20,000, proportion significant = 0.1, real power limit = 0.755055.

Lower Bound l | Empirical Probability | Theoretical Estimate $\mathit{p}\left({\mathit{\eta}}^{*}\right)$ | ${\mathit{\eta}}^{*}$ |
---|---|---|---|

0.600 | 1.000 | 1.000 | 0.109 |

0.610 | 1.000 | 1.000 | 0.108 |

0.620 | 1.000 | 1.000 | 0.107 |

0.630 | 1.000 | 1.000 | 0.106 |

0.640 | 1.000 | 1.000 | 0.105 |

0.650 | 1.000 | 1.000 | 0.105 |

0.660 | 1.000 | 1.000 | 0.104 |

0.670 | 1.000 | 0.999 | 0.096 |

0.680 | 1.000 | 0.995 | 0.085 |

0.690 | 1.000 | 0.980 | 0.074 |

0.700 | 1.000 | 0.934 | 0.062 |

0.710 | 1.000 | 0.821 | 0.049 |

0.720 | 0.999 | 0.608 | 0.037 |

0.730 | 0.991 | 0.309 | 0.024 |

0.740 | 0.924 | 0.048 | 0.009 |

0.750 | 0.693 | 0.000 | 0.001 |

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Palejev, D.; Savov, M.
On the Convergence of the Benjamini–Hochberg Procedure. *Mathematics* **2021**, *9*, 2154.
https://doi.org/10.3390/math9172154

**AMA Style**

Palejev D, Savov M.
On the Convergence of the Benjamini–Hochberg Procedure. *Mathematics*. 2021; 9(17):2154.
https://doi.org/10.3390/math9172154

**Chicago/Turabian Style**

Palejev, Dean, and Mladen Savov.
2021. "On the Convergence of the Benjamini–Hochberg Procedure" *Mathematics* 9, no. 17: 2154.
https://doi.org/10.3390/math9172154