meta.shrinkage: An R Package for Meta-Analyses for Simultaneously Estimating Individual Means

Abstract

Meta-analysis is an indispensable tool for synthesizing statistical results obtained from individual studies. Recently, non-Bayesian estimators for individual means were proposed by applying three methods: the James–Stein (JS) shrinkage estimator, isotonic regression estimator, and pretest (PT) estimator. In order to make these methods available to users, we develop a new R package meta.shrinkage. Our package can compute seven estimators (named JS, JS+, RML, RJS, RJS+, PT, and GPT). We introduce this R package along with the usage of the R functions and the “average-min-max” steps for the pool-adjacent violators algorithm. We conduct Monte Carlo simulations to validate the proposed R package to ensure that the package can work properly in a variety of scenarios. We also analyze a data example to show the ability of the R package.

1. Introduction

Meta-analysis is a tool for synthesizing statistical results obtained from individually published studies [1]. Meta-analyses have been employed in a variety of scientific studies [2,3,4,5], including studies on the influence of COVID-19 [6,7,8].

Usually, the goal of meta-analyses is to summarize individual studies to find some common effect [5,9]. The idea of estimating the common mean was originated from mathematical statistics and stratified sampling designs under fixed effect models (pp. 55–103 of [10,11,12,13]). In biostatistical methodologies, the estimation method based on random effect models [14] is popular. In either model, the goal of meta-analyses is usually to estimate the common mean by combining the estimators of individual means (Section 2.1).

In some scenarios, however, the goal of estimating the common mean is questionable. In these scenarios, meta-analyses can still be informative by looking at individual studies’ means (e.g., by a forest plot). Aside from these simple meta-analyses, Bayesian posterior means provide a more sophisticated summary of individual means in a meta-analysis [15,16,17,18,19].

Recently, Taketomi et al. [20] proposed non-Bayesian estimators of individual means by applying three methods: the James–Stein (JS) shrinkage estimator, isotonic regression estimator, and pretest (PT) estimator. Their frequentist estimators were shown to be superior to the individual studies’ estimators via decision theoretic criteria and Monte Carlo simulation experiments. These frequentist estimators also found their suitable applications to real data examples.

In this article, we propose a new R package meta.shrinkage in order to implement the frequentist estimators in [20]. Our package can calculate seven estimators (namely ${\mathit{\delta }}^{\mathrm{JS}}$, ${\mathit{\delta }}^{\mathrm{JS}+}$,${\mathit{\delta }}^{\mathrm{RML}}$, ${\mathit{\delta }}^{\mathrm{RJS}}$, ${\mathit{\delta }}^{\mathrm{RJS}+}$, ${\mathit{\delta }}^{\mathrm{PT}}$ and ${\mathit{\delta }}^{\mathrm{GPT}}$; see Section 3 for their definitions). We introduce this R package along with the usage of the R functions and the “average-min-max” steps for the pool-adjacent violators algorithm. We conduct Monte Carlo simulations to validate the proposed R package to ensure that the package can work properly in a variety of scenarios. We made the package freely available in the Comprehensive R Archive Network (CRAN): Available online: https://CRAN.R-project.org/package=meta.shrinkage (accessed on 14 November 2021).

This article is organized as follows. Section 2 gives the background, including a quick review of meta-analyses. Section 3 introduces the proposed R package. Section 4 conducts simulation studies to validate the proposed R package. Section 5 includes a data example to illustrate the proposed R package. Section 6 concludes the article with discussions. The appendices give the R code to reproduce the numerical results of this article.

2. Background

2.1. Meta-Analysis

This subsection reviews the basic concepts for meta-analysis.

To clarify the concepts, we introduce some notations and assumptions for a meta-analysis. Define $G$ as the number of studies, where $G$ stands for groups in the meta-analysis. For each $i=1,2,\dots ,G$, let $Y_i$ be an estimator for a target estimand ${\mu }_i$ that is unknown. Let $y_i$ be a realized value of the random variable $Y_i$. We assume that the error is normally distributed so that $Y_i~N\left({\mu }_i,{\sigma }_i^2\right)$, where ${\sigma }_i^2>0$ is a known variance for the error distribution. Thus, ${\mu }_i$ is the mean of $Y_i$. The observed data are $\left\{y_i: i=1,2,\dots ,G\right\}$ in this meta-analysis. We will not consider a setting where the variance is unknown [21,22], as this setting does not follow the framework of meta-analyses based on “summary data”. Without a loss of generality, we assume that ${\mu }_i=0$ corresponds to the null value.

Traditionally, the objective of meta-analyses is to estimate the common mean, denoted as $\mu$. It is defined by the fixed effect model assumption of $\mu \equiv {\mu }_1=\dots ={\mu }_G$ or the random-effects model assumption of ${\mu }_i~N\left(\mu ,{\tau }^2\right)$ for $i=1,2,\dots ,G$, where ${\tau }^2$ is the between-study variance. In this article, we assume neither, since the aforementioned models do not always fit the data at hand. For instance, if the studies have ordered means (e.g., ${\mu }_1\le \dots \le {\mu }_G$), the model is neither fixed nor random [20]. Indeed, many real meta-analyses have some covariates to systematically explain the reason for increasing means ${\mu }_1\le \dots \le {\mu }_G$ or decreasing means ${\mu }_1\ge \dots \ge {\mu }_G$ (see Section 5). In such circumstances, there is no general way to define the common mean $\mu$. Below, we discuss what meta-analyses can do in the absence of the common mean.

2.2. Improved Estimation of Individual Means

Meta-analyses often display individual estimates $\left(y_1,\dots ,y_G\right)$ along with their 95% confidence intervals (CIs): $\left(y_1,\dots ,y_G\right)\pm 1.96\times \left({\sigma }_1,\dots ,{\sigma }_G\right)$. Similarly, the funnel plot shows $\left(y_1,\dots ,y_G\right)$ against$\left({\sigma }_1,\dots ,{\sigma }_G\right)$ (see [1,5,23,24] for these plots). These meta-analyses are possible without the assumptions of the fixed effect or random effect models. Therefore, looking at the individual estimates $\left(y_1,\dots ,y_G\right)$ is a part of meta-analysis.

Taketomi et al. [20] pointed out the need for improving the individual estimates by shrinkage estimation methods [11,12,25,26]. They first regard $\mathit{Y}\equiv \left(Y_1,\dots ,Y_G\right)$ as an estimator of $\mathit{\mu }\equiv \left({\mu }_1,\dots ,{\mu }_G\right)$. Then, they consider an estimator $\mathit{\delta }\left(\mathit{Y}\right)\equiv \left({\delta }_1\left(\mathit{Y}\right),\dots ,{\delta }_G\left(\mathit{Y}\right)\right)$ that improves upon $\mathit{Y}$ in terms of the weighted mean square error (WMSE) criteria:

$$E\left[{\displaystyle \sum }_{i=1}^G\frac{{\left({\delta }_i\left(\mathit{Y}\right)-{\mu }_i\right)}^2}{{\sigma }_i^2}\right]<E\left[{\displaystyle \sum }_{i=1}^G\frac{{\left(Y_i-{\mu }_i\right)}^2}{{\sigma }_i^2}\right]=G, \exists \left({\mu }_1,\dots ,{\mu }_G\right),$$

(1)

$$E\left[{\displaystyle \sum }_{i=1}^G\frac{{\left({\delta }_i\left(\mathit{Y}\right)-{\mu }_i\right)}^2}{{\sigma }_i^2}\right]\le E\left[{\displaystyle \sum }_{i=1}^G\frac{{\left(Y_i-{\mu }_i\right)}^2}{{\sigma }_i^2}\right]=G, \forall \left({\mu }_1,\dots ,{\mu }_G\right).$$

(2)

The above $\mathit{\delta }\left(\mathit{Y}\right)$ is called an improved estimator of $\mathit{Y}$. If “$\forall \left({\mu }_1,\dots ,{\mu }_G\right)$” in Equation (2) holds only for a restricted parameter space, $\mathit{\delta }\left(\mathit{Y}\right)$ is locally improved. Two locally improved estimators are relevant in this article. The first one is under ordered means, where the parameter space is restricted to ${\mu }_1\le {\mu }_2\le \dots \le {\mu }_G$. The second one is under the sparse normal means [27], where many ${\mu }_i$ values are zero (e.g., $\left({\mu }_1,\dots ,{\mu }_{10}\right)=\left(-1,0,0,0,0,0,0,0,0,1\right)$).

The inverse variance weights in Equation (1) and (2) make it convenient to apply the classical decision theory [20]. Consequently, Taketomi et al. [20] were able to theoretically verify the (local) improvement of their estimators upon $\mathit{Y}$ in terms of the WMSE. In practice, one may also be interested in the total MSE (TMSE) criterion, defined as $E\left[{\displaystyle \sum }_{i=1}^G{\left({\delta }_i\left(\mathit{Y}\right)-{\mu }_i\right)}^2\right]$ [11,25], while the TMSE makes the theoretical analysis complex [11]. Section 4 will employ the TMSE criterion to assess the performance of all the improved estimators that will be introduced in the proposed R package.

Below, we introduce our R package, which can compute several improved estimators suggested by [20]. The goal of our package is to compute $\mathit{\delta }\left(\mathit{y}\right)\equiv \left({\delta }_1\left(\mathit{y}\right),\dots ,{\delta }_G\left(\mathit{y}\right)\right)$ from $\mathit{y}$ that are the realized values for a random vector $\mathit{Y}\equiv \left(Y_1,\dots ,Y_G\right)$.

3. R Package meta.shrinkage

This section introduces our proposed R package meta.shrinkage, which can compute seven improved estimators for individual means, denoted as ${\mathit{\delta }}^{\mathrm{JS}}$, ${\mathit{\delta }}^{\mathrm{JS}+}$, ${\mathit{\delta }}^{\mathrm{RML}}$, ${\mathit{\delta }}^{\mathrm{RJS}}$, ${\mathit{\delta }}^{\mathrm{RJS}+}$, ${\mathit{\delta }}^{\mathrm{PT}}$, and ${\mathit{\delta }}^{\mathrm{GPT}}$. We will divide our explanations into four sections: Section 3.1 for${\mathit{\delta }}^{\mathrm{JS}}$ and ${\mathit{\delta }}^{\mathrm{JS}+}$, Section 3.2 for ${\mathit{\delta }}^{\mathrm{RML}}$, Section 3.3 for ${\mathit{\delta }}^{\mathrm{RJS}}$ and ${\mathit{\delta }}^{\mathrm{RJS}+}$, Section 3.4 for ${\mathit{\delta }}^{\mathrm{PT}}$, and ${\mathit{\delta }}^{\mathrm{GPT}}$.

Before embarking on details, we explain the basic variables used in the package. Let $y_i~N\left({\mu }_i,{\sigma }_i^2\right)$ be an estimate for ${\mu }_i$ under a known variance ${\sigma }_i^2>0$ for $i=1,2,\dots ,G$. Thus, one needs to prepare $\left\{\left(y_i,{\sigma }_i\right);i=1,2,\dots ,G\right\}$ to perform a meta-analysis. Accordingly, the input variables in the R console are two vectors:

• y: a vector for $y_i$s;
• s: a vector for ${\sigma }_i$s.

Below is an example for the input variables for a dataset of $G=14$ in the R console.

This dataset comes from the data analysis of [20], in which gastric cancer data [28] was analyzed. The values for $y_i$s are Cox regression estimates for the effect of chemotherapy on disease-free survival (DFS) for gastric cancer patients, and the ${\sigma }_i$ values are the SEs.

3.1. James–Stein Estimator

The James–Stein (JS) estimator is defined as

$${\mathit{\delta }}^{\mathrm{JS}}\equiv \left({\delta }_1^{\mathrm{JS}},\dots ,{\delta }_G^{\mathrm{JS}}\right)\equiv \left(1-\frac{G-2}{{{\displaystyle \sum }}_{i=1}^G{Y}_i^2/{\sigma }_i^2}\right)\mathit{Y}.$$

This estimator is a variant from the primitive JS estimator [29], which was derived under homogeneous variances (${\sigma }_i=1$ for $\forall i$). The JS estimator reduces the WMSE by shrinking the vector $\mathit{Y}$ toward $\mathbf{0}$. The degree of shrinkage is determined by the factor $\left(G-2\right)/({\displaystyle \sum }_{i=1}^G{Y}_i^2/{\sigma }_i^2)$ that typically takes its value from 0 (0% shrinkage) to 1 (100% shrinkage). In rare cases, it becomes greater than one (overshrinkage).

It was proven that ${\mathit{\delta }}^{\mathrm{JS}}$ has a smaller WMSE than $\mathit{Y}$ when $G\ge 3$ [20]. That is, Equations (1) and (2) hold. Thus, ${\mathit{\delta }}^{\mathrm{JS}}$ is an improve estimator without any restriction.

The positive-part JS estimator can further reduce the WMSE by avoiding the overshrinkage phenomenon of $\left(G-2\right)/({\displaystyle \sum }_{i=1}^G{Y}_i^2/{\sigma }_i^2)>1$:

$${\mathit{\delta }}^{\mathrm{JS}+}\equiv \left({\delta }_1^{\mathrm{JS}+},\dots ,{\delta }_G^{\mathrm{JS}+}\right)\equiv {\left(1-\frac{G-2}{{{\displaystyle \sum }}_{i=1}^G{Y}_i^2/{\sigma }_i^2}\right)}^{+}\mathit{Y},$$

where ${(.)}^{+}\equiv \max \left(0,.\right)$.

In our R package, the function “js(.)” can compute ${\mathit{\delta }}^{\mathrm{JS}}$ and ${\mathit{\delta }}^{\mathrm{JS}+}$.

Below is the usage in the R console.

The values in the JS column are the shrunken values of y. The values are equivalent between the columns of JS and JS_plus because there is no overshrinkage.

3.2. Restricted Maximum Likelihood Estimators under Ordered Means

We consider the restricted maximum likelihood (RML) estimator when the individual means are ordered. Without loss of generality, we consider the increasing order ${\mu }_1\le \dots \le {\mu }_G$. This indicates that $\mathit{\mu }$ belongs to $\left\{\left({\mu }_1,\dots ,{\mu }_G\right):{\mu }_1\le \dots \le {\mu }_G\right\}$. If there are such parameter constraints, they should be incorporated into the estimators in order to improve the estimation accuracy [20].

The RML estimator satisfying ${\delta }_1\le \dots \le {\delta }_G$ is calculated by

$${\delta }_i^{\mathrm{RML}}=\underset{1\le s\le i}{\max }\underset{i\le t\le G}{\min }\frac{{{\displaystyle \sum }}_{j=s}^t{Y}_j}{t-s+1}.$$

The above formula is called the pool-adjacent violators algorithm (PAVA). The computation requires the “average-min-max” steps (Figure 1). We developed an R function “rml(.)” in our R package to perform the matrix-based computation of Figure 1. This R function initially computes all the elements of the matrix (Figure 1). Then, it applies the command “max(apply(z, 1, min))”, where “1” indicates that “min” applies to the rows of the matrix “z”. This yields easy-to-understand code in the R program.

To understand the reason why the matrix-based computation (Figure 1) is necessary, we give an example for calculating ${\delta }_2^{\mathrm{RML}}$ from a dataset $\mathit{Y}=\left(Y_1,Y_2,Y_3\right)$. By setting $i=2$ and $G=3$ in Figure 1, the calculation for ${\delta }_2^{\mathrm{RML}}$ proceeds as follows:

$$\left[\begin{array}{cc}\frac{Y_1+Y_2}{2}& \frac{Y_1+Y_2+Y_3}{3}\\ Y_2& \frac{Y_2+Y_3}{2}\end{array}\right]\Rightarrow \left[\begin{array}{c}\min \left(\frac{Y_1+Y_2}{2},\frac{Y_1+Y_2+Y_3}{3}\right)\\ \min \left(Y_2,\frac{Y_2+Y_3}{2} \right)\end{array}\right]\Rightarrow \max \left\{\min \left(\frac{Y_1+Y_2}{2},\frac{Y_1+Y_2+Y_3}{3}\right),\min \left(Y_2,\frac{Y_2+Y_3}{2} \right) \right\}.$$

The matrix in the preceding formula keeps four sub-averages of $\left(Y_1,Y_2,Y_3\right)$, including $Y_2$. Any one of the four components of the matrix can be ${\delta }_2^{\mathrm{RML}}$. Hence, the matrix is necessary as well as sufficient to calculate ${\delta }_2^{\mathrm{RML}}$. That aside, matrices are easy to manipulate in R.

The RML estimator ${\mathit{\delta }}^{\mathrm{RML}}\equiv \left({\delta }_1^{\mathrm{RML}},\dots ,{\delta }_G^{\mathrm{RML}}\right)$ gives a smaller WMSE than $\mathit{Y}$ [20].

For theories and applications of the PAVA, we refer to [30,31,32,33,34]. We decided not to use the “pava(.)” function available in the R package “Iso” [33] to ensure the independence of our package from others. Nonetheless, we checked that “rml(.)” and “pava(.)” gave numerically identical results.

So far, we have assumed that the studies (i.e., $i=1,2,\dots ,G$) are ordered to achieve ${\mu }_1\le \dots \le {\mu }_G$. However, usually, the studies in a raw dataset may be arbitrarily ordered, and hence, one needs to find covariates to order the studies. For instance, one can use publication years if the ${\mu }_i$ values increase with them. More generally, we assume that there exists an increasing sequence of covariates ($x_1<\dots <x_G$) or a decreasing sequence of covariates ($x_1>\dots >x_G$) to achieve the order ${\mu }_1\le \dots \le {\mu }_G$.

In our R package, the function “rml(.)” can compute ${\mathit{\delta }}^{\mathrm{RML}}$. The function allows users to enter covariates when studies are not ordered. For instance, we enter the estimates (y), SEs (s), and the proportion of males (x) from the COVID-19 data with $G=11$ [7,20].

In this data, the estimates (y) are the log risk ratios (RRs) calculated from two-by-two contingency tables examining the association (mortality vs. hypertension). As found in the previous studies [7,20], there was a decreasing sequence of the proportion of males ($x_1>\dots >x_{11}$) that could achieve the order ${\mu }_1\le \dots \le {\mu }_{11}$. Then, we have the following:

We see that the estimates are ordered, which is consistent with the prescribed order ${\mu }_1\le \dots \le {\mu }_{11}$. By the option “test=TRUE”, one can test if the ${\mu }_i$ values are properly ordered by a sequence of covariates. Figure 2 shows the output including the LOWESS plot and a correlation test based on Kendall’s tau via “cor.test(x,y,method=“kendall”)”. The test confirmed that the means were ordered by a “decreasing” sequence ($x_1>\dots >x_{11}$). We suggest the 10% significance level to declare the increasing or decreasing trend. In meta-analyses, 5% is too strict and not realistic since the number of studies is limited.

3.3. Shrinkage Estimators under Ordered Means

The estimator introduced above (in Section 3.2) can be improved by the JS type shrinkage. Based on the idea of Chang [35], Taketomi et al. [20] proposed a JS-type estimator under the following order restriction:

$${\mathit{\delta }}^{\mathrm{RJS}}\equiv \left(1-\frac{G-2}{{{\displaystyle \sum }}_{i=1}^G{Y}_i^2/{\sigma }_i^2}\right)\mathit{Y}I\left(Y_1\le \dots \le Y_G\right)+{\mathit{\delta }}^{\mathrm{RML}}\left(1-I\left(Y_1\le \dots \le Y_G\right)\right),$$

where “RJS” stands for “restricted JS”, $I(.)$ is the indicator function, where $I\left(A\right)=1$ or $I\left(A\right)=0$ if $A$ is true or false, respectively. Note that ${\mathit{\delta }}^{\mathrm{RJS}}$ has a smaller WMSE than ${\mathit{\delta }}^{\mathrm{RML}}$ [20,35], meaning that ${\mathit{\delta }}^{\mathrm{RJS}}$ gives more precise estimates than ${\mathit{\delta }}^{\mathrm{RML}}$.

The RJS estimator can be further corrected by the following positive-part RJS estimator:

$${\mathit{\delta }}^{\mathrm{RJS}+}\equiv {\left(1-\frac{G-2}{{{\displaystyle \sum }}_{i=1}^G{Y}_i^2/{\sigma }_i^2}\right)}^{+}\mathit{Y}I\left(Y_1\le \dots \le Y_G\right)+{\mathit{\delta }}^{\mathrm{RML}}\left(1-I\left(Y_1\le \dots \le Y_G\right)\right).$$

Consequently, ${\mathit{\delta }}^{\mathrm{RJS}+}$ has a smaller WMSE than ${\mathit{\delta }}^{\mathrm{RJS}}$[20].

In our R package, the function “rjs(.)” can compute ${\mathit{\delta }}^{\mathrm{RJS}}$ and ${\mathit{\delta }}^{\mathrm{RJS}+}$ possibly with the aid of covariates.

For instance, one can analyze the COVID-19 data as follows:

In the above commands, the input includes the optional argument “id” that signifies the leading authors and publication years of the 11 studies. We did so simply to make an informative output. If the input does not include this argument, the output shows the ordered sequence of 1, 2, …, 11 as shown in Section 3.2.

3.4. Estimators under Sparse Means

We now consider discrete shrinkage schemes by pre-testing $H_0:{\mu }_i=0$ vs. $H_1:{\mu }_i\ne 0$ for $i=1,2,\dots ,G$. The idea was proposed by Bancroft [36], who developed pretest estimators (see also more recent works [20,37,38,39,40,41,42,43]). In the meta-analytic context, Taketomi et al. [20] adopted the general pretest (GPT) estimator of Shih et al. [41], which is defined as follows:

$${\delta }_i^{\mathrm{GPT}}=Y_{i}I\left(\left|\frac{Y_i}{{\sigma }_i}\right|>z_{{\alpha }_1/2}\right)+q{Y}_{i}I\left(z_{{\alpha }_2/2}<\left|\frac{Y_i}{{\sigma }_i}\right|\le z_{{\alpha }_1/2}\right), i=1,2,\dots ,G.$$

Here $0\le {\alpha }_1\le {\alpha }_2\le 1$, $0<q<1$, and $z_p$ is the upper pth quantile of $N\left(0,1\right)$ for $0<p<1$. To implement the GPT estimator, the values of ${\alpha }_1$, ${\alpha }_2$, and $q$ must be chosen. For any value of ${\alpha }_1$ and ${\alpha }_2$, as well as a function $q$, one can show that ${\mathit{\delta }}^{\mathrm{GPT}}\equiv \left({\delta }_1^{\mathrm{GPT}},\dots ,{\delta }_G^{\mathrm{GPT}}\right)$ has smaller WMSE and TMSE values than $\mathit{Y}$, provided $\mathit{\mu }\approx \mathbf{0}$ [20].

One may choose $q=1/2$ (50% shrinkage), as suggested by [20,41]. To facilitate the interpretability of the pretests, one may choose ${\alpha }_1=0.05$ (5% level) and ${\alpha }_2=0.10$ (10% level). The resultant estimator is

$${\delta }_i^{\mathrm{GPT}}=Y_{i}I\left(\left|\frac{Y_i}{{\sigma }_i}\right|>1.96\right)+q{Y}_{i}I\left(1.645<\left|\frac{Y_i}{{\sigma }_i}\right|\le 1.96\right), i=1,2,\dots ,G.$$

The special case of ${\alpha }_1={\alpha }_2=\alpha =0.05$ leads to the usual pretest (PT) estimator

$${\delta }_i^{\mathrm{PT}}=Y_{i}I\left(\left|\frac{Y_i}{{\sigma }_i}\right|>1.96\right),$$

In our R package, the function “gpt(.)” can compute ${\mathit{\delta }}^{\mathrm{PT}}$ and ${\mathit{\delta }}^{\mathrm{GPT}}$. The significance levels and the shrinkage parameter are flexibly chosen by the following arguments:

• alpha1: significance level for ${\alpha }_1$ (0 < alpha1 < 1);
• alpha2: significance level for ${\alpha }_2$ (0 < alpha2 < 1);
• q: degrees of shrinkage for $q$ (0 < q < 1).

If users do not specify them, the default values ${\alpha }_1=0.05$, ${\alpha }_2=0.10$, and $q=0.5$ will be realized. The following is an example:

The output shows that ${\mathit{\delta }}^{\mathrm{PT}}$ and ${\mathit{\delta }}^{\mathrm{GPT}}$ lead to 0%, 50%, or 100% shrinkage of $Y$. The estimates of ${\mathit{\delta }}^{\mathrm{PT}}$ yield 0 for 12 studies, and the estimates of ${\mathit{\delta }}^{\mathrm{GPT}}$ yield 0 for 10 studies.

4. Simulation: Validating the R Package

This technical section is devoted to the numerical verification of the proposed package via Monte Carlo simulation experiments. Users of the package may skip this section.

We conducted simulations to investigate the operating performance of the seven estimators implemented in the proposed R package (Section 3). Our simulation design added new scenarios to the original ones in [20]. Hence, the simulation not only added new knowledge on the performance of the seven estimators, but it also validated the proposed R package.

4.1. Simulation Design

We considered the four scenarios for the true parameters $\mathit{\mu }=\left({\mu }_1,\dots ,{\mu }_G\right)$:

Scenario (a):

Ordered and non-sparse: $\mathit{\mu }=\left(-2,-2,-1,-1,0,0,1,1,2,2\right)$;

Scenario (b):

Ordered and sparse: $\mathit{\mu }=\left(0,0,0,0,0,0,0,0,2,4\right)$;

Scenario (c):

Unordered and non-sparse: $\mathit{\mu }=\left(1,-1,1,-1,1,-1,1,-1,1,-1\right)$;

Scenario (d):

Unordered and sparse: $\mathit{\mu }=\left(0,0,0,1,0,0,0,1,0,0\right)$;

Scenario (e):

Ordered and non-sparse: $\mathit{\mu }=\left(2,2,1,1,0,0,-1,-1,-2,-2\right)$;

Scenario (f):

Ordered and sparse: $\mathit{\mu }=\left(0,0,0,0,0,0,0,0,-2,-4\right)$.

These scenarios were not considered by our previous simulation studies [20].

We generated normally distributed data $Y_i~N\left({\mu }_i,{\sigma }_i^2\right)$ where ${\sigma }_i^2~{\chi }_{df=1}^2/4$ was restricted (truncated) in ${\sigma }_i^2\in \left[0.009,0.6\right]$, $i=1,2,\dots ,G$, as previously considered [20,44]. Using the data $\mathit{Y}\equiv \left(Y_1,\dots ,Y_G\right)$, we applied the proposed R package meta.shrinkage to compute ${\mathit{\delta }}^{\mathrm{JS}}$, ${\mathit{\delta }}^{\mathrm{JS}+}$,${\mathit{\delta }}^{\mathrm{RML}}$, ${\mathit{\delta }}^{\mathrm{RJS}}$, ${\mathit{\delta }}^{\mathrm{RJS}+}$, ${\mathit{\delta }}^{\mathrm{PT}}$, and ${\mathit{\delta }}^{\mathrm{GPT}}$ for estimating $\mathit{\mu }$. We examined how these estimators improved upon the standard estimator $\mathit{Y}$ in terms of the TMSE and WMSE.

Our simulations were based on 10,000 repetitions using ${\mathit{Y}}^{\left(r\right)}$, where $1\le r\le \mathrm{10,000}$. Let $\mathit{\delta }\left({\mathit{Y}}^{\left(r\right)}\right)=\left({\delta }_1\left({\mathit{Y}}^{\left(r\right)}\right),\dots ,{\delta }_G\left(Y^{\left(r\right)}\right)\right)$ be one of the seven estimators in the $r$th repetition. To assess the TMSE, we computed it via the Monte Carlo average:

$$\mathrm{TMSE}\equiv \frac{1}{\mathrm{10,000}}{\displaystyle \sum }_{r=1}^{\mathrm{10,000}}\left[{\displaystyle \sum }_{i=1}^G{\left({\delta }_i\left({\mathit{Y}}^{\left(r\right)}\right)-{\mu }_i\right)}^2\right].$$

As the TMSE and WMSE gave the same conclusion, we reported on the former.

Appendix A provides the R code for the simulations, which can reproduce the results of the following section.

4.2. Simulation Results

Figure 3 compares the estimators $\mathit{Y}$, ${\mathit{\delta }}^{\mathrm{JS}}$, ${\mathit{\delta }}^{\mathrm{JS}+}$,${\mathit{\delta }}^{\mathrm{RML}}$, ${\mathit{\delta }}^{\mathrm{RJS}}$, ${\mathit{\delta }}^{\mathrm{RJS}+}$, ${\mathit{\delta }}^{\mathrm{PT}}$, and ${\mathit{\delta }}^{\mathrm{GPT}}$.

In Scenarios (a) and (e), the smallest TMSE values were achieved by ${\mathit{\delta }}^{\mathrm{RML}}$, ${\mathit{\delta }}^{\mathrm{RJS}}$, and ${\mathit{\delta }}^{\mathrm{RJS}+}$, which appropriately accounted for the ordered means. Thus, these ordered mean estimators provided some advantages over the standard estimator $\mathit{Y}$. Here, users needed to specify the option “decreasing = FALSE” (Scenario (a)) or “decreasing = TRUE” (Scenario (e)) to capture the true ordering of the means. On the other hand, ${\mathit{\delta }}^{\mathrm{PT}}$ and ${\mathit{\delta }}^{\mathrm{GPT}}$ produced unreasonably large TMSE values, since they wrongly imposed the sparse mean assumptions.

In Scenario (b), the smallest TMSE values were attained by ${\mathit{\delta }}^{\mathrm{PT}}$ followed by ${\mathit{\delta }}^{\mathrm{GPT}}$, as they took advantage of the sparse means. The TMSE values for ${\mathit{\delta }}^{\mathrm{RML}}$, ${\mathit{\delta }}^{\mathrm{RJS}}$, and ${\mathit{\delta }}^{\mathrm{RJS}+}$ were also small by accounting for the ordered means. Hence, these pretest and restricted estimators produced significant advantages over the standard estimator $\mathit{Y}$.

In Scenario (c), ${\mathit{\delta }}^{\mathrm{JS}}$ and ${\mathit{\delta }}^{\mathrm{JS}+}$ performed the best, but the advantage over $\mathit{Y}$ was modest. In this scenario, ${\mathit{\delta }}^{\mathrm{RML}}$, ${\mathit{\delta }}^{\mathrm{RJS}}$, and ${\mathit{\delta }}^{\mathrm{RJS}+}$ produced quite large TMSE values and performed the worst, since they wrongly assumed the ordered means. Additionally, ${\mathit{\delta }}^{\mathrm{PT}}$ and ${\mathit{\delta }}^{\mathrm{GPT}}$ produced large TMSE values since they wrongly assumed the sparse means. This was the only scenario where the standard estimator $\mathit{Y}$ was enough.

In Scenario (d), the smallest TMSE values were attained by ${\mathit{\delta }}^{\mathrm{GPT}}$, as it captured the sparse means. The performance of ${\mathit{\delta }}^{\mathrm{PT}}$, ${\mathit{\delta }}^{\mathrm{JS}}$, and ${\mathit{\delta }}^{\mathrm{JS}+}$ was also good. On the other hand, ${\mathit{\delta }}^{\mathrm{RML}}$, ${\mathit{\delta }}^{\mathrm{RJS}}$, and ${\mathit{\delta }}^{\mathrm{RJS}+}$ gave large TMSE values due to the unordered means.

In summary, our simulations demonstrated that the seven estimators implemented in the proposed R package exhibited desired operating characteristics. If the true means were ordered, the restricted estimators (${\mathit{\delta }}^{\mathrm{RML}}$, ${\mathit{\delta }}^{\mathrm{RJS}}$, and ${\mathit{\delta }}^{\mathrm{RJS}+}$) showed definite advantages over the standard estimator $\mathit{Y}$. In addition, the pretest estimators (${\mathit{\delta }}^{\mathrm{PT}}$ and ${\mathit{\delta }}^{\mathrm{GPT}}$) produced the best performance under the sparse means. Finally, the JS estimators (${\mathit{\delta }}^{\mathrm{JS}}$ and ${\mathit{\delta }}^{\mathrm{JS}+}$) modestly but uniformly improved upon $\mathit{Y}$ across all the scenarios.

We therefore conclud that there are good reasons to apply the proposed R package to estimate $\mathit{\mu }$ in order to improve the accuracy of estimation.

5. Data Example

This section analyzes a dataset to illustrate the methods in the proposed package, demonstrating their possible advantages over the standard meta-analysis. Appendix B provides the R code, which can reproduce the following results.

We used the blood pressure dataset containing $G=10$ studies, where each study examined the effect of a treatment to reduce blood pressure. The dataset is available in the R package mvmeta. Available online: https://CRAN.R-project.org/package=mvmeta (accessed on 14 November 2021). Each study provided the treatment’s effect estimate on the systolic blood pressure (SBP) and the treatment’s effect on the diastolic blood pressure (DBP), as shown in

Table 1. The 10 studies from the blood pressure data. Each study provided the treatment’s effect on the systolic blood pressure (SBP) and the treatment’s effect on the diastolic blood pressure (DBP).

	Treatment Effect on SBP	SE	Treatment Effect on DBP	SE
Study 1	−6.66	0.72	−2.99	0.27
Study 2	−14.17	4.73	−7.87	1.44
Study 3	−12.88	10.31	−6.01	1.77
Study 4	−8.71	0.30	−5.11	0.10
Study 5	−8.70	0.14	−4.64	0.05
Study 6	−10.60	0.58	−5.56	0.18
Study 7	−11.36	0.30	−3.98	0.27
Study 8	−17.93	5.82	−6.54	1.31
Study 9	−6.55	0.41	−2.08	0.11
Study 10	−10.26	0.20	−3.49	0.04

. In the following analysis, we focus on the treatment’s effect estimates for the SBP and regard those for the DBP as covariates.

We aimed to improve the individual treatment effects on the SBP by the methods in the R package meta.shrinkage. For this purpose, we utilized the covariate information to implement meta-analyses under ordered means (Section 3.2 and Section 3.3).

To apply the proposed methods, we changed the order of the 10 studies by the increasing order of the covariates (see the first column of

Table 2. The treatment effect estimates on the SBP based on the 10 studies from the blood pressure data. The 10 studies are ordered by the covariates (treatment effect estimates on the DBP).

	Covariate	$\mathit{Y}$	${\mathit{\delta }}^{\mathbf{JS}}$	${\mathit{\delta }}^{\mathbf{JS}+}$	${\mathit{\delta }}^{\mathbf{RML}}$	${\mathit{\delta }}^{\mathbf{RJS}}$	${\mathit{\delta }}^{\mathbf{RJS}+}$
Study 2	−7.87	−17.93	−17.91	−17.91	−16.05	−16.05	−16.05
Study 8	−6.54	−10.26	−10.25	−10.25	−16.05	−16.05	−16.05
Study 3	−6.01	−14.17	−14.16	−14.16	−12.88	−12.88	−12.88
Study 6	−5.56	−11.36	−11.35	−11.35	−10.6	−10.6	−10.6
Study 4	−5.11	−8.71	−8.70	−8.70	−9.76	−9.76	−9.76
Study 5	−4.64	−10.6	−10.59	−10.59	−9.76	−9.76	−9.76
Study 7	−3.98	−8.7	−8.69	−8.69	−9.76	−9.76	−9.76
Study 10	−3.49	−12.88	−12.87	−12.87	−9.76	−9.76	−9.76
Study 1	−2.99	−6.66	−6.65	−6.65	−6.66	−6.66	−6.66
Study 9	−2.08	−6.55	−6.54	−6.54	−6.55	−6.55	−6.55

). Under this order, we defined $Y_i$, where $i=1,\dots ,10$ as the treatment effect estimates on the SBP (see $\mathit{Y}$ in the second column of Table 2). Table 2 shows a good concordance between $\mathit{Y}$ and the covariates; the smallest covariate (−7.87) yielded the smallest outcome ($Y_1=-17.93$), and the largest covariate (−2.08) yielded the largest outcome ($Y_{11}=-6.55$). However, the values of $Y_i$ were not perfectly ordered.

We therefore considered the order-restricted estimators (${\mathit{\delta }}^{\mathrm{RML}}$, ${\mathit{\delta }}^{\mathrm{RJS}}$, and ${\mathit{\delta }}^{\mathrm{RJS}+}$) by imposing the assumption that the true treatment effects were ordered. Table 2 shows that these restricted estimators satisfied ${\delta }_1\le \dots \le {\delta }_{10}$ (see the fifth through the seventh columns of Table 2). Since it was reasonable to impose a concordance between the treatment effects on the SBP and DBP, these estimators may have been advantageous over the standard estimates ($\mathit{Y}$). In this data example, the JS estimators (${\mathit{\delta }}^{\mathrm{JS}}$ and ${\mathit{\delta }}^{\mathrm{JS}+}$) were almost identical to the standard estimates $\mathit{Y}$. Hence, there would be little advantage to shrinkage in the JS estimators.

By the “rml(.)” function, we tested if the ${\mu }_i$ values were ordered by a sequence of covariates. Figure 4 shows a correlation-based test and the LOWESS plot. The test confirmed that the means were ordered by an increasing sequence ($x_1<\dots <x_{10}$). The P-value was highly significant (p = 0.002). Therefore, we validated the assumption of ordered means.

6. Conclusions and Future Extensions

This article introduced an R package meta.shrinkage (https://CRAN.R-project.org/package=meta.shrinkage (accessed on 15 November 2021)), which we made freely available on CRAN. It was first released on 19 November 2021 (version 0.1.0), following our original methodological article published on 20 October 2021 [20]. We hope that the timely release of our package facilitates the appropriate use of the proposed methods for interested readers. As the precision and reliability of the developed statistical methods are important, we conducted extensive simulation studies to validate the proposed R package (Section 4). We also analyzed a data example to show the ability of the R package (Section 5).

To implement isotonic regression in our R package, we proposed a matrix-based algorithm for the PAVA (Figure 1). This algorithm is easy to program in the R environment, where matrices are convenient to manipulate. However, if one tries to implement the PAVA in other programing environments, the matrix-based algorithm may be inefficient, especially for a meta-analysis with a very large number of studies. The number $G>500$ makes the proposed algorithm slow to compute the output. However, real meta-analyses rarely have more than 100 studies, so this issue may not arise at the practical level.

An extension of the present R packages to multiple responses is an important research topic. An example includes a meta-analysis of verbal and math test scores [45], consisting of two responses. A similar instance involves math and stat tests [44,46]. There is much room for meta-analyzing bivariate and multivariate responses [47,48,49,50,51,52,53,54,55,56]. Multivariate shrinkage estimators of multivariate restricted and unrestricted normal means, such as those in [57,58,59,60], can be considered for this extension.

The proposed R package can only handle normally distributed data; it cannot handle data that are non-normally distributed, asymmetrically distributed, or discrete-valued. To analyze such data, one should consider extensions of the meta-analysis methods in [20] toward asymmetric distributions for skewed or discrete response variables. Shrinkage and pretest estimators exist, such as [61] for the exponential distribution, [62] for the gamma distribution, and [63,64] for the Poisson distribution. Thus, meta-analytical applications of shrinkage estimators to asymmetric or non-normal models are relevant research directions.

We left a comparison of the estimators in our package with the Bayesian random-effects meta-analyses. For instance, the performance of the proposed estimators could be compared with the Bayesian posterior mean estimators, which could be computed from the bayesmeta package [15]. However, one needs to specify the prior mean and prior standard deviation to perform the Bayesian meta-analyses. Therefore, it remains unclear how to perform a fair comparison between the Bayesian and non-Bayesian estimators, as found in [65,66]. A carefully designed comparative study will be helpful for guiding users to apply two R packages: meta.shrinkage and bayesmeta.