On the Appropriateness of Fixed Correlation Assumptions in Repeated-Measures Meta-Analysis: A Monte Carlo Assessment

Abstract

In repeated-measures meta-analyses, raw data are often unavailable, preventing the calculation of the correlation coefficient r between pre- and post-intervention values. As a workaround, many researchers adopt a heuristic approximation of r = 0.7. However, this value lacks rigorous mathematical justification and may introduce bias into variance estimates of pre/post-differences. We employed Monte Carlo simulations (n = 500,000 per scenario) in Fisher z-space to examine the distribution of the standard deviation of pre-/post-differences (σ_D) under varying assumptions of r and its uncertainty (σ_r). Scenarios included r = 0.5, 0.6, 0.707, 0.75, and 0.8, each tested across three levels of variance (σ_r = 0.05, 0.1, and 0.15). The approximation of r = 0.75 resulted in a balanced estimate of σ_D, corresponding to a “midway” variance attenuation due to paired data. This value more accurately offsets the deficit caused by assuming a correlation, compared to the traditional value of 0.7. While the r = 0.7 heuristic remains widely used, our results support the use of r = 0.75 as a more mathematically neutral and empirically defensible alternative in repeated-measures meta-analyses lacking raw data.

1. Introduction

Many authors aiming to conduct a meta-analysis involving paired data on continuous numerical variables typically use the arithmetic difference between pre- and post-intervention measurements as the effect size. However, in doing so, they often lack raw data, which forces them to rely solely on aggregated statistics (i.e., pre- and post-means and their corresponding standard deviations). This reliance complicates the direct and accurate estimation of the mean of the pre/post-differences—and, more critically, its variability. The main reason is that in many individual studies, the correlation coefficient r between pre- and post-values either is not reported or cannot be computed, making it impossible to use this parameter in indirect variance estimation. Some statistical software, such as UNISTAT, assumes a correlation of 0.5 by default, in case this information is not readily available [1]. However, many authors—including the present author—commonly adopt an arbitrary value of r = 0.7 [2,3,4,5,6,7], an approach based on earlier work beginning with Rosenthal [8]. This communication article aims to examine whether this approach is empirically defensible, whether there is a mathematically valid rationale for it, and the consequences of its use. Finally, this article explores potentially sound alternatives to the already proposed fixed-r assumptions.

2. Materials and Methods

Let μ_pre and μ_post be the mean of the values between two timepoints in a repeated-measures design study, i.e., before and after the intervention (pre- and post-values, respectively) and r be the Pearson’s correlation coefficient between pre- and post-values. Additionally, let σ_pre and σ_post be the standard deviations corresponding to μ_pre and μ_post, respectively.

The mean of the pre-/post-differences μ_D is given by the formula μ_D = μ_post − μ_pre. Of note, the standard deviation σ_D of μ_D is given by the formula σ_D = (σ_pre² + σ_post² − 2rσ_preσ_post)^1/2.

To approximate the expected σ_D (℮|σ_D|), we used Monte Carlo simulations (500,000 per condition) in Fisher z-space, applying the formula z = tanh⁻¹(r) = 1/2ln[(1 + r)/(1 − r)] and back-transforming to r ∈ (−1,1) using r = tanh(z). While the Central Limit Theorem (CLT) does not imply normality for r itself, it motivates the approximation of the Fisher z-transformation of r as approximately normal in large samples with variance 1/(n − 3) for sample size n. When standardized normal distribution is implemented, σ_D = [2σ²(1 − r)]^1/2 becomes σ_D = (1 − r)^1/2 [9].

All simulations were performed using R version 4.5.1 (R Core Team, 2024); the complete R code is provided as Appendix A.

ChatGPT 4 was implemented to partly construct the R code and generate Figures 2 and 4.

3. Results

3.1. What Are the Consequences of r Uncertainty?

Let us consider a hypothetical infinite-sample scenario and investigate what happens to the σ_D when μ_D is constant and r is a random variable that is distributed normally presenting a mean of 0.5, under the assumption of equal pre-/post-variances (σ_pre = σ_post = σ).

By substituting σ_pre = σ_post = σ into the formula σ_D = (σ_pre² + σ_post² − 2rσ_preσ_post)^1/2, we receive σ_D = [2σ²(1 − r)]^1/2. Thus, the expected σ_D, ℮|σ_D|, is given by the formula ℮|σ_D|= 2^1/2σ℮|(1 − r)^1/2|, where ℮|(1 − r)^1/2| is the expected (1 − r)^1/2. Note that, despite r being hypothesized to be distributed normally, it must be clipped to [0,1] to be interpretable; therefore, (1 − r)^1/2 is not normally distributed. Consequently, ℮|(1 − r)^1/2| generally has no closed form but can be approximated numerically or via Taylor expansion, particularly if the distribution of r is tightly concentrated around its mean [10].

Applying second-order Taylor expansion for f(X) = X^1/2 = (1 − r)^1/2, ℮|f(X)| ≈ f(μ_X) + 1/2f″(μ_X)Var(Χ).

By substituting f(X) = (1 − r)^1/2, μ_X = 0.5, Var(Χ) = σ_X² = σ_r², f(μ_X) = μ_X^1/2, and f″(μ_X) = −1/4μ_X^−3/2, ℮|(1 − r)^1/2| ≈ (0.5)^1/2 − 1/8(0.5)^−3/2σ_r² = (0.5)^1/2 − 1/2(0.5)^1/2σ_r².

Consequently, ℮|σ_D| = 2^1/2σ℮|(1 − r)^1/2| => ℮|σ_D| = 2^1/2σ[(0.5)^1/2 − 1/2(0.5)^1/2σ_r²] => ℮|σ_D|= σ[1 − 1/2σ_r²].

For a reasonable value of σ_r, e.g., σ_r = 0.1, we receive ℮|σ_D| ≈ σ[1 − 1/2(0.1)²] = 0.995σ. This implies that even considerable uncertainty in r results in a slight inflation of the variability in the pre/post-difference.

In its general form (i.e., for any r), ℮|σ_D| = 2^1/2σ{(1 − r)^1/2 − 1/[8(1 − r)^3/2]σ_r²}. For ℮|σ_D|< 0, the Taylor approximation breaks down, yielding negative or undefined results. Solving for ℮|σ_D|= 0, the critical value for this breakdown is σ_r(crit) = 2^3/2(1 − r). A graphical representation of the validity of the Taylor approximation is provided in Figure 1.

3.2. Does the r = 0.7 Heuristic Satisfy the “Midway” Hypothesis for Equal Pre/Post-Variances?

Let us examine the case under the assumption of r = 1/2^1/2 ≈ 0.707 ≈ 0.7 and the presence of equal pre/post-variances (σ_pre = σ_post = σ).

Under these circumstances, the formula σ_D = (σ_pre² + σ_post² − 2rσ_preσ_post)^1/2 would become σ_D = (2σ² − 2rσ²)^1/2 = σ[2(1 − (1/2^1/2)] ≈ 0.765σ (Figure 2).

The standard deviation of the pre/post-difference is smaller than the standard deviation of the pre- and post-values due to their positive correlation. Of note, assuming r = 1/2^1/2 ≈ 0.707 helps to counterbalance the over- and underestimation of the true variability in repeated-measures designs; however, this is an arbitrary choice that has no further mathematical documentation.

In detail, in the case of the absence of any correlation (r = 0), the formula σ_D = (σ_pre² + σ_post² − 2rσ_preσ_post)^1/2 gives σ_D = (2^1/2)σ ≈ 1.414 σ. Accordingly, when there is an absolute correlation (r = 1), the formula σ_D = (σ_pre² + σ_post² − 2rσ_preσ_post)^1/2 gives σ_D = 0.

Under these circumstances, a “midway” assumption would have to result in σ_D ≈ 0.707 σ. Solving σ_D = (σ_pre² + σ_post² − 2rσ_preσ_post)^1/2 for σ_D ≈ 0.707 σ, under the assumption of equal pre/post-variances (σ_pre = σ_post= σ), we obtain r = 0.75.

3.3. Simulation of Uncertainty of σ_D to Examine the Feasibility of the Proposed r = 0.75 for Equal Pre/Post-Variances

The above-mentioned theoretical approach to estimate σ_D has the disadvantage of using a truncated normal distribution for r. This approximation is considered tolerable as σ_r decreases.

A more accurate approach to estimate σ_D is the use of 500,000 Monte Carlo simulations per condition in Fisher z-space using the formula z = tanh⁻¹(r) = 1/2ln[(1 + r)/(1 − r)] and back-transforming to r ∈ (−1,1) using r = tanh(z). In this special case, σ_D = [2σ²(1 − r)]^1/2 would become σ_D = (1 − r)^1/2, as σ = 1 for the implemented standardized normal distribution.

In detail, we used Monte Carlo simulations to analyze five sets of scenarios (targeting r = 0.5, r = 0.6, r = 0.707, r = 0.75, and r = 0.8) to investigate ℮|σ_D|. For each set of scenarios, we examined three levels of uncertainty of σ (σ_r): 0.05, 0.1, and 0.15. As expected, increasing the correlation reduces variability in the pre/post-differences. A box-and-whisker plot showing the distribution of σ_D for all considered levels of uncertainty of σ_r is depicted in Figure 3; note that for r = 0.75 and σ_r = 0.1, ℮|σ_D| is close to the theoretical approximation of 0.707σ.

Furthermore, Figure 4 demonstrates the limit behavior of ℮|σ_D| versus σ_r²; for reasonable values of σ_r² (i.e., ≤0.2), ℮|σ_D| is slightly affected. Of note, for σ_D² < 0, the Taylor expansion breaks down, yielding negative or imaginary results that are no longer valid.

3.4. The “Midway” Hypothesis for Proposed r Under Unequal Pre/Post-Variances

Let us examine the case of unequal pre/post-variances (σ_pre ≠ σ_post = σ), where σ_pre/σ_post = b ≠ 1. In that case, σ_D = σ_post (b² + 1 − 2rb)^1/2. Of note, σ_D is minimum for r = 1 (σ_D(min) = σ_post |b − 1|) and maximum for r = 0 (σ_D(max) = σ_post (b² + 1)^1/2). As such, the “midway” assumption would be satisfied for σ_post = (σ_D(min) + σ_D(max))/2 = [|b − 1| + (b² + 1)^1/2]/2.

Solving for r, we get r = {b² + 1 − {[|b − 1| + (b² + 1)^1/2]/2}²}/2b. The function r(b) is non-monotonic, with a peak at b = 1, where r = 0.75 (Figure 5).

Moreover, solving r = {b² + 1 − {[|b − 1| + (b² + 1)^1/2]/2}²}/2b for b > 0, the empirically used r = 0.7 corresponds to b values of ~0.85 and ~1.17.

4. Discussion

This study provides a detailed and mathematically sound approach for the heuristic approximation of the correlation coefficient r between pre/post-values of continuous numerical variables in meta-analyses, particularly when r is absent or cannot be computed from the provided data. Moreover, our analysis shows how variation in assumed r influences derived metrics like σ_D, and we suggest that such assumptions be treated transparently and tested via sensitivity analysis when raw data are not available.

The proposed value of r = 0.75 is justified as it represents the midpoint between the minimum and the maximum variance of the pre/post-difference under the assumption of equal variances. The latter theoretical approach yields to symmetry in the Taylor approximation and could be reasonable in the absence of access to raw data. In this setting, any other arbitrarily chosen value, including the empirically used value of r = 0.7—though not strictly problematic—deviates from the described “mid-point” variance approach. Moreover, we have shown that the empirically used r = 0.7 corresponds to σ_pre/σ_post = b values of ~0.85 and ~1.17 under the “midpoint hypothesis”. As such, r = 0.7 predisposes per se an asymmetry in pre/post-variances. Whether this might effectively buffer cluster-induced heterogeneity in variance structures remains to be investigated.

The initial assumption of r = 0.5 as the mean of the correlation distribution between pre/post-measurements in paired data is not derived from a formal mathematical theorem but rather reflects a long-standing empirical convention adopted across various applied disciplines. This assumption has been pragmatically used in statistical software (e.g., UNISTAT version 10), methodological guides (e.g., the Campbell Collaboration’s effect size calculator), and in the meta-analytic literature, where it often appears alongside sensitivity analyses exploring values such as 0.3, 0.5, and 0.7 [1,9].

Rosenthal proposed that, under the prerequisite of statistical significance, a value of 0.7 can be arbitrarily attributed to r, if it is unknown [8]. This approach has since been followed by various authors in the field of social research [4,5,7]. However, its use has rapidly extended to a wide range of fields, including biosciences [2,3,6,11,12,13]. As we have demonstrated, this approach does not equally counterbalance the under- and over-estimation of correlation-attributable variance when normality is met and equal pre/post-variance can be assumed. In contrast, implementing a value of r = 0.75 can accurately balance the variance deficit due to the correlation of paired data. Of note, we have demonstrated that all the above-mentioned values of r (either r = 0.5, r = 0.707, or r = 0.75) can tolerate a considerable amount of σ_r.

Whatever the assumption for r might be, the goal is to provide a reasonable estimate of the within-subject correlation, where a moderate effect is anticipated but not precisely known. Despite this assumption lacking a universal theoretical foundation, one may be tempted to justify it heuristically by noting that the correlation coefficient r takes values from the bounded interval [0, 1] (in pre/post-designs where positive correlation is expected) and that 0.5 might represent a midpoint or central tendency of plausible values in empirical settings. However, this is not a consequence of the CLT. While the CLT ensures that the sampling distribution of an estimated correlation becomes approximately normal with increasing sample size, it centers on the true population value of r—which remains unknown in many studies. Similarly, the Law of Large Numbers ensures that the mean of observed sample correlations across studies will converge on the true population mean, but again, this does not specify what that mean should be.

Beyond the need for approximation, an aspect that has been somewhat overlooked by many researchers is when μ_pre and μ_post (pre- and post-means), σ_pre and σ_post (pre- and post-standard deviations), n (sample size), and the p-value of the pre/post-difference are all known or can be directly computed by available data. In these cases, the true value of r can be computed, and therefore there is no need for an indirect approximation. In detail, (i) knowing the p-value, the t-value can be determined using inverse t-distribution; (ii) knowing the t-value, the Cohen’s d can be computed using the formula d = t/n^1/2; (iii) the Cohen’s d can be converted to the correlation coefficient r via the formula r = d/(d² + 4)^1/2; and (iv) σ_D can be directly obtained as (σ_pre² + σ_post² − 2rσ_preσ_post)^1/2. Solving for r from the latter, the well-established formula r = (σ_pre² + σ_post² − σ_D²)/2σ_preσ_post, proposed by the Cochrane Collaboration, can be obtained [14].

While many prior approaches have relied on fixed assumptions of r, the present analysis introduces a probabilistic framework, modeling r as a normally distributed variable around the theoretical “true” mean. This strategy explicitly incorporates uncertainty and allows for plausible between-study variation in r, thereby improving the realism and robustness if the standard deviation estimates. However, this approach does not fully address potential cluster effects, which account for the systematic variation in correlation structure across studies due to shared methodological, contextual, or population-level factors [15,16]. The assumed normal distribution of r is independent across simulations and does not capture hierarchical relationships or structural dependencies due to, e.g., the same researchers. As such, the model may still underestimate total heterogeneity in the presence of latent clustering.

The primary limitation of this study is that the data used is simulated rather than real. Ideally, future research should explore how different assumptions about the correlation coefficient r—including both the empirically adopted value of r = 0.7 and the theoretically motivated r = 0.75—approximate results from real datasets, particularly under the assumption that pre- and post-intervention variances are equal. Additionally, this line of investigation should be extended to scenarios where pre- and post-variances are unequal. In such cases, it may be possible to identify a best-fitting value of r that depends on the ratio of pre- to post-variance b. The more extensive and diverse the datasets are, especially if they span a wide range of biomedical fields, the greater the potential validity and generalizability of the findings are anticipated to be.

Apart from r approximation, another valid approach is the implementation of a sensitivity analysis for different values of r. This method has been applied experimentally for r ∈ ℝ ∩ [0.5, 0.7] when pseudo-individual participant data is used to synthesize clinical study evidence [17]. A similar approach has been adopted in clinical meta-analyses by other researchers for a wide range of r values [18]. Additionally, the use of r derived from the raw data of even one study might further support the quality of the evidence [18]. In these cases, the correlation coefficient r can be further approximated as the mean of a distribution estimated via the maximum likelihood approach, assuming normality and a known variance. If the variance is unknown and cannot be approximated, at least three studies directly reporting r are required to estimate both the mean and the variance.

5. Conclusions

In conclusion, when r cannot be directly computed, it can be approximated; this study supports the use of r = 0.75, which represents half of the maximum variance that can be attributed to the correlation between paired data in the case of normality and equal variance. As such, we recommend using r = 0.75 as a default in the absence of raw data, paired with sensitivity analyses.