Wang et al. [] conducted a study to “determine the effectiveness of and the differences between different physical exercise programs in improving preschool children’s physical fitness”. The study is identified as a cluster-randomized controlled trial (cRCT) in the article. As described in the article, preschool children were recruited from five kindergartens, and groups were randomized by kindergarten to four interventions and one control group. That is, there was one kindergarten assigned to each of the five study groups, rendering the study design invalid for accurately estimating the causal effect of treatment assignment.
In cRCTs, the presence of correlated data within each cluster is expected and often observed, which means that assumptions of independence for errors (or model residuals) across study participants within clusters may not hold. Moreover, in studies where clusters, rather than individuals, are assigned to study conditions, individuals are nested within clusters, which creates a hierarchical data structure []. Hence, the available degrees of freedom for testing the intervention effect rely on the number of individual participants and the number of clusters allocated to each condition []. In a purported cRCT with one cluster allocated to each of the conditions, such as Wang et al.’s study, it is not possible to estimate the variation attributed to cluster membership separately from the variation attributed to treatment assignment. Therefore, intervention effects and uncertainty cannot be estimated using such a design [] and the trial should not be labeled as a cRCT.
Although assigning clusters (i.e., kindergartens) instead of individuals can be justified in Wang et al.’s study, the confounding of treatment and clusters by having one cluster per condition does not allow the level of causal inference about the intervention effect that can be drawn from cRCTs. Therefore, the study should be labeled as a quasi-experimental study with numerical results interpreted as evidence of associations to clarify the weaker causal inferences from quasi-experimental compared to randomized designs [].
Funding
Supported in part by the Gordon and Betty Moore Foundation and NIH grants R25HL124208, R25DK090880, and P20GM109096. The opinions expressed are those of the authors and do not necessarily represent those of the NIH or any other organization.
Conflicts of Interest
David B. Allison and his institution (Indiana University) have received payments for consultation, grants, contracts, in-kind donations, and contributions from multiple for-profit and not-for-profit entities interested in the statistical design and analysis of experiments, but not directly related to the research questions addressed in the paper in question.
References
- Wang, G.; Zeng, D.; Zhang, S.; Hao, Y.; Zhang, D.; Liu, Y. The Effect of Different Physical Exercise Programs on Physical Fitness among Preschool Children: A Cluster-Randomized Controlled Trial. Int. J. Environ. Res. Public Health 2023, 20, 4254. [Google Scholar] [CrossRef]
- Murray, D.M.; Varnell, S.P.; Blitstein, J.L. Design and analysis of group-randomized trials: A review of recent methodological developments. Am. J. Public Health 2004, 94, 423–432. [Google Scholar] [CrossRef] [PubMed]
- Jamshidi-Naeini, Y.; Brown, A.W.; Mehta, T.; Glueck, D.H.; Golzarri-Arroyo, L.; Muller, K.E.; Tekwe, C.D.; Allison, D.B. A practical decision tree to support editorial adjudication of submitted parallel cluster randomized controlled trials. Obesity 2022, 30, 565–570. [Google Scholar] [CrossRef]
- Varnell, S.P.; Murray, D.M.; Baker, W.L. An evaluation of analysis options for the one-group-per-condition design: Can any of the alternatives overcome the problems inherent in this design? Eval. Rev. 2001, 25, 440–453. [Google Scholar] [CrossRef] [PubMed]
- Shadish, W.R.; Cook, T.D.; Campbell, D.T. Experimental and Quasi-Experimental Designs for Generalized Causal Inference; Houghton Mifflin: Boston, MA, USA, 2002. [Google Scholar]
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).