Next Article in Journal
Dermoscopic Features of Acute, Subacute, Chronic and Intermittent Subtypes of Cutaneous Lupus Erythematosus in Caucasians
Previous Article in Journal
Associations among Physician–Patient Communication, Patient Satisfaction, and Clinical Effectiveness of Overactive Bladder Medication: A Survey of Patients with Overactive Bladder
 
 
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Commentary

Misleading Meta-Analyses during COVID-19 Pandemic: Examples of Methodological Biases in Evidence Synthesis

by 1 and 2,*
1
ELKH-DE Public Health Research Group of the Hungarian Academy of Sciences, Department of Public Health and Epidemiology, Faculty of Medicine, University of Debrecen, Kassai út 26, 4028 Debrecen, Hungary
2
Institute of Social and Preventive Medicine (ISPM), University of Bern, Mittelstrasse 43, 3012 Bern, Switzerland
*
Author to whom correspondence should be addressed.
J. Clin. Med. 2022, 11(14), 4084; https://doi.org/10.3390/jcm11144084
Received: 10 June 2022 / Revised: 4 July 2022 / Accepted: 12 July 2022 / Published: 14 July 2022

Abstract

:
Not all evidence is equal. Evidence-based public health and medicine emanate from the principle that there is a hierarchy of evidence, with systematic reviews and meta-analyses (SRMAs) being at the top, as the highest level of evidence. Despite this, it is common in literature to find SRMAs with methodological issues that can distort the results and can thus have serious public health or clinical implications. During the Coronavirus Disease 2019 (COVID-19) pandemic, the importance of evidence and the way in which evidence was produced was stress tested and revealed a wide array of methodological biases that might have led to misleading conclusions and recommendations. We provide a critical examination of methodological biases in selected SRMAs on COVID-19, which have been widely used to guide or justify some pharmaceutical and nonpharmaceutical interventions with high public health and clinical significance, such as mask wearing, asymptomatic transmission, and ivermectin. Through these selected examples, we highlight the need to address biases related to the methodological quality and relevance of study designs and effect size computations and considerations for critical appraisal of available data in the evidence synthesis process for better quality evidence. Such considerations help researchers and decision makers avoid misleading conclusions, while encouraging the provision of the best policy recommendations for individual and public health.

1. Introduction

Evidence-based public health and medicine researchers have continuously exposed the fact that while many evidence synthesis efforts for widely used healthcare interventions show potential benefits, there is little high-quality evidence supporting their effectiveness and safety [1]. In the past years, there has been a massive explosion of unnecessary, misleading, and conflicted systematic reviews and meta-analyses (SRMAs) [2]. This phenomenon has persisted during the Coronavirus disease 2019 (COVID-19) pandemic. This has led to evidence synthesis efforts often serving mostly as easily produced publishable units or marketing tools, instead of promoting evidence-based public health. Although there are examples of evidence synthesis efforts that have supported best practices for healthcare during the pandemic [3], poor methodological quality and misleading SRMAs can be harmful given the major prestige and influence these types of studies have acquired. Even when rigorous SRMAs are too slow or difficult to conduct, the pandemic served as a reminder that it is still possible to recommend best practices and avoid methodological issues that are uncommon. Here, we present selected frequent methodological biases of SRMAs that had high visibility and were used to guide decisions on asymptomatic transmission, mask wearing, and ivermectin.

2. Asymptomatic Transmission in COVID-19

Information on asymptomatic transmission has high public health relevance as it dictates population-level measures, epidemiological practice, and routine outpatient care. In 2020, an SRMA of observational data showed the secondary attack rate was lower in contacts of people with asymptomatic infection than those with symptomatic infection (relative risk 0.35, 95% CI 0.10–1.27), suggesting that there is asymptomatic transmission, but at a lower rate compared to symptomatic transmission [4]. The article has been cited over 600 times and shared more than 1600 times on social media (e.g., Twitter).
In their analysis, the authors included five studies that conducted detailed contact investigations with sufficient data to estimate a secondary attack rate according to the symptom status of index cases. When examining the included studies, the authors appeared to have misclassified the preprint study by Chaw et al. [5], as reporting the secondary attack of asymptomatic COVID-19 patients, while this study actually provided the aggregate secondary attack for both presymptomatic and asymptomatic patients. Two months after the publication of the meta-analysis, the Chaw et al. paper was published in a peer-reviewed journal providing the secondary attack rate separately for asymptomatic patients (3 cases of 106 contacts) [6]. In addition, for studies having no events in a group, the authors added 0.5 to each cell in the 2 × 2 table. This arbitrary correction may lead to bias or even reverse the result of a meta-analysis, particularly when the sample sizes of the two groups are unbalanced. This is the case for two of the five studies included in the meta-analysis. For example, in the study by Cheng et al. [7], there were zero events in the asymptomatic group, and 22 events in the symptomatic group. However, the number of patients in the asymptomatic group was much smaller than in the symptomatic group, (91 vs. 2664); thus, adding 0.5 to arms that are unbalanced can distort the results, hence the authors’ conclusions.
Similarly, in the study by Park et al. [8], describing a COVID-19 outbreak in a call center in South Korea, there were four patients in the asymptomatic group, with none of their contacts acquiring secondary infections, and 210 symptomatic patients with 34 secondary events, translating to a secondary attack rate of 16.2%. Adding 0.5 to such unbalanced arms can generate inaccurate conclusions (i.e., 0/4 vs. 0.5/4 is 0% vs. 12.5%). In such cases, the Freeman–Tukey double-arcsine variance-stabilizing transformation can address issues arising from the computation of the proportion in the presence of zero event counts [9]. When running the analysis (using Metaprop command in STATA 16.1 [10]) after taking this into account and applying inclusion and exclusion criteria provided by the authors, we find 0% asymptomatic transmission (95% CI: 0.00–0.00) and 5.3% (1.5–11.2%) symptomatic transmission (Figure 1).
Additionally, adding results on asymptomatic transmission by Chaw et al., provided after peer review, generated similar conclusions (Figure 2).
In another meta-analysis, Freeman-Tukey double-arcsine variance-stabilizing transformation was applied to address the issue arising from zero events, showing overall 1% (95% CI 0–2%) asymptomatic transmission [13]. However, when rerunning the analysis in STATA 16.1 based on data provided in the forest plot by the authors, the overall rate is 0.4% (95% CI 0–1.4%), suggesting on average 250 asymptomatic patients are needed to generate a single infection (Figure 3).

3. Wearing Facemasks and COVID-19

Facemasks are suggested to create a barrier against a high percentage of the viral particles released from a wearer’s mouth and nose [19], and high-quality medical masks may decrease particle emission during breathing [20]. A SRMA of mainly retrospective studies with poor methodological quality on mask wearing shows that they may reduce the risk of infection particularly in healthcare settings [21], but a SRMA of randomized controlled trials (RCTs) have not supported these findings [22]. The effectiveness of mask wearing in addition to adherence, quality, fitting, and other contextual factors is also subject to new emerging viral variants with higher viral loads [23]. During the pandemic, a SRMA showed that mask wearing was associated with a 53% reduction in COVID-19 transmission (95% CI; 0.29–0.75, I2 = 84%) [24]. In this analysis, the authors included data from six original studies. This article has been cited over 60 times, picked up by more than 130 media outlets, and shared more than 9000 times on social media (e.g., Twitter).
There are several concerns regarding the analysis presented in this work. The first (major) concern was the inclusion in the meta-analysis of heterogeneous study designs, including an ecological study [25], one cross-sectional study [26], two case-control studies [27,28], one retrospective cohort study [29], and an RCT [30]. The merging of different study designs is methodologically problematic as the distinct study designs inherently carry dissimilar biases and answer research questions of different natures. For instance, while a prospective cohort study or RCT may be appropriate for a research question dealing with temporality, cross-sectional studies have very low relevance. In addition, ecological studies are generally used to generate a hypothesis [31].
Secondly, the authors appear to have missed other relevant studies, including two prospective cohort studies [32,33]. A prospective cohort study in Spain showed no significant association of risk of transmission with reported mask wearing by contacts (HR 1.55, 95%CI: 0.76–3.16) [32]. Similarly, a cohort study in Switzerland showed that constant mask wearing in public was not associated with infection risk in households (HR 0.94, 95% CI 0.43–2.09) [33]. Both these prospective studies were not included in the meta-analysis, and it is reasonable to assume that there are limitations associated with the search strategy of this SRMA, which may have led to omitting other relevant studies. Third, the authors considered the odds ratio (OR) equal to relative risk (RR) and combined these effect estimates together, which is a very common issue observed in meta-analyses. OR can be treated as RR under two circumstances according to Zhang et al. [34]: (i) if the OR value is between 0.5 and 2.5 or (ii) when there is a relatively low incidence of the observed outcome (<10%). If this is not the case, the OR should be converted to RR using specific formulas. The reported OR for the association of mask wearing and incidences of COVID-19 by Doung-Ngern et al. [28], Lio et al. [27], and Wang et al. [29] were all lower than 0.5 (0.23, 0.30, and 0.21, respectively), which could lead to biased estimates if the necessary adjustments are not performed. Instead, the overall conclusion of the paper could be that the use of a mask alone is not sufficient to provide protection against SARS-CoV-2 transmission, with higher-quality studies (e.g., prospective and clinical trials) showing in general a null effect. Similar issues have been observed also in another meta-analysis on this topic [35].

4. Ivermectin: An Example of How Critical Appraisal of Evidence Aids Good Science

Ivermectin, a horse dewormer and antiparasitic medication, has been widely promoted across the world for the treatment of COVID-19. After an in vitro experiment [36], a debate was fueled on the use of ivermectin as a therapy for COVID-19, with numerous subsequent studies challenging the initial in vitro study results.
A relatively small RCT compared an ivermectin dose of 12 mg/daily for 5 days (n = 22) or ivermectin 12 mg and doxycycline daily for 5 days (n = 23) against a placebo (n = 23) [37]. Ivermectin monotherapy resulted in reducing the time for viral clearance with a mean duration 9.7 days vs. 12.7 days for the placebo (p = 0.02), but ivermectin plus doxycycline did not show a comparable reduction (11.5 days). A preprint of a prospective controlled (nonrandomized) trial compared two to three doses of the drug in combination with 5 to 10 days of doxycycline (n = 70) with standard care (n = 70) [38]. Ivermectin/doxycycline therapy was associated with a reduced time to recovery of 10.6 days compared to 17.9 days for the placebo (p < 0.0001), but during the writing of this commentary, these results were not yet peer-reviewed or published.
A retrospective study of 280 patients hospitalized with confirmed COVID-19, reviewed the impact of ivermectin use in COVID-19 in four Florida hospitals [39]. The mortality was lower among patients who received ivermectin (n = 173, 15% mortality) compared to those receiving standard of care (n = 107, 25.2% mortality). After further adjustment, differences in mortality between the groups remained statistically significant (OR = 0.27, 95% CI: 0.09–0.80, p = 0.03). These findings, however, require randomized controlled trials for confirmation.
While meta-analyses on ivermectin and COVID-19 have shown a potential beneficial role of ivermectin, there are several serious methodological concerns observed in these meta-analyses, such as the inclusion of small and flawed RCTs [40] and positive results on ivermectin being driven mainly by methodologically low-quality studies [41]. However, critical evaluation from researchers has led to the retraction of the published meta-analyses. Nevertheless, this example has raised questions on the integrity of the evidence-generating process during COVID-19 [42]. A recent analysis has corroborated such concerns, as it showed that most healthcare interventions tested in Cochrane Reviews are not effective according to methodologically high-quality evidence [1]. Overall, these findings highlight the imperative of high-quality evidence synthesis and the importance of sharing data of individual studies, which would provide an opportunity to conduct meta-analyses of individual patient data (IPD).

5. Conclusions

Methodologically sound meta-analyses are important to not only improve our knowledge on a specific topic and understand gaps, but also to provide methodological rigor for scientific guidance for policy and decision-making. While pandemic situations require high-speed work in a context of tremendous professional pressure, it is important to preserve methodological rigor and ethical standards to ensure public health and clinical practice are informed by the best available evidence. Examples of methodological issues in meta-analyses related to COVID-19 are an indicator of common and repeated flaws in meta-analyses in general, generating misleading conclusions with serious health, economic, and social implications. While our work was not systematic, but rather focused on highly cited and influential meta-analysis in COVID-19, our paper highlights the importance of investigating systematically the extent to which meta-analyses published during the COVID-19 pandemic carry methodological issues that led to distorted conclusions. For high-quality and rigorous evidence synthesis, it is imperative for researchers to have a clear understanding of study methodological quality and designs, confounding and different types of biases, statistical models, and expertise of relevant meta-analytic methods for the specific topic under investigation [43]. What is more important, scientific inquiry should be driven by understanding the facts and seeking the truth, rather than using scientific methods to support preconceived perceptions or ‘eminence-based’ healthcare. Among others, using up-to-date methodologically sound guidelines for the conduct of meta-analysis, following PRISMA for reporting of systematic reviews [44], and applying GRADE (i.e., Grading of Recommendations, Assessment, Development, and Evaluations) [45] could help improve the quality of systematic reviews and meta-analysis and facilitate unbiased evidence-based recommendations.

Author Contributions

T.M. conceptualized the study and reviewed the literature; E.L. reviewed and confirmed the data; T.M. wrote the first draft of the manuscript; E.L. reviewed and revised the subsequent and finalized draft of the manuscript. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Not applicable.

Acknowledgments

We are grateful to our colleagues for their invaluable comments.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Howick, J.; Koletsi, D.; Ioannidis, J.P.A.; Madigan, C.; Pandis, N.; Loef, M.; Walach, H.; Sauer, S.; Kleijnen, J.; Seehra, J.; et al. Most healthcare interventions tested in Cochrane Reviews are not effective according to high quality evidence: A systematic review and meta-analysis. J. Clin. Epidemiol. 2022, 148, 160–169. [Google Scholar] [CrossRef] [PubMed]
  2. Ioannidis, J.P. The Mass Production of Redundant, Misleading, and Conflicted Systematic Reviews and Meta-analyses. Milbank Q. 2016, 94, 485–514. [Google Scholar] [CrossRef] [PubMed][Green Version]
  3. Siemieniuk, R.A.; Bartoszko, J.J.; Ge, L.; Zeraatkar, D.; Izcovich, A.; Kum, E.; Pardo-Hernandez, H.; Qasim, A.; Martinez, J.P.D.; Rochwerg, B.; et al. Drug treatments for covid-19: Living systematic review and network meta-analysis. BMJ 2020, 370, m2980. [Google Scholar] [CrossRef] [PubMed]
  4. Buitrago-Garcia, D.; Egli-Gany, D.; Counotte, M.J.; Hossmann, S.; Imeri, H.; Ipekci, A.M.; Salanti, G.; Low, N. Occurrence and transmission potential of asymptomatic and presymptomatic SARS-CoV-2 infections: A living systematic review and meta-analysis. PLoS Med. 2020, 17, e1003346. [Google Scholar] [CrossRef] [PubMed]
  5. Chaw, L.; Koh, W.C.; Jamaludin, S.A.; Naing, L.; Alikhan, M.F.; Wong, J. Analysis of SARS-CoV-2 transmission in different settings, among cases and close contacts from the Tablighi cluster in Brunei Darussalam. medRxiv 2020. [Google Scholar] [CrossRef]
  6. Chaw, L.; Koh, W.C.; Jamaludin, S.A.; Naing, L.; Alikhan, M.F.; Wong, J. Analysis of SARS-CoV-2 Transmission in Different Settings, Brunei. Emerg. Infect. Dis. 2020, 26, 2598–2606. [Google Scholar] [CrossRef]
  7. Cheng, H.Y.; Jian, S.W.; Liu, D.P.; Ng, T.C.; Huang, W.T.; Lin, H.H. Contact Tracing Assessment of COVID-19 Transmission Dynamics in Taiwan and Risk at Different Exposure Periods Before and After Symptom Onset. JAMA Intern. Med. 2020, 180, 1156–1163. [Google Scholar] [CrossRef]
  8. Park, S.Y.; Kim, Y.M.; Yi, S.; Lee, S.; Na, B.J.; Kim, C.B.; Kim, J.I.; Kim, H.S.; Kim, Y.B.; Park, Y.; et al. Coronavirus Disease Outbreak in Call Center, South Korea. Emerg. Infect. Dis 2020, 26, 1666–1670. [Google Scholar] [CrossRef]
  9. Lin, L.; Xu, C. Arcsine-based transformations for meta-analysis of proportions: Pros, cons, and alternatives. Health Sci. Rep. 2020, 3, e178. [Google Scholar] [CrossRef]
  10. Nyaga, V.N.; Arbyn, M.; Aerts, M. Metaprop: A Stata command to perform meta-analysis of binomial data. Arch. Public Health 2014, 72, 39. [Google Scholar] [CrossRef][Green Version]
  11. Luo, L.; Liu, D.; Liao, X.; Wu, X.; Jing, Q.; Zheng, J.; Liu, F.; Yang, S.; Bi, H.; Li, Z.; et al. Contact settings and risk for transmission in 3410 close contacts of patients with COVID-19 in Guangzhou, China. Ann. Intern. Med. 2020, 173, 879–887. [Google Scholar] [CrossRef]
  12. Zhang, W.; Cheng, W.; Luo, L.; Ma, Y.; Xu, C.; Qin, P.; Zhang, Z. Secondary transmission of coronavirus disease from presymptomatic persons, China. Emerg. Infect. Dis. 2020, 26, 1924. [Google Scholar] [CrossRef]
  13. Qiu, X.; Nergiz, A.I.; Maraolo, A.E.; Bogoch, I.I.; Low, N.; Cevik, M. The role of asymptomatic and pre-symptomatic infection in SARS-CoV-2 transmission-a living systematic review. Clin. Microbiol. Infect. 2021, 27, 511–519. [Google Scholar] [CrossRef]
  14. Chen, Y.; Wang, A.H.; Yi, B.; Ding, K.Q.; Wang, H.B.; Wang, J.M.; Shi, H.B.; Wang, S.J.; Xu, G.Z. Epidemiological characteristics of infection in COVID-19 close contacts in Ningbo city. Chung Hua Liu Hsing Ping Hsueh Tsa Chih. 2020, 41, 667–671. [Google Scholar]
  15. Gao, M.; Yang, L.; Chen, X.; Deng, Y.; Yang, S.; Xu, H.; Chen, Z.; Gao, X. A study on infectivity of asymptomatic SARS-CoV-2 carriers. Respir. Med. 2020, 169, 106026. [Google Scholar] [CrossRef]
  16. Mandić-Rajčević, S.; Masci, F.; Crespi, E.; Franchetti, S.; Longo, A.; Bollina, I.; Velocci, S.; Amorosi, A.; Baldelli, R.; Boselli, L.; et al. Contact tracing and isolation of asymptomatic spreaders to successfully control the COVID-19 epidemic among healthcare workers in Milan (Italy). medRxiv 2020. [Google Scholar] [CrossRef]
  17. Zeng, J.; Qiu, L.P.; Zou, Y. Epidemiological outcome of close contacts of coronavirus disease 2019 cases in Sichuan province. Chin. J. Public Health 2020, 36, 503–506. [Google Scholar] [CrossRef]
  18. Jiang, X.L.; Zhang, X.L.; Zhao, X.N.; Li, C.B.; Lei, J.; Kou, Z.Q.; Sun, W.K.; Hang, Y.; Gao, F.; Ji, S.X.; et al. Transmission potential of asymptomatic and presymptomatic SARS-CoV-2 infections: A three-family cluster study in China. J. Infect. Dis. 2020, 22, 22. [Google Scholar]
  19. Ma, Q.-X.; Shan, H.; Zhang, H.-L.; Li, G.-M.; Yang, R.-M.; Chen, J.-M. Potential utilities of mask-wearing and instant hand hygiene for fighting SARS-CoV-2. J. Med. Virol. 2020, 92, 1567–1571. [Google Scholar] [CrossRef]
  20. Asadi, S.; Cappa, C.D.; Barreda, S.; Wexler, A.S.; Bouvier, N.M.; Ristenpart, W.D. Efficacy of masks and face coverings in controlling outward aerosol particle emission from expiratory activities. Sci. Rep. 2020, 10, 15665. [Google Scholar] [CrossRef]
  21. Chu, D.K.; Akl, E.A.; Duda, S.; Solo, K.; Yaacoub, S.; Schünemann, H.J.; Chu, D.K.; Akl, E.A.; El-harakeh, A.; Bognanni, A.; et al. Physical distancing, face masks, and eye protection to prevent person-to-person transmission of SARS-CoV-2 and COVID-19: A systematic review and meta-analysis. Lancet 2020, 395, 1973–1987. [Google Scholar] [CrossRef]
  22. Nanda, A.; Hung, I.; Kwong, A.; Man, V.C.-M.; Roy, P.; Davies, L.; Douek, M. Efficacy of surgical masks or cloth masks in the prevention of viral transmission: Systematic review, meta-analysis, and proposal for future trial. J. Evid. Based Med. Healthc. 2021, 14, 97–111. [Google Scholar] [CrossRef]
  23. Hetemäki, I.; Kääriäinen, S.; Alho, P.; Mikkola, J.; Savolainen-Kopra, C.; Ikonen, N.; Nohynek, H.; Lyytikäinen, O. An outbreak caused by the SARS-CoV-2 Delta variant (B.1.617.2) in a secondary care hospital in Finland, May 2021. Eurosurveillance 2021, 26, 2100636. [Google Scholar] [CrossRef]
  24. Talic, S.; Shah, S.; Wild, H.; Gasevic, D.; Maharaj, A.; Ademi, Z.; Li, X.; Xu, W.; Mesa-Eguiagaray, I.; Rostron, J.; et al. Effectiveness of public health measures in reducing the incidence of covid-19, SARS-CoV-2 transmission, and covid-19 mortality: Systematic review and meta-analysis. BMJ 2021, 375, e068302. [Google Scholar] [CrossRef]
  25. Krishnamachari, B.; Morris, A.; Zastrow, D.; Dsida, A.; Harper, B.; Santella, A.J. The role of mask mandates, stay at home orders and school closure in curbing the COVID-19 pandemic prior to vaccination. Am. J. Infect. Control 2021, 49, 1036–1042. [Google Scholar] [CrossRef]
  26. Xu, H.; Gan, Y.; Zheng, D.; Wu, B.; Zhu, X.; Xu, C.; Liu, C.; Tao, Z.; Hu, Y.; Chen, M.; et al. Relationship Between COVID-19 Infection and Risk Perception, Knowledge, Attitude, and Four Nonpharmaceutical Interventions During the Late Period of the COVID-19 Epidemic in China: Online Cross-Sectional Survey of 8158 Adults. J. Med. Internet Res. 2020, 22, e21372. [Google Scholar] [CrossRef]
  27. Lio, C.F.; Cheong, H.H.; Lei, C.I.; Lo, I.L.; Yao, L.; Lam, C.; Leong, I.H. Effectiveness of personal protective health behaviour against COVID-19. BMC Public Health 2021, 21, 827. [Google Scholar] [CrossRef]
  28. Doung-Ngern, P.; Suphanchaimat, R.; Panjangampatthana, A.; Janekrongtham, C.; Ruampoom, D.; Daochaeng, N.; Eungkanit, N.; Pisitpayat, N.; Srisong, N.; Yasopa, O.; et al. Case-Control Study of Use of Personal Protective Measures and Risk for SARS-CoV 2 Infection, Thailand. Emerg. Infect. Dis. 2020, 26, 2607–2616. [Google Scholar] [CrossRef]
  29. Wang, Y.; Tian, H.; Zhang, L.; Zhang, M.; Guo, D.; Wu, W.; Zhang, X.; Kan, G.L.; Jia, L.; Huo, D.; et al. Reduction of secondary transmission of SARS-CoV-2 in households by face mask use, disinfection and social distancing: A cohort study in Beijing, China. BMJ Glob. Health 2020, 5, e002794. [Google Scholar] [CrossRef]
  30. Bundgaard, H.; Bundgaard, J.S.; Raaschou-Pedersen, D.E.T.; von Buchwald, C.; Todsen, T.; Norsk, J.B.; Pries-Heje, M.M.; Vissing, C.R.; Nielsen, P.B.; Winsløw, U.C.; et al. Effectiveness of Adding a Mask Recommendation to Other Public Health Measures to Prevent SARS-CoV-2 Infection in Danish Mask Wearers: A Randomized Controlled Trial. Ann. Intern. Med. 2021, 174, 335–343. [Google Scholar] [CrossRef]
  31. Saunders, C.; Abel, G. Ecological studies: Use with caution. Br. J. Gen. Pract. 2014, 64, 65–66. [Google Scholar] [CrossRef] [PubMed]
  32. Marks, M.; Millat-Martinez, P.; Ouchi, D.; Roberts, C.h.; Alemany, A.; Corbacho-Monné, M.; Ubals, M.; Tobias, A.; Tebé, C.; Ballana, E.; et al. Transmission of COVID-19 in 282 clusters in Catalonia, Spain: A cohort study. Lancet Infect. Dis. 2021, 21, 629–636. [Google Scholar] [CrossRef]
  33. Dupraz, J.; Butty, A.; Duperrex, O.; Estoppey, S.; Faivre, V.; Thabard, J.; Zuppinger, C.; Greub, G.; Pantaleo, G.; Pasquier, J.; et al. Prevalence of SARS-CoV-2 in Household Members and Other Close Contacts of COVID-19 Cases: A Serologic Study in Canton of Vaud, Switzerland. Open Forum Infect. Dis. 2021, 8, ofab149. [Google Scholar] [CrossRef] [PubMed]
  34. Zhang, J.; Yu, K.F. What’s the relative risk? A method of correcting the odds ratio in cohort studies of common outcomes. JAMA 1998, 280, 1690–1691. [Google Scholar] [CrossRef][Green Version]
  35. Gomez-Ochoa, S.A.; Muka, T. Meta-analysis on facemask use in community settings to prevent respiratory infection transmission shows no effect. Int. J. Infect. Dis 2021, 103, 257–259. [Google Scholar] [CrossRef]
  36. Caly, L.; Druce, J.D.; Catton, M.G.; Jans, D.A.; Wagstaff, K.M. The FDA-approved drug ivermectin inhibits the replication of SARS-CoV-2 in vitro. Antivir. Res. 2020, 178, 104787. [Google Scholar] [CrossRef]
  37. Ahmed, S.; Khan, W.A. A five-day course of ivermectin may reduce the duration of COVID-19 illness. Int. J. Infect. Dis. 2021, 110, 93–94. [Google Scholar] [CrossRef]
  38. Hashim, H.A.; Maulood, M.F.; Rasheed, A.M.; Fatak, D.F.; Kabah, K.K.; Abdulamir, A.S. Controlled randomized clinical trial on using Ivermectin with Doxycycline for treating COVID-19 patients in Baghdad, Iraq. medRxiv 2020. [Google Scholar] [CrossRef]
  39. Rajter, J.C.; Sherman, M.S.; Fatteh, N.; Vogel, F.; Sacks, J.; Rajter, J.J. Use of Ivermectin Is Associated With Lower Mortality in Hospitalized Patients With Coronavirus Disease 2019: The Ivermectin in COVID Nineteen Study. Chest 2021, 159, 85–92. [Google Scholar] [CrossRef]
  40. Lawrence, J.M.; Meyerowitz-Katz, G.; Heathers, J.A.J.; Brown, N.J.L.; Sheldrick, K.A. The lesson of ivermectin: Meta-analyses based on summary data alone are inherently unreliable. Nat. Med. 2021, 27, 1853–1854. [Google Scholar] [CrossRef]
  41. Hill, A.; Mirchandani, M.; Ellis, L.; Pilkington, V. Ivermectin for the prevention of COVID-19: Addressing potential bias and medical fraud. J. Antimicrob. Chemother. 2022, 77, 1413–1416. [Google Scholar] [CrossRef]
  42. O’Mathuna, D.P. Ivermectin and the Integrity of Healthcare Evidence During COVID-19. Front. Public Health 2022, 10, 788972. [Google Scholar] [CrossRef]
  43. Muka, T.; Glisic, M.; Milic, J.; Verhoog, S.; Bohlius, J.; Bramer, W.; Chowdhury, R.; Franco, O.H. A 24-step guide on how to design, conduct, and successfully publish a systematic review and meta-analysis in medical research. Eur. J. Epidemiol. 2020, 35, 49–60. [Google Scholar] [CrossRef]
  44. Page, M.J.; McKenzie, J.E.; Bossuyt, P.M.; Boutron, I.; Hoffmann, T.C.; Mulrow, C.D.; Shamseer, L.; Tetzlaff, J.M.; Akl, E.A.; Brennan, S.E.; et al. The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. BMJ 2021, 372, n71. [Google Scholar] [CrossRef]
  45. Balshem, H.; Helfand, M.; Schünemann, H.J.; Oxman, A.D.; Kunz, R.; Brozek, J.; Vist, G.E.; Falck-Ytter, Y.; Meerpohl, J.; Norris, S.; et al. GRADE guidelines: 3. Rating the quality of evidence. J. Clin. Epidemiol. 2011, 64, 401–406. [Google Scholar] [CrossRef]
Figure 1. Reanalysis of data on asymptomatic and presymptomatic transmission [7,8,11,12].
Figure 1. Reanalysis of data on asymptomatic and presymptomatic transmission [7,8,11,12].
Jcm 11 04084 g001
Figure 2. Reanalysis of data on asymptomatic and presymptomatic transmission adding Chaw et al. results [6,7,8,11,12].
Figure 2. Reanalysis of data on asymptomatic and presymptomatic transmission adding Chaw et al. results [6,7,8,11,12].
Jcm 11 04084 g002
Figure 3. Reanalysis of secondary attack rates from truly asymptomatic and presymptomatic transmission [6,7,8,11,12,14,15,16,17,18].
Figure 3. Reanalysis of secondary attack rates from truly asymptomatic and presymptomatic transmission [6,7,8,11,12,14,15,16,17,18].
Jcm 11 04084 g003
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Llanaj, E.; Muka, T. Misleading Meta-Analyses during COVID-19 Pandemic: Examples of Methodological Biases in Evidence Synthesis. J. Clin. Med. 2022, 11, 4084. https://doi.org/10.3390/jcm11144084

AMA Style

Llanaj E, Muka T. Misleading Meta-Analyses during COVID-19 Pandemic: Examples of Methodological Biases in Evidence Synthesis. Journal of Clinical Medicine. 2022; 11(14):4084. https://doi.org/10.3390/jcm11144084

Chicago/Turabian Style

Llanaj, Erand, and Taulant Muka. 2022. "Misleading Meta-Analyses during COVID-19 Pandemic: Examples of Methodological Biases in Evidence Synthesis" Journal of Clinical Medicine 11, no. 14: 4084. https://doi.org/10.3390/jcm11144084

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop