Next Article in Journal
High-Protein, Low-Glycaemic Meal Replacement Improves Physical Health-Related Quality of Life in High-Risk Persons for Metabolic Syndrome—A Subanalysis of the Randomised-Controlled ACOORH Trial
Previous Article in Journal
Probiotics Administration in Cystic Fibrosis: What Is the Evidence?
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Association of Vitamin D Supplementation with Cardiovascular Events: A Systematic Review and Meta-Analysis

1
West China Hospital, Sichuan University, Chengdu 610041, China
2
Affiliated Hospital of Chengdu University, Chengdu 610106, China
3
Sichuan University Library, Chengdu 610044, China
*
Author to whom correspondence should be addressed.
These authors contributed equally to this work.
Nutrients 2022, 14(15), 3158; https://doi.org/10.3390/nu14153158
Submission received: 7 July 2022 / Revised: 23 July 2022 / Accepted: 28 July 2022 / Published: 30 July 2022
(This article belongs to the Section Micronutrients and Human Health)

Abstract

:
Background: low vitamin D status has been associated with an increased incidence of cardiovascular events. However, whether vitamin D supplementation would reduce the incidence of cardiovascular events remains unclear. Purpose: To perform a systematic review and meta-analysis of the effect of vitamin D supplementation on the mortality and incidence of cardiovascular events. Data Sources: We searched Medline, Embase, and the Cochrane Central Register of Controlled Trials from their inception until 3 May 2022. Study Selection: Two authors searched for randomized clinical trials that reported vitamin D supplementation’s effect on cardiovascular events outcomes. Data Extraction: Two authors conducted independent data extraction. Data Synthesis: We identified 41,809 reports; after exclusions, 18 trials with a total of 70,278 participants were eligible for analysis. Vitamin D supplementation was not associated with the mortality of cardiovascular events (RR 0.96, 95% CI 0.88–1.06, I2 = 0%), the incidence of stroke (RR 1.05, 95% CI 0.92–1.20, I2 = 0%), myocardial infarction (RR 0.97, 95% CI 0.87–1.09, I2 = 0%), total cardiovascular events (RR 0.97, 95% CI 0.91–1.04, I2 = 27%), or cerebrovascular events (RR 1.01, 95% CI 0.87–1.18, I2 = 0%). Limitation: Cardiovascular events were the secondary outcome in most trials and thus, might be selectively reported. Conclusion: In this meta-analysis of randomized clinical trials, vitamin D supplementation was not associated with a lower risk of cardiovascular events than no supplementation. These findings do not support the routine use of vitamin D supplementation in general.

1. Introduction

Vitamin D insufficiency is highly prevalent among elderly persons worldwide [1]. A large number of observational studies have powerfully demonstrated that low level of vitamin D can negatively affect cardiovascular health and increase the incidence of cardiovascular events and mortality [2,3]. Thus, vitamin D supplementation may be a readily available, safe, and economical modality for preventing cardiovascular events [4]. However, evidence for causality from randomized controlled trials (RCTs) of the association between vitamin D supplementation and cardiovascular events is inconclusive or negative.
Recently, four systematic reviews and meta-analyses of RCTs evaluating vitamin D supplementation and cardiovascular outcomes were published [5,6,7,8]. Three reviews [5,6,7] showed no effect on cardiovascular mortality, stroke, and myocardial infarction. However, these meta-analyses typically included vitamin D and calcium co-administered trials. The calcium side-effects on cardiovascular events are uncommon but critically important, making those reviews challenging to interpret [9,10]. In parallel, the fourth review focuses on trials that utilized vitamin D as the monotherapy and suggested insufficient evidence to support vitamin D supplementation to reduce cardiovascular events. This review included only 11,841 participants, and its accuracy was limited. Subsequently, according to available evidence, the U.S. Preventive Services Task Force stated that vitamin D supplementation did not appear effective in preventing cardiovascular disease [11]. Since the publication of these systematic reviews, two additional large RCTs [12,13] have been published, supporting the potential for better-quality evidence from meta-analyses. The present systematic review and meta-analysis aimed to update the effect of vitamin D supplementation on preventing cardiovascular events.

2. Methods

2.1. Protocol and Guidance

We did the study following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines [14]. The predetermined protocol was registered in the PROSPERO database (CRD42019119641). The study did not require ethical approval.

2.2. Eligibility Criteria

Eligible criteria were as follows: (1) studies involving participants >18 years; (2) vitamin D supplements alone at any dose. (Trials of vitamin D plus calcium (or other treatment) vs. calcium alone were considered vitamin D alone interventions); (3) studies in which placebo controls or no treatment were given to the other group; (4) reported at least one of the following outcomes of interest: cardiovascular mortality, myocardial infarction, stroke, total cardiovascular events, and cardiovascular mortality and cerebrovascular events; and (5) randomized controlled trials (RCT).
Exclusion Criteria: (1) observational studies, review articles, case reports, poster abstracts, and editorials; (2) trials of vitamin D analogs or hydroxylated vitamin D; (3) trials where all participants took vitamin D; (4) trials in pregnant or lactating women; and (5) trials of critically ill patients.

2.3. Information Sources and Search Strategy

The retrieval strategy was developed and implemented by a medical librarian (PX). Articles published before 3 May 2022, were included. A computerized search of Medline, Embase and the Cochrane Central Register of Controlled Trials databases was conducted. Language restrictions were not applied. Details of the retrieval strategy can be obtained in Supplement Materials Table S1. We screened references of critical articles and reviews for additional potentially relevant articles. For ongoing and unpublished studies, we searched relevant clinical trial registries.

2.4. Study Selection

After deleting duplicates, all titles and abstracts were independently screened to identify potentially relevant articles by two authors (P.Y.-Y. and L.Z.-R.). Subsequently, the two authors assessed the full text of potentially relevant studies. The final list of included trials was decided on the discussion between authors. Discrepancies were resolved by consensus or arbitrated by a third author (Z.Y.).

2.5. Data Collection

Two authors (P.Y.-Y. and Z.Y.) independently performed full-text assessments and extracted the following data from full articles: study characteristics, patient characteristics, and vitamin D supplementation. Discrepancies between the two authors were resolved by discussion.

2.6. Risk of Bias Assessment

The risk of bias in each trial was independently assessed by two authors (P.Y.-Y. and L.Z.-R.) using the Cochrane Collaboration’s tool [15,16]. Discrepancies were resolved by consensus.

2.7. Quality of Evidence

Two authors (P.Y.-Y. and L.Z.-R.) independently used the Grading of Recommendation, Assessment, Development, and Evaluation (GRADE) approach to rate the evidence quality of each outcome. Depending on the strength of evidence, it was classified as high, moderate, low, or very low [17]. The GRADE guidance used the domains of study design limitations, inconsistency, indirectness, publication bias, and imprecision in results. Discrepancies were resolved by consensus.

2.8. Data Synthesis

We analyzed data from the included trials using Review Manager (RevMan, version 5.4, the Nordic Cochrane Center, the Cochrane Collaboration) and R (version 4.1.2; R Project for Statistical Computing). Meanwhile, every analysis was based on the intention-to-treat approach. The meta-analysis was conducted using random-effects models despite the level of heterogeneity. The risk ratio (RR) with 95% CI was used as the summary measure. All statistical inference tests reflected a 2-sided of p < 0.05. I2 values were calculated to evaluate variation among trials due to heterogeneity [18]. If there were more than 10 RCTs in a meta-analysis, publication bias was assessed by funnel plot techniques.

2.9. Trial Sequential Analysis

To explore whether cumulative data of included trials were sufficient to evaluate the results, we conducted a trial sequential analysis [19]. The purpose of the trial sequential analysis was to preserve the overall risk of type I error by 5% while retaining the power of 80% and achieving a 15% relative risk reduction from the intervention.

2.10. Subgroup Analysis

Subgroup analyses were implemented based on the number of patients (≥2000 and <2000), number of events (≥200 and <200), mean age (≥70 and <70), sex (female and both), baseline 25(OH)D (≥50 and <50), after administration 25(OH)D in the experimental group (≥75 and <75), published year (before 2017 and in or after 2017), type of vitamin D (vitamin D3 and vitamin D2), daily dose equivalent (≥2000 IU/d and <2000 IU/d), residential status (community and institution), the timing of treatment (daily and intermittently), and length of follow-up (≥3 years and <3 years).

2.11. Sensitivity Analyses

To assess the robustness of our statistical results, we performed a series of sensitivity analyses: (1) excluding trials that were unknown or had a high risk of bias, (2) excluding the largest trial, (3) excluding quasi-randomized or cluster-randomized trials, (4) excluding trials with a high risk of bias in each domain, (5) using random-effects models.

3. Results

3.1. Characteristics of Included Studies

The study selection process is shown in Supplementary Materials Figure S1. The systematic search identified 41,809 records initially, of which 18 trials [20,21,22,23,24,25,26,27,28,29,30,31,32,33,34] met inclusion criteria. Table 1 shows the characteristics of these trials included in the systematic review. In total, 70,278 participants were enrolled.

3.2. Quality of Evidence and Risk of Bias

GRADE summary findings for all outcomes are shown in Table 2. Risk-of-bias assessments are described in Supplementary Materials Figures S2 and S3. Of the 18 included trials, nine were at low risk of bias, eight were at unclear risk, and one was at high risk. The Funnel plots revealed no publication bias for all five outcomes (Supplementary Materials Figures S4–S8).

3.3. Cardiovascular Mortality

Of these, 9 RCTs with 63,227 participants (31,620 with vitamin D supplementation and 31,607 without vitamin D) were included in the meta-analysis for cardiovascular mortality. The supplementation of vitamin D was not associated with reductions in cardiovascular mortality compared without vitamin D (RR 0.96, 95% CI 0.88–1.06, I2 = 0%; Figure 1).

3.4. Myocardial Infarction and Stroke

Stroke was reported in 12 trials with a total of 46,093 participants. Similarly, the supplementation with vitamin D was not related to reducing stroke compared with those not supplemented (RR 1.05, 95% CI 0.92–1.20, I2 = 0%). For myocardial infarction, 14 studies reported 1118 events (552 with vitamin D supplementation as well as 566 without vitamin D). Supplementing with vitamin D did not reduce the risk of myocardial infarction compared with not supplementing vitamin D (RR 0.97, 95% CI 0.87–1.09, I2 = 0%; Figure 1).

3.5. Total Cardiovascular Events and Cerebrovascular Events

A total of 39,046 participants were enrolled in seven trials reporting cardiovascular events. A similar incidence of cardiovascular events was observed between the vitamin D and no vitamin D groups (RR 0.97, 95% CI 0.91–1.04, I2 = 27%; Figure 1. Total cerebrovascular events were similar between the two groups (RR 1.01, 95% CI 0.87–1.18, I2 = 0%; Figure 1).

3.6. Other Analyses

Prespecified subgroup analyses of all outcomes revealed no interactions between the number of patients, number of events, mean age, sex, baseline 25(OH)D, published year, type of vitamin D, daily dose equivalent of vitamin D, the timing of treatment, residential status, and length of follow-up (Table 3). The results of the sensitivity analyses were broadly consistent with the primary analysis (Supplementary Materials Table S2).
Trial sequential analyses of the effects of vitamin D supplementation on cardiovascular outcomes were shown in Supplementary Materials Figures S9–S13. In the trial sequential analysis, the pooled sample size exceeded the calculated optimum sample size for total cardiovascular events but not for cardiovascular mortality, stroke, myocardial infarction, and cerebrovascular events.

4. Discussion

In this meta-analysis of 18 RCTs with a total of 70,278, supplementation of vitamin D was not associated with reducing the risk of cardiovascular mortality, myocardial infarction, stroke, total cardiovascular events, or cerebrovascular events. No significant difference between subgroup analyses was found. Evidence of trial sequential analyses indicated that vitamin D supplementation did not decrease the relative risk of myocardial infarction and total cardiovascular events by 15%. It is unlikely that further similar trials will alter the conclusions of these trial sequential analyses. However, the evidence was not enough on the outcomes of cardiovascular mortality, stroke, myocardial infarction, and cerebrovascular events.

4.1. Comparison with Other Studies

Our findings were consistent with previous systematic reviews on this topic [5,6,8]. Low-quality evidence of a Cochrane review suggested no significant difference in cardiovascular mortality (RR 0.98, 95% CI 0.90–1.07). A meta-analysis by Bolland et al. showed negative findings with stroke (RR 1.00, 95% CI 0.88–1.13) and myocardial infarction (RR 1.04, 95% CI 0.91–1.17) [6]. The two systematic reviews included trials with mixed interventions of vitamin D supplementation combined with calcium supplementation, which is problematic because calcium has been proven to be associated with an increased risk of cardiovascular events [38,39,40]. In the meta-analysis by Ford et al., evidence of supplementation with vitamin D alone does not appear to protect against myocardial infarction (RR 0.95, 95% CI 0.82–1.10) or stroke (RR 1.08, 95% CI 0.91–1.29) [8].
In this study, we provided a different method compared with recent meta-analyses. Our study additionally included two large RCTs, which take up 34% of the participants in this study, providing more robust results of these relationships. Moreover, our study used different selection criteria: unlike these meta-analyses, we did not include more than ten trials of vitamin D co-administered with calcium, six trials [41,42,43,44,45,46,47] of vitamin D analogs, or hydroxylated vitamin D, and one trial by Sato et al. [48] which was retracted in 2017. Furthermore, we quantified a new outcome: total cardiovascular events.

4.2. Implications for Future Research

Though the available evidence did not show meaningful effects of vitamin D supplementation on cardiovascular outcomes, there have been several points for future research. First, the baseline 25(OH)D concentrations of included trials may be too high. Cardiovascular events appear when the serum 25(OH)D levels are below 37 nmol/L [4]. However, only three trials [30,32,34] involving 28,520 participants reported a mean baseline 25(OH)D level of less than 37 nmol/L. The future RCTs enrolling populations with lower baseline 25(OH)D might produce different results. Second, cardiovascular disease is a chronic disease. The follow-up time of patients included in the study is only a few years. Through the different blood sampling times after administration, we can see that the vitamin D level of blood samples will decrease with the extension of sampling time. Positive results may be obtained if the drug is continuously administered for a long time and the follow-up time is extended [28]. Third, eligible trials may be carried out in the wrong populations. Like RCTs of aspirin used to prevent cardiovascular events, the cardiovascular benefits are mainly observed in participants with a high risk of cardiovascular events, such as patients with a prior history of atherosclerotic cardiovascular disease [49]. In this study, however, most participants came from primary prevention trials. Further vitamin D trials aiming to prevent cardiovascular events should design through risk factor optimization (e.g., secondary prevention).

4.3. Strengths and Limitations

We conducted the present systematic review based on a protocol published in the PROSPERO database, which followed a rigorous methodological approach derived from the Cochrane Handbook. Additionally, we assessed the quality of evidence with GRADE and the minimum information size by using trial sequential analysis. Compared with previous reviews [5,6,8], the strengths of this study included a lower degree of heterogeneity, higher quality of evidence, and large sample size, which was adequate to assess rare outcomes, as confirmed by the trial sequential analysis. Some articles [9,10] have shown that the side effects of calcium on cardiovascular events are extremely important, Previous meta-analyses usually include trials of vitamin D and calcium combinations, so it is difficult to explain the role of vitamin D. We aim to give a clearer answer of vitamin D in the occurrence of cardiovascular disease.
We are, however, aware of several limitations. First, our study focused on published trials reporting cardiovascular events. In contrast, these results were not reported in many vitamin D supplementation studies, resulting in selective reporting bias.
Second, the relatively short duration of vitamin D supplementation and follow-up left it challenging to detect benefits in preventing cardiovascular events.
Third, the 15 included trials were heterogeneous in terms of vitamin D dose, duration of treatment, comorbid conditions, population characteristics, and most important, baseline 25(OH) D level. Outcomes definitions among trials differed depending on contemporary consensus definitions, contributing to clinical heterogeneity.

5. Conclusions

This meta-analysis of randomized clinical trials suggests that vitamin D supplementation is not associated with a decrease in cardiovascular events. There is no evidence that the routine use of these supplements is effective in preventing cardiovascular events.

Supplementary Materials

The following supporting information can be downloaded at: https://www.mdpi.com/article/10.3390/nu14153158/s1, Figure S1: Study selection process [50]; Figure S2: Risk of bias summary; Figure S3: Risk of bias graph; Figure S4: Trial sequential analysis for Cardiovascular mortality, RRR = 15%; Figure S5: Trial sequential analysis for Stroke, RRR=15%; Figure S6: Trial sequential analysis for Myocardial infarction, RRR = 15%; Figure S7: Trial sequential analysis for Cardiovascular events, RRR = 15%; Figure S8: Trial sequential analysis for Cerebrovascular events, RRR = 15%; Figure S9: Funnel plot for Cardiovascular mortality; Figure S10: Funnel plot for Stroke; Figure S11: Funnel plot for Myocardial infarction; Figure S12: Funnel plot for Cardiovascular events; Figure S13: Funnel plot for Cerebrovascular event; Table S1: Search strategy; Table S2: Sensitivity analyses.

Author Contributions

F.F. and Y.Z. conceived the study and designed the protocol. P.X. performed the literature search. Y.-Y.P., Z.-R.L. and Y.Z. selected the studies. Y.-Y.P. and Y.Z. extracted the relevant information. Y.-Y.P. and Z.-R.L. synthesized the data. Y.-Y.P. wrote the first draft of the paper. F.F., X.-C.P., Y.-Y.P., Y.Z., Z.-R.L. and P.X. critically revised successive drafts of the paper and approved the final version. F.F., X.-C.P. and Y.Z. are the guarantors of the review. All authors have read and agreed to the published version of the manuscript.

Funding

The work was supported by National Natural Science Foundation of China (82172842, and 81803104 and 81672386), the Sichuan Province Science and Technology Support Program (2021YFSY008, 2020YFS0276), West China Nursing Discipline Development Special Fund Project (HXHL21008), the Technology Innovation Project of Chengdu Science and Technology Bureau (2019-YF05-00459-SN), Postdoctoral research and Development Fund and Translational medicine fund of West China Hospital (2020HXBH119 and CGZH19002). National Key R&D Program of China (2018YFA0108604) (Lu Ma), the 1·3·5 project for disciplines of excellence-Clinical Research Incubation Project, West China Hospital, Sichuan University (21HXFH046) (Fang Fang), the innovation team project of Affiliated Hospital of Clinical Medicine College of Chengdu University (CDFYCX202203) (Yu Zhang), the project of Sichuan Science and Technology Bureau (22ZDYF0798) (Fang Fang), and Clinical Incubation Program of West China Hospital, SCU (2018HXFU008) (Lu Ma). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

The datasets used and analyzed during the current study are available from the corresponding author on reasonable request.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Holick, M.F. The vitamin D epidemic and its health consequences. J. Nutr. 2005, 135, 2739s–2748s. [Google Scholar] [CrossRef] [PubMed]
  2. Weyland, P.G.; Grant, W.B.; Howie-Esquivel, J. Does sufficient evidence exist to support a causal association between vitamin D status and cardiovascular disease risk? An assessment using Hill’s criteria for causality. Nutrients 2014, 6, 3403–3430. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  3. Lee, J.H.; O’Keefe, J.H.; Bell, D.; Hensrud, D.D.; Holick, M.F. Vitamin D deficiency an important, common, and easily treatable cardiovascular risk factor? J. Am. Coll. Cardiol. 2008, 52, 1949–1956. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  4. Wimalawansa, S.J. Vitamin D and cardiovascular diseases: Causality. J. Steroid Biochem. Mol. Biol. 2018, 175, 29–43. [Google Scholar] [CrossRef]
  5. Bjelakovic, G.; Gluud, L.L.; Nikolova, D.; Whitfield, K.; Wetterslev, J.; Simonetti, R.G.; Bjelakovic, M.; Gluud, C. Vitamin D supplementation for prevention of mortality in adults. Cochrane Database Syst. Rev. 2014, Cd007470. [Google Scholar] [CrossRef] [PubMed]
  6. Bolland, M.J.; Grey, A.; Gamble, G.D.; Reid, I.R. The effect of vitamin D supplementation on skeletal, vascular, or cancer outcomes: A trial sequential meta-analysis. Lancet Diabetes Endocrinol. 2014, 2, 307–320. [Google Scholar] [CrossRef]
  7. Barbarawi, M.; Kheiri, B.; Zayed, Y.; Barbarawi, O.; Dhillon, H.; Swaid, B.; Yelangi, A.; Sundus, S.; Bachuwa, G.; Alkotob, M.L.; et al. Vitamin D Supplementation and Cardiovascular Disease Risks in More Than 83,000 Individuals in 21 Randomized Clinical Trials: A Meta-analysis. JAMA Cardiol. 2019, 4, 765–776. [Google Scholar] [CrossRef] [PubMed]
  8. Ford, J.A.; MacLennan, G.S.; Avenell, A.; Bolland, M.; Grey, A.; Witham, M. Cardiovascular disease and vitamin D supplementation: Trial analysis, systematic review, and meta-analysis. Am. J. Clin. Nutr. 2014, 100, 746–755. [Google Scholar] [CrossRef]
  9. Anderson, J.L.; May, H.T.; Horne, B.D.; Bair, T.L.; Hall, N.L.; Carlquist, J.F.; Lappe, D.L.; Muhlestein, J.B.; Intermountain Heart Collaborative Study, G. Relation of vitamin D deficiency to cardiovascular risk factors, disease status, and incident events in a general healthcare population. Am. J. Cardiol. 2010, 106, 963–968. [Google Scholar] [CrossRef] [PubMed]
  10. Pentti, K.; Tuppurainen, M.T.; Honkanen, R.; Sandini, L.; Kröger, H.; Alhava, E.; Saarikoski, S. Use of calcium supplements and the risk of coronary heart disease in 52-62-year-old women: The Kuopio Osteoporosis Risk Factor and Prevention Study. Maturitas 2009, 63, 73–78. [Google Scholar] [CrossRef]
  11. Moyer, V.A.; U.S. Preventive Services Task Force. Vitamin, mineral, and multivitamin supplements for the primary prevention of cardiovascular disease and cancer: U.S. Preventive services Task Force recommendation statement. Ann. Intern. Med. 2014, 160, 558–564. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  12. Chatterjee, R.; Fuss, P.; Vickery, E.M.; LeBlanc, E.S.; Sheehan, P.R.; Lewis, M.R.; Dolor, R.J.; Johnson, K.C.; Kashyap, S.R.; Nelson, J.; et al. Vitamin D Supplementation for Prevention of Cancer: The D2d Cancer Outcomes (D2dCA) Ancillary Study. J. Clin. Endocrinol. Metab. 2021, 106, 2767–2778. [Google Scholar] [CrossRef] [PubMed]
  13. Neale, R.E.; Baxter, C.; Romero, B.D.; McLeod, D.S.A.; English, D.R.; Armstrong, B.K.; Ebeling, P.R.; Hartel, G.; Kimlin, M.G.; O’Connell, R.; et al. The D-Health Trial: A randomised controlled trial of the effect of vitamin D on mortality. Lancet Diabetes Endocrinol. 2022, 10, 120–128. [Google Scholar] [CrossRef]
  14. Liberati, A.; Altman, D.G.; Tetzlaff, J.; Mulrow, C.; Gotzsche, P.C.; Ioannidis, J.P.; Clarke, M.; Devereaux, P.J.; Kleijnen, J.; Moher, D. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: Explanation and elaboration. Ann. Intern. Med. 2009, 151, W65–W94. [Google Scholar] [CrossRef] [Green Version]
  15. Shinichi, A. Cochrane Handbook for Systematic Reviews of Interventions. Online Kensaku 2014, 35, 154–155. [Google Scholar]
  16. Higgins, J.P.; Altman, D.G.; Gøtzsche, P.C.; Jüni, P.; Moher, D.; Oxman, A.D.; Savovic, J.; Schulz, K.F.; Weeks, L.; Sterne, J.A. The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials. BMJ 2011, 343, d5928. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  17. Guyatt, G.H.; Oxman, A.D.; Vist, G.E.; Kunz, R.; Falck-Ytter, Y.; Alonso-Coello, P.; Schunemann, H.J.; Group, G.W. GRADE: An emerging consensus on rating quality of evidence and strength of recommendations. BMJ (Clin. Res. Ed.) 2008, 336, 924–926. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  18. Higgins, J.P.; Thompson, S.G. Quantifying heterogeneity in a meta-analysis. Stat. Med. 2002, 21, 1539–1558. [Google Scholar] [CrossRef]
  19. Brok, J.; Thorlund, K.; Gluud, C.; Wetterslev, J. Trial sequential analysis reveals insufficient information size and potentially false positive results in many meta-analyses. J. Clin. Epidemiol. 2008, 61, 763–769. [Google Scholar] [CrossRef]
  20. Inkovaara, J.; Gothoni, G.; Halttula, R.; Heikinheimo, R.; Tokola, O. Calcium, vitamin D and anabolic steroid in treatment of aged bones: Double-blind placebo-controlled long-term clinical trial. Age Ageing 1983, 12, 124–130. [Google Scholar] [CrossRef]
  21. Komulainen, M.; Kröger, H.; Tuppurainen, M.T.; Heikkinen, A.M.; Alhava, E.; Honkanen, R.; Jurvelin, J.; Saarikoski, S. Prevention of femoral and lumbar bone loss with hormone replacement therapy and vitamin D3 in early postmenopausal women: A population-based 5-year randomized trial. J. Clin. Endocrinol. Metab. 1999, 84, 546–552. [Google Scholar] [CrossRef] [PubMed]
  22. Bischoff, H.A.; Stähelin, H.B.; Dick, W.; Akos, R.; Knecht, M.; Salis, C.; Nebiker, M.; Theiler, R.; Pfeifer, M.; Begerow, B.; et al. Effects of vitamin D and calcium supplementation on falls: A randomized controlled trial. J. Bone Miner. Res. 2003, 18, 343–351. [Google Scholar] [CrossRef] [PubMed]
  23. Trivedi, D.P.; Doll, R.; Khaw, K.T. Effect of four monthly oral vitamin D3 (cholecalciferol) supplementation on fractures and mortality in men and women living in the community: Randomised double blind controlled trial. BMJ 2003, 326, 469. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  24. Grant, A.M.; Avenell, A.; Campbell, M.K.; McDonald, A.M.; MacLennan, G.S.; McPherson, G.C.; Anderson, F.H.; Cooper, C.; Francis, R.M.; Donaldson, C.; et al. Oral vitamin D3 and calcium for secondary prevention of low-trauma fractures in elderly people (Randomised Evaluation of Calcium Or vitamin D, RECORD): A randomised placebo-controlled trial. Lancet 2005, 365, 1621–1628. [Google Scholar] [CrossRef]
  25. Lappe, J.M.; Travers-Gustafson, D.; Davies, K.M.; Recker, R.R.; Heaney, R.P. Vitamin D and calcium supplementation reduces cancer risk: Results of a randomized trial. Am. J. Clin. Nutr. 2007, 85, 1586–1591. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  26. Prince, R.L.; Austin, N.; Devine, A.; Dick, I.M.; Bruce, D.; Zhu, K. Effects of ergocalciferol added to calcium on the risk of falls in elderly high-risk women. Arch. Intern. Med. 2008, 168, 103–108. [Google Scholar] [CrossRef] [Green Version]
  27. Zhu, K.; Devine, A.; Dick, I.M.; Wilson, S.G.; Prince, R.L. Effects of calcium and vitamin D supplementation on hip bone mineral density and calcium-related analytes in elderly ambulatory Australian women: A five-year randomized controlled trial. J. Clin. Endocrinol. Metab. 2008, 93, 743–749. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  28. Sanders, K.M.; Stuart, A.L.; Williamson, E.J.; Simpson, J.A.; Kotowicz, M.A.; Young, D.; Nicholson, G.C. Annual high-dose oral vitamin D and falls and fractures in older women: A randomized controlled trial. JAMA 2010, 303, 1815–1822. [Google Scholar] [CrossRef] [Green Version]
  29. Witham, M.D.; Price, R.J.; Struthers, A.D.; Donnan, P.T.; Messow, C.M.; Ford, I.; McMurdo, M.E. Cholecalciferol treatment to reduce blood pressure in older patients with isolated systolic hypertension: The VitDISH randomized controlled trial. JAMA Intern. Med. 2013, 173, 1672–1679. [Google Scholar] [CrossRef]
  30. Baron, J.A.; Barry, E.L.; Mott, L.A.; Rees, J.R.; Sandler, R.S.; Snover, D.C.; Bostick, R.M.; Ivanova, A.; Cole, B.F.; Ahnen, D.J.; et al. A Trial of Calcium and Vitamin D for the Prevention of Colorectal Adenomas. N. Engl. J. Med. 2015, 373, 1519–1530. [Google Scholar] [CrossRef] [PubMed]
  31. Jorde, R.; Sollid, S.T.; Svartberg, J.; Schirmer, H.; Joakimsen, R.M.; Njølstad, I.; Fuskevåg, O.M.; Figenschau, Y.; Hutchinson, M.Y. Vitamin D 20,000 IU per Week for Five Years Does Not Prevent Progression From Prediabetes to Diabetes. J. Clin. Endocrinol. Metab. 2016, 101, 1647–1655. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  32. Scragg, R.; Stewart, A.W.; Waayer, D.; Lawes, C.M.M.; Toop, L.; Sluyter, J.; Murphy, J.; Khaw, K.T.; Camargo, C.A., Jr. Effect of Monthly High-Dose Vitamin D Supplementation on Cardiovascular Disease in the Vitamin D Assessment Study: A Randomized Clinical Trial. JAMA Cardiol. 2017, 2, 608–616. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  33. Zittermann, A.; Ernst, J.B.; Prokop, S.; Fuchs, U.; Dreier, J.; Kuhn, J.; Knabbe, C.; Birschmann, I.; Schulz, U.; Berthold, H.K.; et al. Effect of vitamin D on all-cause mortality in heart failure (EVITA): A 3-year randomized clinical trial with 4000 IU vitamin D daily. Eur. Heart J. 2017, 38, 2279–2286. [Google Scholar] [CrossRef] [Green Version]
  34. Manson, J.E.; Cook, N.R.; Lee, I.M.; Christen, W.; Bassuk, S.S.; Mora, S.; Gibson, H.; Gordon, D.; Copeland, T.; D’Agostino, D.; et al. Vitamin D Supplements and Prevention of Cancer and Cardiovascular Disease. N. Engl. J. Med. 2018, 380, 33–44. [Google Scholar] [CrossRef] [PubMed]
  35. Brohult, J.; Jonson, B. Effects of Large Doses of Calciferol on Patients with Rheumatoid Arthritis: A Double-Blind Clinical Trial. Scand. J. Rheumatol. 1973, 2, 173–176. [Google Scholar] [CrossRef] [PubMed]
  36. Avenell, A.; MacLennan, G.S.; Jenkinson, D.J.; McPherson, G.C.; McDonald, A.M.; Pant, P.R.; Grant, A.M.; Campbell, M.K.; Anderson, F.H.; Cooper, C.; et al. Long-Term Follow-Up for Mortality and Cancer in a Randomized Placebo-Controlled Trial of Vitamin D3and/or Calcium (RECORD Trial). J. Clin. Endocrinol. Metab. 2012, 97, 614–622. [Google Scholar] [CrossRef] [Green Version]
  37. Lehouck, A.; Mathieu, C.; Carremans, C.; Baeke, F.; Verhaegen, J.; Van Eldere, J.; Decallonne, B.; Bouillon, R.; Decramer, M.; Janssens, W. High doses of vitamin D to reduce exacerbations in chronic obstructive pulmonary disease: A randomized trial. Ann. Intern. Med. 2012, 156, 105–114. [Google Scholar] [CrossRef] [Green Version]
  38. Lewis, J.R.; Zhu, K.; Prince, R.L. Adverse events from calcium supplementation: Relationship to errors in myocardial infarction self-reporting in randomized controlled trials of calcium supplementation. J. Bone Min. Res. 2012, 27, 719–722. [Google Scholar] [CrossRef]
  39. Bolland, M.J.; Avenell, A.; Baron, J.A.; Grey, A.; MacLennan, G.S.; Gamble, G.D.; Reid, I.R. Effect of calcium supplements on risk of myocardial infarction and cardiovascular events: Meta-analysis. BMJ 2010, 341, c3691. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  40. Bolland, M.J.; Grey, A.; Avenell, A.; Gamble, G.D.; Reid, I.R. Calcium supplements with or without vitamin D and risk of cardiovascular events: Reanalysis of the Women’s Health Initiative limited access dataset and meta-analysis. BMJ 2011, 342, d2040. [Google Scholar] [CrossRef] [Green Version]
  41. Ott, S.M. Calcitriol Treatment Is Not Effective in Postmenopausal Osteoporosis. Ann. Intern. Med. 1989, 110, 267–274. [Google Scholar] [CrossRef]
  42. Grady, D.; Halloran, B.; Cummings, S.; Leveille, S.; Wells, L.; Black, D.; Byl, N. 1,25-Dihydroxyvitamin D3 and muscle strength in the elderly: A randomized controlled trial. J. Clin. Endocrinol. Metab. 1991, 73, 1111–1117. [Google Scholar] [CrossRef] [PubMed]
  43. Sato, Y.; Maruoka, H.; Oizumi, K. Amelioration of hemiplegia-associated osteopenia more than 4 years after stroke by 1 alpha-hydroxyvitamin D-3 and calcium supplementation. Stroke 1997, 28, 736–739. [Google Scholar] [CrossRef] [PubMed]
  44. Sato, Y.; Kuno, H.; Kaji, M.; Saruwatari, N.; Oizumi, K. Effect of ipriflavone on bone in elderly hemiplegic stroke patients with hypovitaminosis D. Am. J. Phys. Med. Rehabil. 1999, 78, 457–463. [Google Scholar] [CrossRef] [PubMed]
  45. Sato, Y.; Manabe, S.; Kuno, H.; Oizumi, K. Amelioration of osteopenia and hypovitaminosis D by 1 alpha-hydroxyvitamin D3 in elderly patients with Parkinson’s disease. J. Neurol. Neurosurg. Psychiatry 1999, 66, 64–68. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  46. Gallagher, J.C.; Fowler, S.E.; Detter, J.R.; Sherman, S.S. Combination treatment with estrogen and calcitriol in the prevention of age-related bone loss. J. Clin. Endocrinol. Metab. 2001, 86, 3618–3628. [Google Scholar] [CrossRef] [PubMed]
  47. Dukas, L.; Bischoff, H.A.; Lindpaintner, L.S.; Schacht, E.; Birkner-Binder, D.; Damm, T.N.; Thalmann, B.; Stahelin, H.B. Alfacalcidol reduces the number of fallers in a community-dwelling elderly population with a minimum calcium intake of more than 500 mg daily. J. Am. Geriatr. Soc. 2004, 52, 230–236. [Google Scholar] [CrossRef]
  48. Sato, Y.; Iwamoto, J.; Kanoko, T.; Satoh, K. Low-dose vitamin D prevents muscular atrophy and reduces falls and hip fractures in women after stroke: A randomized controlled trial. Cerebrovasc. Dis. 2005, 20, 187–192. [Google Scholar] [CrossRef]
  49. Mora, S.; Manson, J.E. Aspirin for Primary Prevention of Atherosclerotic Cardiovascular Disease: Advances in Diagnosis and Treatment. JAMA Intern. Med. 2016, 176, 1195–1204. [Google Scholar] [CrossRef]
  50. Page, M.J.; McKenzie, J.E.; Bossuyt, P.M.; Boutron, I.; Hoffmann, T.C.; Mulrow, C.D.; Shamseer, L.; Tetzlaff, J.M.; Akl, E.A.; Brennan, S.E.; et al. The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. BMJ 2021, 372, n71. [Google Scholar] [CrossRef]
Figure 1. Forest plot comparing the effects of vitamin D on cardiovascular events. For different events, we presented in different rows in forest plots. A. cardiovascular mortality; B. stroke; C. myocardial infarction; D. cardiovascular events; E. cerebrovascular events. The black line represents the 95% confidence interval of each study. The blue box represents the weight of each study. The black diamond refers to results in pooled analysis [12,13,20,21,23,25,26,27,28,29,30,31,32,33,34,35,36,37].
Figure 1. Forest plot comparing the effects of vitamin D on cardiovascular events. For different events, we presented in different rows in forest plots. A. cardiovascular mortality; B. stroke; C. myocardial infarction; D. cardiovascular events; E. cerebrovascular events. The black line represents the 95% confidence interval of each study. The blue box represents the weight of each study. The black diamond refers to results in pooled analysis [12,13,20,21,23,25,26,27,28,29,30,31,32,33,34,35,36,37].
Nutrients 14 03158 g001aNutrients 14 03158 g001b
Table 1. Characteristics of the 18 Randomized Clinical Trials.
Table 1. Characteristics of the 18 Randomized Clinical Trials.
Author & Year Mean/Median Age (SD/Range), y FemaleParticipants (Vitamin D/no Vitamin D)Baseline 25(OH)D (nmol/L) (Vitamin D/no Vitamin D)After Administration 25(OH)D (nmol/L) (Vitamin D/no Vitamin D)InterventionControl Primary OutcomeRegionFollow-Up
Brohult
1973 [35]
52 (18 to 69)68%25/25NS/NSNS/NSvitamin D3 (100,000 IU) dailyplacebobone mineral densitySweden1-year
Inkovaara 1983 [20]79.5 (65 to 97)17%45/42NS/NSNS/NSvitamin D3 (1000 IU) dailyplaceboBiochemistryTampere, Finland1-year
Komulainen 1998 [21]52.7 (52 to 53)100%228/230NS/NSNS/NSvitamin D3 (300 IU) daily for four year and (100 IU) daily for the last yearplacebobone mineral densityKuopio, Finland5-year
Trivedi
2003 [23]
75 (65 to 85)24%1345/1341NS/NS74/53vitamin D3 (100,000 IU) four-monthlyplaceboFracturethe United Kingdom5-year
Lappe
2007 [25]
66.7 (7.3)100%446/44572/7296/71vitamin D3 (1000 IU) plus calcium (1400 to 1500 mg) dailycalcium (1400 to 1500 mg) dailybone mineral densityNebraska, the USA4-year
Prince
2008 [26]
77.2 (4.6)100%151/15145/4460/44vitamin D2 1000 IU plus calcium 1000 mg dailycalcium (1000 mg) dailyFallsWestern Australia1-year
Zhu
2008 [27]
74.8 (2.6)100%39/4070/67106/64 vitamin D2 (1000 IU) plus calcium (1200 mg) dailycalcium (1200 mg) dailybone mineral densityPerth, Australia5-year
Sanders
2010 [28]
76 (73 to 80)100%1131/112553/4552/45vitamin D3 (500,000 IU) yearlyplacebofalls and fracturesVictoria, Australia3-year
Avenell
2012 [36]
77 (6)85%2649/264338/38NS/NSvitamin D3 (800 IU) dailyplaceboFracturethe United Kingdom3.75-year
Lehouck 2012 [37]68 (9)20%91/9150/50130/54vitamin D 100,000 IU every 4 weeks placebo Exacerbations in Chronic Obstructive Pulmonary DiseaseLeuven, Belgium1-year
Witham
2013 [29]
7748%80/7945/4565/45vitamin D3 (100,000 IU) three-monthlyplacebo25OHD levelsScotland1-year
Baron
2015 [30]
58 (7)37%1130/112925/2545/25vitamin D3 (1000 IU) daily plus calcium (1200 mg) dailycalcium (1200 mg) dailyadenomas incidencethe United States3-year
Jorde
2016 [31]
62 (9)49%256/25560/6195/59vitamin D3 (20,000 IU) weeklyplaceboprogression to type 2 diabetesNorway5-year
Zittermann 2017 [33]55 (48 to 62)27%199/20131/35100/40vitamin D3 (4000 IU) dailyplaceboall-cause mortalityGermany3-year
Scragg
2017 [32]
NS42%2558/255264/63135/66vitamin D3 initial (200,000 IU) then vitamin D3 (100,000 IU) monthlyplaceboCVD and deathAuckland, New Zealand3.3-year
Manson
2018 [34]
67.1 (7.1)51%12,917/12,94430/3141/29vitamin D3 (2000 IU) dailyplacebocancer and major cardiovascular eventsthe United States6-year
Chatterjee 2021 [12]60 (9.9)44.5%1194/119170/70NS/NSvitamin D3 4000 IU dailyplacebocancer and major cardiovascular eventsthe United States2.9-year
Neale
2022 [13]
6945.9%10,661/10,649NS/NSNS/NSvitamin D3 60,000 IU monthlyplacebomortalityMultiple *5.7-year
NS, no show. * Multiple races or ethnic groups were involved.
Table 2. Summary of Findings and Strength of Evidence in Studies Comparing Vitamin D vs. Control for CVD.
Table 2. Summary of Findings and Strength of Evidence in Studies Comparing Vitamin D vs. Control for CVD.
OutcomeNo. of Patients (Studies)Risk Ratio (95% CI)I2Absolute Effect Estimates
(per 1000)
Quality
ControlVitamin DDifference
Cardiovascular mortality41538
(6)
RR 0.96
(0.83 to 1.11)
22%2423−1 (−3 to 1)High
Stroke46093
(12)
RR 1.05
(0.92 to 1.20)
0%18191 (−1 to 4)High
Myocardial infarction46184
(14)
RR 0.97
(0.87 to 1.09)
0%2524−2 (−6 to 3)High
Cardiovascular events39046
(7)
RR 0.97
(0.91 to 1.04)
27%6866−2 (−6 to 3)High
Cerebrovascular events39359
(10)
RR 1.01
(0.87 to 1.18)
0%16160 (−2 to 3)High
Table 3. Subgroup analysis of the effect of vitamin D supplementation for CVD.
Table 3. Subgroup analysis of the effect of vitamin D supplementation for CVD.
Cardiovascular MortalityStrokeMyocardial InfarctionCardiovascular EventsCerebrovascular Events
Subgroup TitleRR, 95%CIpRR, 95%CIpRR, 95%CIpRR, 95%CIpRR, 95%CIp
No. of patients
≥20000.96 [0.87, 1.05]0.241.04 [0.90, 1.19]0.480.96 [0.86, 1.08]0.650.97 [0.88, 1.06]0.160.99 [0.84, 1.16]0.93
<20001.45 [0.72, 2.93]1.25 [0.73, 2.16]1.12 [0.58, 2.19]1.16 [0.91, 1.47]1.76 [0.76, 4.10]
No. of events
≥2000.95 [0.85, 1.07]0.541.03 [0.89, 1.19]0.480.96 [0.84, 1.09]0.770.95 [0.87, 1.04]0.040.99 [0.83, 1.17]0.57
<2001.04 [0.81, 1.32]1.17 [0.83, 1.65]1.00 [0.80, 1.24]1.19 [0.97, 1.46]1.10 [0.78, 1.56]
Age
≥700.92 [0.81, 1.03]0.181.09 [0.92, 1.30]0.440.99 [0.86, 1.14]0.720.89 [0.81, 0.99]0.081.10 [0.86, 1.41]0.33
<701.06 [0.89, 1.26]0.99 [0.80, 1.24]0.94 [0.77, 1.15]1.06 [0.90, 1.25]0.94 [0.77, 1.15]
Sex
FemaleNANA1.26 [0.64, 2.48]0.481.34 [0.73, 2.44]0.291.15 [0.62, 2.12]0.641.19 [0.52, 2.72]0.68
Both0.96 [0.88, 1.06]1.04 [0.91, 1.19]0.96 [0.85, 1.07]0.99 [0.89, 1.10]1.00 [0.86, 1.17]
Baseline 25(OH)D (nmol/L)
≥500.92 [0.77, 1.11]0.561.06 [0.85, 1.32]0.570.95 [0.81, 1.11]0.960.94 [0.84, 1.06]0.361.01 [0.81, 1.26]0.73
<500.98 [0.86, 1.12]1.03 [0.87, 1.22]0.94 [0.80, 1.12]1.04 [0.86, 1.25]0.96 [0.76, 1.20]
After administration 25(OH)D (nmol/L) *
≥751.25 [0.74, 2.13]0.871.05 [0.65, 1.68]0.860.93 [0.77, 1.12]0.341.07 [0.92, 1.26]0.320.96 [0.57, 1.63]0.97
<750.99 [0.84, 1.18]1.01 [0.85, 1.20]0.96 [0.83, 1.10]0.91 [0.82, 1.01]1.01 [0.85, 1.20]
Published year
Before 20170.92 [0.82, 1.03]0.171.12 [0.94, 1.33]0.480.98 [0.85, 1.12]0.830.89 [0.81, 0.99]0.041.12 [0.89, 1.43]0.23
In or after 20171.06 [0.90, 1.26]0.95 [0.77, 1.17]0.95 [0.78, 1.16]1.03 [0.94, 1.13]0.93 [0.76, 1.14]
Type of vitamin D
Vitamin D30.96 [0.87, 1.06]0.481.05 [0.92, 1.20]0.480.97 [0.87, 1.08]0.751.00 [0.90, 1.10]0.761.01 [0.86, 1.18]0.99
Vitamin D23.00 [0.13, 70.30]1.00 [0.21, 4.88]1.35 [0.18, 10.30]0.83 [0.26, 2.67]1.01 [0.27, 3.78]
Daily dose equivalent
≥2000 IU1.06 [0.90, 1.26]0.150.96 [0.78, 1.18]0.480.93 [0.77, 1.13]0.631.03 [0.94, 1.13]0.040.93 [0.76, 1.14]0.23
<2000 IU0.92 [0.81, 1.03]1.11 [0.94, 1.32]0.99 [0.86, 1.14]0.89 [0.81, 0.99]1.12 [0.89, 1.43]
Timing
Daily0.98 [0.88, 1.10]0.561.05 [0.89, 1.23]0.480.99 [0.84, 1.17]0.711.03 [0.92, 1.15]0.431.00 [0.81, 1.25]0.95
Intermittently0.92 [0.77, 1.11]1.05 [0.84, 1.32]0.95 [0.81, 1.11]0.96 [0.83, 1.10]1.01 [0.81, 1.26]
Residential status
Community1.00 [0.87, 1.15]0.441.00 [0.86, 1.17]0.480.98 [0.87, 1.11]0.600.99 [0.90, 1.09]NA0.99 [0.85, 1.16]0.20
Institution0.93 [0.81, 1.06]1.18 [0.92, 1.51]0.90 [0.68, 1.20]NA1.80 [0.73, 4.46]
Follow-up
≥3 years0.96 [0.87, 1.06]0.491.04 [0.91, 1.20]0.480.96 [0.86, 1.08]0.890.98 [0.89, 1.09]0.340.99 [0.84, 1.16]0.16
<3 years1.31 [0.54, 3.16]1.15 [0.58, 2.29]0.89 [0.28, 2.78]1.22 [0.79, 1.87]1.90 [0.77, 4.70]
NA, No available. * The mean or median level of 25(OH)D in the experimental group after administration.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Pei, Y.-Y.; Zhang, Y.; Peng, X.-C.; Liu, Z.-R.; Xu, P.; Fang, F. Association of Vitamin D Supplementation with Cardiovascular Events: A Systematic Review and Meta-Analysis. Nutrients 2022, 14, 3158. https://doi.org/10.3390/nu14153158

AMA Style

Pei Y-Y, Zhang Y, Peng X-C, Liu Z-R, Xu P, Fang F. Association of Vitamin D Supplementation with Cardiovascular Events: A Systematic Review and Meta-Analysis. Nutrients. 2022; 14(15):3158. https://doi.org/10.3390/nu14153158

Chicago/Turabian Style

Pei, Yi-Yan, Yu Zhang, Xing-Chen Peng, Zhe-Ran Liu, Ping Xu, and Fang Fang. 2022. "Association of Vitamin D Supplementation with Cardiovascular Events: A Systematic Review and Meta-Analysis" Nutrients 14, no. 15: 3158. https://doi.org/10.3390/nu14153158

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop