Next Article in Journal
The Risk of Age-Related Macular Degeneration Is Reduced in Type 2 Diabetes Patients Who Use Metformin
Next Article in Special Issue
Erythromycin Restores Osteoblast Differentiation and Osteogenesis Suppressed by Porphyromonas gingivalis Lipopolysaccharide
Previous Article in Journal
Synthesis and Properties of Sucrose- and Lactose-Based Aromatic Ester Surfactants as Potential Drugs Permeability Enhancers
Previous Article in Special Issue
The Potential Therapeutic Role of Metformin in Diabetic and Non-Diabetic Bone Impairment
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Pharmaceuticals
Volume 16
Issue 2
10.3390/ph16020222
pharmaceuticals-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Denosumab Is Superior to Raloxifene in Lowering Risks of Mortality and Ischemic Stroke in Osteoporotic Women

by
Ting-Chun Liu
1,
Chien-Ning Hsu
2,3,
Wen-Chin Lee
1,
Shih-Wei Wang
2,
Chiang-Chi Huang
1,
Yueh-Ting Lee
1,
Chung-Ming Fu
1,
Jin-Bor Chen
1 and
Lung-Chih Li
1,4,*
1
Division of Nephrology, Department of Internal Medicine, Kaohsiung Chang Gung Memorial Hospital and Chang Gung University College of Medicine, Kaohsiung 833, Taiwan
2
Department of Pharmacy, Kaohsiung Chang Gung Memorial Hospital and Chang Gung University College of Medicine, Kaohsiung 833, Taiwan
3
School of Pharmacy, Kaohsiung Medical University, Kaohsiung 807, Taiwan
4
Institute for Translational Research in Biomedicine, Kaohsiung Chang Gung Memorial Hospital and Chang Gung University College of Medicine, Kaohsiung 833, Taiwan
*
Author to whom correspondence should be addressed.
Pharmaceuticals 2023, 16(2), 222; https://doi.org/10.3390/ph16020222
Submission received: 1 December 2022 / Revised: 19 January 2023 / Accepted: 24 January 2023 / Published: 1 February 2023
(This article belongs to the Special Issue Pharmacological Targets of Bone Homeostasis: Osteoclasts and Osteoblasts)
Browse Figures
Review Reports Versions Notes

Abstract

:
Both osteoporosis and cardiovascular disease (CVD) share similar pathways in pathophysiology and are intercorrelated with increased morbidity and mortality in elderly women. Although denosumab and raloxifene are the current guideline-based pharmacological treatments, their impacts on cardiovascular protection are yet to be examined. This study aimed to compare mortality rate and cardiovascular events between denosumab and raloxifene in osteoporotic women. Risks of CVD development and all-cause mortality were estimated using Cox proportional hazard regression. A total of 7972 (3986 in each group) women were recruited between January 2003 and December 2018. No significant difference between denosumab and raloxifene was observed in composite CVDs, myocardial infarction, or congestive heart failure. However, comparison of the propensity score matched cohorts revealed that patients with proportion of days covered (PDC) ≥60% had lower incidence of ischemic stroke in the denosumab group than that in the raloxifene group (aHR 0.68; 95% CI 0.47–0.98; p = 0.0399). In addition, all-cause mortality was lower in the denosumab group than in the raloxifene group (aHR 0.59; 95% CI 0.48–0.72; p = 0.001), except in patients aged <65 y/o in this cohort study. We concluded that denosumab is superior to raloxifene in lowering risks of all-cause mortality and certain ischemic strokes in osteoporotic women.

1. Introduction

Osteoporosis and resultant fracture have become a public health problem, leading to a heavy global economic, social, and health burden. A previous study reported the estimation that 200 million women worldwide suffer from osteoporosis [1]. In particular, global deaths and disability-adjusted life-years attributable to osteoporosis and its related fracture increased from 207,367 and 8,588,936 in 1990 to 437,884 and 16,647,466 in 2019, respectively [2]. Aging has a major impact on the arterial system and heart, leading to an increase in cardiovascular disease (CVD) including atherosclerosis, hypertension, myocardial infarction, and stroke [3]. Both osteoporosis and CVD events are associated with physical disability, higher health care costs, impaired quality of life, and increased mortality [4,5]. The relationship between osteoporosis and CVD could be explained by their common risk factors such as age, smoking, alcohol consumption, physical activity, and menopause. Recent evidence has shown that osteoporosis and CVD share similar pathophysiological pathways and that osteoporotic patients are at a higher risk of developing major CVDs, such as stroke and ischemic heart disease [6].
The receptor activator of nuclear factor-κB ligand (RANKL)/the receptor activator of nuclear factor-κB (RANK)/osteoprotegerin (OPG) is one of the key signaling pathways shared by osteoporosis and CVD. RANKL–RANK interaction on the surface of pre-osteoclasts that activates downstream signals that initiate osteoclast activation, differentiation, and function [7]. OPG acts as a decoy receptor for RANKL, preventing RANK–RANKL interactions and blocking the resulting downstream osteoclastogenic cascade, and exerts an anti-calcific effect within the vasculature [8]. Denosumab is a fully human monoclonal antibody against human RANKL that mimics OPG, thus preventing it from activating RANK, and inhibits osteoclastogenesis, thereby decreasing bone resorption and increasing bone density. According to the Fracture Reduction Evaluation of Denosumab in Osteoporosis Every 6 Months (FREEDOM) trial, denosumab reduces the risk of vertebral, nonvertebral, and hip fractures. It can increase bone mineral density (BMD) at the lumbar spine and the total hip [9]. Treatment with a recombinant fusion protein, Fc-OPG, has been shown to inhibit vascular calcification in animal studies [10]. A study conducted with human RANKL knock-in (huRANKL-KI) mice showed that treatment with denosumab reduced aortic calcium deposits in prednisolone-treated huRANKL-KI mice by up to 50% based on calcium measurement [11]. However, no related study was conducted in humans. Considering the above information, we hypothesized that denosumab may have an influence in reducing cardiovascular events by slowing the progression of vascular calcification and decreasing the risk of CVD or heart failure.
Estrogen deprivation is associated with decreased bone mass and increased osteoclast formation [12,13,14]. Raloxifene is a benzothiophene non-steroidal derivative that is used as a second-generation selective estrogen receptor modulator (SERM). It binds to estrogen receptor and produces estrogen-like effects on bone, reducing resorption and increasing BMD in postmenopausal women. Raloxifene also acts as an estrogen agonist in pre-osteoclasts, inhibiting their proliferative capacity [15]. The Multiple Outcomes of Raloxifene Evaluation (MORE) trial, the pivotal treatment trial of raloxifene, demonstrated significant reductions in the risk of vertebral fractures after three years. [16]. Although raloxifene therapy for four years did not adversely affect the risk of cardiovascular events in the overall cohort, it reduced the risk of cardiovascular events in the subset of women with increased cardiovascular risk [17].
Denosumab has been shown to exert protective effects against CVD [18]. Chen et al. [19] reported that denosumab suppress the progression of coronary arterial calcification. Raloxifene may offer cardiovascular protection by normalizing the lipid profile, reducing oxidative stress, and improving endothelial function via increased nitric oxide production [20,21,22]. A recently published meta-analysis of randomized controlled trials demonstrated favorable effects of raloxifene on the lipid profile in women [23,24]. However, no direct comparison of the mortality and cardiovascular protective effects between denosumab and raloxifene has been made. As denosumab and raloxifene are currently recommended in selected patients by several osteoporosis management guidelines [25,26,27,28], it is urgently necessary to obtain evidence of these two medications regarding their cardiovascular benefits. The present study aimed to compare the effects of denosumab and raloxifene on mortality and CVD prevention.

2. Results

2.1. Patient Characteristics

A total of 33,576 adult osteoporotic patients receiving denosumab or raloxifene were identified, of whom 30,434 met the inclusion criteria (denosumab: 16,754 patients; raloxifene: 13,680 patients) (Figure 1).
Compared with patients initiated with raloxifene, those initiated with denosumab were older (73.18 ± 9.72 vs. 69.58 ± 10.76 years old) and had more comorbidities, including history of dementia, liver diseases, diabetes, renal diseases, hypertension, hyperlipidemia, and obstructive sleep apnea (Table 1). In the propensity score matched (PSM) cohort, 7972 denosumab- and raloxifene-matched pairs were analyzed over a six-year follow-up period. The baseline characteristics were well balanced in the matched groups and are presented in Table 1. Approximately 45% of the study cohort had a history of hypertension, 27% of hyperlipidemia, and 23% of diabetes. During the follow-up period, most patients were concomitantly administered medications for calcium (29%)/VitD (21%), followed by antihypertensive agents ACEi/ARB/Aliskiren (27%), statins (20%), anti-diabetics (18%), and thrombotic prevention. The frequency of these medications was similar between the denosumab and raloxifene groups (Table 1).

2.2. CVD Incidence

During the six-year follow-up period, the incidence of all CVDs was 7.80% (n = 311) and 10.71% (n = 427), and the incidence of all-cause mortality was 4.47% (n = 178) and 8.10% (n = 323) in the denosumab and raloxifene groups, respectively. Among the major CVDs, the highest event rate was ischemic stroke (4.59% in denosumab, 7.12% in raloxifene), followed by congestive heart failure (3.24% vs. 4.04% in denosumab vs. raloxifene groups) and myocardial infarction (1.00% vs. 1.56% in denosumab vs. raloxifene groups) (Table 2).
The cumulative incidence of mortality and CVDs between the denosumab and raloxifene groups is illustrated in Figure 2. No significant difference was observed in composite CVD incidence (Figure 2a: log-rank test, p = 0.3362) and heart failure (Figure 2c: log-rank test, p = 0.4819) between the two groups. Notably, the incidence of ischemic stroke was significantly lower in the denosumab group than in the raloxifene group (Figure 2b; log-rank test, p = 0.0304).
Following adjustment for the baseline characteristics, comorbidities, and the use of concomitant medications, no significant difference was observed between denosumab and raloxifene in MI, ischemic stroke, heart failure, and all-cause CVDs. However, in the stratified analyses, we found that among patients with proportion of days covered (PDC) ≥60%, the incidence of ischemic stroke was significantly lower in the denosumab group than in the raloxifene group (adjusted hazard ratio (aHR) 0.68; 95% CI 0.47–0.98; p = 0.0399) (Figure 3).

2.3. All-Cause Mortality

Of note, all-cause mortality was lower in the denosumab group than in the raloxifene group. Kaplan–Meier analysis showed that the all-cause mortality was significantly lower in the denosumab group than in the raloxifene group (Figure 2d; log-rank test, p = 0.0022). The stratified analyses further confirmed the superiority of denosumab to raloxifene in lowering the risk of all-cause mortality (aHR 0.59; 95% CI 0.48–0.72; p = 0.001). In patients less than 65 years of age, denosumab users showed lower all-cause mortality, though the difference was not significant. Among osteoporotic women with varying degrees of renal impairment, the superiority of denosumab to raloxifene in lowering the risk of all-cause mortality was apparent (Figure 3).

3. Discussion

Multimorbidity is a growing concern, especially in elderly patients and patients with chronic diseases [29,30,31,32,33]. Organ protection therefore emerges as a key concept in the management of chronic diseases including hypertension, diabetes, and chronic kidney disease. Osteoporosis and resultant fracture have been well-recognized as a silent threat to elderly patients and postmenopausal women. Both osteoporosis and CVD are common in these patient populations. As CVD and osteoporosis share some pathophysiological similarities, we quantified the cardiovascular benefits of denosumab and raloxifene based on the concept of organ protection.
Denosumab and raloxifene are recommended for osteoporosis treatment in postmenopausal women. Dose adjustment of both medications is not required in renal function impairment. In the present study, we showed that denosumab is superior to raloxifene in reducing the risk of ischemic stroke, in particular, in patients with PDC ≥60% and with eGFR 30~59.9 mL/min/1.73 m2. In line with our findings, the results of the Raloxifene Use for The Heart (RUTH) trial revealed that raloxifene may carry increased risks of venous thromboembolism and fatal stroke [34].
Comparative event rates of MI or heart failure in the denosumab and raloxifene groups were found in our study. Raloxifene has been reported to improve the levels of serum low-density lipoprotein cholesterol, total cholesterol, and non-high-density lipoprotein cholesterol in the post-hoc analysis of the MORE trial [35]. However, in the RUTH trial, a large randomized controlled trial, raloxifene did not significantly affect the risk of coronary heart disease [34]. On the other hand, in recent years, the RANKL/RANK/OPG pathway has been discovered not only as a regulator of bone remodeling but also as a regulator of vascular calcification [36]. Despite the fact that Samelson et al. [37] reported that denosumab had no effect on the progression of aortic calcification or the incidence of CV adverse events in postmenopausal women, 24% patients did not finish complete doses of denosumab, and the follow-up period was only three years. Recently, Suzuki et al. [38] found that denosumab ameliorated aortic arch calcification in dialysis patients. The calcified area apparently diminished through 30 months of treatment and it was deemed necessary for ≥2 years for decalcification due to denosumab. Hsu et al. [18] demonstrated that denosumab use is associated with a significant reduction in cardiovascular events compared to alendronate use with better adherence of >60% medication possession ratio. Despite no significant differences in MI and heart failure between the denosumab and raloxifene groups, our study did not obtain evidence of vascular calcification in X-ray or CT, owing to data limitations. Further study is necessary to elucidate whether denosumab can improve vascular calcification.
We demonstrated the lower risk of all-cause mortality in the denosumab group compared with the raloxifene group. Effective osteoporosis treatment has been reported to reduce mortality, per se, because in addition to decreasing bone fracture, treating osteoporosis might improve the ability of an individual to cope with or recover from an acute illness, probably by maintaining physiological reserve and preventing frailty [39]. Head-to-head comparison of denosumab and raloxifene in various bone fractures is lacking. However, denosumab was proven to effectively reduce risks of vertebral, non-vertebral, and hip fracture in the FREEDOM trail [9], whereas raloxifene only reduced the risk of vertebral fracture in the MORE trial [16]. Wu et al. also demonstrated that high adherence users of denosumab, defined by receiving three or four doses, had lower risks of all-cause mortality than low adherence users of denosumab, defined by one or two doses (aHR 0.64, 95% CI 0.48–0.86) [40]. Moreover, the RANKL/RANK/OPG pathway plays numerous roles in various organs [41]. This pathway has been shown to participate in intestinal immunity [42], central nervous system inflammation [43], skin inflammation [44], and fever that occurs during infection [45]. This preclinical evidence supports the potential benefit of denosumab treatment in patient populations suffering from various infectious and inflammatory diseases. Although these potential benefits might explain the lower risks of all-cause mortality shown in our study, additional clinical investigations are required to examine the therapeutic benefit of denosumab in these settings.
Although we performed analysis using one of the largest and most representative medical research databases in Taiwan [46], there are certain limitations to the current study. First, from the database studied, we were unable to identify specific causes of death in individual patients. Second, there were certain unmeasured confounding variables, such as smoking, alcohol, and home nutrient supplement. Finally, laboratory results, such as 1,25(OH)2 vitamin D and parathyroid hormone, are not part of routine care nor health insurance imbursed tests. Despite these limitations, the present study was valuable because we demonstrated the differences in lowering risks of all-cause mortality and ischemic stroke in denosumab compared to raloxifene in osteoporotic women.

4. Materials and Methods

4.1. Data Source

This retrospective cohort study was conducted using electronic health record (EHR) data from the Chang Gung Research Database (CGRD), an electronic healthcare delivery system derived from a group of Chang Gung Memorial Hospitals (CGMH) in geographically distinct parts of Taiwan. Briefly, CGMH is the largest healthcare delivery system in Taiwan, providing approximately 10–12% of the healthcare services of the Taiwan National Health Insurance (NHI) program [46]. The Taiwan NHI program is a compulsory, single-payer health insurance program that covers over 99% of Taiwan’s population [47]. The CGRD, containing detailed individual patient-level EHR including diagnosis, prescription, and laboratory test results, has been validated in some disease populations [46,48]. We conducted this analysis using the CGRD data from January 2003 to December 2018.

4.2. Study Design and Study Cohort

By using the EHR database, we first identified patients aged 30–89 years at the time of initiation of denosumab or raloxifene between 1 January 2003 and 31 December 2017 to be included in the study. To develop the new user cohort, only patients with ≥1-year records before treatment initiation were included in the study. To assess the risk of CVD, patients were excluded with history of stroke, myocardial infarction, heart failure, and cancer. Patients receiving procedures for cardiovascular diseases, including percutaneous coronary intervention (PCI) and coronary artery bypass surgery (CABG), were also identified before treatment initiation and excluded from analyses. Because raloxifene was used in women, male patients were excluded from the analysis. Operational definitions and codes for disease conditions and procedures are listed in Table S1.

4.3. Comparison Groups

New users of denosumab were defined as patients who never taken raloxifene within one year before treatment initiation, and the earliest date of denosumab prescribed was defined as the index date. The same criteria were applied for new users of raloxifene without prior denosumab treatment. PSM was applied to balance differences in baseline demographic and clinical characteristics between the denosumab and raloxifene groups. The propensity score of initiating denosumab or raloxifene was estimated by logistic regression with above mentioned baseline characteristics, with a 1:1 ratio.

4.4. Outcomes and Follow-Up

The outcomes of interest were composite of major CVD, including myocardial infarction, congestive heart failure, and ischemic stroke. These outcomes were assessed based on hospital discharge diagnoses. Individual outcomes of composite CVD were also analyzed.

4.5. Study Covariates

Baseline variables considered in the analyses included patient demographics; comorbid conditions, such as the Charlson Comorbid Index [49]. The following study covariates were measured at baseline as potential confounders to be adjusted in the analyses: procedures of PCI and CABG; preexisting hyperlipidemia; and prior medication use: anti-thrombotic medications (anti-coagulants, anti-platelets), aspirin, glucose-lowering agents, lipid-lowering agents, antihypertensive agents, vitamin D, or calcium supplementations, as well as other osteoporosis therapy besides study drugs.

4.6. Statistical Analysis

Data were summarized as mean ± standard deviation (SD) for continuous variables and n (%) for categorical variables in the study cohort. PSM was employed to balance the differences in baseline demographic and clinical characteristics between the denosumab and raloxifene groups. New users of denosumab and raloxifene were matched at a 1:1 ratio using the greedy algorithm of the PSM method. The covariate balance between the denosumab and raloxifene groups was measured using the standardized mean difference (SDM), and an SDM of <0.1 was considered as no meaningful difference [50].
We used the Cox proportional model to estimate the aHR of the study outcomes between the new users of denosumab and raloxifene. Time to CVD endpoint was used using Kaplan–Meier analysis with log-rank tests. Cox proportional hazard regression was performed for composite incident CVD events and individual cardiovascular events.
Importantly, to assess the heterogeneous effects of denosumab (versus raloxifene) with different baseline characteristics, stratified analyses were performed in the matched cohorts by age < 65 years (vs. age ≥ 65 years) and baseline eGFR groups (≥60, 30–59.9, and <30 mL/min/1.73 m2). A two-tailed test (p value < 0.05) was considered statistically significant. All statistical analyses were performed using SAS 9.4 (SAS Institute, Cary, NC, USA).

5. Conclusions

In conclusion, denosumab is superior to raloxifene in lowering risks of all-cause mortality and certain ischemic strokes in osteoporotic women. Taken into the concept of organ protection in the pharmacological treatment of osteoporosis, denosumab could be a better choice than raloxifene.

Supplementary Materials

The following supporting information can be downloaded at: https://www.mdpi.com/article/10.3390/ph16020222/s1. Table S1. Codes for inclusion/exclusion criteria, baseline comorbid conditions and medication use.

Author Contributions

Conceptualization, L.-C.L. and T.-C.L.; methodology, C.-N.H.; validation, Y.-T.L., C.-M.F., S.-W.W. and C.-C.H.; formal analysis, C.-N.H.; investigation, L.-C.L. and T.-C.L.; writing—original draft preparation, T.-C.L.; writing—review and editing, W.-C.L., J.-B.C., S.-W.W. and C.-C.H.; supervision, L.-C.L.; project administration, L.-C.L.; funding acquisition, L.-C.L. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by the Kaohsiung Chang Gung Memorial Hospital, grant number CORPG8L0541 and CFRPG8J0031.

Institutional Review Board Statement

The study was conducted in accordance with the Declaration of Helsinki, and approved by the Institutional Review Board of Kaohsiung Chang Gung Memorial Hospital, Taiwan (protocol code 201900898B0 and date of approval was 14 June 2019).

Informed Consent Statement

Patient consent was waived due to EHR review according to the regulations of the Committee on Human Research at Kaohsiung CGMH in Taiwan.

Data Availability Statement

Data is contained within the article and supplementary material.

Acknowledgments

The authors thank Hsiao-Ching Kuo in the Department of Pharmacy, Kaohsiung Chang Gung Memorial Hospital for her assistance with programming and data analyses.

Conflicts of Interest

The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript; or in the decision to publish the results.

References

  1. Lane, N.E. Epidemiology, etiology, and diagnosis of osteoporosis. Am. J. Obstet. Gynecol. 2006, 194, S3–S11. [Google Scholar] [CrossRef] [PubMed]
  2. Shen, Y.; Huang, X.; Wu, J.; Lin, X.; Zhou, X.; Zhu, Z.; Pan, X.; Xu, J.; Qiao, J.; Zhang, T.; et al. The Global Burden of Osteoporosis, Low Bone Mass, and Its Related Fracture in 204 Countries and Territories, 1990-2019. Front. Endocrinol. 2022, 13, 882241. [Google Scholar] [CrossRef] [PubMed]
  3. Lakatta, E.G.; Levy, D. Arterial and cardiac aging: Major shareholders in cardiovascular disease enterprises: Part I: Aging arteries: A "set up" for vascular disease. Circulation 2003, 107, 139–146. [Google Scholar] [CrossRef] [PubMed]
  4. Rajsic, S.; Gothe, H.; Borba, H.H.; Sroczynski, G.; Vujicic, J.; Toell, T.; Siebert, U. Economic burden of stroke: A systematic review on post-stroke care. Eur. J. Health Econ. 2019, 20, 107–134. [Google Scholar] [CrossRef] [PubMed]
  5. Srivastava, M.; Deal, C. Osteoporosis in elderly: Prevention and treatment. Clin. Geriatr. Med. 2002, 18, 529–555. [Google Scholar] [CrossRef] [PubMed]
  6. Lello, S.; Capozzi, A.; Scambia, G. Osteoporosis and cardiovascular disease: An update. Gynecol. Endocrinol. 2015, 31, 590–594. [Google Scholar] [CrossRef] [PubMed]
  7. Kim, H.S.; Nam, S.T.; Mun, S.H.; Lee, S.K.; Kim, H.W.; Park, Y.H.; Kim, B.; Won, K.J.; Kim, H.R.; Park, Y.M.; et al. DJ-1 controls bone homeostasis through the regulation of osteoclast differentiation. Nat. Commun. 2017, 8, 1519. [Google Scholar] [CrossRef]
  8. Harper, E.; Forde, H.; Davenport, C.; Rochfort, K.D.; Smith, D.; Cummins, P.M. Vascular calcification in type-2 diabetes and cardiovascular disease: Integrative roles for OPG, RANKL and TRAIL. Vascul. Pharmacol. 2016, 82, 30–40. [Google Scholar] [CrossRef]
  9. Cummings, S.R.; San Martin, J.; McClung, M.R.; Siris, E.S.; Eastell, R.; Reid, I.R.; Delmas, P.; Zoog, H.B.; Austin, M.; Wang, A.; et al. Denosumab for prevention of fractures in postmenopausal women with osteoporosis. N. Engl. J. Med. 2009, 361, 756–765. [Google Scholar] [CrossRef]
  10. Morony, S.; Tintut, Y.; Zhang, Z.; Cattley, R.C.; Van, G.; Dwyer, D.; Stolina, M.; Kostenuik, P.J.; Demer, L.L. Osteoprotegerin inhibits vascular calcification without affecting atherosclerosis in ldlr(-/-) mice. Circulation 2008, 117, 411–420. [Google Scholar] [CrossRef] [Green Version]
  11. Helas, S.; Goettsch, C.; Schoppet, M.; Zeitz, U.; Hempel, U.; Morawietz, H.; Kostenuik, P.J.; Erben, R.G.; Hofbauer, L.C. Inhibition of receptor activator of NF-kappaB ligand by denosumab attenuates vascular calcium deposition in mice. Am. J. Pathol. 2009, 175, 473–478. [Google Scholar] [CrossRef] [PubMed]
  12. Recker, R.; Lappe, J.; Davies, K.; Heaney, R. Characterization of perimenopausal bone loss: A prospective study. J. Bone Miner. Res. 2000, 15, 1965–1973. [Google Scholar] [CrossRef] [PubMed]
  13. Akhter, M.P.; Lappe, J.M.; Davies, K.M.; Recker, R.R. Transmenopausal changes in the trabecular bone structure. Bone 2007, 41, 111–116. [Google Scholar] [CrossRef] [PubMed]
  14. D'Amelio, P.; Grimaldi, A.; Di Bella, S.; Brianza, S.Z.M.; Cristofaro, M.A.; Tamone, C.; Giribaldi, G.; Ulliers, D.; Pescarmona, G.P.; Isaia, G. Estrogen deficiency increases osteoclastogenesis up-regulating T cells activity: A key mechanism in osteoporosis. Bone 2008, 43, 92–100. [Google Scholar] [CrossRef] [PubMed]
  15. D'Amelio, P.; Isaia, G.C. The use of raloxifene in osteoporosis treatment. Expert. Opin. Pharmacother. 2013, 14, 949–956. [Google Scholar] [CrossRef] [PubMed]
  16. Ettinger, B.; Black, D.M.; Mitlak, B.H.; Knickerbocker, R.K.; Nickelsen, T.; Genant, H.K.; Christiansen, C.; Delmas, P.D.; Zanchetta, J.R.; Stakkestad, J.; et al. Reduction of vertebral fracture risk in postmenopausal women with osteoporosis treated with raloxifene: Results from a 3-year randomized clinical trial. Multiple Outcomes of Raloxifene Evaluation (MORE) Investigators. JAMA 1999, 282, 637–645. [Google Scholar] [CrossRef]
  17. Barrett-Connor, E.; Grady, D.; Sashegyi, A.; Anderson, P.W.; Cox, D.A.; Hoszowski, K.; Rautaharju, P.; Harper, K.D.; Investigators, M. Raloxifene and cardiovascular events in osteoporotic postmenopausal women: Four-year results from the MORE (Multiple Outcomes of Raloxifene Evaluation) randomized trial. JAMA 2002, 287, 847–857. [Google Scholar] [CrossRef] [PubMed]
  18. Hsu, T.W.; Hsu, C.N.; Wang, S.W.; Huang, C.C.; Li, L.C. Comparison of the Effects of Denosumab and Alendronate on Cardiovascular and Renal Outcomes in Osteoporotic Patients. J. Clin. Med. 2019, 8, 932. [Google Scholar] [CrossRef]
  19. Chen, C.L.; Chen, N.C.; Wu, F.Z.; Wu, M.T. Impact of denosumab on cardiovascular calcification in patients with secondary hyperparathyroidism undergoing dialysis: A pilot study. Osteoporos. Int. 2020, 31, 1507–1516. [Google Scholar] [CrossRef]
  20. Saitta, A.; Morabito, N.; Frisina, N.; Cucinotte, D.; Corrado, F.; D'Anna, R.; Altavilla, D.; Squadrito, G.; Minutoli, L.; Arcoraci, V.; et al. Cardiovascular effects of raloxifene hydrochloride. Cardiovasc. Drug. Rev. 2001, 19, 57–74. [Google Scholar] [CrossRef]
  21. Mijatovic, V.; van der Mooren, M.J.; Kenemans, P.; de Valk-de Roo, G.W.; Netelenbos, C. Raloxifene lowers serum lipoprotein(A) in healthy postmenopausal women: A randomized, double-blind, placebo-controlled comparison with conjugated equine estrogens. Menopause 1999, 6, 134–137. [Google Scholar] [CrossRef] [PubMed]
  22. Huo, S.; Shi, W.; Ma, H.; Yan, D.; Luo, P.; Guo, J.; Li, C.; Lin, J.; Zhang, C.; Li, S.; et al. Alleviation of Inflammation and Oxidative Stress in Pressure Overload-Induced Cardiac Remodeling and Heart Failure via IL-6/STAT3 Inhibition by Raloxifene. Oxid. Med. Cell. Longev. 2021, 2021, 6699054. [Google Scholar] [CrossRef] [PubMed]
  23. Shi, H.; Santos, H.O.; de Souza, I.G.O.; Hoilat, G.J.; Martins, C.E.C.; Varkaneh, H.K.; Alkhwildi, J.A.; Hejji, A.T.; Almuqayyid, F.; Abu-Zaid, A. The Effect of Raloxifene Treatment on Lipid Profile in Elderly Individuals: A Systematic Review and Meta-analysis of Randomized Clinical Trials. Clin. Ther. 2021, 43, 297–317. [Google Scholar] [CrossRef] [PubMed]
  24. Yang, F.; Li, N.; Gaman, M.A.; Wang, N. Raloxifene has favorable effects on the lipid profile in women explaining its beneficial effect on cardiovascular risk: A meta-analysis of randomized controlled trials. Pharmacol. Res. 2021, 166, 105512. [Google Scholar] [CrossRef] [PubMed]
  25. Shoback, D.; Rosen, C.J.; Black, D.M.; Cheung, A.M.; Murad, M.H.; Eastell, R. Pharmacological Management of Osteoporosis in Postmenopausal Women: An Endocrine Society Guideline Update. J. Clin. Endocrinol. Metab. 2020, 105, 587–594. [Google Scholar] [CrossRef]
  26. Kanis, J.A.; Cooper, C.; Rizzoli, R.; Reginster, J.Y.; Scientific Advisory Board of the European Society for Clinical and Economic Aspects of Osteoporosis (ESCEO); Committees of Scientific Advisors and National Societies of the International Osteoporosis Foundation. European guidance for the diagnosis and management of osteoporosis in postmenopausal women. Osteoporos. Int. 2019, 30, 3–44. [Google Scholar] [CrossRef] [PubMed]
  27. The North American Menopause Society. Management of Osteoporosis in Postmenopausal Women: The Position Statement of The North American Menopause Society'' Editorial, P. Management of osteoporosis in postmenopausal women: The 2021 position statement of The North American Menopause Society. Menopause 2021, 28, 973–997. [Google Scholar] [CrossRef]
  28. Arceo-Mendoza, R.M.; Camacho, P.M. Postmenopausal Osteoporosis: Latest Guidelines. Endocrinol. Metab. Clin. North Am. 2021, 50, 167–178. [Google Scholar] [CrossRef]
  29. Fraser, S.D.; Taal, M.W. Multimorbidity in people with chronic kidney disease: Implications for outcomes and treatment. Curr. Opin. Nephrol. Hypertens. 2016, 25, 465–472. [Google Scholar] [CrossRef]
  30. Bowling, C.B.; Plantinga, L.; Phillips, L.S.; McClellan, W.; Echt, K.; Chumbler, N.; McGwin, G.; Vandenberg, A.; Allman, R.M.; Johnson, T.M., 2nd. Association of Multimorbidity with Mortality and Healthcare Utilization in Chronic Kidney Disease. J. Am. Geriatr. Soc. 2017, 65, 704–711. [Google Scholar] [CrossRef]
  31. Lee, W.C.; Lee, Y.T.; Li, L.C.; Ng, H.Y.; Kuo, W.H.; Lin, P.T.; Liao, Y.C.; Chiou, T.T.; Lee, C.T. The Number of Comorbidities Predicts Renal Outcomes in Patients with Stage 3(-)5 Chronic Kidney Disease. J. Clin. Med. 2018, 7, 493. [Google Scholar] [CrossRef] [PubMed]
  32. Cicek, M.; Buckley, J.; Pearson-Stuttard, J.; Gregg, E.W. Characterizing Multimorbidity from Type 2 Diabetes: Insights from Clustering Approaches. Endocrinol. Metab. Clin. North Am. 2021, 50, 531–558. [Google Scholar] [CrossRef] [PubMed]
  33. Lin, K.M.; Lu, C.L.; Hung, K.C.; Wu, P.C.; Pan, C.F.; Wu, C.J.; Syu, R.S.; Chen, J.S.; Hsiao, P.J.; Lu, K.C. The Paradoxical Role of Uric Acid in Osteoporosis. Nutrients 2019, 11, 2111. [Google Scholar] [CrossRef] [PubMed]
  34. Barrett-Connor, E.; Mosca, L.; Collins, P.; Geiger, M.J.; Grady, D.; Kornitzer, M.; McNabb, M.A.; Wenger, N.K.; Raloxifene Use for The Heart Trial, I. Effects of raloxifene on cardiovascular events and breast cancer in postmenopausal women. N. Engl. J. Med. 2006, 355, 125–137. [Google Scholar] [CrossRef] [PubMed]
  35. Dayspring, T.; Qu, Y.; Keech, C. Effects of raloxifene on lipid and lipoprotein levels in postmenopausal osteoporotic women with and without hypertriglyceridemia. Metabolism 2006, 55, 972–979. [Google Scholar] [CrossRef]
  36. Hofbauer, L.C.; Brueck, C.C.; Shanahan, C.M.; Schoppet, M.; Dobnig, H. Vascular calcification and osteoporosis--from clinical observation towards molecular understanding. Osteoporos. Int. 2007, 18, 251–259. [Google Scholar] [CrossRef]
  37. Samelson, E.J.; Miller, P.D.; Christiansen, C.; Daizadeh, N.S.; Grazette, L.; Anthony, M.S.; Egbuna, O.; Wang, A.; Siddhanti, S.R.; Cheung, A.M.; et al. RANKL inhibition with denosumab does not influence 3-year progression of aortic calcification or incidence of adverse cardiovascular events in postmenopausal women with osteoporosis and high cardiovascular risk. J. Bone. Miner. Res. 2014, 29, 450–457. [Google Scholar] [CrossRef]
  38. Suzuki, S.; Suzuki, M.; Hanafusa, N.; Tsuchiya, K.; Nitta, K. Denosumab Recovers Aortic Arch Calcification During Long-Term Hemodialysis. Kidney. Int. Rep. 2021, 6, 605–612. [Google Scholar] [CrossRef]
  39. Eriksen, E.F.; Lyles, K.W.; Colon-Emeric, C.S.; Pieper, C.F.; Magaziner, J.S.; Adachi, J.D.; Hyldstrup, L.; Recknor, C.; Nordsletten, L.; Lavecchia, C.; et al. Antifracture efficacy and reduction of mortality in relation to timing of the first dose of zoledronic acid after hip fracture. J. Bone Miner. Res. 2009, 24, 1308–1313. [Google Scholar] [CrossRef]
  40. Wu, P.H.; Lin, M.Y.; Huang, T.H.; Lee, T.C.; Lin, S.Y.; Chen, C.H.; Kuo, M.C.; Chiu, Y.W.; Chang, J.M.; Hwang, S.J. Kidney Function Change and All-Cause Mortality in Denosumab Users with and without Chronic Kidney Disease. J. Pers. Med. 2022, 12, 185. [Google Scholar] [CrossRef]
  41. Ono, T.; Hayashi, M.; Sasaki, F.; Nakashima, T. RANKL biology: Bone metabolism, the immune system, and beyond. Inflamm. Regen. 2020, 40, 2. [Google Scholar] [CrossRef] [PubMed]
  42. Bando, J.K.; Gilfillan, S.; Song, C.; McDonald, K.G.; Huang, S.C.; Newberry, R.D.; Kobayashi, Y.; Allan, D.S.J.; Carlyle, J.R.; Cella, M.; et al. The Tumor Necrosis Factor Superfamily Member RANKL Suppresses Effector Cytokine Production in Group 3 Innate Lymphoid Cells. Immunity 2018, 48, 1208–1219.e1204. [Google Scholar] [CrossRef] [PubMed]
  43. Guerrini, M.M.; Okamoto, K.; Komatsu, N.; Sawa, S.; Danks, L.; Penninger, J.M.; Nakashima, T.; Takayanagi, H. Inhibition of the TNF Family Cytokine RANKL Prevents Autoimmune Inflammation in the Central Nervous System. Immunity 2015, 43, 1174–1185. [Google Scholar] [CrossRef] [PubMed]
  44. Loser, K.; Mehling, A.; Loeser, S.; Apelt, J.; Kuhn, A.; Grabbe, S.; Schwarz, T.; Penninger, J.M.; Beissert, S. Epidermal RANKL controls regulatory T-cell numbers via activation of dendritic cells. Nat. Med. 2006, 12, 1372–1379. [Google Scholar] [CrossRef]
  45. Hanada, R.; Leibbrandt, A.; Hanada, T.; Kitaoka, S.; Furuyashiki, T.; Fujihara, H.; Trichereau, J.; Paolino, M.; Qadri, F.; Plehm, R.; et al. Central control of fever and female body temperature by RANKL/RANK. Nature 2009, 462, 505–509. [Google Scholar] [CrossRef]
  46. Shao, S.C.; Chan, Y.Y.; Kao Yang, Y.H.; Lin, S.J.; Hung, M.J.; Chien, R.N.; Lai, C.C.; Lai, E.C. The Chang Gung Research Database-A multi-institutional electronic medical records database for real-world epidemiological studies in Taiwan. Pharmacoepidemiol. Drug Saf. 2019, 28, 593–600. [Google Scholar] [CrossRef] [PubMed]
  47. Cheng, T.M. Taiwan's new national health insurance program: Genesis and experience so far. Health Aff. 2003, 22, 61–76. [Google Scholar] [CrossRef]
  48. Huang, Y.T.; Chen, Y.J.; Chang, S.H.; Kuo, C.F.; Chen, M.H. Discharge status validation of the Chang Gung Research database in Taiwan. Biomed. J. 2021, 45, 907–913. [Google Scholar] [CrossRef]
  49. Sundararajan, V.; Henderson, T.; Perry, C.; Muggivan, A.; Quan, H.; Ghali, W.A. New ICD-10 version of the Charlson comorbidity index predicted in-hospital mortality. J. Clin. Epidemiol. 2004, 57, 1288–1294. [Google Scholar] [CrossRef]
  50. Austin, P.C. Using the standardized difference to compare the prevalence of a binary variable between two groups in observational research. Commun. Stat. Simul. Comput. 2009, 38, 1228–1234. [Google Scholar] [CrossRef]
Pharmaceuticals 16 00222 g001 550
Figure 1. Study flowchart and patient selection process. Propensity score was calculated based on baseline comorbidities: peripheral vascular diseases, dementia, pulmonary disease, connective ti-sue disorder, peptic ulcer, liver diseases, diabetes, diabetes complications, paraplegia, renal disease, severe liver diseases, metastatic cancer, hypertension, hyperlipidemia, abnormal thyroid function, obstructive sleep apnea, and prior medication (oral anticoagulants, antiplatelets, aspirin, statins, fibrates, other lipid-lowering agents, anti-diabetics, angiotensin-converting enzyme inhibitors (ACEI)/angiotensinreceptor blockers (ARBs)/Aliskiren, diuretics, bisphosphonates, Forteo, and calcitonin preparations). CABG, coronary artery bypass surgery; MI, myocardial infarction; PCI, percutaneous coronary intervention; RRT, renal replacement therapy; and SCr, serum creatinine.
Figure 1. Study flowchart and patient selection process. Propensity score was calculated based on baseline comorbidities: peripheral vascular diseases, dementia, pulmonary disease, connective ti-sue disorder, peptic ulcer, liver diseases, diabetes, diabetes complications, paraplegia, renal disease, severe liver diseases, metastatic cancer, hypertension, hyperlipidemia, abnormal thyroid function, obstructive sleep apnea, and prior medication (oral anticoagulants, antiplatelets, aspirin, statins, fibrates, other lipid-lowering agents, anti-diabetics, angiotensin-converting enzyme inhibitors (ACEI)/angiotensinreceptor blockers (ARBs)/Aliskiren, diuretics, bisphosphonates, Forteo, and calcitonin preparations). CABG, coronary artery bypass surgery; MI, myocardial infarction; PCI, percutaneous coronary intervention; RRT, renal replacement therapy; and SCr, serum creatinine.
Pharmaceuticals 16 00222 g001
Pharmaceuticals 16 00222 g002 550
Figure 2. Cumulative incidence of mortality and cardiovascular outcomes in the denosumab and raloxifene groups. (a) Composite cardiovascular (CVD) events (log-rank test, p = 0.3362); (b) ischemic stroke (log-rank test, p = 0.0304); (c) heart failure (log-rank test, p = 0.4819); and (d) all-cause mortality (log-rank test, p = 0.0022).
Figure 2. Cumulative incidence of mortality and cardiovascular outcomes in the denosumab and raloxifene groups. (a) Composite cardiovascular (CVD) events (log-rank test, p = 0.3362); (b) ischemic stroke (log-rank test, p = 0.0304); (c) heart failure (log-rank test, p = 0.4819); and (d) all-cause mortality (log-rank test, p = 0.0022).
Pharmaceuticals 16 00222 g002
Pharmaceuticals 16 00222 g003 550
Figure 3. Stratified analyses of the risk of composite CVD, ischemic stroke, heart failure, and all–cause mortality between the denosumab and raloxifene groups (denosumab versus raloxifene). CVD, cardiovascular disease; aHR, adjusted hazard ratio; PDC, proportion of days covered; and eGFR, estimated glomerular filtration rate. # Adjusted for age, sex, baseline eGFR, mean dose of drug, CVD-related comorbidities, baseline comorbidities (abnormal thyroid function, obstructive sleep apnea, and fracture), and concomitant medications (anti-thrombotic agents, lipid-lowering agents, antidiabetics, antihypertensive agents, and other osteoporosis therapy).
Figure 3. Stratified analyses of the risk of composite CVD, ischemic stroke, heart failure, and all–cause mortality between the denosumab and raloxifene groups (denosumab versus raloxifene). CVD, cardiovascular disease; aHR, adjusted hazard ratio; PDC, proportion of days covered; and eGFR, estimated glomerular filtration rate. # Adjusted for age, sex, baseline eGFR, mean dose of drug, CVD-related comorbidities, baseline comorbidities (abnormal thyroid function, obstructive sleep apnea, and fracture), and concomitant medications (anti-thrombotic agents, lipid-lowering agents, antidiabetics, antihypertensive agents, and other osteoporosis therapy).
Pharmaceuticals 16 00222 g003
Table
Table 1. Characteristics of the study cohorts before and after PSM.
Table 1. Characteristics of the study cohorts before and after PSM.
Before PSMAfter PSM
Denosumab
(n = 5703)		Raloxifene
(n = 5079)		 Denosumab
(n = 3986)		Raloxifene
(n = 3986)
Nn(%)n(%)SMDp-ValueNn(%)n(%)SMDp-Value
Age at the index date (years), mean (SD)10,78273.18±9.7269.58±10.760.35<0.0001797271.31± 9.9771.32± 10.000.000.9587
Baseline eGFR, mean (SD)10,78276.07±30.9573.01±29.440.10<0.0001797274.11± 29.5674.29± 30.160.010.7977
Baseline comorbidities
         Peripheral vascular diseases15490(1.58)64(1.26)0.030.164810954(1.35)55(1.38)0.000.9232
         Dementia464300(5.26)164(3.23)0.10<0.0001320165(4.14)155(3.89)0.010.5683
         Pulmonary disease1084618(10.84)466(9.18)0.060.0042783387(9.71)396(9.93)0.010.7348
         Connective tissue disorder527301(5.28)226(4.45)0.040.0465393196(4.92)197(4.94)0.000.9587
         Peptic ulcer22271160(20.34)1067(21.01)0.020.39251659841(21.10)818(20.52)0.010.5257
         Liver diseases1291739(12.96)552(10.87)0.060.0008942464(11.64)478(11.99)0.010.6271
         Diabetes25201422(24.93)1098(21.62)0.08<0.00011851930(23.33)921(23.11)0.010.8113
         Diabetes complications756445(7.80)311(6.12)0.070.0007542273(6.85)269(6.75)0.000.8587
         Paraplegia6428(0.49)36(0.71)0.030.14164324(0.60)19(0.48)0.020.4445
         Renal disease1007623(10.92)384(7.56)0.12<0.0001687345(8.66)342(8.58)0.000.9047
         Severe liver diseases4122(0.39)19(0.37)0.000.92173112(0.30)19(0.48)0.030.2078
         Metastatic cancer43(0.05)1(0.02)0.020.375721(0.03)1(0.03)0.001.0000
         Hypertension50582911(51.04)2147(42.27)0.18<0.000136261822(45.71)1804(45.26)0.010.6856
         Hyperlipidemia29911760(30.86)1231(24.24)0.15<0.000121541085(27.22)1069(26.82)0.010.6865
         Thyroid function abnormal288176(3.09)112(2.21)0.050.004618384(2.11)99(2.48)0.030.2620
         Obstructive sleep apnea269204(3.58)65(1.28)0.15<0.000113672(1.81)64(1.61)0.020.4890
Prior medications
         Oral anticoagulants175121(2.12)54(1.06)0.08<0.00019848(1.20)50(1.25)0.000.8389
         Anti-platelet1536858(15.04)678(13.35)0.050.01191154597(14.98)557(13.97)0.030.2029
         Aspirin1183667(11.70)516(10.16)0.050.0109886460(11.54)426(10.69)0.030.2257
         Statins22441368(23.99)876(17.25)0.17<0.00011605818(20.52)787(19.74)0.020.3866
         Fibrates225147(2.58)78(1.54)0.070.000214372(1.81)71(1.78)0.000.9328
         Anti-diabetics20331143(20.04)890(17.52)0.060.00081490750(18.82)740(18.56)0.010.7739
         ACEI/ARB/Aliskiren30461794(31.46)1252(24.65)0.15<0.000121621096(27.50)1066(26.74)0.020.4498
         Diuretics518217(3.81)301(5.93)0.10<0.0001368186(4.67)182(4.57)0.000.8309
         Alendronate1431770(13.50)661(13.01)0.010.45661045511(12.82)534(13.40)0.020.4453
         Forteo (Teriparatide)279160(2.81)119(2.34)0.030.1310226116(2.91)110(2.76)0.010.6856
         Calcium31921363(23.90)1829(36.01)0.27<0.000123431157(29.03)1186(29.75)0.020.4759
         Vit. D2264945(16.57)1319(25.97)0.23<0.00011698837(21.00)861(21.60)0.010.5115
Abbreviations: ACEI, angiotensin converting enzyme inhibitor; ARB: angiotensin receptor blockers; PSM, propensity score matching; SMD, standardized mean difference. SMD < 0.1 was considered as no sign of imbalance.
Table
Table 2. The major adverse cardiovascular outcomes in the denosumab and raloxifene groups.
Table 2. The major adverse cardiovascular outcomes in the denosumab and raloxifene groups.
NDenosumab
(n = 3986)		Raloxifene
(n = 3986)		p-Value
n(%)n(%)
Primary outcomes
    Any CVD738311(7.80)427(10.71)<0.0001
         Myocardial infarction10240(1.00)62(1.56)0.0284
         Ischemic stroke467183(4.59)284(7.12)<0.0001
         Congestive heart failure290129(3.24)161(4.04)0.0556
In-hospital death from any cause501178(4.47)323(8.10)<0.0001
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Liu, T.-C.; Hsu, C.-N.; Lee, W.-C.; Wang, S.-W.; Huang, C.-C.; Lee, Y.-T.; Fu, C.-M.; Chen, J.-B.; Li, L.-C. Denosumab Is Superior to Raloxifene in Lowering Risks of Mortality and Ischemic Stroke in Osteoporotic Women. Pharmaceuticals 2023, 16, 222. https://doi.org/10.3390/ph16020222

AMA Style

Liu T-C, Hsu C-N, Lee W-C, Wang S-W, Huang C-C, Lee Y-T, Fu C-M, Chen J-B, Li L-C. Denosumab Is Superior to Raloxifene in Lowering Risks of Mortality and Ischemic Stroke in Osteoporotic Women. Pharmaceuticals. 2023; 16(2):222. https://doi.org/10.3390/ph16020222

Chicago/Turabian Style

Liu, Ting-Chun, Chien-Ning Hsu, Wen-Chin Lee, Shih-Wei Wang, Chiang-Chi Huang, Yueh-Ting Lee, Chung-Ming Fu, Jin-Bor Chen, and Lung-Chih Li. 2023. "Denosumab Is Superior to Raloxifene in Lowering Risks of Mortality and Ischemic Stroke in Osteoporotic Women" Pharmaceuticals 16, no. 2: 222. https://doi.org/10.3390/ph16020222

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop