Next Article in Journal
Differences in Physical Activity Patterns among Korean Adolescents during and after COVID-19
Previous Article in Journal
Advanced Techniques for Bone Restoration and Immediate Loading after Implant Failure: A Case Report
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Review

Self-Reported Medication Adherence Measured with Morisky Scales in Rare Disease Patients: A Systematic Review and Meta-Analysis

by
Ana María García-Muñoz
,
Desirée Victoria-Montesinos
,
Begoña Cerdá
,
Pura Ballester
*,
Eloisa María de Velasco
and
Pilar Zafrilla
Faculty of Pharmacy and Nutrition, Universidad Católica San Antonio de Murcia (UCAM), Campus de los Jerónimos, 30107 Murcia, Spain
*
Author to whom correspondence should be addressed.
These authors contributed equally to this work.
Healthcare 2023, 11(11), 1609; https://doi.org/10.3390/healthcare11111609
Submission received: 21 March 2023 / Revised: 27 May 2023 / Accepted: 29 May 2023 / Published: 31 May 2023

Abstract

:
Background: The visibility of Rare Diseases is a new challenge for society. These diseases are numerous, heterogeneous in nature and distribution, characterized by a high mortality rate but low prevalence, and usually presenting a severe evolution. Adherence to medication studies in rare diseases are uncommon, due to treatment scarcity. Objectives: The main purpose of this study is to do a meta-analysis, evaluating the level of adherence to medication in the most prevalent rare diseases. Methods: This work is a systematic review, and meta-analysis was registered in the International Prospective Register of Systematic Reviews (PROSPERO) (Registration number: CRD42022372843) and conducted according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement. Adherence to treatment in this systematic review and meta-analysis was collected from all studies included, based on the crude numerators and denominators reported, using either the Morisky Medication Adherence Scale 4 or -8. Results: A total of 54 records were identified through database searches, or after screening relevant manuscripts’ references. Finally, 18 studies were included in this systematic review and meta-analysis. A total of 1559 participants (54.18% women) aged less than 84 years old were included. Twelve studies used the MMAS-8. In 8 of them, they established the level of adherence to treatment in three categories (low, medium, and high), with the mean prevalence in each of them being 41.4%, 30.4%, and 28.2%, respectively. Conclusions: The results observed in adherence to treatment in patients with rare diseases show great variability, due to the different aspects involved in the greater or lesser applicability of the medication.

1. Introduction

In recent years, a new challenge has arisen for society: the visibility of Rare Diseases (RDs) at a multidisciplinary level, integrating health, education, and social awareness [1]. Humanizing RD is one of the Sustainable Development Goals (SDGs) included by the United Nations in the 2030 Agenda [2].
These diseases are numerous, and heterogeneous in nature and distribution, with a high mortality rate [3]. Usually, they involve a severe evolution of the condition, with multiple motor, sensory, and cognitive impairments, often presenting a high level of clinical complexity, making their recognition, diagnosis, and treatment difficult [4].
The actual number of people living with RDs is difficult to be determined. In several parts of the world, the rarity criterion is determined by regulations intended to encourage industry investment in RDs’ drug discovery, or marked by the notion of patients’ prevalence in a region [5]. The accepted prevalence of RD in the United States is 1 in 1500 people; in Australia it is stipulated at 1 in 10,000 [6]; and 1 in 2500 citizens in Japan [7]. In Spain, we adhered to the criterion established by the European Union, considering a disease as rare when it affects no more than 1 person in 2000 [3,4,8].
Epidemiological studies are difficult, as most research addressing incidence comes from national, often local, registries on specific diseases or groups of diseases [3]. Around 6000 and 8000 RDs affect 30 million people in the European Union [9]. The ISPOR Rare Disease Special Interest Group published a global systematic review, stating that the average prevalence of a RD was between 40 and 50 cases/100,000 people, and, despite all the variations, a coordinated effort is needed to standardize objective criteria and avoid qualitative descriptors [7]. However, accuracy of RD prevalence is crucial. Knowing that, we could better determine patients’ health care system unmet needs, improve condition management, and estimate the number of individuals benefiting from novel drug development, existing therapies for RDs, or ongoing clinical trials.
Adherence, understood as patient compliance of medical recommendations [10], is a problematic phenomenon well-studied in highly prevalent chronic pathologies, such as diabetes and hypertension, or in moderately prevalent diseases, such as HIV [11]. Numerous factors influence adherence to drug therapy, some related to the patient and clinicians, and others to the medications (frequency of administration, length of treatment, or tolerability). Optimal communication between all actors (prescribers, pharmacists, and patients) influences treatment adherence [12].
Lack of adherence is a serious public health problem affecting healthcare systems worldwide, especially when scarce medications are available, as is the RD scenario [13]. In a recently published manuscript of RDs, a cross-sectional investigation conducted in 139 Wilson disease patients at the National Reference Center for Wilson’s Disease (CRMR) revealed that, as in many chronic diseases, patients were weakly adherent [14]. However, a Polish piece of research in the same condition described that 74.1% of symptomatic patients were adherent to the prescribed medication [15]. In adopting a patient-centered care approach, pharmacists could play an important role identifying and resolving medication-related problems and contributing to improve treatment adherence, impacting on healthcare system sustainability [16].
To the best of our knowledge, adherence studies in patients with RDs are unusual [15], as most of them lack approved and effective therapies. The main purpose of this study is to analyze, with the methodology of meta-analysis, the published works about adherence to medication of the most prevalent rare diseases.

2. Materials and Methods

This systematic review and meta-analysis was registered in the International Prospective Register of Systematic Reviews (PROSPERO, Registration number: CRD42022372843) and conducted according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement [17].

2.1. Eligibility Criteria

The following inclusion criteria were established: (a) Participants: the participants had a rare disease diagnosis; (b) Outcome: treatment adherence measured by Morisky Medication Adherence Scale (MMAS, 4- or 8- items); and (c) Study design: no restriction, with the exception of systematic reviews, and/or meta-analyses, qualitative and case studies. Studies were limited to those published in English or Spanish. The exclusion criteria included studies: (a) With participants without a RD; (b) With adherence measured with a scale different to the MMAS; (c) Based on data from the same survey/study; (d) That were not randomized controlled trials, cross-sectional, or longitudinal studies, specifically excluding case-control studies, cohort studies, case reports, case series, qualitative studies, systematic reviews, meta-analyses, experimental animal studies, in vitro studies, and expert opinions or consensus statements; (e) In a language other than English or Spanish.

2.2. Information Sources and Search Strategy

Two researchers (AMG-M and DV-M) systematically searched the PubMed, Scopus, Web of Science, and Cochrane Database of Systematic Reviews databases, with a date limit from January 2005 to November 2022. Studies were identified via the following search terms: (a) “Morisky Medication Adherence Scale”, “MMAS-4”, “MMAS-8”, “Morisky Green Levine”, “Morisky Green Levine Medication Adherence Scale”, “Medication Adherence Questionnaire”; (b) “Rare disease”, “Cystic Fibrosis”, “Hemophilia A”, “Hemophilia B”, “Idiopathic Pulmonary Fibrosis”, “Myasthenia Gravis”, “Sickle Cell Disease”, “Primary biliary cholangitis”, “Fabry disease”, “Pulmonary arterial hypertension”, “Wilson’s disease”, “Narcolepsy”. The search terms were adapted for each database, in combination with database-specific filters (provided in Supplementary Table S1). In the search, we used those rare diseases with the highest prevalence and in which the pharmaceutical industry invests the most money [18,19,20].

2.3. Selection Process

After identifying eligible studies, Mendeley (Version for Windows 10; Elsevier, Amsterdam, Netherlands) was used to remove the duplicates. Two members of the research team (A.M.G.-M. and D.V.-M.) conducted the selection process independently, and screened all titles and abstracts to identify potentially relevant articles for further review in the full-text phase. A third researcher (E.M.-G.) participated in resolving discrepancies.

2.4. Data Items

Study details, such as sample size, country, study design, and type of medication used, were extracted. The proportion of participants with adherence to treatment was extracted by one researcher (D.V.-M.); meanwhile, another researcher (A.M.G.-M.) checked the data for accuracy. In case of a discrepancy between these two researchers, a third researcher (E.M.-G.) reviewed the information.

2.5. Risk of Bias Assessment

Two researchers (D.V.-M. and A.M.-G.) independently assessed the risk of study bias of the included studies. The assessment of the risk of bias was carried out using a specific tool for prevalence and proportion studies [21]. This comprehensive tool evaluates a wide range of potential biases, examining 10 items that cover various aspects of internal and external validity. These include, but are not limited to, sample representativeness, sample size, non-respondents, data collection method, case definition, measurement tool validity and reliability, and statistical analysis, thus offering a broad evaluation of the potential biases in the studies we reviewed. Each item is classified with the answer “yes” (low risk) or “no” (high risk), with a score of 0 and 1 point, respectively. Depending on the score, the study will be classified as “low risk of bias” (scores of 0–3), “moderate risk of bias” (scores of 4–6), or “high risk of bias” (scores of 7–9).

2.6. Outcome Measures

Adherence to treatment in this systematic review and meta-analysis was collected from all studies included, based on the crude numerators and denominators reported using either MMAS-4 [22] or -8 [23]. In those studies that used the MMAS-4 scale, participants were divided as “with adherence to treatment” and “non-adherence to treatment”. On the other hand, those that used the MMAS-8 scale classified the sample into “high adherence” (8), “medium adherence” (6–8), and “low adherence” (<6). For comparison purposes, those subjects with “high adherence” and “medium adherence” in MMAS-8 were reclassified as “with adherence to treatment”.

2.7. Synthesis Methods

Using Stata (Version 16.1; StataCorp., College Station, TX, USA) and the metaprop package [24], the proportion of multiple studies was pooled by applying a random-effects, using the DerSimonian and Laird method and a general linear mixed model (GLMM) [25]. The exact, or Clopper-Pearson, method was used to establish 95% confidence intervals (95% CIs) for the proportions from the selected individual studies [26], and a Freeman-Tukey transformation was used to normalize the results, before calculating the pooled proportion [27].The estimated effect was also performed with different transformation methods, such as the arcsine and logit transformations [27,28]. Intragroup heterogeneity of pooled proportions was also calculated using the I2 statistic and its p-value. Small-study effects and publication bias were examined using Egger’s test and funnel plots.
Sub-group analyses were conducted by type of disease, age group, study site, and Risk of Bias score. For the subgroup according to age, the studies were divided into two groups according to the mean age: “children, adolescents and young adults” with an age between 0 and 24 years, and another group formed by those subjects with a mean age above 24 years old, called “adults and seniors”. In addition, random-effects meta-regression analyses, using the method of moments, were estimated to independently assess whether treatment adherence differed by mean age, year of publication, or quality score of the studies.

3. Results

3.1. Study Selection

A total of 54 records were identified through database searches and in other articles’ bibliographies (Figure 1). After screening for duplicates, 27 records remained. Finally, 25 studies were obtained for full-text review. Of those studies, 7 were excluded to avoid redundancy, as were extracting data from the same study [29,30,31] or not showing adherence to treatment data [32,33,34,35]. Finally, 18 studies [14,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52] were included in this systematic review and meta-analysis.

3.2. Study Characteristics

Table 1 summarized the main characteristics of the 18 included studies. A total of 1559 participants (54.18% women) aged 0−83 years were included in this systematic review and meta-analysis.
Based on the type of measurement used, there were 12 studies that used the MMAS-8 [14,36,37,38,41,43,45,46,47,48,49,50]. In 8 of them [14,36,38,43,46,47,48,50], they established the level of adherence to treatment in three ranges (low, medium, and high), with the mean prevalence in each of them being 41.4%, 30.4%, and 28.2%, respectively. On the other hand, 6 studies used the MMAS-4 [39,40,42,44,51,52]. The mean adherence reported for this type of scale was 52.9%. According to sex, 15 studies reported the overall proportion of adherence in both men and women [14,37,38,39,40,41,43,44,45,46,47,48,49,51,52], and 3 studies only included one sex [36,42,50] (i.e., only women).
Regarding the type of rare disease, there were 6 studies in which the participants had sickle cell disease (SCD) [38,39,40,41,45,49], also including myelodysplastic syndromes and β-thalassemia in one of them [39]. Four studies included subjects with myasthenia gravis (MG) [37,46,51,52], and two studies included subjects with congenital hypogonadotropic hypogonadism [36,50]. The rest of the subjects in the different studies had different RDs, such as congenital Wilson’s disease [14], Fabry disease [44], cystic fibrosis [43], hemophilia A [42], pulmonary arterial hypertension [48], and amyotrophic lateral sclerosis [47].
In terms of geographical regions, 9 different countries were identified, including 4 regions in Europe [14,39,42,43,47,48], 2 in South America [37,52], 1 in Asia [44,51], 1 in North America [38,40,45,46], and 1 in Africa [41,49]. Two studies did not specify the country, since it was performed via online, on Facebook and other social media [36,50].

3.3. Risk of Study Bias

All the studies showed a low risk of bias, presenting scores between zero and three points. Two studies showed a total of three points [38,40]. The main sources of bias were related to national sample representation [37,38,40,41,43,44,45,47,48,49,50,51,52]. A summary of the risk of bias is presented in the Supplementary Table S2.

3.4. Results of Syntheses

Treatment Adherence

Figure 2 shows that the overall proportion of treatment adherence was 57.14% (95% CI: 44.09% to 69.73%; p < 0.001, I2 = 95.54%; GLMM: 54.95% (95% CI: 32.30% to 77.60%; p < 0.001). The results obtained with the logit transformation and the arcsine transformation were similar (logit transformation: 53.4% (95% CI: 43.4% to 63.5%; p < 0.001, I2 = 90.37%); arcsine transformation: 55.7% (95% CI: 44.0% to 67.4%; p < 0.001, I2 = 88.90%). The Egger’s test showed no significant differences for any of the variables analyzed in the meta-analysis (p = 0.19), indicating an absence of publication bias. However, visual assessment using the funnel plot suggests that publication bias exists, although the Egger’s test result is not statistically significant (provided in Supplementary Figure S1).
A subgroup analysis was performed for those diseases in which two or more studies were found that measured adherence using MMAS-4 or MMAS-8. In this analysis, considerable variation in adherence was observed, depending on the type of disease, with the highest adherence in myasthenia gravis (63.19%, 95% CI: 39.16 to 84.28), followed by Sickle Cell Disease (56.91%, 95% CI: 30.61 to 81.38), and congenital hypogonadotropic hypogonadism (41.01%, CI: 33.35 to 48.89).
Figure 3 shows the subgroup analysis in relation to age group. The overall proportion was slightly higher in the seniors’ group (60.90%, 95% CI: 46.39% to 74.52%; p < 0.001) than in the youth group (49.15%, 95% CI: 23.09% to 75.45%; p < 0.001).
In the subgroup analysis according to the study site, it was observed that the studies conducted in Europe showed a greater overall proportion (68.38%, 95% CI: 49.34% to 84.74%; p < 0.001), whereas, in studies conducted online, the overall proportion was lower (41.01%, 95% CI: 33.35% to 48.89%).
According to the risk of bias, it was observed that those studies with a lower score on this scale had higher overall proportions: Total score equal to 0: 61.65%, 95% CI: 41.98% to 79.54%, and Total score equal to 1: 70.86%, 95% CI: 46.52% to 90.22%.
Table 2 shows the random-effects meta-regression models of mean age, year of publication, and quality score of the studies, with respect to overall treatment adherence. Treatment adherence was not associated with these parameters (p > 0.05 in all variables). The random-effects meta-regression models of mean age, with regards to the overall treatment adherence, are shown in Figure 4.

4. Discussion

To the best of our knowledge, this is the first meta-analysis that has comprehensively examined the overall proportion of treatment adherence in different RDs. The main findings of this study are as follows: (a) a total of 57.14% of 1559 participants from 9 countries have adequate adherence to the treatment prescribed for their specific disease; (b) no significant differences were found in medication adherence considering the age group; (c) adherence was not associated with mean age. There was a large variation in adherence to treatment, depending on the type of disease and medication used.
In all the RD included in this meta-analysis, different types of treatment have been studied for years to mitigate or reduce side effects, even though they are not entirely effective [53]. Adherence to drug treatment has been found to be essential in all diseases, including RDs. However, the difficulties presented by this type of patient, both physical and psychological, may impact adherence rate being not as high as it should be. In patients with RDs, poor adherence or treatment interruption will lead to a worsening of the disease itself [54].
Of the diseases analyzed in this meta-analysis, SCD has shown the highest adherence. It is a rare genetic disease, caused by a mutation of the beta-chain of hemoglobin, producing an alteration of the erythrocyte shape, and resulting in a large formation of cell aggregates, which can eventually lead to hypoxia, hypercoagulability, increased platelet activation, and increased neutrophil adhesiveness. Adebiyi and collaborators described that factors such as climate, fetal hemoglobin levels, and even certain infections may play a definitive role in the manifestation of this disease [55]. There are multiple treatments for this disease; however, in the studies used in this meta-analysis, the use of hydroxyurea has predominated [41,45,49].
The mean adherence measured with MMAS-8 in the six included studies [38,39,40,41,45,49] was 56.91%. In a systematic review and meta-analysis of 14 studies performed by Loiselle et al. [56], with a sample of 921 persons, results similar to those of this meta-analysis were observed, showing an overall adherence to pharmacological treatment, measured by various methods, of 50% in patients with SCD. In a systematic review by Walsh et al. [57], they observed that self-reported measures of adherence, such as the MMAS-4 or MMAS-8, tend to have a higher compliance rate (48–89%) than those measured by objective methods, such as urinalysis. These authors observed a higher adherence rate in patients who used hydroxyurea as a drug, a result that coincides with that observed in our meta-analysis, which shows an adherence rate of 57.25% in patients treated with hydroxyurea. Increasing adherence is important, since this RD primarily affects children. One solution may be setting preventive clinic visits. A systematic review, conducted in 2010 by Dean and collaborators [58], concluded that behavioral and educational interventions are effective in achieving greater adherence, and help parents and caregivers to correctly give medication to their children.
On the other hand, this systematic review and meta-analysis included a total of 4 studies [37,46,51,52] with patients suffering from MG. MG is an autoimmune disease of neuromuscular origin, characterized by different symptoms that depend on the degree of involvement of the patient’s striated muscle [59]. Current treatment is based on generalized and nonspecific immunosuppression. It is a disease that responds well to pharmacological treatment, so adherence is essential for this type of patient.
The heterogeneity of MG results in multiple therapeutic approaches, depending on the subtype. Therefore, Guptill et al. discussed that clinical trials to find an effective treatment must be performed in a population that is as homogeneous as possible, which is sometimes a complicated objective, due to the fact that it is a RD [60]. In this meta-analysis, adherence of 63.19% was observed, measured by MMAS-4 and MMAS-8; the value was higher than those observed in patients with SCD. These differences may be due to the type of treatment used. In patients with MG, chronic treatment is carried out with easily applied immunosuppressive drugs, such as corticosteroids, which produce a remission or decrease in symptoms in 70% of patients, as mentioned in the work of Alhaidar and colleagues [61]. These are usually used in combination with other immunosuppressive drugs, such as cyclosporine or methotrexate, to reduce the dose of corticosteroids, achieving an even greater improvement in the patient’s symptoms [60].
Other rare diseases analyzed in this meta-analysis have very different adherence to treatment, ranging from 16.42% to 88.21%. As mentioned above, this review has included the most common rare diseases studied by the pharmaceutical [19,20]. Adherence to pharmacological treatment is one of the major concerns of health professionals (physicians, pharmacists), due to the importance of reducing the symptoms of this type of disease.
Congenital hypogonadotropic hypogonadism [62], pulmonary arterial hypertension [63], cystic fibrosis [64], Wilson’s disease [65], etc. are diseases whose drugs can help patients maintain their quality of life through chronic treatment. However, adherence to treatment will depend on different aspects, some related to the patient (lack of motivation, depression, denial, cognitive deterioration, etc.), and others related to the pharmacological treatment (complexity, side effects, time, etc.) [66].
The factors that can increase adherence to pharmacological treatment are the support of both family and health professionals, the ease of taking the drugs, the benefits perceived by the patient, and the establishment of a routine for taking the medication. Another factor that can favor increased adherence in patients with this type of disease is the dispensing of these drugs in community pharmacies and primary care centers [64].
This systematic review and meta-analysis has several limitations. First, despite performing an exhaustive search for different rare diseases, it is possible that some relevant articles were not included. Second, unpublished literature has not been included. Third, the studies included in the article used self-report questionnaires, which may result in a “social desirability and recall bias”. In addition, the gray literature was not used in this review, which may lead to loss of information. Finally, there are very few studies using a sample with a rare disease, and only the MMAS was used to measure adherence, so the results should not be generalized and should be interpreted with caution.

5. Conclusions

The results observed in adherence to treatment in patients with rare diseases had great variability, due to the different aspects involved in the greater or lesser applicability of the treatment. Low adherence is a heterogeneous and multifactorial problem that requires not only health professionals, as providers or pharmacists, but also the intervention of psychologists and a multidisciplinary team, including the family. In addition, such treatment must be established individually, according to individuals’ characteristics.

Supplementary Materials

The supporting information can be downloaded at: https://www.mdpi.com/article/10.3390/healthcare11111609/s1.

Author Contributions

A.M.G.-M. and D.V.-M.: collected, analyzed, and interpreted the data, and drafted the manuscript. B.C. and P.Z.: designed the study, interpreted the data, and reviewed the manuscript; E.M.d.V.: served as a third evaluator in data curation and reviewed the manuscript; P.B.: interpreted the data and prepared the manuscript. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Not applicable.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Belzer, L.T.; Wright, S.M.; Goodwin, E.J.; Singh, M.N.; Carter, B.S. Psychosocial Considerations for the Child with Rare Disease: A Review with Recommendations and Calls to Action. Children 2022, 9, 933. [Google Scholar] [CrossRef] [PubMed]
  2. Collin-Histed, T.; Gershkowitz, J.; Stevens, B.; Timmins, G. The Patient Perspective on Rare Diseases. In Lysosomal Storage Disorders; John Wiley & Sons, Ltd.: Hoboken, NJ, USA, 2022; pp. 299–307. ISBN 978-1-119-69731-2. [Google Scholar]
  3. Nguengang Wakap, S.; Lambert, D.M.; Olry, A.; Rodwell, C.; Gueydan, C.; Lanneau, V.; Murphy, D.; Le Cam, Y.; Rath, A. Estimating Cumulative Point Prevalence of Rare Diseases: Analysis of the Orphanet Database. Eur. J. Hum. Genet. 2020, 28, 165–173. [Google Scholar] [CrossRef] [PubMed]
  4. EURODIS. What Is a Rare Disease? Available online: https://www.eurordis.org/information-support/what-is-a-rare-disease/ (accessed on 19 December 2022).
  5. Hedley, V.; Hannah, M.; Charlotte, R.; Ségolène, A. Report on the State of the Art of Rare Disease Activities in Europe; European Commission: Brussels, Belgium, 2018. [Google Scholar]
  6. ICORD. Naoko Yamamoto (Disease Control Division, Ministry of Health, Labour and Welfare, Japan). Rare Disease Policies in Japan. In Proceedings of the 7th ICORD International Conference for Rare and Intractable Diseases and Orphan Drugs—C3: Connection and Collaboration, for Creation 2012, Tokyo, Japan, 4–6 February 2012. [Google Scholar]
  7. Richter, T.; Nestler-Parr, S.; Babela, R.; Khan, Z.M.; Tesoro, T.; Molsen, E.; Hughes, D.A. Rare Disease Terminology and Definitions—A Systematic Global Review: Report of the ISPOR Rare Disease Special Interest Group. Value Health 2015, 18, 906–914. [Google Scholar] [CrossRef] [PubMed]
  8. Haendel, M.; Vasilevsky, N.; Unni, D.; Bologa, C.; Harris, N.; Rehm, H.; Hamosh, A.; Baynam, G.; Groza, T.; McMurry, J.; et al. How Many Rare Diseases Are There? Nat. Rev. Drug Discov. 2020, 19, 77–78. [Google Scholar] [CrossRef] [PubMed]
  9. European Commission Rare Diseases. Available online: https://research-and-innovation.ec.europa.eu/research-area/health/rare-diseases_en (accessed on 5 December 2022).
  10. Kripalani, S.; Yao, X.; Haynes, R.B. Interventions to Enhance Medication Adherence in Chronic Medical Conditions: A Systematic Review. Arch. Intern. Med. 2007, 167, 540–550. [Google Scholar] [CrossRef]
  11. Patel, V.; Flisher, A.J.; Hetrick, S.; McGorry, P. Mental Health of Young People: A Global Public-Health Challenge. Lancet 2007, 369, 1302–1313. [Google Scholar] [CrossRef]
  12. Pereira, E.D.B.; de Matos Cavalcante, A.G. Prescription Is Not Enough: The Importance of Adherence to Pharmacological Treatment of COPD. J. Bras. Pneumol. Publicacao Of. Soc. Bras. Pneumol. E Tisilogia 2022, 48, e20220058. [Google Scholar] [CrossRef]
  13. Mongkhon, P.; Ashcroft, D.M.; Scholfield, C.N.; Kongkaew, C. Hospital Admissions Associated with Medication Non-Adherence: A Systematic Review of Prospective Observational Studies. BMJ Qual. Saf. 2018, 27, 902–914. [Google Scholar] [CrossRef]
  14. Jacquelet, E.; Poujois, A.; Pheulpin, M.-C.; Demain, A.; Tinant, N.; Gastellier, N.; Woimant, F. Adherence to Treatment, a Challenge Even in Treatable Metabolic Rare Diseases: A Cross Sectional Study of Wilson’s Disease. J. Inherit. Metab. Dis. 2021, 44, 1481–1488. [Google Scholar] [CrossRef]
  15. Masełbas, W.; Członkowska, A.; Litwin, T.; Niewada, M. Persistence with Treatment for Wilson Disease: A Retrospective Study. BMC Neurol. 2019, 19, 278. [Google Scholar] [CrossRef]
  16. Rajiah, K.; Sivarasa, S.; Maharajan, M.K. Impact of Pharmacists’ Interventions and Patients’ Decision on Health Outcomes in Terms of Medication Adherence and Quality Use of Medicines among Patients Attending Community Pharmacies: A Systematic Review. Int. J. Environ. Res. Public. Health 2021, 18, 4392. [Google Scholar] [CrossRef]
  17. Page, M.J.; McKenzie, J.E.; Bossuyt, P.M.; Boutron, I.; Hoffmann, T.C.; Mulrow, C.D.; Shamseer, L.; Tetzlaff, J.M.; Akl, E.A.; Brennan, S.E.; et al. The PRISMA 2020 Statement: An Updated Guideline for Reporting Systematic Reviews. BMJ 2021, 372, n71. [Google Scholar] [CrossRef]
  18. Szmelter, A. Global Pharmaceutical Industry: Characteristics and Trends. Available online: https://www.igi-global.com/chapter/global-pharmaceutical-industry/www.igi-global.com/chapter/global-pharmaceutical-industry/216205 (accessed on 5 December 2022).
  19. Torreya Global Pharma Industry Study. Torreya The Future of the Global Pharmaceutical Industry. 2017. Available online: https://torreya.com/publications/torreya_global_pharma_industry_study_october2017.pdf (accessed on 28 May 2023).
  20. Orphanet. The Portal for Rare Diseases and Orphan Drugs. Available online: http://www.orpha.net/consor/www/cgi-bin/index.php?lng=ES (accessed on 5 December 2022).
  21. Hoy, D.; Brooks, P.; Woolf, A.; Blyth, F.; March, L.; Bain, C.; Baker, P.; Smith, E.; Buchbinder, R. Assessing Risk of Bias in Prevalence Studies: Modification of an Existing Tool and Evidence of Interrater Agreement. J. Clin. Epidemiol. 2012, 65, 934–939. [Google Scholar] [CrossRef]
  22. Morisky, D.E.; Green, L.W.; Levine, D.M. Concurrent and Predictive Validity of a Self-Reported Measure of Medication Adherence. Med. Care 1986, 24, 67–74. [Google Scholar] [CrossRef]
  23. Morisky, D.E.; Ang, A.; Krousel-Wood, M.; Ward, H.J. Predictive Validity of a Medication Adherence Measure in an Outpatient Setting. J. Clin. Hypertens. 2008, 10, 348–354. [Google Scholar] [CrossRef]
  24. Nyaga, V.N.; Arbyn, M.; Aerts, M. Metaprop: A Stata Command to Perform Meta-Analysis of Binomial Data. Arch. Public Health 2014, 72, 39. [Google Scholar] [CrossRef]
  25. Lin, L.; Chu, H. Meta-Analysis of Proportions Using Generalized Linear Mixed Models. Epidemiol. Camb. Mass 2020, 31, 713–717. [Google Scholar] [CrossRef]
  26. Newcombe, R.G. Two-Sided Confidence Intervals for the Single Proportion: Comparison of Seven Methods. Stat. Med. 1998, 17, 857–872. [Google Scholar] [CrossRef]
  27. Barendregt, J.J.; Doi, S.A.; Lee, Y.Y.; Norman, R.E.; Vos, T. Meta-Analysis of Prevalence. J. Epidemiol. Community Health 2013, 67, 974–978. [Google Scholar] [CrossRef]
  28. Lin, L.; Xu, C. Arcsine-Based Transformations for Meta-Analysis of Proportions: Pros, Cons, and Alternatives. Health Sci. Rep. 2020, 3, e178. [Google Scholar] [CrossRef]
  29. Badawy, S.M.; Thompson, A.A.; Lai, J.-S.; Penedo, F.J.; Rychlik, K.; Liem, R.I. Health-Related Quality of Life and Adherence to Hydroxyurea in Adolescents and Young Adults with Sickle Cell Disease. Pediatr. Blood Cancer 2017, 64, e26369. [Google Scholar] [CrossRef] [PubMed]
  30. Badawy, S.M.; Thompson, A.A.; Penedo, F.J.; Lai, J.-S.; Rychlik, K.; Liem, R.I. Barriers to Hydroxyurea Adherence and Health-Related Quality of Life in Adolescents and Young Adults with Sickle Cell Disease. Eur. J. Haematol. 2017, 98, 608–614. [Google Scholar] [CrossRef] [PubMed]
  31. Galadanci, N.A.; Umar Abdullahi, S.; Vance, L.D.; Musa Tabari, A.; Ali, S.; Belonwu, R.; Salihu, A.; Amal Galadanci, A.; Wudil Jibir, B.; Bello-Manga, H.; et al. Feasibility Trial for Primary Stroke Prevention in Children with Sickle Cell Anemia in Nigeria (SPIN Trial). Am. J. Hematol. 2017, 92, 780–788. [Google Scholar] [CrossRef] [PubMed]
  32. Pernell, B.M.; DeBaun, M.R.; Becker, K.; Rodeghier, M.; Bryant, V.; Cronin, R.M. Improving Medication Adherence with Two-Way Short Message Service Reminders in Sickle Cell Disease and Asthma. A Feasibility Randomized Controlled Trial. Appl. Clin. Inform. 2017, 8, 541–559. [Google Scholar] [CrossRef]
  33. Viola, A.S.; Drachtman, R.; Kaveney, A.; Sridharan, A.; Savage, B.; Delnevo, C.D.; Coups, E.J.; Porter, J.S.; Devine, K.A. Feasibility of Medical Student Mentors to Improve Transition in Sickle Cell Disease. J. Pediatr. Psychol. 2021, 46, 650–661. [Google Scholar] [CrossRef]
  34. Fogarty, H.; Gaul, A.; Syed, S.; Aleksejenko, N.; Geoghegan, R.; Conroy, H.; Crampton, E.; Ngwenya, N.; Tuohy, E.; McMahon, C. Adherence to Hydroxyurea, Health-Related Quality of Life Domains and Attitudes towards a Smartphone App among Irish Adolescents and Young Adults with Sickle Cell Disease. Ir. J. Med. Sci. 2022, 191, 809–816. [Google Scholar] [CrossRef]
  35. Ivarsson, B.; Hesselstrand, R.; Rådegran, G.; Kjellström, B. Adherence and Medication Belief in Patients with Pulmonary Arterial Hypertension or Chronic Thromboembolic Pulmonary Hypertension: A Nationwide Population-Based Cohort Survey. Clin. Respir. J. 2018, 12, 2029–2035. [Google Scholar] [CrossRef]
  36. Dzemaili, S.; Tiemensma, J.; Quinton, R.; Pitteloud, N.; Morin, D.; Dwyer, A.A. Beyond Hormone Replacement: Quality of Life in Women with Congenital Hypogonadotropic Hypogonadism. Endocr. Connect. 2017, 6, 404–412. [Google Scholar] [CrossRef]
  37. Vitturi, B.K.; Pellegrinelli, A.; Valerio, B.C.O. Medication Adherence in Patients with Myasthenia Gravis in Brazil: A Cross-Sectional Study. Acta Neurol. Belg. 2020, 120, 83–89. [Google Scholar] [CrossRef]
  38. Viswanathan, K.; Swaminathan, N.; Viswanathan, R.; Lakkaraja, M. Caregiver’s Health Locus of Control and Medication Adherence in Sickle Cell Disease. J. Natl. Med. Assoc. 2015, 107, 51–55. [Google Scholar] [CrossRef]
  39. Thuret, I.; Hacini, M.; Pégourié-Bandelier, B.; Gardembas-Pain, M.; Bisot-Locard, S.; Merlat-Guitard, A.; Bachir, D. Socio-Psychological Impact of Infused Iron Chelation Therapy with Deferoxamine in Metropolitan France: ISOSFER Study Results. Hematol. Amst. Neth. 2009, 14, 315–322. [Google Scholar] [CrossRef]
  40. Treadwell, M.J.; Law, A.W.; Sung, J.; Hackney-Stephens, E.; Quirolo, K.; Murray, E.; Glendenning, G.A.; Vichinsky, E. Barriers to Adherence of Deferoxamine Usage in Sickle Cell Disease. Pediatr. Blood Cancer 2005, 44, 500–507. [Google Scholar] [CrossRef]
  41. Raji, S.O.; Lawani, A.O.; James, B.O. Prevalence and Correlates of Major Depression among Nigerian Adults with Sickle Cell Disease. Int. J. Psychiatry Med. 2016, 51, 456–466. [Google Scholar] [CrossRef]
  42. Lamiani, G.; Strada, I.; Mancuso, M.E.; Coppola, A.; Vegni, E.; Moja, E.A. Pro-Adherence Study Group Factors Influencing Illness Representations and Perceived Adherence in Haemophilic Patients: A Pilot Study. Haemophilia 2015, 21, 598–604. [Google Scholar] [CrossRef]
  43. Knudsen, K.B.; Pressler, T.; Mortensen, L.H.; Jarden, M.; Skov, M.; Quittner, A.L.; Katzenstein, T.; Boisen, K.A. Associations between Adherence, Depressive Symptoms and Health-Related Quality of Life in Young Adults with Cystic Fibrosis. SpringerPlus 2016, 5, 1216. [Google Scholar] [CrossRef]
  44. Karaca, C.; Dincer, M.T.; Ozcan, S.G.; Sarac, B.; Ahmadzada, S.; Alagoz, S.; Bakir, A.; Kiykim, E.; Trabulus, S.; Seyahi, N. The Impact of the COVID-19 Pandemic on Fabry Disease Patients: An Examination of Mood Status, Therapy Adherence, and COVID-19 Infection. Orphanet J. Rare Dis. 2022, 17, 338. [Google Scholar] [CrossRef]
  45. Badawy, S.M.; Thompson, A.A.; Holl, J.L.; Penedo, F.J.; Liem, R.I. Healthcare Utilization and Hydroxyurea Adherence in Youth with Sickle Cell Disease. Pediatr. Hematol. Oncol. 2018, 35, 297–308. [Google Scholar] [CrossRef]
  46. Bacci, E.D.; Coyne, K.S.; Poon, J.-L.; Harris, L.; Boscoe, A.N. Understanding Side Effects of Therapy for Myasthenia Gravis and Their Impact on Daily Life. BMC Neurol. 2019, 19, 335. [Google Scholar] [CrossRef]
  47. Introna, A.; D’Errico, E.; Modugno, B.; Scarafino, A.; Fraddosio, A.; Distaso, E.; Tempesta, I.; Mastronardi, A.; Simone, I.L. Adherence to Riluzole in Patients with Amyotrophic Lateral Sclerosis: An Observational Study. Neuropsychiatr. Dis. Treat. 2018, 14, 193–203. [Google Scholar] [CrossRef]
  48. Grady, D.; Weiss, M.; Hernandez-Sanchez, J.; Pepke-Zaba, J. Medication and Patient Factors Associated with Adherence to Pulmonary Hypertension Targeted Therapies. Pulm. Circ. 2018, 8, 2045893217743616. [Google Scholar] [CrossRef]
  49. Galadanci, N.A.; Abdullahi, S.U.; Tabari, M.A.; Abubakar, S.; Belonwu, R.; Salihu, A.; Neville, K.; Kirkham, F.; Inusa, B.; Shyr, Y.; et al. Primary Stroke Prevention in Nigerian Children with Sickle Cell Disease (SPIN): Challenges of Conducting a Feasibility Trial. Pediatr. Blood Cancer 2015, 62, 395–401. [Google Scholar] [CrossRef] [PubMed]
  50. Dwyer, A.A.; Tiemensma, J.; Quinton, R.; Pitteloud, N.; Morin, D. Adherence to Treatment in Men with Hypogonadotrophic Hypogonadism. Clin. Endocrinol. 2017, 86, 377–383. [Google Scholar] [CrossRef] [PubMed]
  51. Aşiret, G.D.; Kapucu, S.; Kaymaz, T.T.; Kurt, C.E.B. Psychosocial Adjustment and Adherence to Medication in Patients with Myasthenia Gravis. Gazi Med. J. 2021, 32, 3. [Google Scholar]
  52. Idiaquez, J.F.; Gonzalez, S.; Lasso-Penafiel, J.; Barnett, C. Pharmacological treatment compliance and a description of its associated factors in patients with myasthenia gravis. Rev. Neurol. 2018, 66, 15–20. [Google Scholar]
  53. Gimenez-Lozano, C.; Páramo-Rodríguez, L.; Cavero-Carbonell, C.; Corpas-Burgos, F.; López-Maside, A.; Guardiola-Vilarroig, S.; Zurriaga, O. Rare Diseases: Needs and Impact for Patients and Families: A Cross-Sectional Study in the Valencian Region, Spain. Int. J. Environ. Res. Public. Health 2022, 19, 10366. [Google Scholar] [CrossRef]
  54. Gorini, F.; Santoro, M.; Pierini, A.; Mezzasalma, L.; Baldacci, S.; Bargagli, E.; Boncristiano, A.; Brunetto, M.R.; Cameli, P.; Cappelli, F.; et al. Orphan Drug Use in Patients with Rare Diseases: A Population-Based Cohort Study. Front. Pharmacol. 2022, 13, 869842. [Google Scholar] [CrossRef]
  55. Adebiyi, M.G.; Manalo, J.M.; Xia, Y. Metabolomic and Molecular Insights into Sickle Cell Disease and Innovative Therapies. Blood Adv. 2019, 3, 1347–1355. [Google Scholar] [CrossRef]
  56. Loiselle, K.; Lee, J.L.; Szulczewski, L.; Drake, S.; Crosby, L.E.; Pai, A.L.H. Systematic and Meta-Analytic Review: Medication Adherence Among Pediatric Patients with Sickle Cell Disease. J. Pediatr. Psychol. 2016, 41, 406–418. [Google Scholar] [CrossRef]
  57. Walsh, K.E.; Cutrona, S.L.; Kavanagh, P.L.; Crosby, L.E.; Malone, C.; Lobner, K.; Bundy, D.G. Medication Adherence Among Pediatric Patients with Sickle Cell Disease: A Systematic Review. Pediatrics 2014, 134, 1175–1183. [Google Scholar] [CrossRef]
  58. Dean, A.J.; Walters, J.; Hall, A. A Systematic Review of Interventions to Enhance Medication Adherence in Children and Adolescents with Chronic Illness. Arch. Dis. Child. 2010, 95, 717–723. [Google Scholar] [CrossRef]
  59. Suresh, A.B.; Asuncion, R.M.D. Myasthenia Gravis; StatPearls Publishing: Tampa, FL, USA, 2022. [Google Scholar]
  60. Guptill, J.T.; Soni, M.; Meriggioli, M.N. Current Treatment, Emerging Translational Therapies, and New Therapeutic Targets for Autoimmune Myasthenia Gravis. Neurotherapeutics 2016, 13, 118–131. [Google Scholar] [CrossRef]
  61. Alhaidar, M.K.; Abumurad, S.; Soliven, B.; Rezania, K. Current Treatment of Myasthenia Gravis. J. Clin. Med. 2022, 11, 1597. [Google Scholar] [CrossRef]
  62. Gao, Y.; Yu, B.; Mao, J.; Wang, X.; Nie, M.; Wu, X. Assisted Reproductive Techniques with Congenital Hypogonadotropic Hypogonadism Patients: A Systematic Review and Meta-Analysis. BMC Endocr. Disord. 2018, 18, 85. [Google Scholar] [CrossRef]
  63. Klinger, J.R.; Elliott, C.G.; Levine, D.J.; Bossone, E.; Duvall, L.; Fagan, K.; Frantsve-Hawley, J.; Kawut, S.M.; Ryan, J.J.; Rosenzweig, E.B.; et al. Therapy for Pulmonary Arterial Hypertension in Adults: Update of the CHEST Guideline and Expert Panel Report. Chest 2019, 155, 565–586. [Google Scholar] [CrossRef]
  64. Macdonald, M.; Martin-Misener, R.; Helwig, M.; Smith, L.J.; Godfrey, C.M.; Curran, J.; Murphy, A. Experiences of Adults with Cystic Fibrosis in Adhering to Medication Regimens: A Qualitative Systematic Review. JBI Database Syst. Rev. Implement. Rep. 2016, 14, 258–285. [Google Scholar] [CrossRef]
  65. Appenzeller-Herzog, C.; Mathes, T.; Heeres, M.L.S.; Weiss, K.H.; Houwen, R.H.J.; Ewald, H. Comparative Effectiveness of Common Therapies for Wilson Disease: A Systematic Review and Meta-Analysis of Controlled Studies. Liver Int. 2019, 39, 2136–2152. [Google Scholar] [CrossRef]
  66. Kleinsinger, F. The Unmet Challenge of Medication Nonadherence. Perm. J. 2018, 22, 18–033. [Google Scholar] [CrossRef]
Figure 1. Identification of studies via databases and registers.
Figure 1. Identification of studies via databases and registers.
Healthcare 11 01609 g001
Figure 2. Overall proportion of treatment adherence (%) [14,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52].
Figure 2. Overall proportion of treatment adherence (%) [14,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52].
Healthcare 11 01609 g002
Figure 3. Subgroup analysis in relation to age group for treatment adherence (%) [14,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52].
Figure 3. Subgroup analysis in relation to age group for treatment adherence (%) [14,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52].
Healthcare 11 01609 g003
Figure 4. Random-effects meta-regression models of mean age.
Figure 4. Random-effects meta-regression models of mean age.
Healthcare 11 01609 g004
Table 1. Characteristics of the studies included (N = 18).
Table 1. Characteristics of the studies included (N = 18).
ReferenceYearCountryStudy DesignTotal (n)Women (%)Age (Mean)Rare DiseaseToolTherapyAdherence (%)
Aşiret et al. [51] 2021TurkeyCross-sectional5464.844.1Myasthenia gravisMMAS-4Cholinesterase inhibitors, Glucocorticoids (methyl prednisolone, prednisone vb.), Ciclosporin94.5
Bacci et al. [46]2019USACross-sectional24264.058.4Myasthenia gravisMMAS-8Intravenous immunoglobulin63.7
Badawi et al. [45]2018USACross-sectional3441.014.0Sickle cell diseaseMMAS-8Hydroxyurea26.0
Dwyer et al. [50]2016OnlineCross-sectional101037.0Congenital hypogonadotropic hypogonadismMMAS-8Testosterone replacement therapy or fertility-inducing treatment via exogenous42.6
Dzemaili et al. [36]2017Social media sitesCross-sectional5510020.7Congenital hypogonadotropic hypogonadismMMAS-8Hormone replacement38.2
Galadanci et al. [49]2015NigeriaRandomized clinical trial2552.06.8Sickel cell diseaseMMAS-8Hydroxyurea100
Grady et al. [48]2018United KingdomCross-sectional26370.661.6Pulmonary arterial hypertensionMMAS-8Ambrisentan, bosentan, sildenafil, tadalafil, iloprost, epoprostenol, ERA + PDE5i, Iloprost (nebulized) + PDE5i, IV/SC Prostanoid + ERA, IV/SC Prostanoid + PDE5i, Trial drug + ERA +/− PDE5i88.2
Idiázquez et al. [52]2018ChileCross-sectional2657.755.5Myasthenia gravisMMAS-4Cholinesterase inhibitors or immunosuppressors38.5
Introna et al. [47]2018ItalyCross-sectional4540.063.8Amyotrophic lateral sclerosisMMAS-8Riluzole tablet or oral suspension of riluzole62.2
Jacquelet et al. [14]2021FranceCross-sectional13950.439.0Wilson’s diseaseMMAS-8D-Penicillamine, Trientine 2HCl, Zinc acetate, Zinc sulfate and Zinc sulfate79.1
Karaca et al. [44]2022TurkeyCross-sectional6752.237.0Fabry diseaseMMAS-4 Enzyme replacement therapy16.4
Knudsen et al. [43]2016DenmarkCross-sectional6759.024.1Cystic fibrosisMMAS-8NR25.8
Lamiani et al. [42]2015ItalyCross-sectional 50039.7Hemophilia AMMAS-4On-demand and prophylaxis76.0
Raji et al. [41]2016NigeriaCross-sectional20586.325.4Sickle cell diseaseMMAS-8Hydroxyurea34.1
Thuret et al. [39]2009FranceCross-sectional7054.044.6Sickle cell disease myelodysplastic syndromes and β-thalassemiaMMAS-4DFO, deferiprone, deferiprone + DFO or deferasirox72.0
Treadwell et al. [40]2005USACross-sectional1553.512.1Sickle cell diseaseMMAS-4Chelation therapy20.0
Viswanathan et al. [38]2015USACross-sectional4348.85.7Sickle cell diseaseMMAS-8Hydroxyurea and Penicillin69.0
Vitturi et al. [37]2020BrazilCross-sectional5881.046.3Myasthenia gravisMMAS-8NR44.8
DFO: Deferoxamine; ERA: endothelin antagonist; MMAS-4: Morisky Medication Adherence Scale-4; MMAS-8: Morisky Medication Adherence Scale-8; MDS: myelodysplastic syndromes; NR: Not reported; PDE5i: phosphodiesterase type-5 inhibitor; SCD: Sickle cell disease.
Table 2. Characteristics of the meta-regression model.
Table 2. Characteristics of the meta-regression model.
VariableCoefficientLower Limit
Confidence Interval
Upper Limit
Confidence Interval
p-Value
Age mean0.32−0.441.080.403
Year of publication−0.34−3.783.100.845
Quality score−5.42−19.788.940.459
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

García-Muñoz, A.M.; Victoria-Montesinos, D.; Cerdá, B.; Ballester, P.; de Velasco, E.M.; Zafrilla, P. Self-Reported Medication Adherence Measured with Morisky Scales in Rare Disease Patients: A Systematic Review and Meta-Analysis. Healthcare 2023, 11, 1609. https://doi.org/10.3390/healthcare11111609

AMA Style

García-Muñoz AM, Victoria-Montesinos D, Cerdá B, Ballester P, de Velasco EM, Zafrilla P. Self-Reported Medication Adherence Measured with Morisky Scales in Rare Disease Patients: A Systematic Review and Meta-Analysis. Healthcare. 2023; 11(11):1609. https://doi.org/10.3390/healthcare11111609

Chicago/Turabian Style

García-Muñoz, Ana María, Desirée Victoria-Montesinos, Begoña Cerdá, Pura Ballester, Eloisa María de Velasco, and Pilar Zafrilla. 2023. "Self-Reported Medication Adherence Measured with Morisky Scales in Rare Disease Patients: A Systematic Review and Meta-Analysis" Healthcare 11, no. 11: 1609. https://doi.org/10.3390/healthcare11111609

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop