The Impact of the Chronic Disease Self-Management Program on Health Literacy: A Pre-Post Study Using a Multi-Dimensional Health Literacy Instrument

This study assessed the impact of the Chronic Disease Self-Management Program (CDSMP) on different domains of health literacy using a pre-post study design. Participants aged over 16 years and with one or more self-reported chronic diseases were recruited for the CDSMP in western Sydney (a highly diverse area of New South Wales, Australia) between October 2014 and September 2018. Health literacy was assessed pre- and immediately post-intervention using the Health Literacy Questionnaire (HLQ), with differences in mean scores for each HLQ domain analysed using paired sample t-tests. A total of 486 participants were recruited into the CDSMP. Of those, 316 (65.0%) completed both pre- and post-intervention surveys and were included in the analysis. The median age of the participants was 68 years, the majority were female (62.5%), and most were born in a country other than Australia (80.6%). There were statistically significant (P < 0.001) improvements across all nine domains of the HLQ. This is the first study evaluating the potential impact of the CDSMP on improving different domains of health literacy amongst a diverse sample of participants with chronic diseases using a multi-dimensional instrument. The absence of a control population in this study warrants caution when interpreting the results.


Introduction
Non-communicable (chronic) disease is a global health problem. Chronic diseases (e.g., cardiovascular diseases, cancer, diabetes, chronic respiratory diseases) are the leading cause of disease burden and mortality worldwide, with the attributable share of total global burden of disease increasing steadily from 44% in 1990 to 61% in 2016 [1]. In Australia, chronic disease accounts for 84-88% of the total disease burden [2].
In the absence of a cure, chronic disease must be managed over time as it evolves with shifting severity, pace, and treatments [3]. Holman has written eloquently about the features of chronic illnesses, emphasising their dynamic and evolving nature and variations in severity over time, which require competent, often daily decision making and management by patients [3]. Given this, developing patients' self-management skills and confidence has become an increasingly important part of clinical in Australia, 47% of residents are overseas-born, and one in two speak a language other than English at home [15]. Since 2013, WSLHD has hosted a demonstration program to improve the integration of care between primary and tertiary health systems. One important part of this program was to better support the large group in the population who were living with (often complex) chronic disease, and who were frequent users of the healthcare system. This included making the established CDSMP (described below) available for eligible participants across the health district [16].

Intervention
The CDSMP was developed at Stanford University and has been broadly disseminated across populations and several countries [17][18][19][20][21][22]. The CDSMP comprises structured small-group interventions (2.5 hours each) over six weeks and an accompanying reference book. The program includes aspects encountered throughout the chronic illness trajectory (e.g., fatigue, medication/symptom management, decision-making, communication with providers, and behavioural changes related to nutrition and exercise) [23], and seeks to build self-efficacy for self-management through the mastery of skills (action planning and feedback), symptom reinterpretation, modelling of self-management behaviours, problem-solving strategies, and social persuasion [24] (see Figure 1). The CDSMP is peer-led by trained individuals with a personal experience living with a chronic condition [23].
The content of the CDSMP maps onto the domains of health literacy measured in the HLQ. It teaches participants, for example, how to actively manage their health (HLQ Domain 3) and provides them with information to do so (Domain 2; Domain 8; Domain 9). Topics related to communication skills, making decisions and working with health professionals and organisations may help participants to actively engage with healthcare providers (Domain 6) and feel supported and understood by them (Domain 1), as well as navigate the healthcare system more effectively (Domain 7). The small-group format of the program may also increase perceived social support for health (Domain 4).
In this study, the CDSMP program was delivered by trained bilingual and Indigenous community leaders recruited from multicultural health services, local non-government organisations and volunteer networks. To qualify, all leaders completed 24 h of training across four full-day sessions hosted by master trainers. A total of 27 leaders were trained between 2014 and 2016. Each CDSMP session was led by two trained leaders who followed a detailed Chronic Disease Self-Management Leader's Manual and Fidelity Manual to deliver the program. Programs were delivered in several languages (e.g., Mandarin; Cantonese; Turkish; Tagalong; see Table 1) via 'on-the-spot' translation by the group leader. All participants were provided with Living a Healthy Life with Chronic Conditions reference text, either in English or simplified Chinese.

Participants and Recruitment
We used multiple recruitment strategies, with a non-random selection of the study participants. Individuals were referred to the CDSMP through the WSLHD's multicultural health services, existing chronic disease services (e.g., Connecting Care; Chronic Care Rehabilitation Service), and via clinicians within the Local Health District and general practice network. The inclusion criteria were individuals who (a) had one or more (self-reported) chronic diseases, (b) were over 16 years of age, and (c) had the (self-reported) ability to attend the 6-week face-to-face CDSMP.

Outcome Measures
The participants were asked to complete paper-based questionnaires providing demographic information and assessing health literacy across a range of domains using the HLQ, as described below. The data were collected upon commencement (pre-intervention) and at the conclusion (post-intervention) of the CDSMP. Although facilitators were present and could assist in reading difficult words, participants were required to answer questions independently.
Health literacy was measured using the HLQ [13]; a multi-dimensional tool that measures health literacy across 9 distinct conceptual domains ( Figure 2) [13,35]. The HLQ has 44 questions, with 4 to 5 questions per domain. Domains 1 to 5 assess the strength of participants' agreement with a statement on a 4-point ordinal scale (strongly disagree = 1 to strongly agree = 4) while domains 6 to 9 assess participants' perceived ease in task completion on a 5-point ordinal scale (cannot do or always difficult = 1 to very easy = 5). The median reliability of the HLQ domains is 0.88, with a range from 0.77 to 0.90 [13]. The questionnaire takes between 7 to 40 min to complete. Participants enrolled in non-English CDSMP groups in this study completed a translated HLQ which were available from the instrument developers.

Statistical Analysis
Descriptive statistics were used to summarise demographic characteristics and the pre-and post-intervention HLQ scores. The pre-intervention demographic data and mean HLQ domain scores between those who completed the program and those who did not complete the program were compared using chi-square tests and independent sample t-tests (see Table A1). Paired sample t-tests were used to assess whether the HLQ domain mean scores differed pre-and post-intervention. Exploratory multivariable linear regression modelling was utilised to investigate the associations between the HLQ domain score changes and a range of demographic variables.
All data were analysed using IBM SPSS Version 25 and R version 3.5.2, with P < 0.05 considered statistically significant. The expectation maximisation algorithm was used to substitute the missing HLQ values, following the HLQ User Manual; up to two missing values were imputed for domains with four or five questions. Missing data that could not be imputed were excluded pairwise from the analysis.

Ethics
The approval to evaluate and publish the results of this program was obtained from the Westmead Scientific Advisory QA Committee and the Secretary of the WSLHD Human Research Ethics Committee (Protocol number: 1809-04 QA).

Demographic Characteristics
In total, 486 participants were recruited for the CDSMP between October 2014 and September 2018. Of those, 38 were excluded from the analysis as they did not complete the pre-intervention questionnaire. An additional 132 participants did not complete the post-intervention questionnaire or were removed from the analysis due to indistinguishable pre-and post-intervention questionnaires, leaving a sample of 316 (65.0%) participants included in the analysis (see Figure 3). The pre-intervention demographic characteristics of participants who completed both the pre-and post-intervention questionnaires are summarised in Table 1. Overall, the median age of participants was 68 years. There was approximately twice as many female participants than male participants. The majority were non-native English speakers, did not live alone, had a healthcare card, and were born in Australia, the Greater China regions or India. Health Care Cards are issued by the Australian Department of Human Services to people who live in Australia and already receive government payments, benefits or allowances. Approximately 15% of participants indicated that they required assistance to complete the questionnaires. Compared to the participants who did not complete the program, those who did were less likely to speak English at home (P < 0.001), and more had private health insurance (P = 0.04). There were also differences related to country of birth (P < 0.001) and teaching groups (P < 0.001). Other demographic characteristics were statistically similar (See Table A1). The participants who did not complete the program also had significantly higher mean scores on two health literacy domains pre-intervention compared to those who completed the program (see Table A2): Domain 7 (Navigating the healthcare system) and Domain 8 (Ability to find good health information). However, these groups did not differ significantly on the other seven domains.

Health Literacy
The participants' mean pre-and post-intervention HLQ domain scores and the mean domain score changes are presented in Table 2. There were statistically significant (P < 0.001) improvements in mean scores in all nine domains of the HLQ. The effect sizes (Cohen's d) ranged between 0.32 (Domain 4: Social support for health) and 0.58 (Domain 2: Having sufficient information to manage my health). Exploratory multivariable linear regression modelling was conducted (see Table A3) to determine whether the magnitude of change in the HLQ domain scores differed as a function of sociodemographic and health factors. Each HLQ domain score was modelled individually, with all demographic variables shown in Table 1 (except country of birth, due to collinearity with teaching group) included as explanatory variables. The outcomes of these analyses indicated a differential effect of age, gender, living alone, education level, particular chronic health conditions (arthritis, heart problems), and teaching group on the amount of improvement observed across HLQ domains.

Discussion
This is the first study to evaluate the impact of the CDSMP on health literacy using a multi-dimensional measure. Analyses showed that the CDSMP was effective in improving all domains of health literacy among a broad cross-section of participants. Statistically significant improvements (corresponding to small to moderate effect sizes) in all nine health literacy domains were observed; and these changes in reported confidence and skills can be matched to the educational content of the intervention. The largest effect sizes were observed for the domains 'Having sufficient information to manage my health', 'Ability to find good health information', 'Navigating the healthcare system' and 'Appraisal of health information'. The smallest effect size was observed for 'Feeling understood and supported by healthcare providers'. Exploratory analysis indicated that the differences in health literacy scores according to demographic characteristics were domain-specific, providing a complex picture with no clear patterns. However, there were some notable differences, including that the self-reported ability to 'actively manage health' decreased with age; people who lived alone had lower scores on 'feeling understood and supported by healthcare professionals'; and that, in general, increasing education was associated with higher self-reported understanding of health information.
This study adds to the body of evidence supporting the potential effectiveness of self-management programs for patients with chronic disease [17,24,29,30] by showing that programs such as the CDSMP can support the development of sustainable and potentially transferable health literacy skills immediately post-intervention. Importantly, these findings were observed in a highly diverse cohort of traditionally 'hard-to-reach' adults and so provide practical insights into intervention delivery and assessment in this context. While we were able to recruit 486 people into the program using multiple recruitment strategies, 35% did not complete the post-intervention questionnaire and were excluded from the analysis, and those who were included had higher HLQ scores on some domains, and were more likely to have private health insurance. They were, however, less likely speak English at home compared to those who were excluded. High attrition is common in health literacy interventions [36] and it has been acknowledged that work is needed to investigate how to recruit and retain 'hard to reach' participants throughout trials [36]. Related to this, our finding that we were able to retain people who spoke languages other than English at home provides some support for the use of in-language programs in diverse settings.
The findings of this study should be considered in light of the limitations of a single-arm pre-post study design with non-random selection of participants, including being unable to account for concurrent events or programs that could have contributed to the change in participants' health literacy during the intervention period and the possibility of recruiting participants who were highly motivated to improve their health. Given the use of convenience sampling, we do not have any information on the total number of eligible participants, and our sample is not representative of the population in WSLHD or those with chronic disease in Australia. Compared to the total population of Western Sydney Local Health District, for example, our sample had disproportionately more females than males. Although, in Australia, females aged 15 years and over are more likely than males to have one or more chronic conditions, the distribution was more heavily skewed in our sample (62% compared to 36%), compared to population data (57% compared to 51%) [37]. The two main countries of birth (other than Australia) for participants in this study, however, did reflect those of the total population in WSLHD [38]. Future studies applying more rigorous study designs (e.g., matched controls, waitlist-randomised control trial) will help to strengthen evidence for the effectiveness of CDSMP on improving health literacy.
There are also limitations pertaining to measurement instruments used in this study, particularly the use of translated versions of the HLQ. Although the instrument developers note that a "strict protocol is followed for each translation to help ensure each version of the HLQ is linguistically, culturally and psychometrically robust" [39], the validity and reliability of the translated versions of HLQ used in our study have not been reported in a peer-reviewed format. Studies using skills-based (rather than self-report) measures of health literacy may also yield different results. It has recently been argued that self-report measures of health literacy (such as the HLQ) assess participants' confidence in their capacity to comprehend and use health materials, rather than their actual ability, and that, conceptually, feelings of perceived confidence (i.e. self-efficacy) differ from health literacy (i.e. skills) [39]. In line with this, a recent meta-analysis found that health literacy had a positive impact on the self-care activities of people living with diabetes only for studies which assessed health literacy using self-report measures and not for those which used skills-based instruments [39]. Comparing the effect of the CDSMP on skills-based and self-report measures of health literacy, therefore, represents an important direction for future research.
To continue to address questions about the key mechanisms through which chronic disease self-management programs are effective [40], future research should build on the current study by also assessing intermediate and longer-term impacts and outcomes in addition to the short-term impacts on health literacy. Structural equation modelling could then be used to develop a more comprehensive program logic and evaluate whether short term impacts (e.g., changes in health literacy) mediate intermediate impacts (improved self-management) and longer-term outcomes (e.g., improved clinical indices).
Although there is now evidence regarding the effectiveness of chronic disease self-management programs-including on health literacy skills identified in this study-Harris et al., [41] have detailed some of the challenges related to implementing and sustaining programs such as the CDSMP in practice. They highlight the importance of collaboration between organisations to provide and coordinate self-management programs (e.g., state health services; non-government organisations; primary care providers), as well as the need for referral pathways which are sustained over time and facilitate direct communication [41]. The education of staff on the evidence of the effectiveness of these programs was also identified as an enabler of engagement with self-management support [41]. A model for implementation has been provided (by Western Sydney Local Health District, as described here) which addresses many of the potential threats to sustainability, including through the development of care pathways and shared care plans between primary, community and hospital providers, and the use of community-based care facilitators to assist in care planning, navigation, transitional care between services, and patient education and self-management [16]. Implementation research exploring models such as these represents an important direction for future work in this field.

Conclusions
This is the first study to evaluate the potential impact of the CDSMP on improving different domains of health literacy amongst a diverse sample of participants with chronic disease using a multi-dimensional measure. The absence of a control population in this study means that some caution should be applied to interpreting the results.

Conflicts of Interest:
The authors declare no conflict of interest. Table A1. Demographic characteristics of all participants (pre-intervention) (n = 448).

Appendix C
Main effect P-values for each variable in the analysis of covariance are reported in the Appendix C  Table A3. The attribute with a statistical significance indicates its potential of being a covariate for the Health Literacy Questionnaire (HLQ) score change in its corresponding domain. As an individual can have more than one chronic condition, the chronic disease category is not mutually exclusive. Also, the country of birth category was not included in the analysis of covariance due to collinearity with teaching group. questionnaire. a Country of birth was not included in the analysis of covariance due to collinearity with teaching group. b Less than 5 observations stayed within the group.