A Mobile App for Chronic Disease Self-Management for Individuals with Low Health Literacy: A Multisite Randomized Controlled Clinical Trial

Abstract

The purpose of this study was to evaluate the effects of a mobile app designed to improve chronic disease self-management in patients aged 40 years and older with low health literacy and who had at least one chronic health condition, and to assess the impact of delivering information at different levels of reading difficulty. A randomized controlled trial was completed at two sites. Individuals aged 40 years and older screened for low health literacy who had at least one chronic health condition were randomly assigned to a tailored information multimedia app with text at one of three grade levels. Four primary outcomes were assessed: patient activation, chronic disease self-efficacy, health-related quality of life, and medication adherence. All groups showed overall increases in activation, self-efficacy, and health-related quality of life, but no change in medication adherence. No between-group differences were observed. The mobile app may have been effective in increasing participants’ levels of several psychosocial variables, but this interpretation can only be advanced tentatively in light of the lack of control-experimental group differences. Reading difficulty level was not significantly related to outcomes.

1. Introduction

Health literacy is the degree to which individuals have the ability to find, understand, and use information and services to inform health-related decisions and actions for themselves and others [1]. It is related to health status and health outcomes across a wide range of contexts and health conditions. The 2003 United States (US) National Assessment of Adult Literacy showed that more than 75 million Americans had only basic health literacy skills, indicating that one in four Americans have problems understanding information about healthcare [2]. More recent studies of literacy and numeracy skills in the general population suggest that the situation has not changed [3,4]. Research also shows that the problem is not limited to the US, with similar findings in Canada, Europe, and Africa [5,6,7].

Further, health literacy is lower in persons from racial and ethnic minoritized groups, as well as in persons 40 years of age and older [2,8] and may be an important factor in health disparities [9,10]. In the US, 24% of Black people (9.5 million) and 41% of Hispanic people (21 million) have below-basic levels of health literacy [2]. Members of minoritized groups have lower levels of health literacy and compelling evidence links race and ethnicity to disparities in health via health literacy [11,12,13,14,15,16,17,18]. Members of racially minoritized groups and persons in middle age or older are also more frequently affected by chronic diseases such as cancer, high blood pressure, heart attack, stroke, diabetes, asthma, hepatitis, HIV infection, mental health disorders and many others. The twin burdens of chronic disease and low levels of health literacy thus fall disproportionately on those most in need—members of minorities and persons in middle age or older, all of whom may experience one or more chronic conditions while not having the health literacy skills they need to cope [9,19]. Interventions to improve health literacy are thus clearly needed.

Providing information for patients in clinical settings on self-management of their health conditions may be a useful strategy for addressing low health literacy. In the traditional methods that use pamphlets or handouts supported by conversation in a brief clinical encounter, however, information is often not actually read or remembered [20,21], and recommendations may not be implemented among people diagnosed with chronic conditions [22,23].

One strategy to increase the impact of such information on patient behavior is tailoring. Tailoring information, defined as using various methods to create individualized communications for patients [24], aims to reduce the burden of self-management on health consumers by giving them useful information that is relevant to their needs or concerns and that they can understand and use. Various information interventions have been developed to improve patients’ health literacy [12,25,26], including matching health education content to patient characteristics and tailoring health messaging to make it more directly relevant to patients. While these techniques have often been successful [25,27], creating individually tailored health information is labor-intensive and thus may not be widely available.

A possible solution to the problem of giving patients the information they want and need in a form they can use has been the development of computer-based interventions to automate message tailoring [28,29]. Computer-based tailoring creates the possibility that high-quality, individualized health information can be made available to those who need it. Information can be delivered to patients when they want it, and it can target content they are interested in, a process of providing precision health information [30]. Analogous to the processes underlying the precision medicine approach to the somatic treatment of diseases [31], computer-based tailoring can take a patient’s personal characteristics, including their expressed concerns or problems, and provide detailed information to help them understand their health conditions and develop self-management skills.

A potentially critical variable in the tailoring process is ensuring that the content is appropriate to the patient’s level of health literacy. If information is not understood, even if tailored, it may not impact patients’ behavior. Studies show that while experts recommend that materials for patients be written at a 4th or 6th grade level and match the patients’ level of health literacy [32,33,34], multiple studies show that most patient education materials are at much more difficult levels [35,36,37,38,39].

Similarly, while interventions have been developed to improve health literacy [40,41] they are difficult to scale to the levels needed to meet the challenge of low health literacy (millions of people worldwide) due to their cost [29,41,42]. Effective interventions with the potential for wider dissemination at reasonable costs are urgently needed. For many of the problems that are the focus of CDSM, well-defined behavioral strategies exist for their management (e.g., cognitive behavioral therapy for sleep [43] or mood problems [44]) holding the possibility that effective tailoring might help patients develop relevant knowledge and behavioral skills.

Assessments of CDSM programs have focused on several outcomes, and in this project, we focused on four that we judged were most relevant to the effects of a CDSM-targeting intervention and that would allow us to compare our results with those of other researchers. Patient activation [45] is defined as the extent to which they are actively involved in their healthcare, and has been related to a number of important health status variables, including emergency department visits, receiving breast cancer screening, as well as laboratory measures such as hemoglobin A1C (related to diabetes) and HDL (high-density lipoprotein) related to cardiovascular disease risk [46]. Activation has also been associated with self-management behaviors [47] as well as quality of life [48], and studies show that interventions that improve patient skills can increase activation [49]. Finally, activation is related to self-management behaviors [50], and changes in activation are related to changes in health outcomes [51] and healthcare costs [52].

Self-efficacy, or a person’s belief in his or her capacity to reach specific goals [53], is a key concept in understanding health behavior [54], especially in relation to patients’ health behavior [54] and in CDSM [55]. In-person and internet-delivered CDSM programs have a positive impact on self-efficacy [56,57] which in turn has been related to self-management [58,59,60]. Another highly relevant potential outcome of improved CDSM skills is improved health-related quality of life (HRQOL). Self-reported health status has often been studied as an outcome in CDSM studies [57,61,62], and is clearly an important aspect of improved self-management skills. Finally, medication adherence is an essential aspect of self-management behavior. Poor medication adherence is common in patient populations, with estimates of adherence ranging from 55 [63] to 75% [64]. Low levels of adherence have been linked to numerous adverse health outcomes [65], including increased healthcare costs [66], and studied as an outcome in other studies of CDSM [62,63].

The objective of this study was to assess whether a mobile app for CDSM providing individually tailored health information would have a positive impact on participants’ activation, self-efficacy, quality of life, and medication adherence. We chose CDSM because we believed it was a logical target for a health literacy intervention. In an approach that cuts across specific diseases, CDSM targets problems and skills needed to cope with issues such as fatigue, pain, stress, depression, sleep disturbance, and treatment adherence. Studies show that in-person CDSM classes improve patients’ functioning and reduce healthcare utilization [55,56,57,61,62,67,68], but their availability is limited due to a lack of qualified personnel, cost, and accessibility. It was also hypothesized that information presented at reading levels consistent with expert recommendations (3rd to 6th grade levels) would have a greater impact on these variables than information presented at an 8th grade level.

2. Materials and Methods

2.1. Development

In this project, we worked to address issues of providing health information to individuals with low levels of health literacy by developing a tailored information app focused on chronic disease self-management (CDSM) [61] with three versions: one with text at 8th grade level (a control condition), a second at 6th grade (an experimental group), and a third at 3rd grade level supported by audio narration (a second experimental group) [69].

Initial guidance on app content was drawn from a review of existing sources on CDSM supplemented by a qualitative study that explored patients with chronic health conditions needs for information about managing their conditions [70]. A multidisciplinary team comprising representatives from medicine, nursing, psychology, pharmacy, public health, and education was assembled to develop content. Team members with special expertise in developing culturally and ethnically appropriate education materials were included as well (K.T-P., K.K.).

App content was developed by individual team members and reviewed by the team for appropriateness of content and format as well as for the usefulness and relevance of graphic content. Once drafts of the modules were available, they were subjected to usability testing (including asking for comments on the module content as well as the format) with groups of potential users with low levels of health literacy, asking them not only for feedback on the interface and ease of use of the app but also on app content. Modules were revised and retested as needed. The app was conceived as a series of topical modules that would consist of a series of screens within the app. Each module included screens conveying an orientation to its purpose, assessment of the participants’ current status by way of questions, general health information on each topic, individually tailored content, and a summary. Self-test questions were included to help participants understand how well they learned the module’s contents.

Information was presented as text on a series of screens, supplemented by pictures, graphics, and narrated animations consistent with the principles of multimedia learning [71]. An outline of each module’s contents is provided in Supplementary Table S1, and example screenshots pages that included suggestions about self-care through improving diet is presented and suggestions about strategies for working with the participant’s doctor are included in Supplementary Figures S1 and S2. Additional information on the development of the modules and user experience in working with them has been reported in [72]. Modules with the same content were created at three levels of reading difficulty based on the Fry [73] and Flesch Reading Ease [74,75] scores of the text they contained (3rd grade, with text narrated; 6th and 8th) using Health Literacy Advisor^® (Bethesda, MD, USA: Health Literacy Innovations LLC), a software plugin working with Microsoft Word 2017^®.

2.2. Participants

Participants were recruited from participants in previous studies, local health clinics and medical practices, and by word of mouth. At the Atlanta site, a paid recruiter visited local churches where she could screen potential participants as well as give them information about participation. Information on race and ethnicity was collected as required by the U.S. National Institutes of Health for grant recipients [76]. Race and ethnicity were self-reported by participants. Gender was also self-reported, with transgender participants considered as the gender of their chosen identity.

2.3. Screening

Participants were initially screened to determine their potential eligibility using a brief interview that elicited medical history, medication use, and education. They were administered a short form of the Rapid Estimate of Adult Literacy in Medicine (REALM) [77] using a previously validated cut-off for health literacy at or below the 8th grade level [78].

2.4. Inclusion and Exclusion Criteria

To be eligible to participate in the study, participants were required to be 40 years of age and older, have at least one chronic health condition for which they were currently treated, have an education level less than 16 (i.e., not be a college graduate), and score below the cut-off score on the short form of the REALM. Persons 40 years of age and older were included because of the increased prevalence of chronic health conditions in people in this age range [79,80]. The criterion of having less than a college education was derived from our findings in a previous study [30] that no participants who had successfully completed a college degree had inadequate health literacy. We did not include specific criteria for level of technological skill or diversity with respect to race or ethnicity. It should be noted that while participants were assessed to have low levels of health literacy on a standard measure, they may have had some disease-specific knowledge or skills as evidenced by the fact that they had been diagnosed with a health condition.

In this study, we chose to define chronic health conditions broadly as we wished to understand not only self-management but also to take into account the observation that the most common chronic condition is multimorbidity [81]. We thus chose to cast a wide net to include conditions with varying levels of severity and likely impact on participants’ daily lives. We included all the conditions listed in

Table 1. Health conditions for which participants could be considered eligible.

Arthritis, rheumatoid or osteoarthritis *, 1

Osteoporosis *

Asthma *

Chronic obstructive pulmonary disease (COPD) *

Angina *

Congestive heart failure (CHF) *

Heart attack *

Hypertension

Dyslipidemia

Multiple sclerosis; treated with oral or injectable medications * (in FCI 2 as part of neurological conditions)

Parkinson’s disease * (in FCI 2 as part of neurological conditions)

Migraine, if occurring at least 3 times per month and treated

Dementia (including mild cognitive impairment if diagnosed and treated)

Seizure disorder

Head injury; eligible if taking medications for pain or seizure prophylaxis

Cerebrovascular accident or transient ischemic attacks

Peripheral vascular disease *

Diabetes I or II *

Ulcer * (in FCI 2 as part of gastrointestinal [GI] conditions)

Hernia * (in FCI 2 as part of GI conditions)

Gastroesophageal reflux disease * (in FCI 2 as part of GI conditions)

Depression *

Anxiety or panic disorder *

Bipolar I or II

Psychotic disorder

Psychosis in the context of mood disorder

Cataracts

Glaucoma

Macular degeneration

Vision impairment (screening procedure described in text) *

Hearing impairment, check whether using amplification *

Back pain, including degenerative disc disease *

Obesity *

HIV/AIDS

Hepatitis, if taking medication

Chronic fatigue syndrome/myalgic encephalomyelitis

Substance abuse

Notes: ¹ Conditions listed as part of the Functional Comorbidity Index [82] are marked with an asterisk; ² Functional Comorbidity Index [82].

, with the additional criterion that participants’ conditions had to be treated with at least one medication prescribed (or recommended in the case of over-the-counter medications) by a healthcare provider. The list was based on the conditions included in the Functional Comorbidity Index [82], as it focused on the relation of health conditions to physical function, but was expanded to include a number of conditions requiring treatment that were commonly used in US Medicare reporting [83] and other multimorbidity indexes [84], such as hypertension and dyslipidemia. We also expanded the number of mental health conditions to include psychotic and bipolar disorders, as well as HIV/AIDS and chronic fatigue syndrome/myalgic encephalomyelitis. This left what we judged to be a useful balance between a detailed assessment of patients’ status, broad inclusion criteria to support generalizability of findings, and participant assessment burden.

2.5. Measures

Participants completed an extensive battery of measures as part of a baseline assessment, with self-reported outcome measures administered immediately after completing the intervention and then three months later. Most self-report measures were administered by computer, with questions read aloud by the interviewing software to minimize the impact of participants’ reading skills on their ability to respond to questions. At the baseline visit, participant demographic information, level of education, and medical history were assessed in an individual interview. As noted above, the medical history interview was based on the medical conditions comprising the Functional Comorbidity Scale [82,83] but expanded to include additional health conditions common in middle-aged and older adults [83]. Self-report measures were administered via audio computer-assisted self-interview software (Bethesda, MD, USA: Questionnaire Development System version 4.0B) that read all questions aloud to participants to keep the effect of reading ability on participant responses to a minimum.

In order to provide a standardized assessment of participants’ reading skills, trained assessors individually administered the Woodcock-Johnson Psycho-Educational Battery [85] Passage Comprehension subtest. This measure provides a grade-equivalent score that helps characterize participants’ reading levels. The FLIGHT/VIDAS health literacy scale was used in this study because of its desirable psychometric characteristics [86] that include a wide range of scores; other health literacy measures often have ceiling effects that reduce the range of observed scores that make them less useful in statistical analyses [87].

2.6. Outcomes

The four outcomes we studied were assessed with widely used measures of each construct. Activation was evaluated with the Patient Activation Measure [45]. We used the ten-item version based on the 13-item short form that has been shown to have good validity and reliability [88,89]. Higher scores are associated with changes in patients’ engagement in medical treatment and belief in their self-management skills [49]. Self-efficacy was evaluated with the Chronic Disease Self-Efficacy Scale, used in multiple studies of CDSM programs. It has also demonstrated reliability and validity [90,91]. Higher scores are associated with increases in patients’ perceptions of their ability to lead a regular life in the face of the challenges of a chronic health condition and are inversely correlated with health distress, illness intrusiveness, and activity limitations [91]. The Medical Outcomes Study, Short Form 36 General Health subscale (MOS SF36 [92]), is one of the most widely used measures for understanding psychosocial functioning related to health. It has well-established reliability and validity for use with persons with chronic health conditions [93,94] and is related to both the physical and emotional domains assessed by the SF36 [95]. Higher scores indicate higher levels of perceived health, as higher scores mean that participants have responded more positively when asked to rate their health overall. In this study, it is used as a measure of HRQOL.

The Gonzalez-Lu questions were included as a measure of adherence because they have been validated against electronically recorded medication adherence and for their simplicity [96,97]. As the four questions were highly intercorrelated, we reduced them to single score via principal axis factor analysis and used resulting factor scores as a single measure. This approach avoids the limitations of simply summing a group of items [98] and has good psychometric properties [99]. Scores on this summary measure expressed as standard (z scores) have a mean of zero, with more positive scores indicating higher levels of self-reported adherence.

2.7. Procedures

After initial screening, potential participants were scheduled for an in-person eligibility visit when, after obtaining verbal consent, they completed measures of health literacy, reading comprehension, a hearing and vision screening, and a medical history interview to determine their eligibility. Participants’ visual acuity was assessed with the Rosenbaum Pocket Visual Screener [100], while auditory acuity was assessed by evaluating the participant’s ability to correctly respond to open-ended sentence completion problems from the Woodcock-Johnson Psycho-Educational Battery [85] heard through the computer audio, such as “A bird swims, a fish …?”. Participants were required to respond to four items to confirm their ability to understand the audio included in the modules.

Eligible persons were then scheduled for a baseline visit, during which they completed self-report measures and some individually administered measures of academic skills. At the first intervention visit, participants were randomly assigned by research assistants to one of the three groups (3rd [intervention], 6th [intervention], or 8th [control] grade reading levels) and returned for the intervention visits. Randomization was conducted by permuted blocks of 3 and 6 with a list generated by an online resource (London, UK: Sealed Envelope, Ltd.). Intervention visits occurred over two to three weeks, with a maximum of two sessions per week during which the participants worked through the CDSM modules for a total of three sessions. During intervention sessions, participants worked through the modules on tablet computers (Microsoft Surface Pros^®; Redmond, WA, USA: Microsoft, Inc.), as preliminary work suggested that many of them would have difficulty interacting with the modules on smaller screens.

In the first session, participants reviewed an introductory module that explained the purpose of the information, an adherence module that emphasized not only strategies for treatment adherence but also how to work with health care professionals, and a module on stress, its effects, and management techniques. In the second session, participants reviewed modules on sleep, mood, pain, and memory. Finally, in the third session, they worked with modules on fatigue, shortness of breath, and anger.

After completing the modules, participants returned within several days for the first follow-up visit, during which they again responded to self-report measures and completed an individual semi-structured interview that elicited their reactions to the modules and the extent to which they had adopted any of the recommendations they contained. Three months later, participants returned for a second follow-up visit, during which they again responded to self-report measures and completed the same semi-structured interview.

2.8. Human Subjects Approval

All study procedures were completed under protocols approved by the Nova Southeastern University Institutional Review Board (2018-685-NSU) and the Emory University Institutional Review Board (MODCR001-IRB00087112). All participants provided verbal consent for screening and written informed consent for all other study procedures.

2.9. Statistical Analyses

Planned analyses assessed the study hypotheses that persons receiving the intervention would (1) show significant increases in measures of activation, chronic disease self-efficacy, health-related quality of life and medication adherence and (2) that persons receiving the information at 6th and 3rd grade levels (experimental conditions) would show greater change than those receiving the material at the 8th grade level (control condition).

A preliminary review of the data and descriptive measures were obtained using SPSS version 28 (Armonk, NY, USA: IBM). Mixed effects random intercept models were evaluated with the statistical program R, version 4.2.1 [101] using the lme4 package version 1.1-34 [102]. All models included participant age, gender, race, and site of data collection, as well as time and treatment group and their interaction. Tests of study hypotheses were completed after maximum likelihood estimation using Satterthwaite approximations of p values [103]. Tests of the statistical significance of within- and between-group differences were obtained using the emmeans 1.8.7 package with Tukey corrections for multiple comparisons [104].

Based on the evaluation of between-group differences at the two sites as well as theoretical considerations, model covariates were chosen to control for likely confounders as well as observed between-site differences. They were chosen based on considerations of likely confounding impacts on outcome measures (age, gender, race, and education) and observed between-site differences in level of health literacy and level of multimorbidity (FLIGHT/VIDAS health literacy scale and number of health conditions). In addition, because of observed differences in participant characteristics at the two sites, the site itself was included as a covariate.

2.9.1. Sample Size

Target sample size was determined during the planning phase of the project using the mixed effects model simulation routine in PASS 16 [105], which showed that a minimum sample size of 30 per group would provide a power greater than 0.90 to evaluate study hypotheses as the interaction of treatment groups with time. Effect sizes for the analysis were based on previous observations of the effects of a similar intervention [106].

2.9.2. Evaluation of Minimal Clinically Important Difference (MCID)

Although in the original protocol of the study we only planned to evaluate between and within group treatment effects based on their statistical significance alone, as post hoc assessments of treatment effects we calculated effect sizes as Cohen’s d statistic [107]. We therefore provide the d statistic based on the chi-square-based likelihood ratio test, evaluating models of increasing complexity by adding treatment effects and testing the differences between models. The resulting chi-square value was then converted to a more familiar d statistic using the esc version 0.5.1 package in R [108], facilitating comparisons with other studies evaluating chronic disease self-management interventions using this effect size measure. We also provide an interpretation of the obtained effect sizes in the context of the minimal clinically important difference (MCID) to facilitate the interpretation of study results. Although direct estimates of the MCID for the outcomes were not available for all instruments, we followed the practice of others in viewing effect sizes of 0.20 to 0.50 (small to medium effects [109]) as clinically meaningful. In this report, effect sizes greater than 0.20 are interpreted as potentially clinically meaningful, while those greater than or equal to 0.50 are interpreted as meeting a more rigorous criterion and being more likely to represent an MCID [109,110].

3. Results

Figure 1 presents the CONSORT diagram for participant flow during the trial for both sites combined, and

Table 2. Description of participants completing at least one intervention session.

Variable Count	Atlanta	Fort Lauderdale	Total	Χ2	df	p	Effect Size (d)
Men	65	79	144
Women	118	47	165	22.15	1	<0.001	0.56
White	7	34	41
Nonwhite	176	92	268	37.78	1	<0.001	0.71
Variable Mean (SD)				t	df	p	ES
Age in Years	58.10 (8.61)	56.95 (8.09)	57.63 (8.41)	1.18	307	0.24	0.14
Education Years	12.02 (1.76)	11.64 (1.97)	11.86 (1.85)	1.75	307	0.08	0.20
Total Number of Health Conditions	5.92 (2.66)	7.26 (2.77)	6.47 (2.78)	4.28	307	<0.001	0.50
WJ Reading Grade 1	6.60 (4.10)	7.74 (3.97)	7.06 (4.08)	2.44	307	0.02	0.28
Flight/Vidas Health Literacy 1	9.59 (3.92)	10.97 (4.09)	10.17 (4.04)	2.89	307	0.004	0.35
PAM Score 1	61.37 (16.03)	61.94 (15.96)	61.61 (15.97)	0.30	291 2	0.76	0.04
CDSE Mean 1	6.96 (2.01)	6.61 (1.93)	6.82 (1.98)	1.46	290 2	0.15	0.18
HRQOL (SF General Health) 1	60.21 (19.38)	60.16 (20.10)	60.19 (19.66)	0.02	288 2	0.98	0.003
Gonzalez Lu Factor Score 1	−0.09 (1.07)	0.14 (1.02)	0.01 (1.06)	1.89	291 2	0.06	0.22

¹ WJ Reading Grade = Woodcock-Johnson Psycho-Educational Battery Passage Comprehension subtest grade equivalent score; FV Health Literacy = FLIGHT-VIDAS Health Literacy Measure; PAM score = Patient Activation Measure; CDSE = Chronic Disease Self-Efficacy Scale; SF General Health = Medical Outcomes Studies, Short Form-36 General Health subscale; Gonzalez-Lu = Self-report of medication adherence factor score (see text); GL = Gonzalez-Lu questions 1–4. ² Degrees of freedom vary due to data loss caused by equipment failure.

presents descriptive data for participants who completed at least one intervention session at each site and for both groups overall. Study activities began on 20 September 2018 at the Fort Lauderdale site and on 20 October 2018 in Atlanta. All study activities concluded in November, 2020. The study concluded at the end of the period of financial support provided by a government grant. Both participant gender and race were differently distributed at the two sites, with relatively more male and white participants at the Fort Lauderdale site. The two groups of participants did not vary in three of the outcome measures, although Fort Lauderdale participants reported slightly greater medication adherence, although the comparison was not significant.

While 334 participants completed the baseline visit, only 315 actually completed at least the first intervention visit. Reasons for their loss included that they did not appear for the first scheduled visit and could not be contacted (n = 9), they did not comply with study procedures (n = 4), they were caregivers who could not be away from the person for whom they cared (n = 3), and their own illness (n = 3).

The random intercept model for the Patient Activation Measure is presented in

Table 3. Model for the Patient Activation Measure.

	Sum of Squares	Mean Squares	Numerator df	Denominator df	F	p
Age	601.86	601.86	1	281.16	4.90	0.03 1
Gender	394.9	394.9	1	274.52	3.21	0.07
Race	82.26	82.26	1	276.27	0.67	0.41
Education	143.46	143.46	1	270.92	1.17	0.28
Health Literacy	309.17	309.17	1	276.78	2.51	0.11
Health Conditions	189.27	189.27	1	281.6	1.54	0.22
Site	17.73	17.73	1	271.62	0.14	0.70
Time	822.22	411.11	2	482.77	3.34	0.04 1
Group	159	79.5	2	276.84	0.65	0.52
Time x Group	92.61	23.15	4	482.3	0.19	0.94

¹ p < 0.05.

, with model-based means for each group at each time displayed in Figure 2. While the interaction of the treatment group with time was not statistically significant, there was a significant effect for time, with all groups showing increases in activation after the intervention. Although level of activation appears to continue to increase between the first and second follow-up visits for the 3rd grade group, the difference between this group’s activation and the other groups was not significant (all p > 0.50) and the within-subjects difference also was not significant (p > 0.50). Calculation of treatment effect size through the likelihood ratio test of models with and without the effect of time resulted in significant chi square (χ² [2] = 10.27, p = 0.006) for the effect of time, associated with a d value of 0.37 (95th CI 0.14–0.60), an intermediate effect size greater than a small effect of 0.20 but not reaching the more rigorous criterion of 0.50.

The model for the Chronic Disease Self-Efficacy Scale is presented in

Table 4. Model for the Chronic Disease Self-Efficacy Scale.

	Sum of Squares	Mean Squares	Numerator df	Denominator df	F	p
Age	1.20	1.17	1	291	0.90	0.34
Gender	1.30	1.31	1	285	1.01	0.32
Race	0.10	0.13	1	297	0.10	0.75
Education	14.60	14.58	1	279	11.24	<0.001 1
Health Literacy	14.10	14.10	1	285	10.88	0.0011 1
Health Conditions	19.30	19.28	1	286	14.87	<0.001 1
Site	0.60	0.62	1	287	0.48	0.49
Time	32.20	16.08	2	470	12.40	<0.001 1
Group	3.80	1.92	2	287	1.48	0.23
Time x Group	7.60	1.91	4	470	1.47	0.21

¹ p < 0.05

Education, health literacy, and total number of health conditions, as well as time were related to this outcome. Model-derived means are plotted in Figure 3. Although the level of self-efficacy appears to decline for the 8th grade group at second follow-up, the difference between the 8th and 3rd grade groups (which appears to increase) was not significant (p = 0.14). The calculation of treatment effect size through the likelihood ratio test resulted in significant chi square (χ2 [2] = 23.64, p < 0.001) for the effect of time, associated with a d value of 0.58 (95th CI 0.34–0.81), meeting the more rigorous criterion of 0.50 for an MCID [111].

The model for health-related quality of life (HRQOL assessed with the SF-36 General Health scale) is presented in

Table 5. Model for MOS SF-36 General Health scale.

	Sum of Squares	Mean Squares	Numerator df	Denominator df	F	p
Age	46	46	1	287	0.47	0.50
Gender	250	250	1	276	2.52	0.11
Race	13	13	1	276	0.13	0.72
Education	1139	1139	1	274	11.51	<0.001 1
Health Literacy	259	259	1	277	2.62	0.11
Health Conditions	1739	1739	1	280	17.57	<0.001 1
Site	172	172	1	274	1.74	0.19
Time	1558	779	2	474	7.87	<0.001 1
Group	272	136	2	277	1.37	0.26
Time x Group	355	89	4	474	0.9	0.47

¹ p < 0.05.

and model-corrected means are presented in Figure 4. In this model, in addition to the effect of time, participants’ HRQOL was positively related to their level of education and inversely related to the number of health conditions they reported. Inspection of Figure 4 shows that the most pronounced effect on this outcome measure was observed at the three-month follow-up. Within group analyses showed that the 8th grade group improved significantly from baseline to the second follow-up (t [486] = 2.94, p = 0.01), while the 3rd grade group improved significantly only between the first and second follow-up (t [486] = 2.66, p = 0.02). Although the 6th grade group appears to have improved from the first to the second follow-up, this change was not significant (t [475] = 1.61, p = 0.24). Calculation of treatment effect size through the likelihood ratio test of models resulted in a significant chi square (χ2 [2] = 15.62, p < 0.001) for time, associated with a d value of 0.44 (95th CI 0.22–0.66), greater than 0.20 but not reaching the more rigorous criterion of 0.50.

Finally, the model for self-reported medication adherence (Gonzalez-Lu factor score) is presented in

Table 6. Model for Gonzalez-Lu adherence score.

	Sum of Squares	Mean Squares	Numerator df	Denominator df	F	p
Age	1.12	1.119	1	282	3.19	0.07
Gender	1.35	1.355	1	274	3.87	0.0503
Race	0.63	0.633	1	274	1.81	0.18
Education	0.01	0.008	1	270	0.02	0.88
Health Literacy	1.61	1.609	1	275	4.59	0.03 1
Health Conditions	0.94	0.94	1	280	2.68	0.10
Site	0.79	0.786	1	272	2.24	0.14
Time	0.28	0.138	2	476	0.39	0.67
Group	5.73	2.864	2	276	8.18	<0.001 1
Time x Group	3.09	0.772	4	476	2.2	0.07

¹ p < 0.05.

and model-corrected means are presented in Figure 5. For this outcome, no overall effect of time was observed, nor was there an interaction of time with the treatment group. Both the 8th grade and 3rd grade groups reported significantly greater adherence at baseline (t [450] = 3.66, p < 0.001) and (t [456] = 3.67, p < 0.001), respectively. These group differences were maintained at the first follow-up, but only the 8th grade group was still significantly different from the 6th grade group at the second follow-up (t [516] = 2.52, p = 0.03). Although the 6th grade group appeared to improve adherence between the first and second follow-ups, this improvement was not significant (t [477] = 2.07, p = 0.10). Calculation of treatment effect size through the likelihood ratio test resulted in a non-significant chi square (χ2 [2] = 0.93, p = 0.63) for time, associated with a d value of 0.11 (95th CI −0.11–0.33), a small effect size that did not meet either criterion for an MCID [111].

4. Discussion

The purpose of this study was to investigate the impact of a tailored information app for CDSM on people 40 years of age and older with chronic health conditions and low health literacy. The study focused on individuals 40 years of age and older because of the greater prevalence of chronic health conditions in people in this age range. We hypothesized that the app would have a positive effect on four outcomes that reflect patients’ attitudes and beliefs about their ability to manage their health: activation, self-efficacy, HRQOL, and medication adherence. Results partially support this hypothesis, as we saw a significant positive impact of the app on three of the outcomes, activation, self-efficacy, and health-related quality of life that represented small to medium effect sizes consistent with MCID values from other studies. We did not find a significant change in participants’ self-report of medication adherence, and it was not associated with an MCID. However, this interpretation must be advanced tentatively, as although changes in the outcome measures were observed over time, differences between experimental (3rd and 6th grade reading difficulty) and control conditions (8th grade reading level) were not observed, making it difficult to determine whether the changes observed were the result of time or participants’ interaction with the app.

Outcome measures for this study were chosen because of their relevance to middle-aged and older person’s ability to manage their health and their use in previous studies, allowing us to compare the results of the automated app with those of similar in-person and internet-delivered CDSM interventions. Patient activation was a key outcome and has been assessed in other studies in relation to disease self-management [112,113]. The effect sizes obtained in the present study are similar to those found in other studies of behavioral interventions to improve activation. In a meta-analysis of intervention studies aiming to improve the activation of patients with chronic diseases, for example, Lin found average effect sizes of 0.33 for activation and 0.57 for self-efficacy, similar to our findings [112]. Other researchers, for example, have also found that CDSM programs can have a positive impact on self-efficacy for disease management [56,61,62] when compared to relevant control conditions.

A number of studies of behavioral interventions, however, have included HRQOL as an outcome, frequently finding positive effects of a CDSM intervention, including a trial of delivering it via the internet [57]. Improved health-related quality of life was observed in this study as well, associated with a moderate effect size (0.50) that has been described as clinically meaningful for HRQOL outcomes in other areas [113].

In a similar vein, Ritter and Lorig [91], in a review of seven chronic disease self-management programs offered in small groups or by the Internet, report effect sizes ranging from 0.18 (a value that approaches the minimum effect size often considered to represent a MCID) to 0.61, a moderately large effect size that is larger than the stricter criterion for MCID of 0.50. They also report effect sizes from both in-person and Internet-delivered studies, with an average effect size of 0.41 for in-person and 0.28 for Internet studies [92] (Table 5, p. 1270). The effect sizes obtained in our study thus compare favorably with those in other studies, which included control conditions consistent with an effect of the app on patient self-reports.

These results are thus similar to those observed in studies of in-person CDSM programs that have been shown to have a positive impact on activation [111], self-efficacy, and health-related quality of life. This suggests that some of the drawbacks of providing in-person CDSM interventions (cost, lack of trained personnel, accessibility) may be addressed by providing patients access to CDSM as a digital therapeutic. While it is true that the development of the mobile app was expensive initially, after initial deployment, this sort of app can be inexpensive to maintain while providing access to large numbers of patients. We [114] and others [115] have shown that initial development and ongoing deployment costs can be substantially offset by benefits such as improved self-management behavior. Lindsay et al. [52], for example, showed that a substantial increase in activation was associated with lower follow-up costs, especially in high-risk populations.

Given the common critique of applications that do not provide patients information at the 3rd to 6th grade level, it is not clear why we did not find an effect of the modules’ text difficulty favoring the two experimental groups (3rd and 6th grade text) over the control (8th grade text). A possible explanation comes from research on educational applications in other situations in which researchers found a “contiguity effect” [116] in which learners presented text and graphics in close proximity, as was done in this study, resulting in superior learning compared to a condition in which graphic and text elements were separated. In more recent studies, research has shown that the use of multimedia as an adjunct to traditional instruction may enhance comprehension and learner motivation [117]. Other research has shown that incorporating information technology in interventions may enhance patient engagement [118], an effect that was not controlled in our research design. Future research may help clarify whether multimedia instruction can help persons with low health literacy even when the text of the information presented is too difficult when presented as text alone.

In recruitment efforts for this study, we relied in part on several community organizations providing service for persons with low educational and economic resources. In the future, it might be possible to enlist their aid in supporting further deployment of the app, perhaps even including efforts to further develop interpersonal support and to crowdsource information about the problems of these individuals and the solutions they may have developed.

Limitations of this study include the nature of the sample, which was primarily Black, Indigenous, or Other People of Color (BIPOC), and differences between the two study sites on some variables. While we included variables on which the sites differed as covariates in statistical models as well as using site as an additional covariate, it is possible that the observed differences may have affected the study’s outcomes. On the other hand, in the search for persons with low health literacy, we succeeded in recruiting a large number of participants with health literacy skills that might place them at a disadvantage in managing their health, and the study shows that the intervention was successful in increasing activation, self-efficacy, and HRQOL. The lack of between-site differences on outcome variables also supports the usefulness of the intervention, as it appears to have been efficacious in two diverse settings. A significant limitation is the lack of a finding on control vs. experimental group outcomes. Although a lack of differences related to reading difficulty is a concern, the clear effect of time suggests that the intervention had a significant impact on participants. Even without significant between-group differences, the alternative that participants improved in activation, self-efficacy, and HRQOL spontaneously over time is implausible. Additional research on possible mediators and moderators of change in these measures would be useful in further understanding these findings and enhancing the efficacy of the app. For example, the recruitment, screening, assessment, and intervention processes provided substantial opportunities for personal interaction that may have affected outcomes. In the future, it might be useful to clarify the extent to which intervention effects are related to personal interactions and intervention effects, although it should be noted that the effect sizes seen in the present study are similar to those of other studies that provided varying degrees of interpersonal contact. The fact that the intervention has only been tested with these groups is another limitation that may affect the generalizability of our findings. It should also be noted that although participants were assessed as having low health literacy on a standard measure, they had demonstrated some health-related skills, as demonstrated by their having been diagnosed with a health condition. Finally, although in the development process we attempted to include potential users at all stages of development, even greater input from them might have resulted in different content for the app. Our focus on the broad effects of the intervention leaves open the question of whether it may have had differential effects related to specific diagnoses, an issue that may be further explored.

It should also be noted that the study included individuals 40 years of age and older because of their increased likelihood of experiencing chronic health conditions. There are important differences between individuals in their forties and those who are much older with respect to health literacy [2,119], history of educational opportunity [120], and use of digital health resources [121]. Further exploration of the effects of age within this group of persons aged 40 years and older may be useful in understanding how to make the app useful to individuals across this age range. This may be especially important given age-related issues in the acceptance and use of eHealth by middle aged and older adults [122,123].

5. Conclusions

In this study, we investigated the effects of a mobile app for CDSM on persons aged 40 years and older with low levels of health literacy and chronic health conditions. Although we hypothesized that modules that presented content at the 8th grade level would be less effective than those at lower levels, this hypothesis was not supported. Changes in self-report measures over time, however, suggest that the app may have had positive effects for all groups. Given the study design, it was not possible to assess the uptake of the app in a real-world situation. Future development of the app will include additional analyses of possible mediators and moderators of its effects to better understand how it works and ultimately have an even greater impact. Given the recent availability of generative artificial intelligence models, we plan to explore the potential to integrate interactive language capabilities into the app. Additional research will also focus on strategies for making the app widely available at little or no cost to users in order to increase uptake.