A Pilot Randomised Control Trial of an Online Acceptance and Commitment Therapy (ACT) Resilience Training Program for People with Multiple Sclerosis

Abstract

Background: This pilot study explored the effectiveness and feasibility of an online version of a group acceptance and commitment therapy (ACT) resilience training intervention for people with multiple sclerosis (PwMS), called e-READY for Multiple Sclerosis (MS). Methods: Fifty-six PwMS were randomized to intervention (n = 31) or waitlist control (WLC) (n = 25). The primary outcome, resilience, and secondary outcomes (quality of life (QoL), distress, psychological flexibility) were assessed at pre- and post-intervention and 12-week follow-up. Results: Intervention participants reported greater pre- to post-intervention improvements in anxiety (d = 0.56) and stress (d = 0.62) than WLC. Gains were maintained at follow-up. Confidence intervals revealed a trend for the intervention group to report greater improvements than WLC across all outcomes. Reliable Change Index data showed that, compared to WLC, there were trends for more intervention participants to evidence clinically significant improvements in physical health QoL. Recruitment response was weak, intervention retention was good, adherence to program progression guidelines was satisfactory, program usability satisfaction was high, and study protocol attrition at post-intervention and follow-up was low and high, respectively. Most participants viewed the intervention as enjoyable, helpful, and resilience-building, and would recommend it to other PwMS. Qualitative feedback validated the usefulness of intervention tools and digital delivery mode and bolstered resilience through improved ACT-related skills. Conclusions: Effectiveness and feasibility results from this proof-of-concept study provide preliminary support for the e-READY for MS program.

1. Introduction

Multiple sclerosis (MS) is one of the most common acquired neurological diseases. It is a complex degenerative illness that involves demyelination of the central nervous system nerve fibres. People with MS (PwMS) experience lower quality of life (QoL) [1] and are more likely to report clinically significant psychological distress than people in the general population [2]. In particular, prevalence rates of depression (30.5%) and anxiety (22.1%) are high [2]. Psychological distress is often intensified by the demands of adjusting to the illness whilst simultaneously having the potential to exacerbate some MS symptoms [3]. Although QoL and distress in PwMS are important intervention targets, few psychological programs in the MS field have also focused on enhancing resilience, which is an important resource in successfully adapting to negative life events.

Resilience is the ability to “bounce back” and successfully adapt to stressful life events. It involves a process of negotiation and adaptation to stress or trauma that is influenced by an individual’s internal resources (e.g., mindfulness, acceptance, cognitive flexibility, and active coping) and external resources (e.g., social support, financial capital, and community services) [4].

Resilience is important for healthy aging, sustained QoL, and flexible adaptation to the fluctuating demands of MS over the long haul of the illness [5]. Indeed, resilience has been linked to better patient-reported psychosocial and mental health outcomes [5,6,7,8] and motor functioning [9] and has been shown to ameliorate the adverse effects of fatigue and pain on QoL in PwMS [10]. Promoting resilience in PwMS is particularly important given that they report lower resilience than community samples and people with other chronic illnesses [10,11].

To our knowledge, three resilience training interventions for PwMS have been published. One was based on positive psychology, delivered via teleconference to older adults (≥45 years) with MS and evaluated in a two-group pilot randomised controlled trial (RCT) [12]. The intervention group evidenced greater improvements in resilience and satisfaction with social roles than wait-list control (WLC) participants. A second intervention was informed by a purpose-built resilience framework and delivered by teleconference to PwMS and their carers [13]. A single-arm feasibility study showed higher program satisfaction among PwMS, although both dyad partners reported improvements in various resilience protective factors. A third intervention called READY for MS is the most widely evaluated resilience training program for PwMS and is the focus of this study [14].

READY for MS was derived from a group resilience training program called “REsilience Activities for every DaY” (READY) that was effectively applied to people with cancer [15], congenital heart disease [16], and diabetes [17] and staff in a workplace setting [18]. A modified version of the program for PwMS was developed and titled READY for MS [14].

READY for MS is based on acceptance and commitment therapy (ACT) [19], a contemporary variant of cognitive behaviour therapy. ACT is underpinned by the psychological flexibility framework. Broadly, the goal of ACT is to increase psychological flexibility, which involves behaving consistently with one’s chosen values even in the presence of unwanted intrusive internal experiences, such as emotional discomfort or self-critical thinking [19]. Psychological flexibility is the cornerstone of psychological health and is a source of resilience [20]. Furthermore, psychological flexibility processes have been shown to mediate the beneficial effects of ACT-based resilience training [14]. According to the ACT framework, six core processes enhance psychological flexibility: (1) acceptance—openness to experience, (2) cognitive defusion—observing thoughts rather than taking them literally, (3) present moment awareness (mindfulness)—open and responsive awareness of the present, (4) self-as-context—flexible self-awareness and perspective-taking, (5) values—freely chosen personally meaningful life directions, (6) committed action—values-guided effective action. The inverse of these processes leads to psychological inflexibility. Reviews suggest that ACT interventions show promise in promoting mental health and QoL in people with chronic illnesses [21], people with neurological disorders [22], and specifically PwMS [23,24]. As detailed below, evaluations of READY for MS have supported its theoretical foundation by demonstrating that therapeutic change occurs through psychological flexibility, the mechanism of action proposed by the ACT framework [19].

READY for MS was first evaluated in Australia using a single-arm pre–post group intervention pilot study with 12-week follow-up (n = 37) [14]. Participants significantly improved in resilience, QoL, depression, stress, and three psychological flexibility processes (defusion, values, and acceptance). Values and defusion emerged as mediators of improvements in QoL and stress, respectively. Program feasibility was supported by positive participant feedback, high rates of recruitment, attendance, retention, and homework engagement, and good intervention fidelity.

The READY for MS program was later translated into Italian and evaluated in a pilot RCT (n = 39), which had a nested qualitative study and pre- and post-intervention and 12-week follow-up assessments [25]. READY for MS was compared to an active control group that received relaxation. Control group participants were offered the intervention after completing their final assessment. Hence, control participants were able to report on the benefits and weaknesses of READY for MS relative to the relaxation intervention. Qualitative (but not quantitative) data provided evidence in favour of READY for MS.

The Italian READY for MS program was subsequently evaluated in a single-arm multi-centre effectiveness study (n = 237) with pre- and post-intervention and 12-week follow-up assessments and a nested qualitative study [26]. The results showed that participants improved in resilience, anxiety, depression, stress, health-related QoL, and psychological flexibility. Improvements on most outcomes occurred at post-intervention and were maintained at 12-week follow-up. No demographic or illness variables predicted these improvements. Psychological flexibility mediated improvements in resilience, anxiety, depression, stress, and health-related QoL. Qualitative data confirmed the positive psychosocial impacts of the intervention.

Findings from these prior studies evaluating the READY for MS program informed the development of a study protocol for a multi-centre cluster RCT comparing READY with a group relaxation program in 240 PwMS from eight centres in Italy [27]. In addition, healthcare professionals trained in READY for MS have reported improvements in distress, wellbeing, resilience, and clinical skills from before to after training [28,29].

Frontline MS services have requested options for delivering the READY for MS program that do not rely on an in-person group format for several reasons. First, some organisations have insufficient resources to train and employ healthcare practitioners to deliver the program. Second, because some organisations offer services to PwMS across far-reaching geographical areas, it is logistically difficult to provide an in-person group intervention to PwMS who reside in regional or remote regions. Third, physical disability prevents some PwMS from regularly accessing face-to-face group sessions. Fourth, for many PwMS, the emergence of the COVID-19 pandemic and associated strict public health social distancing measures have precluded participation in face-to-face psychosocial interventions, despite evidence suggesting that the pandemic has had adverse impacts on the mental health and QoL of PwMS [30,31]. Finally, results of meta-analyses have shown that ACT delivered in self-help formats [32], including online [33,34], promotes mental health and psychological flexibility. Given these service delivery considerations and the empirical evidence supporting the online delivery of ACT interventions, the READY for MS program was modified for online delivery.

The Present Study

The primary aim of the present pilot RCT was to explore the effectiveness of the online READY for MS program (e-READY for MS). We predicted that, compared to a WLC group, e-READY for MS participants would improve significantly more on the primary outcome resilience and on the secondary outcomes QoL, distress (anxiety, depression, and stress), and psychological flexibility.

A secondary aim of this study was to explore the feasibility of converting a facilitated group resilience training program into an online intervention that has no practitioner and group support. Regarding the latter, qualitative data from prior evaluations of READY for MS have indicated that group effects (e.g., sense of interpersonal belonging) and facilitator attributes (e.g., empathic facilitation) are potent non-specific therapeutic elements of the group delivery of READY for MS [26]. Hence, in this study, we explore the feasibility of an online version of the program by investigating recruitment, retention, program engagement, online program usability, and participant perceptions of intervention helpfulness and satisfaction.

2. Materials and Methods

2.1. Study Design

The study design is an RCT with two conditions (e-READY for MS and WLC) and three online survey assessments: pre-intervention, post-intervention, and 12-week follow-up. E-READY for MS participants completed all three assessments. WLC participants completed the pre-intervention questionnaire and, after 7 weeks, equivalent to the 7-week e-READY for MS program, they completed a second identical questionnaire. After completion of the second assessment, WLC participants were given access to e-READY for MS. After completion of the program, WLC participants completed the post-intervention questionnaire. The study protocol received ethical clearance from The University of Queensland’s Human Research Ethics Committee (21 March 2019 clearance number: 2018001953). All participants read the study information sheet and the consent form and then gave informed consent by activating the “I agree” field on the screen; alternatively, they could activate the “I disagree” field and not consent to study participation. The study was registered on the Australian New Zealand Clinical Trials Registry (ANZCTR 20 May 2021, registration number: ACTRN12621000602820) before commencement of participant enrolment.

2.2. Recruitment

Participants were recruited from a pool of MS patients enrolled in a patient support program called “b•yond”. B•yond is a program of Biogen Australia Pty Ltd. and administered by Partizan Worldwide Pty Ltd., an Australian company that provides health support to patients with chronic disease. The eligibility criteria for enrolling in the b•yond program include taking an eligible Biogen medication for relapse-remitting MS. The study inclusion criteria were 18 years and older and fluency in English. No incentive was provided for participation in the study. Participation was voluntary and participants were free to withdraw at any time without prejudice or penalty. Recruitment occurred between 19 May and 31 August 2022 during the COVID-19 pandemic.

Regarding the randomisation process, after providing consent, participants were randomly allocated to either the WLC or intervention group using a custom algorithm based on a programmatic equivalent of a ‘coin-toss’. The algorithm generated a random fraction between 0 and 1, then rounded it to the nearest whole number, where 1 = intervention allocation and 0 = WLC allocation. The allocation of each participant was completely independent of all previous allocations (i.e., each ‘coin-toss’ was completed independently for each participant). All participants were then automatically directed to complete the online pre-intervention assessment.

Partizan sent an email invitation to participate in the study to all 958 MS patients enrolled in the b•yond program. Of these, 608 emails were opened, and 58 clicked the link. A second study invitation email was sent to 909 MS patients; 544 emails were opened and 17 clicked the link.

Figure 1 depicts a Consolidated Standards of Reporting Trials (CONSORT) flow chart of assessment and intervention completion in both study groups. A total of 56 participants consented to enrolment in the study and were randomized to the intervention (n = 31) or WLC group (n = 25). One intervention group participant was withdrawn from the study because they no longer met eligibility for membership of the b•yond program (i.e., ceased Biogen medication). Therefore, the final study sample was composed of 55 participants (intervention group n = 30; WLC group n = 25). Only 40% of the intervention participants completed the post-intervention assessment and 20% completed follow-up. In contrast, 76% of the WLC group completed their second assessment, but only 8% completed the post-intervention assessment after finishing the e-READY for MS program.

2.3. Intervention

The e-READY for MS program is an online version of the in-person group READY for MS program. The e-program consists of seven weekly core modules plus a booster module five weeks after the seventh module. The core seven module topics and order of presentation are as follows: (1) Introduction to the READY Resilience Model, (2) Mindfulness, (3) Acceptance, (4) Defusion I, (5) Defusion II and Self-as-context, (6) Values and Meaningful Action, (7) Review and Future Planning. The booster module provides a review of program information. Program strategies include psychoeducation, experiential exercises, guided meditations, metaphors, self-reflection tasks, self-monitoring, in-session practice activities, and home practice assignments. Participants receive a workbook that has two components: (1) module content and (2) the READY Personal Plan, which is comprised of in-session and home practice exercises. Participants record their personal practice in the Personal Plan. The Personal Plan constitutes a personalized resource to help participants apply program strategies to their specific context. The workbook was provided as a PDF file; however, a hardcopy of the Personal Plan was also sent to participants.

The creator of READY for MS worked with Partizan information technology (IT) staff to convert the e-READY for MS manual and resources into an online program. The e-READY for MS program was hosted on Partizan’s b•yond online platform. Regarding the first seven core modules, a module was released every seven days after the program commencement date, regardless of rate of progression through the modules. However, it was recommended that modules be completed weekly in sequential order. Participants could revisit modules. It was estimated that modules would take between 40 and 60 min to complete. The booster module was released five weeks after the seventh module was distributed. No practitioner support was provided. However, four days after completing a module, automated emails were sent that encouraged participants to complete the corresponding home practice. Additionally, automated reminder emails or SMS messages were sent to prompt timely completion of modules.

We conducted a face-validity pilot of the e-READY for MS program with four PwMS (3 females, 1 male) enrolled in the b•yond program. Three Partizan healthcare staff also completed the program. These pilot participants provided feedback via their responses to six open-ended questions. Overall, participants reported high levels of satisfaction with the program. The most important feedback with implications for the RCT phase was the need for flexibility in how the program was completed to accommodate fluctuations in MS illness and competing demands of career and family.

2.4. Measures

The primary outcome (resilience) and secondary outcomes (QoL, distress, psychological flexibility) were assessed at pre- and post-intervention and the 12-week follow-up. Measures of participant MS and socio-demographic characteristics were assessed at pre-intervention. Participant satisfaction with e-READY for MS was assessed at post-intervention and follow-up. All measures entail self-report and were completed online. Program usability was assessed at post-intervention. All outcome measures have been used in prior MS research. All observed internal reliability coefficients for all multi-item measures were ≥0.70.

2.4.1. Resilience

The 15-item Resilience Scale (RS-15) [35] was used to assess resilience. Each positively phrased item is scored on a 7-point Likert scale (1 = disagree to 7 = agree) and asks participants about their ability to manage stressors (e.g., “I usually take things in my stride”). Item scores are summed, with higher scores indicating higher resilience (range 15–105). The RS-15 is a short version of the original 25-item scale [36] and is a widely used measure of resilience, including in MS research [4,14], and has demonstrated good psychometric properties [35]. Observed McDonald’s omegas range across all assessments was 0.95–0.95.

2.4.2. QoL

QoL was measured by the Multiple Sclerosis Quality of Life Instrument (MSQoL-54) [37]. It consists of the generic QoL scale, the Short-Form-36 (SF-36) [38], and 18 MS-specific QoL items [37]. It consists of 8 health QoL subscales (physical health, role limitations due to physical problems, pain, energy, health perceptions, social function, health distress, sexual function) and 4 mental health QoL subscales (emotional wellbeing, role limitations due to emotional problems, health distress, cognitive function). Sample items include “How much bodily pain have you had during the past 4 weeks? How much of the time during the past 14 weeks was your health a worry in your life?”. Responses to questions are recorded using either a Likert scale with three to six options or a dichotomous “yes/no” option. Scores for each subscale are created by transforming item scores linearly to 0–100 range of scores and then averaging the transformed items within scales, with higher scores indicating greater QoL. Based on the final subscales scores, the two composites of the MSQoL-54, physical health and mental health, are calculated as a weighted sum of corresponding sub-scales (range 0–100). Higher scores on both composite scores indicate greater QoL [37]. The MSQoL-54 has good validity, internal consistency, and test–retest reliability [37]. Observed McDonald’s omegas range across all assessments for physical health QoL was 0.92–0.95 and for mental health QoL 0.92–0.93.

2.4.3. Distress

The short 21-item version of the Depression, Anxiety, and Stress Scales (DASS-21) [39] was used to assess distress. The measure comprises three subscales: depression (e.g., “I felt I had nothing to look forward to”); anxiety (e.g., “I felt I was close to panic”); and stress (e.g., “I found it hard to wind down”). Items are scored on a 4-point scale (0 = not at all to 3 = most of the time). Items are summed for each subscale and then multiplied by two to provide scores comparable to the 42-item parent scale (range 0–42 for each sub-scale). Higher scores indicate greater distress. The DASS-21 has good reliability and convergent validity in clinical and community samples [39,40] and has been widely used in MS research (i.e., [41]). Observed McDonald’s omegas range across all assessments for depression was 0.86–0.91, anxiety 0.70–0.77, and stress 0.82–0.86.

2.4.4. Psychological Flexibility

The 30-item psychological flexibility scale of the standardised widely used Multidimensional Psychological Flexibility Inventory (MPFI) was employed to assess psychological flexibility [42]. This scale assesses the six core ACT processes, which are resilience protective factors (acceptance, present moment awareness, self-as-context, defusion, values, committed action). Participants rated the extent to which they agreed with each item on a 6-point scale (1 = never true to 6 = always true). The MPFI has good psychometric properties in clinical and community samples [42,43,44] and has been recently used in MS research [27]. Observed McDonald’s omegas range across all assessments was 0.97–0.98.

2.4.5. Illness Duration and Type of MS

Participants were asked to provide the number of months since diagnosis and indicate whether they had relapse-remitting or chronic progressive MS.

2.4.6. MS Disease Severity

MS disease severity was assessed via a self-report version of the physician’s Disease Steps Scale [45] that has been used in prior MS research [46]. It measures the degree of mobility limitations associated with MS. Respondents nominate their level of mobility in eight scenarios using a response scale ranging from 1 (normal) to 8 (bedridden). A ninth scenario assesses the presence of symptoms that limit activity but are not related to mobility (e.g., eyesight or memory problems). Participants were classified into three broad categories: mild (scenarios 1–2 and 9), moderate (scenarios 3–5), and severe MS (scenarios 6–8).

2.4.7. Cognitive Impairment

Cognitive impairment was assessed using the 6-item cognitive sub-scale of the Mayo-Portland Adaptability Inventory (MPAI), which assesses impairment in communication, memory, attention, problem solving, visuospatial abilities, and common knowledge [47,48]. Each item is rated on a standardized 4-point scale (0 = no impairment to 3 = complete or nearly complete loss of function), with higher scores indicating poorer cognitive functioning. Observed McDonald’s omega was 0.84. The MPAI has previously been used in MS populations where high positive correlations between MS patient and carer ratings of cognitive impairment have been demonstrated [49]. The criteria used for severe cognitive impairment that may jeopardise the reliability of self-report was a score of two or more standard deviations above the mean and a score of three (indicating severe impairment) on two or more of the six items. None of the participants reached criteria indicative of severe cognitive impairment.

2.4.8. Socio-Demographics

Participants indicated their gender, age (via date of birth), education, residential location (via postcode), employment, household composition, finances (i.e., “how do you manage on your income”), country of birth, and language mostly spoken at home.

2.4.9. Program Feasibility

To address the secondary aim of exploring program feasibility, we investigated data on recruitment, retention, program engagement, online program usability, and participants’ perceptions of the helpfulness of the intervention. Regarding participant engagement, we collected data on number of modules completed, time engaged in the program, and the interval between modules.

Perceptions of Intervention Helpfulness and Satisfaction

In the post-intervention and follow-up questionnaires, six items inquired about participants’ perceptions of helpfulness and satisfaction regarding the e-READY for MS program as follows: (1) helpfulness of the READY Program, (2) the READY Personal Plan, (3) and the READY Resource Book, (4) enjoyableness of the READY Program, (5) recommendation of the READY program to others, (6) helpfulness of the READY program to become more resilient. Items were rated on a 5-point Likert scale (1 = strongly disagree to 5 = strongly). In the post-intervention and follow-up questionnaires, participants were also asked six open-ended questions to elicit feedback on program satisfaction and helpfulness and suggested program improvements (e.g., “In what ways have you become more resilient?” and “How has the READY program impacted on how you feel, think about, or manage your MS?”).

Online Program Usability

The 10-item System Usability Scale (SUS) [50] measured online program usability at post-intervention [51]. It provides a global view of subjective assessments of usability of a digital program. Items are rated on a 5-point scale (0 = strongly disagree to 4 = strongly agree). Scores are summed and the sum score is multiplied by two, with a total score ranging from 0 to 80. Higher scores indicate higher system usability.

2.5. Data Analysis Approach

A power analysis conducted before study commencement indicated that, to detect an effect size of d = 0.44, p < 0.01, a total sample size of 90 (45 per group) was required. However, the total sample size achieved for this pilot study was 56 (31 intervention and 25 WLC); therefore, the analyses are severely underpowered due to the small sample. Because of the low power, a less stringent significance level of p < 0.10 was employed, corresponding to 90% confidence intervals (CIs) [52,53].

In line with the recommendations by Lee et al. [53] and to provide an estimated range of possible intervention effects, we inspected the standardized mean differences (with their 90% and 85% CIs) between intervention and WLC groups on primary and secondary outcomes at post-intervention/second assessment. Standardized mean differences, also known as Cohen’s d, represent a measure of effect size. In addition to the magnitude of the intervention effect, the standardized mean differences also represent the direction of the intervention effect for positive mental health outcomes (i.e., resilience, QoL, and psychological flexibility) and negative mental health outcomes (i.e., distress), positive and negative values, respectively. In addition, the 90% and 85% CIs provide information about the range of the intervention effects with CIs that do not cross zero (i.e., CIs ranging from 0.2 to 3.1 or −0.99 to −0.22), indicating that the intervention effect is statistically significant at the 0.10 level and the 0.15 level, respectively [53].

The overall percentage of missing data for the observed values of primary and secondary outcomes was 44.24%. An intention-to-treat (ITT) sample using 400 imputations was created for replicable standard error estimates due to the high level of missing data [54]. However, the ITT post-intervention data contravened homogeneity of variance assumptions. In addition, if missing data exceeds 40%, as in the present study, ITT analyses are likely to be unreliable [55]. Moreover, given that 58% of the intervention participants did not engage in the intervention, ITT analyses are likely to be diluted and are not recommended [56]. Therefore, we explored the per-protocol (PP) sample. The Little’s Missing Completely at Random test [57] on PP post-intervention data χ² (47) = 45.03, p = 0.556 and 12-week follow-up scores χ² (1) = 0.026, p = 0.871 showed that data were missing completely at random. Percentage of missing data in the observed values of the primary and secondary outcomes in the PP pre–post dataset for intervention and WLC groups combined was 21.82%. Therefore, primary analyses exploring changes in the outcomes from pre- to post-treatment in the two groups were conducted on the PP sample.

Regarding preliminary analyses, to check on the randomization procedure, one-way ANOVAs and chi-square tests were conducted to examine whether the intervention and WLC differed on primary and secondary outcomes, MS illness characteristics, and socio-demographics. Furthermore, post-intervention assessment completers and non-completers (i.e., post-intervention for the intervention group and second assessment for the WLC group) were compared on these study variables.

To address the first aim, comparisons between the intervention and WLC groups on changes in primary and secondary outcome variables from pre-intervention to post-intervention were examined using 2 (group: intervention vs. WLC) × 2 (time: pre-intervention vs. post-intervention) repeated measures ANOVAs performed on the PP sample. Effect sizes were calculated with Cohen’s d, with effect sizes of 0.20, 0.50, and 0.80 considered as small, medium, and large effects, respectively [58].

Clinically significant changes from pre- to post-intervention on the primary and secondary outcomes in the treatment and WLC groups were investigated using the Reliable Change Index (RCI) [59]. The RCI is calculated by comparing a participant’s actual change on a scale to the expected spread of the distribution of change scores if no actual change had occurred. The RCI is an indicator of clinically significant change for a single participant. Due to the small sample size, the Fisher’s exact test was used (instead of chi-square test) to determine if the intervention and WLC groups significantly differed in the proportion of participants who exhibited reliable change from pre- to post-intervention.

To address the second aim, data on recruitment, retention, program engagement, online program performance, and participants’ perceptions of intervention helpfulness and satisfaction were investigated. In addition to presenting descriptive data on these factors, we conducted correlations to explore relations between program engagement indices and post-intervention outcomes and pre-intervention socio-demographics and MS illness variables. To further examine variables related to program engagement, we used ANOVAs and chi-square tests to compare intervention group program uses (n = 13) and non-users (n = 18) on demographics, MS illness variables, and outcomes.

Responses to the six open-ended questions eliciting program feedback were analysed according to Braun and Clarke’s [60] recommendations for thematic analysis. All responses were transcribed, read several times, and then organized thematically into categories by G.L. Codes were assigned to each theme. Calculation of the number and percentage of responses per theme provided an indication of the relative frequency of that category within the sample and is in line with qualitative approaches described by Miles et al. [61]. Although each theme category was mutually exclusive, responses containing multiple explanations were coded into each of the relevant theme categories. The allocation of responses to the categories was discussed with K.P. Any differences were negotiated and reconciled until agreement was achieved.

3. Results

3.1. Participant Characteristics

Sample characteristics are summarized in

Table 1. Descriptive Data on Socio-demographics and MS Illness Characteristics at Pre-intervention in the Total Sample as well as in the Intervention and WLC Groups.

Variable	Total Sample (n = 55)		Intervention (n = 30)		WLC (n = 25)
	% (n)	M (SD)	% (n)	M (SD)	% (n)	M (SD)
Socio-demographics
Gender: female	85.45 (n = 47)		83.33 (n = 25)		88.00 (n = 22)
Age		48.48 (13.08)		50.89 (14.22)		45.35 (10.85)
Household composition:
Single live alone	12.73 (n = 7)		16.67 (n = 5)		8.00 (n = 2)
Couple with no children	34.55 (n = 19)		36.67 (n = 11)		32.00 (n = 8)
Couple with children	42.27 (n = 26)		36.67 (n = 11)		60.00 (n = 15)
Single living with others	1.82 (n = 1)		3.33 (n = 1)		0.00 (n = 0)
Single with children	1.82 (n = 1)		3.33 (n = 1)		0.00 (n = 0)
Highest education:
High school	14.55 (n = 8)		16.67 (n = 5)		12.00 (n = 3)
Trade/Apprenticeship	32.73 (n = 18)		33.33 (n = 10)		32.00 (n = 8)
Bachelor’s degree	29.09 (n = 16)		16.67 (n = 5)		44.00 (n = 11)
Post-graduate degree	21.82 (n = 12)		33.33 (n = 10)		8.00 (n = 2)
Employment status:
Full-time	42.64 (n = 24)		46.67 (n = 14)		40.00 (n = 10)
Part-time/casual	27.27 (n = 15)		23.23 (n = 7)		32.00 (n = 8)
Retired	16.36 (n = 9)		20.00 (n = 6)		12.00 (n = 3)
Student	3.64 (n = 2)		0.00 (n = 0)		8.00 (n = 2)
Unable to work	8.93 (n = 5)		10.00 (n = 3)		8.00 (n = 2)
Finances (how manage on income):
Manage ‘easy’ on income	29.09 (n = 16)		33.33 (n = 10)		24.00 (n = 6)
Manage ‘not too bad’	43.63 (n = 24)		46.67 (n = 14)		40.00 (n = 10)
Difficulties some of the time	23.64 (n = 13)		16.67 (n = 5)		32.00 (n = 8)
Difficulties all the time	3.64 (n = 2)		3.33 (n = 1)		4.00 (n = 1)
Residential location:
Australia	91.91 (n = 50)		90.00 (n = 27)		92.00 (n = 23)
New Zealand	9.09 (n = 5)		10.00 (n = 3)		8.00 (n = 2)
English spoken at home	96.36 (n = 53)		93.33 (n = 28)		100.00 (n = 22)
MS illness characteristics
Months since diagnosis		69.22 (77.74)		79.13 (81.09)		57.32 (73.37)
Relapse-remitting MS	91.91 (n = 50)		86.67 (n = 26)		96.00 (n = 24)
Chronic progressive MS	9.09 (n = 5)		13.33 (n = 4)		4.00 (n = 1)
MS severity:
Mild	70.91 (n = 39)		73.33 (n = 22)		68.00 (n = 17)
Moderate	27.27 (n = 15)		23.33 (n = 7)		32.00 (n = 8)
Severe	1.82 (n = 1)		3.33 (n = 1)		0.00 (n = 0)

. A total of 85.45% participants were female, with a mean age of 48.48 years (SD = 13.08, range 24.50–83.89). Almost half (47.27%) of the participants were living with a partner and children, 34.55% lived with a partner and no children, and most of the remainder were single and living alone (12.73%). Almost one third (32.73%) reported as the highest education trade/apprentice, 14.55% high school, 29.09% bachelor’s degree, and 21.82% post-graduate degree. Regarding employment status, 42.64% full-time, 27.27% part-time/casual, 16.36% retired, and 3.64% student. Almost one quarter of the participants (23.64%) reported difficulties some of the time managing their income, 3.64% difficulties all the time, 43.63% managed ‘not too badly’, and 29.09% managed easily. As for residential location, 9.09% lived in New Zealand, while the remainder lived in Australia: 32.73% New South Wales, 23.64% Queensland, 20.00% Victoria, 5.46% Tasmania, 5.45% Australian Capital Territory, 1.82% South Australia, and 1.82% Western Australia. Regarding country of birth, 74.55% reported Australia, 9.09% New Zealand, 3.62% United Kingdom, 7.28% ‘other’ (one each from Brazil, Germany, Greece, and South Africa), while 5.45% did not disclose their country of birth. Almost all participants (96.36%) spoke English at home.

As for MS characteristics, the mean time since diagnosis was 69.22 months (SD = 77.74, range 0–324). Almost all participants (91.91%) reported relapse-remitting MS, while 9.09% chronic progressive MS. Regarding illness severity assessed by the Disease Steps Scale, 70.91% reported mild, 27.27% moderate, and 1.82% severe MS. The percentage of participants taking specific Biogen MS medications is summarised as follows: 41.82% (n = 23) Tecfidera, 38.18% (n = 21) Tysabri, 10.91% (n = 6) Vumerity, 7.27% (n = 4) Plegridy, and 1.82% (n = 1) Avonex.

3.2. Preliminary Analyses

There were no significant differences between the intervention and WLC groups on pre-intervention primary and secondary outcomes, MS illness characteristics, or socio-demographics. Post-intervention assessment completers and non-completers (i.e., post-intervention for the intervention group and second assessment for the WLC group) did not differ significantly on pre-intervention primary and secondary outcomes, MS illness characteristics, or socio-demographics.

Of the total sample, 74.55% (n = 41) of participants reported mild to moderate psychological distress at pre-intervention as follows: depression 27.27% (n = 15) mild, 12.72% (n = 7) moderate; anxiety 12.73% (n = 7) mild, 23.64% (n = 13) moderate; stress 45.45% (n = 25) mild, 21.82% (n = 12) moderate. Furthermore, 23.64% (n = 13) reported severe to extremely severe distress at pre-intervention as follows: depression 7.27% (n = 4) severe, 9.09% (n = 5) extremely severe; anxiety 7.27% (n = 4) severe, 7.27% (n = 4) extremely severe; stress 1.82% (n = 1) severe, 1.82% (n = 1) extremely severe.

3.3. Changes in Primary and Secondary Outcomes

Means, standard deviations, and estimated 90% confidence intervals for primary and secondary outcomes and results of the repeated measures ANOVAs are reported in

Table 2. Means, Standard Deviations, and Estimated 90% Confidence Intervals for the Intervention and WLC Groups and ANOVA Statistics for the Primary and Secondary Outcomes as a Function of Group by Time.

	Intervention			WLC
	Pre-Intervention (n = 12)	Post-Intervention (n = 19)	Follow-Up (n = 6)	Pre-Intervention (n = 12)	Second Assessment (n = 19)	ANOVA Statistics
	M (SD) 90% CI	M (SD) 90% CI	M (SD) 90% CI	M (SD) 90% CI	M (SD) 90% CI	F	d
Resilience	77.50 (15.89) 76.26–90.41	85.92 (12.04) 78.43–93.40	89.50 (14.46) 77.60–101.40	80.05 (14.87) 72.61–83.86	79.35 (16.93) 73.40–85.29	1.20	0.20
QoL—Physical health	58.81 (18.17) 48.71–64.48	66.64 (15.35) 58.35–74.94	67.38 (21.04) 50.07–84.69	52.85 (17.89) 45.80–58.33	55.85 (17.80) 49.25–62.44	1.73	0.31
QoL—Mental health	59.58 (20.10) 49.39–69.75	73.51 (18.20) 63.49–83.54	75.61 (27.47) 53.00–98.21	58.75 (20.92) 49.55–65.72	64.48 (21.69) 56.51–72.45	0.59	0.00
Depression	11.53 (8.51) 5.71–13.62	5.83 (3.95) 2.45–9.21	6.00 (7.59) 1.24–6.52	10.79 (10.55) 7.15–13.44	9.88 (8.19) 7.19–12.57	0.80	0.00
Anxiety	8.04 (8.49) 3.08–9.25	4.33 (4.74) 1.49–7.17	4.67 (4.32) 1.11–8.02	8.61 (5.49) 6.90–11.80	8.08 (6.35) 5.83–10.34	3.42 * (p = 0.075)	0.56
Stress	14.20 (8.29) 9.15–15.85	8.67 (6.11) 5.17–12.16	12.67 (8.26) 5.87–13.51	15.04 (6.67) 13.65–18.98	13.95 (7.68) 11.17–16.72	3.99 * (p = 0.055)	0.62
Psychological flexibility	3.79 (0.95) 3.57–4.47	4.24 (0.90) 3.81–4.68	4.31 (1.15) 3.37–5.26	3.69 (0.84) 3.36–4.07	3.87 (0.87) 3.53–4.22	1.18	0.15

Note: * p < 0.10. 90% CI = estimated 90% confidence interval.

. The time x group effect was non-significant for the primary outcome resilience. Regarding the secondary outcomes, there were significant time x group effects for anxiety [F(1,29) = 3.42, p < 0.10, d = 0.56] and stress [F(1,29) = 3.99, p < 0.10, d = 0.62] but not for physical and mental health QoL, depression, and psychological flexibility. Hence, compared to the WLC group, the intervention group evidenced greater pre- to post-intervention decreases in anxiety and stress. The results correspond to a medium effect size for both anxiety and stress.

Figure 2 presents the standardized mean differences between intervention and WLC groups on primary and secondary outcomes at post-intervention with 90% and 85% CIs. These data show a trend for the intervention group to evidence markedly greater improvements on all outcomes. In particular, the 90% CIs replicated the results of repeated measure ANOVAs, indicating that the intervention group evidenced significantly greater improvements in anxiety and stress than the WLC group. The 85% CIs revealed a trend for the intervention group to also show greater improvements in physical health QoL and depression than the WLC group.

The results of the within-subjects effect for repeated measures ANOVAs indicated no significant difference from post-treatment to follow-up in anxiety, [F(1,5) = 1.31, p > 0.10, d = 0.42], or stress, [F(1,5) = 1.02, p > 0.10, d = 0.11]. Therefore, intervention gains were maintained at the 12-week follow-up.

3.4. Clinically Significant Change

The percentages of participants who had shown clinically significant change expressed as either improvement or deterioration in primary and secondary outcome scores for both the intervention and WLC are reported in

Table 3. Percentage of Participants in the Intervention and WLC Groups with Reliable Change from Pre-intervention to Post-intervention/Second Assessment and Post-intervention to Follow-up.

	Improvement			Deterioration
	Intervention Pre to Post (n = 12)	Intervention Post to Follow-Up (n = 6)	WLC Pre to Second Assessment (n = 19)	Intervention Pre to Post (n = 12)	Intervention Post to Follow-Up (n = 6)	WLC Pre to Second Assessment (n = 19)
	% (n)	% (n)	% (n)	% (n)	% (n)	% (n)
Resilience	8.33% (n = 1)	0.00 (n = 0)	0.00 (n = 0)	0.00 (n = 0)	0.00 (n = 0)	0.00 (n = 0)
QoL—Physical health	41.67 (n = 5)	0.00 (n = 0)	15.79 (n = 3)	0.00 (n = 0)	16.67 (n = 1)	0.00 (n = 0)
QoL—Mental health	41.67 (n = 5)	0.00 (n = 0)	10.53 (n = 2)	0.00 (n = 0)	16.67 (n = 1)	0.00 (n = 0)
Depression	41.67 (n = 5)	50.00 (n = 3)	31.58 (n = 6)	0.00 (n = 0)	50.00 (n = 3)	21.05 (n = 4)
Anxiety	25.00 (n = 3)	50.00 (n = 3)	36.84 (n = 7)	16.67 (n = 2)	0.00 (n = 0)	10.53 (n = 2)
Stress	58.33 (n = 7)	66.67 (n = 4)	42.11 (n = 8)	16.67 (n = 2)	33.33 (n = 2)	15.79 (n = 3)
Psychological flexibility	8.33% (n = 1)	0.00 (n = 0)	0.00 (n = 0)	0.00 (n = 0)	0.00 (n = 0)	0.00 (n = 0)

. Overall, there was a tendency for more intervention participants to report clinically significant improvements on most outcomes relative to the WLC group. Fisher’s exact tests were used to determine if the groups significantly differed in the proportion of participants who exhibited reliable change from pre- to post-treatment assessment. All Fisher’s exact tests performed on the RCI improvement data were non-significant; however, the Fisher’s exact test p-value for physical health QoL approached significance p = 0.109, indicating a trend for more intervention participants to report clinically significant pre to post improvements on this outcome than WLC participants. Similarly, all Fisher’s exact tests performed on the RCI deterioration data were non-significant; however, the Fisher’s exact test p-value for depression approached significance p = 0.139, indicating a trend for more WLC participants to report clinically significant pre to post deterioration on this outcome than intervention participants.

3.5. Program Engagement

Descriptive data on program engagement are reported in

Table 4. Descriptive Data on Program Engagement in the Intervention and WLC Groups.

	Intervention (n = 30)			WLC (n = 25)
	% (n)	M (SD)	Median	% (n)	M (SD)	Median
Number of modules completed (range 1–8)		4.85 (3.21)	4.00		4.92 (2.47)	5.00
Completed module 1 Introduction to READY	43.33 (n = 13)			52.00 (n = 13)
Completed module 2 Mindfulness	30.00 (n = 9)			48.00 (n = 12)
Completed module 3 Acceptance	30.00 (n = 9)			40.00 (n = 10)
Completed module 4 Defusion I	26.67(n = 8)			32.00 (n = 8)
Completed module 5 Defusion II and Self-as-context	20.00 (n = 6)			32.00 (n = 8)
Completed module 6 Values	20.00 (n = 6)			24.00 (n = 6)
Completed module 7 Review	20.00 (n = 6)			20.00 (n = 5)
Completed booster module	20.00 (n = 6)			8.00 (n = 2)
Minutes engaged in module 1 Introduction to READY a		26.61 (33.35)	18.20		32.99 (67.60)	16.38
Minutes engaged module 2 Mindfulness a		28.40 (31.11)	25.53		24.90 (23.37)	17.58
Minutes engaged module 3 Acceptance a		21.29 (19.84)	18.73		26.26 (22.11)	28.97
Minutes engaged module 4 Defusion I a		27.26 (32.89)	12.78		67.35 (93.26)	40.18
Minutes engaged module 5 Defusion II and Self-as-context a		33.99 (30.09)	30.83		39.45 (30.29)	34.63
Minutes engaged module 6 Values a		47.42 (49.55)	38.22		105.83 (98.69)	79.63
Minutes engaged module 7 a		92.70 (172.75) †	28.53		34.81 (6.51)	32.67
Minutes engaged booster a		41.53 (44.03)	25.14		53.00 (63.57)	53.00
Minutes engagement per module (modules 1–7) a		29.19 (34.44) †	17.85		34.88 (32.21)	23.84
Hours of program engagement (modules 1–7, plus booster)		2.93 (4.43) †	1.14		3.54 (4.19)	1.91
Interval between Modules b
Days between finishing a module and starting the next module (modules 1–7) c		8.57 (1.38)			9.76 (6.74)
Days between module 1 and 2 c		8.57 (1.38)			9.76 (6.74)
Days between module 2 and 3 c		12.78 (6.52)			7.50 (7.51)
Days between module 3 and 4 bc		8.78 (6.80)			10.50 (12.97)
Days between module 4 and 5 c		6.38 (5.68)			11.63 (7.56)
Days between module 5 and 6 c		5.33 (5.28)			9.63 (9.83)
Days between module 6 and 7 c		8.17 (5.67)			11.67 (7.61)
Weeks between module 7 and booster d		6.00 (6.03)			7.00 (7.81)

Note: ^a It was estimated that modules would take between 40 and 60 min to complete. ^b Time between finishing one module and starting the next module. ^c Recommended time between finishing a module and starting the next is 7 days. ^d Recommended time between finishing module 7 and starting the booster is 5 weeks. ^† One participant (intervention group) took 1376 min to finish module 7. This outlier was excluded from the calculation of the relevant engagement means.

. The mean number of modules completed for both intervention and WLC groups was close to five out of eight modules. Only 43.33% and 52.00% of the intervention and WLC participants, respectively, completed one or more modules; hence, intervention uptake was relatively low. The number of participants completing modules gradually decreased over the course of the program. Only 20% and 8% of intervention and WLC participants, respectively, completed the booster. Overall, there was a tendency for WLC participants to spend more time engaged in the program than intervention participants. That is, the mean number of minutes participants engaged in each of the core program modules (1 to 7) was higher for the WLC group (34.88 vs. intervention 29.19), as was the overall mean number of hours engaged in the entire program (WLC 3.54 vs. intervention 2.93). Modules with the highest mean minutes of engagement were the values, review, and booster modules, which are the modules that take longer to complete.

The mean number of days between the seven core modules was 8.57 (intervention group) and 9.76 (WLC), which are similar to the recommended seven-day interval between modules. However, mean number of days between the seven core modules varied from 5.33 to 12.78 in the intervention group and 7.00–11.67 for the WLC. For each group, the mean number of weeks between completing the seventh module and the booster was close to the recommended five-week interval (6.00 intervention group, 5.50 WLC). Module revisit data are not displayed in Table 4 because only module one was revisited and by one participant, who was in the intervention group.

3.6. Relations between Program Engagement and Outcomes, Socio-Demographics, and MS Illness Variables

We conducted correlations to investigate whether the number of modules completed and the mean number of minutes engaged in each module were correlated with post-intervention outcome scores in the intervention group. The results showed that greater mean number of minutes engagement per module was significantly related to better physical health QoL (r = 0.77, p < 0.05) and greater levels of psychological flexibility (r = 0.88, p < 0.05) at post-intervention. We also conducted correlations to examine whether socio-demographics and MS illness variables were correlated with the number of modules completed and the mean number of minutes engaged in each module in the intervention group. The results showed that none of the socio-demographics and MS illness variables were related to levels of engagement in the e-READY for MS program.

To further examine study variables that are related to program engagement, we used ANOVAs and chi-square tests to compare intervention group program uses (completed one or more modules, n = 13) and non-users (did not start a module, n = 18) on demographics, MS illness variables, and outcomes. Program users did not differ from program non-users on any of the study variables.

3.7. Perceptions of Intervention Helpfulness and Satisfaction

Table 5. Participant Feedback on the e-READY for MS Program.

	M (SD)			Median
	Intervention Post (n = 12)	Intervention Follow-Up (n = 6)	WLC Post (n = 4)	Intervention (n = 12)	Intervention Follow-Up (n = 6)	WLC Post (n = 4)
1. Overall, I found the READY program helpful	3.75 (1.06)	4.33 (0.82)	4.75 (0.50)	3.50	4.50	5.00
2. I found the READY Personal Plan helpful	3.75 (0.75)	4.33 (0.52)	4.00 (0.82)	4.00	4.00	4.00
3. I found the READY Resources Book helpful	3.75 (0.87)	4.17 (0.75)	4.00 (0.82)	4.00	4.00	4.00
4. Doing the READY program was enjoyable	3.67 (1.07)	4.00 (0.89)	4.33 (1.15)	4.00	4.00	5.00
5. I would recommend the READY program to others living with MS	3.92 (1.08)	4.33 (0.82)	4.75 (0.50)	4.00	4.50	5.00
6. The READY program has helped me become more resilient	3.25 (1.22)	4.00 (1.26)	4.25 (1.50)	3.00	4.50	5.00

Note: Ratings on a 5-point scale: 1 = strongly disagree, 2 = disagree, 3 = not sure, 4 = agree, 5 = strongly agree.

summarises descriptive data for the intervention and WLC groups on post-treatment participant ratings for the six items inquiring about the helpfulness of the e-READY for MS program. Almost all ratings hovered around the 4-rating, indicating that most participants agreed that they enjoyed the e-READY for MS program, found the program and its resources helpful, would recommend the program to other PwMS, and that the program helped them become more resilient. The highest mean ratings across intervention and WLC groups were for the helpfulness of the overall program (3.75–4.75) and for recommending the program to other PwMS (3.92–4.75). The lowest mean rating was for the item stating that the program had helped participants become more resilient (3.25), and this occurred at the post-intervention phase in the intervention group. However, intervention participants who completed the follow-up assessments and those in the WLC group rated this item higher (4.00–4.25).

Table 6. Participant Responses to Open-ended Feedback Questions.

Q1—In what ways have you become more resilient? n = 19 (intervention = 15; WLC = 4)
Themes/Categories	% (n)	Sample response
Non-specific improved coping or resilience	42.11 (8)	“Able to deal with situations better”
Flexible perspective-taking	21.05 (4)	“I am developing an ability to step back from problems and pain and view from a distance”
Acceptance	15.79 (3)	“Allowing negative emotions and feeling to be”
Already resilient no change	15.79 (3)	“I don’t think I’ve become more resilient. I have always been resilient”
Awareness of inner experiencing	15.79 (3)	“Recognition of my thoughts and emotions”
Defusion	15.79 (3)	“Not becoming involved/wrapped up in negative thoughts or feelings and being able to instead observe them, and allowing them to pass”
Realigning with values	10.52 (2)	“Realigning with my values” and “found some meaning
Uncertain	10.52 (2)	“Not sure”
More mindful	5.26 (1)	“Practicing mindfulness”
Q2—What did you value most about the READY program? n = 18 (intervention 14; WLC 4)
Themes/Categories	% (n)	Sample response
Strategies/tools/resources	50.00 (9)	“Learning all the strategies”
Accessible information delivery	27.78 (5)	“The information was delivered in a way that was easy to understand” and “Being able to do it when I was able”
Practice exercises	16.67 (3)	“Practical exercises which helped to reinforce what I was learning by putting the skills into practice immediately”
Mindfulness	11.11 (2)	“Presence and self-awareness activities”
Uncertain	11.11 (2)	“I can’t say that I spent too much time on it—I have a busy working life that exhausts me”
Values	5.56 (1)	“Helped me re-focus on my values”
Q3—What are the most helpful skills you learnt from the READY program? n = 15 (intervention 12; WLC 3)
Themes/Categories	% (n)	Sample response
Defusion	53.33 (8)	“Having a broader perspective and stepping back using defusion has been helpful” and “To observe thoughts and minimise trying to fight them when they’re uncomfortable”
Mindfulness	53.33 (8)	“Meditation (most importantly finding a form of meditation that works for me)”
Acceptance	13.33 (2)	“Accepting”
Self-as-context	6.67 (1)	“Observer-self skills”
Resilience	6.67 (1)	“Resilience”
Values	6.67 (1)	“Awareness of values and how they drive me”
Uncertain	6.67 (1)	“Uncertain”
Q4—How has the READY program impacted on how you feel, think about, or manage your MS? n = 17 (intervention 13; WLC 4)
Themes/Categories	% (n)	Sample response
No/Uncertain	41.18 (7)	“Not about MS but about anxiety and work stress”
Resilient mindset to MS	35.29 (6)	“This program has positively impacted on my overall wellbeing, and although I already had quite a positive, resilient mindset in relation to managing my MS I strongly believe that these new skills will help me continue to maintain a good mindset as the MS progresses” and “I recently had some disease progression for the first time in four years and although it was very disheartening, I have picked myself up and kept going”
Non-specific MS adjustment gains	17.65 (3)	“Yes—it supports my approach but gives me specific content to apply and think about and consider”
Direction and meaning	11.76 (2)	“Found some direction & meaning”
Reinforced current coping strategies	5.88 (1)	“It has confirmed that I am dealing with my MS in a healthy way, seeking social connection, making exercise goals and using exercise as a mindfulness and self-care tool”
Q5—What aspects of the READY program would you like to see done differently? n = 13 (intervention 10; WLC 3)
Topics	% (n)	Sample response
None	23.08 (3)	“Nothing really, it was great”
Changes to audio files:	15.38 (2)	“Ability to speed up the talking, to pause it and scroll back and forth instead of it always going back to start the whole thing again”
Case study videos	7.69 (1)	“Possibly some brief case study videos of how people living with MS have implemented strategies to become more resilient”
Clearer Personal Plan Layout	7.69 (1)	“The personal plan was sometimes a little redundant and confusing”
Not immediately after MS diagnosis	7.69 (1)	“The timing for me was difficult—directly after a diagnosis—starting medication and working full time managing a family. Finding time to not feel overwhelmed while adjusting to a new reality”
Additional booster	7.69 (1)	“Perhaps another booster or two. I found it difficult to make time to practice all techniques as I went on holiday, work got busy, etc.”
Target low-resilience people	7.69 (1)	“Perhaps best for targeted audiences—those with lower resilience”
Non-gendered language	7.69 (1)	“Use of inclusive non-gendered language, (i.e., they/them instead of he/she)”
YouTube videos	7.69 (1)	“I didn’t find value in some of the YouTube videos but can appreciate that they gave the content delivery variety and helped to keep me engaged”
Uncertain	7.69 (1)	“Uncertain”
Response not aligned to question	7.69 (1)
Q6—Other feedback on the READY program? n = 9 (intervention 6; WLC 3)
Themes/Categories	% (n)	Sample response
Appreciation	55.56 (5)	“Thank you for providing access to this program, I have found it valuable.”
Technical difficulties a	44.44 (4)	Audio file rewind/fast forward/pause control difficulties Some audio-guided meditations could not be fully played Late arrival of Personal Plan Modules released before week had ended
Program Improvements a	44.44 (4)	Faster pace of speech in audio files, but acknowledged slow pace important for cognitively impaired PwMS Greater flexibility managing progression through program Develop app application Delay program delivery by a few months for newly diagnosed

Note: ^a Each sample response bullet point represents a paraphrased response for ease of understanding and presentation. Although each theme category was mutually exclusive, responses containing multiple explanations were coded into each of the relevant theme categories; hence, percentages do not add up to 100.

summarises the themes derived from content analyses performed on responses to the six open-ended questions eliciting feedback on the e-READY for MS program. Due to the relatively small number of respondents, intervention participants’ responses obtained from the post-intervention and follow-up questionnaires and WLC participants’ responses obtained at post-intervention were pooled and subjected to content analyses. Themes were derived from responses to each question except for question five, which asked about how the program could be improved. Almost every suggested improvement was unique and informative. Hence, to retain all information provided, we reported each suggested improvement topic with the corresponding verbatim response (see Table 6).

The most reported theme related to Question 1, which inquired about how participants had become resilient, was non-specific improved coping (42.11%). Most of the themes (six of nine) that emerged from responses to this question related to one of the six psychological flexibility processes. Interestingly, three participants reported they were already resilient. The most reported theme related to Question 2, which inquired about the program component that participants liked most, was the intervention’s strategies and resources (50%). Accessible information delivery was the second most reported theme (27.78%). Regarding Question 3, which asked about the most helpful skills learned, the majority of the themes (five of seven) related to the psychological flexibility processes. The most reported psychological flexibility processes were defusion (53.33%) and mindfulness (53.33%). The two most reported themes related to Question 4, which inquired about how the program impacted managing MS, were “no/uncertain” (41.18%) and “resilient mindset to MS” (32.29%). Question 5 elicited suggestions for improving e-READY for MS. However, some responses to the sixth and final question, which sought general comments, also referred to program improvements. Inspection of these responses across the two questions shows that, while three respondents (23.08%) said no changes were required, suggested changes reported by other participants included improvements to the online content (use of case study videos, changes to current videos, non-gendered language), clearer layout of the Personal Plan booklet, better platform functionality (audio files), not delivering the program immediately after MS diagnosis, increased program duration (adding a booster module), and a refined target group (people with low resilience). Finally, two additional themes emerged in response to general feedback Question 6: program appreciation (55.56%) and technical difficulties (44.44%).

3.8. Online Program Usability

We compared our System Usability Scale data to norms that indicate the following cut-off scores: ≥68 = above average and <68 = below average [62]. Of the intervention (n = 11) and WLC (n = 4) participants who completed the scale, 66.67% (n = 10; intervention group n = 6, WLC n = 4) evaluated the e-READY for MS program as above average, indicating these participants found the program easy to use, well-integrated, and consistent (intervention group M = 62.36, SD = 16.78, range 26–78; WLC M = 71.98, SD = 5.51, range = 68–80).

4. Discussion

The results of the present pilot RCT provided some support for the prediction that, compared to a WLC group, e-READY for MS participants would improve significantly more on the primary outcome resilience and on the secondary outcomes distress, QoL, and psychological flexibility. Specifically, intervention participants reported significantly greater pre- to post-intervention improvements in anxiety and stress than WLC participants, and these gains were maintained at the 12-week follow-up. These intervention effects were of a medium size. Notably, the standardized mean differences between intervention and WLC groups on primary and secondary outcomes at post-intervention exhibited a trend for the intervention group to report markedly greater improvements than the WLC group across all outcomes, especially physical health QoL, depression, anxiety, and stress. In addition, RCI data showed that, compared to the WLC group, there were trends for more intervention participants to evidence clinically significant improvements in physical health QoL and for fewer intervention participants to manifest a deterioration in depressive symptoms. In the context of an underpowered study, these intervention effects, although preliminary, are nevertheless notable.

The absence of significant intervention effects for the primary outcome resilience may be due to several factors other than the study being underpowered. First, it is possible that few participants achieved a level of program engagement that reached a threshold necessary for boosting resilience. In support of this proposal, the results of this study showed that more time engaged in the program was related to higher psychological flexibility at post-intervention. Given that the psychological flexibility strategies foster resilience [63], it is possible that less than optimal engagement in the program leads to insufficient mastery of these skills to bring about detectable improvements in resilience. Evaluations of the group READY for MS program that have evidenced significant improvements in resilience have also reported relatively high levels of session attendance [14,26]. Second, delayed improvements in resilience have emerged in the post-intervention to follow-up phase in two studies evaluating the group READY for MS program [14,26]. It seems that an adequate amount of time following post-intervention is necessary to consolidate the acquisition and application of the psychological flexibility strategies in daily living, thereby bolstering resilience. However, the high attrition at follow-up in the present study reduced the power to detect possible delayed improvements in resilience.

A secondary aim of this study was to explore the feasibility of converting a facilitated group-delivered READY for MS program into an online intervention that excludes practitioner and group support, which are important non-specific therapeutic elements of the parent program. For this purpose, we considered data on recruitment, retention, program engagement, online program usability, and participants’ perceptions of intervention helpfulness and satisfaction. Regarding recruitment, the response was relatively poor, with only 5.85% of the potential participant pool enrolled in the study. There are several factors that may explain the weak recruitment response. First, the 15-week recruitment period was relatively short. Second, it is possible that many in the potential participant pool considered that they were adequately resourced through the b•yond program and were consequently not motivated to invest time and energy in a resilience training intervention. In this regard, it should be noted that the b•yond program provides a wide range of resources and supports, including consultations with experienced MS nurses, psychoeducation about living with MS and medical treatments, personalised support and information about third party services and resources to assist with wellbeing, and articles on mental health and mindfulness. However, it should also be noted that almost a quarter (23.64%) of the total sample reported clinically significant levels of distress at pre-intervention, suggesting that their enrolment in the study was a form of mental health help-seeking. Third, due to the increased reliance on tele- and digital health services, the shifting of social activities to online, and the dependence on digital technologies when working from home during the COVID-19 pandemic, it is possible that many potential participants experienced the burden of digital overload, burnout, and/or fatigue [64]. Indeed, perceived digital information overload is related to poorer wellbeing in help seekers and a greater likelihood of discontinuing utilisation of online health resources [65].

Intervention uptake was relatively poor for the intervention group. Of those randomly assigned to one of the two study conditions, less than half (40.00%) of the intervention group engaged with e-READY for MS, whereas 76.00% of WLC participants engaged in the program when it was offered to them. Retention was relatively high among those who engaged in the e-READY for MS program, with only two intervention dropouts. The intervention dropout rate (6.67%) is low compared with that of the treatment arms of MS controlled trials of psychological interventions (M = 12.86%; range = 0.00–31.3%) reviewed by Fiest et al. [66]. However, dropout of the study assessment protocol was high at follow-up, with 80% of the intervention participants not completing the follow-up questionnaire. Study protocol attrition is also problematic in the broader literature evaluating digital mental health interventions, ranging from 27% to as high as 87% [67].

Program engagement data showed that, on average, just over half of the program was completed by participants, which is comparable to engagement levels in clinical trials of digital mental health interventions [67]. Interestingly, not only was intervention uptake greater in the WLC group than in the intervention group but WLC participants who undertook the intervention also tended to engage more with the program than intervention participants. It is possible that the seven-week period that WLC participants were unable to access the intervention increased their commitment to the study procedure and their motivation to engage in the e-READY for MS program when it became available. Supporting this view is Eysenbach’s [68] proposal that participants in traditional RCTs go through various clinical filtering steps prior to entering treatment, which is in itself a demonstration of commitment to a study’s procedures. In contrast, the intervention participants in the present study had immediate and easy access to e-READY for MS, which in turn may have lowered motivation or commitment to the program [69].

To investigate potential study variables associated with the less-than-optimal level of module completion, we investigated whether the socio-demographic and MS illness variables were significantly correlated with indices of program engagement. The results showed that none of these variables emerged as significant correlates. We, therefore, turn to other possible explanations. First, participants may have only engaged with content that they considered personally pertinent, which is one of the advantages of digital interventions. Evidence suggests that participants who do not complete an entire online mental health intervention may nevertheless benefit from it (e.g., [70]). Second, given the supports and resources available through the b•yond program, some participants may have considered that they were sufficiently skilled in some of the e-READY for MS strategies and only selected content that was novel for them. Third, the absence of practitioner support in the e-READY for MS program may have diminished sustained program engagement in some participants. Indeed, digital interventions that do not provide practitioner support demonstrate lower user engagement than those that offer therapist guidance [71].

The average amount of time engaged in each module was close to what was intended by the program developer. The mean number of days between modules was similar to the recommended seven-day interval between the seven core modules and the recommended five-week interval between the seventh module and the booster. These data suggest that, in general, participants adhered to the guidelines for progression through the program.

Most participants (66.67%) reported above average levels of satisfaction with the online performance of the program. However, a third rated below average levels of satisfaction with the online program performance and some participants reported technical difficulties in operating the program when responding to open-ended feedback questions (see Table 6). User dissatisfaction with online program performance is likely to contribute to intervention non-adherence and dropout. Hence, further work is required to optimize the online performance of the e-READY for MS program.

Regarding ratings of intervention helpfulness and satisfaction, most participants indicated that they enjoyed the e-READY for MS program, found the program and its resources helpful, would recommend the program to other PwMS, and that the program helped them become more resilient. Responses to open-ended questions eliciting feedback on the e-READY for MS program revealed themes indicating that the program had bolstered resilience through improved coping skills, most of which were related to the six psychological flexibility processes. Participant feedback validated the usefulness of intervention coping tools and the digital mode of delivering them. Just over half of the participants identified one or more positive impacts of the intervention on how they managed their MS. This is notable given that the program presents generic resilience-building strategies and does not provide information on MS, except for five case studies presented in module 6, which illustrate how PwMS have applied the READY strategies to their management of MS-related stressors. However, just under half of the participants reported no such positive impacts or uncertainty about them. This may be due to unmet personalised expectations that the program would focus on MS because it was embedded amongst a suite of MS-specific resources on the b•yond platform.

Although almost a quarter of participants reported that changes to the program were unnecessary, 10 different program improvements were suggested. However, each suggested change was reported by one participant, apart from the audio file functionality modification (reported by two participants). Nevertheless, most of the suggested improvements could be easily addressed in future iterations of the e-READY for MS program. Such modifications are likely to improve user satisfaction and, hence, the social validity of the intervention. Consequently, these suggested changes can serve as a guide for further refinements to the e-READY for MS program.

It is important to note that most of the intervention feedback themes that emerged in responses to four of the six open-ended questions related to one or more of the six psychological flexibility processes. In addition, greater engagement in the program was significantly correlated with greater mastery of the psychological flexibility strategies at post-intervention. Given that e-READY for MS is based on the ACT psychological flexibility model, these findings support the theoretical integrity of the program, deemed a critical element of effective online mental health interventions [64].

Limitations

Interpretation of study results should be tempered by consideration of the methodological limitations as follows. First, the generalisability of findings is limited by non-random sampling from a relatively well-serviced group of PwMS, all of whom were taking medications for relapse-remitting MS and comprised an underrepresentation of people with severe MS. Second, the 56 participants enrolled in the study was well below the target sample size of 90 established by a priori power analyses. Hence, the study was markedly underpowered to detect effect sizes of d = 0.40 and is likely to account for the lack of significant intervention effects in multivariate analyses. Third, because the study was underpowered, a less stringent significance level was used for the primary analyses, which may increase Type I error rate. Fourth, the relatively high number of intervention participants who failed to complete the follow-up assessment further reduced power to detect 12-week follow-up intervention effects. Fifth, given that we used a WLC design, we cannot rule out the possibility that intervention effects are due to non-specific therapeutic effects (e.g., digital placebo effects) and nor can we demonstrate superiority to other established intervention approaches. Future work should compare the e-READY for MS program to an active control. Sixth, all measures involved self-report online. Finally, due to the small sample size, we were unable to undertake planned analyses to determine whether psychological flexibility mediated significant improvements in the outcomes, as predicted by the framework underpinning the e-READY for MS program. In addition, given the high attrition at the third assessment for WLC participants who undertook the intervention, we were unable to conduct intended analyses to examine whether WLC participants improved from baseline to post-intervention.

5. Conclusions

Although this pilot study was statistically underpowered, the quantitative and qualitative data converge in providing preliminary support for the mental health benefits of the e-READY for MS program. The significant beneficial effects of e-READY for MS on anxiety and stress are noteworthy given the high prevalence of distress among PwMS [2] and that elevated stress can exacerbate MS symptoms [3]. In support of program feasibility, the intervention dropout was low, adherence to program progression guidelines was good, program usability satisfaction was high for most participants, and perceptions of intervention helpfulness and satisfaction were generally very positive. Future applications of e-READY for MS should consider participants’ suggested program improvements, investigate the psychological flexibility processes as potential mediators of improvements in outcomes, and examine program cost-effectiveness. Overall, the effectiveness and feasibility results provide pilot support for the e-READY for MS program. Moreover, the findings converge with preliminary data from a multi-centre pilot RCT of a UK version of e-READY for MS [72] and early findings from a single-arm pilot of online teleconference delivery of the group version of READY for MS in Greece [73]. The data from these pilot trials provide proof-of-concept that the face-to-face group READY for MS program can be effectively converted to an online delivery mode, which leads to improvements in mental health in PwMS.