The Evaluation of the Suitability, Quality, and Readability of Publicly Available Online Resources for the Self-Management of Fear of Cancer Recurrence

Cancer survivors often rely on the internet for health information, which has varying levels of readability, suitability, and quality. There is a need for high-quality online self-management resources for cancer survivors with fear of cancer recurrence (FCR). This study evaluated the readability, suitability, and quality of publicly available online FCR self-management resources. A Google search using FCR-related keywords identified freely available FCR self-management resources for cancer survivors in English. Resource readability (reading grade level), suitability, and quality were evaluated using relevant assessment tools. Descriptive statistics and cluster analysis identified resources with higher suitability and quality scores. Mean resource (n = 23) readability score was grade 11 (SD = 1.6, Range = 9–14). The mean suitability score was 56.0% (SD = 11.4%, Range = 31.0–76.3%), indicating average suitability and the mean quality score was 53% (SD = 11.7%, Range = 27–80%), indicating fair quality. A cluster of 15 (65%) resources with higher suitability and quality scores was identified. There were no significant associations between suitability or quality scores and the type of organisation that published the resources. Online FCR self-management resources varied in readability, suitability and quality. Resources with higher quality and suitability scores relative to other resources are identified for use by healthcare professionals and cancer survivors. Resources that are more culturally appropriate, with lower reading grade levels and detailed self-management strategies are needed.


Introduction
Fear of cancer recurrence (FCR), defined as "the fear, worry, or concern about cancer returning or progressing" [1], is a prevalent issue among cancer survivors with more than half reporting moderate FCR and 19-45% experiencing high/clinical FCR [2].Additionally, FCR is frequently identified by survivors as a major concern and one of their most common unmet supportive care needs across multiple cancer sites [3,4].FCR manifests on a continuum, ranging from rational concerns about recurrence to clinically significant FCR, characterised by persistently high levels of worry/preoccupation and hypervigilance to bodily symptoms [1,5].FCR can be present from diagnosis and remain stable or worsen throughout survivorship, particularly when these fears go unaddressed [6,7].High levels of FCR are related to poorer mental health and quality of life in both cancer survivors [4] and caregivers [8] and greater healthcare use [9], burdening both cancer survivors, their caregivers and the healthcare system.
Stepped or matched care models, where recommended interventions are matched to FCR level reported have been proposed and piloted as a means of efficiently and effectively addressing cancer survivors varying levels of FCR [10].Delivery of matched care requires interventions of varying intensity, from self-management resources that can be provided to all cancer survivors, or those with minimal/mild FCR, as part of an initial step/universal care, through to therapist-delivered interventions for more severe FCR.Provision of less intensive interventions early in survivorship may mitigate worsening of FCR over time [11].
To date, largely face-to-face psychologist-delivered treatments targeting more severe clinical levels of FCR have demonstrated efficacy in reducing FCR and ameliorating associated psychological distress and quality of life impairments.A meta-analysis of 23 controlled trials [7] evaluating the effect of psychological interventions on FCR, 20 of which were delivered face-to-face, found post-treatment improvements in FCR that were maintained at follow-up.While these interventions may offer an effective means of addressing more severe FCR, barriers, including time and costs associated with travel and taking time off work for treatment, may hinder accessibility [12][13][14].Limited research has focused on interventions that can feasibly address moderate but burdensome levels of FCR.Brief interventions delivered by non-mental health professionals (e.g., nurses or oncologists) have demonstrated potential [15][16][17], but require significant investment in education and training to deliver [18].
Self-management, defined as the active management of one's own physical and psychosocial wellbeing, may help overcome the barriers of face-to-face interventions described above [19].Low-intensity interventions, such as online self-management resources that provide psychoeducation about FCR, and strategies for managing FCR, may help make FCR support more widely available to survivors experiencing minimal or moderate FCR.Online psychological interventions for FCR have demonstrated moderate, but variable, effects in reducing FCR [20].While these online interventions may increase access to effective FCR treatment, many still require a substantial time investment (e.g., about 5 h for iConquerFear [14]), which may make them unappealing for some cancer survivors with mild-moderate FCR.Further, FCR reductions over time in the attention control group in the trial of another online intervention (FoRtitude) suggest that for some survivors with moderate FCR, information alone may be sufficient to reduce FCR, particularly those with higher self-efficacy, which was found to predict FCR improvements [21].
Approximately 69% of cancer survivors rely on the internet for health information [22].There are large amounts of cancer-related information available online with varying levels of readability, suitability, and quality [23,24].The lack of consistent guidelines and regulation of online health information risks inaccurate or low-quality information being unhelpful or even harmful [23].Unmet information needs and inadequate information provision have been theorised [25] and found to be related to greater FCR [16].A recent review of 45 online resources for adult cancer survivors who have completed primary treatment was conducted and their quality, suitability, readability, and usefulness in addressing prevalent unmet needs were assessed [26].Findings showed that only 58% of these unmet needs were addressed, for which cancer recurrence was only partially addressed (mean = 0.6/1.0)[26].There is a clear need for high quality low-intensity online selfmanagement resources for cancer survivors with mild to moderate FCR that are suitable and readable.These resources may prevent FCR from worsening and reduce the impact on mental health and quality of life.To fill this gap, we aimed to evaluate the suitability, quality, and readability of publicly available online resources designed to enable cancer survivors with mild-moderate FCR levels self-manage their concerns.

Sample and Setting
Online resources qualified as a self-management resource if they included suggestions for cancer survivors about how to manage FCR, as opposed to mere information provision about the FCR experience.To evaluate online self-management resources for FCR, an online search replicating how cancer survivors might search for FCR information online was conducted.Consumers commonly locate health information by entering short phrases and engaging with websites returned on the first page [24].As such, combinations of common keywords and layperson phrases were formulated after consultation with the librarian at the Liverpool Hospital (South West Sydney Clinical Campuses).The autocomplete function on Google Search also informed the development of search terms as the suggested phrases are generated based on queries commonly entered in the search engine.The author BS reviewed and approved the search terms (see Table 1).

1.
How to manage fear of cancer recurrence 2.
Anxiety and cancer 3.
Cancer patients and fear of cancer recurrence 4.
Manage fear of cancer recurrence 5.
How to stop worrying about fear of cancer recurrence 6.
How to fight emotions regarding cancer recurrence 7.
How to deal with fear of cancer recurrence 8.
What if my cancer comes back? 9.
Fear of cancer recurrence-Self manage The search terms were then entered into Google Search sequentially using the Google Chrome web browser, connected to an Australian IP address.Browsing history, cookies, and cache files were deleted prior to searching to ensure that the retrieval of Uniform Resource Locators (URLs) was not affected by search history and preferences.The search was conducted in Sydney, Australia (25 February 2020).The first page of results for each keyword or phrase were collated in a Google Document by TS with the title and URL of the originating resource noted and screened against the eligibility criteria by TS and VW.
Resources were included if they were freely available online (e.g., webpages, videos, or downloadable fact sheets) in English; addressed cancer survivors directly; and provided detailed information on FCR and management options.Questionnaires, testimonials, scientific journal articles, advertisements, books, podcasts, and resources targeting healthcare professionals were excluded.Resources eligible for inclusion were entered in a Microsoft Excel spreadsheet which was used for the subsequent evaluation of their characteristics, readability, suitability, and quality.

Data Collection
Resources were evaluated for readability, suitability, quality, and other characteristics (e.g., location, type).Readability is the ease with which sentences and paragraphs can be understood by the reader [27].Suitability is the level of comprehension and acceptability of written information [28].Quality refers to the integration of evidence-based self-management advice [29,30].Ten (43.5%) resources were independently rated by TS and VW, with ratings compared to establish inter-rater reliability (see details Section 2.2.5).Rating discrepancies were resolved through discussion with BS.Remaining resources were assessed by TS.

Readability Indices
Readability was assessed using the Gunning Fog Index (GFI) [31], the Simplified Measure of Gobbledygook (SMOG) [32], and the Flesch-Kincaid Grade level [33] using an online tool [34].GFI [31] and SMOG [32] estimate years of formal education required to understand the text on first reading.The Flesch-Kincaid Grade Level indicates the difficulty of a passage and the required reading grade, as per school grade levels in the United States (US) [33].Lower levels (ideally 8th grade and below) correspond to higher readability.All raw text from each resource was compiled in a Google Document [35], and manually formatted so that images, tables, and irrelevant text (e.g., hyperlinks and advertisements) were removed.The formatted text was then pasted into the readability tool [34].To account for variability in scores generated by the different measures, a mean reading grade across these three indices was calculated [36,37].

Suitability
Suitability was assessed using the Suitability Assessment of Materials (SAM) [28], a validated 22-item assessment tool [38,39].SAM appraises six categories affecting readability and comprehension [28]: content, literacy demand, graphics, layout and typography, learning stimulation/motivation, and cultural appropriateness.Each category was assessed using 2-5 items rated from 0 (not suitable) to 2 (most suitable), with higher scores representing greater suitability.Item scores for each category were summed, divided by the total possible score per category, then converted to a percentage, whereby superior = 70-100%, adequate = 40-69%, and unsuitable < 39% [28].

Resource Characteristics
Additional information about the resources, including location, responsible organisation name, organisation category (i.e., cancer support organisation, medical/research centre, professional association, health education/publishing), format (e.g., website versus online booklet/PDFs), year updated, audio-visual elements (yes/no), presence of selfmanagement skills (yes/no) and advertisements (yes/no) were extracted to determine their association with resource quality and suitability.

Bias: Inter-Rater Reliability
Intraclass Correlation Coefficient (ICC) estimates and their 95% confidence intervals were calculated on the SAM and DISCERN total scores using IBM SPSS Statistics 26 (IBM Corp., Armonk, NY, USA), based on a mean rating (k = 2), consistency, two-way randomeffects model [41].

Data Analysis
Descriptive statistics were calculated to summarise the overall readability, suitability and quality of the resources.To identify groups of resources scoring high versus low for both quality and suitability, a hierarchical cluster analysis using Ward's linkage [42,43] was performed on SAM and DISCERN scores.Characteristics of resources in each cluster were compared using Fisher's Exact Test (p < 0.05).Characteristics of resource associated with high SAM or DISCERN scores (not both as in the cluster analysis) were also assessed separately using t-tests [44].All the analyses were performed using IBM SPSS Statistics 26 (IBM Corp., Armonk, NY, USA).

Search Results
Thirty-seven online resources were identified and screened for eligibility.Fourteen were subsequently excluded for the following reasons: (1) inadequate information regarding FCR self-management (i.e., minimal or no description about how to manage FCR) (n = 8); (2) redundancy (multiple retrievals from Google using different search terms; n = 4); (3) not targeted towards cancer survivors (n = 2).Twenty-three resources were included in the final sample (see Table 2).

Resource Characteristics
Of the included resources, three quarters (n = 17, 74%) were from the US.More than half (n = 15, 65.2%) were published by national cancer organisations and less than a quarter (n = 4, 17%) by independent health organisations (e.g., Mayo Clinic Health System [45]).78% (n = 18) of resources were webpages as opposed to downloadable fact sheets or booklets, only two (9%) resources contained videos.Nine (39%) resources contained advertisements.See Appendix A for additional resource characteristics.

Readability
Readability was assessed by calculating the average of the GFI, SMOG, and Flesch-Kincaid Reading Grade Level, with higher reading grade levels indicating lower readability.Average reading grade scores ranged from grade level 9-14, which can be interpreted as ranging between secondary to tertiary school level.The mean reading grade score was grade level 11 (SD = 1.6), equivalent to secondary school.

Characteristics of Resources with Higher Suitability and Quality
There were no significant differences between organisation category and cluster (Fisher exact test, p = 0.318).Resources published in Australia (n = 4/4, 100%) were more likely to be included in the superior resource cluster compared with USA (n = 5/17, 29%) (Fisher exact test, p = 0.0211).Being published in the UK or Australia was independently associated with significantly higher mean quality (DISCERN t(21) = 2.1, p = 0.048) and suitability (SAM score t(18.08)= 3.13, p = 0.006).No significant associations were found for the other characteristics (format, year updated, audio-visual elements, presence of self-management skills, and advertisements).

Discussion
This is the first study to evaluate the suitability, quality, and readability of publicly available online resources for the self-management of FCR, despite FCR being a highly prevalent concern among cancer survivors.Twenty-three resources were evaluated, 15 of which were of higher suitability and quality (Cluster 1), and 8 were of lower suitability and quality (Cluster 2).The type of organisation did not impact on the suitability and quality of the resources.However, resources with higher suitability and quality were more likely to be published in the UK or Australia.None of the resources met the criteria for acceptable readability (8th grade and below).
The internet is a common source of cancer-related information.Despite this, our findings indicated low readability of online resources for FCR self-management.Readability of all resources was higher than the recommended 8th grade level [48].The high reading grade of the included resources, indicating more difficult text requiring higher levels of education to understand, complements a recent evaluation of publicly available resources for adult cancer survivors who have completed treatment, where the average reading level was 11 (equivalent to secondary school) with only one resource meeting the recommended 8th grade [26].These findings are also consistent with an evaluation of online resources for self-management of depression, which found that readability of the included resources required at least secondary/high school level education [30].Another study which evaluated the readability of Australian online health information (covering 12 common health conditions including cancer) found 251 web pages from 137 websites had an average reading grade of 10.5 (Flesch-Kincaid)/12.1 (SMOG), considerably higher than the average Australian reading level of grade 8 [48].Low readability of online resources increases the likelihood of individuals disregarding or misinterpreting health information, potentially leading to self-management deficiencies and worsening of health outcomes [48,49].This risk is amplified for cancer survivors of differing ethnic/cultural backgrounds, with limited fluency in the dominant language of the country they reside in, and/or who have low health literacy.Readability of online FCR self-management resources needs to be improved for these resources to benefit the broad spectrum of cancer survivors affected by FCR.
Only one resource was rated 'superior' in terms of suitability, while the rest were rated as 'adequate'.There was limited use of graphics, learning, stimulation, and motivation strategies (e.g., interactive features, behaviours modelled/specific, and motivation/selfefficacy) across resources.Similarly, limited use of graphics were found for online resources for cancer survivors, which the authors note may further compound readability issues [26].This is problematic, as these strategies are likely to enhance learning, engagement and retention of skills, which are critical to enabling self-management, [38].The resources also demonstrated limited cultural appropriateness, with low ratings regarding congruency of presented information with the logic, language, and experiences of cultural backgrounds other than the White majority.This result may be partly because we only included Englishlanguage resources in this study, which typically did not target a specific cultural group, and SAM, as a literacy assessment tool, has been criticised for its limited application when there is limited information about the intended cultural audience [36].However, it is essential that FCR self-management resources cater for people from culturally and linguistically diverse backgrounds considering the growing diversity of the many countries populations and the fact that FCR can impact Indigenous and minority people differently [50].
The quality of the evaluated resources was generally moderate, although this did not significantly differ across categories.One of the common issues diminishing quality was a limited reporting of information sources for resource content, for this also presents as a problem for online resources targeted towards cancer survivors who have completed treatment [26].Transparent attribution of information sources is a key element impacting judgements about credibility, as this enables readers to verify the information independently [51].This is a common issue with online health information generally [52,53].In this instance, it may impact on engagement with online FCR self-management resources, and if information is not evidence-based, could also lead to misleading or unhelpful recommendations.
Included resources rated slightly worse in terms of outlining FCR self-management options than presenting reliable information.Many resources failed to describe benefits and risks associated with each recommended approach to self-managing FCR, and what would happen without self-management/treatment. Similarly in the evaluation of online resources for cancer survivors who have completed treatment, there was minimal description of the risks associated with the suggested self-management strategies [26].Presenting information about risks and benefits of FCR self-management strategies is critical to informed decision-making, as sometimes self-management strategies may not benefit survivors [11] and in some cases focusing more on FCR may increase distress, especially in people who typically use avoidant coping.Describing potential outcomes of foregoing FCR self-management may motivate survivors to try self-management, considering that evidence suggests that for many cancer survivors FCR often persists and sometimes worsens if left unmanaged [4,54].
This was the first study to evaluate publicly available online self-management resources for FCR.Study strengths include use of search methodology likely to reflect common searches for online information about FCR by cancer survivors and validated tools for assessing readability, suitability, and quality.Study limitations include only resources published in English being evaluated, meaning our findings are only relevant to patients literate in English.The search for resources was conducted in 2020, which means that there may be more recently developed or updated resources that have been missed.Nonetheless, this study provides a useful framework for developing and evaluating FCR self-management resources and identifies resources of high quality and suitability for use in practice.Future research could extend our work by adopting the same approach in evaluating the suitability and quality of the FCR self-management resources published more recently and in other languages, seeing as FCR seems to affect cancer survivors regardless of where they live [2].This is particularly important, as immigrant patients, who have been found to have greater unmet needs regarding FCR, [50,55] may turn to information published in their language from their home country, as they commonly do for other forms of health information [56].While we used validated instruments to assess resource readability, suitability and quality, the DISCERN instrument used to assess quality was originally designed to assess the quality of information regarding medical treatments rather than self-management strategies, which complicated assessment of resource quality.
While we assessed resource accessibility in terms of their readability and suitability (e.g., content, layout, and typography), we did not evaluate more technical factors impacting accessibility (e.g., user interface and web functionality that contribute to ease of navigation, readability, and comprehension) [57].Future development and evaluation of online FCR self-management resources could use the Web Content Accessibility Guidelines (WCAG), which outline technical requirements and features that enable easy and effective website use [58].Although these guidelines were developed to improve web accessibility for people with disabilities, they can also be applied to ensure usability for other populations, such as people affected by cancer.Finally, there is a need for data regarding which resources are commonly recommended by oncology health professionals, used by cancer survivors and their effects on FCR to enable correlation of resource readability, suitability and quality with usage and impact.

Practical Implications
This review identifies some key areas for improvement in the readability, suitability and quality of existing online FCR self-management resources and factors for future resource developers to consider.Readability can be enhanced by using readability tools (such as the online tool used in our review) to evaluate resource content and ensure the language used fits the literacy level of the general population.Resource suitability can be improved through greater use of graphics to aid reader comprehension, incorporating learning stimulation and motivation (e.g., posing scenarios or questions for readers) to promote long-term retention of new knowledge/skills, and modelling specific self-management behaviours (e.g., instructions presented in steps or video format) to facilitate learning.Quality can be enhanced through transparent reporting of information sources (e.g., providing links to the original sources), balanced description of the benefits and risks for each self-management strategy, and clear description of potential consequences if a condition, such as FCR, is left unmanaged.Development of future resources could apply the readability, suitability (SAM) and quality (DISCERN) tools used in our evaluation to avoid identified resource shortcomings.Involving consumers in the development of resources may also help ensure that they present content that suits the needs and preferences of the target population [59].
For patients and clinicians seeking additional FCR self-management support resources, our results provide a useful guide for which resources to consider (see Table 2 for resources ranked by quality rating).Provided they can understand the resource contents, many patients and clinicians may not discern/have a preference where a resource comes from, but some may prefer local resources.It is worth noting that Australian and UK resources were generally of higher suitability and quality, while the suitability and quality of resources from the USA were more variable, so those based in the USA may wish to be more selective.High quality and suitability FCR self-management resources could also serve as a form of 'universal care' for FCR provided to all survivors as part of a stepped care approach to managing FCR, which could help prevent FCR from worsening [10].There is preliminary evidence from the Fear-Less study suggesting that stepped care models including an FCR self-management resource as the initial step can reduce FCR [11].However, it should be noted that the FCR self-management resource in the Fear-Less study was more intensive and included some therapist guidance, compared with the typically brief and entirely self-management resources evaluated here.

Conclusions
Many cancer survivors rely on the internet for self-management information and are affected by FCR.Consequently, it is essential that publicly available online FCR selfmanagement resources deliver high quality, evidence-based information that can be read and understood by the diverse spectrum of people living with and beyond cancer.There is potential for improvement of the readability, suitability and quality of existing FCR self-management resources, particularly in terms of readability, which required a much higher level of education than recommended.As well as improving existing resources, further research is needed to evaluate FCR self-management resources in languages other than English, and from non-Western countries.

Table 2 .
Characteristics of included FCR self-management resources.

Table 3 .
Mean Suitability Assessment of Materials ratings for included resources (n = 23).
Item scoring: 1 to 5; Mean: Mean of 23 resources; SD: standard deviation.The bold headings in this table help differentiate the different measure sub scales.