Comparison of eHealth Literacy Scale (eHEALS) and Digital Health Literacy Instrument (DHLI) in Assessing Electronic Health Literacy in Chinese Older Adults: A Mixed-Methods Approach

This study compared the reliability, construct validity, and respondents’ preference of the Chinese version of 8-item eHEALS (C-eHEALS) and 21-item DHLI (C-DHLI) in assessing older adults’ electronic health (eHealth) literacy using a mixed-methods approach. A web-based, cross-sectional survey was conducted among 277 Chinese older adults from September to October 2021, and 15 respondents were subsequently interviewed to understand their preference of scale to use in practice. Results showed that the internal consistency and test-retest reliability of both scales were satisfactory. For the construct validity, the C-DHLI score showed stronger positive correlations with having Internet use for health information and higher educational attainments, occupational skill levels, self-rated Internet skills, and health literacy than the C-eHEALS score. In addition, younger age, higher household income, urban residence, and longer Internet use history were only positively correlated with C-DHLI score. Qualitative data suggested that most interviewees perceived the C-DHLI as more readable than C-eHEALS for its clear structure, specific description, short sentence length, and less semantic complexity. Findings revealed that both scales are reliable tools to measure eHealth literacy among Chinese older adults, and the C-DHLI seemed to be a more valid and favored instrument for the general Chinese older population based on the quantitative and qualitative results.


Introduction
The Internet is now well integrated into our daily lives. It has become the most readily accessible resource for obtaining health information, bringing many opportunities for selfcare [1]. As the major users of healthcare services, older adults are one of the groups who are most likely to benefit from using the Internet for health information [2]. However, older adults are still one of the groups with the most difficulty in using digital health, which further exacerbates digital divide and health disparities [3,4]. Therefore, improving skills in searching for, understanding, and evaluating online health information, which can be reflected as eHealth literacy, would be particularly essential for them.
Electronic health (eHealth) literacy, first mentioned by Norman and Skinner in 2006, is defined as the ability of individuals to seek, find, understand, and appraise health information from electronic resources and apply such knowledge to addressing health problems [5]. The roles of eHealth literacy in improving or maintaining the health of the elderly have been more and more emphasized [6]. Different levels of eHealth literacy indicate that not everyone has the capacity to benefit from digital health [7], which may be a contributing factor to the health disparities. For example, older individuals who are low eHealth literacy may not able to take full advantage of the Internet in obtaining health information and support in their health, thereby solidifying the health disparities [8,9]. social structures (e.g., demographics) are connected to health disparities through the microlevel conduits, including eHealth usage and eHealth literacy [8]. A recent systematic review also showed that socio-demographic factors, such as age, educational attainment, and socioeconomic status, were important influencing factors of eHealth literacy for Chinese older adults [22]. Second, older adults are still one of the most vulnerable groups in the digital divide [7,23]. Compared with young generations, older adults reported having lower Internet use, poorer Internet use skills, and less Internet use for health information [24]. Internet use behaviors, such as Internet use history, frequency, and health-related Internet use, were found to be predictive of eHealth literacy among older adults [21,25,26]. Third, health literacy, defined as "the degree to which individuals have the capacity to obtain, process, and understand basic health information and services needed to make appropriate health decisions [27]", can be seen as an embedded core concept within eHealth literacy based on the Lily Model [5]. Therefore, these variables can be considered as the related constructs to measure older adults' eHealth literacy.
In addition, given that eHealth literacy capabilities can be seen as important personal resources, it is also necessary to understand respondents' personal preferences about the measurements. The present study will utilize qualitative interviews to determine respondents' perceptions of the two scales. Such approach can provide important information for understanding respondents' preferences regarding the scales used in research and clinical settings [28].
Based on the above, the purpose of this study was to compare the reliability, construct validity, and respondents' perceptions of the C-eHEALS and C-DHLI using a mixedmethods approach. It is expected that findings of the study would help to identify a valid and reliable instrument for assessing eHealth literacy among Chinese older adults.

Participants and Procedures
A web-based cross-sectional survey was conducted among Chinese older adults from September to October 2021. A non-probability, convenience sample of older adults above 55 years of age in 30 provinces and municipalities of mainland China was recruited. The inclusion criteria of the participants were: (1) being aged 55 or over, (2) having Internet use experience, and (3) having cognitive ability to complete a self-reported survey. Eligible participants were invited to participate in the web-based survey via social networking sites (e.g., WeChat). The online questionnaire, which included the informed consent and study questionnaire, was accessed through a hyperlink or a QR code of Sojump (a major platform for online surveys in China). Participants self-administered the online questionnaire after giving their informed consent, which took around 15 min to complete. After two weeks, approximately 20% of them were randomly selected and invited to complete another questionnaire which consisted of the two eHealth literacy scales again. No incentives were given to the participants.

Measures
Socio-demographic characteristics included the respondents' information about age, gender, types of residence, education, occupation (currently or before retirement), and monthly household income.
Internet use-related factors included respondents' Internet use history (years), selfrated Internet use skills (response options from 1 = "Poor" to 5 = "Excellent") and Internet use for health information (0 = "No", 1 = "Yes"). eHealth literacy was assessed by two instruments: the 8-item C-eHEALS and the 21-item C-DHLI. As introduced before, in the C-DHLI, respondents were asked how often they experience specific situations or how difficult they find it to perform specific tasks by searching for health information from the Internet. The 4-point Likert scale has response options ranging from "1 = very easy" to "4 = very difficult" or from "1 = never" to "4 = often". Scores need to be reversed, and a higher score represents a higher level of eHealth literacy. In this study, the three optional items about protecting privacy were not included in the validation analysis due to low completion rates [12]. Another instrument-C-eHEALS-measures one's perceived ability to use Internet for health information. Items are measured by a 5-point Likert scale with responses from "1 = strongly disagree" to "5 = strongly agree". Total scores are summed to range from 8-40, with higher scores indicating higher perceived eHealth literacy [10].
Health literacy was assessed by a 14-item health literacy scale (HLS-14) developed by M. Suka et al., which contains three dimensions: functional, interactive, and critical [29]. The response was rated on a 5-point Likert scale (1 = "Strongly disagree" to 5 = "Strongly agree", with higher scores indicating higher health literacy.

Qualitative Interviews
After the web-based survey, participants who had shown interest in taking part in the follow-up qualitative study were invited to provide their contact information. Those with various socio-demographic backgrounds were randomly selected and approached by one research team member for their consent to be interviewed. During the interviews, their perceptions and preference for the two eHealth literacy scales were asked and recorded (See Appendix A for sample interview questions). The average length of the individual interview was around 20 min.
The recruitment for interviewees stopped when data saturation emerged (i.e., where no new insights/themes/issues emerged from the interviews) [30,31]. A total of 15 respondents were finally included in the qualitative interviews.
Ethical approval was obtained from the Survey and Behavioral Research Ethics Committee of the Chinese University of Hong Kong (No. SBRE-21-0005).

Data Analysis
Descriptive statistics regarding respondents' socio-demographic characteristics, Internet use behaviors, eHealth literacy scores on two scales, and health literacy were described by mean ± standard deviation (SD) for continuous variables (normality, median and quartile for non-normality) and number (percentage) for categorical variables.

Reliability
Internal consistency of both scales was assessed by Cronbach's coefficient alpha. A threshold of 0.7 was considered acceptable, a value >0.8 indicated good internal consistency, and a value >0.9 indicated excellent internal consistency [32].
Test-retest reliability was calculated by the interclass correlation coefficient (ICC), with values less than 0.5, between 0.5 and 0.75, between 0.75 and 0.9, and greater than 0.90, indicating poor, moderate, good, and excellent reliability, respectively [33].

Construct Validity
Based on the theoretical and empirical evidence, nine a-priori hypotheses were used to evaluate the construct validity of the C-DHLI and C-eHEALS as follows. The process of construct validation test used in this study followed the methodology in examining scale construct validity used in many previous studies [34][35][36].

H1:
Older age would be negatively associated with eHealth literacy among Chinese older adults.

H2:
Higher educational attainments would be positively associated with eHealth literacy among Chinese older adults.
H3: Urban residence would be positively associated with eHealth literacy among Chinese older adults.

H4:
Higher occupational skill levels would be positively associated with eHealth literacy among Chinese older adults.

H5:
Higher household income would be positively associated with eHealth literacy among Chinese older adults.
H6: Longer Internet use history would be positively associated with eHealth literacy among Chinese older adults.
H7: Higher self-rated Internet skills would be positively associated with eHealth literacy among Chinese older adults.

H8:
Having used Internet for health information would be positively associated with eHealth literacy among Chinese older adults.

H9:
Higher health literacy would be positively associated with eHealth literacy among Chinese older adults.
One-way analyses of variance (ANOVA) were used to analyze responses for C-eHEALS and C-DHLI scores, with age category, types of residence, education, occupation, household income, Internet use history, self-rated Internet skills, and using the Internet for health information as group factors, followed by LSD post-hoc tests. In addition, Pearson or Spearman correlations were also used to test the above hypotheses for the effect size and direction.

Qualitative Data Analysis
The qualitative data were transcribed and translated into English. We used inductive thematic analysis to analyze the interview data related to the respondents' preference decisions [37]. The authors coded all transcripts and then reviewed the codes together. After the initial coding, labels were attached to text fragments that appeared to be important to the interview questions, and the analysis was carried out in an iterative fashion to develop a set of themes regarding to the interview topic (i.e., their preference between the scales). The authors compared the raw data with emerging theme label and further refined the themes by merging, adding, and removing redundant themes. The results were presented to a research team that consisted of a qualitative expert, a psychologist, and two public health postgraduate students for finalization.

Descriptive Statistics
The characteristics of respondents are presented in Table 1. A total of 277 participants were included, and 110 (39.7%) of them were 60 years old or over. Over half (51.3%) were male, and half (50.9%) lived in cities. Most of them (58.8%) had education attainments of middle school or high school. Only 18.4% had a monthly household income of less than RMB 2500. For their Internet use, 35.7% had an Internet use history of over 10 years, and most (60.3%) rated their Internet skills as "Average". 79.4% of participants had used the Internet for health information.

Reliability
Sixty-two participants completed the C-eHEALS and C-DHLI again two weeks after baseline. The internal consistency of both scales was good, with both Cronbach's alpha of 0.94. For the test-retest reliability, both C-eHEALS and C-DHLI showed excellent test-retest reliability on their total scores by 0.92 and 0.94 of ICC between baseline and two weeks later (p < 0.001).

Construct Validity
As shown in Tables 1 and 2, the results confirmed a-prior-defined hypotheses to evaluate the construct validity of C-eHEALS and C-DHLI. All hypotheses were fulfilled regarding the C-DHLI, whereas only five hypotheses (i.e., H2: education, H4: occupation, H7: self-rated Internet skills, H8: using the Internet for health information, and H9: health literacy) were fulfilled in the C-eHEALS. Table 2 presents the correlation coefficients of two eHealth literacy scales and key variables for the nine hypotheses, with all the directions as expected in both scales. For the C-DHLI, higher educational attainments (ρ = 0.479), higher household income (ρ = 0.403), higher self-rated Internet skills (ρ = 0.492), having used the Internet for health information (ρ = 0.432), and higher health literacy (ρ = 0.562) were moderately correlated with higher C-DHLI scores (p < 0.001). Specifically, the functional literacy dimension (ρ = 0.402) of health literacy showed a moderate positive correlation with C-DHLI scores but no significant correlation with C-eHEALS. Nevertheless, age (r = −0.217) showed a weak-to-moderate negative correlation with C-DHLI scores; unban residence (ρ = 0.240), higher occupational skill levels (ρ = 0.298), and longer Internet use history (ρ = 0.346) also showed a weak-to-moderate positive correlation with higher C-DHLI scores (p < 0.001). Table 2. Correlations between two eHealth literacy scales and construct variables (N = 277).

Variables
eHEALS Score DHLI Score For C-eHEALS, higher self-rated Internet skills (ρ = 0.242), having used the Internet for health information (ρ = 0.328), and higher health literacy (ρ = 0.391) were weaklyto-moderately correlated with higher C-eHEALS scores; higher educational attainments (ρ = 0.179, p < 0.01) and occupational skill levels (ρ = 0.150, p < 0.05) were weakly correlated with higher C-eHEALS scores. Other variables showed no significant correlations with C-eHEALS scores. Table 3, among the 15 interviewees, five of them (33.3%) preferred to use the C-eHEALS in practical settings, labeled A1-A5, and eight (53.3%) preferred to use the C-DHLI, labeled B6-B13. Two (13.3%) did not give an explicit choice and were labeled C14-C15. Through an iterative process, the data were saturated on two themes: item readability and content comprehensiveness, that influenced respondents' preference to use a particular scale.

Item Readability
The scale's readability is one of the most important factors affecting respondents' preference for which scale to use. For the five interviewees who preferred the C-eHEALS, three interviewees (i.e., A1, A2, and A3) stated the items in both C-eHEALS and C-DHLI were easy to read and understand. With regards to their demographic characteristics, they generally had higher education attainment, higher household income, urban residence, longer Internet use history, having used the Internet for health information, and higher C-eHEALS and C-DHLI scores. They had no difficulty answering both scales; thus, the scale's brevity (i.e., fewer item numbers) became the main reason for their choice.
"Both scales are easy to answer, but the C-DHLI with more than twice as many items as C-eHEALS seems too long and tedious. I prefer to use the shorter one (i.e., C-eHEALS)." [Interviewee A2] However, two other interviewees (i.e., A4 and A5) found both scales quite difficult to understand and answer by themselves, especially Interviewee A5 who could hardly answer both scales without the other's help. They picked the C-eHEALS only due to its fewer item numbers. They had lower education and household income, town or village residence, short Internet use history, and had never used the Internet for health information. Their eHealth literacy scores on both scales were low.
Of the eight interviewees who preferred to use the C-DHLI in practical settings, most stated the C-DHLI was easier to understand as it had more specific descriptions. Specifically, they explained that they felt the C-DHLI items described more distinct skill categories, while items of C-eHEALS require long time for them to comprehend. For example, the first three items of C-eHEALS were about "what health resources . . . ", "where to find . . . ", and "how to find . . . ". However, when they were translated into Chinese, some interviewees' first impression was that their meanings were similar, so it took more time for them to understand their differences. The same happened for item 4 (" . . . how to use the Internet to answer my health questions") and item 5 (" . . . how to use health information to help me") of the C-eHEALS.
"I have difficulty distinguishing the items of C-eHEALS and feel some of them quite similar. Thus, when answering the questions, I just roughly choose an option. But for the C-DHLI, I can tell the difference between items with specific scenarios, so I can exactly understand what this scale would like to ask." [Interviewee B10] In addition, some-particularly those with lower education (e.g., middle school) and shorter Internet use history-described less semantic complexity and shorter length of the C-DHLI's items as reducing their burden in answering the scale. Additionally, Interviewee B7 also emphasized the higher readability of C-DHLI than C-eHEALS, not only about the items but also about item options, "C-DHLI's items are shorter, easier, and more straightforward. Moreover, the answer options of C-DHLI regarding difficulty or frequency are more objective and easier to answer than eHEALS's options about the agreement." [Interviewee B7] Another two interviewees (i.e., C14 and C15) did not give an explicit preference for either scale. They explained that they rarely used the Internet for health information and eHealth literacy was unimportant in their daily life. They commented the C-DHLI is relatively easier to understand but more time-consuming.

Content Comprehensiveness
At the beginning of the interview, the definition of eHealth literacy was introduced to each interviewee. After that, they were asked to comment on the extent which the two scales have captured the concept. Interviewees mentioned that the C-DHLI provided a more comprehensive description about health-related Internet skills/contexts than the C-eHEALS.
"I think the C-DHLI covers all skills/scenarios that I would potentially encounter on the Internet, and some I have not even encountered yet . . . " [Interviewee B8] Some interviewees indicated that the clear structure of C-DHLI increased the clarity of what skills eHealth literacy aimed to measure, whereas the C-eHEALS described more about an overall perception of eHealth use.
"The subtitles of C-DHLI help improve its structure, so I can know what specific skills it would like to measure. The C-DHLI also help me understand which skills I need to improve in the future." [Interviewee B6] However, Interviewee A3 also complained about the length of C-DHLI: "I think eHEALS already reflects the definition of eHealth literacy, whereas the C-DHLI is a bit too long and may contain some useless information." [Interviewee A3]

Discussion
This study showed that both C-eHEALS and C-DHLI were reliable tools in assessing the eHealth literacy of Chinese older adults. The C-DHLI showed itself to be more valid than the C-eHEALS with better construct validity based on the nine a-priori hypotheses. Additionally, qualitative results revealed that the C-DHLI was preferred by interviewees across different backgrounds, since it was perceived as more readable (e.g., having a clear structure, specific descriptions, shorter sentence length, and less semantic complexity) than C-eHEALS. The C-eHEALS was more favored by those who can easily understand it for having fewer items (and therefore being less time-consuming) than the C-DHLI.
The quantitative results showed that both C-eHEALS and C-DHLI had good internal consistency and test-retest reliability in Chinese older adults. The result was consistent with the findings of a Korean study which validated the Korean version of DHLI and eHEALS in older adults (Cronbach's alpha K-DHLI: α = 0.93, K-eHEALS: α = 0.90; ICC K-DHLI: 0.844, K-eHEALS: 0.769) [13]. As found in previous studies, eHEALS demonstrated good readability in older adults in Iran, the US, and Norway [38][39][40][41], with Cronbach's α ranging from 0.85 to 0.99. As the DHLI was developed in 2017 and has only been validated in older adults in South Korea, this study expands evidence that the DHLI is an internally consistent and temporally stable measure for the elderly's eHealth literacy assessment in the Chinese context. Researchers and healthcare professionals can consider both C-eHEALS and C-DHLI to assess Chinese older adults' ability to utilize online health information for health purposes in practice settings.
Overall, the C-DHLI demonstrated better construct validity than the C-eHEALS in the older Chinese population. Specifically, all nine hypotheses were fulfilled in the C-DHLI, and five of them were fulfilled in the C-eHEALS. In addition, for the five hypotheses fulfilled in both scales, the correlations between C-DHLI and the hypothesized factors (i.e., education, occupation, self-rated Internet skills, using the Internet for health information, and health literacy) were stronger than such with the C-eHEALS scores. As stated before, the disparities in social structure, digital divide, and health literacy are related to older adults' eHealth literacy, which would further exacerbate health disparities [8]. Based on the results, the C-DHLI can better reflect the impact of such disparities on eHealth literacy among Chinese older adults compared with the C-eHEALS. Some possible explanations for the findings were proposed: First, the C-eHEALS measures one's overall perception of eHealth use by eight items, while the C-DHLI contains seven specific skill categories on eHealth use, with twenty-one items in total. It was found that multi-item measures were likely to be more reliable and valid (e.g., [42]). As eHealth literacy is a multi-dimensional construct, the overall measure (i.e., C-eHEALS) may be less significant in construct validity than the multi-dimensional measure (i.e., C-DHLI). Second, the C-DHLI is developed based on one's actual performance on eHealth usage. Hence, its items are specific skill orientations (e.g., operational skills and information evaluation) closely related to older adults' basic knowledge and skills. Thus, the C-DHLI may demonstrate stronger correlations with the hypothesized construct than the C-eHEALS, which emphasizes one's self-efficacy in eHealth usage. This point was further demonstrated in the present study from the ninth construct hypothesis regarding health literacy. Functional literacy, as one dimension of health literacy, reflects the basic reading and understanding skills of health information [43]. This study found functional literacy was moderately correlated with the C-DHLI score (r = 0.402, p < 0.001) while not significantly associated with the C-eHEALS score. Some previous studies have criticized the weak relationship between one's self-efficacy and actual performance on eHealth use [16,17]. Although eHEALS has been validated in older populations, its validity may warrant more discussions.
The results from the qualitative data confirm that a clear difference of eHealth literacy levels exists within older individuals, potentially influenced by their socio-demographic characteristics and Internet use history/skills [21]. Respondents' perceptions of the scale are also influenced by their background, especially their education levels, affecting their preference in choosing eHealth literacy assessment in practice. Based on the qualitative results, the C-eHEALS may be preferred by those (especially those with higher educational attainments and SES) who can easily understand both scales. They tend to choose the one with fewer items (i.e., C-eHEALS) since it takes less time to complete. However, most interviewees (across different background levels) preferred to use the C-DHLI, as they perceived it easier to understand and more comprehensive than the C-eHEALS. As many older adults are beginners to health technology, the clear structure and specific skill/scenario description of the C-DHLI can help them navigate through the scale. Moreover, they also found that shorter sentence length and lower semantic complexity of its items decreased their response burden. The results suggest that respondents would prefer to use a scale which is perceived to be more readable. The qualitative findings may indicate that the C-eHEALS can be applied to certain groups of elderly, such as those with higher education and SES, but the C-DHLI is more acceptable to the general older population.
This study confirmed the reliability and validity of the two eHealth literacy instruments in Chinese older adults. However, as mentioned before, as the digital health technologies have advanced a lot in recent years (e.g., the popularity of social media), refinement of the instrument is therefore warranted, for example, to capture the characteristics of online interactions. In addition, the C-DHLI, as a newly developed instrument, has been translated into Chinese in 2021 by our research team and validated in 277 Chinese older adults for the first time. More evidence regarding its psychometric properties in larger and more representative samples in China is also needed. There are several limitations of the study that should also be noted. First, the present study was a web-based survey with the sample being Chinese older adults who voluntarily participated in this study. Selection bias may exist as the participants were older adults who had experience with the Internet and were more likely to be interested in accessing health information online. Second, reporting bias (e.g., social desirability and recall bias) may exist as the questionnaire was self-reported and self-administered. Third, nine a-priori hypotheses were used to examine the construct validity in the present study, which is often considered less powerful than criterion validation. However, with strong theories and specific expectations, it is possible to acquire substantial evidence that the measurement instrument is measuring what it purports to measure [44].

Conclusions
To conclude, the Chinese versions of eHEALS and DHLI were reliable in assessing eHealth literacy, but the C-DHLI was more valid than the C-eHEALS among Chinese older adults. In addition, qualitative results showed that the C-DHLI was more favored by respondents with different backgrounds, due to its clear structure, specific descriptions, shorter sentence length, and lesser semantic complexity. The C-eHEALS was preferred by people who can easily understand it due to its brevity (i.e., fewer item numbers). The findings suggest that the C-DHLI may be more applicable to general older populations in China for eHealth literacy assessment; the C-eHEALS can be considered in certain elderly groups (e.g., those with higher education or SES) in daily practice. In addition, the refinement of the eHEALS is still strongly recommended to capture the evolution of digital health technologies over time.  Informed Consent Statement: Informed consent was obtained from all subjects involved in the study.