Trust in and Use of COVID-19 Information Sources Differs by Health Literacy among College Students

People’s health information-seeking behaviors differ by their health literacy levels. This study assessed the relationship between health literacy and college students’ levels of trust in and use of a range of health information sources of COVID-19. We collected data from August to December 2020 among college students (n = 763) through an online survey. We used a health literacy measure containing three self-reported survey questions, developed by the CDC. We assessed the extent to which participants trusted and used any of the sixteen different sources of information about COVID-19. Respondents reported high levels of trusting and using COVID-19 information from the CDC, health care providers, the WHO, state/county/city health departments, and official government websites when compared to other sources. After controlling for demographic characteristics (i.e., gender, age, race, ethnicity, and income), those who reported having lower health literacy were significantly less likely to trust and use COVID-19 information from these health authorities when compared to participants who reported having higher health literacy. Students with lower self-reported health literacy indicated not trusting or using official health authority sources for COVID-19 information. Relying on low-quality information sources could create and reinforce people’s misperceptions regarding the virus, leading to low compliance with COVID-19-related public health measures and poor health outcomes.


Introduction
The regular and comprehensive communication of public health information is an important step in containing the spread of COVID-19, protecting people at risk of serious complications or death, and reducing the burden on the health system [1]. Diffusion of knowledge, especially about the COVID-19 virus, is also critical to promoting social, political, and economic development [2]. The COVID-19 pandemic created an infodemic [3], where a vast amount of information, including misinformation and disinformation, spread rapidly and impeded effective crisis management [4,5]. Misinformation refers to false information that is created and spread, regardless of an intent to harm or deceive, and How does health literacy play a role in our participants' trust in and use of a range of potential health information sources for COVID-19?

Procedures and Participants
Data for the present analyses were derived from a larger cross-sectional online survey study, designed to investigate college students' experiences during the COVID-19 pandemic [20]. Recruitment and data collection, using Qualtrics, were conducted from August 2020 to December 2020. The larger study was approved by Oklahoma State University Institutional Review Board. Detailed procedures are reported by McMaughan et al. [20]. Of the 849 students who submitted survey responses, 72 were dropped for straight line, missing, speedy or fake responses. An additional 14 were dropped due to failing the validation item associated with the trust in sources section, resulting in a final sample of 763 participants in our data analyses for this current study.

Health Literacy
We used a health literacy measure developed by the Centers for Disease Control and Prevention (CDC) in the 2016 Behavioral Risk Factor Surveillance System (BRFSS) [21]. The measure contains three self-reported survey questions: (1) assessing individuals' abilities to find information, "How difficult is it for you to get advice or information about health or medical topics if you needed it?"; (2) understanding oral information, "How difficult is it for you to understand information that doctors, nurses, and other health professionals tell you?"; (3) understanding written information, "You can find written information about health on the Internet, in newspapers and magazines, and in brochures in the doctor's office and clinic. In general, how difficult is it for you to understand written health information?". Each item was assessed on a 4-point Likert type scale ranging from 1 = "very difficult" to 4 = "very easy". We calculated the sum score, with possible scores ranging from 3 to 12. A higher score indicated that the participant reported having lower health literacy. Furthermore, to identify the individuals who were at high risk of having a low health literacy, we defined low health literacy as a response of "very difficult" or "difficult" to at least one of these three questions [22]. In addition, the item assessing an individual's ability to find information included a response of "I don't look for health information", and the item assessing the understanding of written information included a response of "I don't pay attention to written health information". Both responses were coded as 0.

Information Trust and Use
We also assessed the extent to which participants trusted and used sixteen sources (see Table 1) of information about COVID-19.

Demographics
Demographics included gender, age, ethnicity, race, and annual family income. Race and ethnicity were determined by asking participants to select from a list of racial (i.e., White, Black or African American, American Indian or Alaskan Native, Asian, and Native Hawaiian or Pacific Islander) and ethnic (i.e., Hispanic, Latinx, or of Spanish origin) categories, those which best matched their own identity. Multiple options could be selected. Gender included three options: male (including transgender men), female (including transgender women), and self-described (non-binary, gender-fluid, agender, etc.), with an option to provide an open answer. Participants were asked to identify their annual family income through the following options: less than $ (U.S. Dollar) 20,000, $20,000-$34,999, $35,000-$49,999, $50,000-$74,999, $75,000-$99,999, and over $100,000.

Data Analysis
Individuals who chose either "I don't look for health information" or "I don't pay attention to written health information" were treated separately from those who selfreported their levels of difficulty in finding information, understanding oral information, and understanding written information. We performed independent t-tests to compare trust in and use of different sources for COVID-19 information, between those who chose either "I don't look for health information" or "I don't pay attention to written health information", and those who self-reported their levels of difficulty in finding information, understanding oral information, and understanding written information.
Among participants who indicated levels of difficulty in finding information, understanding oral information, and understanding written information, we investigated the relationship between their self-reported health literacy capacity (independent variable) and their trust in and use of each source of COVID-19 information (outcome variable), and we performed bivariate linear regressions (without covariates) and multiple linear regressions (controlling for demographic characteristics). We included the demographic variables (gender, age, ethnicity, race, and annual family income) in our multiple linear regression analysis as covariates, because these demographic variables are associated with health literacy based on prior research [23]. We conducted separate regressions for each COVID-19 health information source, in terms of participants' trust in and use of them. We used Stata 16 for statistical analysis. The significance level was set at α = 0.05.
A total of 87 participants (11.40%) chose either "I don't look for health information" (54 participants) or "I don't pay attention to written health information" (48 participants).
Among those who indicated their levels of difficulty in finding information, understanding oral information, and understanding written information (n = 676, 88.60%), their health literacy scores ranged from 5 to 12 (M = 9.75, SD = 1.56). The health literacy score data were moderately skewed towards the high end, which indicated that most of our participants reported having adequate health literacy (n = 526, 68.94%). About 19.66% of our participants (n = 150) were at a high risk of having low health literacy (i.e., provided a response of "very difficult" or "difficult" to at least one of the three health literacy questions).  We observed the same patterns among our participants who were at a high risk of having low health literacy (n = 150), as well as among those who chose either "I don't look for health information" or "I don't pay attention to written health information" (n = 87). However, we noticed that those who reported being at risk of having low health literacy had a lower levels of trust in various sources when compared to those who reported having adequate health literacy. Those who chose either "I don't look for health information" or "I don't pay attention to written health information" had the lowest levels of trust for various sources among these three groups.

Use
Use of a source was positively associated with trust in that source across all the sixteen sources (all p < 0.001). Among those who indicated their levels of difficulty in finding information, understanding oral information, and understanding written information (n = 676), the most commonly used sources (used more than We observed the same patterns among our participants who reported being at a high risk of having low health literacy, as well as among those who chose either "I don't look for health information" or "I don't pay attention to written health information". However, we noticed that those who reported being at risk of having low health literacy used various sources less frequently when compared to those who reported having adequate health literacy. Those who chose either "I don't look for health information" or "I don't pay attention to written health information" had the lowest frequency of using various sources among these three groups. Among those who reported their health literacy (n = 676), our unadjusted linear regression models indicated that students who reported having lower health literacy were more likely to trust COVID-19 information from social media (b = −0.04, p = 0.006) when compared to those who reported having higher health literacy. However, they were less likely to trust coworkers (b = 0.04, p = 0.022), (former) President Trump (b = 0.06, p = 0.010), and radio and podcasts (b = 0.04, p = 0.022).
As shown in Table 3, after controlling for demographic characteristics (i.e., gender, age, race, ethnicity, and income), students who reported having lower health literacy were also less likely to trust COVID-19 information from family members (b = 0.06, p = 0.005), coworkers (b = 0.04, p = 0.009), (former) President Trump (b = 0.05, p = 0.019), and radio and podcasts (b = 0.04, p = 0.018) when compared to those who reported having higher health literacy. Our independent t-test results indicated that those who chose either "I don't look for health information" or "I don't pay attention to written health information" had significantly lower trust in doctors and other health care providers (p = 0.038), official government websites (p = 0.043), and the CDC (p = 0.006) for COVID-19 information when compared to those who reported their levels of difficulty in finding information, understanding oral information, and understanding written information.

Use
Among those who reported their health literacy (n = 676), our unadjusted linear regression models indicated that students who reported having lower health literacy more frequently used social media (b = −0.13, p = 0.001) as sources of COVID-19 information than students who reported having higher health literacy. However, when compared to people who reported having higher health literacy, those who reported having lower health literacy less frequently used health care providers (b = 0.09, p = 0.027) as sources of health information about COVID-19.
As shown in Table 3, after controlling for demographic characteristics (i.e., gender, age, race, ethnicity, income), students who reported having lower health literacy more frequently used social media (b = −0.09, p = 0.032) as sources of COVID-19 information than students who reported having higher health literacy. However, those reporting a lower health literacy capacity were less likely to use health care providers (b = 0.12, p = 0.005), and the CDC (b = 0.11, p = 0.016) to obtain information about COVID-19 when compared to participants who reported having higher health literacy.
Our independent t-test results indicated that those who chose either "I don't look for health information" or "I don't pay attention to written health information" had a significantly lower frequency of using magazines and newspapers (p = 0.034), doctors and other health care providers (p = 0.002), official government websites (p = 0.002), the WHO (p = 0.031), the CDC (p = 0.018), and state/county/city health departments (p = 0.017) as sources of COVID-19 information when compared to those who reported their levels of difficulty in finding information, understanding oral information, and understanding written information.

Discussion
In our study of college students during the COVID-19 pandemic, we found that college students who chose either "I don't look for health information" or "I don't pay attention to written health information" had significantly lower trust in and use of official health authority sources (e.g., health care providers, official government websites, and the CDC) of COVID-19 information when compared to students who reported their levels of difficulty in finding information, understanding oral information, and understanding written information. Additionally, we also found that those who reported lower health literacy had a lower frequency of using official health authority sources (e.g., health care providers and the CDC) of COVID-19 information when compared to students who reported higher health literacy; however, students who reported lower health literacy used social media for COVID-19 information more frequently. This pattern of information-seeking behavior is similar to a previous study investigating the relationship between health literacy and use of and trust in sources for general health information, which found that lower health literacy was associated with lower odds of trusting in specialist doctors and dentists for health information, as well as using medical websites, but higher odds of using social media, among a U.S. nationally representative adult sample [13]. Similarly, a recent study conducted among a German-speaking adult population of Switzerland also found that those with lower health literacy tended to not trust in, and more rarely use, health professionals and health authorities as sources of COVID-19 information when compared to those with higher health literacy [24].
Several factors might help explain why people with low health literacy neither trust nor use health professionals and health authorities for COVID-19 information gathering. First, people with lower health literacy have difficulty understanding physician instructions and have negative perceptions of their healthcare experience, such as receiving inadequate health information, which contributes to a low trust in health care providers [13,25,26]. In fact, many people with low health literacy work hard to hide the fact that they have difficulty understanding oral or written instructions from health care providers; in addition, people with low health literacy often do not have a regular health care provider [27,28].
These factors create a challenge for health care providers in gaining trust among those with low health literacy and providing health recommendations.
The second possible explanation as to why people with low health literacy neither trust nor use health professionals and health authorities for COVID-19 information is that individuals with low health literacy have low trust in scientists, especially during the COVID-19 pandemic [29,30]. For example, people with low health literacy tend to not embrace the COVID-19 vaccine, due to a lack of trust in the government and scientists that stems from the uncertain attitudes towards possible herd immunity, the complexity of the scientific and political discourse surrounding COVID-19 vaccines, as well as questions about the vaccination effectiveness as new variants keep evolving [29]. Additionally, official rules and recommendations for preventive measures (e.g., mask mandates) based on scientific findings and clinical trials keep changing, because of the evolving knowledge about how this virus is behaving [30], which leads to negative feelings, such as being overwhelmed, confused, upset, and scared [8]. In fact, people with lower health literacy are more likely to avoid health information related to COVID-19, in order to reduce the above unwanted emotions [9].
Third, the level of trust in the CDC declined significantly during the COVID-19 pandemic, and many people view the CDC as strongly politized [31]. Another study also found that about 40 percent of adults in the U.S. felt that the CDC was paying too much attention to politics when issuing guidelines and recommendations for the COVID-19 policy [32]. Such distrust in the government stimulates the spread of fake news and mis/disinformation [33,34], which further reduces the trust in COVID-19 health information from government sources. Our current study finding indicates that college students with low health literacy have had an especially low trust in the government (e.g., the CDC) during this pandemic.
Several strategies can help to increase trust in health care providers and government agencies among people with low health literacy. First, it is critical to provide easy-tounderstand health information, with plain, jargon-free language through various channels, in order to meet people where they are. Second, health care providers should use the teachback technique with their low health literacy patients, in order to ensure these patients understand the information and instructions they have been given. When applying the teach-back technique, health care providers ask their patients to repeat, in their own words, what they have been told, using a caring tone of voice and attitude to create a "shame-free" environment for patients [35]. Third, collaborating closely with community organizations and social workers to create health messages with no political interference is another effective strategy for government agencies to utilize, in order to build trust in the public [29,36]. Another strategy for government agencies to maintain public trust is to ensure clear information and unambiguous health instructions related to COVID-19 that represent government transparency and effective communication [37]. Lastly, a prior study pointed out that the CDC should create COVID-19 health information with a clearer and more explicit focus on the science, and should provide rationales for any decision made regarding the guidelines and recommendations of COVID-19 policy [32].
Moreover, we found that college students who reported having low health literacy tended to use various sources (including health care providers and public health authorities, such as the CDC, the WHO, state/county/city health departments, and official government websites) less frequently, and trusted them less when compared to those who reported having an adequate health literacy capacity, which is consistent with a recent study [24]. This finding indicates that people with a higher health literacy might compare information among different sources, in order to check for trustworthiness, while those with a lower health literacy do not [24]. Furthermore, in one of our recent studies, we found that college students with lower health literacy were more likely to intentionally avoid information about COVID-19 [9]. Our findings confirm that college students with low health literacy are at risk of COVID-19 knowledge deficits, due to their high information avoidance. They might not learn about the most important preventive behaviors and the value of vaccinations, which could lead to compliance violations and vaccine hesitancy. Our findings also indicate that there is a critical need for higher education institutions to create learning outcomes and to provide training, in order to enhance health literacy skills among college students [38].
We found that health care providers, the CDC, the WHO, state/county/city health departments, and official government websites were frequently used and highly trusted sources of COVID-19 information among college students. This finding aligns with a previous study, which reported that the U.S. adult population used and trusted health care providers the most for general health information [13]. Another study also reported that doctors and government health agencies were the most trusted source, among a U.S. nationally representative adult sample, of general health information [39]. Social media had low trust among our participants, which is consistent with a recent study concluding that people had low trust towards social media as a source of COVID-19 information [24], as well with as a previous study, reporting that social media received low trust among the U.S. adult population as a source of general health information [13]. We also found that (former) President Trump was one of the least used and trusted sources for COVID-19 information. These results were similar to those reported in our recent qualitative study, which recruited participants from the same university, and found that college students identified various pieces COVID-19 misinformation, especially from social media and politicians [8].
However, COVID-19 related information-seeking behavior patterns might be different across countries. For example, a recent study conducted by De Gani's team among 1012 participants (age mean = 46.2, SD = 16.9) living in the German-speaking part of Switzerland reported that television and the Internet were the most used information sources for COVID-19; health authorities and health professionals were used by less than half of the respondents, but these two sources were reported as being highly trusted [24]. Although we collected our data at the same time as De Gani's team did, we found that television was not a frequently used source among our college participants, and they used health authority-and health professional-related sources frequently. These two study samples consisted of different age groups, however, which might contribute to such differences, as age is a significant predictor of information-seeking behaviors [40]. Another possible explanation is that COVID-19 information dissemination and mitigation strategies are used differently across countries [41].
Interestingly, after adjusting for gender, age, race, ethnicity, and income, the association between lower health literacy and higher trust in social media for COVID-19 information became non-significant. The 2007 HINTS data indicated that younger adults had a higher level of trust in online health information, regardless of the information quality; one possible explanation is that young adults perceive themselves as being less vulnerable to low-quality health information, due to this age group being generally healthier than older adults [42]. Therefore, our finding might also indicate that age plays a more critical role than health literacy levels in terms of trust in social media.
Social media is one of the least trusted sources for COVID-19 information among our participants; however, we found that students who reported having a lower health literacy more frequently used social media as sources of COVID-19 information than students who reported having higher health literacy. Social media can play a positive or negative role in providing health information related to COVID-19. Social media can be used to assist in seeking, understanding, and sharing health information; however, the quality and accuracy of the health information from social media need to be evaluated cautiously [43]. On one hand, social media helps to reduce social isolation and improve mental health, as it provides a connection between people and their peers, friends, and family [44]. On the other hand, COVID-19 misinformation is more likely to be shared on social media than official sources [45], and people experience negative feelings, such as fear and confusion, when they identify various pieces of misinformation related to COVID-19 on social media [8].

Limitations
Although this study reported how health literacy plays a role in COVID-19 informationseeking behaviors, it has some limitations. First, the cross-sectional survey design of this study limits our ability to infer causal relationships. Second, different types of health literacy measures are associated with people's patterns of information source usage [46]. There are many different instruments used to measure individuals' health literacy [47]. Moreover, some studies have developed health literacy measures that are specific to COVID-19 [24,46]. Due to the fact that people have been constantly exposed to COVID-19-related information during this pandemic, they tend to have higher COVID-19-specific health literacy scores than their general health literacy scores [24]. Different measures of health literacy could produce different results, as different measures may assess slightly different skills [47]. Third, our convenience sampling method and relatively small sample size temper our ability to generalize our findings to the entire U.S. or other countries. Lastly, our COVID-19 information source list was not exhaustive, although the sources used here have been shown to be the most frequently used in many other studies.

Conclusions
This study makes an important contribution to our understanding of the patterns in information source preferences, specific to COVID-19, among college students attending a land-grant university located in the South-Central region of the U.S., with variations in self-reported health literacy levels. We found that college students who reported having lower health literacy reported lower levels of using and trusting official sources for COVID-19 information when compared to their peers who reported having higher health literacy. Relying on low-quality information sources and misinformation could create and reinforce people's misperceptions regarding the virus, and further lead to less compliance with COVID-19-related public health measures.