Developing a Short Assessment of Environmental Health Literacy (SA-EHL)

Environmental health literacy (EHL) is defined as the understanding of how the environment can impact human health, yet there are few tools to quantify EHL. We adapted the Short Assessment of Health Literacy (SAHL) to create the Short Assessment of Environmental Health Literacy (SA-EHL). Using the Amazon mTurk platform, users (n = 864) completed the 18-item SAHL and the 17-item SA-EHL. The SA-EHL was originally tested with 30 items; 13 items were removed because they were outside the acceptable difficulty parameters (DIFF: −0.4–4.0) or because of limited variance (>90% correct or incorrect), resulting in the final 17 items. Overall, participants scored highly on the SAHL, with 89.9% exhibiting high literacy. In contrast, the majority had low EHL (<1.0% high literacy, 99.2% low literacy) measured by the SA-EHL. The two scales were not correlated with each other (R2 = 0.013) as measured via linear regression and dichotomous variables. Scores on the SAHL and the SA-EHL were positively correlated with education. The SAHL was positively correlated with age, gender and marital status, whereas the SA-EHL was not. The SA-EHL can be used to gauge EHL for communities, and the results used to improve interventions and research translation materials.


Introduction
Environmental health literacy is most simply defined as the understanding that environmental exposures can impact health [1,2]. The discipline of environmental health incorporates public health, chemistry, environmental science, toxicology, risk communication and biology [1][2][3]. The concept of EHL follows that of health literacy: as knowledge is gained, individuals can identify relevant information, apply said information to their situation and, therefore, make informed decisions [1,2].
The conceptual framework posited for EHL uses a modified version of Bloom's Taxonomy, organizing levels of literacy across the following categories: Recognize, Understand, Analyze, Apply, Evaluate and Create [1]. Thus, a learner recognizes important environmental health concepts, progresses to understanding them, and then uses this foundational knowledge to apply to their own situation and ultimately incorporate intervention strategies and behavior changes that mitigate the environmental hazard [1,3]. A second conceptual framework specifies three dimensions of EHL, beginning with awareness and knowledge, progressing to skills and self-efficacy, and culminating in community change [4]. Both of these frameworks are compatible and reflect individual-versus population-level changes in practices or policies that would reduce the burden of environmentally mediated disabilities or diseases.
However, there are few standardized methods to assess changes in an individual-level EHL. Evaluating individual shifts is necessary to track the adoption of environmental health knowledge within populations because increased EHL is linked with improved decision-making at both the individual and community level [3,4]. Furthermore, the ability to discern individual shifts within specific socio-demographic groups can inform

Materials and Methods
We developed the SA-EHL to be in a similar format as the SAHL, a validated tool for evaluating health literacy. We developed the SA-EHL scale as per the guidelines of DeVellis [16]. In short, we: (i) generated a comprehensive list of items; (ii) reviewed items with experts; (iii) piloted the items using an online, convenience sample of adults in the United States (US); (iv) evaluated and revised items; and (v) optimized the scale. Finally, we administered the revised scale to a larger sample of adults in the US (Supplemental Information).

Scale Development, Item Generation
Environmental health literacy was the construct to be measured, with a focus on word recognition, and was developed to be delivered in the same manner as the SAHL. We developed the item list following a review of terms commonly used in environmental health (review of commonly used words in environmental health papers, presentations and websites), with a focus on words used to communicate research on air quality, water quality and environmental pollution. The SA-EHL was initially developed as a 30-item scale. One point was awarded for each correct response.

Expert Review and Inclusion of Validation Items
The SA-EHL was reviewed by experts in environmental health in the College of Public Health and Human Sciences at Oregon State University, and the Oregon Health Authority. These experts either work with programs to improve environmental health via communityengaged research or report-back on research results, science communication and public health messaging. The experts individually generated lists of items that were commonly used in their communication materials with the lay public (well water contamination, air contamination and toxic waste) and identified terms specific to environmental health. The lists were compared, and a short list was generated. The decision was made to focus on words that are foundational to the discipline, which relies on identifying and detecting hazards in environmental media, characterizing the dose or concentration of said hazards, determining how individuals are exposed and applying numerical calculations to estimate dose. For each item, our experts considered the relevance of the item to the construct, the clarity of the item to the accurate response, the appropriateness of the distractors and common use of the term in the field of environmental health.

Data Collection and Validation
This study used a convenience sample of adults drawn from across the U.S. via the Amazon Mechanical Turk (mTurk) online Web-based platform. This platform was used to collect socio-demographic data, and administered the SAHL and the SA-EHL. mTURK facilitates recruitment of adults (age 18+) to complete human intelligence tasks (HITS). These HITS are simple, designed to be quick and involve either human coding or interpretation of images, questions, etc. Upon successful completion of each HIT, mTurk participants receive Amazon credit. Restricting analysis to mTurk workers in the US, the sample contains a higher percentage of female, white, young and ideologically liberal individuals, relative to the general US populace [17][18][19]. However, prior research has found substantial benefits in using the mTurk platform, specifically regarding the rapid recruitment of large populations, increased compliance with study protocols and the ability to produce valid and reliable data, specifically in relationship to health literacy measures [18][19][20].
Recruitment followed a two-stage process. In the first stage, interested participants completed a pre-survey to ensure attentiveness to questions and to ensure responses were not automated. Questions included age, gender, zip code and an attention filter question. Participants who completed the pre-survey received USD 0.03 Amazon credit as per mTurk compensation guidelines. Participants who successfully completed the pre-survey were invited to the full survey via email with a unique URL. We pilot tested the protocol and revised the SA-EHL before collecting a larger sample.
A total of 166 participants piloted the survey between 24 and 27 October 2016, which included the 30-item SA-EHL scale with 2 distractors per item. A distribution of the responses for each item was reviewed. Specifically, the distractors were assessed for lack of response or majority incorrect answers, to select the most appropriate distractor. A single distractor was chosen for each item, but scale items were unchanged. Data collected from this pilot test were used to inform the scale development, but not scale validation, and are not included in this manuscript.
The revised 30-item SA-EHL was disseminated via mTurk from 21 to 25 November 2016. As previously described [8], a total of 1168 initiated the filter survey and 1153 completed the filter and were invited to the full survey. A total of 869 individuals completed the questionnaire. Full demographic information is available in Table 1. The complete survey took approximately 20 min to complete, and compensation was offered at USD 1.00 via Amazon credit.

Short Assessment of Health Literacy (SAHL)
The SAHL is an 18-point scale, with each correct response awarded one point [11]. Higher scales are related to health literacy [11]. Each item is accompanied by one correct response, one distractor and the option "Don't know" [11]. Table 2 contains the items, responses and distribution of responses. Table 2. List of SAHL and SA-EHL items (left column) and possible responses (correct response, distractor, "Don't know"/skipped).

Short Assessment of Environmental Health Literacy (SA-EHL)
The Short Assessment of Environmental Health Literacy began as a 30-item scale, where each correct response is awarded one point ( Table 2). Higher scores are related to higher environmental health literacy. Table 2 contains the full list of items initially selected for specificity to environmental health, along with the correct response and the distractor, and the distribution of responses.
The responses were tabulated, and items were eliminated if they met either of the following criteria: (i) >90% correct (indicating the item was too easy); (ii) <10% correct (indicating the item was too difficult); or (iii) >10% "Don't know" (indicating the item was either too difficult or the word was unknown). Nine items meeting one or more of these criteria were eliminated due to limited variance ( Table 2).
Item Response Theory Item response theory was conducted on both the SAHL and the SA-EHL. In this population, the DIFF for the SAHL fell between −4.0 and 4.0. We applied this inclusion range to the SA-EHL, thereby eliminating an additional four items. The remaining 17 items were used for all further analysis.

Statistical Analysis
Univariate analyses and histograms were used to evaluate the distribution of responses for each scale. Scale diagnostics and psychometric properties were examined for both the SAHL and SA-EHL scales by performing a factor analysis, reviewing a scree plot and calculating the internal consistency. Factor analysis incorporated an orthogonal varimax rotation. Factors were retained with Eigenvalues greater than 1. Next, statistical tests assessed criterion-related validity between the SAHL and SA-EHL. Kappa statistics, correlations, regression coefficients and Bland-Altman plots compared the relationship between the absolute values. Ranking ability was tested by reporting the percent agreement of the two measures that fell within the same binary or tertile cutoffs. Lastly, we tested construct validity by comparing absolute scores on SA-EHL between socio-demographics (i.e., education) theoretically related to environmental health literacy. Bi-variate analyses included t-tests, chi-square and ANOVA depending on the format of the variables. All analyses were conducted using StataSE version 13 (StataCorp, College Station, TX, USA).

Human Subjects' Protection
As previously described [8], participants provided consent online prior to beginning the questionnaire. Consent was obtained first for the screener survey, and again for the full questionnaire. All study activities and data collection procedures were approved by the Institutional Review Board of Oregon State University (IRB protocol #7622).

Results
A full description of participant demographics was described previously [8] and is included in Table 1. A total of 911 individuals initiated the full questionnaire with 869 (95%) completing the full survey, and 864 completing both the SAHL and the SA-EHL. The sample was predominantly white (85.2%), with 6.3% reporting as Black or African American, 4.9% as Asian or Asian Indian, 2.2% as multi-racial, less than 1% as American Indian or Alaska Native, and 2% preferred to not answer or left the question blank.

SAHL Univariate and Scale Diagnosis
Responses to the SAHL were evaluated as previously described [11]. Table 2 reports the original SAHL items and wording. Percent of correct responses for the items ranged from 2.4% to 97.5%. The mean score was 15.7 (SD = 1.90) and the median score was 16.0. A histogram is shown in Figure 1A, showing a left-skewed distribution. Cronbach's coefficient alpha was 0.8041. A one-factor analysis was conducted, and only one factor was identified with an Eigenvalue greater than one. A scree plot (Supplemental Figure S1A) shows the drop in Eigenvalue magnitude after the first factor. Continuous scores were separated using a binary scale to delineate low (score of 0-13) and high (score of [14][15][16][17][18] literacy. The majority of participants had high literacy (90%) compared to low literacy (10%). To compare ranking ability between the two scales, tertiles were created for SAHLE (0-6, 7-12 and 13-18). Similarly, evaluation of the SAHL via three categories (low, medium and high) identified a majority of participants with high literacy (Table 3).
identified with an Eigenvalue greater than one. A scree plot (Supplemental Figure S1A) shows the drop in Eigenvalue magnitude after the first factor. Continuous scores were separated using a binary scale to delineate low (score of 0-13) and high (score of [14][15][16][17][18] literacy. The majority of participants had high literacy (90%) compared to low literacy (10%). To compare ranking ability between the two scales, tertiles were created for SAHLE (0-6, 7-12 and 13-18). Similarly, evaluation of the SAHL via three categories (low, medium and high) identified a majority of participants with high literacy (Table 3).    1 The SAHL method uses a binary scale where scores from 0 to 13 are considered low literacy and scores 14 to 18 are considered high literacy. Agreement between the two measures dichotomized was low with kappa statistic = 0.0018, 10.8% agreement and p = 0.18. 2 The SA-EHL scale was divided into tertiles (0-6, 7-12 and 13-17) and applied to the responses. Agreement between the two measures as tertiles was low with kappa statistic = 0.005, 7.6% agreement and p = 0.14.

SA-EHL Univariate and Scale Diagnosis
The full 30 items and response options are listed in Table 2. However, all scale diagnostics, criterion and construct validity analyses included only the final 17 items in the scale. The percent of correct responses for the final items (n = 17) ranged from 15.2% to 83.1%. The mean overall score was 7.8 (2.2 SD) and the median was 8.0. A histogram is shown in Figure 1B, showing a normal distribution. Cronbach's coefficient alpha was 0.56. Factor analysis was conducted and a scree plot (Supplemental Figure S1B) shows the drop in Eigenvalue magnitude after the first factor, although four factors were >1 Eigenvalue. As with the SAHL, a one-factor analysis was conducted. In order to compare cutoffs with SAHLE, continuous scores were separated using a binary scale to delineate low (score of 0-13) and high (score of 14-18) literacy) and tertiles for low (0-6), medium (7)(8)(9)(10)(11)(12), and high (13-17) levels of literacy.

Criterion-Related Validity
When looking at distribution ( Figure 1A,B), we see that the distribution of scores is different between the SAHL and the SA-EHL. Therefore, we evaluated the SA-EHL via the SAHL criteria, as well as via a tertile calculation, as described in Table 3. Application of the SAHL criteria to the EHL resulted in a level of high literacy below 1% (Table 3). Evaluating the EHL categorically by tertile (low, medium and high) revealed the majority of respondents exhibited medium literacy (69.6%), with 72.5% classified as medium or high environmental health literacy. There was no agreement between the two scales using either the binary (kappa statistic = 0.0018, 10.8% agreement and p = 0.18) or the tertile method (kappa statistic = 0.005, 7.6% agreement and p = 0.14).
The two scales were poorly correlated with each other. A simple linear regression revealed no correlation (R 2 = 0.013, p < 0.001) (Figure 2). A Bland-Altman analysis was conducted ( Figure 3) and did not reveal any pattern of agreement between the two scales. A low correlation was observed between the SAHL and the SA-EHL (correction of r = 0.11, p < 0.001). When the SAHLE was regressed on EHL_17, it produced a small but significant regression coefficient of 0.13 (standard error = 0.04, 95% CI 0.05, 0.21, and p < 0.001). The model fit was low but significant (not shown in table-model statistics F (1, 862) = 11.12, model p < 0.001, R 2 = 0.012). Even though the model produced significant correlations, the magnitude was low and not meaningful. The mean difference between the scores of the two scales was 7.85 (2.8 SD).  Individual items within the SAHL and the SA-EHL were evaluated using linear regressions. Across all items, the R 2 value was ≤0.3, indicating little to no correlation between any items on the SAHL and the SA-EHL.  Individual items within the SAHL and the SA-EHL were evaluated using linear regressions. Across all items, the R 2 value was ≤0.3, indicating little to no correlation between any items on the SAHL and the SA-EHL. Individual items within the SAHL and the SA-EHL were evaluated using linear regressions. Across all items, the R 2 value was ≤0.3, indicating little to no correlation between any items on the SAHL and the SA-EHL.

Construct Validity
Construct validity was determined using a bivariate analysis between the absolute score and hypothetically related variables for each scale. We evaluated known predictors or characteristics of health literacy, including age, gender, income, education and marital status. Race/ethnicity was not assessed as the respondent population was predominantly White (77.7%, Table 1). Four characteristics were positively correlated with the SAHL: age (p < 0.001); gender (p < 0.001); education (p < 0.001); and marital status (p < 0.001) ( Table 1). None of the other co-variates were correlated with the SAHL.
For the SA-EHL, only education was positively related to the absolute score for the SA-EHL (p < 0.001), where individuals with a college degree had the highest scores within the SA-EHL. All other co-variates were not correlated with the SA-EHL (Table 1).

Discussion
We developed and assessed the Short Assessment of Environmental Health Literacy (SA-EHL) tool. This scale was adapted from the Short Assessment of Health Literacy [11]. Our tool performed differently from the SAHL, as assessed by the distribution of responses and poor scale correlation. In contrast, our prior research with the WELLS tool identified moderate correlations between health literacy and EHL scales [8]. However, the WELLS tool relied heavily on numeracy, which may explain the correlation between the scales [8]. In contrast, the SAHL and SA-EHL relies on word recognition. Here, our sampled population demonstrated very high health literacy via the SAHL, yet moderate-to-poor environmental health literacy using the SA-EHL. In fact, the mean score for health literacy was twice as high as the mean score for environmental health literacy. This may be due to either conflicting definitions of terms, or unfamiliarity with environmental health terminology. A review of EHL found that a common barrier was the use of confusing or unknown terms [4], either due to low literacy or cultural context [3,22]. Of note, items with very poor recognition (>90% incorrect or >90% "Don't know") were removed from our SA-EHL to mitigate this issue. However, many of the terms surveyed have varying definitions based on popular versus scientific use. Science communication experts have advocated that such dual-use terms be avoided to aid in comprehension [23]. Finally, the reliance on word recognition may also contribute to the poor correlation, as people may be more familiar with health-related terms, and less familiar with environmental health terms. This recognition does not always translate to true literacy, with evidence that word recognition assessments may overestimate health literacy [24]. Ultimately, our tool indicates that competence in health literacy does not directly translate to environmental health literacy. We did find that EHL was associated with educational attainment. This may be a function of increased education leading to higher literacy and, therefore, stronger word recognition and comprehension. This was identified previously with the WELLS scale being correlated with both income and education [8].
Recently, a survey of environmental health researchers, clinicians, educators and community partners was conducted to identify essential competencies in EHL [25]. Respondents were asked to rank the knowledge of specific environmental health terms. While the term "hazard" was ranked as essential by 67.2%, other terms were viewed as less important: "dose" (57.5%); "risk" (57.9%); and "frequency of exposure" (56.6%) [25]. Rather, the panel focused on specific skills they felt were essential for EHL, such as ability to find information to reduce risks in their life, or ability to identify established hazards in their environment [25]. Our results indicate that basic knowledge of frequently used terms in environmental health may be needed before individuals can build these higher-level competencies.
The SA-EHL is designed to assess the lower levels of EHL, comprising recognition and knowledge, or the dimension of awareness and understanding [1,4]. Additional tools will be needed to assess higher-level competencies. However, this tool can be used to generate a standardized, baseline level of EHL founded on recognition and word familiarity. Once this baseline is set, additional interventions or education can be conducted to scaffold individuals and communities to other important concepts in EHL [9]. Additionally, this baseline can help researchers and public health professionals better tailor messaging around environmental health hazards. Specifically, we envision this tool, among others, to be a useful way to ensure materials match the EHL of the target audience. By gauging baseline EHL through the use of focus groups or community forums, researchers and practitioners can avoid developing complex or repetitive materials. This is the theory behind health literacy screeners, which are often used to adjust the literacy level of materials, or the format in which they are disseminated (e.g., use of pictorial descriptions instead of text) [26,27]. In the same way, the SA-EHL may be used as an EHL screener to inform more appropriate messaging. This is a priority especially given environmental hazards following disasters, including wildfires, hurricanes and chemical releases [28][29][30].
We acknowledge several limitations with this study. The SA-EHL scale reflects words that were selected by public health and environmental health researchers. When compared to the aforementioned survey, several of these terms are considered essential, such as "dose", "exposure" and "risk" [25], yet the final SA-EHL scale is not representative of the full range of environmental health. Furthermore, the test population was highly educated and not fully representative of the United States [17][18][19]. This highlights an important issue for environmental health educators, which is the inclusion of definitions for common terms that are considered scientifically essential but may be misunderstood by a more lay audience. The results reported herein may not be applicable to other countries with differing cultural, linguistic and socioeconomic contexts. Countries with different pollution burdens may also differ in baseline awareness of environmental health concerns. It is interesting that the SA-EHL was correlated strongly with education, and not with additional factors, whereas the SAHL was correlated with multiple factors. This, along with the additional limitations described, suggests that this preliminary research be a foundation for additional work testing the SA-EHL in diverse populations. Finally, there remains a question as to what the test is fully measuring-environmental health literacy, or familiarity with words?
When evaluating word difficulty, Lee and Kauchak found that words with a lower frequency of occurrence in the English language had a higher level of perceived difficulty and were less frequently correctly defined [31]. In contrast, more frequent words were perceived as easier, and more correctly defined. Interestingly, there was no correlation between word length and ability to define the word [31], although longer words were perceived as more difficult [31].
There remain many strengths with this study. Using mTurk, the tool was tested in over 800 individuals across the United States. We adapted a validated health literacy tool, the Short Assessment of Health Literacy, allowing for an analysis of the overlap between health literacy and EHL. However, replications and further testing of this new measure of environmental health literacy should be considered.

Conclusions
The Short Assessment of Environmental Health Literacy was developed to create a standardized tool capable of assessing an individual's awareness (recognition) and knowledge of environmental health-specific terminology. While focused on word recognition, this tool can be valuable to gauge the baseline EHL of a population, thereby enabling researchers and public health practitioners to tailor educational materials [22]. This may mitigate the existing EHL barrier of confusing or unknown terminology [4]. Additionally, studies that are returning data to study participants and communities have reported qualitative increases in EHL with the return of data [32][33][34][35]. Thus, there is a need for standardized scales that can inform researchers and health care professionals about baseline EHL levels [5,6,36] and assess changes in EHL upon receipt of data [3]. The SA-EHL is designed to be a short, rapid assessment of baseline knowledge related to environmental health terms and concepts. The data from the SA-EHL can be used to measure changes in EHL and to tailor educational or outreach materials to the language of the target audience.

Institutional Review Board Statement:
The study was conducted according to the guidelines of the Declaration of Helsinki, and approved by the Institutional Review Board of Oregon State University (IRB 7299).

Informed Consent Statement:
Informed consent was obtained from all participants involved in the study.

Data Availability Statement:
The data presented in this study are available on request from the corresponding author.