2. Study 1: Factor Analysis and Rasch Model
The author’s purpose in the first study was to evaluate the psychometric properties of the of the ten CCWS items as given in Table 1
. In addition to item descriptive statistics and an evaluation of the internal consistency of the items, the author also assessed the items’ contributions to the latent variable of climate change worry. Two approaches were used for the for the latter task: (1) the classical true score test theory approach involving factor analysis [59
] and (2) a more contemporary approach to invariant measurement using a Rasch model [60
]. In using each approach for studying the item relationship with the construct of climate change worry, the author was interested in evaluating the unidimensionality of the items (i.e., is a single latent variable responsible for the intercorrelations of the ten items?). The author also was interested in assessing the extent to which the items may function invariantly with respect to the respondent’s gender. Similarly, the author explored gender differences on mean CCWS scores.
Finally, the author examined the correlation of CCWS scores with scores on an item that assessed the participant’s liberal versus conservative political orientation. Although the IPCC has documented that the Earth’s climate is changing and that this has stemmed from anthropogenic forcing of the climate system [1
], people differ with respect to the ways that mitigation and adaption efforts should proceed [66
]. Political identifications and attitudes particularly relate to: the degree to which people believe that climate change and disruption is actually occurring, that climate change is a problem that should be addressed, and in beliefs about mitigating climate change and adapting to it [66
]. Specifically, Smith and Leiserowitz observed that greater worry about climate change was associated with support for policy recommendations to address climate change [18
2.1. Materials and Methods
The participants were 600 undergraduate and graduate students from a large public university in the southeastern United States; the sample contained 300 each of men and women. The sample ranged in age from 18 to 51 years; M = 22.3 years and SD = 5.9 years. The racial composition of the participant sample was: 68.0% White, 10.0% Asian, 4.3% Black, 2.7% Latinx, 2.5% Indian subcontinent, and 12.5% multiracial or other race. Politically, 50.2% of the respondents indicated that they were liberal or liberal leaning and 35.9% were conservative or conservative leaning. Of the remaining participants, 9.4 indicated that they were moderate (between liberal and conservative) and 8.2% did not indicate their political orientation. The participants were recruited via an email message that was sent to the student that described the research project, which was part of a larger effort to gather data on the psychology of weather and climate. The incentive for participating was entry into a random draw to receive one of four $50 amazon.com gift cards. Students also had the chance to win one of the cards without participating in the research.
The participants completed the CCWS items online using the Qualtrics survey platform (http://www.Qualtrics.com
). They also completed a 12-item weather history survey that inquired about their past experiences of severe weather (i.e., floods, tornadoes, hurricanes, winter storms, or thunderstorms) that resulted in either injuries or damage to property. This survey also inquired about their previous evacuation experiences due to floods or hurricanes. The response options for this survey were “yes” or “no”. The online presentation order of the CCWS and the weather history survey was randomized. Finally, the participants then responded to demographic items that inquired about their age, gender identification, and race. The participants then responded to a single item that asked them to use a 7-point fully anchored rating scale to indicate their political orientation on a liberal (left side of scale) to conservative (right side of scale) scale.
The students were sent an email inviting their participation in the research and that also provided a link to the survey in Qualtrics. The first page of the online survey consisted of the informed consent document. If participants agreed participate, then the project measures appeared in random order. The demographics items always appeared last in the survey, followed by a debriefing statement. The project measures and procedure were reviewed and approved by the Institutional Review Board at the author’s university (Approval: MOD00007528, Parent Protocol STUDY00001233).
2.1.4. Data Analysis
The author used both (1). factor analysis and structural equation modeling (SEM) approaches and (2). Rasch modeling to examine the latent structure of the CCWS items and to examine any differences in the functioning of the items with respect to participant gender. The author employed each of these psychometric approaches to provide a thorough examination and analysis of the items [64
]. Response scales such as the one employed on the CCWS often function as ordered categories rather than a true interval-level scale. This means that item intercorrelations are best assessed through polychoric correlations [68
]. The author used both the Lavaan package in R and the Factor program to calculate the polychoric correlations and to perform the exploratory factor analysis [70
]. Factor is an open-source program available at: http://psico.fcep.urv.es/utilitats/factor/index.html
. The factor analysis used an unweighted least squares factor extraction procedure because this method produced superior results compared to maximum likelihood methods when factor analyzing categorical data that are not normally distributed [75
]. Because a single latent variable (i.e., climate change worry) was expected in the analysis of the data, no factor rotation methods were employed. The author used the Coefficient Alpha package in R to calculate the internal consistency statistics [76
]. To check for the invariance of factor solutions across gender, the author used the Lavaan structural equation modeling package within the R statistics environment [74
]. The semTools R package also was used to evaluate the extent of model invariance between men and women as constraints were added progressively to the models [77
While traditional factor analysis is concerned with understanding the variance that items share in indicating a latent variable, Rasch modeling goes further to provide a description of both the items as well as the people who have responded to them (60,63,65). The goal of Rasch measurement is to create an invariant measurement model by calibrating responses to items with estimates of the amount of trait or ability a person possesses. For this project, the author used the Facets and Winsteps programs to perform the Rasch modeling [78
2.2.1. Descriptive Statistics
shows the polychoric correlations of the combined sample of men and women who responded to the CCWS. The bottom rows of Table 2
also provide the descriptive statistics for each item and the item-to-total correlations. The items exhibited a mix of positive and negative skewness. Overall, many items exhibited heavier distributions in the tails (i.e., negative kurtosis). These results were reflected in Mardia’s tests for multivariate skewness (not statistically significant) and kurtosis (p
< 0.0001) [80
]. Thus, the data were not considered to be normally distributed.
2.2.2. Factor Analysis
Prior to conducting the factor analysis, the author examined Bartlett’s test of sphericity that the polychoric correlations in Table 1
were an identity matrix (i.e., 1’s on the diagonal and zero values elsewhere). Bartlett’s test (χ2
= 6673.2; df = 45; p
= 0.00001) suggested the correlations were not an identity matrix, indicating that factor analysis was appropriate. The author also examined the Kaiser–Mayer–Olkin (KMO) test that at least one latent variable may explain the correlations in Table 1
. The KMO value of 0.95 suggested that factor analysis would be useful. Several indices also suggested that a single factor existed among the item, beginning with the parallel analysis criteria of Timmerman and Lorenzo-Seva [81
]. Three additional indices that are calculated by the Factor program also suggested a single, unidimensional factor. First, the unidimensional congruence (UniCo) coefficient was 0.994 (95% CI: 0.991–0.997). Second, the explained common variance (ECV) coefficient was 0.926 (95% CI: 0.909–0.942). UniCo and ECV values that are greater than 0.95 and 0.85, respectively, suggest that items can be treated as unidimensional. Third, the mean of item residual absolute loadings (Minreal) provides an indication of additional variance that was unexplained. The Minreal index was 0.223 (95% CI: −0.196–0.245), which was below the cutoff of 0.30, suggesting that the mean residual loadings were minimal. Given the values of these indices, a one-factor solution was chosen in the factor analysis.
shows factor loadings of the items and their communalities. The single factor explained 73.6% of the variance. Likely due to the larger sample size, the robust mean and variance adjusted chi square statistic was statistically significant, suggesting a lack of model fit, χ2
= 302.72; df = 35; p
= 0.00001. The root mean-square error of approximation (RMSEA) was 0.043, which indicated a well-fitting model. Beyond this, several fit indices suggested an acceptable degree of fit: the non-normed fit index = 0.97, the comparative fit index (CFI) = 0.99, and the adjusted goodness of fit index = 0.99. Finally, the root mean square of residuals was small (0.052), suggesting that the one-factor solution was optimal.
With respect to the internal consistency and unidimensionality of the CCWS items, Cronbach’s alpha was high; α = 0.95, (95% CI: 0.95–0.96) [82
]. Because McDonald’s omega (ω) provides a more conservative estimate of internal consistency, the author also calculated this statistic as well [83
]. Here, McDonald’s omega also exhibited a high value, ω = 0.95, (95% CI: 0.95–0.96). Hancock’s index, H = 0.97 (95% CI: 0.96–0.97), was calculated and suggested that a single common factor well-represented the CCWS items [85
To what extent is this factor solution invariant for men and women in the sample? Table 4
shows the changes in the chi square, the CFI, and RMSEA as model constraints were added to assess the degree of invariance that may exist in the factor solution between men and women. Generally, the results of the invariance analyses support the conclusion that the CCWS items function very similarly in the assessment of climate change worry regardless of gender. Using a criterion of 0.005 for the change in the CFI and the RMSEA, the constrained equivalence of factor loadings (metric invariance), intercepts (scalar invariance), and the residuals of items (residual invariance) did not result in large changes in the model fit indices [86
]. Thus, these types of invariance in the CCWS measurement model were established for men and women. The only criteria that was not satisfied concerns the constraint of the means of each group on the latent factor of climate change worry. Mean differences on the CCWS scores will be examined further below.
2.2.3. Rasch Item Analysis
The Rasch model explained 71.3% of the variance in the CCWS items. The CCWS exhibited a good separation of the participants, relpart. = 0.92, χ2 (599) = 6959.3, p < 0.0001. Such a reliability value suggests that the CCWS items are sensitive to the differences that participants exhibited on the latent trait of climate change worry. The Rasch analysis also provided a strong degree of separation of the items with respect to their distinctiveness and to their ordering as participants responded to them, relitem = 0.99, χ2 (9) = 2921.1, p < 0.0001.
shows the frequency with which the different ordinal response categories were used by the 600 participants in responding to the CCWS items. Respondents used the first four categories rather uniformly (i.e., 21%–25%); the Always response category was used somewhat less at 11% overall. The average measures are those that were modeled to generate the observations at each response category. The expected measures are those corresponding response values that exist if the data fit the Rasch model [79
]. The small discrepancies between the average and expected measures in Table 5
suggest that the data fit the Rasch model well. Similarly, the outfit column in Table 5
has an expected value of 1.0 for each category if the data fit the model. Finally, the Rasch–Andrich thresholds show the step calibrations of the rating scale as logits on the latent variable of climate change worry [79
]. The thresholds show the amount of worry needed to go from one point on the response scale to the next (e.g., a 1.55 logit step is involved in moving from the Sometimes to Often response categories). Each of the thresholds in Table 5
exceeds the recommended minimum value of 1.4 logits [60
]. Figure 1
shows the actual and the modeled proportion of responses within each response category of the CCWS items given the level of climate change worry, modeled as a latent variable. The close correspondence between the data (straight lines with x) and the model (smooth solid lines) provide visual support of the extent of Rash model fit.
shows the Rasch model item characteristics along with statistics that indicate the degree to which the data fit the model for each item. The construct level (also known as the item difficulty) convey in logit units the level of climate change worry that the item assesses. The larger the value of the construct level, the greater the amount of climate change worry a person must have to respond to the item that they often
worry. The modeled standard error appears in the second column of Table 6
. The item discrimination coefficients appear in the third column of Table 6
. These coefficients indicate the slope of the item characteristic curves and convey the degree to which items discriminate between people who possess different levels of climate change worry. The right columns of Table 6
suggest that the data fit the Rasch model well because the values for the infit and outfit mean squares were in the optimal region of 0.5 to 1.5 [79
]. The infit statistic is sensitive to aberrant response patterns within the measurement region of an item whereas the outfit statistic is sensitive to outlying data points that do not fit the model well [60
depicts a Wright map of both the items and the respondents on the logit scale of climate change worry [78
]. The items and their response categories have been indicated on the right-hand side of the figure in order of increasing climate change worry (i.e., item difficulty). Generally, the ten CCWS items provide good coverage over the mid- and upper-ranges of climate change worry. Item coverage of lower and minimal levels of climate change worry was somewhat sparse.
shows the test characteristic curve for the ten CCWS items. The amount of climate change worry as a latent variable is depicted along the horizontal axis while the item response categories are shown on the vertical axis. The data fit the model well, as shown by the close proximity of the data (x) with the model (smooth curve). The figure also shows the upper and lower bounds of a 90% confidence interval. Overall, the figure suggested that the data fit the model quite well.
2.2.4. Demographic Relationships with CCWS Scores
The author transformed the Rasch score on the CCWS for each person to remove the negative values (i.e., multiplied the Rasch score by 5 and added 50). When using four categories of the participants’ racial identification (Black, White, Asian, and Other), no statistically significant differences were observed in the mean of each group on the CCWS; F (3595) = 1.54, p = 0.20. The respondents’ political orientation scores were correlated significantly with their scores on the CCWS; r = −0.45, p < 0.0001. That is, self-ratings of being politically liberal were associated with higher scores on the CCWS whereas greater identification with conservative attitudes was associated with a lower score. Finally, the author examined the extent to which women and men differed in the mean scores on the CCWS given the relationships the CCWS scores demonstrated with political orientation. Women exhibited significantly higher scores (M = 47.6, SD = 12.1) compared to men (M = 45.4, SD = 11.8); F (1548) = 5.63, p = 0.02, η2 = 0.02. Overall, both political identification and gender accounted for 21% of the variance in CCWS scores. Finally, there were no statistically significant relationships among the CCWS score and the total or individual item responses to the weather history survey.
The results from Study 1 were useful in establishing the psychometric properties of the CCWS items in several ways. First, the ten CCWS items appear highly internally consistent with respect to the way that the study participants interpreted and responded to them. Second, and relatedly, the factor analysis of the polychoric correlations suggested that a single latent variable, climate change worry, fit the data well. Third, the results of a confirmatory factor analysis provided support that even with constraints through the equality of residuals, the factor structure of the CCWS is invariant between men and women in the sample. Fourth, Rasch modeling provided further psychometric information at the item and person levels regarding the use and performance of the items and the five-point rating scale. In this regard, the items provide good coverage in reflecting the levels of climate change worry people possess. There is, however, a lack of items measuring very low or minimal levels of climate change worry. The demographic analyses revealed that the mean scores by men and women on the CCWS differed and that climate change worry was moderately correlated with liberal–conservative political identification. Further, prior experiences of severe or extreme weather that resulted in property damages or injuries were not associated with scores on the CCWS. This result was consistent with prior research [66
3. Study 2: Test–Retest Reliability
The author’s purpose in the second study was to evaluate the stability of the CCWS scores over time when the scale was administered to a sample of people over two occasions. The observation of highly related scores over time provides evidence that CCWS may function similarly when used with groups that resemble the reliability sample [59
3.1. Materials and Methods
The material for this study consisted of printed, hard-copy packet that contained the CCWS and a demographic form. The participants were 54 (45 women) undergraduate students from a large university in the southeastern United States. The mean age of the participants was 20.9 years (SD = 1.06 years). The racial composition of the participant sample was: 63% White, 24% Black, 6% Asian American, and 7% other, biracial, or multiracial.
Participation in the research fulfilled the students’ course requirements. The participants had a number of different research and non-research opportunities from which to choose. The 54 participants completed the research in person in groups of five to ten people each. When the participants registered for the research, they also were scheduled for their second session that was exactly two weeks after the first session. At the first session, the participants were provided with an overview of the research and completed a written informed consent document. They then completed the CCWS and were excused. At the second session, participants again completed the CCWS and were then provided with a written debriefing statement that summarized the purposes of the research. Care was taken at the first session to minimize the possibility of carry-over effects to the second session. The research protocol was part of a larger effort to evaluate the test–retest reliability of several weather and psychology measures. The research protocol was approved by the Institutional Review Board of the university where the author conducted the research (Protocol ID#: STUDY00003846).
The Pearson correlation of the participants’ total scores on the CCWS at the first administration with their total scores at the second administration suggested that the sample produced reliable scores; r = 0.91, p < 0.001. As another check on the stability of the scores over the two-week test–retest interval, the author conducted a paired t-test of peoples’ scores on the CCWS on the two testing occasions. This analysis revealed that the CCWS scores also did not change in a statistically significant way from the first administration (M = 22.26, SD = 6.98) to the second administration (M = 22.89, SD = 7.00), t (53) = 1.53, p = 0.13.
Beyond this test–retest reliability assessment of scores, the author also evaluated the internal consistency of the CCWS items at the first and second administrations using Cronbach’s coefficient alpha (α). At both the first (α = 0.90) and second administrations (α = 0.91), the CCWS items demonstrated good internal consistency. Regarding McDonald’s omega, both the first administration (ω = 0.90) and the second administration (ω = 0.92) of the CCWS suggested that peoples’ responses to the items were internally consistent.
The results of Study 2 provided evidence in two ways that respondents’ total scores on the CCWS were stable over time. First, when the respondents’ scores were correlated across a two-week test–retest interval, other than the passage of time the scores shared 83% of the variance. Second, the respondents’ mean scores were compared across the two occasions and did not differ with respect to their location (i.e., mean values). Additionally, with a new sample beyond that used in Study 1, the CCWS items demonstrated satisfactory internal consistency. Thus, the results from Study 2 establish the important psychometric property of temporal stability in the CCWS scores [59
4. Study 3: Relationship of CCWS Scores with Measures of Worry, Fear of Weather, Anxiety, Depression and Stress
Having established the unidimensional structure of the CCWS items and the temporal stability of peoples’ responses to the measure, the author’s purpose in Study 3 was to explore the relationship of respondents’ scores on the CCWS with their scores on several related measures. Such a survey of correlations with known and established measures would permit an evaluation of the CCWS’s convergent and divergent validity. With respect to convergent validity, it was expected that as a measure of worry the CCWS would exhibit some degree of correlation with the Penn State Worry Questionnaire (PWSQ), a broad-based measured that inventories the degree to which people report worry in their lives generally [43
]. Because the CCWS was designed to assess personal worry specifically about climate change rather than about broader areas of life, the correlations with the PSWQ were not expected to be large.
Because worry can be related to fear, the author expected that scores on the CCWS would be related to two measures of fear about weather conditions. Conceptually, if climate change manifests as future severe weather events that may be more intense and damaging, then worry about climate change should be related to fears about the weather [3
]. In assessing the CCWS’s convergent validity with respect to fear, the author used two measures, the first of which was the Storm Fear Questionnaire (SFQ), a 15-item measure that was created to assess clinically-severe levels of storm fear in people [57
]. The second measure was the Fear of Weather Scale (FOWS), an 87-item instrument designed to assess the degree of fear people experience towards various types of weather events and the consequences of these events for their lives [22
The author examined the CCWS scores with those of stress, anxiety, and depression as measured by the Depression, Anxiety, and Stress Scale (DASS) [90
]. The DASS has become a benchmark resource in defining, understanding and assessing the emotional states of depression, anxiety, and stress. For this study the author hypothesized that the CCWS would exhibit a greater degree of correlation with the DASS stress subscale, and somewhat lower correlations with the anxiety and depression subscales [45
]. Further, the author also expected that anxiety would be related to scores on the weather fear measures and thus relate to CCWS scores more indirectly.
Beyond the pairwise correlations of the aforementioned measures with the CCWS, the author examined the extent to which the measures may be used together to account for CCWS scores. From Study 1, it was expected that peoples’ political orientations would be a significant predictor of climate change worry. The author predicted that stress would also be associated with climate change worry given the theoretical relationship of stress with worry [52
]. Because fear of severe and extreme weather events may encompass what people would anticipate under a changed climate, it was expected that such fears also would be associated with CCWS scores. Finally, the author hypothesized that the emotional state of anxiety, rather than being directly predictive of climate change worry, would be more related with fear of weather and storms and thus exert its influences in that manner.
The participants were 417 undergraduate and students from a large public university in the southeastern United States. The sample was mostly female (n = 353, 85%). The sample ranged in age from 18 to 37 years, M = 20.8 years, SD = 1.9 years. The racial composition of the participant sample was: 72.9% White, 7.7% Black, 4.5%, Asian, 3.8% Latinx, 1.7% Indian subcontinent, and 9.4% multiracial or other race. Regarding political attitudes, 58.9% of the respondents indicated that they were liberal or liberal leaning and 37.1% were conservative or conservative leaning. There were 16 (3.8%) who did not indicate their political orientation. The participants completed the research as part of a research pool in the university’s College of Education in exchange for course credit. The participation in the study was completely voluntary and people had a range of alternative studies in the research pool from which to choose.
In addition to the CCWS the participants completed four measures that included the PSWQ, FOWS, SFQ, and DASS. Researchers have used the Penn State Worry Questionnaire extensively in the past to explore its relationships with other emotions in clinical and experimental research [43
]. The PSWQ consists of 16 self-report items that were designed to measure the trait of worry. People make their responses to the items on a five-point rating scale (1 to 5) with respect to how characteristic the item is of the respondent. The PSWQ is scored by summing the items; the scores range from 16 to 80. Higher scores are suggestive of clinically significant levels of worry [43
]. The PSWQ demonstrated good test–retest reliability (r
= 0.92) at an 8 to 10-week interval and exhibited good internal consistency (α = 0.93). For the present participants, the author observed that the PSWQ also possessed a high degree of internal consistency (α = 0.94, CI: 0.93 to 0.95).
The Storm Fear Questionnaire is a 15-item self-report measure of “the cognitive, affective, and behavioral aspects thought to be associated with storm phobia in adults.” ([21
], p. 114) The authors conceptualized storms to encompass a variety of severe or extreme weather events such as thunder, lightning, hurricanes, and tornadoes, although the items only refer to storms in the SFQ. People respond to the SFQ items using a five-point rating scale (0 to 4) to indicate the extent to which each item is true for them. The total score on the SFQ are obtained by summing the item ratings; Scores range from 0 to 60 with higher scores indicating progressively greater degrees of storm-related fear and incapacity resulting from such fear. The SFQ items demonstrated very good internal consistency (α = 0.95). The two-week test–retest reliability was acceptable (r
= 0.78). With the sample of participants in this project, the author observed that the SFQ possessed good internal consistency (α = 0.94, CI: 0.93 to 0.95).
The Fear of Weather Scale contains 65 self-report items that assess peoples’ level of fear to various types of weather (e.g., thunderstorms) and the sensory components of that weather (roaring sounds of wind). People indicated their fear of each weather component by using a 0 (No fear) to 6 (Terror) fully-anchored rating scale [22
]. The item ratings are summed to yield a total score, which can range from 0 to 390; higher scores indicate greater fear of weather. The FOWS also possesses 22 items to assess the effects and consequences of severe weather on people with respect to property damages, loss of control over belongings, and loss of services (e.g., water, electricity). The FOWS exhibited good internal consistency in prior research [22
]. Within the current sample of this study, the internal consistency of the 65 items assessing fear of weather was quite higher (α = 0.98, CI: 0.97 to 0.98).
The Depression, Anxiety, and Stress Scale is a 42-item measure designed to assess the emotional states rather than traits [90
]. Each subscale consists of 14 items that measure recent characteristics of depressive emotions (gloomy outlook, pessimism, self-disparaging, disinterest, lack of motivation, etc.), anxiety (apprehensiveness, trembly, autonomic symptoms of arousal, worry about performance and loss of control), and stress (tension, easily upset, irritable, difficulty relaxing, easily startled). The DASS instruction set asks people to respond to the items using the timeframe of “over the past week” and a four-point rating scale that ranges from 0 (“Did not apply to me at all”) to 3 (“Applied to me very much or most of the time”). The DASS has seen extensive use in experimental and clinical research as evidenced by over 400 studies that have used the measure within the PsycINFO database. Most recently, the shorter version of the DASS (i.e., DASS-21) was favorably evaluated for its psychometric properties and its robustness against patient self-report bias [92
]. The DASS has demonstrated stability through confirmatory factor analyses and also good levels of internal consistency (α’s ranging from 0.84 to 0.91) [90
]. For the present sample, the three DASS subscale exhibited strong internal consistencies: anxiety (α = 0.94, CI: 0.93 to 0.95), depression (α = 0.96, CI: 0.95 to 0.97), and stress (α = 0.95, CI: 0.94 to 0.95).
The participants enrolled in the study online and then were directed first to the informed consent document. The measures of the project were administered through the Qualtrics survey platform. If the participants gave their consent for the study, then the Qualtrics platform presented the project measures in random order. The demographic items always appeared last in the survey, followed by a debriefing statement. The project measures and procedure were reviewed and approved by the Institutional Review Board at the author’s university (Protocol ID#: STUDY00006902).
4.1.4. Data Analysis
The author used the R statistics package to calculate the descriptive statistics and the Pearson correlations to investigate the hypothesized relationships in this study. Then, the Lavaan package in R was used to conduct the path analysis to assess the contributions of political orientation, fear of weather, fear of storms, and stress upon CCWS scores [74
The correlations of the CCWS with the with the other measures in Study 3 appear in Table 7
. As hypothesized, although the scores on the CCWS and the PSWQ were correlated to a statistically significant extent, the correlation was comparatively small in magnitude. This result suggested that the type of general and broad worry that characterizes high scorers on the PSWQ does not contribute in a noteworthy way to the specific worry people expressed about climate change, and thus supports the instrument’s divergent validity Similar to the results from Study 1, political orientation was related in a statistically significant and noteworthy way with CCWS scores. Again, an increasingly democratic, liberal political orientation was associated with increasing levels of climate change worry. Although women (M
= 24.8) did produce CCWS scores that were numerically higher than men (M
= 23.0) as observed in Study 1, the differences within the present study were not statistically significant (p
= 0.17). This result may have been due to the fact that there were fewer men (n
= 64) in this study compared to Study 1.
Regarding weather, both the SFQ and the FOWS exhibited comparable and statistically significant associations with scores on the CCWS. For each weather measure, increasing fears of storms or of weather elements that could be severe or extreme were associated with increases in climate change worry.
The DASS subscales each demonstrated statistically significant associations with scores on the CCWS. Increases in scores across each of the three subscales (i.e., stress, anxiety, and depression) were associated with increases in climate change worry. Contrary to what was expected, the three DASS measures exhibited comparable correlations with the CCWS. The Stress subscale, however, exhibited the most robust correlation with the CCWS as evidenced by a partial correlation analysis. After controlling for the association of the DASS anxiety subscale, the correlation of Stress with CCWS remained statistically significant, rStress,CCWS|Anxiety = 0.12, p = 0.014. A similar result was observed when controlling for the DASS depression subscale: rStress,CCWS|Depression = 0.12, p = 0.016. When controlling for the associations of the Stress subscale, the CCWS correlations with the Anxiety and depression subscales were not statistically significant. These results suggested that the emotional state of stress was more strongly related to climate change worry.
The contributions of the variables to the prediction of climate change worry are depicted in the path diagram in Figure 4
. This model exhibited a good fit to the data, as indicated by a comparative fit index (CFI) of 0.98 and a Tucker–Lewis index of 0.96. Similarly, both the RMSEA (0.056) and the standardized room mean square residual (0.021) were small and suggestive of a good model fit. Feelings of state anxiety (DASS anxiety) were predictive of scores on the FOWS and the SFQ; anxiety was somewhat more related with fear of storms. In turn, the FOWS and the SFQ were comparably predictive of CCWS scores. A stressful emotional state (DASS stress) also was predictive of increasing CCWS scores as was a progressively more liberal or democratic political orientation. Together, the predictive measures explained 32% of the variance in CCWS scores. Scores on the PSWQ and participant gender were not significantly predictive of scores on the CCWS.
The results of this study document the convergent and divergent validity of the CCWS. The small but significant correlation between the CCWS and the PSWQ likely relates to the fact that both instruments are measures of worry (i.e., both measures use the word, worry, in their respective items). Further, because the CCWS was specifically designed to measure climate change-related worry, it is understandable that the correlation with the PSWQ is small given the more general and broad scope of the latter.
Fear of weather (FOWS) and fear of storms (SFQ) both exhibited a moderate degree of association with climate change worry, a result which supported the convergent validity of the CCWS. These results are consistent with the fact that a disrupted global climate system may lead to more severe weather outbreaks or more frequent extreme events [1
]. People with existing fears of weather or storms may thus worry increasingly about climate change. This may especially be the case if people have memories or recollection of severe weather and have noted trends towards greater severity in the weather over time.
Research in the field of worry suggested that stress would relate more strongly to worry than other affects like anxiety or depression [45
]. Although the three DASS measures exhibited correlations of comparable magnitudes with the CCWS, only the DASS stress subscale exhibited the most robust correlation with climate change worry. DASS anxiety was more predictive of fears of severe weather elements and fear of storms than climate change worry. This result makes sense in that the general apprehensiveness about future events (here, weather and storms) may amplify peoples’ fears of extreme weather events.
Political orientation was the most predictive of climate change worry. This result is consistent with the higher degree of correlation among these variables reported in Study 1 above and also in line with prior research results that have documented a liberal–conservative difference in beliefs about the realities of climate change, the causes of climate change, and policies to address it [18
]. This result also raises the interesting question of what else may contribute to climate change worry that could help to change a person’s beliefs and attitudes so that they are more prepared to behave more sustainability and thus to mitigate global climate disruption and adapt to its varied impacts.
5. General Discussion
Global climate change may affect peoples’ behaviors (e.g., move from a flood plain or coastal region), their thoughts or attitudes (e.g., accept or deny the reality of climate change) and can lead to the experience of emotions such as anxiety, fear, or worry [13
]. Within this article, the author reported on the initial development of a measure of worry about climate change.
The author chose to assess climate change worry rather than other affects such as fear or anxiety for three reasons. First, worry is a component of anxiety along with somatic arousal [55
]. Thus, worry is a more fundamental psychological feature and one that is rooted in the symbols of thought and language [45
]. Although people may typically know that they worry, how much they worry, or what they worry about so that they can self-report these experiences accurately, such may not be the case in self reports of the somatic arousal that underlies anxiety. Self-reports of arousal may not correspond to arousal measured psychophysiologically [93
]. In addition, people may be differentially sensitive to interoceptive cues of their arousal [94
]. Second, and relatedly, with its foundation in language and thought, worry about climate change is salient at this time when messages about climate change appear frequently within the media. From 2017 to 2019, climate change and the occurrences of severe and extreme weather have resulted in nearly 34,000 print and online articles in over 200 newspapers worldwide and approximately 3000 television and stories according to the Ebsco Newspaper Source Plus database. If people have not experienced climate change directly, then there is an increasing likelihood that they will have learned about it through the media [20
]. Third, fear pertains to the anxiety and apprehension that comes with an imminent threat [51
]. Because climate change may be unrealized for some people, the assessment of climate change fear may be less informative at this point, but certainly should be considered for the future [95
]. In this regard, fear of weather or storms that people experienced in the past or anticipate for the future may relate to climate change worry [1
Beyond these conceptual considerations, the three studies in this article were informative in the creation of a new measure of climate change worry. The ten CCWS items exhibited unidimensionality and invariance at the residual level for men and women in Study 1. In addition, a Rasch model was developed that further supported the unidimensionality of the items and that illustrated the relationship of the participants and the CCWS items with respect to their level of climate change worry (Figure 2
). The results from Study 2 suggested that over a two-week test–retest interval that scores on the CCWS were both highly correlated and that the scores on the measure did not change significantly with time. The results from study 3 supported the validity of the CCWS by examining its relationship with other established measures. Although the CCWS was correlated with the PSWQ, the magnitude of the correlation was small. Climate change worry was much more related to the state-levels of stress (DASS stress), which was expected on the basis of theory [13
]. Similarly, the CCWS correlations with fear of weather (FOWS and SFQ) were expected and observed given the general relationships of worry with fear [42
]. Following from the results of previous research, it was not surprising that climate change worry was associated with political orientation (studies 1 and 3) [18
This research is limited in two significant ways, the first of which pertains to the author’s choice to assess personal worry about climate change rather than a more encompassing one that concerned inter-group conflicts and/or resource scarcity [7
]. Thus, the CCWS and the research reported here represents a first step to assess and quantify climate change worry. Subsequent development efforts with the scale will focus on the societal and regional contributions to climate change worry. A second limitation concerns the reliance upon university student samples from the southeastern United States in the three studies. People who are older, who live in different geographical or climatic regions, or who have a different socioeconomic status compared to students may provide responses to the CCWS that differ from the patterns observed and reported in this project. With these caveats in mind, the student samples provided an important starting point for the measurement of climate change worry given that the university is situated in an area that regularly experiences extreme of heat, flooding, severe thunderstorms and tornadoes, and that is susceptible to hurricanes or tropical storms that make landfall in the southeastern United States. In addition, the responses of student participants are valuable insofar as they reflect the experiences of people who will study, document, enact policies, and live with the results of climate change in the future.
Future research with the CCWS could occur along several lines, the first of which involves examining the functionality of the instrument with a diverse sample that is more representative of the racial and socioeconomic composition of residents in the United States. Second, the researcher will explore the development of additional items that relate to more distal and global impacts of climate change. Third, additional research should be conducted to explore the interrelationships of fear, anxiety, depression, and worry within the realm of climate change. Although the distinctions between these constructs may be somewhat clearer in the clinical psychology literature [11
], the nature of the constructs and their interrelations within climate psychology warrants further theoretical and empirical research [25
]. Specifically, can climate change with its multiple effects and impacts by themselves lead to anxiety and depressive disorders? Alternatively, do the effects of climate change contribute to the worsening or longer duration of incipient and existing disorders? Perhaps both scenarios are possible [13
]. Subsequent research will hopefully bring clarity to these questions. The CCWS may be one among several tools available to researchers to use for this purpose.