Toward Sustainable Mental Health: Development and Validation of the Brief Anxiety Scale for Climate Change (BACC) in South Korea

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

The paper develops and validates an Anxiety Scale for Climate Change (Brief). The study first conducts an Exploratory Factor Analysis (EFA) with 300 respondents, followed by a Confirmatory Factor Analysis (CFA) with 400 respondents. The paper proposes a 13-question Brief Anxiety Scale for adolescents and adults. Below, I outline several areas that require improvement prior to accepting the paper for publication. 

1. The paper is focused on South Korea. This information should be in the title of the paper.
2. What are the limitations of the study? 
3. How generalizable is the new instrument?
4. The paper uses the Mental Health Screening Tool for Anxiety Disorders (MHS:A) and the Mental Health Screening Tool for Depressive Disorders (MHS:D). Could you explain the selection of these two tools? Enhance the motivation behind the use of these two tools by citing the primary references and demonstrating their application. 
5. Why not compare the results of the developed scale with the full 22-question scale? Are we missing something by reducing the scale? 
6. Rephrase the paragraph "Moreover, the kurtosis and skewness values for all the questions met the criteria of skewness at < |2| and < |7|, respectively." Is the kurtosis lower than |7|?
7. The methodology is long. Include a summary of a pseudocode showing how the method is implemented in R. 

Author Response

Thank you for your valuable feedback. Please see the attachment. 

Author Response File: Author Response.pdf

Reviewer 2 Report

Comments and Suggestions for Authors

Introduction
The authors do not show how climate change is connected to anxiety. They only mention in a very vague general way. They should consider specific points of climate change that link to emotional responses such as anxiety.
The English language writing is poor, e.g.
distress [7]; furthermore, it has been widely recognized as a transdiagnostic feature across anxiety and mood disorders.

It is recommended that the paper should be reviewed by a native English speaker. 

The authors focus on coping. However, this is not the heart of the study; they should explain the importance of the psychometric properties of the studies in terms of content and construct validity of the study variable.
The authors indicate that it is important to include clinical criteria. However, in their sample they do not show a clinical population.

Methods,
The authors should mention the resources they used to review relevant information on climate anxiety, for example, databases or mention if they used any type of artificial intelligence tool.
The authors indicate that they only relied on the DSM V, they do not show a solid theoretical support to develop their items. This point should be clarified.
It is worrying that the authors included adolescents in their study and did not have the approval of an IRB, which raises serious doubts about the procedures and results of the study.
As it is ethically based to pay adolescents $5 to participate in a study, this is malpractice. This could generate only desired responses and therefore invalidate the results.

Other points that discredit the work is the sample, which is insufficient to propose the creation of a new scale. It is recommended that they use or expand the scales we have that have been created by experts, e.g. Clayton, (2020).
The rationale lacks empirical foundation.
Overall, the methods used have not been sufficiently robust to propose the creation of a new scale on climate anxiety.

 

 

Comments on the Quality of English Language

The English language writing is poor, e.g.
distress [7]; furthermore, it has been widely recognized as a transdiagnostic feature across anxiety and mood disorders.

It is recommended that the paper should be reviewed by a native English speaker. 

Author Response

Response to Reviewer 2

We would like to express our sincere gratitude to Reviewer 2 for their constructive and insightful feedback, which helped improve the clarity and quality of our manuscript. Please find our detailed responses to each point below.

Comment 1:

The authors do not show how climate change is connected to anxiety. They only mention in a very vague general way. They should consider specific points of climate change that link to emotional responses such as anxiety.

Response 1:

Thank you for this insightful comment. We agree that it is essential to clearly explain the psychological mechanisms through which climate change leads to anxiety, and we acknowledge that this aspect was not sufficiently addressed in the original introduction. In response, we have added a new paragraph to the Introduction to provide a more detailed account of climate anxiety. This paragraph draws on previous research to explain how key features of climate change—such as its unpredictability, uncontrollability, and persistent nature—contribute to anxiety. It also highlights how these factors, when combined with concerns about one’s future or the safety of one’s children, can ultimately lead to clinically significant levels of anxiety.

Revision: Page 2, Paragraph 1, Lines 46-54.

“Among the various mental health effects of climate change, anxiety has emerged as the most prominent emotional response [2]. One of the key reasons is that climate change represents a real, persistent, and long-term threat that is difficult to predict, control, or resolve [3]. Anxiety, by nature, is a psychological response linked to the activation of the behavioral inhibition system (BIS), which regulates individuals’ sensitivity to threat and their tendency to avoid potential harm [4]. In the context of climate change, this unresolved and existential threat can intensify uncertainty and worry about the future, including concerns for one’s own safety and that of one’s children[5]. As a result, such prolonged emotional responses may develop into clinically significant levels of anxiety [3].”

 

Comment 2:

The English language writing is poor, e.g. distress [7]; furthermore, it has been widely recognized as a transdiagnostic feature across anxiety and mood disorders. It is recommended that the paper should be reviewed by a native English speaker. 

Response 2:

Thank you for your valuable feedback. Prior to submission, the manuscript underwent professional native English editing through the academic editing service Editage. Nevertheless, we fully acknowledge the importance of clarity and readability in academic writing and sincerely appreciate your suggestion. Accordingly, we have arranged for a second round of professional editing, and the revised version has now undergone an additional native-level language review, including the sentence you kindly pointed out. A certificate of editing has been attached for your reference.

Revision: Page 2, Paragraph 4, Lines 72-74

"It can manifest as a behavioral expression of climate-related distress [10] and is widely recognized as a transdiagnostic feature across anxiety and mood disorders [11]."

 

Comment 3:

The authors focus on coping. However, this is not the heart of the study; they should explain the importance of the psychometric properties of the studies in terms of content and construct validity of the study variable.

Response 3:

Thank you for this valuable comment. We would like to clarify that the references to coping strategies in the manuscript were not intended to position coping as the main focus of the study. Rather, they were included to highlight a key limitation of existing climate anxiety scales, namely the conceptual overlap between symptomatology and coping behaviors. This distinction is particularly important for the clinical assessment of climate anxiety, and it served to motivate the development of a new scale that focuses exclusively on symptom-based evaluation. That said, we fully agree that the core contribution of this study lies in the psychometric validation of the newly developed scale. In response to your suggestion, we have revised the Discussion section to more clearly articulate the BACC’s content validity—established through a multi-step development process—and its construct validity, supported by consistent factor structures across samples and CFA model fit indices.

Revision: Pages 13, Paragraph 2, Lines 442-453.

“The final scale demonstrated strong internal consistency, robust factor structure, and good discriminant validity with good model fit indices (CFI = 0.96, TLI = 0.95, RMSEA = 0.077, SRMR = 0.071), thereby supporting its construct validity.

The content validity of the BACC was established through a multi-step development process. Items were initially constructed based on DSM-5 diagnostic criteria for GAD and reviewed in relation to existing climate anxiety measures. Based on expert consensus and external feedback, appropriate items were selected or revised. In addition, item difficulty was adjusted to capture a continuum from adaptive to pathological levels of climate anxiety. To address limitations of previous tools, new items were also developed to reflect clinically important symptoms not previously assessed—such as irritability (item 7), which is particularly relevant in adolescent populations.”

 

Comment 4:

The authors indicate that it is important to include clinical criteria. However, in their sample they do not show a clinical population.

Response 4:

Thank you for this valuable input. The clinical applicability of a new anxiety scale is a critical consideration. The BACC was developed based on the DSM-5 diagnostic criteria for generalized anxiety disorder to identify symptoms of potential clinical relevance. As outlined in the Development Procedure section, the items were intentionally calibrated to assess both pathological and adaptive levels of climate anxiety by adjusting the overall difficulty level. As this study relied on community samples of adolescents and adults, we were unable to directly evaluate the scale's diagnostic utility. Future research should involve clinical or high-risk populations to assess the BACC's sensitivity, specificity, and potential cutoff scores. We have clearly described this limitation in the revised Discussion section.

Revision: Page 14, Paragraph 4, Lines 520-527.

“Fourth, although the BACC was developed based on DSM-5 criteria and established theoretical frameworks, its clinical applicability has not yet been tested in clinical or high-risk populations. The BACC was developed to assess both pathological and adaptive levels of climate anxiety by calibrating item difficulty. Consequently, the present study evaluated the scale's validity using a community-based sample. Future research should include replication studies with culturally and demographically diverse samples and assess the scale’s diagnostic utility in populations with clinically significant levels of climate-related anxiety.”

 

Comment 5:

The authors should mention the resources they used to review relevant information on climate anxiety, for example, databases or mention if they used any type of artificial intelligence tool.

Response 5:

Thank you for this helpful comment. To identify relevant literature on climate anxiety, we conducted searches using academic databases such as Google Scholar, PsycINFO, and PubMed. We confirm that no artificial intelligence tools were used in the process of searching or synthesizing the literature. As AI tools were not utilized in this instance, this particular point was not included in the manuscript. The databases used for the literature review have been specified in the revised Development Procedure section.

Revision: Page 3, Paragraph 7, Lines 144-145.

“To conduct this review, relevant literature was searched using Google Scholar, PsycINFO, and PubMed.”

 

Comment 6:

The authors indicate that they only relied on the DSM V, they do not show a solid theoretical support to develop their items. This point should be clarified.

Response 6:

Thank you for this insightful comment. In developing the items for the BACC, we referred not only to the DSM-5 diagnostic criteria for generalized anxiety disorder, but also to theoretical frameworks from previous studies on climate anxiety. Specifically, we incorporated conceptual elements such as excessive worry, emotional distress, uncontrollable rumination, and functional impairments, which have been consistently identified as core features of climate anxiety in the literature. However, we agree that this theoretical foundation was not clearly articulated in the original version of the manuscript. In response to your suggestion, we have revised the Development Procedure section (2.1.1) to explicitly describe the theoretical basis that informed item construction.

Revision: Page 4, Paragraph 1, Lines 159-164.

“In addition to these diagnostic foundations, the item development process also drew upon theoretical frameworks of climate anxiety [3, 8, 9]. The reviewed studies underscore key features of climate anxiety, including excessive worry, emotional distress, uncontrollable rumination, and functional impairments. These features were incorporated into the scale's item construction to ensure its capacity to adequately capture core psychological symptoms associated with climate anxiety.”

 

Comment 7:

It is worrying that the authors included adolescents in their study and did not have the approval of an IRB, which raises serious doubts about the procedures and results of the study.

Response 7:

Thank you for your important feedback. We would like to clarify that this study was fully approved by the Institutional Review Board (IRB) of Korea University (IRB No. KUIRB-2024-0314-01), and informed consent was obtained from all participants, including legal guardians for adolescent participants. While the IRB information was previously mentioned at the end of the Introduction and in the Institutional Review Board Statement section (6. Patents) of the manuscript, we acknowledge that this may not have been sufficiently prominent. To ensure that readers can more clearly identify the ethical approval, we have moved the IRB statement to the Methods section of the manuscript. Specifically, the IRB approval and consent procedures are now clearly stated in the Participants and Data Collection subsections of both Study 1 and Study 2.

Revision:

• Page 4, Paragraph 4, Lines 184-186: “This study was approved by the Institutional Review Board of Korea University (KUIRB-2024-0314-01), and informed consent was obtained from all participants.”
• Page 9, Paragraph 3, Lines 341-343: “The procedures for Study 2 were also approved by the Institutional Review Board of Korea University under the same protocol (KUIRB-2024-0314-01), and informed consent was obtained from all participants.”

 

Comment 8:

As it is ethically based to pay adolescents $5 to participate in a study, this is malpractice. This could generate only desired responses and therefore invalidate the results.

Response 8:

Thank you for your comment. We would like to respectfully clarify that the compensation provided (approximately $5 USD or 5,000 KRW) was a standard and modest amount in accordance with national research guidelines and ethical standards in South Korea. This level of compensation was not intended to serve as an incentive to influence participants' responses, but rather to compensate them for their time and effort. Additionally, the study protocol, including the compensation procedure, was reviewed and approved by the Institutional Review Board (IRB) of Korea University (KUIRB-2024-0314-01). We are therefore confident that the procedure adhered to appropriate ethical standards and did not compromise the validity of the data.

 

Comment 9:

Other points that discredit the work is the sample, which is insufficient to propose the creation of a new scale.

Response 9:

Thank you for your insightful feedback. We acknowledge your concerns regarding the adequacy of the sample size for scale development. However, we would like to clarify that the sample sizes used in both studies align with widely accepted guidelines in psychometric research.

Specifically, Study 1 involved 300 participants and was used to evaluate the factor structure of the initial 21-item pool. This exceeds the commonly recommended participant-to-item ratio of 10:1 for exploratory factor analysis (EFA), providing sufficient statistical power for factor extraction (Costello & Osborne, 2005). After careful consideration of the EFA results, 13 items were selected to comprise the final version of the scale.

Study 2 employed an independent sample of 400 participants to conduct confirmatory factor analysis (CFA) on this refined 13-item structure. This sample size aligns with established recommendations for CFA, particularly given the relatively simple two-factor model.

Furthermore, the utilization of two distinct samples—one for EFA and the other for CFA—further enhances the methodological rigor and robustness of the scale validation process. This approach is also consistent with other recent scale development studies published in Sustainability, where comparable sample sizes (e.g., 200 for EFA and 400 for CFA) were employed.

Taken together, we believe that the sample sizes used in the present study were appropriate and adequate for the initial development and validation of the BACC.

*Reference

Costello, A. B.; Osborne, J. Best Practices in Exploratory Factor Analysis: Four Recommendations for Getting the Most from Your Analysis. Pract. Assess. Res. Eval. 2005, 10(1). https://doi.org/10.7275/jyj1-4868.

Author Response File: Author Response.pdf

Reviewer 3 Report

Comments and Suggestions for Authors

This study developed and the Brief Anxiety Scale for Climate Change (BACC), a self-report measure designed to assess symptoms of climate anxiety. The comments are as follows. 

1. In the methods part, the association (linkage) between Study 1 and Study 2 should be clearly explained. 

2. There are many types of climate change, particulary in extreme weather, e.g., heat wave, cold spell, and related disaster, e.g., flood and drought. Have the authors considered them and put them in recognizing their different impact on anxiety and designing the scale? The authors need to explain it in details.

3. Climate change is a global issue, but its impact on population health, including mental health, is local, based on the diversity of natural and socio-demographical factors by countries and districts. Thus, the mental health concern (including anxiety) and scale from Korean population in this study should be cautiously interpreted in populations in other countries/regions. The authors need to disuss it and regard it as a limitation. 

4. References 19,20,21,23, the first letters of words in the titile of articles do not need to be as capital letters. E.g., "A Brief Measure for Assessing Generalized Anxiety Disorder: The GAD-7" should be as: "A brief measure for assessing generalized anxiety disorder: the GAD-7".

5. Some writing need to be corrected. E.g., in P-value, P should be as capital letter and in italic form.

Comments on the Quality of English Language

This study requires a moderate language correction. 

Author Response

Response to Reviewer 3

We would like to express our sincere gratitude to you for the constructive and insightful feedback, which contributed to enhancing the clarity and quality of our manuscript. Please find our detailed responses to each point below.

Comment 1:

In the methods part, the association (linkage) between Study 1 and Study 2 should be clearly explained. 

Response 1:

Thank you for your valuable comment. We agree that clarifying the linkage between Study 1 and Study 2 enhances the logical flow of the manuscript. In response, we have added a new sentence at the beginning of the “3.1. Method” section to explain that Study 2 was designed as a follow-up study to independently validate the factor structure and reliability of the BACC identified in Study 1.

Revision: Page 9, Paragraph 2, Lines 334-337:

“Study 2 was conducted to independently validate the two-factor structure identified in Study 1 and to reassess the reliability of the final 13-item version of the BACC. By employing a new sample, this follow-up study aimed to further examine the psychometric stability of the scale and its applicability to adolescent and adult populations.”

 

Comment 2:

There are many types of climate change, particulary in extreme weather, e.g., heat wave, cold spell, and related disaster, e.g., flood and drought. Have the authors considered them and put them in recognizing their different impact on anxiety and designing the scale? The authors need to explain it in details.

Response 2:

Thank you for this insightful comment. We acknowledge the importance of recognizing the diverse manifestations of climate change, including extreme weather events and related natural disasters. The BACC was not designed to assess psychological responses to specific types of climate events (e.g., heatwaves, floods) individually. Instead, it was intentionally developed to capture individuals’ subjective perception of climate change as a generalized, persistent, and threatening phenomenon. Although this scale was developed in South Korea, it was intended for broader application. We considered that individuals’ experiences of climate change may vary depending on their country’s environmental and sociopolitical context. To enhance the scale’s conceptual breadth and international applicability, we opted to focus on general psychological responses to climate change rather than event-specific reactions. To clarify this intention while also acknowledging its limitations, we have added a sentence to the discussion section explaining the rationale behind this design decision.

Revision: Page 14, Paragraph 3, Lines 512-519

“Moreover, although the manifestations of climate change—such as rising temperatures or changes in precipitation—may differ across regions [32], the scale was intentionally designed to capture individuals’ psychological responses to climate change as a persistent and abstract threat, rather than reactions to specific events. This approach contributes to enhancing its generalizability; however, cultural and regional factors may nonetheless influence how climate anxiety is experienced and expressed. Therefore, cross-cultural validation studies are needed to evaluate the applicability of the BACC across diverse sociocultural settings.”

 

Comment 3:

Climate change is a global issue, but its impact on population health, including mental health, is local, based on the diversity of natural and socio-demographic factors by countries and districts. Thus, the mental health concern (including anxiety) and scale from Korean population in this study should be cautiously interpreted in populations in other countries/regions. The authors need to disuss it and regard it as a limitation. 

Response 3: Thank you for your thoughtful comment regarding the cultural specificity of our sample. We fully agree with your important point. In response, we have taken two steps to address this concern. First, we have revised the title of the manuscript to explicitly include “in South Korea,” clearly indicating the national context in which the scale was developed and validated. Second, we have added a paragraph to the Discussion section (Limitations) to explicitly state the limitations in generalizing the findings beyond the South Korean context. We also emphasized the potential influence of cultural and socio-demographic factors on how climate-related distress is perceived and expressed and highlighted the need for future cross-cultural validation studies.

Revision:

• Page 1, Title revised to:

“Toward Sustainable Mental Health: Development and Validation of the Brief Anxiety Scale for Climate Change (BACC) in South Korea”

Page 14, Paragraphs 3-4, Lines 510-527.

“Third, while the BACC demonstrated sound psychometric properties among South Korean adolescents and adults recruited through online surveys, the generalizability of these findings to other populations and cultural contexts may still be limited. Moreover, although the manifestations of climate change—such as rising temperatures or changes in precipitation—may differ across regions [32], the scale was intentionally designed to capture individuals’ psychological responses to climate change as a persistent and abstract threat, rather than reactions to specific events. This approach contributes to enhancing its generalizability; however, cultural and regional factors may nonetheless influence how climate anxiety is experienced and expressed. Therefore, cross-cultural validation studies are needed to evaluate the applicability of the BACC across diverse sociocultural settings.

Fourth, although the BACC was developed based on DSM-5 criteria and established theoretical frameworks, its clinical applicability has not yet been tested in clinical or high-risk populations. The BACC was developed to assess both pathological and adaptive levels of climate anxiety by calibrating item difficulty. Consequently, the present study evaluated the scale's validity using a community-based sample. Future research should include replication studies with culturally and demographically diverse samples and assess the scale’s diagnostic utility in populations with clinically significant levels of climate-related anxiety.”

 

Comment 4:

References 19,20,21,23, the first letters of words in the titile of articles do not need to be as capital letters. E.g., "A Brief Measure for Assessing Generalized Anxiety Disorder: The GAD-7" should be as: "A brief measure for assessing generalized anxiety disorder: the GAD-7".

Response 4:

Thank you for your close and helpful review. We have carefully revised the capitalization in the titles of references 19, 20, 21, and 23 according to the proper sentence-case format, as suggested. Please note that the reference numbers have been updated in the revised manuscript due to formatting adjustments and the insertion of additional sources. We sincerely appreciate your attention to detail, which helped improve the consistency and formatting quality of our reference list.

Revision: Page 17 – 18, Line 645-651; 654-656

[22] Spitzer, R. L.; Kroenke, K.; Williams, J. B.; Löwe, B. A brief measure for assessing generalized anxiety disorder: The GAD-7. Arch. Intern. Med. 2006, 166 (10), 1092–1097. https://doi.org/10.1001/archinte.166.10.1092

[23] Kroenke, K.; Spitzer, R. L.; Williams, J. B. The PHQ-9: Validity of a brief depression severity measure. J. Gen. Intern. Med. 2001, 16 (9), 606–613. https://doi.org/10.1046/j.1525-1497.2001.016009606.x

[24] McWilliams, L. A.; Cox, B. J.; Enns, M. W. Use of the coping inventory for stressful situations in a clinically depressed sample: factor structure, personality correlates, and prediction of distress. J. Clin. Psychol. 2003, 59 (12), 1371–1385. https://doi.org/10.1002/jclp.10228

[26] Kim, S.-H.; Park, K.; Yoon, S.; Choi, Y.; Lee, S.-H.; Choi, K.-H. A brief online and offline (paper-and-pencil) screening tool for generalized anxiety disorder: The final phase in the development and validation of the mental health screening tool for anxiety disorders (MHS: A). Front. Psychol. 2021, 12, 639366. https://doi.org/10.3389/fpsyg.2021.639366

 

Comment 5:

Some writing need to be corrected. E.g., in P-value, P should be as capital letter and in italic form.

Response 5:

Thank you for your important feedback. We have carefully reviewed the entire manuscript and revised all instances of P-value formatting. Specifically, we ensured that P is consistently capitalized and italicized (e.g., P < .001) throughout the text and tables, in accordance with academic style guidelines.

 

*P.S. Finally, thank you for your valuable feedback regarding the English expression. Prior to submission, the manuscript underwent professional native English editing through the academic editing service Editage. Nevertheless, we fully recognize the importance of clarity and readability in academic writing and sincerely appreciate your suggestion. Accordingly, we arranged an additional round of professional editing. Please find the certificate of editing attached.

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

Comments and Suggestions for Authors

Revised paper is ok

Reviewer 2 Report

Comments and Suggestions for Authors

The authors have included a majority of observations

Reviewer 3 Report

Comments and Suggestions for Authors

The authors have revised the manuscript according to the reviewer's comments. Thus, the manuscript can be accepted aftr proofreading.