DTM-Based Analysis of Hot Topics and Evolution of China’s Energy Policy

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

Title: DTM -Based Analysis of Hot Topics and Evolution of China's Energy Policy

 The topic of this manuscript has good theoretical value and practical significance, but there are still the following problems:

 1. The abstract talks about 1,872 energy policy documents; the author should have summarised them in an appendix or a table to make it more transparent.

2. The introduction should introduce the international background of energy policy first and then turn to the Chinese case.

3. In the literature review section, it is recommended that a detailed review of energy policy be done and that the contribution of this study be highlighted.

4. What are the characteristics of the Dynamic Topic Model (DTM)? What are the advantages of DTM in exploring the evolution of energy policy? Please explain.

5. Figure 1. Distribution of energy policies by year of publication, Figure 2. Topic coherence of the number of topics. The picture needs to be more apparent. What is the economic logic behind it? Please explain.

6. The author divides energy policy into four stages: the Energy Strategy Initiation Stage (1980-1990). Energy Strategy Initiation Stage (1980-1990), Energy Development Acceleration Stage (1991-2005). Critical Energy Transition Stage (2006-2015). Modern Energy System Optimisation Stage (2016-2023). What are the criteria for classification? Is it authoritative?

7. Some additional quantitative analyses could be considered in the analysis of Topic Evolution Paths.

8. Could there be comparative analyses internationally or between provinces? This might be a worthwhile element to analyze.

9. The whole paper lacks quantitative analyses, and additions are suggested. In addition, the current study is

10. Conclusion and policy implications: consider adding a discussion section that elaborates on the contribution to the international dimension.

11. Please check the references carefully.

12. Please check the entire text carefully.

Comments on the Quality of English Language

Minor editing of English language required.

Author Response

Dear editor and reviewers,

Thanks for your letter and the reviewers’ comments concerning our manuscript (ID: sustainability-3137515). Those comments are all valuable and very helpful for revising and improving our manuscript, as well as having the important guiding significance to our studies. We have studied comments carefully and tried our best to revise the manuscript. Revised portions are marked in red in the revised manuscript. The point-to-point responses to the reviewer’s comments are listed as follows:

 

Reviewer 1#

The topic of this manuscript has good theoretical value and practical significance, but there are still the following problems:

(1) The abstract talks about 1,872 energy policy documents; the author should have summarised them in an appendix or a table to make it more transparent.

A1: Thanks for your comments. According to your comments, we have summarized and categorized the 1872 energy policy texts based on their effectiveness levels, issuing departments, main content, etc. Please refer to Table 1 in the revised manuscript.

(2) The introduction should introduce the international background of energy policy first and then turn to the Chinese case.

A2: Thanks for your instructive suggestions. According to your comments, we have rewritten the Section 1. Introduction. We provided a more detailed description of the refined themes related to the research question in this article, including the key issues or gaps in current international energy policy research. Please see the first paragraph of the Section 1. Introduction for more information.

• In the literature review section, it is recommended that a detailed review of energy policy be done and that the contribution of this study be highlighted.

A3: Thanks for your careful review of our manuscript. According to your comments, we have rewritten the Section 2. Literature review. In this section, we have categorized and summarized both domestic and international research on energy policy, drawing from a variety of high-quality journals. We have identified current research gaps and outlined the innovative framework of this paper. Also, the introduction has been revised to highlight the theoretical contributions of this study, particularly in the third paragraph. Please see the revised manuscript.

(4) What are the characteristics of the Dynamic Topic Model (DTM)? What are the advantages of DTM in exploring the evolution of energy policy? Please explain.

A4: Thanks for your valuable suggestions. In Section 3.1.1, the paper provides a detailed description of the Dynamic Topic Model (DTM) and highlights its advantages in analyzing the evolution of energy policy topics compared to other models. Please see the Section 3.1.1 in the revised manuscript.

(5) Figure 1. Distribution of energy policies by year of publication, Figure 2. Topic coherence of the number of topics. The picture needs to be more apparent. What is the economic logic behind it? Please explain.

A5: Thanks for your careful review of our manuscript. Figure 1 depicts the annual distribution of the 1872 energy policy texts, while Figure 2 utilizes DTM's coherence analysis to extract and summarize the topics from these texts, identifying five key topics. Despite their visual similarities, the two figures are fundamentally distinct. As a result, Sections 3.2.1 Data Acquisition and 3.2.3 Determining the Optimal Number of Topics have been revised in this paper. Please see the revised manuscript.

(6) The author divides energy policy into four stages: the Energy Strategy Initiation Stage (1980-1990). Energy Strategy Initiation Stage (1980-1990), Energy Development Acceleration Stage (1991-2005). Critical Energy Transition Stage (2006-2015). Modern Energy System Optimisation Stage (2016-2023). What are the criteria for classification? Is it authoritative?

A6: Thanks for your comments. The Five-Year Plans (FYP) are foundational documents created every five years by the Chinese government to guide economic and social development. Each plan is influenced by its unique historical context and development goals, reflecting China's strategic growth trajectory during different periods. Additionally, each FYP outlines specific energy policies, including those related to energy structure adjustments, technological innovations, and efficiency improvements, allowing for a comprehensive exploration of the evolution of China's energy policy themes. So, we divided the energy policies into four stages based on China's Five-Year Plans and the adjustment directions of energy policies. Please see the Section 3.2.2 Sample Stage Division in the revised manuscript.

(7) Some additional quantitative analyses could be considered in the analysis of Topic Evolution Paths.

A7: Thanks for your instructive suggestions. According to your comments, we have included Table 3 Topic Types and Regression Results, as well as Table 4 Characteristics of China's Energy Policy Themes Network Structure. Table 3 presents the results of linear regression analysis on the topics for each year, identifying hot and cold topics. Table 4 examines the characteristics of the topic collaboration network, utilizing various network metrics to assess the level of collaboration within the network and its evolutionary trends. Please see the Section 4.3 Analysis of Topic Evolution Paths in the revised manuscript.

(8) Could there be comparative analyses internationally or between provinces? This might be a worthwhile element to analyze.

A8: Thanks for your instructive suggestions. Since the energy policy text data in this paper primarily originates from energy policies issued at the central level, conducting comparative analyses at the international and interprovincial levels would require collecting policy texts from other countries and various provinces in China. This would involve a much larger dataset and more complex data processing. Therefore, this issue has been addressed in section 5.3 Limitations and Avenues for Future Research, to guide future efforts in enhancing research in this area. Thank you again for your insightful questions. This paper utilizes DTM and SNA methods for quantitative text analysis of energy policies. While the empirical analysis was somewhat limited, the study has been enhanced by incorporating a linear regression of hot/cold topics for each year and exploring the collaborative characteristics of the topic network, enriching the content of the research.

(9) The whole paper lacks quantitative analyses, and additions are suggested.

A9: Thanks for your instructive suggestions. This paper utilizes DTM and SNA methods for quantitative text analysis of energy policies. While the empirical analysis was somewhat limited, the study has been enhanced by incorporating a linear regression of hot/cold topics for each year and exploring the collaborative characteristics of the topic network, enriching the content of the research. The results of the empirical analysis are presented in Table 3 and Table 4 in the revised manuscript.

(10) Conclusion and policy implications: consider adding a discussion section that elaborates on the contribution to the international dimension.

A10: Thanks for your comments. According to your comments, we divided Section 5.1 Discussion and Conclusions into two parts: Section 5.1 Discussion and Section 5.2 Conclusions. In Section 5.1 Discussion, we explored the research background, reasons for the results, and differences from previous research findings. In Section 5.2 Conclusions, we provided a comprehensive summary of the research results. Please see the Section 5.1 Discussion and Section 5.2 Conclusions in the revised manuscript.

(11) Please check the references carefully.

A11: Thank you for bringing the quality issues with the references to our attention. After a thorough review, we have carefully revised the references in accordance with the guidelines of Sustainability, adding high-quality articles from reputable journals such as Atmospheric Pollution Research, Energy Policy, and Journal of Cleaner Production. The number of references has also been increased to 53 as outlined in the reference section of the revised manuscript.

(12) Please check the entire text carefully.

A12: Thanks for your review of our manuscript. We have meticulously revised the overall framework design, content arrangement, logical structure, language expression, and word choice of the article. Please see the revised manuscript.

 

Reviewer 2 Report

Comments and Suggestions for Authors

The manuscript "DTM—Based Analysis of Hot Topics and Evolution of China's Energy Policy" aims to extract complete energy policy themes, employing  DTM (Dynamic Topic Model) for both horizontal and popularity analyses. This study employs DTM and social network analysis methods to examine 1,872 energy policy documents issued in China between 1980 and 2023, focusing on hot topic detection and trend evolution. It's a thought-provoking article. However, some parts of the manuscript need further clarification and improvement.

- Manuscripts submitted to Sustainability should include an Abstract, Keywords, Introduction, Materials and Methods, Results, Discussion, and Conclusions sections. Please see Instructions for Authors.

- The abstract should be a maximum of about 200 words.

- Please report the search to allow for easy replication by others (details on who developed and executed the search, all search limitations and filters applied, etc.).

- Please provide a research methodology flowchart showing the reader the results of the literature search and full screening process (number of laws, duplicate screening, verification, etc.).

- Figure 2 shows a coherence score of 0.335. Is it a good or bad result? Should low coherence need to be fixed?

- In the Discussion section, authors should discuss the results and how they can be interpreted from the perspective of previous studies and the working hypotheses. The findings and their implications should be discussed in the broadest context possible, and the limitations of the work should be highlighted. Future research directions may also be mentioned.

Author Response

Dear editor and reviewers,

 

Thanks for your letter and the reviewers’ comments concerning our manuscript (ID: sustainability-3137515). Those comments are all valuable and very helpful for revising and improving our manuscript, as well as having the important guiding significance to our studies. We have studied comments carefully and tried our best to revise the manuscript. Revised portions are marked in red in the revised manuscript. The point-to-point responses to the reviewer’s comments are listed as follows:

 

Reviewer 2#

The manuscript "DTM—Based Analysis of Hot Topics and Evolution of China's Energy Policy" aims to extract complete energy policy themes, employing DTM (Dynamic Topic Model) for both horizontal and popularity analyses. This study employs DTM and social network analysis methods to examine 1,872 energy policy documents issued in China between 1980 and 2023, focusing on hot topic detection and trend evolution. It's a thought-provoking article. However, some parts of the manuscript need further clarification and improvement.

• Manuscripts submitted to Sustainability should include an Abstract, Keywords, Introduction, Materials and Methods, Results, Discussion, and Conclusions sections. Please see Instructions for Authors.

A1: Thanks for your careful review of our manuscript. According to your comments, we have carefully reviewed the author guidelines and designed the framework of the paper, specifically including the Abstract, Keywords, Introduction, Literature Review, Materials and Methods, Results, Discussion, and Conclusions sections. Please see the revised manuscript.

• The abstract should be a maximum of about 200 words.

A2: Thanks for your instructive suggestions. According to your comments, we have revised the abstract to ensure it is within 200 words. Please refer to the abstract section for specific modifications.

• Please report the search to allow for easy replication by others (details on who developed and executed the search, all search limitations and filters applied, etc.).

A3: Thanks for your valuable suggestions. According to your comments, we have provided a detailed explanation of the energy policy retrieval process, focusing on the developer and website of the Peking University Law Database, as well as the parameter settings and steps involved in data retrieval. This will allow others to replicate the process based on the outlined steps. The specific modifications can be found in section 3.2.1 Data Acquisition.

(4) Please provide a research methodology flowchart showing the reader the results of the literature search and full screening process (number of laws, duplicate screening, verification, etc.).

A4: Thanks for your comments. we have included Figure 2 Research Roadmap, which clearly outlines the data retrieval process, topic mining process, and result analysis process. Please see 3.2.3 Text Data Preprocessing in the revised manuscript.

(5) In the Discussion section, authors should discuss the results and how they can be interpreted from the perspective of previous studies and the working hypotheses. The findings and their implications should be discussed in the broadest context possible, and the limitations of the work should be highlighted. Future research directions may also be mentioned.

A5: Thanks for your comments. According to your comments, we divided Section 5.1 Discussion and Conclusions into two parts: Section 5.1 Discussion and Section 5.2 Conclusions. In Section 5.1 Discussion, we explored the research background, reasons for the results, and differences from previous research findings. In Section 5.2 Conclusions, we provided a comprehensive summary of the research results. Please see the Section 5.1 Discussion and Section 5.2 Conclusions in the revised manuscript.

In addition, we have included section 5.3 Limitations and Avenues for Future Research, where we address the limitations in the text and suggest ways for future research to tackle these issues. The specific changes can be found in Section 5.3.

Round 2

Reviewer 1 Report

Comments and Suggestions for Authors

Many thanks to the authors for their careful replies, the article has been much improved with the following issues:

1. Figure 5. Heatmap of Core Topic Popularity Across Different Stages. Please explain in some detail for this figure.

2. Table 4. Characteristics of the Network Structure of China's Energy Policy Themes. The basis for the delineation of time periods can be updated clearly.

3. Please check the whole text carefully.

 

Comments on the Quality of English Language

Minor editing of English language required.

Author Response

Reviewer 1#

Many thanks to the authors for their careful replies, the article has been much improved with the following issues:

(1) Figure 5. Heatmap of Core Topic Popularity Across Different Stages. Please explain in some detail for this figure.

A1: Thanks for your comments. According to your comments, We have added an explanation for the figure 5 in section "4.2 Temporal Analysis of Core Topic Popularity in Energy Policies". Please see the revised manuscript.

(2) Table 4. Characteristics of the Network Structure of China's Energy Policy Themes. The basis for the delineation of time periods can be updated clearly.

A2: Thanks for your careful review of our manuscript. In section 3.2.2 Sample Stage Division, China's energy policy development is categorized into four stages based on adjustments to energy policies and critical milestones within the Five-Year Plans. We have also adjusted the corresponding stage divisions and results in the subsequent sections due to segmentation criteria and data updates. Specific details can be found in the provided data, processing code, and results in the text. Thank you once again for your question. Please see the 3.2.2 Sample Stage Division in the revised manuscript.

• Please check the whole text carefully.

A3: Thanks for your review of our manuscript. We have meticulously revised the overall framework design, content arrangement, logical structure, language expression, and word choice of the article. Please see the revised manuscript.

 

 

Reviewer 2 Report

Comments and Suggestions for Authors

Thanks to the authors for the changes that have been made to the manuscript. Unfortunately, the paper still needs major revisions.

- the authors state in the Abstract that they identify five core topics with high coherence (Lines 15-16). However, Fig. 3 shows a low level of coherence.

- Jung & Kim (https://doi.org/nd25) are missing the statements that the authors cite in their manuscript  (Lines 331-334). The manuscript needs to be proofread.

- Please find information about Topic coherence on the IBM website (https://shorturl.at/bHG8D).

- In the Discussion section, authors should discuss the results and how they can be interpreted from the perspective of previous studies and the working hypotheses. The authors should continue the discussion section.

- Authors should read the Instructions for Authors and prepare Manuscripts according to the Sustainability requirements. 

- There needs to be a citation to source number 54 in the reference list.

Author Response

Reviewer 2#

Thanks to the authors for the changes that have been made to the manuscript. Unfortunately, the paper still needs major revisions.

• the authors state in the Abstract that they identify five core topics with high coherence (Lines 15-16). However, Fig. 3 shows a low level of coherence.

A1: Thanks for your instructive suggestions. According to your comments, We have further explored the core topics, setting the number of topics from 1 to 15 with a step size of 1, and calculated their coherence scores, which range from 0 to 1. According to the results, the model achieves the best convergence when the number of topics is set to 5, as shown in Figure 3 of this paper. Referring to the IBM website (https://shorturl.at/bHG8D), which states, "Remember that, while popular, topic coherence scores are not an absolute indicator of topic quality. Nevertheless, they can be one useful metric for gauging model performance," we also analyzed the reasons behind this result (0.335). One possible reason is that we used Chinese keywords instead of English keywords; another reason could be the model's insensitivity to extracting energy-related keywords. Therefore, under the premise that this result cannot be changed, we visualized the topic results to further explain (Figure 4, PyLDAvis Result Visualization). When the number of topics is set to 5, the boundaries between the topics are clear, and the classification effect is good. Please see the revised manuscript.

• Jung & Kim (https://doi.org/nd25) are missing the statements that the authors cite in their manuscript (Lines 331-334). The manuscript needs to be proofread.

A2: Thanks for your comments. According to your comments, we carefully reviewed the literature citations and found that reference number 54 is Vayansky and Kumar (10.1016/j.is.2020.101582). Therefore, we have cited the statements “some techniques are also not representative of real-world data relationships . This is due to assumptions regarding key parameters in the calculation process and inefficiency of many optimization methods, which often attempt to overcome uncertainty by performing many time-consuming iterations to determine the best value for the parameter.” from this article. The specific modifications can be found in section 3.2.3 Determining the Optimal Number of Topics.

• Please find information about Topic coherence on the IBM website (https://shorturl.at/bHG8D).

A3: Thank you for providing valuable information on the topic model. It has been extremely helpful for our text analysis. After reading the material, we now have a deeper understanding of topic coherence. Following Jacob Murel, Ph.D.'s approach, we visualized the topic results to demonstrate the robustness of extracting five topics.

(4) In the Discussion section, authors should discuss the results and how they can be interpreted from the perspective of previous studies and the working hypotheses. The authors should continue the discussion section.

A4: Thanks for your comments. Based on your feedback, paragraphs 2-4 of the discussion section have been revised. The analysis now includes a more comprehensive examination of the research results, incorporating both the time and spatial dimensions. Additionally, a detailed discussion is provided in relation to the research background and previous findings by other authors. Please see the revised manuscript.

(5) Authors should read the Instructions for Authors and prepare Manuscripts according to the Sustainability requirements.

A5: Thanks for your comments. According to your comments, we have carefully reviewed the "Author Guidelines" and made detailed revisions to the article structure, figure and table formats, and reference formats as required by Sustainability.

• There needs to be a citation to source number 54 in the reference list.

A6: Thanks for your comments. According to your comments, We have added reference number 54 to the bibliography as per your suggestion. Specific modifications can be found in reference [54].

   

Round 3

Reviewer 2 Report

Comments and Suggestions for Authors

Unfortunately, the authors have not improved the quality of the LDA model they use to describe the results. Although the coherence score is not an absolute indicator, it is “one useful metric for gauging model performance”. Also, the authors still state in the Abstract that they identify five core topics with high coherence (Lines 15-16). However, Figure 3 shows a poor score of coherence in their model.

Authors should read the Instructions for Authors and prepare Manuscripts according to the Sustainability requirements. 

Author Response

Dear editor,

 

Thanks for your letter and the reviewers’ comments concerning our manuscript (ID: sustainability-3137515). We have tried our best to revise the manuscript. Revised portions are marked in red in the revised manuscript. The point-to-point responses to the reviewer’s comments are listed as follows:

 

Reviewer 1#

Unfortunately, the authors have not improved the quality of the LDA model they use to describe the results. Although the coherence score is not an absolute indicator, it is “one useful metric for gauging model performance”. Also, the authors still state in the Abstract that they identify five core topics with high coherence (Lines 15-16). However, Figure 3 shows a poor score of coherence in their model.

A1: Thanks for your instructive suggestions. According to your comments, we have adjusted the coherence testing process of this paper by referring to practices related to Topic Coherence on the IBM website(https://shorturl.at/bHG8D). We reduced irrelevant stop words and dynamically adjusted the alpha parameter, resulting in a final coherence value of 0.504. While the result is not perfect, we have put forth our best effort. In future research, we will focus on the latest findings in this field and continue to improve our methods of coherence testing. Thank you for pointing out the detailed issues, we appreciate your rigorous scientific attitude and work ethic.

(2) Authors should read the Instructions for Authors and prepare Manuscripts according to the Sustainability requirements. 

A2: Thanks for your careful review of our manuscript. According to your comments, we have revised the entire document's format in accordance with the author guidelines. This includes adjustments to the author's email annotations, the size and resolution of figures and tables, as well as the structure of the chapter contents. Please see the revised manuscript.