Dynamic Water-Energy-Carbon Trade-Off Optimization for Heavy Industry Decarbonization via Deep Reinforcement Learning: A UK Case Study
Round 1
Reviewer 1 Report
Comments and Suggestions for AuthorsAvoid very long sentences eg.first sentence in abstract. This must be corrected throughout the manuscript
“Consequently, while local carbon emissions decrease, part of the environmental burden is shifted to regional power and water systems”. This sentence needs clarification. How this claim is justified?
Apart from CCUS are there any ways to cut off carbon emissions in cement industry? Discussion is needed on this in introduction section
Please don’t indicate numbering for nomenclature section, it must be kept before introduction or at the end . refer author guidelines
First paragraph of section 3 has some repetition of section 1. Please avoid repetition , make the article crisp
WCMPI is proposed by the authors are it is available already in literature? Reference needed
References are needed for all the equations used
Which tool was used for solving the equations?
What kind of data were obtained? They are not mentioned clearly in literature
Details of CCUS are missing
Fig 4 and 5 must be improved
References are needed for cost components, formulas used??
Why hydro and renewables are considered separately? What are the energy sources considered as renewables in this work? Make it clear
List the limitations of this work?
Water saving potential is missing in conclusion
Scope for future work must be included
What the scope for real time applicability of the results of this work or methodology adopted in the cement industry?
How total costs details were arrived?
Article must be made crisp
Comments on the Quality of English LanguageLong sentences must be avoided. Readability must be improved
Author Response
Reviewer 1, Round 1
Author 1, Comment 1: Avoid very long sentences eg., first sentence in abstract. This must be corrected throughout the manuscript.
Response: Thanks for highlighting this. Your comment is now incorporated in the revised version.
Author 1, Comment 2: “Consequently, while local carbon emissions decrease, part of the environmental burden is shifted to regional power and water systems”. This sentence needs clarification. How is this claim justified?
Response: We revised the relevant paragraph in the introduction to explain the mechanism explicitly. The revised text now states that, although stack-level carbon emissions decline, CCUS increases electricity demand for capture and compression and raises cooling-water demand. As a result, part of the burden is shifted upstream to the regional power system and laterally to regional water systems. We also added supporting citations to the water-footprint and hydropower-footprint literature.
Changes: The corresponding clarification is highlighted in red in the manuscript [lines # 40-57].
Author 1, Comment 3: Apart from CCUS are there any ways to cut off carbon emissions in cement industry? Discussion is needed on this in introduction section.
Response: To discuss the wider aspects of cement decarbonization pathways beyond CCUS, the authors added a new paragraph in the introduction that briefly discusses clinker-factor reduction through supplementary cementitious materials, alternative and low-carbon fuels, process-efficiency improvements, waste-heat recovery, electrification of selected process steps, and material-efficiency measures. We also clarified that carbon capture remains important in deep-decarbonization pathways because a large share of cement emissions is process-related and originates from limestone calcination. The corresponding discussion and references are highlighted in red in the manuscript.
Changes: See the revised details in the revised manuscript [Lines # 58-69]
Author 1, Comment 4: Please don’t indicate numbering for nomenclature section, it must be kept before introduction or at the end. refer author guidelines.
Response: Thanks for pointing this out. The authors have removed the section numbering for the nomenclature section and moved it before the introduction section.
Changes: See updated Table 1 on page 2 of the revised version.
Author 1, Comment 5: First paragraph of section 3 has some repetition of section 1. Please avoid repetition, make the article crisp.
Response: To address this issue, the authors rewrote the first paragraph of Section 3 (Section 2in the revised version) so that it is more focused and directly oriented toward the motivation for the research gap. The revised paragraph now emphasizes the coupled system challenge and the central research question without restating the broader introductory background.
Changes: The corresponding revision is highlighted in red in the manuscript [Page # 3, Line # 113-123].
Author 1, Comment 6: WCMPI is proposed by the authors are it is available already in literature? Reference needed.
Response: We clarified in the manuscript that WCMPI is proposed by the “authors” in this study and is not presented as an already established named index from the literature, as per our knowledge through our literature review. Instead, it is introduced as a study-specific metric motivated by the broader literature on water footprints, carbon footprints, and environmental trade-offs associated with CCUS.
To make this clear, we revised the abstract, the end of the introduction, and the dedicated WCMPI subsection so that the novelty of the metric is stated explicitly. We also added supporting references to the related footprint and trade-off literature. The corresponding revisions are highlighted in red in the manuscript.
Changes: See the updated abstract and last paragraph of introduction [Line # 105-112].
Author 1, Comment 7: References are needed for all the equations used.
Response: We performed an equation-by-equation review of all the sections and added citations wherever the formulation follows existing literature. In particular, we added references for the standard MDP framework, the energy-hub and water-energy balance equations, and the continuous-control reinforcement-learning formulation. For equations that are specific to this study, such as the proposed WCMPI definition, the customized coupling matrix, and the study-specific state, action, and reward design, we revised with the relevant updated text to state explicitly that these formulations are proposed or adapted in this work, while still citing the broader literature that motivated them.
Changes: These changes are seen in the manuscript with updated references for equations.
Author 1, Comment 8: Which tool was used for solving the equations?
Response: Thank you for this important question. We clarified in the manuscript that
the governing equations were not solved using a separate symbolic or algebraic equation
solver such as GAMS, MATLAB symbolic tools, or a dedicated nonlinear programming
package. Instead, the equations were evaluated numerically within a custom Python simulation environment built with “gymnasium”. The control policy was then optimized using
the reinforcement-learning implementations of SAC and PPO provided through PyTorch
and “Stable-Baselines3”. We added this clarification to the implementation paragraph and highlighted it in red in the manuscript.
Changes: See the highlighted changes in the revised version [Lines #282-320].
Author 1, Comment 9: What kind of data were obtained? They are not mentioned clearly in literature.
Response: We revised the “Dataset & Modeling Methodology” subsection to state the data categories explicitly. The revised section highlights: (i) hourly energy-system data, including grid carbon intensity, baseline thermal generation, and offshore wind availability from NESO; (ii) water-regulation data, including abstraction limits from the Environment Agency; and (iii) scenario-definition inputs used to construct drought, low-renewable, and stressed operating cases. We also clarified that these data are used directly in the state representation, constraints, and reward calculations of the MDP environment.
Changes: The corresponding revision is highlighted in red in the manuscript [Line # 282-298].
Author 1, Comment 10: Details of CCUS are missing.
Response: We added a short clarifying paragraph in the methodology section describing the CCUS configuration considered in this study. The revised text now explains that the model represents a post-combustion amine-based capture chain, including flue-gas absorption, solvent regeneration, CO2 compression, and the associated cooling and auxiliary energy requirements. We also clarified that transport and permanent storage are acknowledged as part of the wider CCUS chain, but that the present model focuses on the on-site capture and compression stages because they dominate the plant-level water-energy-carbon interactions. The corresponding revision is highlighted in red in the manuscript.
Changes: The changes can be seen in the revised version [Lines # 170-178].
Author 1, Comment 11: Figures 4 and 5 must be improved.
Response: We have updated these figures for better visibility and understanding.
Author 1, Comment 12: References are needed for cost components, formulas used??
Response: We reviewed the cost-related components and clarified their basis in the manuscript. In particular, we revised the reward-function description and the quantitative benchmarking paragraph to show how the cost terms are constructed. The operating-cost component is now linked explicitly to standard energy-hub operating-cost formulations from the literature, while the carbon- and water-related valuation terms are tied to the footprint and nexus literature used in this study. We also clarified that the regulatory fine term is study-specific and is based on constraint-violation penalties associated with abstraction and dispatch limits. These supporting references and explanations have been added and highlighted in red in the manuscript.
Changes: The changes can be seen in the revised version [Lines # 235-242].
Author 1, Comment 13: Why are hydro and renewables considered separately? What are the energy sources considered as renewables in this work? Make it clear.
Response: Thanks for highlighting this, and the authors have now clarified this in both the “Methodology” and the “Dataset & Modeling Methodology” subsection. In the revised manuscript, it is highlighted that hydropower is treated separately because it is modeled as a dispatchable pumped-storage resource with its own internal reservoir state and controllable release decisions. By contrast, the renewable term $Erent$ (in Equation 7) represents exogenous non-dispatchable renewable availability. In this study, the renewable input specifically corresponds to offshore wind data obtained from the UK dataset. These clarifications have been added to the revised manuscript.
Changes: The changes can be seen in the revised version [Lines # 282-298].
Author 1, Comment 14: List the limitations of this work?
Response: We agree that the manuscript should explicitly state the main limitations of the present study. We therefore added a dedicated “Limitations” subsection before the conclusions. The new text now clarifies the principal limitations related to regional specificity of the UK case study, the plant-level scope of the CCUS representation, the simplified renewable-source representation, the calibration dependence of some model coefficients, and the gap between simulation performance and full industrial deployment. These additions are highlighted in red in the manuscript.
Changes: The changes can be seen in the revised version [Lines # 452-467, Section 4.9].
Author 1, Comment 15: Water saving potential is missing in conclusion.
Response: We revised the conclusion to explicitly highlight that the proposed framework reduces dependence on freshwater abstraction by shifting part of the cooling-water demand to reclaimed municipal wastewater. The revised conclusion now also states that this coordinated strategy helps avoid the 2.15-5.17% hydrological stress increase observed under unmanaged carbon-mitigation pathways. This addition is highlighted in red in the manuscript.
Changes: The changes can be seen in the revised version [Lines # 468-498, Section 5].
Author 1, Comment 16: Scope for future work must be included.
Response: We revised the conclusion to include a future-work statement, which reveals that future studies should extend the framework to multi-region and full-chain CCUS settings, include broader renewable portfolios and network constraints, and validate the learned control policies through plant-level deployment studies. This addition is highlighted in red in the manuscript.
Changes: The changes can be seen in the revised version [Lines # 493-498, Section 5].
Author 1, Comment 17: What is the scope for real time applicability of the results of this work or methodology adopted in the cement industry?
Response: We added clarification in the discussion section to explain the industrial scope of
the framework. The revised text now states that, once trained, the DRL policy can pro-
vide near-instant operational decisions from live plant and market signals, which makes it
suitable as a real-time decision-support layer. We also clarify that practical deployment
in cement plants would still require plant-specific calibration, integration with SCADA or
supervisory control systems, sensor-quality assurance, safety interlocks, and phased pilot
validation before autonomous closed-loop implementation. This addition is highlighted in
red in the manuscript.
Changes: The changes can be seen in the revised version [Section 4, and 4.7 particularly].
Author 1, Comment 18: How total costs details arrived?
Response: The revised text now clarifies that total cost is calculated by combining three
categories at each simulation time step: (i) operating-cost terms associated with thermal
generation and plant energy use; (ii) regulatory and feasibility penalties due to abstraction
or dispatch violations; and (iii) carbon- and water-related valuation terms derived from the
methodology. These components are then summed over the full dispatch horizon for each
scenario, so the reported values represent cumulative system-level cost.
Changes: This clarification is highlighted in red in the manuscript [Lines # 325-342, Section # 4.6].
Author 1, Comment 19: The article must be made crisp.
Response: We have thoroughly revised the manuscript to improve its quality.
Reviewer 2 Report
Comments and Suggestions for Authors1. The research background presented in the Introduction is well developed. However, the literature review lacks sufficient depth. Regarding studies on the WEC nexus, the manuscript mainly focuses on research methods and model applications, yet only provides a single example, which is not comprehensive. In addition to methodologies, aspects such as research regions and research objects should also be systematically reviewed. A more thorough reading, synthesis, and critical assessment of the literature are needed. This would better demonstrate the authors’ understanding of the research topic and provide a solid basis for identifying research gaps.
2. In the results analysis, the choice of specific methods and models should be more clearly justified.
3. The manuscript should clearly indicate the original sources of the data used.
4. In addition to the implications section, the paper should include a dedicated discussion section to engage with the existing literature. By comparing the study’s key findings with related research, the authors can more effectively highlight the contribution of this paper.
Author Response
Reviewer 2, Round 1
Reviewer 2, Comment 1: The research background presented in the Introduction is well developed. However, the literature review lacks sufficient depth. Regarding studies on the WEC nexus, the manuscript mainly focuses on research methods and model applications, yet it only provides a single example, which is not comprehensive. In addition to methodologies, aspects such as research regions and research objects should also be systematically reviewed. A more thorough reading, synthesis, and critical assessment of the literature are needed. This would better demonstrate the authors’ understanding of the research topic and provide a solid basis for identifying research gaps.
Response: We revised the Introduction to provide a broader and more critical review of WEC-nexus studies. In the revised text, we now distinguish among: (i) deterministic and mixed-integer energy-hub optimization studies; (ii) stochastic and uncertainty-aware water-energy nexus models; (iii) technology-specific footprint studies on hydropower and CCUS; and (iv) emerging DRL-based operational approaches. We also clarify how the literature differs by research object, including utility-scale water systems, multi-energy hubs, and industrial decarbonization technologies, and we note that many prior studies are either calibrated to generic systems or to non-industrial settings rather than to heavy-industry CCUS applications in a specific regional context. Based on this broader synthesis, we then sharpen the identified research gap as both methodological and contextual. These additions are highlighted in blue in the revised manuscript.
Changes: The changes can be seen in the revised version [Section I: Introduction].
Reviewer 2, Comment 2: In the results analysis, the choice of specific methods and models should be more clearly justified.
Response: We revised the quantitative benchmarking discussion to state explicitly why SAC, PPO, and the rule-based baseline were selected. In the revised text, SAC is justified as the main method because the problem involves continuous control, stochastic exogenous signals, and tightly coupled nonlinear constraints. PPO is retained as a widely used policy-gradient benchmark, while the rule-based controller provides an engineering baseline for interpretability. This clarification is now highlighted in blue in the revised manuscript.
Changes: The changes can be seen in the revised version [Section # 4.6].
Reviewer 2, Comment 3: The manuscript should clearly indicate the original sources of the data used.
Response: Thank you for this important comment. We agree that the original sources of the data should be stated clearly. This point has now been clarified explicitly in the revised manuscript, particularly in the “Dataset & Modeling Methodology” subsection. The revised text states that the energy-system inputs, including hourly grid carbon-intensity, baseline
thermal-generation, and offshore-wind availability data, were obtained from UK dispatch records reported by the National Energy System Operator (NESO) and accessed through the cited UK dataset source. It also states that the water-regulation inputs were derived from the Environment Agency planning guideline used in this study. In addition, we clarified
that the renewable input term corresponds specifically to offshore wind from the same UK dataset. These source clarifications are highlighted in blue in the revised manuscript.
Changes: The changes can be seen in the revised version [Section # 4.1].
Reviewer 2, Comment 4: In addition to the implications section, the paper should include a dedicated discussion section to engage with the existing literature. By comparing the study’s key findings with related research, the authors can more effectively highlight the contribution of this paper.
Response: Dear Reviewer, to address this, we added a dedicated discussion subsection entitled “Discussion in Relation to Existing Literature” before the limitations section. In this new subsection, we compare our findings with prior work on energy-hub optimization, stochastic water-energy modeling, water-footprint studies of CCUS and hydropower, and DRL-based energy-hub scheduling. This comparison allows us to position the present contribution more clearly, especially by showing how the paper extends prior studies from static or generic hub formulations toward a dynamic, heavy-industry WEC framework with explicit CCUS, wastewater reuse, abstraction constraints, and regional operating signals. These additions are highlighted in blue in the revised manuscript.
Changes: The changes can be seen in the revised version [Section # 4.8].
Reviewer 3 Report
Comments and Suggestions for AuthorsThis paper aims to develop and apply a dynamic optimization model to balance trade-offs between water, energy, and carbon for industrial decarbonization. This paper will address the limitations of static optimization using Deep Reinforcement Learning (DRL) in a Markov Decision Process (MDP) context. Additionally, this paper introduces a metric for diagnosis, “Water-Carbon Mitigation Penalty Index,” to evaluate the water footprint costs associated with the carbon mitigation process. Ultimately, this paper proposes to show how AI-based scheduling can assist the industry, such as the cement sector, to achieve net-zero targets and avoid excessive secondary environmental burdens.
The key strength of this paper lies in its clear identification of a real-world issue; decarbonization strategies frequently introduce new resource stresses, most notably in the realm of water and energy. The methodological innovation introduced within this submission lies within the combined use of reinforcement learning techniques and coupled constraints, which represents a new approach for the field. The incorporation of a diagnostic index into a real-world context, namely the UK industrial cluster, enhances the practical contribution of this work. Lastly, this submission supports its conclusions with concrete evidence of reductions in nexus costs and carbon emissions.
The reviewer offers the following suggestions for enhancing the manuscript:
- The abstract can be improved. The objectives, methods, and results are intermixed, thus creating a loss of clarity.
- The quantitative results are significant; however, without a context to create an understanding of their implications. For example, in lines 17-18, the statement indicates that "Simulation results indicate that distributed carbon-mitigation strategies will have adverse impacts upon local hydrology (2.15-5.17% reduction).” The reviewer could not find or confirm the 2.15-5.17% reduction in local hydrology as stated in the main text. Furthermore, the author should clearly discuss and explain what it means to have a reduction of local hydrology equal to 2.15-5.17%.
- More and more discussions are needed on the recognition of the limitations of this work (e.g., computational demands, availability of data, and ability to scale). What are the main obstacles, and how can they be overcome?
- The work focuses on the cement industry within the UK. While this area of focus is acceptable, there is little discussion of how findings can be generalized to be acceptable for a global forum like the MDPI Journal of Water.
Overall, this paper is an informative and technically sound contribution. However, it would greatly benefit from a minor revision based on the comments above.
Author Response
Reviewer 3, Round 1
Reviewer 3, Comment 1: The reviewer offers the following suggestions for enhancing the manuscript:
1. The abstract can be improved. The objectives, methods, and results are intermixed, thus creating a loss of clarity.
Response: Thank you for this helpful suggestion. We revised the abstract to improve its structure and clarity. In the revised version, the abstract now states the study objective, then summarizes the methodological framework, and finally presents the main quantitative findings and their implications in a clearer sequence.
2. The quantitative results are significant; however, without a context to create an understanding of their implications. For example, in lines 17-18, the statement indicates that Simulation results indicate that distributed carbon-mitigation strategies will have adverse impacts upon local hydrology (2.15-5.17% reduction).” The reviewer could not find or confirm the 2.15-5.17% reduction in local hydrology as stated in the main text. Furthermore, the author should clearly discuss and explain what it means to have a reduction of local hydrology equal to 2.15-5.17%.
Response: We revised the abstract and the results discussion to state the result more precisely as a 2.15-5.17% increase in local hydrological stress relative to the baseline operating condition under unmanaged carbon-mitigation pathways. We also added explanatory text in the results section clarifying that this percentage represents additional pressure on the local water system caused by greater freshwater-abstraction demand during decarbonization, especially under stressed conditions such as drought. These clarifications are highlighted in purple in the manuscript.
3. More and more discussions are needed on the recognition of the limitations of this work (e.g., computational demands, availability of data, and ability to scale). What are the main obstacles, and how can they be overcome?
Response: Thank you for this valuable comment. We expanded the “Limitations” subsection to discuss three additional challenges: computational demands of DRL training, dependence on the availability and quality of plant and regulatory data, and scalability of the framework across plants, regions, and wider CCUS chains. We also added brief discussion of how these obstacles may be addressed, including offline training, surrogate-model acceleration, staged calibration, data assimilation, digital monitoring, and hierarchical or transfer-learning strategies.
4. The work focuses on the cement industry within the UK. While this area of focus is acceptable, there is little discussion of how findings can be generalized to be acceptable for a global forum like the MDPI Journal of Water.
Response: To address this comment, we added a discussion clarifying that, although the numerical case study is calibrated to the UK cement sector, the framework itself is structurally transferable to other industrial regions. The revised text now explains that the state variables, constraints, and reward design can be recalibrated using local grid, water-policy, and industrial-process data, so the main generalizable contribution is the nexus- aware control methodology rather than the direct transfer of UK-specific numerical values. This clarification is highlighted in purple in the revised manuscript.
Changes: The changes can be seen in the revised version [These additions are highlighted in purple in the revised manuscript [Abstract: Section 4.7: Section # 4.9].
Reviewer 3, Comment 2: Overall, this paper is an informative and technically sound contribution. However, it would greatly benefit from a minor revision based on the comments above.
Response: Thanks for your valuable comments. The authors have thoroughly reviewed and revised the paper accordingly.
Round 2
Reviewer 1 Report
Comments and Suggestions for AuthorsMY COMMENTS HAVE BEEB ADDRESSED
Reviewer 2 Report
Comments and Suggestions for Authorsall comments has been addressed
