Submit to this Journal Review for this Journal Propose a Special Issue

Article Menu

Share Help Cite Discuss in SciProfiles

Open AccessArticle

Peer-Review Record

Generation of Benchmark Problems for Optimal Design of Water Distribution Systems

Water 2019, 11(8), 1637; https://doi.org/10.3390/w11081637

by Ho Min Lee¹, Donghwi Jung²

, Ali Sadollah³

, Do Guen Yoo⁴ and Joong Hoon Kim^5,*

Reviewer 1:

Joseph Kasprzyk

Reviewer 2:

Ching-Fu Chang

Water 2019, 11(8), 1637; https://doi.org/10.3390/w11081637

Submission received: 5 July 2019 / Revised: 6 August 2019 / Accepted: 8 August 2019 / Published: 8 August 2019

(This article belongs to the Section Urban Water Management)

Round 1

Reviewer 1 Report

Summary:

The authors present a very interesting idea -- that existing water distribution system benchmark problems are fundamentally limited because they assume a fixed network. The paper is well written, explaining how new benchmark problems can be created and used to test metaheuristics.

However, I feel as though there are some limitations that should be addressed. Overall, there is a fairly long history of multi-objective optimization within water distribution systems, going back at least 14 years at this point (e.g., Kapelan et al 2005). Algorithm parameters are ignored, and the choice of metaheuristics within the test suite is also not justified. In spite of these limitations, the authors have put forth a compelling premise.

Therefore, although the methods are fairly outdated, the paper could be valuable, especially if the authors are willing to share the code such that this study can truly be a contribution -- a new approach for creating benchmarks in water distribution systems.

General Comments:

The organization of the paper is somewhat non-standard. The authors provide a simple example first, then transition to a more complicated example, which doesn't leave space for a traditional "methods" section. They may want to consider revising the paper to have a more standard organization.

Some of the methodological choices are not well-justified. The results focus on the percentage of feasible solutions found, or how many infeasible solutions are in the set, rather than trying to approximate the true optimal. The idea that one cannot prove mathematical optimality in metaheuristic optimization is not a major hindrance, given the increasing use of these approaches in practice. In reality, managers are wanting to find the closest approximation to optimal solutions, which can save money and improve engineering performance. I'd like to see the methodology tuned more toward this idea.

Ideally, I would like to see the code from this paper shared, since the authors advertise that they are presenting new benchmark problems. Otherwise, some of the language in the paper should be changed to reflect the idea that this is just a demonstrative study, and more details should be provided such that researchers could replicate this work without code.

Specific Comments:

The current abstract is well-written, and does a nice job introducing the problem in general. However, it could benefit from more specifics. My suggestion would be to eliminate lines 13-17 and replace it with a definitive statement of what this paper contributes: "This paper contributes a set of N new benchmark problems for metaheuristic optimization of water distribution systems" and then use the extra space to provide a few more details.

The literature review in lines 41-46 could be considered redundant considering the discussion in lines 77-82; the two reviews should be resolved. The authors would also be well-served to add the so-called Anytown Network problem to their review (Walski et al 1987). It appears as though this model is missing from the review, which is a major omission.

Lines 68-70: The authors may want to direct the authors to Fu et al (2013) that discussed some of those design considerations for Water Distribution Systems.

Lines 87-88: The idea of "characteristic modifications" is unclear. Can this be re-phrased?

Figure 1: My suggestion is to provide more labels, if possible, to this figure to aid understanding.

Lines 104-105: It is somewhat unconventional to show all of these results for a simplified problem, and later move into the paper's true "results." Is it necessary to show this example? I would prefer more of a Methodology section that sets up how the optimizers were used, and other study details.

Line 115-116: Are the other data available in the original reference? It seems to be a bit of a logical leap that the authors are already providing answers to the optimization problem. Since the goal is to provide benchmarks, it would be useful to discuss any limitations of solving these problems. Are they all easy to solve? See also my comment above regarding lines 104-105.

Line 136: Is the Hazen-Williams roughness calculation required? Perhaps the authors could comment about other formulations that could be replaced here.

Line 149-150: Using the known worst cost as a benchmark is an interesting choice. If other authors have done this, please cite those studies. Intuition suggests that searching for the "worst" value in a feasible set may be similarly challenging to finding the global optimum in a feasible set, especially given the larger problems where simply generating a feasible solution is quite challenging.

Line 158-160: How was this set of optimizers chosen? Also, I'm surprised there wasn't a discussion of the optimizer's parameters; it has been discussed for quite a while that the performance of these algorithms can greatly depend on both how the algorithm is designed and the value of its parameters (e.g., Goldberg 1987).

Line 164: Please clarify the metric here. Are you trying to determine the percentage of the feasible set that is found in each run? I am not familiar with this metric, since the ultimate goal of optimization is to find the optimal solution. Again, if this metric is presented in prior literature it should be cited; it would ideally also be explained in a Methodology section.

Figures 3-6: My suggestion would be to combine these figures such that they look better in print, since if they are rendered this large it would make it difficult for the reader to compare across panels. It would also be worthwhile to have more explanation on how to interpret the figures -- I'm assuming this demonstrates which of the problem dimensions causes the most trouble for each algorithm.

Line 192-193: Again, I do not think that having infeasible solutions in the set at late numbers of generations is that important, if there exist good solutions in the population. Have the authors considered creating a "best known approximation" of the optimal value? I feel as though that would be more meaningful -- they could look at all optimization results at a particular number of function evaluations, and find the best value. This is a close enough approximation of the optimal point, and mimics how these tools would be used in practice (e.g., Basdekas 2013).

line 207-208: "the choice of method should be dependent on the optimization problem": This is a great idea, but how is it to be done in practice? I'd like to see more specific guidance of how this study can inform future practice.

Given that this work advertises that it is creating new benchmark problems, it would be extremely helpful for the authors to share the code for these problems (and even the algorithms) for the public. It would greatly increase the contribution of this paper.

References:

Basdekas, L., 2014. Is Multiobjective Optimization Ready for Water Resources Practitioners? Utility’s Drought Policy Investigation. Journal of Water Resources Planning and Management 140, 275–276. https://doi.org/10.1061/(ASCE)WR.1943-5452.0000415

Fu, G., Kapelan, Z., Kasprzyk, J., Reed, P., 2013. Optimal Design of Water Distribution Systems Using Many-Objective Visual Analytics. Journal of Water Resources Planning and Management 139, 624–633. https://doi.org/10.1061/(ASCE)WR.1943-5452.0000311

Kapelan, Z.S., Savic, D.A., Walters, G.A., 2005. Multiobjective design of water distribution systems under uncertainty. Water Resources Research 41. https://doi.org/10.1029/2004WR003787

Goldberg, D.E., 1998. The Race, the Hurdle, and the Sweet Spot: Lessons from Genetic Algorithms for the Automation of Design Innovation and Creativity (No. 98007). Illinois Genetic Algorithms Laboratory.

Walski, T.M., Brill, E.D., Jr., Gessler, J., Ian C. Goulter, Roland M. Jeppsen, Kevin Lansey, Han‐Lin Lee, Jon C. Liebman, Larry Mays, David R. Morgan, Lindell Ormsbee, 1987. Battle of the Network Models: Epilogue. Journal of Water Resources Planning and Management 113, 191–203. https://doi.org/10.1061/(ASCE)0733-9496(1987)113:2(191)

Author Response

Reviewers' comments:

Reviewer #1: The authors present a very interesting idea -- that existing water distribution system benchmark problems are fundamentally limited because they assume a fixed network. The paper is well written, explaining how new benchmark problems can be created and used to test metaheuristics. However, I feel as though there are some limitations that should be addressed. Overall, there is a fairly long history of multi-objective optimization within water distribution systems, going back at least 14 years at this point (e.g., Kapelan et al 2005). Algorithm parameters are ignored, and the choice of metaheuristics within the test suite is also not justified. In spite of these limitations, the authors have put forth a compelling premise. Therefore, although the methods are fairly outdated, the paper could be valuable, especially if the authors are willing to share the code such that this study can truly be a contribution -- a new approach for creating benchmarks in water distribution systems.

Response: We appreciate the reviewer’s helpful comments. Detailed descriptions of how these comments were addressed are provided below. Please note that additions/modification to the original paper are highlighted in red in the revised paper.

Thank you for your comment. In this study, a total of five factors (the number of pipes (n), the number of candidate pipe diameter options (m), the pressure constraint (p), roughness coefficient (c), and nodal demand multiplier (d)) were selected to control the size and complexity of the water distribution network design problems and the performance and reliability of metaheuristic algorithms were evaluated. As the reviewer pointed out, we considered only five representative metaheuristic algorithms and single objective function (cost minimization). The purpose of this paper, however, is to propose a new methodology to generate various problems of water distribution network design. In future work, we will consider the application of various metaheuristic algorithms and multi-objective optimal design problems, and this is reflected in the conclusion of the manuscript.

On the other hand, we would like to share with other researchers on the water distribution network design benchmarks we presented in our study, and we think that is meaningful. Therefore, if the manuscript is published, we will share it with researchers who need our data by possible way (such as sharing via email, web pages, etc.).

Revised Part:

In addition, cost minimization was selected as an objective function in this study, and the node pressure requirement was used as a hydraulic constraint. However, real water distribution system designs have several objectives (such as system reliability and greenhouse gas emissions) and constraints (such as water flow velocity limitation and water quality requirement). Therefore, in future studies, various combinations of objectives and constraints with other problem characteristic modification factors will be considered including multi-objective optimization problems; then they will be applied to benchmark problem generation for performance and reliability comparisons among optimization techniques. In addition, five algorithms were compared by proposed water distribution system design benchmarks in this study, however in the future studies, other optimization algorithms and variants will be considered.

General Comments:

Response: We revised our manuscript by changing the existing engineering benchmark problem generation section into a methodology section based on the opinion of the reviewer. On the other hand, in the case of the modified two-loop example networks included in the methodology section, the metaheuristic optimization techniques were not applied to the water distribution network design benchmarks. We checked all possible designs for simple water distribution network design benchmarks and wanted to quantitatively show that the size and complexity of the problems change as the five factors proposes in this study change.

Response: In the two-loop example network, which was presented in the methodology section, its size is very small, so we can check all the candidate designs for the networks which was generated in this study with various characteristics. Therefore, the global optimal design can be known for these small problems and it can be used as a quantitative index of optimal design. However, for example, in the case of the Hanoi network, some of the existing studies have predicted the global optimal value for the original Hanoi network design problem, but it is difficult to obtain the global optimal design if the characteristics of the benchmark problem change, and we cannot be sure that the derived design is the global optimal design. Therefore, in this study, we selected a quantitative index (improvement ratio) of the evaluation how the cost of the derived design can be reduced based on the worst feasible solution (setting all pipes as the maximum pipe diameter). Such a quantitative index is not a criterion for judging that the derived design is a global optimal design, but it means that it represents a relatively near optimal value, so it can be applied to the evaluation of the design in terms of engineer.

Revised Part: A complete enumeration for each generated design problem was performed and the results of this are shown in Table 3. The size of the two-loop example network was very small, so all the candidate designs for the modified two-loop networks which was generated in this study with various characteristics can be checked. The global optimum cost increased linearly, and the number of candidate designs and feasible designs increased exponentially as the number of pipes increased. The ratio of feasible designs decreased as the number of pipes increased. The global optimum cost decreased, and the number of candidate designs and number of feasible solutions increased as the number of candidate pipe diameter options increased. The feasible design ratio varied as the number of candidate pipe diameter options increased. Therefore, the parameters n and m controlled the size of the benchmark problems.

The known globally optimal solution was used as a reference value for the measurement of performance among the metaheuristic algorithms. However, globally optimal solutions of engineering optimization problems, including water distribution system designs, are generally unknown. In addition, the globally optimal solution changes with different factors, such as n and m. Therefore, the global lowest cost cannot be used as a reference value for measuring the performance of the applied optimization algorithms. Thus, the known worst solution in the feasible solution area is used as the reference. In the case of the water distribution system cost optimization problem, the global optimal design among the candidate designs cannot be founded without the solution search is performed, while we intuitively know that the worst design is to set all the diameters to the maximum. This is because as the diameter increases, the cost increases (Equation 1), but the head loss in the pipe decreases and the constraint can be satisfied (Equation 2).

Response: As mentioned earlier, we want to share the data of our research, and if the paper is published, we will share information with researchers who need our data by possible way.

Specific Comments:

Response: Thank you for your comment. The abstract was revised to reflect the comment by reviewer.

Revised Part: Engineering benchmark problems with specific characteristics have been used to compare the performance and reliability of metaheuristic algorithms, and water distribution system design benchmarks are also widely used. However, only a few benchmark design problems have been considered in the research community. Due to the limited set of previous benchmarks, it is challenging to identify the algorithm with the best performance and highest reliability among a group of algorithms. Therefore, in this study, a new water distribution system design benchmark problem generation method is proposed considering problem size and complexity modifications of a reference benchmark. The water distribution system design benchmark problems are used to performance and reliability comparison among metaheuristic algorithms. The optimal design results are able to quantify the performance and reliability of the compared algorithms, and each metaheuristic algorithm has its own strengths and weaknesses. Finally, using the method proposed in this study, guidelines are derived for selecting an appropriate metaheuristic algorithm for water distribution system design.

Response: Lines 41-46 illustrate examples of various metaheuristic algorithms applied to water distribution network design. However, in lines 77-82, the previous studies which proposed new water distribution network design benchmark are presented, and therefore, it is necessary to distinguish between these two. We added references such as Anytown Network problem as the comment by the reviewer.

Revised Part: The water distribution system design problem is a widely used benchmark problem in the field of engineering. Several metaheuristic algorithms have been applied to the optimal design of water distribution systems with various characteristics; Simpson et al. [17] applied Genetic Algorithms (GAs), Cunha and Sousa [18] applied Simulated Annealing, Maier et al. [19] applied Ant Colony Optimization (ACO), Montalvo et al. [20] applied Particle Swarm Optimization (PSO), and Geem [21] applied the Harmony Search Algorithm (HSA). More recently, Sadollah et al. [22, 23] employed the Water Cycle Algorithm (WCA) and the Mine Blast Algorithm (MBA) for the optimal design of water distribution systems.

Optimally designing a water distribution system is a widely used benchmark problem for measuring the performance of optimization methods. Several benchmark water distribution systems have been introduced in previous studies. The two-loop network problem is one of the simplest problems and was suggested by Alperovits and Shamir [27]. The Anytown network was introduced by Walski et al. [28]. The Hanoi network and the GoYang network problems, which are medium scale problems, were introduced by Fujiwara and Khang [29] and Kim et al. [30], respectively. Reca and Martinez [31] introduced the Balerma network, which is a large irrigation network. More recently, Bragalli et al. [32] conducted a study using three benchmark networks: Fossolo, Pescara, and Modena.

Lines 68-70: The authors may want to direct the authors to Fu et al (2013) that discussed some of those design considerations for Water Distribution Systems.

Response: The reference was added according to the comment by the reviewer.

Revised Part: The objective function for determining the lowest cost design of a water distribution system with nodal pressure constraints is calculated from the diameter and length of the pipes as follows [25]:

Equation (1)

where Cc (Di) is the construction cost according to pipe diameter per unit length, Li is the pipe length, Di is the pipe diameter, Pj is the penalty function for ensuring that the pressure constraints are satisfied, N is the number of pipes, and M is the number of nodes.

If a design solution does not meet the nodal pressure requirements, a penalty function is added to the objective function as follows [25]:

Equation (2)

where hj is the nodal pressure at node j, hmin is the minimum pressure requirement at node j, and α and β are penalty function constants. Note that other hydraulic or water quality requirements, such as allowable flow velocity, water age, and residual chlorine concentration, can also be considered in water distribution system design [26].

Lines 87-88: The idea of "characteristic modifications" is unclear. Can this be re-phrased?

Response: The explanation was revised more clearly according to the comment.

Revised Part: However, in a previous water distribution system design benchmark in the literatures, the network layout, objective function, candidate pipe diameter option, and hydraulic constraints are all fixed. Therefore, previous water distribution system design benchmarks in the literatures have limited ability to measure the performance and reliability of optimization methods, because each problem has its own unique characteristics. Thus, in this study, water distribution system design problems are generated using characteristic modifications, such as the number of pipes, the number of candidate pipe diameter options, the pressure constraint, roughness coefficient and nodal demand multiplier, of previous water distribution system design benchmarks, and the generated problems are applied to measuring the performance of metaheuristic algorithms. Such benchmark problems have several advantages. Firstly, the number of alternative designs, which is represented by problem size, can be easily altered by a user. In addition, the complexity and difficulty of the optimization problems can be changed.

The benchmark water distribution system problems in this study were generated by modifying five individual characteristics of a reference water distribution system design benchmark. The factors that were modified are shown in Table 1. The number of pipes (n) and the number of candidate pipe diameter options (m) are used as problem size modification factors. The pressure constraint (p), roughness coefficient (c), and nodal demand multiplier (d) are potential problem complexity modification factors.

Figure 1: My suggestion is to provide more labels, if possible, to this figure to aid understanding.

Response: The labels of figures 1 and 2 were added to reflect the comment.

Response: As mentioned earlier, the two-loop network example was not actually comparing the results by applying the metaheuristic algorithms, but to show how we generated the benchmark problems and how the size and complexity of the problem changed. In addition, the purpose of this study is to change the characteristics of the water distribution network design problems. Therefore, we think that the detailed explanation about the applied metaheuristic algorithms does not meet the purpose of this study, and we list the references instead. Please understand.

Revised Part: If k values are considered for each factor, then 5×k cases of the benchmark water distribution system design problems will be generated—each one changing a single factor for each problem—with a reference design benchmark as the default problem. In this study, a simple water distribution system, which is a modified form of the two-loop network design problem introduced by Alperovits and Shamir [27] is used as an example to estimate the relative influence of each factor. This system consists of a single water source (junction 1), six demand nodes (junctions 2-7), eight pipes, and two loops. In our example, each factor takes three values (k=3). The layout of the example network and the values assigned to each factor are shown in Figure 1 and Table 2, respectively; the default factors are in a bold font. 15 (5×3) benchmark design problems were generated.

Response: As mentioned before, we evaluated all the candidate designs for the modified two-loop network design problems that generated in our study, including the original two-loop network. The results are shown in Table 3. In the case of modified two-loop network design benchmarks, the size of the problem is small, and we can check all possible designs, and the global optimal design also can be derived.

Meanwhile, the global optimum costs increased and the numbers of feasible solutions and feasible design ratios decreased as the pressure constraint and nodal demand multiplier increased and also as the roughness coefficient decreased. The total number of candidate designs was not changed by variations of the factors p, c, and d. Therefore, these factors are considered to be problem complexity modification factors.

Line 136: Is the Hazen-Williams roughness calculation required? Perhaps the authors could comment about other formulations that could be replaced here.

Response: Rather than calculating the HW roughness itself, we used it to change the characteristics of the problem by converting the HW roughness value used in the original design problem (please see table 4).

Response: An attempt to evaluate the performance of algorithms based on worst value in the feasible solutions is our own idea. If the target problem is an optimization for a general mathematical equation, finding the global optimal solution is very similar to finding the global worst solution. However, in the case of the water distribution network cost optimization problem, the global optimal design among the candidate designs cannot be founded without the solution search is performed, while we intuitively know that the worst design is to set all the diameters to the maximum. This is because as the diameter increases, the cost increases (see equation 1), but the head loss in the pipe decreases and the constraint can be easily met (see equation 2). We explained this point more clearly in the revised manuscript.

Revised Part: The known globally optimal solution was used as a reference value for the measurement of performance among the metaheuristic algorithms. However, globally optimal solutions of engineering optimization problems, including water distribution system designs, are generally unknown. In addition, the globally optimal solution changes with different factors, such as n and m. Therefore, the global lowest cost cannot be used as a reference value for measuring the performance of the applied optimization algorithms. Thus, the known worst solution in the feasible solution area is used as the reference. In the case of the water distribution system cost optimization problem, the global optimal design among the candidate designs cannot be founded without the solution search is performed, while we intuitively know that the worst design is to set all the diameters to the maximum. This is because as the diameter increases, the cost increases (Equation 1), but the head loss in the pipe decreases and the constraint can be satisfied (Equation 2).

Response: We know that the choice of optimization algorithm and the setting of parameters are important. In this study, we first consider the representative metaheuristic algorithms which have been applied to the problem of the optimal design of the water distribution networks To determine suitable parameters of metaheuristic algorithms, we conducted a sensitivity analysis for water distribution network design problem with default characteristics (Table 4), and then the parameters show best results for default problem are applied to the other problems. We clarified this point in the manuscript regarding the comment. In addition, we will apply more diverse algorithms in the future studies.

Revised Part: In this study, five algorithms were compared: Random Search (RS), Genetic Algorithms (GAs) [33], Simulated Annealing (SA) [34], Harmony Search Algorithm (HSA) [35]), and Water Cycle Algorithm (WCA) [36]. Each metaheuristic algorithm was tested with 20 individual runs for each of the 20 cases shown in Table 4. The parameters of applied metaheuristic algorithms were determined through a solution search of the water distribution system design problem. If the characteristic factors are the default parameter set (Table 4), then the optimization parameters of each metaheuristic algorithm were varied, and the combination of parameters that derive the best result is determined. The determined parameter values were applied to 20 cases of benchmark problems. The maximum number of function evaluations was used as the stopping criterion and was set to 50,000, and the results were compared at 10,000th, 30,000th and final function evaluation.

Line 164: Please clarify the metric here. Are you trying to determine the percentage of the feasible set that is found in each run? I am not familiar with this metric, since the ultimate goal of optimization is to find the optimal solution. Again, if this metric is presented in prior literature it should be cited; it would ideally also be explained in a Methodology section.

Response: First, the design problem of the water distribution network is an engineering optimization problem. Since the constraint condition must be considered, it is not reasonable to evaluate each design with only the objective function. We therefore concluded that it is not fair to quantitatively evaluate the results including infeasible designs, and used the success rate as an alternative indicator.

Revised Part: Here, the ratio of an algorithm’s optimal solution cost to the known worst solution cost is defined as the improvement ratio. Note that the known worst solution is the design with the largest diameter for all pipes that still meets the hydraulic requirements. Water distribution system design problems are constrained problems, and the success rates are considered as a basic performance indicator. In this study, four improvement ratio statistics were used to compare the five algorithms: the mean and standard deviation of the average improvement ratios, and the mean and standard deviation of the best improvement ratios. In this study, the best solution included in the final population was selected (in 20 individual runs) for each case and applied to the evaluation index. Therefore, a run that was considered to be a success if there are one or more feasible solutions were included in the last function evaluation, and if there was no feasible solution in the last function evaluation, the run was considered as fail.

The SA found 19.375% of the feasible solutions on average at the final function evaluation; it had the second worst results after the RS. The SA was weak—especially with variations in demand and number of pipes. The GAs, HSA, and WCA all found feasible solutions in all individual runs for each case in the final function evaluation. However, the GAs had slow conversion results compared to the HSA and WCA at 10,000th, 30,000th and final function evaluation—especially with variations in demand and number of pipes.

Response: In accordance with the comment by the reviewer, the figures were improved to make them more readable, and added explanations. However, since each figure shows each performance index (the mean and standard deviation of the average improvement ratios and the mean and standard deviation of the best improvement ratios), combining the figures may make it difficult for the reader to understand. Also, we want to compare the change in the solutions of the optimization iterations, and therefore we selected this format. Please understand.

Response: Like the reviewer's opinion, we also think that if there are good solutions in the final population, it is not a problem to include some infeasible solutions. However, in our study, the best solution included in the final population was selected (in 20 independent runs) for each case and applied to the evaluation index. Therefore, a run that is considered to be a success if there are one or more feasible solutions are included in the last population, and if there is no feasible solution in the last population, the run is considered as fail (It does not determine success based on all solutions in the final population).

The GAs showed better results than the others with respect to the varying number of pipes in terms of the mean and standard deviation at the final function evaluation. On the other hand, in terms of the varying number of candidate pipe diameters, the performance of the GAs was insufficient. In addition, in terms of the convergence rate, the GAs showed low efficiency in early function evaluations when compared with the HSA and the WCA. The GAs found infeasible solutions at the 10,000th and 30,000th function evaluations. Moreover, feasible solutions found by the GAs had low improvement ratio values.

Response: In this study, we can quantitatively evaluate which algorithms are applicable to problems with certain characteristics, and can be used as a guide for selecting algorithms according to the characteristics of the problems. We explained this point more clearly in the revised manuscript.

Revised Part: Furthermore, as the size and complexity of the problems increased, the all algorithms’ performance and reliability weakened consistently. Therefore, the choice of method should be dependent on the characteristics of optimization problem. For example, the SA can be applied to simple optimization problems with several individual runs, and the GAs can be used to find optimal solutions for large problems with repeated function evaluations. The HSA is suitable for the problems include high complexities, and the WCA can be applied to the problems which need long computation time. In addition, the existing optimization algorithms require improvement by enhancing the optimization process and introducing additional engineering approaches considering characteristics of given problem.

Response: As mentioned above, we suggest a new methodology, so it is meaningful to share the information of our research. Therefore, if the article is published, we will share it with researchers who need our data by possible way.

Author Response File: Author Response.pdf

Reviewer 2 Report

This study points to an interesting topic, but the content does no seem to advance the field of research nor does it show enough novelty, at least not in the current stage.

I suggest a major revision with the underlying goals of elaboration on key innovation/results and paraphrasing key paragraphs for clarity.

Detailed comments are as follows:

Line 21-22: I think it means the problems are applied to algorithms to compare the performance of algorithms. Paraphrasing is suggested for clarity.

Line 34-39 (However,.....): It would be great to provide several key examples where a particular metaheuristic algorithm that did great in mathematical benchmark problems actually failed in some engineering problems. Alternatively, elaboration on the part "engineering optimization problems have their own unique characteristics" could support your point.

Line 41: Are the algorithms applied to an existing optimal design or are they applied to the searching of such design?

Line 47-50: Elaboration is suggested. What are good examples of such characteristics, how they cannot be freely assigned, and why that affects the quantitative evaluation of the effectiveness of algorithms.

Line 50 (engineering design benchmark problems): Based on the previous paragraphs, water distribution design is a subset of engineering design. If in this study only the water distribution system design benchmarks are generated, I suggest narrowing down to that.

Introduction in general: I think a brief overview of how metaheuristic algorithms works to find the optimal solution and their known advantages/disadvantages would be a great addition to the introduction. It would be even better that this overview can be given within the context of engineering problem design, while highlighting some key differences between engineering problems and mathematical problems.

The fact that some problems appear in previous studies does not necessarily make them "traditional". A clear definition of "traditional water distribution system design problems" should be given, as this study seems to improve on such problems.

Line 54: missing verb?

Line 62: It seems like a fixed cost function is assumed here without elaboration. Installation of pipes of the same length and diameter could have different costs; for example, the excavation cost may depend on the soil/rock properties, as well as timing and location. If cost is assumed to be solely depending on the diameter, the assumption should be justified, or at the very least, explicitly noted.

Line 71-74: Rephrasing is suggested for better clarity; maybe add a few examples or conceptual diagrams.

Line 83-85: I suggest laying out such background information in the introduction. Also, here the term "standard water distribution system design benchmark" is used together with "traditional water distribution system design benchmarks". What's the difference between "standard benchmark" and "traditional benchmark"? Clear definitions should be given.

Line 101-103: The interaction among multiple factors cannot be tested with this design. For example, if the pressure requirement is particularly stringent, then the effect of changes in roughness coefficient would be more significant. If the pressure requirement is loose, then head loss would not be a problem and thus the importance of roughness would be reduced. This should be explicitly noted and justified, or argued if the authors believe otherwise.

Table 2: Why three values and how were they selected?

Line 115 (a complete enumeration survey): I am having difficulty in understanding this. How was the survey conducted? What was the cost function adopted in the survey? It seems like this and the next paragraph are primarily explaining the control of each parameter, without the details of how this was done. Elaboration is suggested.

Connection between Section 2 and 3: It is confusing that after seeing a case study in the Section 2 we are seeing another case study here in Section 3. Now it seems like section 2 is a methodology section where the case study is simply a demonstration of the approach applied in this study, while section 3 is the actual case study of interest. If that is the case, it should be made clear either in the titles or the first few sentences of both sections.

Line 144-150: It seems like this paragraph aims to justify the use of the known worst solution instead of the known optimal solution as the reference. If that is the case, rephrasing is suggested for clarity.

Line 155: Over what are the statistics calculated? What's the difference between the average improvement ratios and the improvement ratios (i.e., averaging over what), and over what are the mean and standard deviation taken? Different algorithm runs? Different plausible designs? Because the results are hugely based on discussions over these statistics, it is difficult for readers unfamiliar with your research design to follow the rest of the paper without clear explanations.

Line 158-160: Why five, and why those five? If those five algorithms are to be compared, a brief overview of the key features, advantages/disadvantages, and/or iconic applications should be given.

Line 161-162: Why are there three "maximum number"? Are they applied in different scenarios? The selection of the numbers should also be justified.

Line 168: What does "slow conversion results" mean? Conversion from what to what?

Line 172-173 (average improvement ratios): Averaged over what?

Line 173-175: The explanation of the statistics should be expanded and should be given when they were first mentioned in the manuscript.

Figures 3 through 6: keep each figure on the same page; in addition to comparing different maximum FEs with each algorithm, it could be interesting to compare all algorithms given one number of FEs; which may provide a clearer picture of the differences among algorithms.

Line 183-187: To support your point, either elaboration or conceptual diagrams simultaneously showing the statistics and the success rates should be provided. Otherwise the readers are forced to go back and forth over several pages for Table 5 and Figures 3 through 6.

Line 194: What is fitness value?

Line 195-199: Rephrasing is suggested for clarity.

Line 202: Sensitive to what?

Line 206-207: Rephrasing is suggested for clarity.

Line 208-210: The choice of method depends on the context of the problem. This is quite widely known. The next sentence is also quite general, as improvements in the optimization process is always welcomed. Because these sentences seem like summary remarks for the discussion, narrowing the scope down to something specifically found in this study would be better.

Line 215-216: Again, what does "traditional" mean? Fixed characteristics?

Line 225-227:

This seems like the concluding remark of the results, as the next paragraph is focused on limitations and future research. If that is the case, it does not seem to have a direct linkage with the results. Elaboration is suggested, otherwise this concluding remark seems to be a general one that one could make without running all the optimizations. Furthermore, in the abstract it was mentioned that guidelines for selecting an appropriate metaheuristic algorithm will be given. After reading the abstract I was expecting explicit and specific guidelines that are directly based on the findings of this study, rather than general statements.

These comments may seem harsh, but in a nutshell it's largely due to lack of clarity, which makes it difficult to see the novelty in this research. I would suggest that the authors try to highlight and clarify the key innovation and contribution of this research to the field of interest.

Author Response

Reviewers' comments:

Reviewer #2: This study points to an interesting topic, but the content does no seem to advance the field of research nor does it show enough novelty, at least not in the current stage. I suggest a major revision with the underlying goals of elaboration on key innovation/results and paraphrasing key paragraphs for clarity.

Detailed comments are as follows:

Line 21-22: I think it means the problems are applied to algorithms to compare the performance of algorithms. Paraphrasing is suggested for clarity.

Response: Thank you for your comment. We explained this part more clearly in the revised manuscript

Revised Part: Therefore, in this study, a new water distribution system design benchmark problem generation method is proposed considering problem size and complexity modifications of a reference benchmark. The water distribution system design benchmark problems are used to performance and reliability comparison among metaheuristic algorithms. The optimal design results are able to quantify the performance and reliability of the compared algorithms, and each metaheuristic algorithm has its own strengths and weaknesses.

Response: We agreed with reviewer's opinion and revised the manuscript to reflect this.

Revised Part: However, engineering optimization problems have their own unique characteristics. Therefore, a metaheuristic algorithm with good performance and reliability in mathematical benchmark problems does not guarantee suitable results in real-world engineering problems. Consequently, the performance and reliability of metaheuristic algorithms for real problems should be verified by applying them to engineering problems with specific characteristics [16].

Line 41: Are the algorithms applied to an existing optimal design or are they applied to the searching of such design?

Response: Line 41 implies that the metaheuristic algorithms are applied to the optimal design of water distribution networks in the previous studies.

Response: In the case of the existing water distribution network optimal design problem, the layout and constraints of the problem are fixed, so that the size and complexity cannot be freely changed. Therefore, in this study, we propose a methodology to change the characteristics of the problem and generate the various design benchmarks. Through this, we quantitatively evaluated the performance of the metaheuristic algorithms on the problems with certain characteristics.

Revised Part: However, previous water distribution system design problems in the literatures have a disadvantage in that their characteristics cannot be freely assigned. In the previous benchmark problems, the layout and constraints of the problem were fixed, therefore the size and complexity could not be freely changed. Thus, it is difficult to quantitatively evaluate which metaheuristic algorithm is effective, because the results are dependent on the characteristics of a given design problem. Therefore, in this study, engineering design benchmark problems are generated by modifying existing water distribution system design benchmarks and applying them to measure the performance and reliability of metaheuristic algorithms.

Response: We revised the manuscript to reflect this comment.

Response: The purpose of this study is to change the characteristics of the water distribution network design problems and propose a new benchmark generation method. Therefore, we think that the detailed explanation about the applied metaheuristic algorithms does not meet the purpose of this study, and we list the references instead. Please understand.

Response: The traditional water distribution system design problem is a benchmark with fixed properties used in existing literature. According to the opinion of the reviewer, the ‘traditional’ was revised to ‘previous’.

Revised Part: Due to the limited set of previous benchmarks, it is challenging to identify the algorithm with the best performance and highest reliability among a group of algorithms.

However, previous water distribution system design problems in the literatures have a disadvantage in that their characteristics cannot be freely assigned. In the previous benchmark problems, the layout and constraints of the problem were fixed, therefore the size and complexity could not be freely changed.

However, in a previous water distribution system design benchmark in the literatures, the network layout, objective function, candidate pipe diameter option, and hydraulic constraints are all fixed. Therefore, previous water distribution system design benchmarks in the literatures have limited ability to measure the performance and reliability of optimization methods, because each problem has its own unique characteristics. Thus, in this study, water distribution system design problems are generated using characteristic modifications, such as the number of pipes, the number of candidate pipe diameter options, the pressure constraint, roughness coefficient and nodal demand multiplier, of previous water distribution system design benchmarks, and the generated problems are applied to measuring the performance of metaheuristic algorithms.

Line 54: missing verb?

Response: We revised the manuscript to reflect this comment.

Revised Part: Therefore, in this study, engineering design benchmark problems are generated by modifying existing water distribution system design benchmarks and applying them to measure the performance and reliability of metaheuristic algorithms.

Response: The cost function is affected by a variety of environmental conditions. However, in general, it has been assumed in previous studies that the cost of water distribution network design is calculated according to the diameter and pipe length. We added the references for the cost function used in this study.

Equation (1)

If a design solution does not meet the nodal pressure requirements, a penalty function is added to the objective function as follows [25]:

Equation (2)

Line 71-74: Rephrasing is suggested for better clarity; maybe add a few examples or conceptual diagrams.

Response: We revised the manuscript to reflect this comment.

Revised Part: Difficulties in designing an optimal water distribution system include the relation between the pipe diameter (decision variables of the problem) and the cost (problem’s objective function) being nonlinear, the energy equation (hydraulic constraint) for the head loss calculation includes nonlinear terms, and the pipe flow direction is not fixed for looped-type water distribution systems [25]. Therefore, the optimal design problems of water distribution systems are complex nonlinear constrained problem, and the mathematical approaches cannot be applied efficiently.

Response: We agree with this comment, there would be confusion, and it was supplemented throughout the paper.

Revised Part: Due to the limited set of previous benchmarks, it is challenging to identify the algorithm with the best performance and highest reliability among a group of algorithms.

Response: The purpose of this study is to generate a variety of design problems based on a change in the size and complexity of the reference problem. Therefore, in this study, please understand that when one problem characteristic changes, the other characteristics of the problem is fixed to the default value (please see table 2).

Table 2: Why three values and how were they selected?

Response: The application of the three values contained in table 2 for each parameter is an assumed value for confirming the increase and decrease in the size and complexity of the problem according to the change in the problem characteristics as in table 3.

Response: The cost values were calculated by equation (1), which was objective function in this study. In this section, all possible designs for each modified two-loop network design problem were evaluated and we expressed this as a complete enumeration survey. In the case of modified two-loop network design benchmarks, the size of the problem is small, and we can check all possible designs, and the global optimal design also can be derived (the optimization algorithm was not applied in this section). We changed the name to complete enumeration, thinking that there is room for confusion.

Response: As described above, section 2 exemplifies how the size and complexity of a problem change when each feature of a problem changes with respect to a two-loop network design problem that can be evaluated for all designs, in order to clarify the meaning of the proposed method. On the other hand, in section 3, the proposed method is applied to a more complex water distribution network, and the performance and reliability of metaheuristic algorithms are compared.

Response: In the case of the water distribution network cost optimization problem, the global optimal design among the candidate designs cannot be founded without the solution search is performed, while we intuitively know that the worst design is to set all the diameters to the maximum. This is because as the diameter increases, the cost increases (see equation 1), but the head loss in the pipe decreases and the constraint can be easily met (see equation 2). We explained this point more clearly in the revised manuscript.

Response: In this study, we run 20 times for each case to evaluate the average performance of the algorithm and calculate the mean and standard deviation of the average improvement ratios for the 20 solutions. In addition, for actual engineering problems, the best solution by optimization algorithm is also important, and therefore, the mean and standard deviation of the best improvement ratio among the improvement ratios of 20 solutions are considered. We explained this point more clearly in the revised manuscript.

Line 158-160: Why five, and why those five? If those five algorithms are to be compared, a brief overview of the key features, advantages/disadvantages, and/or iconic applications should be given.

Line 161-162: Why are there three "maximum number"? Are they applied in different scenarios? The selection of the numbers should also be justified.

Response: The maximum number is fixed to 50,000, and the results at 10,000th and 30,000th function evaluation are used to compare the intermediate performance quantitatively. To prevent confusion, we modified this section.

Revised Part: The maximum number of function evaluations was used as the stopping criterion and was set to 50,000, and the results were compared at 10,000th, 30,000th and final function evaluation.

Line 168: What does "slow conversion results" mean? Conversion from what to what?

Response: Comparing the results at 10,000th, 30,000th, and 50,000th function evaluation, the search speed of the GAs is slower than the other algorithms for the changes of the parameters n and d, and it is clearly modified in the revised manuscript.

Revised Part: The GAs showed better results than the others with respect to the varying number of pipes in terms of the mean and standard deviation at the final function evaluation. On the other hand, in terms of the varying number of candidate pipe diameters, the performance of the GAs was insufficient. In addition, in terms of the convergence rate, the GAs showed low efficiency in early function evaluations when compared with the HSA and the WCA.

Line 172-173 (average improvement ratios): Averaged over what?

Response: As mentioned earlier, the average improvement ratio of 20 solutions for each case is obtained, and the mean value is calculated for the four factor values applied

Line 173-175: The explanation of the statistics should be expanded and should be given when they were first mentioned in the manuscript.

Response: As previously mentioned, the explanation of the statistics is on lines 167-172.

Revised Part: Water distribution system design problems are constrained problems, and the success rates are considered as a basic performance indicator. In this study, four improvement ratio statistics were used to compare the five algorithms: the mean and standard deviation of the average improvement ratios, and the mean and standard deviation of the best improvement ratios. In this study, the best solution included in the final population was selected (in 20 individual runs) for each case and applied to the evaluation index.

Response: It is possible to present the success rate of table 5 as a figure or to present the result of figures 3-6 as a table, but it is considered that it will be less readable. We use the success rate as a backup indicator to support the performance quantification results in figure 3-6, therefore please understand that we used these table and figures.

Line 194: What is fitness value?

Response: We modified this explanation more clearly.

Line 195-199: Rephrasing is suggested for clarity.

Response: We revised the manuscript to reflect this comment.

Revised Part: The HSA showed the best results when varying the number of candidate pipe diameter options, while it had the worst results when the number of pipes was varied. The HSA also showed the best results in terms of the means of the average improvement ratio and best improvement ratio for three factor variations: pressure constraint, roughness coefficient, and nodal demand multiplier. On the other hand, the HSA performed poorly for these three characteristics in terms of standard deviations.

Line 202: Sensitive to what?

Response: Line 202 means that the WCA is less sensitive to factor variation. We modified this section more clearly.

Revised Part: The WCA produced an average performance in relation to the other metaheuristic algorithms. Additionally, the WCA found relatively good solutions in the early function evaluations compared to the other algorithms. Therefore, the WCA is less sensitive to variation of problem characteristics than the other applied algorithms and has good adaptability. However, in the latter half of the optimization process, the solutions found by the WCA did not improve further.

Line 206-207: Rephrasing is suggested for clarity.

Response: We revised the manuscript to reflect this comment.

Revised Part: Furthermore, as the size and complexity of the problems increased, the all algorithms’ performance and reliability weakened consistently.

Response: We made it clear that in accordance with the comment of the reviewer. The existing optimization algorithms require improvement by enhancing the optimization process considering the characteristics of given problem.

Revised Part: Therefore, the choice of method should be dependent on the characteristics of optimization problem. For example, the SA can be applied to simple optimization problems with several individual runs, and the GAs can be used to find optimal solutions for large problems with repeated function evaluations. The HSA is suitable for the problems include high complexities, and the WCA can be applied to the problems which need long computation time. In addition, the existing optimization algorithms require improvement by enhancing the optimization process and introducing additional engineering approaches considering characteristics of given problem.

Line 215-216: Again, what does "traditional" mean? Fixed characteristics?

Response: We revised the manuscript to reflect this comment.

Revised Part: Due to the limited set of previous benchmarks, it is challenging to identify the algorithm with the best performance and highest reliability among a group of algorithms.

Line 225-227: This seems like the concluding remark of the results, as the next paragraph is focused on limitations and future research. If that is the case, it does not seem to have a direct linkage with the results. Elaboration is suggested, otherwise this concluding remark seems to be a general one that one could make without running all the optimizations. Furthermore, in the abstract it was mentioned that guidelines for selecting an appropriate metaheuristic algorithm will be given. After reading the abstract I was expecting explicit and specific guidelines that are directly based on the findings of this study, rather than general statements.

Each applied metaheuristic algorithm had its own strengths and weaknesses, and the performance and reliability of all the algorithms weakened as the size and complexity of the problems increased. In addition, each algorithm showed its own convergence characteristic for the given design problem. This implies that finding optimal solutions for engineering problems using a metaheuristic algorithm requires an efficient approach that considers the characteristics of the given design problem.

Response: Thank you for your comments. We think that this study is meaningful because it is the first time to present a new methodology for generating various water distribution network design benchmarks. Based on the reviewers' point of view, the purpose and significance of this study was supplemented throughout the paper.

Author Response File: Author Response.pdf

Round 2

Reviewer 2 Report

The revision made by the authors as well as the authors' response letter are much appreciated, and improvements sure have been made.

Yet, some questions remain unanswered or half-answered, and some minor revisions are suggested as follows.

Check grammar and typos, e.g., lines 20-21, 127-128, 163-164, 177, 264-266 (not an exhaustive list). Please run through a thorough check before the next upload.

The paragraph starting at line 108: I appreciate the authors' explanation in the authors' response. However, the information in that response is not included in the texts here. Please rephrase the paragraph to include that response. If interaction effect is not a focus of the present study, and thus is neglected here, it should be explained. The paragraph starting at line 178: the authors' response is appreciated, yet it does not answer the question.
What are the criteria of being a "representative algorithm"? The selection of the algorithms should be made very clear as the entire result is based on comparing the performance of those algorithms.
The fact that an algorithm was used before, per se, does not make it representative. I'd suggest the authors elaborate on the features, advantages, and disadvantages of each algorithm that sets it apart from others and makes it a good candidate in the context of the present study.
Also, I'd suggest rephrasing the explanation of how parameters of the algorithms were determined. I get the idea from the authors' response that sensitivity analyses were conducted, but based on the texts here it is still a bit confusing. Line 193 (slow conversion results): Based on the authors' response I guess the authors meant "convergence" instead of "conversion"? Namely, the searching algorithm will converge towards the best result it can find as iteration goes on, not converting something into the best result. Line 224: The revision is appreciated, but still there's room for more clarity.
For example, what does "perform poorly in terms of standard deviation" mean? Does it mean large standard deviation?
Although it might seem intuitive to the authors, it is not immediately obvious for readers, as a smaller standard deviation is not always considered "better". Lines 232-237: Discussion of specific guidance should be given, and a concise version of that discussion should go into conclusion. These sentences provide a good starting point of specific guidance. Please elaborate on this in the discussion, and provide a concise version of the discussion in the conclusion.
These study-specific findings, if discussed further and deeper, form the foundation of discussion that sets the present study apart from general statements that could be made without running all the optimizations.

Author Response

Reviewers' comments:

Reviewer #2: The revision made by the authors as well as the authors' response letter are much appreciated, and improvements sure have been made. Yet, some questions remain unanswered or half-answered, and some minor revisions are suggested as follows.

Response: We appreciate the reviewer’s helpful comments. Detailed descriptions of how these comments were addressed are provided in below. Please kindly note that additions/modification to the original paper are highlighted in blue colors in the revised manuscript.

Check grammar and typos, e.g., lines 20-21, 127-128, 163-164, 177, 264-266 (not an exhaustive list). Please run through a thorough check before the next upload.

Response: Thank you for your precious comment. The entire manuscript was proofread and double checked in terms of English usage in the revised manuscript.

The paragraph starting at line 108: I appreciate the authors' explanation in the authors' response. However, the information in that response is not included in the texts here. Please rephrase the paragraph to include that response. If interaction effect is not a focus of the present study, and thus is neglected here, it should be explained.

Response: Thank you for your insight and important comment. We revised the manuscript to reflect this comment highlighted in blue colors.

Revised Part: If k values are considered for each factor, then 5×k cases of the benchmark water distribution system design problems will be generated—each one changing a single factor for each problem—with a reference design benchmark as the default problem. Note that, there are interactions among problem modification factors. For instance, if the pressure constraint is stringent, then the effect of changes in roughness coefficient would be more significant. However, the purpose of this study is to generate a variety of design benchmarks based on a change in the size and complexity of reference benchmark problem. Therefore, in this study, when one problem characteristic factor changes, the other factors are fixed to the default value.

The paragraph starting at line 178: the authors' response is appreciated, yet it does not answer the question. What are the criteria of being a "representative algorithm"? The selection of the algorithms should be made very clear as the entire result is based on comparing the performance of those algorithms. The fact that an algorithm was used before, per se, does not make it representative. I'd suggest the authors elaborate on the features, advantages, and disadvantages of each algorithm that sets it apart from others and makes it a good candidate in the context of the present study.

Response: Thank you for your useful comment. As the reviewer truly mentioned, we should say ‘previously applied algorithms in water distribution system design’ rather than ‘representative algorithms’. Therefore, we revised the manuscript to reflect the reviewer’s comment highlighted in blue colors in the revised manuscript, and also added explanations about applied algorithms.

Revised Part: In this study, five algorithms were compared: Random Search (RS), Genetic Algorithms (GAs) [33], Simulated Annealing (SA) [34], Harmony Search Algorithm (HSA) [35]), and Water Cycle Algorithm (WCA) [36], which were applied to water distribution system design in the previous studies [17, 18, 21, 22]. The RS was used as the subject to be compared. It includes the process of searching solutions by creating uniform random numbers within a range of searching solutions. The GAs is one of the early developed metaheuristic algorithms, and it mimics the evolutionary phenomena that are the most widely utilized. The SA mimics the quenching process. While the GAs evolves several solutions in the population, the SA improves one solution repeatedly. The HSA mimics behaviors of music players, and performs well for the combinatorial optimization problems. The WCA is an almost recent developed metaheuristic algorithm among the considered applied algorithms, and it mimics hydrological cycle process.

Also, I'd suggest rephrasing the explanation of how parameters of the algorithms were determined. I get the idea from the authors' response that sensitivity analyses were conducted, but based on the texts here it is still a bit confusing.

Response: Thank you for your valuable comment. We revised the explanation considering the raised comment for better understanding in the revised manuscript highlighted in blue colors.

Revised Part: Each metaheuristic algorithm was tested 20 individual runs for each of the 20 cases shown in Table 4. The parameters of applied metaheuristic algorithms were determined using sensitivity analysis. The characteristic factors of water distribution system design benchmark were set to the default value (see Table 4), and the optimization parameters of each metaheuristic algorithm were tested. The combination of parameters that derive the best optimization results were determined, and the parameters were applied to 20 cases of benchmark problems. The maximum number of function evaluations was used as the stopping criterion and was set to 50,000, and the results were compared at 10,000th, 30,000th, and final function evaluation.

Line 193 (slow conversion results): Based on the authors' response I guess the authors meant "convergence" instead of "conversion"? Namely, the searching algorithm will converge towards the best result it can find as iteration goes on, not converting something into the best result.

Response: Thank you for your comment. As the reviewer rightly pointed, 'convergence' is correct, not 'conversion'. We have corrected this misrepresentation in the revised manuscript highlighted in blue color.

Revised Part: The GAs, HSA, and WCA all found feasible solutions in all individual runs for each case in the final function evaluation. However, the GAs had slow convergence results compared to the HSA and WCA at 10,000th, 30,000th, and final function evaluation—especially with variations in demand and number of pipes.

Line 224: The revision is appreciated, but still there's room for more clarity. For example, what does "perform poorly in terms of standard deviation" mean? Does it mean large standard deviation? Although it might seem intuitive to the authors, it is not immediately obvious for readers, as a smaller standard deviation is not always considered "better".

Response: Thank you for your insight and detailed comment. The reviewer truly mentioned about this matter. We have revised the explanation for better understanding highlighted in blue color.

Revised Part: . In this study, four improvement ratio statistics were used to compare five existing algorithms: the mean and standard deviation of the average improvement ratios, and the mean and standard deviation of the best improvement ratios. Note that, if the standard deviation of the decision variables is large, it can be estimated that various solutions can be derived at each independent run. Indeed, if the standard deviation of the objective function of the final solution is large, it means that the probability of converging to a good solution is low. Therefore, in this study, reliability is considered to be superior when the standard deviation of improvement ratios is small along with obtaining better cost function.

The HSA showed the best results when varying the number of candidate pipe diameter options, while it had the worst results when the number of pipes was varied. The HSA also showed the best results in terms of the means of the average improvement ratio and best improvement ratio for three factor variations: pressure constraint, roughness coefficient, and nodal demand multiplier. On the other hand, the HSA showed large standard deviations for these three characteristics.

Lines 232-237: Discussion of specific guidance should be given, and a concise version of that discussion should go into conclusion. These sentences provide a good starting point of specific guidance. Please elaborate on this in the discussion, and provide a concise version of the discussion in the conclusion. These study-specific findings, if discussed further and deeper, form the foundation of discussion that sets the present study apart from general statements that could be made without running all the optimizations.

Response: Thank you for your meaningful comment. We have revised the discussion and conclusion in the revised manuscript highlighted in blue colors.

Revised Part: The metaheuristic optimization algorithms have their own strengths and weaknesses, and no method performs perfectly better than the other with respect to all aspects. Furthermore, as the size and complexity of the problems increase, performance and reliability of all reported algorithms weaken consistently. Therefore, the choice of method should be dependent on the characteristics of optimization problem. For instance, the SA can be applied to simple optimization problems with several individual runs, while the GAs can be used to find optimal solutions for large-scale optimization problems offering reliable solution at every single run. The HSA is suitable for the problems include high complexities, however the results should be verified to find reliable solution. The WCA can be efficiently applied to optimization problems which need long computational time, such as real time estimation problems, feature selection, and classification of big data.

These performance and reliability characteristics of algorithms can be used as a guidance to select proper optimization algorithm for given problems. In addition, the existing optimization algorithms require improvement by enhancing the optimization process and introducing additional engineering approaches considering characteristics of given problem.

Each applied metaheuristic algorithm had its own strengths and weaknesses, and the performance and reliability of all studied algorithms weakened as the size and complexity of the problems increased. In addition, each algorithm showed its own convergence characteristic for given design problems. This implies that finding optimal solutions for engineering problems using a metaheuristic algorithm requires an efficient approach that considers the characteristics of the given design problem. In this study, performance and reliability metaheuristic algorithms were evaluated through the water distribution system design benchmarks, and a guidance to select the proper optimal algorithm for given problem characteristics was suggested from the results of this study.

Meanwhile, cost minimization was selected as an objective function in this study, and the node pressure requirement was used as a hydraulic constraint. However, real water distribution system designs have several objectives (such as system reliability and greenhouse gas emissions) and constraints (such as water flow velocity limitation and water quality requirement). Therefore, in future studies, various combinations of objectives and constraints with other problem characteristic modification factors will be considered including multi-objective optimization problems; then they will be applied to benchmark problem generation for performance and reliability comparisons among optimization techniques. In addition, five metaheuristic optimizers were compared using proposed water distribution system design benchmarks in this study, however other optimization algorithms and other variants will be tested in the future studies.

Author Response File: Author Response.pdf

Article Menu

Generation of Benchmark Problems for Optimal Design of Water Distribution Systems

Further Information

Guidelines

MDPI Initiatives

Follow MDPI