Round 1

Reviewer 1 Report

Review of manuscript submission in Processes; Computational Experience with Piecewise-Linear Relaxations for Petroleum Refinery Planning

The paper presents the computational performance of the use of five different piecewise linear relaxations for nonconvex bilinear optimization problems. The authors investigated different partitioning schemes including uniform and nonuniform partitioning schemes for nonconvex optimization, and their results suggest that the tightness of the relaxation depends on the type of partition (uniform or non-uniform) and the number of partitions. The different piecewise linear relaxations were applied to bilinear terms in a petroleum refinery planning problem.

Overall, the paper communicates the main conclusions well. However, I find that a lot of details are missing from the reformulations presented. Even though the first four computational schemes are not new or are available in the literature, the authors need to still provide more details as a background for readers beyond what they provided. For example, why does the big M approach provide tighter relaxation than the McCormick reformulation and the impact of additional binary variables? Also, the de approach needs more background material to support it. Additionally, it would be nice to have more cases to assess and validate the conclusions made in this paper.

Other comments on the manuscript include:

 

1. Line 116, Typo in “Description”
2. Reference error on line 117.
3. Could you make Figure 2 bigger?
4. Line 139. Recurring reference error. Please check.
5. From your study, does tightness in convex relaxation always lead to speed up in computational time?
6. In practice, the global optimum is unavailable or computational difficult to obtain- e.g. BARON/ANTIGONE doesn’t converge in a reasonable time. In such circumstances, how would an algorithm that utilizes these PWLs terminate or guarantee some certificate of optimality?
7. In the conclusion, the author emphasizes the result is consistent with the literature. Can you elaborate on the literature data to support that claim and what significant contributions does your work provide?
8. I searched for the supplementary material to look at more details of the model but could not find it. It would be nice to look at it.

Comments for author File: Comments.docx

Author Response

Response to comments of Reviewer 1

 

Overall, the paper communicates the main conclusions well. However, I find that a lot of details are missing from the reformulations presented. Even though the first four computational schemes are not new or are available in the literature, the authors need to still provide more details as a background for readers beyond what they provided. For example, why does the big M approach provide tighter relaxation than the McCormick reformulation and the impact of additional binary variables? Also, the de approach needs more background material to support it. Additionally, it would be nice to have more cases to assess and validate the conclusions made in this paper.

Reply: More background information has been added on the piecewise linear relaxation schemes in Reformulation as Relaxation Models” section of the revised manuscript (lines 158 to 194). Further, this work focusses on investigating the suitability of piecewise-linear relaxation schemes for refinery planning applications by using the base model of Li et al. (2005). Since the results and the validation are deemed sufficient to show the advantages and disadvantages of the schemes, we do not consider an extension of the study to report on other cases or examples.

 

1. Line 116, Typo in “Description”

Reply: We have corrected this typographical error (line 108, p. 3).

 

2. Reference error on line 117.

Reply: We have corrected this reference error of Figure 2 (line 110, p. 3).

 

3. Could you make Figure 2 bigger?

Reply: We have enlarged Figure 2 (p. 4).

 

4. Line 139. Recurring reference error. Please check

Reply: We have corrected this reference error on Table 2 (line 131, p. 4).

 

5. From your study, does tightness in convex relaxation always lead to speed up in computational time?

Reply: The relaxation tightness reduces the optimality gap but increases the solution time due to a larger reformulated model size entailed. The revised manuscript includes CPU time taken for the relaxation schemes studied (see Figure 4) with corresponding explanation as excerpted below (lines 216 to 223, p. 8):

All the relaxed models were solved within a fraction. CPU times for relaxation schemes are plotted in Figure 04 as a function of number of partitions and uniform partition sizes are taken as a representative case. The average values of CPU times are 0.144, 0.170, 0.180 and 0.216 for bm, de, nf6t and nf5 schemes. It increases as the number of partitions increase. The CPU time is the highest for nf5 scheme and the lowest for the bm scheme, which in also consistent with their relative sizes. Gounaris et al. [37] have also reported CPU times for many relaxed benchmark problems and concluded that the relaxation tightness reduces the optimality gap quickly, but it increases the solution time due to its bigger size.

 

6. In practice, the global optimum is unavailable or computational difficult to obtain- e.g. BARON/ANTIGONE doesn’t converge in a reasonable time. In such circumstances, how would an algorithm that utilizes these PWLs terminate or guarantee some certificate of optimality?

Reply: A piecewise linear relaxation (PWL) scheme as investigated through the five schemes in this work gives a certificate of optimality by computing an upper bound of the objective function value for a maximization problem (or conversely, a lower bound for a minimization problem) which we demonstrate through a nonlinear refinery planning model (with profit maximization as the objective).

 

7. In the conclusion, the author emphasizes the result is consistent with the literature. Can you elaborate on the literature data to support that claim and what significant contributions does your work provide?

Reply: The results from our work are consistent with the literature as explained in the Computational Results section of the revised manuscript particularly in the following excerpt (lines 211 to 213, p. 8):

A convexified model using nf5 is tighter than nf6t but at the expense of a larger size while that of bm as well as de varies with respect to , which is consistent with reported results [37].

as well as the following excerpt (lines 243-247, p. 10):

Figure 12 shows a lowest deviation (about 33 percent) for uniform partitioning () compared to other non-uniform partition sizes, thereby indicating better relaxation quality for the former in employing a piecewise-linear relaxation scheme. This observation is consistent with (Hasan and Karimi 2009) which reports advantages of uniform partitioning in reducing the optimality gap (i.e., between the nonconvex original objective function and that of the relaxed convex objective  function for each partition).

We have also revised the Concluding Remarks section to emphasize the contribution of this work as excerpted in the following (lines 271 to 279, p. 13):

We introduce a new relaxation scheme based on eigenvector decomposition that is shown to be able to give good relaxation results especially for non-uniform partition sizes. As alternatives to conventional measures (e.g., computational time or optimality gap, which are more applicable to large scale industrial-size problems), we consider certain indicators to compare the schemes in terms of convergence and partitioning behavior. Our study encounters a limit on the number of partitions that contributes to relaxation tightness, which does not necessarily correspond to a large number of partitions. We also find that the relation between a relaxed formulation size and its tightness significantly depends on the number and size (uniform/non-uniform) of the partitions. Furthermore, the computational results show better relaxation quality by using uniform partition sizes.

 

8. I searched for the supplementary material to look at more details of the model but could not find it. It would be nice to look at it.

Reply: The supplementary material has been revised with more details added (e.g., with the reformulated relaxed model as advised in Reviewer 2’s comment no. 2 (in a later part of this document).

Author Response File: Author Response.pdf

Reviewer 2 Report

The authors compared the performance of different piecewise-linear relaxation schemes for bilinear terms in refinery planning models. However, most of these schemes have already been presented and compared in the literature. Although the authors came up with the relaxation scheme based on eigenvector decomposition, the performance is not better than that of Big-M scheme. Therefore, there is no significant contribution of this work. This paper should be rejected.
Other comments:
1. The authors did not provide detailed explanation on the fifth scheme which is based on eigenvector decomposition. The relaxation model for this scheme is confusing. For instance, what is the meaning of ξj? Why ξ has no indices as it is the average of Lu and Wp?
2. The reformulated relaxation models cannot be found in the supplementary material.
3. What is the definition of partition size γ? when γ = 1, it is uniformly partitioning. How to achieve
non-uniform partition for other values of γ such as γ = 0.25?
4. In the references 36 and 37, several piecewise-linear relaxation models are compared. It is not clear why the authors used nf5 and nf6t from these references? It seems that the authors did not use the piecewise-linear relaxation model with the best performance. For instance, why the authors did not use nf4r which demonstrated the best performance from the reference 37.
5. It is very strange that bm shows tighter relaxation compared to other schemes when the partitioning
levels and sizes are increased. Did the authors solve each case to optimality?
6. The convergence indicator should be defined mathematically. What is the meaning of high value of the convergence indicator?
7. Some errors could be found in the paper, which requires proof-reading. For instance, Error! Reference source is not found. The reference 27 and 33 are the same.

Author Response

Response to comments of Reviewer 2

 

General Comments

The authors compared the performance of different piecewise-linear relaxation schemes for bilinear terms in refinery planning models. However, most of these schemes have already been presented and compared in the literature. Although the authors came up with the relaxation scheme based on eigenvector decomposition, the performance is not better than that of Big-M scheme. Therefore, there is no significant contribution of this work. This paper should be rejected.

Reply: In this work, we evaluate four existing piecewise linear relaxation schemes by employing them in a refinery planning model to gain insights on their performance and affirm reported observations in the literature. Our work gives observations on an optimality gap through use of a metric called convergence indicator (defined and explained in the revised manuscript) as well as delineate the relation between relaxation tightness and size for a piecewise linear scheme with respect to (non-)uniform partitioning and number of partitions. This work proposes a new PWL scheme based on eigenvector decomposition that shows improve relaxation quality for non-uniform partition particularly for increased number of partitions.

 

1. The authors did not provide detailed explanation on the fifth scheme which is based on eigenvector decomposition. The relaxation model for this scheme is confusing. For instance, what is the meaning of ξj? Why ξ has no indices as it is the average of Lu and Wp?

Reply: Equations (27) to (39) have been corrected to add the required indices as pointed out by the reviewer. We also give more details for a proposed scheme of de in the Reformulation as Relaxation Models section in the revised manuscript (lines 185 to 194, p. 7) as excerpted below:

As the univariate function is convex on the real-valued domain of Â, the affine overestimators and underestimators can be obtained for -direction by applying, e.g., the sandwich algorithm [63], which linearizes the function at point  as performed in equation (30). Then, piecewise-affine relaxations are constructed for  by dividing the domain of  into  subintervals. Auxiliary continuous variables  are introduced to select the domains where a feasible solution exists. These are special ordered set type 2 (SOS2) variables for which not more than two adjacent variables may be nonzero in the final solution.

 

2. The reformulated relaxation models cannot be found in the supplementary material.

Reply: The reformulated relaxation models have been added in the revised supplementary material.

 

3. What is the definition of partition size γ? when γ = 1, it is uniformly partitioning. How to achieve non-uniform partition for other values of γ such as γ = 0.25?

Reply:  is a parameter that specifies uniform or non-uniform partition sizes in which  corresponds to the former (uniform partition sizes) of equal segment lengths while  corresponds to the latter (see lines 174 to 780, p. 6 in the revised manuscript).

 

4. In the references 36 and 37, several piecewise-linear relaxation models are compared. It is not clear why the authors used nf5 and nf6t from these references? It seems that the authors did not use the piecewise-linear relaxation model with the best performance. For instance, why the authors did not use nf4r which demonstrated the best performance from the reference 37.

Reply: The nf5 and nf6t schemes have been chosen because they belong to the incremental cost class and reported to have less non-convergence issue [36, 37]. Moreover, we select nf5 because of its larger size than nf6t to compare in terms of relaxation sizes (or number of variables). We explain this justification in the Reformulation as Relaxation Models section in the revised manuscript (lines 141 to 144, p. 5) as excerpted below:

[…] two incremental cost relaxations called nf5 and nf6t schemes, which have been chosen because they are reported to have less non-convergence issue [36, 37], and a proposed scheme involving decomposition in eigenvector directions called de. Moreover, nf5 is selected because of its larger size than nf6t for comparison in terms of relaxation sizes (or number of variables).

 

5. It is very strange that bm shows tighter relaxation compared to other schemes when the partitioning levels and sizes are increased. Did the authors solve each case to optimality?

Reply: This is an observation from our work, which emphasizes that the relaxation tightness depends on the uniform/non-uniform domain partitioning and the number of partitions. All the instances or cases are solved to optimality mainly based on a default optimality gap in GAMS/CPLEX.

 

6. The convergence indicator should be defined mathematically. What is the meaning of high value of the convergence indicator?

Reply: As suggested, we define convergence indicator in the revised manuscript (refer equation (40)) with elaboration (see lines 227 to 232, p. 9).

 

7. Some errors could be found in the paper, which requires proof-reading. For instance, Error! Reference source is not found. The reference 27 and 33 are the same.

Reply: We thank the reviewer for his pointing out these errors, which we have corrected duly in the revised manuscript.

Author Response File: Author Response.pdf

Reviewer 3 Report

The paper presents results of a numerical comparison of some techniques to deal numerically with the refinery planning optimization problem. I suggest introducing some minor revisions as follows.

a) When citing several works on the construction of piece-wise functions for optimization problems, the works by Y.Sergeyev should not be missing. As more representatives ones, please mention the following papers:

1. Sergeyev Y.D. (1998) Global one-dimensional optimization using smooth auxiliary functions, Mathematical Programming, 81(1), 127-146.
2. Sergeyev Y.D. (1998) On convergence of "Divide the Best" global optimization algorithms, Optimization, 44(3), 303-325.

b) A (sub) section dedicated to the practical usage of the obtained numerical results and their interpretation should be introduced. The question for me as an applied user is: what methods should I use and how to use them to address the stated practical problem?

Author Response

Response to comments of Reviewer 3

 

The paper presents results of a numerical comparison of some techniques to deal numerically with the refinery planning optimization problem. I suggest introducing some minor revisions as follows.

 

1. When citing several works on the construction of piece-wise functions for optimization problems, the works by Y.Sergeyev should not be missing. As more representative ones, please mention the following papers:
   146. Sergeyev Y.D. (1998) Global one-dimensional optimization using smooth auxiliary functions, Mathematical Programming, 81(1), 127-146.
   147. Sergeyev Y.D. (1998) On convergence of "Divide the Best" global optimization algorithms, Optimization, 44(3), 303-325

Reply: We thank the reviewer for the suggestion and have cited the mentioned work in the Introduction section (lines 77 to 78, p. 3, references [47] and [48].

 

2. A (sub) section dedicated to the practical usage of the obtained numerical results and their interpretation should be introduced. The question for me as an applied user is: what methods should I use and how to use them to address the stated practical problem?

Reply:

We thank the reviewer for the comment and added the following explanation in the Computational Results section to indicate a practical use of our results and interpretation (lines 262 to 267, p. 13):

Piecewise-linear relaxations provide an upper bound (lower bound) on the objective function value for a maximization (minimization) problem. By applying several relaxation schemes for a representative refinery planning problem, our computational results shows that the nf5 scheme performs well for uniform partition sizes and can be suitably incorporated as part of a global optimization procedure, whereas non-uniform partitioning works better with the bm scheme particularly for larger number of partitions.

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

The authors did a great job on the first round of revision, and the paper is in better shape now. The biggest application of this work is in proffering multiple convex relaxation solutions that can be used within a B&B search for global optimization of problems with bilinear terms.

Two points the authors should consider addressing.

1. I would suggest the authors spend some time editing English/grammar to improve the quality of the manuscript.

1. It would be helpful if the authors can provide more insight on the tightness of the convex relaxations (for all the methods they presented) and the association between tightness and computational time.

Author Response

Please see the attachment.

Author Response File: Author Response.docx

Reviewer 2 Report

The quality of the manuscript has been improved. However, the novelty of this manuscript is still not very clear and the contribution is not significant as the proposed relaxation scheme based on eigenvector decomposition only performed better than bm when r = 4. This conclusion is only based on one problem instance, which is not convincing. I suggest the authors should add comparative results of these schemes through solving those examples in the literature such as [36] and [37] to demonstrate the advantages of the proposed scheme based on the eigenvector decomposition.

Other minor problems are listed as follows:

1. In section 2, the number of variables and equations in the model should be given to let readers know its scale.
2. The average convergence indicator in Table 3 should better be given for each algorithm separately since different relaxation scheme has evidently different tightness performance and averaging the performance of different algorithms doesn’t make sense.
3. Similar to the last comment, in Fig. 12, the study on the effect of uniform and non-uniform partition sizes should also be applied to each algorithm separately. Especially for the conclusions given in lines 263-266 in page 13, they cannot be seen from Fig. 12.

Author Response

Please see the attachment.

Author Response File: Author Response.docx

Round 3

Reviewer 2 Report

The quality of the paper is improved. The authors should present the computational time for each scheme with different values of r (r <> 1). From the model statistics, there are only 21 bilinear terms, which indicates the presented example is a very small example. I would like to see a larger example with bilinear terms more than 100.

In addition, the authors missed the following two very relevant papers which are related to the piecewise linear approximation.

1. Jie Li, Ruth Misener, C. A. Floudas, Continuous‐time modeling and global optimization approach for scheduling of crude oil operations, AIChE J,  2012, 58, 205-226
2. Jie Li, Ruth Misener, C A Floudas, Scheduling of crude oil operations under demand uncertainty: A robust optimization framework coupled with global optimization, 2012, AIChE Journal 58 (8), 2373-2396

Author Response

Please see the attachment.

Author Response File: Author Response.docx