Comparison of the Point-Collocation Non-Intrusive Polynomial (NIPC) and Non-Intrusive Spectral Projection (NISP) Methods for the
γ
−
R
θ
Transition Model
Round 1
Reviewer 1 Report
This paper addresses uncertainty quantification of laminar-turbulent transition flow. The gamma-Re_theta model is particularly considered as the underlying physics model, and two non-intrusive uncertainty quantification schemes are adopted and compared. The topic of uncertainty quantification is timely and important, and understanding flow transition also exhibits scientific values. However, this paper has several major issues to be publishable:
- Mathematical notions and descriptions are rather abrupt and unclear. For example, the three entries that define the probability space (Theta, Sigma, P) are used without any explanation and the argument theta to define the random variable xi is not explained. The notations in (1),(2) are also need much better clarifcation. s^p, s^||, L2 are not defined. N is not clear in (2.8), gamma is not clear in (2.9), and so on. There are much more cases where notations are not clear so that it is difficult to understand the description.
- The contributions of the paper are not very clear. To the reviewer, it seems like the UQ methods are not new and actually pretty well-established, but the transition model is relatively new (first introduced 10 years ago by [21]); thus, the contribution is application of well-established UQ methodologies to a relatively new problem. If this is the case, the authors need to state it in a much more clear manner; otherwise, the authors should be able to claim sufficient contributions compared to the literature.
- Flow transition is certainly a difficult problem in which many sources of uncertainties may come into play. But, it is not clear from this article why the three coefficients are chosen as the uncertain parameters, and why such lower and upper bounds are set, and how much those settings make sense. The authors should better justify this. The present article simply reads like that those choices are quite arbitrary. Also, it is not very well justified why the uniform distribution is considered for those parameters.
- The presentation of results are not easy to understand. For example, to just show the numbers like table 4 is not a good way of representing the results. Given Figure 2 and similarly Figure 6. It would be much better to overlay the UQ results on top of these pictures so that readers can directly compare the deterministic solution, experimental results, and UQ mean and dispersion. Results in Figures 3 and 7 are not easy to capture, either.
- Given the description in page 10. It reads like 10% uncertainty in the coefficient would just make 2.9% of the results. Is this really significant? If it is, then need better description. Otherwise, the problem is not very well motivated.
- In Figure 11, there are quite substantial gap between the NIPC/NISP results and Monte Carlo. Does this result really imply that the proposed UQ schemes work well for this problem? It looks not..
- Minor issues:
-- Table 5 is split over two pages.
-- Line 293, in this "chapter" -> no chapter in a single article.
-- Figure 4&8: legends and numbers are not legible.
Author Response
Dear Reviewer 1
We appreciate your valuable and sincere comments.
Our manuscript is revised to address each of the comments.
We will attach the file with the answers to the reviewer's comments.
Sincerely,
Chang.
Author Response File: 
Author Response.docx
Reviewer 2 Report
The authors apply three UQ methods to a CFD model, but there are serious flaws in the methodology.
The method called "Non-Intrusive Spectral Projection" (NISP) is often called dense or full grid and is know to be one of the most inefficient methods for multi-dimensional integration. For this type of problems, sparse grids methods are far more efficient, i.e., second order accuracy with Gauss-Legendre points can be achieved with only 181 points and third order requires 1177. More efficient Gauss-Kronkord-Patterson rule can gain third order accuracy with 871 points. The dense grid method has been driven obsolete by the work of people such as: Michael Grieble, Clayton Webster, Fabio Nobile, Dirk Pfluger, and many others.
The method called "Point-Collocation Non-intrusive Polynomial Chaos" (NIPC) method uses a much better basis to construct model approximation (in fact this is the same basis targeted by the sparse grids methods), the reduction of computational work compared to the dense methods is well known. However, the authors compute the coefficients using equation (2.12), where the matrix is badly conditioned especially when p gets large. Sparse grids methods offer convenient ways to find the "solution" without the need to solve a linear system, provided the sample points are chosen in a specific way (e.g., according to a quadrature rule). Oversampling in the NIPC method can be used to control the condition number of the matrix, but this is not needed when the sample points can be controlled, e.g., when using computer simulations. Oversampling is a viable approach when dealing with random points or noisy outputs, e.g., when dealing with experiments.
The NIPC method wit over sampling is often called L2-projection. A more interesting case arises from solving under-determined system of equations of the form (2.12), which is leads to the L1 or "compressed sensing" methods. The L1 approach is an advanced UQ technique and far superior the methods used by the authors.
The numerical problem is very "easy" from the UQ perspective. The total variability in the model response is only a few percent, which means that the Monte Carlo method requires only a few samples. Second order polynomial approximation is also very coarse. I strongly recommend that the authors design a more challenging problem (e.g., bigger range of inputs), and then apply a more current UQ methodology.
Author Response
Dear Reviewer 2
We appreciate your valuable and sincere comments.
Our manuscript is revised to address each of the comments.
We will attach the file with the answers to the reviewer's comments.
Sincerely,
Chang.
Author Response File: Author Response.docx
Reviewer 3 Report
The authors study the uncertainty in coefficients $\gamma-R_{\theta}$
of a laminar-turbulent transition closure model (Menter's), and
demonstrated this investigation for a flat plate and Aerospatiale
A-Airfoil. PCEs are constructed for QoIs skin friction coefficients
and drag and lift coefficients using NIPC and NISP. The QoI
distributions, mean, variance are compared with Monte Carlo. I will
recommend accepting but need to address some issue below first.
The major issues are as follows:
1. The conclusions and results claim that NIPC converges faster than
NISP as PCE order is increased. This is not clearly supported by
Figure 3, 7, 9. I do not see a trendline with a steeper slope for
NIPC than NISP, and in fact NISP has better error than NIPC at
several places. Please make the plot to convey this trend, perhaps
using a plot of log-error, and use a higher-sample Monte Carlo for
reference (true solution).
2. The number of runs used for NIPC and NISP is not a fair
comparison. While NIPC is using an oversampling factor of 2, which
is best practice, NISP is not using best practice. Nobody uses
tensor-product quadrature to do NISP these days, at least sparse
quadrature should be used, and better yet is to use adaptive sparse
quadrature. Please either redo this using sparse quadrature, or
rephrase your descriptions without claiming that NIPC requires less
samples than NISP outright, and mention that sparse quadrature
would also be a better option to make a more even comparison.
3. The dominance of Ce1 in Sobol results is surprising, and really
begs the question why were Ca1 and Ce2 not detected to be
non-sensitive in the initial sensitivity analysis and then
discarded? Please include the initial sensitivity analysis, and
it's okay if there only Ce1 is the most dominating, just say
ordinarily you would discard them but here you want to do top 3 to
illustrate multidimensional PCE.
More comments below:
- p1, line 34-35: "UQ is a method used to..." What you described there
is uncertainty propagation, which is a subset of uncertainty
quantification. For example, UQ also entails uncertainty reduction
(e.g. Bayesian inference or "inverse" UQ), design under uncertainty,
etc. Suggest change to something like "One important part of UQ is
uncertainty propagation, which is used to quantify the
sensitivity..."
- p1, line 40: "Intrusive methods show exponential convergence", this
gives the impression that non-intrusive methods do not have
exponential convergence. That would not be true, non-intrusive is
still a spectral method, and would have the same convergence rate.
- p4, line 120: maybe replace "determined" with "known analytically"
to emphasize that they the norm quantities are easily accessible.
- p4, line 132: GL and GH are just possible (and natural) choices;
though generally any other quadrature rule can be used, such as
Clenshaw-Curtis, Gauss-Lobatto, etc. Of course the finite/infinite
support has to be appropriate.
- p5, line 148: This formula is only true for a total-order expansion
of polynomial order $p$. Please clarify that.
- p5, line 158: With oversampling, it is no longer an
interpolation. Is it still considered NIPC? What if undersampling?
The solution is non-unique. Is it important the right distribution
for sampling $\xi$ is used?
- p5, line 155, 156: These are true for any PCE regardless how it is
constructed (NIPC or NISP, or intrusive). It is now under the NIPC
section which makes it seem like it is only true for NIPC. Please
adjust.
- p7, line 214: "required number of random inputs is 19,683 or 262,144
in the NISP method", do you mean the number of terms in the PCE, if
a total-order expansion were used? If this is what you meant, then
it is not only for NISP, but any PCE that you expand with total
order, in which case please revise that. But if you meant this is
the number of model runs for NISP, then please explain how you get
that? Doesn't that depend on what kind of quadrature rule you use
(how many points, if it is tensor-product or sparse or adaptive), or
how many random points you use if it is Monte Carlo based? This
sentense is also implying NISP is more expensive than NIPC, but I
think that is not justifiable here (and you probably didn't intend),
so please consider rephrasing.
- p7, line 224: What were the variance values you computed for
sensitivity analysis? Please include this in a table for all 9
variables. How different are the top 3 compared to the next 6? For
future can consider Sobol indices that takes into account of
simultaneous variation, rather than one at a time; this can capture
potential correlation effects.
- p8, line 232: okay I see for NISP you always use a tensor-product
quadrature rule. I highly recommend sparse quadrature and adaptive
versions, it is much more efficient and widely available, and nobody
uses tensor-product for NISP these days. Many software packages can
be found on the web.
- p10, line 266: what do you mean by the first sentence? It is
unclear.
- p10, Figure 3: It would be better to plot the absolute error in
semilog-y scale rather than the value itself. Furthermore, I don't
think this is conclusive enough to say NIPC converges faster (i.e.,
steeper slope in the error decrease?). For example, for mean, NISP
got to closer to green line than NIPC right at order 3. Can you run
more Monte Carlo samples? I fear Monte Carlo is now too noisy and
not accurate enough to serve as a reference (true) solution for NIPC
and NISP (those methods converge much faster than Monte Carlo).
- p11, line 293: "chapter" -> "section"
- Figure 7, Figure 9, similar comments as Figure 3.
- p14, Figure 8: in text you mentioned you will discuss why the
histograms are very different in the next section, but you never
did. Please add discussions and explanations why they are so
different for Cl in this figure, especially between the lower order
and higher order PCEs (which we didn't see a big difference for the
other 2 QoIs)?
- p16, Table 9 and 10: Ce1 seems dominating compared to Ca2 and
Ce2. Why was this not detected in the initial 9-input sensitivity
analysis? Why did you retain 3 inputs whereas this result suggests
really only Ce1 is needed?
- Please improve the overall language/English, it's not bad but several places sound awkward and overall paper can be made better.
Author Response
Dear Reviewer 3
We appreciate your valuable and sincere comments.
Our manuscript is revised to address each of the comments.
We will attach the file with the answers to the reviewer's comments.
Sincerely,
Chang.
Author Response File: Author Response.docx
Round 2
Reviewer 1 Report
Most comments are addressed well.
The reviewer still thinks that MC w/ more samples would provide more evidence of the proposed method working, but understands the limitation in computational resources. That said, instead of simply saying that the three pdfs look similar in Fig. 11, it would be better to say in more details like they have peaks at the same CD values thus maximum likely estimate of CD is within a certain error and so on. Being "similar" is a pretty subjective notion; thus, to provide more objective/quantitative information would be better.
Author Response
Reviewer 1
Dear Reviewer1
Thank you for the opportunity to submit a revised version of our manuscript entitled Comparison of the Point-Collocation Non-Intrusive Polynomial Chaos (NIPC) and Non-Intrusive Spectral Projection (NISP) Methods for the g-Req transition model in Applied Sciences.
We revised our manuscript to address each of the comments and the modified sentences are highlighted in blue color.
1: The reviewer still thinks that MC w/ more samples would provide more evidence of the proposed method working, but understands the limitation in computational resources. That said, instead of simply saying that the three pdfs look similar in Fig. 11, it would be better to say in more details like they have peaks at the same CD values thus maximum likely estimate of CD is within a certain error and so on. Being "similar" is a pretty subjective notion; thus, to provide more objective/quantitative information would be better.
è answer:
We appreciate the reviewer’s valuable and sincere comments for our manuscripts.
We entirely agree with the reviewer’s comment.
As reviewer mentioned, instead of saying that the patterns of the distributions are similar among the three methods which described in Fig. 11, so we added the sentence with the objective/quantitative information in the modified manuscript. (in line 390)
“In the case of flat plate, the peak values of the drag coefficients, 0.0041 agree well in three methods; NISP, PC-NIPC and Monte Carlo. This peak is located at a higher value than that (0.00401) of the deterministic solution. In the drag coefficients of Aerospatiale A-Airfoil simulation, there are two peaks in the PDF and the left peak is lower than the right one in every methods. The predicted peak values in NISP and PC-NIPC methods are approximately 0.0206 with 0.03% error with that of Monte Carlo method. The pattern of the distribution of the lift coefficient in A-Airfoil simulation is similar to that of the drag coefficient in the flat plate simulation. The peak value of NISP show smaller difference with 0.46% than that of PC-NIPC, 0.86% when compared with that of Monte Carlo’s result.”.
Author Response File: Author Response.docx
Reviewer 2 Report
I understand that it is simply infeasible for many research communities to keep with the latest UQ advancements, but I still maintain that the methodology in the paper is too outdated. There are many newer and more refined algorithms that can be applied "out-of-the-box" with publicly available software, and any of those would be of greater interest to any community.
Author Response
Reviewer2
Dear Reviewer 2
Thank you for the opportunity to submit a revised version of our manuscript entitled Comparison of the Point-Collocation Non-Intrusive Polynomial Chaos (NIPC) and Non-Intrusive Spectral Projection (NISP) Methods for the g-Req transition model in Applied Sciences.
I understand that it is simply infeasible for many research communities to keep with the latest UQ advancements, but I still maintain that the methodology in the paper is too outdated. There are many newer and more refined algorithms that can be applied "out-of-the-box" with publicly available software, and any of those would be of greater interest to any community.
è answer:
We appreciate the reviewer’s valuable and sincere comments for our manuscripts.
We entirely agree with the reviewer’s comment.
As the reviewer mentioned, we know that our methods regarding to UQ in this study are a little bit outdated. However, we think that many researchers in CFD and transitional flow simulation fields are not still accustomed to uncertainty quantification and they will be interested in our results even though the adopted UQ methods are old-fashioned in UQ fields. Our results will be the basis to move to the next step for application of the newest methodologies of UQ in this field. Also we are adopting the newest methods which the reviewer mentioned to transition model and will obtain meaningful results soon.
Thank you very much for your valuable comments.
Author Response File: Author Response.docx
