When Are Models Useful? Revisiting the Quantification of Reality Checks

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

The submitted manuscript is, in some ways, disconcerting, yet it is rich in content, presents novel results, and offers valuable material for scientific discussion. It is well-founded, instructive, and well-written. I have identified only a few errors or points requiring clarification, which can be easily addressed (see annotated manuscript).

The manuscript consists of three main parts. The first part presents and discusses various efficiency criteria, including error variance and bias, based on the K-moments introduced by the authors. This section provides a clear and useful summary of several previous publications. However, a potential criticism of the authors' perspective is that there may not be an ideal model efficiency criterion. Assessing a model's efficiency depends on the users' needs, which should influence the choice of the criterion. For example, in hydrology, forecasting low flows or floods requires good agreement between modeled and observed discharges across very different value ranges.

The second part compares the relative merits of the various criteria using a theoretical example (section 3.2). A new concept is introduced: the possible transformation of data. I am personally less convinced by this section, as it does not clearly illustrate the insights from the earlier parts of the manuscript. Its conclusions seem highly dependent on the chosen theoretical case study. Furthermore, if the transformation improves performance scores, the fundamental issue remains: model users are interested in the original values. The transformation does not change the underlying fact that the model's performance is poor.

The final part evaluates the rain rates simulated by Global Circulation Models (GCMs), which are used for climate projections. While this section does not directly demonstrate the added value of the K-moments-based criteria, it presents important and original results. The rain rates simulated by GCMs are known to be strongly biased—bias correction methods are typically applied—but they are also poorly correlated with observed rain rates, even when averaged over large areas (such as hemispheres) and long periods (one or more years). Given its topicality and importance, this final part could have stood alone as the focus of a separate paper.

Comments for author File: Comments.pdf

Author Response

Please see attached report

Author Response File: Author Response.pdf

Reviewer 2 Report

Comments and Suggestions for Authors

General comments: the paper submitted is in continuity with the author’s previous works from 2019 to 2023. It is about the use of knowable moments and derived coefficients in time series modelling. Knowable moments of order p use the order statistics (statistics of ranked values). They are expectations of orders statistics of order p (such as the max and the min). The justification is that classical moments are poor in restituting extremes and non-normal distributions. Here, the author explains what the discrepancies of NSE and KGE, which are based on the expected mean and the variance as well as the asymmetry coefficient, in model fitting,  are in comparison to the use of knowable moments. He proposes to retain the knowable moments of order one and two to this purpose. He answers to the question “when are the models useful?” line 171 to 172.  He prposes the use of two criteria namely the K explained variance KUVp and KEVp based on the estimation of te dispersed ratio of order 2. This is by analogy with riteria adopted using classical statistics. Two case studies are presented: a synthetic series with persistence; rainfall series generated by several CMIP6 considered at the yearly time step in comparison to ERA5 “real data”. The paper is clear in general except for a few points. It is very useful from practical point of view.

The transformation proposed may cite the Box and Cox transformation. Line 197

Order moments should be in key words and title

A section should be added to criticize ERA5 rainfall data as it appears in the references

The 8 year moving average should be mentioned

In the text before Eq 5 it seems that it is RB

The median absolute deviation could be mentioned

In Line 133 and elsewhere it should be mentioned maximization with respect to what? rxs and sigmas are two facies of the result. They are linked.

Before Eq 24 it should be added that the sample size is n now. Line 159-160.

In Figure 3 and 4 it would be interesting to plot the case of the log transformation

Why 8 years for both NH and SH? There ere mass transfer between them and maybe a delay. Others works using these models should be cited.

 

 

 

 

 

Author Response

Please see attached report.

Author Response File: Author Response.pdf