Local Severe Storm Tracking and Warning in Pre-Convection Stage from the New Generation Geostationary Weather Satellite Measurements

Round 1

Reviewer 1 Report

Please see attached review

Comments for author File: Comments.pdf

Author Response

Thank you very much for your review, we have made corrections based on your suggestions.

Author Response File: Author Response.docx

Reviewer 2 Report

See attached file.

Comments for author File: Comments.pdf

Author Response

Thank you very much for your review, we have made corrections based on your suggestions.

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

Thank you for your response to my review.

This manuscript is a resubmission of an earlier submission. The following is a list of the peer review reports and author responses from that submission.

Round 1

Reviewer 1 Report

The topic of the article is interesting and important. The article is written clearly.

My basic remark concerns the learning and verification of the forecasting model. The authors evaluate accuracy of categorical forecasts by POD, FAR, CSI and HR and main attention is devoted to values of POD and they are focused on forecasts of “severe” and “medium” forecasts. In several places in the text they emphasize how large values of POD the model reaches. However, the model also has large values of FAR. The resultant CSI, which takes into account both POD and FAR, is not very high. In my opinion, large values of both POD and FAR are caused by applied learning technique when the number of “not important” events is significantly reduced. It leads to over forecasting of “severe” and “medium” forecasts. I do not share the authors' view that a fundamental change in the frequency of forecasted events in the learning data improves the prediction. Unfortunately, evaluation of forecasts of small or none precipitation is not shown. The authors should explain why the model forecasts are useful even if they very often forecast existence of severe precipitation in case when this event is not observed (large values of FAR).

Specific comments:

Section Data – could you describe when the NWP model data are available to be used in your model? The NWP model needs some time to give forecasts.

Line 159 – I think better is “severe” instead of “strong”.

Line 200 – Figure caption is too brief. Please explain meaning of delta T.

Line 296 – Is there any reason why 2nd and 15th days were selected?

Line 474 – In my opinion FAR=0.91 is very large value not relatively high.

Line 481 – I do not understand what, do you mean by “sudden local convective storm system intensity”.

Line 501 – I do not understand the sentence.

Line 502 – “the needs for high POD”, could you explain it? It is related to my basic comment.

Author Response

My basic remark concerns the learning and verification of the forecasting model. The authors evaluate accuracy of categorical forecasts by POD, FAR, CSI and HR and main attention is devoted to values of POD and they are focused on forecasts of “severe” and “medium” forecasts. In several places in the text they emphasize how large values of POD the model reaches. However, the model also has large values of FAR. The resultant CSI, which takes into account both POD and FAR, is not very high. In my opinion, large values of both POD and FAR are caused by applied learning technique when the number of “not important” events is significantly reduced. It leads to over forecasting of “severe” and “medium” forecasts. I do not share the authors' view that a fundamental change in the frequency of forecasted events in the learning data improves the prediction. Unfortunately, evaluation of forecasts of small or none precipitation is not shown. The authors should explain why the model forecasts are useful even if they very often forecast existence of severe precipitation in case when this event is not observed (large values of FAR). 

Our focus is on strong convective weather with greater destructive power. When building the model, it should be completely in accordance with the actual situation. Actually, all of cases tracked by our algorithm represent the real circumstance of convective storm in the area of interest in this study. We use the model trained by these matched samples to predict convective storm in nowcasting application.

Specific comments:

Section Data – could you describe when the NWP model data are available to be used in your model? The NWP model needs some time to give forecasts.

Line 159 – I think better is “severe” instead of “strong”.

Agree. We have changed it.

Line 200 – Figure caption is too brief. Please explain meaning of delta T.

Figure 2 is the flow chart of SWIPE algorithm and RF model training. First, it tracks potential convective cloud clusters. The second step is to divide the convective storm system dataset into three different types (weak, medium, and severe). Finally, the RF algorithm is used to train and develop a convection intensity classification statistical model.

delta T is the cloud top temperature at 10.4 μm band in Two adjacent moments. If the cloud top cooling rate reaches -16 K/hour or lower, the related cloud system will be marked or considered to be a potential or developing convective cloud cluster. We have described this delta T in the caption of Figure 2.

Line 296 – Is there any reason why 2nd and 15th days were selected? 

Thanks for your suggestion. This is a completely random selection. The selected data at 2nd and 15th days are considered as testing data, which are independent sample. The testing data are completely separate from the training data.

Line 474 – In my opinion FAR=0.91 is very large value not relatively high.

If a model with a lower FAR is used, the POD of this model will be lower, which will miss many well-developed convective clouds. Actually, the last two co-authors are professional forecasters at CMA. They always use real-time SWIPE, other radar, nwp and surface data, etc. together for nowcasting application.

Line 481 – I do not understand what, do you mean by “sudden local convective storm system intensity”.  

I mean “SWIPE can track and identify sudden local convective storm system “

Line 501 – I do not understand the sentence.

I mean: The n_estimators, max_depth, and max_features have been tuning in RF model training for many times, which can help us to find an optimal predictive model. It is a common method for choosing the optimal model in machine learning studies.

Line 502 – “the needs for high POD”, could you explain it? It is related to my basic comment. 

The three types of classification here are more difficult than the two types of classification for machine learning algorithm, which induce a higher model's FAR. In the weather forecast process, we have to mark all the clouds that are likely to be called severe convection in the shortest time. In real nowcasting application, the most important thing is to ensure that no cloud or convective system is missed. In theory, a lower FAR is always companied with a lower POD, which will miss many well-developed or potential convective clouds, and the final heavy precipitation cannot be successfully predicted. In short, we will choose a predictive model with a higher POD for nowcasting application.

Author Response File: Author Response.doc

Reviewer 2 Report

This manuscript is too long, and does not clearly show what the results and the significance of the work is.  The abstract is too long to start with, there is a single equation to describe the machine learning techniques or where the different choices for the ration impact the performance of the technique.

The English is very bad throughout, with missing the as well as using direct spoken English terms rather than written professional style, along with direct translations where words are in the wrong order.

The manuscript has a lot of potential as it is trying develop a technique to improve identification and recognisition of convection, but given the way the manuscript is laid out i cannot recommend publishing it at this time.

My main points are :

1) The English needs to be improved throughout the whole manuscript, please have a English first language person check before resubmitting.

2) Equations, you can summarize things better with the mathematics rather than describing in words, this will also help shorten the manuscript as well.

3) Shorten the abstract.

Author Response

This manuscript is too long, and does not clearly show what the results and the significance of the work is.  The abstract is too long to start with, there is a single equation to describe the machine learning techniques or where the different choices for the ration impact the performance of the technique.

The English is very bad throughout, with missing the as well as using direct spoken English terms rather than written professional style, along with direct translations where words are in the wrong order.

The manuscript has a lot of potential as it is trying develop a technique to improve identification and recognisition of convection, but given the way the manuscript is laid out i cannot recommend publishing it at this time.

My main points are:

1) The English needs to be improved throughout the whole manuscript, please have a English first language person check before resubmitting.

An English native speaker has improved our manuscript. 

2) Equations, you can summarize things better with the mathematics rather than describing in words, this will also help shorten the manuscript as well.

We agree with your point and use equations to describe such as POD and FAR in Section 4.2.

3) Shorten the abstract.

We have already shortened our abstract as your suggestion. The current abstract only 327 words.

Author Response File: Author Response.doc

Reviewer 3 Report

The manuscript entitled “Local Severe Storm Tracking and Warning in Pre-Convection Stage from the New Generation Geostationary Weather Satellite Measurements” offers a methodology for the early detection of severe convective storms using a number of remote sensing and numerical model data, through the use of machine learning random forecasts. In general the manuscript is very interesting and addresses a serious issue, namely the accurate and early detection of potential devastating storms. Therefore it deserves to be published in the Remote Sensing journal, after the following comments have been addressed:

1)      First of all the authors state in Line 54 that : “….it is still impossible to accurately predict their occurrence, development, and movement based on the current NWP model….”. It is not impossible, just difficult. There is a big difference, and with the continued development of NWPs (forecasting and nowcasting models) it is becoming more accurate and reliable. Please correct.

2)      More references need to be added for the importance of early monitoring of convective storms.

3)      The manuscript needs more references and less internet links. For example in Line 163 the authors use the reference of TRMM for the GPM and provide a link to the data. Remove the link. The reference is enough.

4)      Lines 174-175: “The uncertainties in this product were well validated and compared using ground-based gauges or radars in globe.”. Add a reference here.

5)      Line 216: “maximum area is about 600 km * 600 km”. Is there a reference for using this area range? Why 600Km and not 500Km? If there is not a reference please provide adequate explanation.

6)      Lines 247-252: The division of the rank of each storm system is based on some other publication? Please remove the link in line 252 and add a proper reference for the ranking given here.

7)      Line 254: Please remove the Wikipedia link. Add a proper reference.

8)      Line 255: “During the period from April to October 2016, a total of 88,351 convective storm events were successfully tracked”. There were 88,351 convective storms in seven months? This seems quite large. Also the authors then state that there were 85,102 slight (or none) convective systems. What do you mean by none? You successfully tracked no convective systems? This paragraph needs rephrasing. Also consider calculating your statistical scores without the “none” storms.

9)      On that subject remove tables 5 and 6 and add the Wilks reference that contains these: “Wilks, Daniel S. (2006). Statistical Methods in the Atmospheric Sciences, Second Edition, Academic Press, London. ISBN 13: 978-0-12-751966-1.”

10)  The authors state that they use GFS data. Do you use GFS forecasts or analysis? This needs to be cleared. Also the authors should comment on the capabilities of GFS in forecasting convective storms with a 0.5 degree resolution. There is a comment in the conclusions that a high resolution model will be better, but the authors should expand on this. Also would it be possible to run such a high resolution model and see the results in contrast with GFS? Even for one case it would be interesting.

11)  In Case-1 the SWIPE algorithm detects the storm 1hour and 23 minutes early. This seems good. In Case-2 however this is below an hour (54 minutes). Is this adequate? The authors should comment on whether or not these times are acceptable, for example for a Civil Protection Agency.

12)  A paragraph should also be included that states the benefits on using this algorithm, over a conventional method (e.g. a high resolution NWP model). As I said this is a very interesting work and the benefits should be very clear to the reader.

Author Response

Review 3

The manuscript entitled “Local Severe Storm Tracking and Warning in Pre-Convection Stage from the New Generation Geostationary Weather Satellite Measurements” offers a methodology for the early detection of severe convective storms using a number of remote sensing and numerical model data, through the use of machine learning random forecasts. In general the manuscript is very interesting and addresses a serious issue, namely the accurate and early detection of potential devastating storms. Therefore it deserves to be published in the Remote Sensing journal, after the following comments have been addressed:

1)      First of all the authors state in Line 54 that : “….it is still impossible to accurately predict their occurrence, development, and movement based on the current NWP model….”. It is not impossible, just difficult. There is a big difference, and with the continued development of NWPs (forecasting and nowcasting models) it is becoming more accurate and reliable. Please correct.

Agree. We have amended this sentence.

2)      More references need to be added for the importance of early monitoring of convective storms.

Thank you for your suggestion. We have made the corresponding changes.

3)      The manuscript needs more references and less internet links. For example in Line 163 the authors use the reference of TRMM for the GPM and provide a link to the data. Remove the link. The reference is enough.

OK. We have changed it.

4)      Lines 174-175: “The uncertainties in this product were well validated and compared using ground-based gauges or radars in globe.”. Add a reference here.

I very much agree with your point and add a reference to explain.

Tang G, Zeng Z, Long D, et al. Statistical and Hydrological Comparisons between TRMM and GPM Level-3 Products over a Midlatitude Basin: Is Day-1 IMERG a Good Successor for TMPA 3B42V7?[J]. Journal of Hydrometeorology, 2015, 17.

This reference describes the IMERG products of GPM Satellite, which is developed by U.S. NASA, and widely used.

5)      Line 216: “maximum area is about 600 km * 600 km”. Is there a reference for using this area range? Why 600Km and not 500Km? If there is not a reference please provide adequate explanation.

According to this reference, the largest area of the MCC can reach nearly 200,000 . The shape of the MCC is irregular and may appear elliptical or long. Considering that the ratio of the major axis length to the minor axis length of the MCC is about 0.7, the length of the MCC in a single direction can exceed 500 km. Thus, in this study, we choose 600km to calculate convective system.

Jirak I L , Cotton W R , Mcanelly R L . Satellite and Radar Survey of Mesoscale Convective System Development[J]. Monthly Weather Review, 2002, 131(10):2428.

6)      Lines 247-252: The division of the rank of each storm system is based on some other publication? Please remove the link in line 252 and add a proper reference for the ranking given here.

This standard used in Wikipedia comes from a book published by the British Meteorological Agency. (https://en.wikipedia.org/wiki/Rain)

7)      Line 254: Please remove the Wikipedia link. Add a proper reference.

Agree. The standards used in Wikipedia are quoted from the Met Office and National Weather Service.

8)      Line 255: “During the period from April to October 2016, a total of 88,351 convective storm events were successfully tracked”. There were 88,351 convective storms in seven months? This seems quite large. Also the authors then state that there were 85,102 slight (or none) convective systems. What do you mean by none? You successfully tracked no convective systems? This paragraph needs rephrasing. Also consider calculating your statistical scores without the “none” storms.

Yes, 88,351 convective storms in seven months. The convective cloud traced by the tracking method is likely to produce precipitation. When the precipitation (from GPM) is less than 2.5mm/h, the convection cloud is considered to be a weak one (or none). Clouds with a precipitation of 0 mm/h should also be included. If these samples are deleted artificially, all the data selected here will not be fully representative of what actually happened in reality. Actually, all of cases tracked by our algorithm represent the real circumstance of convective storm in the area of interest in this study. We use the model trained by these matched samples to predict convective storm in nowcasting application.

9)      On that subject remove tables 5 and 6 and add the Wilks reference that contains these: “Wilks, Daniel S. (2006). Statistical Methods in the Atmospheric Sciences, Second Edition, Academic Press, London. ISBN 13: 978-0-12-751966-1.”

Agree. The original Table 6 describes the scores of each model, which cannot be deleted. Delete the original table 5 and add a formula to the text to describe these criteria.

10)  The authors state that they use GFS data. Do you use GFS forecasts or analysis? This needs to be cleared. Also the authors should comment on the capabilities of GFS in forecasting convective storms with a 0.5 degree resolution. There is a comment in the conclusions that a high resolution model will be better, but the authors should expand on this. Also would it be possible to run such a high resolution model and see the results in contrast with GFS? Even for one case it would be interesting. 

Thanks for your suggestion. We use GFS forecast data (one day ago) in our article. For nowcasting application, the result of the judgment needs to be updated every 10 minutes. Therefore, the GFS analysis data are not available for real-time application. In order to provide more timely information of convective storm to weather forecasters, the GFS data must be updated ahead of satellite observation. Therefore, we will use the GFS forecast data in this study. The high resolution GFS model may show a better effect, but this data is not available now.

11)  In Case-1 the SWIPE algorithm detects the storm 1hour and 23 minutes early. This seems good. In Case-2 however this is below an hour (54 minutes). Is this adequate? The authors should comment on whether or not these times are acceptable, for example for a Civil Protection Agency. 

The short-term forecast needs to issue an early warning about one hour before the precipitation occurs and explain to the public the time, place and intensity of the arrival of the precipitation. Therefore, one hour ahead is completely adequate. I add a reference to explain. The one-hour forecast interval for convective precipitation was also used during the Olympics. Therefore, we think one hour ahead is OK.

Sun J , Chen M , Wang Y . A Frequent-Updating Analysis System Based on Radar, Surface, and Mesoscale Model Data for the Beijing 2008 Forecast Demonstration Project[J]. Weather and Forecasting, 2010, 25(6):1715-1735.

12)  A paragraph should also be included that states the benefits on using this algorithm, over a conventional method (e.g. a high resolution NWP model). As I said this is a very interesting work and the benefits should be very clear to the reader.

Thanks for your suggestion. We have made the corresponding changes.

Author Response File: Author Response.doc

Round 2

Reviewer 2 Report

I am very disappointed to see that none of my concerns from the my first review have been addressed.  In fact it seams that the manuscript has become worse.  There is now no mathematics to justify any of the work that is presented, making it very difficult for me to recommend publishing.  It is still far too long winded, there is bad grammar throughout with incorrect tenses and wrong plurals being used just to name of couple of  the types of mistakes.  The caption on Figure 3 in very bad :move direction"!!!!

I can see from the change tracker that there has been no effort to reduce the size of the manuscript as well as bad paragraph structure throughout.

Please address my concerns from the first review, they still stand.

Reviewer 3 Report

All my comments have been addressed adequately. Therefore I suggest the manuscript be published in its present and revised form.