Next Article in Journal
Discrimination between Ground Vegetation and Small Pioneer Trees in the Boreal-Alpine Ecotone Using Intensity Metrics Derived from Airborne Laser Scanner Data
Previous Article in Journal
Integrating Crowdsourced Data with a Land Cover Product: A Bayesian Data Fusion Approach
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Object-Based Assessment of Satellite Precipitation Products

1
Department of Geosciences and Environment, California State University Los Angeles, Los Angeles, CA 90032, USA
2
Center for Hydrometeorology and Remote Sensing, Department of Civil and Environmental Engineering, University of California Irvine, Irvine, CA 92697, USA
*
Author to whom correspondence should be addressed.
Remote Sens. 2016, 8(7), 547; https://doi.org/10.3390/rs8070547
Submission received: 11 April 2016 / Revised: 13 June 2016 / Accepted: 22 June 2016 / Published: 27 June 2016

Abstract

:
An object-based verification approach is employed to assess the performance of the commonly used high-resolution satellite precipitation products: Precipitation Estimation from Remotely Sensed Information using Artificial Neural Networks (PERSIANN), Climate Prediction center MORPHing technique (CMORPH), and Tropical Rainfall Measurement Mission (TRMM) Multi-Satellite Precipitation Analysis (TMPA) 3B42RT. The evaluation of the satellite precipitation products focuses on the skill of depicting the geometric features of the localized precipitation areas. Seasonal variability of the performances of these products against the ground observations is investigated through the examples of warm and cold seasons. It is found that PERSIANN is capable of depicting the orientation of the localized precipitation areas in both seasons. CMORPH has the ability to capture the sizes of the localized precipitation areas and performs the best in the overall assessment for both seasons. 3B42RT is capable of depicting the location of the precipitation areas for both seasons. In addition, all of the products perform better on capturing the sizes and centroids of precipitation areas in the warm season than in the cold season, while they perform better on depicting the intersection area and orientation in the cold season than in the warm season. These products are more skillful on correctly detecting the localized precipitation areas against the observations in the warm season than in the cold season.

Graphical Abstract

1. Introduction

Precipitation is a crucial hydrologic variable that links the atmosphere with surface land processes. Ground-based instruments, such as rain gauges and ground radar, are commonly used to measure precipitation. Such instruments can provide reliable local precipitation measurements, which, however, have limited range and availability. With the advances in satellite technology, many remote-sensing algorithms have been developed to estimate precipitation at quasi-global scales. The satellite precipitation products provide precipitation estimations with high temporal and spatial resolutions that are applicable for watershed management, flood monitoring, and hydrologic modeling, such as Precipitation Estimation from Remotely Sensed Information using Artificial Neural Networks (PERSIANN) [1,2], Climate Prediction center MORPHing technique (CMORPH) [3], Tropical Rainfall Measuring Mission (TRMM) Multi-Satellite Precipitation Analysis (TMPA) [4], and Integrated Multi-satellitE Retrievals for Global Precipitation Measurement (IMERG) [5].
The satellite precipitation products combine the higher spatial and temporal resolution, but less accurate, infrared images with higher-accuracy, but coarser spatial and temporal resolution, passive microwave precipitation estimates in order to produce precipitation estimations. The quality of these products can vary by season and region depending on the combination methods. Therefore, many verification studies have been carried out to assess the accuracy of satellite precipitation estimations compared to the ground observations (e.g., [6,7,8,9,10,11,12,13]). These studies employed pixel-based verification measures, such as root mean square error, correlation coefficient, and/or categorical statistics [14] (such as false alarm ratio and probability of detection) to integrate the differences between the pixel values of satellite precipitation estimations and that of the ground observations. However, while these measures provide valuable information, the hydrology community requires useful diagnostic information about errors associated with location, spatial characteristics, and geometric patterns of precipitation, since such information is essential for hydrologic applications, especially in distributed hydrological modeling where the variation of the location and spatial patterns of precipitation events affect the modeled runoff volumes [15,16,17].
Meanwhile, spatial verification approaches have emerged to evaluate the precipitation estimations with a focus on the spatial structures and features characterizing precipitation fields (see [18,19,20] and references therein). One subgroup of spatial verification methods is feature-based/object-based approaches, where the spatial properties of precipitation objects obtained from precipitation estimations are verified by those from reference fields or ground observations (e.g., [21,22,23,24,25,26,27,28,29,30,31,32,33,34]). Li et al. [35] developed an object-based verification approach that allows the geometric and spatial evaluation of the separable local-scale precipitation areas. This object-based approach adopts the watershed transformation algorithm to delineate and identify the local-scale precipitation objects. Several key geometric attributes are utilized to evaluate the precipitation objects through a distance measurement. Three verification scores are generated to account for the geometric performance of the precipitation estimations.
A number of studies have adopted object-based approaches to examine the skills of satellite precipitation products. For example, Skok et al. [36] investigated the movement and spatial distribution of precipitation systems obtained from PERSIANN and TRMM 3B42 using the Method for Object-based Diagnostic Evaluation (MODE) [28] over the equatorial Pacific region. In addition, Demaria et al. [37] assessed multiple satellite precipitation products (PERSIANN, CMORPH, and TRMM) against rain gauge observations using the Contiguous Rain Area (CRA) method [21] at the La Plata river basin. The spatial errors associated with precipitation pattern, volume, and spatial location were revealed for three products. They found that CMORPH slightly underestimated the maximum rainfall rate but overestimated the average rainfall; PERSIANN overestimated the storm areas and underestimated average and maximum rainfall rate; TRMM overestimated the maximum rainfall and correlated rainfall volume and average rainfall rate well with ground observations. Furthermore, AghaKouchak et al. [38] developed three geometric indices (i.e., dispersiveness index, connectivity index, and shape index) to compare the precipitation patterns of CMORPH, TRMM, and PERSIANN against those of ground observations stage II with respect to different precipitation thresholds.
In contrast to previous studies listed above, our study evaluates the geometric and spatial features of the local-scale precipitation areas using the object-based verification method developed by Li et al. [35]. Most of the object-based approaches employ a threshold to group the pixels exceeding that threshold into objects [19]. Such thresholding techniques increase the difficulty in identifying the closely spaced but separate local-scale precipitation systems [35]. On the other hand, local-scale precipitation systems present different spatial and temporal features, such as shapes, sizes, orientation, growth/decay rates, and advection. Therefore, in this study, the object-based verification method [35], which can detect and separate local-scale precipitation systems, is used to perform the evaluation. A secondary goal of this study is to examine the seasonal variability of the performance of the satellite precipitation products based on their geometric features. It is worth mentioning that this study aims to provide advanced geometric evaluations of commonly used satellite precipitation products using this object-based verification method, while Li et al. [35] focus on introducing the procedures and details of this object-based verification method. The results of this study can shed some light on how well satellite precipitation products perform on geometrical characterization of precipitation fields at a local scale. It is hoped that hydrologists and/or meteorologists who are interested in the geometric performances of satellite precipitation products can obtain more insights from this evaluation work.

2. Methodology

2.1. Identification of Precipitation Object

The object-based verification approach employed in this study was developed by Li et al. [35] in order to verify to what extent the precipitation estimations depict the geometric and spatial features of the ground observation. The object-based approach uses watershed transformation [39] as the primary algorithm to identify the local-scale precipitation objects. This algorithm partitions precipitation intensity images into segmented precipitation areas based on storm centers, which has the capability to detect and separate the closely spaced localized precipitation areas in the precipitation field. The watershed transformation algorithm is based on topographical distance. A precipitation intensity image is interpreted as a topographic surface, where the value of each pixel represents the altitude at that point. The algorithm locates the storm centers (i.e., local maxima) in the image and associates an object with each of the storm centers. A precipitating pixel is assigned to an object based on the shortest topographic distance between this pixel and the storm center.
To avoid the over-segmentation in the precipitation intensity images, the H-minima transform technique is adopted to suppress the minor fluctuations prior to watershed segmentation. The minor fluctuations are the ones whose magnitudes are less than or equal to a predefined merge threshold. They are filled/suppressed via adjusting the precipitation intensities with the merge threshold. It should be noted that the selection of a merge threshold depends on the scale where validation information is required. A relatively higher merge threshold tends to result in large-scale precipitation objects, which could be considered by a user who is interested in large-scale evaluation of precipitation fields. A relatively lower merge threshold is prone to small-scale precipitation objects, which could be implemented by a user who is interested in a more detailed pattern analysis of precipitation fields. Thus, users can determine a merge threshold based on their particular applications. In this study, a merge threshold of 15 mm/day is used for a local-scale evaluation of satellite precipitation products, based on the results shown in Li et al. [35]. The same merge threshold is applied on the estimated precipitation fields and observed precipitation fields.

2.2. Evaluation of Precipitation Object

After the precipitation objects are identified and delineated, they are categorized into unmatching objects or matching objects according to overlapping criteria. Every observed object searches for a matching estimated object by examining the overlap between the two objects. These two objects are counted as matching objects if they are partially or fully overlapped. Otherwise, they are unmatching objects. For the unmatching objects, an object-based missing ratio and an object-based false alarm ratio is computed to account for the missed objects proportion in the observation field and falsely detected objects proportion in the satellite precipitation field, respectively. The object-based missing ratio is defined as the ratio of missing objects area to the total area of observed objects, while the object-based false alarm ratio is defined as the ratio of false alarm objects area to the total area of estimated objects [35]:
r m i s s = A m i s s A o b s
r f a = A f a A e s t
where r m i s s and r f a denote object-based missing ratio and false alarm ratio, respectively. A m i s s is the area coverage of a missing object in the observed field. A o b s is the area coverage of an observed object. A f a is the area coverage of a false alarm object in the estimated field. A e s t is the area coverage of an estimated object.
With regard to the matching objects, four geometric attributes, including object area ratio, centroid location distance, orientation difference, and intersection area ratio, are selected to evaluate the estimated objects against the observed objects. The object area ratio (ar) computes the ratio of estimated object area to observed object area. The centroid location distance (cd) computes the Euclidean distance between the centroid of the estimated object and the centroid of the observed object. The orientation difference (od) calculates the orientation angle difference between the matching objects. The orientation angle of an object is derived by fitting an eclipse to this object. The intersection area ratio (iar) calculates the ratio of overlapping area to the smaller object area. For each pair of matching objects, the four attributes are defined as [35]:
a r = A e s t A o b s
c d = ( x o b s x e s t ) 2 + ( y o b s y e s t ) 2
o d = {   | δ o b s δ e s t | i f   | δ o b s δ e s t | 90   180 | δ o b s δ e s t | i f   | δ o b s δ e s t | > 90
i a r = {   A o v e r l a p A o b s i f   A o b s A e s t   A o v e r l a p A e s t i f   A o b s > A e s t
where A e s t and A o b s are the area coverage of the matching estimated and observed object, respectively. ( x e s t , y e s t ) and ( x o b s , y o b s ) are the coordinates of the geometric centroids of the matching estimated and observed object, respectively. δ e s t and δ o b s are the orientation angles of the matching estimated and observed object, respectively. A o v e r l a p is the overlapping area between the matching estimated and observed objects.
To obtain commensurable measures, these attributes are converted to interest values that range from 0 to 1 through different interest functions. The interest value of 1 represents a perfect performance of the estimated object for a specific attribute, while the value of 0 represents no skill. An overall interest score is summarized using the interest values and assigned weights, which reveals the quality of the satellite precipitation product in an aggregated sense. The same weight of 25% is assigned to each attribute in this study. Users can adjust the weights according to their particular applications. The overall interest score tends to be high for the matching objects whose size difference is small, centroid distance is small, orientation angle difference is small, and intersection area ratio is large. Whereas, the overall interest score tends to be low for the matching objects whose size difference is large, centroid distance is large, orientation angle difference is large, and intersection area ratio is small. The geometric attributes and verification scores were examined to be stable from the previous application [35]. For a more detailed explanation of this object-based verification approach, the reader is referred to Li et al. [35].

3. Data

The three satellite precipitation products employed in this study are: (1) PERSIANN [1,2]; (2) CMORPH [3]; and (3) TMPA 3B42RT [4]. These three satellite precipitation products are not corrected by gauge data. PERSIANN uses an artificial neural networks approach to estimate the rainfall rate based on the geostationary longwave infrared imagery. The network parameters are adjusted using an adaptive training feature. PERSIANN data is available at 0.25° by 0.25° spatial resolution every 1-h. CMORPH utilizes the infrared geostationary satellite imagery to obtain advection vectors between 3-h passive microwave (PMW) satellite scans. This advection information is used to determine the half-hour precipitation estimates based on the precipitation estimates derived from PMW imagery. Furthermore, these half-hour precipitation estimates are “morphed” by applying the time weighting factors. CMORPH data has a time resolution of 3-h and a spatial resolution of 0.25° by 0.25°. TMPA 3B42RT combines the most recent PMW imageries to derive the precipitation estimates. To fill in the coverage gaps between the PMW imageries, microwave-calibrated infrared precipitation estimates are used. 3B42RT is available at 0.25° by 0.25° spatial resolution every 3-h.
National Oceanic and Atmospheric Administration (NOAA) stage IV multi-sensor composite rain analysis [40] is utilized as the ground-based reference. The stage IV analysis is derived from the rain gauge and regional radar precipitation measurements. These measurements are produced by the National Weather Service River Forecast Centers over the Continental United States (CONUS). It is available at a temporal resolution of 1-h and a spatial resolution of 4 km. All of the satellite precipitation products and the stage IV data were converted to a spatial resolution of 0.25° by 0.25° and a temporal resolution of 24-h prior to the analysis. A mean pixel value is computed for a 0.25° by 0.25° pixel that covers the 4 km grid stage IV data, in order to obtain the stage IV data at a 0.25° spatial resolution. All of the data are aggregated to 24-h/daily accumulated precipitation in this study.

4. Results

The object-based method is applied on three satellite precipitation products against ground references for the warm and cold seasons. The precipitation objects are identified and delineated for each product and ground observation. The performance of each product for depicting the geometric and spatial features of the precipitation systems is evaluated and compared. The evaluations are conducted at 0.25° by 0.25° on a daily scale in the warm season of 2008 (i.e., 1 June to 31 August) and cold season of 2010 (i.e., 1 October to 30 November) over CONUS. It is worth noting that October and November are selected as the cold season in order to minimize, and hopefully eliminate, the effect of snowfall and/or snow-covered land surface which could affect the evaluation results. This is mainly due to the limitations in retrieving precipitation with passive microwave when the land surface is covered by snow or ice [41].

4.1. Identification of Precipitation Object

Local-scale precipitation objects are identified using the object-based approach for each satellite precipitation product and ground observations. Figure 1 shows a warm season example on 23 July 2008, over CONUS for the stage IV observation and three satellite precipitation products. Figure 1a,c,e,g exhibits the daily precipitation distribution of the stage IV, PERSIANN, CMORPH, and 3B42RT on the sample day. Figure 1b,d,f,h illustrates the precipitation objects derived from the stage IV observation and three satellite precipitation products using the watershed transformation. Figure 2 shows a cold season example on 27 October 2010, over CONUS. Figure 2a,c,e,g exhibits the daily precipitation distribution of the stage IV and three satellite precipitation products; meanwhile, Figure 2b,d,f,h illustrates the correspondent precipitation objects. In both examples of warm and cold seasons, the segmented precipitation objects (shown in the right panels of Figure 1 and Figure 2) correspond well with the closely spaced but separate local-scale precipitation areas (shown in the left panels of Figure 1 and Figure 2) for the stage IV observations and three satellite precipitation products. The precipitation objects delineate the storm systems that a relevant expert would possibly observe from the precipitation fields. The distinct precipitation objects represent distinct precipitation areas produced from convective storms in the warm season and frontal storms in the cold season. The spatial distribution of precipitation objects depicts the structure of accumulated precipitation at the local scale for the stage IV observation and each satellite precipitation product. Different satellite precipitation products display different patterns of accumulated precipitation at the local scale. For instance, in both seasons, PERSIANN and CMORPH tend to generate larger precipitation objects in size than 3B42RT. In general, there are more false alarm objects identified in the three products in the cold season example than the warm season example, which indicates that the products tend to perform better on correctly detecting the precipitation events in the warm season example than the cold season example. The cold-season false precipitation could be associated with non-precipitating clouds that falsely appear as rain especially in cold months, as shown in Nasrollahi et al. [42]. Such false precipitation could also be caused by ground contaminations especially for 3B42RT and CMORPH [41,43,44]. Additionally, the precipitation objects shown in the cold season example are prone to be larger than those in the warm season example for the stage IV observation and each satellite precipitation product. This is because different precipitation systems are presented in the cold and warm seasons. The cold season is dominated by frontal systems in which storms usually have larger area coverage, while the warm season is dominated by convective systems in which storms usually have smaller area coverage. This implies that the three satellite precipitation products are capable of depicting such spatial characteristics of different precipitation systems in different seasons. Challenges associated with cold-season precipitation have been discussed in previous studies (e.g., [45]).

4.2. Evaluation of Precipitation Object

After the precipitation objects are identified in both the stage IV observation and three satellite precipitation products, the objects acquired from the products are verified by the objects acquired from the stage IV. Precipitation objects were grouped into matching or unmatching objects. Four key geometric attributes (i.e., area ratio, centroid distance, orientation difference, and intersection area ratio) are examined for matching objects.

4.2.1. Area Ratio

Figure 3a–c illustrates the normalized histograms of the area ratio in the warm season of 2008 for PERSIANN, CMORPH, and 3B42RT. The area ratio is the ratio of satellite estimated object area to ground reference observed object area. The peak of the histogram for CMORPH is around the value of 1, while 3B42RT’s peak is less than the value of 1, and PERSIANN’s peak is more than the value of 1. As seen in the histogram distributions, CMORPH tends to generate the precipitation objects closest in size to the stage IV observations, while 3B42RT tends to generate smaller size precipitation objects, and PERSIANN is prone to produce larger size precipitation objects than the stage IV observations. This is perhaps due to the fact that PERSIANN tends to produce larger rain area than the stage IV in temporal scale, which leads to a larger coverage of rain area after accumulation of precipitation to daily scale. 3B42RT, on the other hand, uses infrared information when there is missing microwave information and relies less on infrared. These algorithmic features are well detected in this object-based analysis. The mean interest values of the area ratio for the three products are illustrated in Table 1. CMORPH possesses the highest mean interest value among others, which indicates that CMORPH is capable of generating the storm objects closest in size compared with the stage IV.
Figure 3d–f shows the normalized histograms of the area ratio in the cold season of 2010 for PERSIANN, CMORPH, and 3B42RT. Similar to the warm season case, CMORPH has the peak around the value of 1, while 3B42RT’s peak is less than the value of 1, and PERSIANN’s peak is more than the value of 1. In general, CMORPH has a tendency to produce the storm objects close in size to stage IV, while 3B42RT tends to underestimate the object sizes, and PERSIANN tends to overestimate the object sizes. The mean interest values of the area ratio for the three products are illustrated in Table 2 for the cold season. Same as the warm season case, CMORPH obtains the highest mean interest value among the others. The three products are all prone to lower mean interest values for the area ratios in the cold season than the warm season. This indicates that the satellite precipitation products are more capable of accurately capturing the storm object sizes in the warm season than the cold season.

4.2.2. Centroid Distance

Figure 4a–c shows the normalized histograms of the centroid distance in the warm season of 2008 for PERSIANN, CMORPH, and 3B42RT. In general, as the centroid distance increases, the number of object pairs declines. 3B42RT tends to obtain smaller centroid distances against the stage IV observations, whereas PERSIANN and CMORPH obtain larger centroid distances. The mean interest values of the centroid distances are listed in Table 1 for the three products. On average, 3B42RT possesses the highest interest value of the centroid distances. This implies that 3B42RT has the capability of placing the storm objects in the correct location compared to the stage IV.
Figure 4d–f shows the normalized histograms of the centroid distances in the cold season for PERSIANN, CMORPH, and 3B42RT. The overall trend of the centroid distance histograms for each product is similar to that of the warm season case. The three products’ mean interest values are listed in Table 2. On average, 3B42RT possesses the highest interest value of centroid distances. This is the same result as the warm season case, indicating that the object centroids can be successfully captured by 3B42RT in both seasons. It is also noted that three products acquire higher interest values of centroid distances in warm season than in cold season. This implies that the products perform better on depicting the object centroids in warm season.

4.2.3. Intersection Area Ratio

Figure 5a–c shows the normalized histograms of the intersection area ratio in the warm season for PERSIANN, CMORPH, and 3B42RT. The number of matching object pairs is distributed quite evenly in the area ratio range for each product. Also, a high peak is observed at the value of 1 for each product. This indicates that a large number of storm objects derived from each product perfectly overlap with the matching storm objects derived from the stage IV observation. The mean interest values of intersection area ratio are illustrated in Table 1 for the three products. On average, PERSIANN shows the highest mean interest value of intersection area ratio. This reveals that PERSIANN behaves the best with respect to the intersection area ratio in the warm season.
Figure 5d–f demonstrates the normalized histograms of the intersection area ratio in the cold season for PERSIANN, CMORPH, and 3B42RT. High peaks are continually noted at the value of one for each product. Each product has a tendency to obtain higher mean interest value of intersection area ratios in the cold season than in the warm season (see Table 1 and Table 2), which demonstrates that each product performs better in the cold season with respect to the intersection area ratio. PERSIANN is prone to the highest interest values of the intersection area ratio; which is the same result as the warm season case.

4.2.4. Orientation Difference

Figure 6a–c exhibits the normalized histograms of the orientation difference in the warm season for PERSIANN, CMORPH, and 3B42RT. It is observed that the histogram trends are similar for each product. As the orientation difference increases, the number of object pairs decreases. The mean interest values of the orientation difference are illustrated in Table 1 for the three products. PERSIANN tends to obtain the highest mean orientation difference interest value. This demonstrates that PERSIANN is capable of correctly depicting the orientation of the storm objects compared to the stage IV.
Figure 6d–f illustrates the normalized histograms of the orientation difference in the cold season for PERSIANN, CMORPH, and 3B42RT. The histogram trends are similar for each product. On average, CMORPH obtains the smallest orientation difference, which leads to its best performance on the mean interest value of orientation difference in the cold season (see Table 2). All the products are prone to higher mean interest values of orientation differences in the cold season compared to the warm season. This indicates that the object orientation can be better captured by the three products in the cold season.

4.2.5. Overall Performances

Three verification scores, namely the overall interest score, object-based missing ratio, and object-based false alarm ratio, are compared among CMORPH, PERSIANN, and 3B42RT. The time series of these verification scores for the warm season of 2008 are illustrated in Figure 7 and the mean of these scores for the three products are shown in Table 3. In general, all the products are prone to low object-based missing ratios and false alarm ratios, which indicates that all the products have the capability of accurately detecting the storm systems in the warm season. As seen in Table 3, CMORPH possesses the lowest missing ratio, while PERSIANN obtains the lowest false alarm ratio. For the overall interest score shown in Figure 7c, 3B42RT and CMORPH perform similarly, while PERSIANN has a discrepancy with these two products. It should be noted that the overall interest score summarizes the performances of four geometric attributes, which provides the skill of the satellite precipitation products in an aggregated sense. CMORPH obtains the highest overall interest value, thereby showing that this product performs well overall on depicting the geometric features of the precipitation pattern at the local scale in the warm season. This is perhaps due to the advection procedures used in CMORPH that allows the well depiction of the geometric features of precipitation systems.
Figure 8 exhibits the time series of the object-based missing ratio, false alarm ratio, and overall interest score for the three satellite precipitation products in the cold season of 2010. All of the products obtain higher object-based missing ratios and false alarm ratios in the cold season than the warm season. This implies that all products fail to correctly detect the storm systems in the cold season against the stage IV observations. As seen in Table 4, 3B42RT obtains the lowest missing ratio, and CMORPH obtains the lowest false alarm ratio. For the overall membership score shown in Figure 8c, three products performed similarly in October, while PERSIANN had a discrepancy with the other two in November. CMORPH continues to obtain the highest overall interest score, which indicates that CMORPH performs the best on depicting the geometric features of the precipitation pattern at the local scale against the observation in the cold season.

5. Summary and Conclusions

The abilities of three satellite precipitation products to depict the geometric and spatial features of local-scale precipitation areas are examined by an object-based verification approach. This approach allows an object-to-object evaluation in the local-scale to reveal the geometric performances of satellite products against the ground observations, whereas such verification is difficult to be achieved by pixel-to-pixel evaluation techniques. The three satellite precipitation products, including PERSIANN, CMORPH, and 3B42RT, are tested and compared with the stage IV observations in the warm season of 2008 and cold season of 2010. The local-scale precipitation objects are identified for each product and observations via watershed transformation and are evaluated using the key geometric attributes and a distance measurement. The main conclusions of this study are as follows:
1. Overall, precipitation object area and object centroids are better depicted by all of these products in the warm season than in the cold season. Meanwhile, the intersection area ratio and object orientation are better captured by these products in the cold season than in the warm season. These three products performed better with respect to object-based missing ratio and false alarm ratio in the warm season than in the cold season, which indicates that they are more capable of correctly detecting the local-scale precipitation area against the observations in the warm season. Therefore, the geometric performances of satellite precipitation products vary for different seasons.
2. PERSIANN performed the best with respect to intersection area ratios for both seasons and performed the best on orientation differences in the warm season. This implies that PERSIANN has the capability of capturing the location and orientation of the storm systems. However, PERSIANN tends to overestimate the precipitation object sizes for both seasons. This is because PERSIANN is prone to produce larger rain area in the temporal scale that leads to the tendency of a larger coverage of rain area after accumulating to the daily scale. In addition, PERSIANN performs well in object-based missing ratios and false alarm ratios in both seasons, which demonstrates that PERSIANN has the capability to correctly detect the precipitation objects against the observation.
3. CMORPH performed the best in regards to the area ratio for both seasons, showing that it has the ability to depict the accurate precipitation area compared to observations. In the cold season, the object orientation was well captured by CMORPH, indicating that CMORPH is capable of capturing the correct orientation of the storm systems. Furthermore, CMORPH performed the best with respect to the object-based missing ratio in the warm season and object-based false alarm ratio in the cold season. This implies its ability to correctly detect the storm systems against the observations in both seasons. For the overall performance, CMORPH obtained the highest overall interest score in both seasons, which demonstrates that the “morphing” technique used can successfully depict the geometric and spatial features of precipitation patterns at the local scale.
4. 3B42RT performed the best in regards to depicting the object centroids for both seasons, indicating that 3B42RT has the capability of placing the storm systems at the correct locations. In contrast with the other products, 3B42RT tends to underestimate the precipitation object sizes for both seasons. Moreover, 3B42RT performed the best in regards to the object-based missing ratio in the cold season, whereas it falsely detected a number of precipitation objects in the warm season. This demonstrates that 3B42RT performed differently in detecting the storm systems in different seasons. Finally, the overall membership scores for 3B42RT were moderate in both seasons.
It is concluded that the discrepancies between various satellite precipitation products in depicting the geometric characteristics of precipitation can be identified through the object-based verification approach. The results of this study are valid for the evaluation of local-scale precipitation areas identified from a 24-h accumulated precipitation field. The geometric capability of satellite precipitation products is evaluated by the four attribute measures and three overall scores. It is worth pointing out that the presented object-based evaluation study provides the means to reveal the potential deficiencies in satellite precipitation products with respect to observations from a geometrical point of view. It is hoped that further advancements in this area will provide future development of precipitation products with more insights on geometric characterization.

Acknowledgments

The authors would like to acknowledge Dan Braithwaite for his help with the data preparation. The authors would also like to thank the reviewers for their constructive comments and suggestions, which led to the significant improvement in this work. This study was supported by the NASA MIRO grant (NNX15AQ06A).

Author Contributions

Jingjing Li collected the data, performed the analysis, and wrote the paper. Kuo-Lin Hsu, Amir AghaKouchak, and Soroosh Sorooshian provided suggestions and edited the paper.

Conflicts of Interest

The authors declare no conflict of interest.

Abbreviations

The following abbreviations are used in this manuscript:
CONUScontinental United States
CMORPHClimate Prediction center MORPHing technique
CRAContiguous Rain Area
MODEMethod for Object-based Diagnostic Evaluation
PERSIANNPrecipitation Estimation from Remotely Sensed Information using Artificial Neural Networks
TRMMTropical Rainfall Measurement Mission
TMPATRMM Multi-Satellite Precipitation Analysis
PMWpassive microwave

References

  1. Hsu, K.; Gao, X.; Sorooshian, S.; Gupta, H.V. Precipitation estimation from remotely sensed information using artificial neural networks. J. Appl. Meteorol. 1997, 36, 1176–1190. [Google Scholar] [CrossRef]
  2. Sorooshian, S.; Hsu, K.; Gao, X.; Gupta, H.V.; Imam, B.; Braithwaite, D. Evaluation of PERSIANN system satellite-based estimates of tropical rainfall. Bull. Am. Meteorol. Soc. 2000, 81, 2035–2046. [Google Scholar] [CrossRef]
  3. Joyce, R.J.; Janowiak, J.E.; Arkin, P.A.; Xie, P. CMORPH: A method that produces global precipitation estimates from passive microwave and infrared data at high spatial and temporal resolution. J. Hydrometeorol. 2004, 5, 487–503. [Google Scholar] [CrossRef]
  4. Huffman, G.J.; Adler, R.F.; Bolvin, D.T.; Gu, G.; Nelkin, E.J.; Bowman, K.P.; Hong, Y.; Stocker, E.F.; Wolff, D.B. The TRMM Multisatellite Precipitation Analysis (TMPA): Quasi-global, multiyear, combined-sensor precipitation estimates at fine scales. J. Hydrometeorol. 2007, 8, 38–55. [Google Scholar] [CrossRef]
  5. Huffman, G.J.; Bolvin, D.T.; Braithwaite, D.; Hsu, K.; Joyce, R.; Xie, P. Algorithm Theoretical Basis Document (ATBD) Version 4.4 for the NASA Global Precipitation Measurement (GPM) Integrated Multi-Satellite Retrievals for GPM (I-MERG); GPM Project; NASA: Greenbelt, MD, USA, 2014.
  6. AghaKouchak, A.; Behrangi, A.; Sorooshian, S.; Hsu, K.; Amitai, E. Evaluation of satellite-retrieved extreme precipitation rates across the central United States. J. Geophys. Res. 2011, 116. [Google Scholar] [CrossRef]
  7. AghaKouchak, A.; Mehran, A.; Norouzi, H.; Behrangi, A. Systematic and random error components in satellite precipitation data sets. Geophys. Res. Lett. 2012, 39. [Google Scholar] [CrossRef]
  8. Arkin, P.; Turk, J. Program to Evaluate High Resolution Precipitation Products (PEHRPP): A contribution to GPM planning. In Proceedings of the 6th GPM International Planning Workshop, Annapolis, MD, USA, 6–8 November 2006.
  9. Ebert, E.E.; Janowiak, J.E.; Kidd, C. Comparison of near-realtime precipitation estimates from satellite observations and numerical models. Bull. Am. Meteorol. Soc. 2007, 88, 47–64. [Google Scholar] [CrossRef]
  10. Sapiano, M.R.P.; Arkin, P.A. An intercomparison and validation of high-resolution satellite precipitation estimates with 3-hourly gauge data. J. Hydrometeorol. 2009, 10, 149–166. [Google Scholar] [CrossRef]
  11. Tian, Y.; Peters-Lidard, C.D.; Eylander, J.B.; Joyce, R.J.; Huffman, G.J.; Adler, R.F.; Hsu, K.; Turk, F.J.; Garcia, M.; Zeng, J. Component analysis of errors in satellite-based precipitation estimates. J. Geophys. Res. 2009, 114. [Google Scholar] [CrossRef]
  12. Kidd, C.; Bauer, P.; Turk, J.; Huffman, G.J.; Joyce, R.; Hsu, K.-L.; Braithwaite, D. Intercomparison of high-resolution precipitation products over northwest Europe. J. Hydrometeorol. 2012, 13, 67–83. [Google Scholar] [CrossRef]
  13. Prakash, S.; Mitra, A.K.; AghaKouchak, A.; Pai, D.S. Error characterization of TRMM Multisatellite Precipitation Analysis (TMPA-3B42) products over India for different seasons. J. Hydrol. 2015, 529, 1302–1312. [Google Scholar] [CrossRef]
  14. Wilks, D.S. Statistical Methods in the Atmospheric Sciences; Academic: Amsterdam, The Netherlands, 2011. [Google Scholar]
  15. Nunes, J.P.; de Lima, J.L.M.P.; Singh, V.P.; de Lima, M.I.P.; Vieira, G.N. Numerical modeling of surface runoff and erosion due to moving rainstorms and the drainage basin scale. J. Hydrol. 2006, 330, 709–720. [Google Scholar] [CrossRef]
  16. Foufoula-Georgiou, E.; Vuruputur, V. Patterns and organization in precipitation. In Spatial Patterns in Catchment Hydrology: Observations and Modeling; Grayson, R., Blöschl, G., Eds.; Cambridge University Press: Cambridge, UK, 2001; pp. 82–104. [Google Scholar]
  17. Nguyen, P.; Thorstensen, A.; Sorooshian, S.; Hsu, K.; AghaKouchak, A. Flood forecasting and inundation mapping using HiResFlood-UCI and near real-time satellite precipitation data: The 2008 Iowa flood. J. Hydrometeorol. 2015, 16, 1171–1183. [Google Scholar] [CrossRef]
  18. Ahijevych, D.; Gilleland, E.; Brown, B.; Ebert, E. Application of spatial verification methods to idealized and NWP gridded precipitation forecasts. Weather Forecast. 2009, 24, 1485–1497. [Google Scholar] [CrossRef]
  19. Gilleland, E.; Ahijevych, D.; Brown, B.G.; Casati, B.; Ebert, E.E. Intercomparison of spatial forecast verification methods. Weather Forecast. 2009, 24, 1416–1430. [Google Scholar] [CrossRef]
  20. Gilleland, E.; Ahijevych, D.B.; Brown, G.; Ebert, E.E. Verifying forecasts spatially. Bull. Am. Meteorol. Soc. 2010, 91, 1365–1373. [Google Scholar] [CrossRef]
  21. Ebert, E.E.; McBride, J.L. Verification of precipitation in weather systems: Determination of systematic errors. J. Hydrol. 2000, 239, 179–202. [Google Scholar] [CrossRef]
  22. Ebert, E.E.; Gallus, W.A. Toward better understanding of the Contiguous Rain Area (CRA) method for spatial forecast verification. Weather Forecast. 2009, 24, 1401–1415. [Google Scholar] [CrossRef]
  23. Nachamkin, J.E. Mesoscale verification using meteorological composites. Mon. Weather. Rev. 2004, 132, 941–955. [Google Scholar] [CrossRef]
  24. Nachamkin, J.E. Application of the composite method to the spatial forecast verification methods intercomparison dataset. Weather Forecast. 2009, 24, 1390–1400. [Google Scholar] [CrossRef]
  25. Nachamkin, J.E.; Chen, S.; Schmidt, J. Evaluation of heavy precipitation forecasts using composite-based methods: A distributions-oriented approach. Mon. Weather Rev. 2005, 133, 2163–2177. [Google Scholar] [CrossRef]
  26. Davis, C.A.; Brown, B.; Bullock, R. Object-based verification of precipitation forecasts. Part I: Methodology and application to mesoscale rain areas. Mon. Weather Rev. 2006, 134, 1772–1784. [Google Scholar] [CrossRef]
  27. Davis, C.A.; Brown, B.; Bullock, R. Object-based verification of precipitation forecasts. Part II: Application to convective rain systems. Mon. Weather Rev. 2006, 134, 1785–1795. [Google Scholar] [CrossRef]
  28. Davis, C.A.; Brown, B.; Bullock, R.; Halley-Gotway, J. The method for object-based diagnostic evaluation (MODE) applied to numerical forecasts from the 2005 NSSL/SPC Spring Program. Weather Forecast. 2009, 24, 1252–1267. [Google Scholar] [CrossRef]
  29. Wernli, H.; Paulat, M.; Hagen, M.; Frei, C. SAL—A novel quality measure for the verification of quantitative precipitation forecasts. Month. Weather Rev. 2008, 136, 4470–4487. [Google Scholar] [CrossRef]
  30. Wernli, H.; Hofmann, C.; Zimmer, M. Spatial forecast verification methods intercomparison project: Application of the SAL technique. Weather Forecast. 2009, 24, 1472–1484. [Google Scholar] [CrossRef]
  31. Marzban, C.; Sandgathe, S. Cluster analysis for verification of precipitation fields. Weather Forecast. 2006, 21, 824–838. [Google Scholar] [CrossRef]
  32. Marzban, C.; Sandgathe, S. Cluster analysis for object-oriented verification of fields: A variation. Mon. Weather Rev. 2008, 136, 1013–1025. [Google Scholar] [CrossRef]
  33. Micheas, A.C.; Fox, N.I.; Lack, S.A.; Wikle, C.K. Cell identification and verification of QPF ensembles using shape analysis techniques. J. Hydrol. 2007, 343, 105–116. [Google Scholar] [CrossRef]
  34. Lack, S.; Limpert, G.L.; Fox, N.I. An object-oriented multiscale verification scheme. Weather Forecast. 2010, 25, 79–92. [Google Scholar] [CrossRef]
  35. Li, J.; Hsu, K.; AghaKouchak, A.; Sorooshian, S. An object-based approach for verification of precipitation estimation. Int. J. Remote Sens. 2015, 36, 513–529. [Google Scholar] [CrossRef]
  36. Skok, G.; Tribbia, J.; Rakovec, J.; Brown, B. Object-based analysis of satellite-derived precipitation systems over the low- and midlatitude Pacific Ocean. Mon. Weather Rev. 2009, 137, 3196–3218. [Google Scholar] [CrossRef]
  37. Demaria, E.M.C.; Rodriguez, D.A.; Ebert, E.E.; Salio, P.; Su, F.; Valdes, J.B. Evaluation of mesoscale convective systems in South America using multiple satellite products and an object-based approach. J. Geophys. Res. 2011, 116. [Google Scholar] [CrossRef]
  38. AghaKouchak, A.; Nasrollahi, N.; Li, J.; Imam, B.; Sorooshian, S. Geometrical characterization of precipitation patterns. J. Hydrometeorol. 2011, 12, 274–285. [Google Scholar] [CrossRef]
  39. Meyer, F. Topographic distance and watershed lines. Signal Process. 1994, 38, 113–125. [Google Scholar] [CrossRef]
  40. Lin, Y.; Mitchell, K.E. The NCEP Stage II/IV hourly precipitation analyses: Development and applications. In Proceedings of the 19th Conference on Hydrology, San Diego, CA, USA, 9–13 January 2005.
  41. Ferraro, R.R.; Smith, E.A.; Berg, W.; Huffman, G.J. A screening methodology for passive microwave precipitation retrieval algorithms. J. Atmos. Sci. 1998, 55, 1583–1600. [Google Scholar] [CrossRef]
  42. Nasrollahi, N.; Hsu, K.; Sorooshian, S. An artificial neural network model to reduce false alarms in satellite precipitation products using MODIS and CloudSat observations. J. Hydrometeorol. 2013, 14, 1872–1883. [Google Scholar] [CrossRef]
  43. Gopalan, K.; Wang, N.; Ferraro, R.; Liu, C. Status of the TRMM 2A12 land precipitation algorithm. J. Atmos. Ocean. Technol. 2010, 27, 1343–1354. [Google Scholar] [CrossRef]
  44. You, Y.; Wang, N.; Ferraro, R.; Meyers, P. A prototype precipitation retrieval algorithm over land for ATMS. J. Hydrometeorol. 2016, 17, 1601–1621. [Google Scholar] [CrossRef]
  45. Sorooshian, S.; AghaKouchak, A.; Arkin, P.; Eylander, J.; FoufoulaGeorgiou, E.; Harmon, R.; Hendrickx, J.M.H.; Imam, B.; Kuligowski, R.; Skahill, B.; et al. Advanced concepts on remote sensing of precipitation at multiple scales. Bull. Am. Meteorol. Soc. 2011, 92, 1353–1357. [Google Scholar] [CrossRef]
Figure 1. The left panels exhibit the precipitation distribution (mm/day) acquired from the observation and three satellite precipitation products on 23 July 2008 at 0.25° by 0.25° over the Continental United States (CONUS). The precipitation intensity lower than a threshold of 1mm/day is represented by white background. Gray area indicates no data. The right panels exhibit the distribution of local-scale precipitation objects that is delineated from precipitation field shown in the left panel. Distinct precipitation objects are represented by distinct colors. (a) Stage IV precipitation distribution; (b) Stage IV precipitation objects distribution; (c) Precipitation Estimation from Remotely Sensed Information using Artificial Neural Networks (PERSIANN) precipitation distribution; (d) PERSIANN precipitation objects distribution; (e) Climate Prediction center MORPHing technique (CMORPH) precipitation distribution; (f) CMORPH precipitation objects distribution; (g) 3B42RT precipitation distribution; (h) 3B42RT precipitation objects distribution.
Figure 1. The left panels exhibit the precipitation distribution (mm/day) acquired from the observation and three satellite precipitation products on 23 July 2008 at 0.25° by 0.25° over the Continental United States (CONUS). The precipitation intensity lower than a threshold of 1mm/day is represented by white background. Gray area indicates no data. The right panels exhibit the distribution of local-scale precipitation objects that is delineated from precipitation field shown in the left panel. Distinct precipitation objects are represented by distinct colors. (a) Stage IV precipitation distribution; (b) Stage IV precipitation objects distribution; (c) Precipitation Estimation from Remotely Sensed Information using Artificial Neural Networks (PERSIANN) precipitation distribution; (d) PERSIANN precipitation objects distribution; (e) Climate Prediction center MORPHing technique (CMORPH) precipitation distribution; (f) CMORPH precipitation objects distribution; (g) 3B42RT precipitation distribution; (h) 3B42RT precipitation objects distribution.
Remotesensing 08 00547 g001aRemotesensing 08 00547 g001b
Figure 2. The left panels exhibit the precipitation distribution (mm/day) acquired from the observation and three satellite precipitation products on 27 October 2010 at 0.25° by 0.25° over CONUS. The precipitation intensity lower than a threshold of 1mm/day is represented by white background. Gray area indicates no data. The right panels exhibit the distribution of local-scale precipitation objects that is delineated from precipitation field shown in the left panel. Distinct precipitation objects are represented by distinct colors. (a) Stage IV precipitation distribution; (b) Stage IV precipitation objects distribution; (c) PERSIANN precipitation distribution; (d) PERSIANN precipitation objects distribution; (e) CMORPH precipitation distribution; (f) CMORPH precipitation objects distribution; (g) 3B42RT precipitation distribution; (h) 3B42RT precipitation objects distribution.
Figure 2. The left panels exhibit the precipitation distribution (mm/day) acquired from the observation and three satellite precipitation products on 27 October 2010 at 0.25° by 0.25° over CONUS. The precipitation intensity lower than a threshold of 1mm/day is represented by white background. Gray area indicates no data. The right panels exhibit the distribution of local-scale precipitation objects that is delineated from precipitation field shown in the left panel. Distinct precipitation objects are represented by distinct colors. (a) Stage IV precipitation distribution; (b) Stage IV precipitation objects distribution; (c) PERSIANN precipitation distribution; (d) PERSIANN precipitation objects distribution; (e) CMORPH precipitation distribution; (f) CMORPH precipitation objects distribution; (g) 3B42RT precipitation distribution; (h) 3B42RT precipitation objects distribution.
Remotesensing 08 00547 g002aRemotesensing 08 00547 g002b
Figure 3. (ac) describes normalized histograms of area ratio for satellite precipitation products PERSIANN, CMORPH, and 3B42RT in the warm season of 2008, respectively; (df) describes normalized histograms of area ratio for satellite precipitation products PERSIANN, CMORPH, and 3B42RT in the cold season of 2010, respectively.
Figure 3. (ac) describes normalized histograms of area ratio for satellite precipitation products PERSIANN, CMORPH, and 3B42RT in the warm season of 2008, respectively; (df) describes normalized histograms of area ratio for satellite precipitation products PERSIANN, CMORPH, and 3B42RT in the cold season of 2010, respectively.
Remotesensing 08 00547 g003
Figure 4. (ac) describes normalized histograms of centroid distances for satellite precipitation products PERSIANN, CMORPH, and 3B42RT in the warm season of 2008, respectively; (df) describes normalized histograms of centroid distances for satellite precipitation products PERSIANN, CMORPH, and 3B42RT in the cold season of 2010, respectively.
Figure 4. (ac) describes normalized histograms of centroid distances for satellite precipitation products PERSIANN, CMORPH, and 3B42RT in the warm season of 2008, respectively; (df) describes normalized histograms of centroid distances for satellite precipitation products PERSIANN, CMORPH, and 3B42RT in the cold season of 2010, respectively.
Remotesensing 08 00547 g004
Figure 5. (ac) describes normalized histograms of intersection area ratios for satellite precipitation products PERSIANN, CMORPH, and 3B42RT in the warm season of 2008, respectively; (df) describes normalized histograms of intersection area ratios for satellite precipitation products PERSIANN, CMORPH, and 3B42RT in the cold season of 2010, respectively.
Figure 5. (ac) describes normalized histograms of intersection area ratios for satellite precipitation products PERSIANN, CMORPH, and 3B42RT in the warm season of 2008, respectively; (df) describes normalized histograms of intersection area ratios for satellite precipitation products PERSIANN, CMORPH, and 3B42RT in the cold season of 2010, respectively.
Remotesensing 08 00547 g005
Figure 6. (ac) describes normalized histograms of orientation differences for satellite precipitation products PERSIANN, CMORPH, and 3B42RT in the warm season of 2008, respectively; (df) describes normalized histograms of orientation angle differences for satellite precipitation products PERSIANN, CMORPH, and 3B42RT in the cold season of 2010, respectively.
Figure 6. (ac) describes normalized histograms of orientation differences for satellite precipitation products PERSIANN, CMORPH, and 3B42RT in the warm season of 2008, respectively; (df) describes normalized histograms of orientation angle differences for satellite precipitation products PERSIANN, CMORPH, and 3B42RT in the cold season of 2010, respectively.
Remotesensing 08 00547 g006
Figure 7. (ac) Time series of object-based missing ratio, object-based false alarm ratio, and overall interest score for satellite precipitation products in the warm season of 2008, respectively.
Figure 7. (ac) Time series of object-based missing ratio, object-based false alarm ratio, and overall interest score for satellite precipitation products in the warm season of 2008, respectively.
Remotesensing 08 00547 g007
Figure 8. (ac) Time series of object-based missing ratio, object-based false alarm ratio, and overall interest score for satellite precipitation products in the cold season of 2010, respectively.
Figure 8. (ac) Time series of object-based missing ratio, object-based false alarm ratio, and overall interest score for satellite precipitation products in the cold season of 2010, respectively.
Remotesensing 08 00547 g008
Table 1. Mean Interest Values of Four Attributes for Satellite Precipitation Products in the Warm Season of 2008.
Table 1. Mean Interest Values of Four Attributes for Satellite Precipitation Products in the Warm Season of 2008.
ProductArea RatioCentroid DistanceIntersection Area RatioOrientation Difference
PERSIANN0.430.720.680.69
CMORPH0.570.790.610.68
3B42RT0.490.820.600.64
Table 2. Mean Interest Values of Four Attributes for Satellite Precipitation Products in the Cold Season of 2010.
Table 2. Mean Interest Values of Four Attributes for Satellite Precipitation Products in the Cold Season of 2010.
ProductArea RatioCentroid DistanceIntersection Area RatioOrientation Difference
PERSIANN0.380.720.710.70
CMORPH0.500.800.670.72
3B42RT0.460.810.640.66
Table 3. Mean of Three Verification Scores for Satellite Precipitation Products in the Warm Season of 2008.
Table 3. Mean of Three Verification Scores for Satellite Precipitation Products in the Warm Season of 2008.
ProductObject-Based Missing RatioObject-Based False Alarm RatioOverall Interest Score
PERSIANN0.030.010.64
CMORPH0.020.030.70
3B42RT0.030.040.68
Table 4. Mean of Three Verification Scores for Satellite Precipitation Products in the Cold Season of 2010.
Table 4. Mean of Three Verification Scores for Satellite Precipitation Products in the Cold Season of 2010.
ProductObject-Based Missing RatioObject-Based False Alarm RatioOverall Interest Score
PERSIANN0.050.090.64
CMORPH0.060.060.69
3B42RT0.030.120.68

Share and Cite

MDPI and ACS Style

Li, J.; Hsu, K.-L.; AghaKouchak, A.; Sorooshian, S. Object-Based Assessment of Satellite Precipitation Products. Remote Sens. 2016, 8, 547. https://doi.org/10.3390/rs8070547

AMA Style

Li J, Hsu K-L, AghaKouchak A, Sorooshian S. Object-Based Assessment of Satellite Precipitation Products. Remote Sensing. 2016; 8(7):547. https://doi.org/10.3390/rs8070547

Chicago/Turabian Style

Li, Jingjing, Kuo-Lin Hsu, Amir AghaKouchak, and Soroosh Sorooshian. 2016. "Object-Based Assessment of Satellite Precipitation Products" Remote Sensing 8, no. 7: 547. https://doi.org/10.3390/rs8070547

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop