Evaluation of the Impact of Rainfall Inputs on Urban Rainfall Models: A Systematic Review

Over the past several decades, urban flooding and other water-related disasters have become increasingly prominent and serious. Although the urban rain flood model’s benefits for urban flood simulation have been extensively documented, the impact of rainfall input to model simulation accuracy remains unclear. This systematic review aims to provide structured research on how rain inputs impact urban rain flood model’s simulation accuracy. The selected 48 peer-reviewed journal articles published between 2015 and 2019 on the Web of ScienceTM database were analyzed by key factors, including rainfall input type, calibration times and verification times. The results from meta-analysis reveal that when a traditional rain measurement was used as the rainfall input, model simulation accuracy was higher, i.e., the Nash–Sutcliffe efficiency coefficient (NSE) of traditional technology for rain measurement was higher than the 0.18 for the new technology rain measurement with respect to flow simulation. In addition, the single-field sub-flood calibration model was better than the multi-field sub-flood calibration model. NSE was higher than 0.14. The precision was better for the verification period; NSE of the calibration value showed a 0.07 higher verification value on average in flow simulation. These findings have certain significance for the development of future urban rain flood models and propose the development direction of the future urban rain flood model. Finally, in view of the rainfall input problem of the urban storm flood model, we propose the future development direction of the urban storm flood model.


Introduction
The frequent occurrence of urban flood events has caused severe losses to human life and property worldwide [1]. Climate change and the urban rain island effect have exacerbated the problem via increasing occurrence of extreme rainfall events [2]. Continuous development and expansion of cities have led to the transformation of natural permeable surfaces into impervious underlay surfaces, which accelerate the formation of runoff [3]. At the same time, continuous economic development has led to more severe flood losses [4]. The increasing complexity and harmfulness of urban flood events have made urban rain and flood issues important research topics. The basis of the study of urban rain and flood problems is to establish urban rain and flood models, calibrate and verify the models by simulating and reproducing historical flood events, and then providing technical support for urban hydrological forecasting, urban planning, and water resource utilization. The goal is to alleviate or even solve the problem of urban flooding.
Scholars have developed hundreds of urban rain and flood models. Model development can be divided into three stages ( Figure 1): the foundational stage , the development stage (1972)(1973)(1974)(1975)(1976)(1977)(1978)(1979)(1980)(1981)(1982)(1983)(1984)(1985)(1986)(1987)(1988)(1989)(1990), and the modern stage (1991-today) [5]. The introduction of mathematical formulas into urban drainage design calculations in 1889 marked the beginning of the foundational stage [6]. This phase mainly used empirical equations based on physical mechanisms to meet the needs of urban runoff calculations. The establishment of the US Environmental Protection Agency in 1971 marked the beginning of the development phase of the storm water management model (SWMM) [7]. The SWMM is a semi-distributed model that integrates surface runoff, surface confluence, and pipe network confluence. Its development is an important sign of the maturity of urban hydrological models [8]. At this stage, the research is mainly focused on urban hydrological processes and hydrodynamic process simulation. With algorithmic innovation, the model is able to meet the basic needs of urban rain and flood simulation. Since 1991, or the modern stage, developments in science and technology have provided a solid foundation for rapid development of urban rain and flood models, and comprehensive integration is now a remarkable feature of current models. The simulation method is more comprehensive, and the modeling process is more convenient. New methods, technologies, and models have been gradually introduced into urban rain and flood simulation, making simulation methods more accurate and complete [9]. Many scholars have conducted detailed systematic reviews of the performance of rain and flood models over time.
Verworn et al. [10] examined today's basic principles of urban drainage, potential future advances, and their relevance to flood protection. Boughton et al. [11] systematically review the development and application of water balance models in Australia over a 40-year period. Winz, Ines et al. [12] traced the theoretical and practical evolution of system dynamics over a period of 50 years. From a review of the literature and selected case studies, the authors identified and discussed a number of the best practices and common pitfalls in the application of system dynamics simulation. Summarizing the calculation methods for urban rainwater discharge and examining development of urban rainwater models, Xia Jun et al. [13] compared commonly used urban rainwater models; with respect to the problems of insufficient data and investigative mechanisms in urban rainwater models, future development and improvement of the urban rainwater model are discussed. These review articles on urban rain flood models have discussed and evaluated the application effects of models in terms of structure principles, application characteristics, and development prospects, and they are all qualitative descriptions. However, there is little work on quantitative and intuitive descriptions of model application effect. In addition, there is almost no discussion about the influence of rainfall input on the effect of model application.
The accuracy of the urban rain flood model is determined by rainfall input, model structure, and model parameters [14,15]. Analysis and research have determined that uncertainty in rainfall input is the greatest source of model error. Based on a literature search of the Web of Science™ database, the present study, using a meta-analysis method, screens published research results on the urban rain flood model over the past five years, extracting model simulation accuracy data. The main analysis is divided into traditional rainfall input (rainfall station observations) and new technology rainfall input (radar rain measurement, numerical forecasting, and Web crawling). This study also systematically reviews the research and model development [16] work regarding recent urban rain flood models. Rainfall observation from rainfall stations has the advantage of high observation accuracy, but the spatiotemporal effect is poor [17]. New technology of rainfall measurement is designed to improve the spatial and temporal resolution of rainfall input to the model, but its observation accuracy is far from sufficient [18]. These limitations are the main reasons that rainfall input affects model accuracy [5]. In addition, rainfall duration, magnitude, model calibration times, and model verification times all have some impact on model accuracy.
Since the 1980s, meta-analysis has been recognized by researchers in different disciplines such as medicine, psychology, and pedagogy [19]. The research idea is to calculate the average effect of the data generated through experimental research and quasi-experimental research, so it belongs to the category of quantitative research methods. Sohn W et al. quantitatively evaluate the relationship between climate and low impact development (LID) effectiveness, which provides new ideas for rainwater management policies [20].
This study summarizes the development process behind urban rain flood models and points out the development characteristics of different stages of development, along with characteristics and model application. Further, it combines meta-analysis methods, quantitatively evaluating the impact of such factors as rainfall input type, calibration times, and verification times on model accuracy, developing quantitative analysis improvements. Finally, this study examines possible applications of emerging technologies in further development of urban rain flood models.  [20]. This study summarizes the development process behind urban rain flood models and points out the development characteristics of different stages of development, along with characteristics and model application. Further, it combines meta-analysis methods, quantitatively evaluating the impact of such factors as rainfall input type, calibration times, and verification times on model accuracy, developing quantitative analysis improvements. Finally, this study examines possible applications of emerging technologies in further development of urban rain flood models.  [21].( The figure is adapted from reference [21]).

Materials and Methods
This review of the impact of rainfall input on the accuracy of urban rain flood models involves two review steps: (1) systematic selection and review of 58 peer-reviewed journal articles for study areas in 15 countries, published in 42 journals, and (2) employment of a meta-analysis [20] method to build a framework to quantitatively evaluate the simulation results of the model based on rainfall input.

Study Search and Selection
To search and select the literature comprehensively and extensively, a research entry point was established, along with determination of which factors most strongly affect the simulation accuracy of urban flood models. After selection and final determination, rainfall as the input condition of the model was determined to be the main factor. The study theme was then determined to be the influence of different rainfall input conditions on the simulation accuracy of urban rain flood models. Four keywords were selected: urban pluvial flood model, urban rain flood model, urban storm water model, and urban hydrological model. To highlight research trends in the urban rain flood model, five years were chosen as the study period. The search was conducted on the Web of Science™ database. The next step was to screen the literature. The keyword search yielded 3464 published papers; 2265 of these were filtered out to complete the first article screening. The second screening yielded 621 related papers. A third screening produced 195 additional papers. Additionally, 104 papers were obtained through full-text review. Finally, according to the research topic of the impact Figure 1. Urban rain flood model research progress graph [21]. (The figure is adapted from reference [21]).

Materials and Methods
This review of the impact of rainfall input on the accuracy of urban rain flood models involves two review steps: (1) systematic selection and review of 58 peer-reviewed journal articles for study areas in 15 countries, published in 42 journals, and (2) employment of a meta-analysis [20] method to build a framework to quantitatively evaluate the simulation results of the model based on rainfall input.

Study Search and Selection
To search and select the literature comprehensively and extensively, a research entry point was established, along with determination of which factors most strongly affect the simulation accuracy of urban flood models. After selection and final determination, rainfall as the input condition of the model was determined to be the main factor. The study theme was then determined to be the influence of different rainfall input conditions on the simulation accuracy of urban rain flood models. Four keywords were selected: urban pluvial flood model, urban rain flood model, urban storm water model, and urban hydrological model. To highlight research trends in the urban rain flood model, five years were chosen as the study period. The search was conducted on the Web of Science™ database. The next step was to screen the literature. The keyword search yielded 3464 published papers; 2265 of these were filtered out to complete the first article screening. The second screening yielded 621 related papers. A third screening produced 195 additional papers. Additionally, 104 papers were obtained through full-text review. Finally, according to the research topic of the impact of key variables (rainfall input type, calibration times, and verification times) on model simulation accuracy and the requirements of meta-analysis methods, 48 documents were finalized ( Figure 2). of key variables (rainfall input type, calibration times, and verification times) on model simulation accuracy and the requirements of meta-analysis methods, 48 documents were finalized ( Figure 2). To explore the impact of rainfall input on the simulation results of urban rain and flood models, we must clarify the types of rainfall. This study divides rainfall into two categories according to method of observation: (1) traditionally observed rainfall, i.e., ground stations (hydrological stations, rainfall stations; 41 studies use ground-based observed rainfall as the model input); (2) new technology for rain measurement, i.e., rainfall data obtained via emerging technologies such as remote sensing, radar, numerical weather forecasting, and Web crawling (seven articles employed this method). Urban flood modeling has been an area of great research interest in recent years ( Figure  (3). The present study represents a systematic review of comprehensive observation of rainfall methods and research trends. Thus, the selected research period is 2015-2019. At the same time, to further enrich the content, 10 reference-related papers were included.

Nash-Sutcliffe Efficiency
The Nash-Sutcliffe efficiency (NSE) is a traditional model performance indicator for evaluating the goodness-of-fit between simulated and observed values. The value of NSE varies between -∞ and 1, with higher values indicating better performance. If NSE > 0.5, then the simulated discharge can be regarded as satisfactory [22].

Root of the Mean Square Error
Root of the mean square error (RMSE) represents the error between the simulated and observed values. RMSE is non-negative, the smaller the value, the better the model simulation effect [22].

The Coefficient of Determination
The coefficient of determination (R 2 ) is often used to describe the degree of fit between data. When R 2 is closer to 1, it means that the reference value of the related equation is higher; on the contrary, when it is closer to 0, it means that the reference value is lower [23][24][25]. To explore the impact of rainfall input on the simulation results of urban rain and flood models, we must clarify the types of rainfall. This study divides rainfall into two categories according to method of observation: (1) traditionally observed rainfall, i.e., ground stations (hydrological stations, rainfall stations; 41 studies use ground-based observed rainfall as the model input); (2) new technology for rain measurement, i.e., rainfall data obtained via emerging technologies such as remote sensing, radar, numerical weather forecasting, and Web crawling (seven articles employed this method). Urban flood modeling has been an area of great research interest in recent years ( Figure 3). The present study represents a systematic review of comprehensive observation of rainfall methods and research trends. Thus, the selected research period is 2015-2019. At the same time, to further enrich the content, 10 reference-related papers were included.
Water 2020, 12, x FOR PEER REVIEW 10 of 20

Overview of Urban Rain Flood Model Research Trends
Since the 1970s, urban flood modeling has developed rapidly, and a variety of urban rain and flood models have been developed [29,30]. Urban flood models can be divided into empirical models, conceptual models, and physical models [31]. The empirical model, also known as the "black box model", only inputs and outputs sequences, but lacks physical mechanisms and sufficient analysis of hydrological processes. The conceptual model is based on water balance construction and has a certain physical significance. It is used in flood control planning and urban drainage and widely used in design [32]. The physical model has a strong physical foundation, takes hydrodynamics as its theoretical basis, requires a lot of data, and is complex to solve; however, it has the highest simulation accuracy and wide application prospects.
As shown in Figure 3, the number of research articles related to urban flood models in the Web of Science™ database has surged since 2000. There were three related articles in 2000, 177 in 2010, and 746 in 2019, or 248.6 times the number of articles published in 2000. This is because, since 2000, the urbanization process has seen unprecedented acceleration. On the other hand, human activity and climate change are leading to more frequent extreme rainfall events, and the characteristics of the urban rain island effect have become clear [33]. These two factors have led to increased rainfall and reduced infiltration in urban areas, leaving them prone to water accumulation and exacerbating frequent flooding disasters in cities [34]. Thus, urban rainwater models have been rapidly developed and applied. The selected articles involve more than 20 urban flood models. As shown in Figure 4, the SWMM model has gradually become the most widely used model in the world since it emerged in 1971 [16,35,36]. It appears in our sample a total of 23 times, close to the total number of occurrences of all other models combined. The SWMM model is easy to operate, has a wide range of applications, and the source code can be shared [37,38]. These advantages make it one of the preferred models for studying urban hydrology. In addition, MIKE, UFDSM, geographic information systems (GIS) + and other urban rainfall runoff models are widely used [2,39].

Nash-Sutcliffe Efficiency
The Nash-Sutcliffe efficiency (NSE) is a traditional model performance indicator for evaluating the goodness-of-fit between simulated and observed values. The value of NSE varies between −∞ and 1, with higher values indicating better performance. If NSE > 0.5, then the simulated discharge can be regarded as satisfactory [22].

Root of the Mean Square Error
Root of the mean square error (RMSE) represents the error between the simulated and observed values. RMSE is non-negative, the smaller the value, the better the model simulation effect [22].

The Coefficient of Determination
The coefficient of determination (R 2 ) is often used to describe the degree of fit between data. When R 2 is closer to 1, it means that the reference value of the related equation is higher; on the contrary, when it is closer to 0, it means that the reference value is lower [23][24][25].

Relative Error
Relative error (RE) refers to the value obtained by multiplying the ratio of the absolute error caused by the measurement to the measured (conventional) true value by 100%, expressed as a percentage. Generally speaking, the relative error can better reflect the credibility of the measurement. It is mostly used in the evaluation of urban rain and flood models to indicate the credibility of the simulated value of flood peak discharge [24,25].

Data Analysis
The selected 48 peer-reviewed articles (Table 1) are characterized by the quantitative relationship description of different key variables to model accuracy, such as rainfall input type, calibration times, and verification times. We analyze the research trend by counting the frequency of related articles appearing in each year. To explain the influence of various factors on the accuracy of the model, we conducted a quantitative analysis of the accuracy problems corresponding to each key variable in the model. Different key variables are used as classification conditions, and evaluation indicators, such as NSE, RMSE, R 2 and RE, are unified as data sources for meta-analysis. In accordance with the classification of research objects, data on research methods were selected from the 48 articles, including research locations, study area scales, model calibration, verification of rainfall events, and model accuracy evaluation indicators. The reviewed literature reveals the effect of different rainfall inputs on model application.
The application effect or model accuracy of an urban rain and flood model is usually determined by comparing the flow process, flood peak flow, total runoff, or water depth of the model with the observed values [28]. However, the number of quantitative indexes between studies is inconsistent. To select the quantitative verification indexes, a weighted average method was used to determine the verification accuracy of the model. An assessment of the hydrologic effectiveness of low impact development (LID) practices for managing runoff with different objectives 2 Identifying fit-for-purpose lumped surrogate models for large urban drainage systems using GLUE 3 Improving urban runoff in multi-basin hydrological simulation by the HYPE model using EEA urban atlas: a case study in the Sege river basin, Sweden 4 An urban pluvial flood simulation model based on diffusive wave approximation of shallow water equations 5 A comparison of SAC-SMA and Adaptive Neuro-fuzzy Inference System for real-time flood forecasting in small urban catchments 6 Urban flood modelling combining cellular automata framework with semi-implicit finite difference numerical formulation 7 Stormwater simulation based on the concept of sustainable development of sponge city construction 8 Mathematical modeling of flood management system in the city of Gdansk, Orunski stream case study 9 Real-time urban inundation prediction combining hydraulic and probabilistic methods 10 A cellular automata fast flood evaluation (CA-ffe) model 11 A methodology for simple 2-D inundation analysis in urban area using SWMM and GIS 12 Simulating urban rainfall-runoff and assessing LID facilities by SWMM model in Dahongmen catchment 13 Prognostic model of total runoff and its components from a partially urbanized small lowland catchment 14 High-resolution modelling of the grass swale response to runoff inflows with Mike SHE 15 Simulation of urban drainage system using disaggregated rainfall data 16 Flood inundation assessment for the Hanoi Central Area, Vietnam under historical and extreme rainfall conditions 17 SWAT plus versus SWAT 2012: comparison of sub-daily urban runoff simulations 18 A case study for the application of an operational two-dimensional real-time flooding forecasting system and smart water level gauges on Roadsin Tainan city, Taiwan  19 Urban flood simulation and prioritization of critical urban sub-catchments using SWMM model and PROMETHEE II approach 20 Effects of using high-density rain gauge networks and weather radar data on urban hydrological analyses 21 GIS-based SWMM model for simulating the catchment response to flood events 22 Combination of GIS and mathematical modeling to predict floods in semiarid areas: case of Rheraya watershed (Western High Atlas, Morocco) 23 Urban stormwater inundation simulation based on SWMM and diffusive overland-flow model 24 Assessing methods for predicting green roof rainfall capture: A comparison between full-scale observations and four hydrologic models 25 Simulation of control efficiency of low impact development measures for urban stormwater 26 Assessing the effects of climate change on water quantity and quality in an urban watershed using a calibrated stormwater model 27 Evaluating the effects of low impact development practices on urban flooding under different rainfall intensities 28 Predicting urban storm water-logging for Chittagong city in Bangladesh 29 Impact of rapid urbanization on rainfall-runoff processes in urban catchment: Case study for Liangshui River basin 30 Urban stormwater forecasting model and drainage optimization based on water environmental capacity 31 Jinan urban rainstorm model based on lattice boltzmann method 32 Modeling the quality and quantity of runoff in a highly urbanized catchment using storm water management model 33 Support vector regression for rainfall-runoff modeling in urban drainage: A comparison with the EPA's storm water management model 34 Distributed simulation of urban storm water based on D8 algorithm 35 Urban rainfall-runoff simulations and assessment of low impact development facilities using SWMM model-A case study of Qinghe catchment in Beijing 36 The use of PCSWMM for assessing the impacts of land use changes on hydrological responses and performance of WSUD in managing the impacts at Myponga catchment, South Australia 37 Analysis of extreme flooding events through a calibrated 1D/2D coupled model: the case of Barcelona (Spain) 38 Assessment of the service performance of drainage system and transformation of pipeline network based on urban combined sewer system model 39 Developing an effective 2-D urban flood inundation model for city emergency management based on cellular automata 40 Modelling and assessment of hydrological changes in a developing urban catchment 41 Evaluation of a hydrology and run-off BMP model in SUSTAIN on a commercial area and a public park in South Korea 42 Precipitation estimation methods in continuous, distributed urban hydrologic modeling 43 High-resolution hydrologic forecasting for very large urban areas 44 Quantitative rainfall analysis and flow simulation for an urban catchment using input from a weather radar 45 Storm runoff prediction using rainfall radar map supported by global optimization methodology 46 Radar data analyses for a single rainfall event and their application for flow simulation in an urban catchment using the SWMM model 47 Effects of using high-density rain gauge networks and weather radar data on urban hydrological analyses 48 Adaptive blending method of radar-based and numerical weather prediction QPFs for urban flood forecasting

Overview of Urban Rain Flood Model Research Trends
Since the 1970s, urban flood modeling has developed rapidly, and a variety of urban rain and flood models have been developed [29,30]. Urban flood models can be divided into empirical models, conceptual models, and physical models [31]. The empirical model, also known as the "black box model", only inputs and outputs sequences, but lacks physical mechanisms and sufficient analysis of hydrological processes. The conceptual model is based on water balance construction and has a certain physical significance. It is used in flood control planning and urban drainage and widely used in design [32]. The physical model has a strong physical foundation, takes hydrodynamics as its theoretical basis, requires a lot of data, and is complex to solve; however, it has the highest simulation accuracy and wide application prospects.
As shown in Figure 3, the number of research articles related to urban flood models in the Web of Science™ database has surged since 2000. There were three related articles in 2000, 177 in 2010, and 746 in 2019, or 248.6 times the number of articles published in 2000. This is because, since 2000, the urbanization process has seen unprecedented acceleration. On the other hand, human activity and climate change are leading to more frequent extreme rainfall events, and the characteristics of the urban rain island effect have become clear [33]. These two factors have led to increased rainfall and reduced infiltration in urban areas, leaving them prone to water accumulation and exacerbating frequent flooding disasters in cities [34]. Thus, urban rainwater models have been rapidly developed and applied.
The selected articles involve more than 20 urban flood models. As shown in Figure 4, the SWMM model has gradually become the most widely used model in the world since it emerged in 1971 [16,35,36]. It appears in our sample a total of 23 times, close to the total number of occurrences of all other models combined. The SWMM model is easy to operate, has a wide range of applications, and the source code can be shared [37,38]. These advantages make it one of the preferred models for studying urban hydrology. In addition, MIKE, UFDSM, geographic information systems (GIS) + and other urban rainfall runoff models are widely used [2,39].  A rational formula was introduced into urban drainage design in 1889; this is considered the beginning of study of urban flood models [21]; however, study of urban hydrology in the true sense began in the 1960s. Continuous expansion of the scale of urban flooding brings ever greater attention to water problems caused by urbanization [40]. After 1980, with improvements in computing power and the emergence of geographic information systems (GIS) and remote sensing technology (RS) in hydrology, a series of urban hydrological-hydrodynamic models surfaced, including SWMM, STORM, and HSPF [29]. Among the many urban flood models, SWMM is still the most widely used. Scholars worldwide use the SWMM model to carry out in-depth research on storm flood simulation A rational formula was introduced into urban drainage design in 1889; this is considered the beginning of study of urban flood models [21]; however, study of urban hydrology in the true sense began in the 1960s. Continuous expansion of the scale of urban flooding brings ever greater attention to water problems caused by urbanization [40]. After 1980, with improvements in computing power and the emergence of geographic information systems (GIS) and remote sensing technology (RS) in hydrology, a series of urban hydrological-hydrodynamic models surfaced, including SWMM, STORM, and HSPF [29]. Among the many urban flood models, SWMM is still the most widely used. Scholars worldwide use the SWMM model to carry out in-depth research on storm flood simulation [41], urban non-point source load estimation [42], and effective evaluation of low impact development (LID) measures [16]. The applicability of this model has also been verified in some non-urban areas. It can be said that SWMM is representative of the urban flood model. After model development entered the modern stage, how to use emerging science and technology to improve the spatial and temporal accuracy of the urban flood model simulation has become a key consideration [5]. GIS, remote sensing, Internet, and Big Data technologies provide multiple possibilities for urban storm flood simulation and verification. Multi-data input, multi-method application, multi-technology paralleling, and multi-method verification of Big Data-driven simulation may become the direction for future development.

Influence of Rainfall Calibration and Verification Times on Model Accuracy
Rainfall input data are taken from the 41 articles with measured rainfall. Among these, 16 papers use NSE as the index for model calibration and verification. These 16 articles involve 24 urban flood models. Accuracy statistics for their calibration and verification are shown in Figure 5.  Figure 5 shows that in the 24 research simulations [3,16,36,41,[43][44][45][46][47][48][49][50][51][52], the accuracy of the model in six verification periods was lower than that in the calibration period, and the accuracy of the model in 18 verification periods was higher than that in the calibration period. Calibration accuracy is 0.07 higher than verification accuracy on average and the overall trend of model calibration accuracy is positive.
This study also summarizes the comparison articles to compare the accuracy of the models for single-run (refers to a single historical flood to calibrate the model) and multiple-run (refers to calibrating the model with multiple historical floods) rainfall-runoff event calibration. The results are shown in Table 2. Among the 48 articles (Table 1) Figure 5 shows that in the 24 research simulations [3,16,36,41,[43][44][45][46][47][48][49][50][51][52], the accuracy of the model in six verification periods was lower than that in the calibration period, and the accuracy of the model in 18 verification periods was higher than that in the calibration period. Calibration accuracy is 0.07 higher than verification accuracy on average and the overall trend of model calibration accuracy is positive.
This study also summarizes the comparison articles to compare the accuracy of the models for single-run (refers to a single historical flood to calibrate the model) and multiple-run (refers to calibrating the model with multiple historical floods) rainfall-runoff event calibration. The results are shown in Table 2. Among the 48 articles (Table 1) RE was used to evaluate the effect in 10.5% and 29.90%, with large errors. Statistical analysis shows NSE higher than 0.14 and R 2 higher than 0.17 on flow simulation; but in water depth simulation, RE is about 6% lower. Combining the two statistical tables, the single-run rainfall runoff event calibration model has higher simulation accuracy than the multi-run calibration model.  The internal reason is that this phenomenon occurs because of insufficient understanding of the urban hydrological process [13]. The study of urbanization hydrological effect is the basis for discriminating urban water problems, exploring urban water cycle rules and forming the theory of urban production and confluence. The mechanism of urban production and confluence is the theoretical basis for constructing urban hydrological models. Fully understanding the process of urban hydrological mechanism requires urban hydrological monitoring and forecasting. Perfect meteorological and hydrological observation data have always been an important condition for people to understand and study the laws of hydrology. Urban hydrological observation and forecasting are important contents and key technologies for urban hydrological research [57]. Strengthening the understanding of hydro-physical processes, conducting in-depth mechanism research and the dynamic calibration of model parameters are of great significance to the improvement of urban rain flood models.

Influence of Rain Measurement Method on Model Accuracy
There are many methods for rainfall observation, divided into two types according to observation technique. One is traditional rain measurement via rainfall stations, hydrological stations, meteorological stations, and other facilities to record rainfall on the ground. Rainfall data obtained via this method are the most commonly used rainfall data for rainfall input to urban flood models. However, this method has serious difficulties, including high station construction cost and low spatial resolution [17]. The other method is new technology measurement, mainly through radar, remote sensing, microwave, and numerical weather forecasting. An important feature of this method is completion of observations prior to rainfall. This method has a wide range of space-time observations [58,59]. Many studies have focused on increasing the time and space of rainfall input. Resolution seeks to improve the accuracy of simulated predictions of urban rain flood models [60,61]; this is likely to remain a developing trend in precipitation measurement. However, the observation accuracy of this method is not enough, and it usually needs to be corrected in conjunction with surface rain measurement [17]. In addition, the observation data usually need to be downscaled and inferred to be applied; this technology is very complex. Of the 48 articles screened for quantitative analysis, 41 use rainfall input data for the urban rain flood model observed by traditional methods; the remaining seven articles use new technology measurement. Figure 6 shows that current rainfall input to urban rain and flood models is still based mainly on ground observation of rainfall; application of new technology for rain measurement is less frequent but is a developing trend.
Water 2020, 12, x FOR PEER REVIEW 14 of 20 accuracy of this method is not enough, and it usually needs to be corrected in conjunction with surface rain measurement [17]. In addition, the observation data usually need to be downscaled and inferred to be applied; this technology is very complex. Of the 48 articles screened for quantitative analysis, 41 use rainfall input data for the urban rain flood model observed by traditional methods; the remaining seven articles use new technology measurement. Figure 6 shows that current rainfall input to urban rain and flood models is still based mainly on ground observation of rainfall; application of new technology for rain measurement is less frequent but is a developing trend.  use RE, with values 5.69% and 7.5%, respectively. For observation of depth, one article uses both RMSE and RE, with values 0.194 mm and 24.55%, respectively. NSE of traditional technology for rain measurement is higher at 0.18 than new technology for rain measurement on flow simulation. Note that for simulation on the same hydrological elements, the simulation results obtained via traditional rain measurement input are better than that of new technology methods.  Traditional rainfall measurement technology is not obsolete, because it continues to play an irreplaceable role in the input layer of the model. However, rain measurement technology continues to evolve, relying on the latest scientific and technological achievements to build a scientific and complete meteorological and hydrological surface and underground monitoring network system. Realizing comprehensive monitoring of urban rainfall, combined with multi-source information technology and satellite remote sensing, meteorological radar, urban video monitoring, and sensor equipment to obtain important information on the urban rainfall process from multiple angles is a new trend in rain measurement technology [57]. In addition, improving the accuracy of rainfall forecasting [68,69] is indirectly used as an effective means to improve the accuracy of urban storm flood simulation.

Application of Web Crawling Rainfall Data in Model Input
In the current information explosion era, Big Data technology uses Big Data to effectively crawl network data as the rainfall model input; this is a topic of intense interest in current urban hydrological research [70]. Not a single selected article uses clear quantitative evaluation criteria; however, a high-quality related paper was found. An article entitled "Critical rainfall thresholds for urban pluvial flooding inferred from citizen observations" was published in 2019 in Science of the Total Environment. The authors explore the possibility of using citizen flood observations to gain new insights [18] using a 10-year radar rain map dataset and 70,000 citizen flood reports in Rotterdam. The investigators trained three binary decision trees based on Big Data and used them to predict flood occurrence based on peak rainfall intensity at different time scales. They concluded that the decision tree correctly predicted 37-52% of all flood events and 95-97% of all non-flood events. This shows that Big Data fusion may be useful in studying urban flood simulation and forecasting. How to transform unstructured data into structured data and use dense data to drive urban rain and flood models will be a focus of future research.
Multi-source fusion of rainfall observations is an inevitable outcome of the combination of ground-level rain measurement and new technology rain measurement to strengthen rainfall observation and quantitative precipitation forecasting in the future [71]. This will further lead to multi-source information coupling technology to achieve high rainfall observation accuracy, which in turn improves the simulation ability of urban rain and flood models, providing a more accurate basis for urban flood control and disaster reduction. The problems of short forecast periods and low accuracy must be solved urgently to improve urban flood disaster forecasting and early warning. The advantages of massive data and efficient processing of Big Data should combine with urban flood disaster simulation principles to study urban flood disaster forecasting and early warning theory and methods. This will become the very direction toward solving the problem of urban rain and flooding events. It is necessary to construct urban flood forecast models and real-time correction methods, coupled with urban storm forecasting models driven by Big Data. In addition, urban flood dynamic simulation models and urban flood disaster loss assessment models are needed, along with an urban flood disaster loss index system and disaster rating evaluation method. For urban areas, developing a dynamic early warning mechanism for flood disasters is an urgent problem.
Next steps should include development of a multi-source data assimilation system and a standardized basic database. Better cooperation with government agencies is required, as well as scientific research institutions and network technology companies to build both a Big Data storage and sharing system and a rainfall proximity forecasting system based on multi-source information and data. This will provide short-term heavy rain forecasting. Another important issue is processing high-precision forecasting information, so as to provide scientific and technological support for urban rain and flood simulation, flood forecasting, flood control, and disaster reduction.

Conclusions
By using meta-analysis to evaluate the impact of rainfall input on the simulation effect of urban rain flood models, we found that the accuracy of the calibration period is higher than that of the verification period when the urban rainwater model is applied. NSE of the calibration value is a 0.07 higher verification value on average in flow simulation. This is because the model uses the measured runoff as the target of model fitting during the calibration period, and only takes into account the flood characteristics of the calibration period during the adjustment of the model parameters, while the model parameters still retain the flood characteristics of the calibration period during the model verification period, making the accuracy below calibration period. In the simulation results, the single-rainfall runoff event calibration model has higher accuracy than the multi-field calibration model. NSE is higher than 0.14 and R 2 is higher than 0.17 on flow simulation; but in water depth simulation, RE is about 6% lower. As this is caused by the parameter rate tending to average, the application of the multi-field flood calibration model should be more reliable.
The variety of rain measurement methods leads to diverse rainfall inputs to the model. The current urban rain flood model rainfall input is still based on ground rain measurement, supplemented by emerging technology rain measurement. NSE of traditional technology for rain measurement is higher by 0.18 than new technology for rain measurement on flow simulation. Given the problems analyzed in this paper, the following guidance applies to the study of rainfall input in future urban rain and flood models: (1) optimize the layout and construction of existing ground rainfall stations, as ground rainfall measurement is still reliable and feasible as a method; (2) develop long-term accurate methods; (3) speed the development of radar, remote sensing satellites, and microwave technologies. These latter technologies are effective methods for observing rainfall and can improve the spatial and temporal resolution of rainfall measurement. This paper conducts a meta-analysis of the literature on the impact of rainfall input on the accuracy of urban rainfall models. Quantification and systematic evaluation are carried out on several aspects of rainfall input type, calibration times and verification times. It further examines the feasibility of network crawling rainfall input. Finally, it provides new ideas for future development of urban rain and flood models.