Next Article in Journal
Prediction of Sunlight- and Salinity-Driven Inactivation Kinetics of Microbial Indicators with Validation in a 3D Water Quality Model
Previous Article in Journal
Water Footprints of Dairy Milk Processing Industry: A Case Study of Punjab (India)
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Water
Volume 16
Issue 3
10.3390/w16030436
water-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Toward Systematic Literature Reviews in Hydrological Sciences

by
David De León Pérez
1,2,*,
Rick Acosta Vega
3,
Sergio Salazar Galán
1,4,
José Ángel Aranda
1 and
Félix Francés García
1
1
Research Group of Hydrological and Environmental Modelling (GIHMA), Research Institute of Water and Environmental Engineering (IIAMA), Universitat Politècnica de València, 46022 Valencia, Spain
2
GeoAgro-Environmental Sciences and Resources Research Center Foundation, Neiva 410001, Colombia
3
Statistical and Quantitative Methods Research Group (GEMC), Universidad del Magdalena, Santa Marta 470004, Colombia
4
Agroecosystems History Laboratory, Universidad Pablo de Olavide, 41013 Sevilla, Spain
*
Author to whom correspondence should be addressed.
Water 2024, 16(3), 436; https://doi.org/10.3390/w16030436
Submission received: 20 December 2023 / Revised: 16 January 2024 / Accepted: 26 January 2024 / Published: 29 January 2024
Browse Figures
Versions Notes

Abstract

:
Systematic literature reviews can provide an objective global overview of background research on hydrological questions. This study presents a methodology to ensure rigor, traceability, and replicability in evaluating state-of-the-art hydrological topics. The proposed methodology involved systematic, objective, and explicit steps. Strategies with defined selection criteria were used to search the relevant literature comprehensively and accurately on hydrological uncertainty as a case study. The most pertinent documents were filtered to build a critical state-of-the-art synthesis for evaluating their quality and relevance. This methodology allows systematic literature analysis to provide an objective summary of the evidence, with a structured procedure for state-of-the-art reviews, which promotes transparency in the search, unlike typical review papers that lack reproducible methodologies that may lead to obtaining a potentially subjective reference selection. Consequently, the proposed methodology improves reference reliability and study reproducibility while generalizing the methodology. This procedure has proven effective and practical for building state-of-the-art research in the study case (State-of-the-art Uncertainty in Hydrological forecasting from 2017 to 2023) and can be considered a relevant tool in hydrology and other STEM disciplines, providing a rigorous and transparent approach that enhances evidence quality and reliability.

1. Introduction

According to Cochrane-Org [1], a systematic literature review (SLR) provides a comprehensive, objective, and transparent overview of all evidence related to a specific question. On the other hand, Siddaway et al. [2] define it as “a review of a formulated question that uses systematic and explicit methods to identify, select, and critically appraise relevant research, as well as to collect and analyze data from the studies included in the review”.
In the hydrological literature, there is a long history of evolution and development of knowledge, including reviews made by authors who, at the time, presented a summary of the current state of the discipline; thus, it is possible to find articles with more than a century of publication that the journal already grouped under the type “Review”, e.g., “Hydraulic illustration of the Wheatstone bridge” [3], an article that presents how to build in the laboratory an analogy of the Wheatstone bridge employing a hydraulic circuit. In the 1920s, Piper [4] discussed the problem of artesian aquifers and their water movement, stating that research on the subject was predominantly qualitative and lacked historical information for adequate quantitative studies. On the other hand, Breusse [5] reviews the geophysical methods used for groundwater exploration and development, pointing out that these methods were widely known and used in petroleum exploration but not so much by hydrogeologists.
However, the first comprehensive historical review of hydrology and its development over 300 years was presented by Linsley [6], who began with the first empirical relationships established by Perrault [7] on the relationship between rainfall and runoff and summarized 49 papers to present his vision of the future of hydrology. He anticipated the use of computers to improve the efficiency and speed of estimating hydrologic values, allowing accurate models to determine discharge hydrographs. The Commission on Hydrology of the Association of Geographers of Japan (Nippon-Chiri-Gakkai) annually updates its review of state-of-the-art hydrology, climatology, and meteorology to keep hydrologists and researchers informed of advances and new developments in the field [8].
There are many review studies on the different hydrological sub-areas in the literature, such as in Ecohydrology [9,10,11,12,13,14], Hydropower [15], Hydroinformatics [16,17,18,19,20,21,22,23,24,25], Gral Hydrology [26,27,28,29,30], Hydrology and Climate Change [31,32,33,34,35], Stochastic Hydrology [36,37,38,39], Forecasting and Uncertainty [40,41,42,43,44,45,46,47,48,49], Nival Hydrology [50,51,52,53], GIS Hydrology and Remote Sensing [54,55,56,57,58], Fluvial Hydrology [59,60], Socio-Hydrology [61,62,63,64], Hydrogeology [65,66,67,68,69,70], Hydropedology [71,72], Oceanography [73,74], Hydrological Regionalization [75,76], and Urban Hydrology [77,78,79,80,81,82,83,84,85,86], studies embodied in traditional review articles in hydrology, which provide, from a narrative approach, an overview of the relevant literature but may have limitations such as subjective selection of reference articles and systematic shortcomings to reach a comprehensive meta-analysis [87].
SLR is a rigorous methodology for assessing a specific topic’s state-of-the-art. It requires prior knowledge and experience in the literature search and review to ensure reliability but provides a valuable tool to select best practices with an objective analysis, especially in hydrology [87]. Based on the availability of digital databases, SLR allows a structured search with objective criteria [2]. However, a critical analysis is required to ensure accuracy and scientific validity [88,89,90].
Delimiting the search criteria is fundamental in an SLR and should include topics, keywords, and research questions. This helps filter the existing literature and evaluate the current state of a research topic [88,90]. SLR is the beginning of a study, and effective communication of its results and conclusions is as important as the results obtained at the end of the research project [91]. In both cases, the communication can be either complete or fragmented through scientific events or papers for the scientific community to know and discuss [91,92]. It should also be disseminated to society in general as a mechanism of democratization and openness of knowledge and expressed in a clear (and non-scientific) language so that they are brushstrokes of knowledge, open to ordinary citizens who do not know the details of science [91,93].
The relevance of this work is based on the need-to-know state-of-the-art reference documents objectively and avoids biases, in addition to the fact that other sciences (Health Sciences, Computer Science, and Social Sciences, among others) have already developed SRL methodologies adapted to their needs with good results [94]. Developed SRL methodologies adapted to their needs with good results (Kitchenham et al., 2009a) [94]. This protocol was applied to a specific case study of the uncertainty in hydrological forecasting. Nevertheless, it is expected to be replicable in other fields of Science, Technology, Engineering, and Mathematics (STEM) research.

2. Materials and Methods

2.1. Materials

To search for relevant and updated information on the topic under investigation, recognized and prestigious databases should be used to guarantee the universality of the information collected and its replicability.
For the SLR performed in this study, Elsevier’s Scopus [95] and Clarivate’s Web of Science [96] bibliographic databases are the primary scientific information sources; in particular, WOS has a package of databases composed of seven different databases (Web of Science Core Collection, Current Contents Connect, Derwent Innovations Index, KCI-Korean Journal Database, MEDLINE, ProQuestTM Dissertations & Theses Citation Index, and Scielo Citation Index). These databases have a broad scope, bringing together thousands of leading academic journals across all fields of knowledge. This broad scope allows for comprehensive identification of the current state of research on the topic of interest. Although other regional or specialized databases index relevant scientific literature, Scopus and WOS stand out for their global coverage of high-impact publications in each discipline. Additionally, both platforms allow useful functionalities and tools to narrow the results to the most relevant studies and extract them in formats that can be processed by other software, such as analysis programs, for example, VOSviewer 1.6.20 [97].
It is worth mentioning that Scopus and the Web of Science (and several journal papers) are restricted subscription databases accessible only to entities or institutions that can pay for their licensing. Although this fact limits their adoption, for the time being, they are still integral options for mapping international scientific knowledge with great accuracy and extension. Institutional accreditation of the Universitat Politècnica de València (Valencia, Spain) provided access to these resources.

2.2. Methods

The methodology developed was primarily based on an adapted combination of the guidance proposed by Nguyen and Singh [98] and Kitchenham [99] and the steps suggested by Muka et al. [100] for designing an SLR, with a specific adaptation to make it applicable to hydrological sciences and STEM areas. This is a novel contribution of the present article, given that it is an adaptation of the health (and informatic) sciences literature for which they have developed precise mechanisms and protocols for research meta-analyses and pioneered the development of SLR methodologies.
The first step involves the research approach, which clearly defines the research question or questions to be addressed in the review. Additionally, a working team to collaborate at all stages of the process and specific roles should be assigned to each team member for an adequate distribution of tasks. In the next step, a detailed protocol is formulated to guide the review. This protocol establishes inclusion and exclusion criteria for the selection of relevant documents. Then, certain search strategies are designed with specific terms related to the research topic (in the case of this study on Uncertainty, Hydrology, and Forecasting).
Once the protocol has been established, a document search and systematic extraction can begin. The search and selection strategies are then applied to the academic databases, initially evaluating the papers’ abstracts and titles using a standardized form (Form 1). Duplicates are verified and identified, particularly when multiple databases are used for the search.
The selected papers were retrieved in the previous step, and a more detailed reading of the preselected documents was performed after evaluating their titles and abstracts. This careful reading evaluates the methodology, results, and conclusions of documents. This evaluation selected documents that provided relevant information to answer the research questions. The second standardized form (Form 2) contained specific information about the study, allowing for subsequent analyses. It is essential to seek the opinions and references of experts in the field to obtain additional information and recommendations from other relevant documents. These steps ensured an exhaustive and accurate search of existing literature related to the research question.
The collected information was analyzed using the final selection of documents. The quality of the information is evaluated, and a descriptive synthesis of the vital information of the selected documents is made so that in cases where the data allow it, a meta-analysis is carried out to combine and statistically analyze the results of similar studies. The SLR report is then written, documenting all relevant steps and findings. Finally, the manuscript was prepared for submission to a scientific journal for publication and sharing the results with the scientific community.
This methodology has been designed to be applicable in several studies in the hydrological sciences and other related fields, providing a structured and objective approach that improves the quality and reliability of the references obtained in the literature review. Figure 1 shows the flow diagram of the developed methodology.

3. Results

This section details the application of the proposed methodology to a specific case study to demonstrate its application.

3.1. Research Question Statement

As a research question to initiate the search and filtering of existing information, the following question is posed: With the current scientific development in recent years, what are the trends and best practices for analyzing and reducing uncertainty in hydrological forecasting?

3.2. Search Protocol and Inclusion and Exclusion Criteria

3.2.1. Search Strategies

An attempt was made to make the search criteria uniform between the two databases, so that the results could respond similarly to the research questions. Regarding terms to include and exclude (see Table 1), the research team conducted a terms test by essay and error to ensure that the literature focused on uncertainty in hydrological forecasting.
Terms to be included:
  • Hydro*: This search term captures documents that include hydrology-related terms, such as hydroclimatology, hydrometeorology, and hydrological, among others.
  • Uncertainty: This term is fundamental because uncertainty is a central theme in this study. The inclusion of this keyword ensured that the documents were related.
  • Forecast: This term is essential for documents that focus on their ability to predict future events.
Terms to be excluded:
  • Hydropower: Excluded because including this term can lead to studies of power generation rather than uncertainty in hydrological forecasts.
  • Hydrogen*: This term was excluded because it could identify documents related to the chemical elements and technologies surrounding it that were unrelated to the research.
  • Photo*: Excluded because previous reviews have found that with the current growth of photovoltaic power generation technologies, many papers related to forecasts or predictions of photovoltaic power generation capacity have appeared.
  • Petro*: Excluded because many papers related to petroleum were detected in the pre-search essay and error stage when this term was excluded.
Based on these terms, an initial search was carried out in both databases, obtaining 1947 documents for Scopus and 5598 documents for WOS (see documents by year in Figure 2).
Careful selection of terms to include and exclude in the initial search is an essential part of the RSL process as it ensures that the results provide a comprehensive, objective, and accurate overview of the background information related to the research question.
After this initial search, the first inclusion and exclusion criteria (filters) were proposed, taking advantage of the automatic tools in the database platforms. As described below, six sequential filters were applied to both databases. We will discuss the first inclusion and exclusion criteria because, after this filtering stage, expert knowledge work must be carried out. The documents acquired from this procedure will undergo individual assessment in Form 1, where a new screening (based on abstract and title) will be utilized for initial selection. The previously selected materials will thoroughly examine methodology and results to determine the final selection for inclusion in Form 2.
  • Filter 1: The publication year from 2017 to 2023 (October) was used (see Table 2). This period was chosen to obtain trends that showed the current landscape of the field (However, at a general level, previous documents that provide important inputs should be taken into consideration). Furthermore, Figure 2 shows that 2017 saw a new ascending slope in academic production on the research topic. This filter returned 834 documents for Scopus and 3072 for the WOS.
  • Filter 2: The second filter was used for Scopus as the Subject Area and for WOS as the Research Area to delimit the classification area, applying the criteria presented in Table 3, resulting in 782 documents for Scopus and 2882 documents for WOS.
  • Filter 3: The languages admitted for the search of the articles were determined (see Table 4); this filter can be used for convenience or for the researchers’ use of the languages. This filter yielded 744 documents for Scopus and 2860 documents for WOS.
  • Filter 4: Keywords at a general level should be contained within the documents (see Table 5), which helps perform filtering that leads to a general orientation toward the subject of studs, obtaining 615 documents for Scopus and 1140 documents for WOS.
  • Filters 5 and 6: Subsequent filters are based on the researcher’s criteria, as they refine their search so that the focus of the investigation is even more precise. In the specific case of this research, Filter 5 is shown in Table 6 and Filter 6 in Table 7, obtaining 544 documents in Filter 5 for Scopus and 806 documents for WOS. In comparison, Filter 6 closes this stage with 256 and 271 documents in Scopus and WOS, respectively. Something to highlight for the study case, “Drought forecasting” and “Flood Forecasting” were not included as keywords because they are specific cases of “Hydrological forecast”.
Something important to highlight, and it is here where the expertise of the researchers is essential in these two final filters, for the study case, “Drought forecasting” and “Flood Forecasting” were not included as keywords because they are specific cases within the “Hydrological forecast” which is our object.

3.2.2. Design Summary Form by Abstract and Title (Form 1)

To extract the initial information for filtering and selecting articles that may contain relevant research information, a form must be developed or constructed to provide a summary of the results. Part of Form 1 prepared for this study is presented in Table 8, and the complete database organized in this form is presented in Supplementary Material S1.
With Form 1, the first evaluation was made based on the title and abstract of the article. Two team members could conduct this evaluation independently to crosscheck the information and determine whether it entered the second stage. In the second stage, two team members analyzed the methodology and results of the papers chosen based on their abstracts and titles during the initial selection process. The second read determines the papers to be included in the final review.

3.2.3. Design of Summary Form 2 to Extract Essential Information from Selected Documents

After consolidating the information using Form 1 (Table 8), a more compact database with information closer to that required for research can be obtained. With this filtered database, the next step is to develop a more detailed reading of the documents to extract meaningful information for the study (methods, metrics, etc.), which must be grouped and organized in a new form that allows consolidation. This second form allowed further analysis of the evaluated topics. In this case study, the collection form contained four blocks (see Table 9) as follows:
  • Section 1: Presents a unique identifier for the document (the same identifier assigned in Form 1), retrieval link, year of publication, and the journal or event in which it was published.
  • Section 2: Information on the methods or methodologies applied. The idea is to list the methods applied by type.
  • Section 3: Contains specific research information, such as data sources, input and output variables, and the model used.
  • Section 4: Benchmarking (in some cases, there are no other models against which it is compared, but in many others, it is evaluated against another method to present strengths and improvements in the developed research), performance metrics, and a column for some observations that the researcher considers important to provide an orientation of what was found in the document review.

3.3. Application of the Initial Systematic Document Search and Extraction Protocol

3.3.1. Evolution of the Number of Documents Found in Each Phase

The initial number of documents satisfying the criteria outlined in numeral Section 3.2 (Table 1) was 1947 for Scopus and 5598 for WOS. The number of documents decreased when initial inclusion and exclusion criteria were applied (Table 10 shows how the number of documents selected decreases when applying different filters).
Both databases effectively eliminated a significant portion of the initial document pool, resulting in a final count of nearly 13% for Scopus and nearly 5% for the WOS of the initial documents obtained. This reduction in volume allows for a more focused and specialized approach to applying Form 1 and reading abstracts to better align with the research question.
As shown in Figure 3, an analysis of the occurrence of keywords was conducted using VOSviewer 1.6.20 which confirmed that this selection captured relevant documents that contained keywords related to the research question.
The specific discipline evaluated in this case study is uncertainty in hydrological forecasting (and possibly applications in meteorology and climatology); therefore, the areas of relevance of the results should be related to Environmental and Earth Sciences, Engineering, Water Resources, and Mathematics. The latter is a crosscutting and fundamental area of scientific development. Figure 4 shows that the documents found in this research on Scopus are categorized in most cases in the thematic area of Environmental Sciences (56.30%), while in WOS, they are categorized as Meteorology Atmospheric Sciences (64.00%).
A review by type of document (see Figure 5) shows that the most abundant form of disseminating research is through the publication of articles in recognized journals, followed to a much lesser extent by publications in congresses, seminars, or conferences. This percentage decreased with the application of the filters.
Reviewing the amount of document production by year (see Figure 6 and above Figure 2), in 2017, there was an ascending slope in research production on the research question topic. On the other hand, 2019 had the highest production (Scopus 16.91% and WOS 19.1%). The amounts for 2023 are evaluated for reference only because the searches were conducted on 15 October 2023, which means that the publication volume (or issues) of journals for the last quarter of the year has not yet been published; however, the documents evaluated correspond to the most current at the time of the search.
After applying the six filters and performing the analysis again with the VOSviewer 1.6.20 software, Figure 7 shows the evaluation according to the number of occurrences by keyword. This generates essential savings in the specific review work that follows this semi-automatic stage since the documents were found.
The final filter included keywords that were exclusively relevant to the research question, as presented in Section 3.1. This facilitates a comprehensive manual analysis in the subsequent stages.

3.3.2. Application Form 1

After performing semi-automatic filtering using the tools for each database, the resulting documents from each database were included in Form 1, and the expert knowledge filtering procedure began with a review of the title and abstract (see Figure 8). In this phase, duplicate items were identified in the two databases.
In the case of Scopus, 149 documents were selected after reviewing the titles and abstracts (7.65% of the initial search) from 256 records previously selected up to Filter 6. As for the WOS, of the 271 records resulting from Filter 6, 108 were selected after reviewing the title and abstract (1.93% of the initial search). Sixty-eight (68) documents were found to be cross-referenced in both databases after applying the semiautomatic filters. Finally, the information was consolidated into a single form, resulting in 217 distributed documents, as shown in Figure 9.

3.3.3. Retrieval of Selected Documents

The document retrieval process begins once the documents are selected from previous filters. It is important to note that, in all cases, the selected records were not open access. To overcome this limitation, this study used researchers’ credentials provided by the Spanish Foundation for Science and Technology (FECYT). These credentials allowed access to the selected databases, making document retrieval possible. Thus, the availability of relevant documents is guaranteed in order to continue their analysis and study.

3.3.4. Documents Final Selection from Scopus and WOS

During this stage (second read of documents), an exhaustive analysis of the methodologies used in each document was performed to evaluate their rigor and relevance to the research in progress. Similarly, the results obtained in each study were examined in detail, seeking evidence to support and contribute to building a solid body of knowledge regarding the research question.
This meticulous review allowed informed decisions to be made regarding the final inclusion or exclusion of papers in the study. Those that demonstrated a sound methodology and relevant results were selected as important background materials. Finally, 129 records were selected, distributed as shown in Figure 10, and grouped into two blocks according to the proposed forecast period: a first group of real-time up to one month (called short-to mid-term) and another one from one month onwards (called Mid- to Long-Term).

3.3.5. Documents Referenced by Colleagues and Cross Reference

At this stage, as mentioned in Section 2.2, consultations were made with colleagues who were experts on the subject (from the same group or other research centers) to obtain new articles referenced by their knowledge (Manual Google search or shared by a colleague). From this evaluation, references were brought to 40 research articles with the same criteria as those obtained from the databases, which were first filtered by abstract and title and then filtered by methodology and results. Finally, 32 documents were selected for this stage (see Figure 11).

3.3.6. Final Database Prepared with Documents Selected to Review

The final selection was 161 documents from the search and filtering of the databases (Scopus and WOS) and selection from colleagues’ recommendations. With this final selection, we will write the review or meta-analysis paper in the next stage (two reviews will be written in the next step of this study, one for the mid-to-long-term scale and the other for the short-to-mid-term scale). The information of this final database was consolidated in summary Form 2 (see above Table 9, and the complete results may be consulted in Supplementary Material S2).

4. Discussion

SLR constitutes studies that apply a structured methodology to synthesize findings from multiple primary studies that focus on a specific question [2,88,90]. A distinctive feature of these reviews is their adherence to a transparent, objective, and replicable methodology [2,100], which is very important in the research process. The literature on SLR has identified several critical steps in conducting a systematic review:
  • Formulation of a straightforward and delimited research question [2,88,90,99,100,101].
  • Developing a detailed protocol with criteria for the inclusion or exclusion of studies [2,90,100,101].
  • Conduction of a comprehensive and systematic literature search from a variety of sources [2,88,90,100].
  • Relevant studies should be selected using established criteria [2,90,100,101].
  • Systematic extraction and analysis of data [2,100,101].
  • Evaluation of the methodological quality of studies [2,90,100,101].
  • Quantitative or qualitative synthesis of the results [2,90,100,101].
Based on these premises, the methodology proposed in this study seeks to integrate the fundamental elements of a systematic review to improve the quality, transparency, and replicability of review studies and to evaluate the state-of-the-art in hydrology and other STEM disciplines.
It is relevant to note that traditional reviews of the state-of-the-art in hydrology often present a narrative approach that can be subject to problems of subjectivity and lack of transparency in selecting reference studies [87]. These limitations stem from the fact that authors may inadvertently assign greater weight to certain studies than others without following clear criteria or stating the rationale for their decisions. In contrast, a meta-analysis approach allows the existing literature to be reviewed more objectively and quantitatively, following the structured processes of search, selection, quality assessment, and statistical analysis. This opens up the possibility of exploring new research questions and relationships between variables that would not be feasible to address in individual studies or in those in which references have been subjectively selected, thus avoiding introducing bias into the research.
To seek impartiality in the review process, the proposed methodology suggests that the filters used in the databases should not be restricted by the names or impact factors of authors and scientific journals. In addition, by not stating the study’s authors in Abstract Form 1, researchers are invited to focus on the initial selection of documents based only on the title and abstract. This contributes to the fact that there is no bias in selection and facilitates a careful evaluation of the heterogeneity of the documents.
SLR is a methodology with growing acceptance in various scientific disciplines, owing to its ability to perform comprehensive, objective, and transparent evaluations of a research topic. Traditional narrative reviews are the predominant approaches used in hydrology. For example, the classic article by Linsley [6] deals with an extensive 300-year review of the development of the rainfall–runoff relationship. Although valuable for its breadth, the approach does not present criteria for the inclusion of references, and therefore, it is essentially subjective in selection and analysis. Another similar case is “A review of single-site models for monthly streamflow generation” [29], which compares four models for the generation of monthly streamflow series selected without an objective protocol and concludes with subjective recommendations.
Mosavi et al. [21] presented an article notable for its narrative approach, addressing machine learning techniques applied to flood forecasting. It provides a broad overview of important machine learning (ML) methodologies, such as ANN, ANFIS, and SVM. Kambalimath and Deka [23] provided an overview of fuzzy logic applications in hydrology and water resources. However, similar to a previous study, a clear literature review protocol is lacking.
In contrast to these articles, “A review of AI methods for the prediction of high-flow extreme hydrology” [19] presents a review with a qualitative approach to artificial intelligence methods for the prediction of extreme hydrological events, which is quite exhaustive in terms of the description of the different techniques. However, similar to previous studies, it provides a critical shortcoming in the methodology used for identifying and evaluating reference studies as it does not present a systematic protocol for the literature review. This may have led to selection bias in the inclusion of the analyzed studies.
The lack of detail on how studies were identified, selected, and analyzed may affect the objectivity and reliability of the results. The lack of a structured and transparent review process limits readers’ ability to adequately assess the quality of the synthesis and its application in future research and practice.
Another example of this type of review article is that of Rhif et al. [22], who presented a detailed review of the applications of wavelet transforms for the analysis of nonstationary time series in various applied scientific fields. However, as is the case at the general level, there is a gap in the methodology for selecting reference papers; the protocol followed for searching, setting, and extracting information from the relevant literature has not been clearly and explicitly described. There was no mention of systematic inclusion/exclusion criteria or structured database search strategy, nor is it a standardized process for assessing the methodological quality and relevance of the studies described. This standard practice in hydrology and STEM can introduce subjectivity and bias into the selection and synthesis of the reference literature.
A very good review is a paper that offers a comprehensive review of 40 years of research on the generation of flow forecasts from an analysis of more than 700 studies [47]. This study classifies forecasting systems into three main categories: Statistics-Based Streamflow Prediction (SBSP), Ensemble Streamflow Prediction (ESP), and Ensemble Prediction Systems (EPS). Detailed technical information, strengths, weaknesses, and applications are also provided. It also discusses the main aspects of forecasting systems, such as uncertainty analysis, data assimilation, model combination, post-processing, and verification. This article discusses the current successes, challenges, and future aspirations in the field and concludes that further collaboration between scientific and operational communities is needed to advance the implementation of these systems. However, it lacks a clear presentation of the process employed in the literature review and selection of documents used in the analysis. It contains robust and valuable information. However, the lack of methodology significantly hinders its reproducibility.
Ghobadi and Kang [18] developed an SLR to apply machine learning to water resource management. They found that machine learning has been widely used because of its ability to handle large datasets and model complex systems. The authors classified the main machine learning techniques into three categories: prediction, clustering, reinforcement learning, and high-lighting–less studied areas, such as spatiotemporal and geo-spatiotemporal challenges in water resource management. They concluded that challenges remain in consistently incorporating spatiotemporal dimensions into models and developing probabilistic approaches that capture uncertainties. However, despite this excellent review, the research protocol was not transparent and replicable, as the search criteria used were not presented.
In view of this general gap in hydrology (and STEM areas), this study proposes a structured and objective methodology, which includes defining the research question a priori, establishing systematic eligibility criteria, conducting broad searches in multiple databases with clearly defined terms, applying standardized forms for data extraction and analysis, and synthesizing the evidence in a transparent and reproducible manner. It is clear that the researchers’ studies will always have a subjective touch because the comments are made from a professional, scientific perspective and from the experience of the authors, but if another author decides to replicate the study, he/she may find in his/her search that the reference documents are the same or practically the same, which will lead him/her to debate the results of his/her colleagues or to confirm them, but it opens the opportunity for the review and evaluation studies of the state-of-the-art to be objective and replicable in the search of the references.
In this study, examples of review articles were selected using the same databases as the case study, following the methodology for filtering them. However, to extract the reference sample, a random selection was made only to present cases that illustrate the large number of review studies that exist in branches related to hydrology. Thus, some were taken to present and explain in this part of the discussion, which was mentioned by Evaristo and McDonnell [87] as situations of subjectivity and the lack of a transparent procedure for the selection of reference studies in most (if not all) existing review articles is the common denominator.
Table 11 summarizes some articles selected as random samples of what is found at a general level, as mentioned in traditional review articles. Most of them did not comply with the available protocol or the search, analysis, or selection of references, implying that they are unlikely to be replicable. Sit et al. [25] and Yildirim et al. [102] compiled relatively straightforward and replicable protocols. Bibliometrics were performed for the first case. Without a doubt, they apply an established protocol of search, filtering, and selection of the documents, reaching at the end the objective of making a bibliometric measurement of the subject in question; in the second case, it is good research, it presents in good measure some steps of a complete protocol, with some shortcomings in some stages. Nevertheless, they provide an excellent and broadly replicable analysis and synthesis of the selected documents, which provides reliability and transparency for this review or evaluation of the state-of-the-art methods.
The methodological approach developed in this article considers policies through a systematized methodological proposal for SLR in hydrology. This methodology contemplates transparent processes of search, selection, analysis, and synthesis of evidence, thereby overcoming the limitations of traditional and more subjective approaches. In other words, it represents a relevant contribution to improving the quality and rigor of literature reviews in this discipline. SLR can strengthen hydrological research by promoting an objective synthesis of existing knowledge on various topics.
While it should be recognized that narrative approaches have been valuable in providing an overview of knowledge in hydrology, the methodology developed in this study offers a more robust and transparent alternative. A systematic approach based on predefined criteria minimizes bias and promotes a more objective literature review. It is important to note that this methodology does not seek to detract from the validity of the narrative approaches used in the past, as it has been valuable in providing an overview of existing knowledge in hydrology. However, given the growing demand for more rigorous and transparent research and the large volume of information in databases, the proposed methodology is presented as a solid and reliable alternative for developing a state-of-the-art method that responds to the research question posed, perhaps by compiling the most and best information available.
It should be noted that adopting this methodology does not imply a limitation in the interpretation of the results or total automation because the inclusion of clear selection criteria and the use of an exhaustive search strategy is the task of the research team.

5. Conclusions

This methodology has been designed to be applicable in several studies in hydrological sciences and other related fields, providing a structured and objective approach that improves the quality and reliability of the references obtained in the literature review. With many publications and scientific developments, manual searching and selection is complex because of the large volume of information. By adopting this methodology, researchers can move toward a more solid and transparent scientific practice, which is fundamental for the advancement of the discipline and for informed decision-making. Through the application of this methodology, several advantages and benefits that contribute to scientific advancement and knowledge development in this discipline were identified. The main conclusions are as follows:
  • Improves the quality and thoroughness of the review papers and state-of-the-art evaluations: The proposed methodology ensures that the review process is transparent, replicable, and objective by establishing clear selection criteria, exhaustive search strategies, and structured summary forms, guaranteeing the inclusion of relevant studies and the systematic extraction of relevant data, which strengthens the quality and reliability of the results.
  • Promotes transparency and replicability and emphasizes openness at all stages of the review process. This allows other researchers to follow the same steps and reproduce the results, thereby facilitating the validation and comparison of the studies. In addition, the methodology provides clear and detailed documentation of the methods used, allowing for a more accurate and reliable assessment.
  • Exhaustive identification of the relevant literature: This methodology ensures a broad and thorough search of the existing literature. By using reliable databases and establishing appropriate search strategies, the collection of relevant studies is maximized, which allows obtaining a broad and updated view of the state-of-the-art in the study area.
  • Foster collaboration and knowledge sharing: This methodology fosters collaboration among researchers and facilitates knowledge sharing within a scientific community. This methodology provides a clear and structured framework that allows researchers to share and compare their results, enrich scientific progress, and stimulate new research.
Traditional literature reviews frequently lack transparency and replicability in terms of study selection, which introduces bias and subjectivity. Such reviews also often lack clear eligibility criteria and a structured search strategy, and critical evaluation of the methodological quality and relevance of the included studies is frequently neglected. This can affect the reliability of conclusions. Most literature reviews are narrative in their synthesis of results rather than employing quantitative methods, such as meta-analysis. By contrast, the proposed methodology introduces objectivity by following a predefined and standardized protocol that reduces bias:
  • Allows comprehensive and reproducible mapping of the relevant literature on research questions.
  • The quality and relevance of the studies are critically evaluated before their synthesis.
  • The evidence is synthesized quantitatively when it is feasible to maximize rigor.
  • This provides excellent reliability and robustness to the conclusions obtained, thereby improving the quality of reviews.

Supplementary Materials

The following supporting information can be downloaded at: https://www.mdpi.com/2073-4441/16/3/436/s1, S1 contains the summary form number 1; S2 contains the summary form number 2; S3 shows the distribution by research area and type of document selected in each semi-automatic filter applied.

Author Contributions

Conceptualization, D.D.L.P., S.S.G., J.Á.A. and F.F.G.; Methodology, D.D.L.P.; Validation, R.A.V., S.S.G. and F.F.G.; Formal Analysis, D.D.L.P. and S.S.G.; Investigation, D.D.L.P. and R.A.V.; Data Curation, D.D.L.P.; Writing—Original Draft, D.D.L.P.; Writing—Review and Editing, D.D.L.P., R.A.V., S.S.G., J.Á.A. and F.F.G.; Visualization, D.D.L.P.; Supervision, S.S.G. and F.F.G.; Project Administration, F.F.G.; Funding Acquisition, D.D.L.P. and F.F.G. All authors have read and agreed to the published version of the manuscript.

Funding

This study was funded by the Colombian Ministry of Science, Technology, and Innovation (MINCIENCIAS) through the Call for Doctorates Abroad 885-2; the Valencian Regional Government through the WATER4CAST research project (PROMETEO/2021/074); and the Spanish Ministry of Science and Innovation through the research project TETISPREDICT (PID2022-141631OB-I00). Funding for open access charge, Universitat Politècnica de València.

Data Availability Statement

This study did not use sensitive or restricted data. The case study data can be extracted from the SCOPUS and WOS databases (with the access credentials of the Universitat Politècnica de València and FECyT) under the search conditions proposed in this methodology. The corresponding analysis can be found in the Supplementary Material.

Acknowledgments

The authors wish to acknowledge Universitat Politècnica de València for access to the scientific databases used in the systematic literature review. The authors express their gratitude to the three anonymous reviewers, the editor, and his assistant for their prompt and thorough review of this research article.

Conflicts of Interest

The authors declare no conflicts of interest.

Correction Statement

This article has been republished with a minor correction to the Funding statement. This change does not affect the scientific content of the article.

References

  1. Cochrane.org. What Are Systematic Reviews? Available online: https://www.cochrane.org/our-evidence/what-are-systematic-reviews (accessed on 15 January 2023).
  2. Siddaway, A.P.; Wood, A.M.; Hedges, L.V. How to Do a Systematic Review: A Best Practice Guide for Conducting and Reporting Narrative Reviews, Meta-Analyses, and Meta-Syntheses. Annu. Rev. Psychol. 2018, 70, 747–770. [Google Scholar] [CrossRef]
  3. Hallock, W. Hydraulic illustration of the Wheatstone Bridge. Science 1898, 8, 206–208. [Google Scholar] [CrossRef]
  4. Piper, A.M. The origin of Artesian pressure. Econ. Geol. 1928, 23, 683–696. [Google Scholar] [CrossRef]
  5. Breusse, J.J. Modern geophysical methods for subsurface water exploration. Geophysics 1963, 28, 633–657. [Google Scholar] [CrossRef]
  6. Linsley, R.K. The relation between rainfall and runoff: Review paper. J. Hydrol. 1967, 5, 297–311. [Google Scholar] [CrossRef]
  7. Perrault, P. Treatise on the Origin of Springs; Ilustrada; Hafner Publishing Company: New York, NY, USA, 1967; ISBN 9780028502007. [Google Scholar]
  8. Yamamoto, S. Current studies on hydrology in Japan. Geogr. Rev. Jpn. 1972, 45, 163–171. [Google Scholar] [CrossRef]
  9. Dyring, M.; Hofmann, H.; Stanton, D.; Moss, P.; Froend, R. Ecohydrology of coastal aquifers in humid environments and implications of a drying climate. Ecohydrology 2023, 16, e2491. [Google Scholar] [CrossRef]
  10. Stein, E.D.; Gee, E.M.; Adams, J.B.; Irving, K.; Van Niekerk, L. Advancing the science of environmental flow management for protection of temporarily closed estuaries and coastal lagoons. Water 2021, 13, 595. [Google Scholar] [CrossRef]
  11. Wei, X.; Zhou, X.; Wang, C. The influence of mountain temperate forests on the hydrology in northeast China. For. Chron. 2003, 79, 297–300. [Google Scholar] [CrossRef]
  12. Wullschleger, S.D.; Meinzer, F.C.; Vertessy, R.A. A review of whole-plant water use studies in tree. Tree Physiol. 1998, 18, 499–512. [Google Scholar] [CrossRef]
  13. Yin, J.; Porporato, A.; D’Odorico, P.; Rodríguez-Iturbe, I. Ecohydrology. In Encyclopedia of Water; Wiley: Hoboken, NJ, USA, 2019; pp. 1–21. [Google Scholar] [CrossRef]
  14. Rinaldo, A.; Rodriguez-Iturbe, I. Ecohydrology 2.0. Rend. Lincei 2022, 33, 245–270. [Google Scholar] [CrossRef]
  15. Moniruzzaman, M.; Chowdhury, M.S.H.; Saha, D.; Billah, M.M.; Helal, A.; Biswash, R.A. Hydrokinetic Turbine Technology and Its Prospect in Bangladesh: A Review. Aust. J. Eng. Innov. Technol. 2022, 4, 01007. [Google Scholar] [CrossRef]
  16. Anctil, F.; Lauzon, N. Generalisation for neural networks through data sampling and training procedures, with applications to streamflow predictions. Hydrol. Earth Syst. Sci. 2004, 8, 940–958. [Google Scholar] [CrossRef]
  17. Chen, Y.; Han, D. Big data and hydroinformatics. J. Hydroinformatics 2016, 18, 599–614. [Google Scholar] [CrossRef]
  18. Ghobadi, F.; Kang, D. Application of Machine Learning in Water Resources Management: A Systematic Literature Review. Water 2023, 15, 620. [Google Scholar] [CrossRef]
  19. Hamitouche, M.; Molina, J.L. A Review of AI Methods for the Prediction of High-Flow Extremal Hydrology. Water Resour. Manag. 2022, 36, 3859–3876. [Google Scholar] [CrossRef]
  20. Mohammadi, B. A review on the applications of machine learning for runoff modeling. Sustain. Water Resour. Manag. 2021, 7, 98. [Google Scholar] [CrossRef]
  21. Mosavi, A.; Ozturk, P.; Chau, K.W. Flood prediction using machine learning models: Literature review. Water 2018, 10, 1536. [Google Scholar] [CrossRef]
  22. Rhif, M.; Abbes, A.B.; Farah, I.R.; Martínez, B.; Sang, Y. Wavelet transform application for/in non-stationary time-series analysis: A review. Appl. Sci. 2019, 9, 1345. [Google Scholar] [CrossRef]
  23. Kambalimath, S.; Deka, P.C. A basic review of fuzzy logic applications in hydrology and water resources. Appl. Water Sci. 2020, 10, 191. [Google Scholar] [CrossRef]
  24. Simmonds, J.; Gómez, J.A.; Ledezma, A. The role of agent-based modeling and multi-agent systems in flood-based hydrological problems: A brief review. J. Water Clim. Chang. 2019, 11, 1580–1602. [Google Scholar] [CrossRef]
  25. Sit, M.; Demiray, B.Z.; Xiang, Z.; Ewing, G.J.; Sermet, Y.; Demir, I. A comprehensive review of deep learning applications in hydrology and water resources. Water Sci. Technol. 2020, 82, 2635–2670. [Google Scholar] [CrossRef]
  26. Gentine, P.; Troy, T.J.; Lintner, B.R.; Findell, K.L. Scaling in Surface Hydrology: Progress and Challenges. J. Contemp. Water Res. Educ. 2012, 147, 28–40. [Google Scholar] [CrossRef]
  27. Matsuyama, H. Review and Prospect on the Field of Hydrology and Water Resources Contributed by Geographers in Recent 30 Years. J. Jpn. Soc. Hydrol. Water Resour. 2018, 31, 467–486. [Google Scholar] [CrossRef]
  28. Merheb, M.; Moussa, R.; Abdallah, C.; Colin, F.; Perrin, C.; Baghdadi, N. Hydrological response characteristics of Mediterranean catchments at different time scales: A meta-analysis. Hydrol. Sci. J. 2016, 61, 2520–2539. [Google Scholar] [CrossRef]
  29. Phien, H.N.; Ruksasilp, W. A review of single-site models for monthly streamflow generation. J. Hydrol. 1981, 52, 1–12. [Google Scholar] [CrossRef]
  30. Singh, V.P. Review of Hydrology for Engineers, Geologists, and Environmental Professionals by Sergio E. Serrano. J. Hydrol. Eng. 2011, 16, 846. [Google Scholar] [CrossRef]
  31. Di Baldassarre, G.; Elshamy, M.; Van Griensven, A.; Soliman, E.; Kigobe, M.; Ndomba, P.; Mutemi, J.; Mutua, F.; Moges, S.; Xuan, Y.; et al. Hydrologie et climat futurs dans le bassin du Nil: Une revue. Hydrol. Sci. J. 2011, 56, 199–211. [Google Scholar] [CrossRef]
  32. Dinu, M.J.; Siddaramaiah, D.G. Uncertainties in predicting impacts of climate change on hydrology in basin scale: A review. Arab. J. Geosci. 2020, 13, 1037. [Google Scholar] [CrossRef]
  33. Mortsch, L.; Hengeveld, H.; Lister, M.; Lofgren, B.; Quinn, F.; Slivitzky, M.; Wenger, L. Climate change impacts on the hydrology of the great lakes-st. Lawrence system. Can. Water Resour. J. 2000, 25, 153–179. [Google Scholar] [CrossRef]
  34. Stadnyk, T.A.; Déry, S.J. Canadian continental-scale hydrology under a changing climate: A review. Water 2021, 13, 906. [Google Scholar] [CrossRef]
  35. Wong, K.V.; Lennon, C. Innovations Related to Hydrology in Response to Climate Change—A Review. Open Hydrol. J. 2015, 9, 17–23. [Google Scholar] [CrossRef]
  36. Bobba, A.G.; Rudra, R.P.; Diiwu, J.Y. A stochastic model for identification of trends in observed hydrological and meteorological data due to climate change in watersheds. J. Environ. Hydrol. 2006, 14, 1–11. [Google Scholar]
  37. Bolgov, M.V. Stochastic hydrology: The development of main ideas in Russia. Inst. Water Probl. Russ. Acad. Sci. 2021, 66, 19–40. [Google Scholar] [CrossRef]
  38. Matalas, N.C. Developments in stochastic hydrology. Rev. Geophys. 1975, 13, 67–73. [Google Scholar] [CrossRef]
  39. Mizuki, C.; Kuzuha, Y. Frequency Analysis of Hydrological Data for Urban Floods—Review of Traditional Methods and Recent Developments, Especially an Introduction of Japanese Proper Methods. Water 2023, 15, 2490. [Google Scholar] [CrossRef]
  40. Beven, K.; Binley, A. GLUE: 20 years on. Hydrol. Process 2014, 28, 5897–5918. [Google Scholar] [CrossRef]
  41. Georgakakos, K.P.; Kavvas, M.L. Precipitation analysis, modeling, and prediction in hydrology. Rev. Geophys. 1987, 25, 163–178. [Google Scholar] [CrossRef]
  42. Koutsoyiannis, D.; Makropoulos, C.; Langousis, A.; Baki, S.; Efstratiadis, A.; Christofides, A.; Karavokiros, G.; Mamassis, N. HESS opinions: “Climate, hydrology, energy, water: Recognizing uncertainty and seeking sustainability”. Hydrol. Earth Syst. Sci. 2009, 13, 247–257. [Google Scholar] [CrossRef]
  43. Li, W.; Duan, Q.; Miao, C.; Ye, A.; Gong, W.; Di, Z. A review on statistical postprocessing methods for hydrometeorological ensemble forecasting. Wiley Interdiscip. Rev. Water 2017, 4, e1246. [Google Scholar] [CrossRef]
  44. Liu, Y.; Gupta, H.V. Uncertainty in hydrologic modeling: Toward an integrated data assimilation framework. Water Resour. Res. 2007, 43, W07401. [Google Scholar] [CrossRef]
  45. Moges, E.; Demissie, Y.; Larsen, L.; Yassin, F. Review: Sources of hydrological model uncertainties and advances in their analysis. Water 2021, 13, 28. [Google Scholar] [CrossRef]
  46. Teng, J.; Jakeman, A.J.; Vaze, J.; Croke, B.F.W.; Dutta, D.; Kim, S. Flood inundation modelling: A review of methods, recent advances and uncertainty analysis. Environ. Model. Softw. 2017, 90, 201–216. [Google Scholar] [CrossRef]
  47. Troin, M.; Arsenault, R.; Wood, A.W.; Brissette, F.; Martel, J.L. Generating Ensemble Streamflow Forecasts: A Review of Methods and Approaches Over the Past 40 Years. Water Resour. Res. 2021, 57, e2020WR028392. [Google Scholar] [CrossRef]
  48. Vannitsem, S.; Bremnes, J.B.; Demaeyer, J.; Evans, G.R.; Flowerdew, J.; Hemri, S.; Lerch, S.; Roberts, N.; Theis, S.; Atencia, A.; et al. Statistical postprocessing for weather forecasts review, challenges, and avenues in a big data world. Bull. Am. Meteorol. Soc. 2021, 102, E681–E699. [Google Scholar] [CrossRef]
  49. Wetterhall, F.; Pappenberger, F.; Alfieri, L.; Cloke, H.L.; Thielen-Del Pozo, J.; Balabanova, S.; Daňhelka, J.; Vogelbacher, A.; Salamon, P.; Carrasco, I.; et al. HESS Opinions “forecaster priorities for improving probabilistic flood forecasts”. Hydrol. Earth Syst. Sci. 2013, 17, 4389–4399. [Google Scholar] [CrossRef]
  50. Arenson, L.U.; Harrington, J.S.; Koenig, C.E.M.; Wainstein, P.A. Mountain Permafrost Hydrology–Practical Review Following Studies from the Andes. Geosciences 2022, 12, 48. [Google Scholar] [CrossRef]
  51. Beria, H.; Larsen, J.R.; Ceperley, N.C.; Michelon, A.; Vennemann, T.; Schaefli, B. Understanding snow hydrological processes through the lens of stable water isotopes. WIREs Water 2018, 5, e1311. [Google Scholar] [CrossRef]
  52. Mohammadzadeh Khani, H.; Kinnard, C.; Lévesque, E. Historical Trends and Projections of Snow Cover over the High Arctic: A Review. Water 2022, 14, 587. [Google Scholar] [CrossRef]
  53. Walvoord, M.A.; Kurylyk, B.L. Hydrologic Impacts of Thawing Permafrost—A Review. Vadose Zone J. 2016, 15, vzj2016.01.0010. [Google Scholar] [CrossRef]
  54. Amani, M.; Ghorbanian, A.; Ahmadi, S.A.; Kakooei, M.; Moghimi, A.; Mirmazloumi, S.M.; Moghaddam, S.H.A.; Mahdavi, S.; Ghahremanloo, M.; Parsian, S.; et al. Google Earth Engine Cloud Computing Platform for Remote Sensing Big Data Applications: A Comprehensive Review. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2020, 13, 5326–5350. [Google Scholar] [CrossRef]
  55. Devendran, A.A.; Lakshmanan, G. A Review On Accuracy and Uncertainty of Spatial Data and Analyses with special reference to Urban and Hydrological Modelling. ISPRS Ann. Photogramm. Remote Sens. Spat. Inf. Sci. 2014, II-8, 171–178. [Google Scholar] [CrossRef]
  56. Evora, N.D.; Coulibaly, P. Recent advances in data-driven modeling of remote sensing applications in hydrology. J. Hydroinformatics 2009, 11, 194–201. [Google Scholar] [CrossRef]
  57. Rango, A. Application of remote sensing methods to hydrology and water resources. Hydrol. Sci. J. 1994, 39, 309–320. [Google Scholar] [CrossRef]
  58. Wu, X.; Lu, G.; Wu, Z. Remote Sensing Technology in the Construction of Digital Twin Basins: Applications and Prospects. Water 2023, 15, 2040. [Google Scholar] [CrossRef]
  59. Kale, V.S. Fluvial hydrology and geomorphology of Monsoon-dominated Indian rivers. Rev. Bras. De Geomorfol. 2005, 6, 63–73. [Google Scholar] [CrossRef]
  60. Wymore, A.S.; Ward, A.S.; Wohl, E.; Harvey, J.W. Viewing river corridors through the lens of critical zone science. Front. Water 2023, 5, 1147561. [Google Scholar] [CrossRef]
  61. Blair, P.; Buytaert, W. Socio-hydrological modelling: A review asking “why, what and how”? Hydrol. Earth Syst. Sci. 2016, 20, 443–478. [Google Scholar] [CrossRef]
  62. Khilchevskyi, V. Socio-hydrology: The pros and cons of a new interdisciplinary field of research. Bull. Taras Shevchenko Natl. Univ. Kyiv Geogr. 2021, 78–79, 9–13. [Google Scholar] [CrossRef]
  63. Pande, S.; Sivapalan, M. Progress in socio-hydrology: A meta-analysis of challenges and opportunities. WIREs Water 2017, 4, e1193. [Google Scholar] [CrossRef]
  64. Watts, G. Hydrology with impact: How does hydrological science inform decision-makers? Hydrol. Res. 2015, 47, 545–551. [Google Scholar] [CrossRef]
  65. DiCarlo, D.A. Stability of gravity-driven multiphase flow in porous media: 40 Years of advancements. Water Resour. Res. 2013, 49, 4531–4544. [Google Scholar] [CrossRef]
  66. Frenelus, W.; Peng, H.; Zhang, J. Evaluation methods for groundwater inflows into rock tunnels: A state-of-the-art review. Int. J. Hydrol. 2021, 5, 152–168. [Google Scholar] [CrossRef]
  67. Hermans, T.; Goderniaux, P.; Jougnot, D.; Fleckenstein, J.H.; Brunner, P.; Nguyen, F.; Linde, N.; Huisman, J.A.; Bour, O.; Lopez Alvis, J.; et al. Advancing measurements and representations of subsurface heterogeneity and dynamic processes: Towards 4D hydrogeology. Hydrol. Earth Syst. Sci. 2023, 27, 255–287. [Google Scholar] [CrossRef]
  68. Molz, F.J.; Liu, H.H.; Szulga, J. Fractional Brownian motion and fractional Gaussian noise in subsurface hydrology: A review, presentation of fundamental properties, and extensions. Water Resour. Res. 1997, 33, 2273–2286. [Google Scholar] [CrossRef]
  69. Nwachukwu, M.A. Prospective techniques for in-situ treatment and protection of aquifers: A sustainable hydrology review. Int. J. Water Resour. Environ. Eng. 2014, 6, 131–143. [Google Scholar] [CrossRef]
  70. Too, V.K.; Omuto, C.T.; Biamah, E.K.; Obiero, J.P. Review of Soil Water Retention Characteristic (SWRC) Models between Saturation and Oven Dryness. Open J. Mod. Hydrol. 2013, 4, 173–182. [Google Scholar] [CrossRef]
  71. Ma, Y.J.; Li, X.Y.; Guo, L.; Lin, H. Hydropedology: Interactions between pedologic and hydrologic processes across spatiotemporal scales. Earth Sci. Rev. 2017, 171, 181–195. [Google Scholar] [CrossRef]
  72. Van-Huyssteen, C.W. A review of advances in hydropedology for application in South Africa. S. Afr. J. Plant Soil 2008, 25, 245–254. [Google Scholar] [CrossRef]
  73. Chris, P. River Discharge to the Coastal Ocean: A Global Synthesis. Oceanography 2011, 24, 143–144. [Google Scholar] [CrossRef]
  74. Gröger, M.; Dieterich, C.; Haapala, J.; Ho-Hagemann, H.T.M.; Hagemann, S.; Jakacki, J.; May, W.; Meier, H.E.M.; Miller, P.A.; Rutgersson, A.; et al. Coupled regional Earth system modeling in the Baltic Sea region. Earth Syst. Dyn. 2021, 12, 939–973. [Google Scholar] [CrossRef]
  75. Guo, Y.; Zhang, Y.; Zhang, L.; Wang, Z. Regionalization of hydrological modeling for predicting streamflow in ungauged catchments: A comprehensive review. Wiley Interdiscip. Rev. Water 2021, 8, e1487. [Google Scholar] [CrossRef]
  76. He, Y.; Bárdossy, A.; Zehe, E. A review of regionalisation for continuous streamflow simulation. Hydrol. Earth Syst. Sci. 2011, 15, 3539–3553. [Google Scholar] [CrossRef]
  77. Arnbjerg-Nielsen, K.; Willems, P.; Olsson, J.; Beecham, S.; Pathirana, A.; Bülow Gregersen, I.; Madsen, H.; Nguyen, V.-T.V. Impacts of climate change on rainfall extremes and urban drainage systems: A review. Water Sci. Technol. 2013, 68, 16–28. [Google Scholar] [CrossRef]
  78. Burn, S.; Maheepala, S.; Sharma, A. Utilising integrated urban water management to assess the viability of decentralised water solutions. Water Sci. Technol. 2012, 66, 113–121. [Google Scholar] [CrossRef]
  79. Ferrans, P.; Torres, M.N.; Temprano, J.; Rodríguez Sánchez, J.P. Sustainable Urban Drainage System (SUDS) modeling supporting decision-making: A systematic quantitative review. Sci. Total Environ. 2022, 806, 150447. [Google Scholar] [CrossRef]
  80. Kourtis, I.M.; Tsihrintzis, V.A. Adaptation of urban drainage networks to climate change: A review. Sci. Total Environ. 2021, 771, 145431. [Google Scholar] [CrossRef]
  81. Kuttler, W. Urban climate. Part 1: Fundamentals and causes [Stadtklima. Teil 1: Grundzüge und ursachen]. Umweltwissenschaften Und Schadst. Forsch. 2004, 16, 187–199. [Google Scholar] [CrossRef]
  82. Mitchell, V.G.; McMahon, T.A.; Mein, R.G. Components of the Total Water Balance of an Urban Catchment. Environ. Manag. 2003, 32, 735–746. [Google Scholar] [CrossRef] [PubMed]
  83. Njue, N.; Stenfert Kroese, J.; Gräf, J.; Jacobs, S.R.; Weeser, B.; Breuer, L.; Rufino, M.C. Citizen science in hydrological monitoring and ecosystem services management: State of the art and future prospects. Sci. Total Environ. 2019, 693, 133531. [Google Scholar] [CrossRef] [PubMed]
  84. Sto Domingo, N.D.; Refsgaard, A.; Mark, O.; Paludan, B. Flood analysis in mixed-urban areas reflecting interactions with the complete water cycle through coupled hydrologic-hydraulic modelling. Water Sci. Technol. 2010, 62, 1386–1392. [Google Scholar] [CrossRef]
  85. Urich, C.; Rauch, W. Modelling the urban water cycle as an integrated part of the city: A review. Water Sci. Technol. 2014, 70, 1857–1872. [Google Scholar] [CrossRef]
  86. Walker, K.P. Urban runoff and combined sewer overflow. Water Environ. Res. 1992, 64, 408–411. [Google Scholar] [CrossRef]
  87. Evaristo, J.; McDonnell, J.J. A role for meta-analysis in hydrology. Hydrol. Process 2017, 31, 3588–3591. [Google Scholar] [CrossRef]
  88. Beltrán, G.Ó.A. Revisiones sistemáticas de la literatura. Rev. Colomb. Gastroenterol. 2005, 20, 60–69. [Google Scholar]
  89. García-Peñalvo, F.J.; García-Holgado, A. Técnicas Para Llevar a Cabo Mapeos y Revisiones Sistemáticas de la Literatura. 2021. Available online: https://zenodo.org/records/4732089 (accessed on 15 October 2023).
  90. Manterola, C.; Astudillo, P.; Arias, E.; Claros, N. Revisiones sistemáticas de la literatura. Qué se debe saber acerca de ellas. Cir. Esp. 2013, 91, 149–155. [Google Scholar] [CrossRef]
  91. Espinosa-Santos, V. Difusion y divulgacion de la investigacion cientifica. Idesia 2010, 28, 5–6. [Google Scholar] [CrossRef]
  92. Rodríguez, M.B.; Quintero Ortega, S.P.; Zafra, S.L. La revisión sistemática de la literatura científica y la necesidad de visualizar los resultados de las investigaciones. Rev. Logos Cienc. Tecnol. 2015, 7, 101–103. [Google Scholar] [CrossRef]
  93. Fourez, G. Alfabetizacion Cientifica y Tecnologica: Acerca de las Finalidades de la Ensenanza de las Ciencias; Buenos Aires: Ediciones Colihue SRL, Ed.; Nuevos Caminos; Reimpresa; Colihue; Colihue: Buenos Aires, Argentina, 1997; ISBN 9789505816378. [Google Scholar]
  94. Kitchenham, B.; Brereton, P.; Turner, M.; Niazi, M.; Linkman, S.; Pretorius, R.; Budgen, D. The Impact of Limited Search Procedures for Systematic Literature Reviews—A Participant-Observer Case Study. In Proceedings of the ESEM 2009: 2009 3rd International Symposium on Empirical Software Engineering and Measurement, Lake Buena Vista, FL, USA, 15–16 October 2009. [Google Scholar] [CrossRef]
  95. Elsevier, B.V. Scopus® Scopus Preview; Elsevier B.V.: Amsterdam, The Netherlands; Available online: https://www.scopus.com (accessed on 15 October 2023).
  96. Clarivate Analytics. Web Of Science®; Claivate Analytics: London, UK; Available online: https://www.webofscience.com/wos (accessed on 15 October 2023).
  97. Centre for Science and Technology Studies. VOSviewer-Visualizing Scientific Landscapes; Leiden University: Leiden, The Netherlands; Available online: https://www.vosviewer.com/download (accessed on 20 March 2023).
  98. Muka, T.; Glisic, M.; Milic, J.; Verhoog, S.; Bohlius, J.; Bramer, W.; Chowdhury, R.; Franco, O.H. A 24-step guide on how to design, conduct, and successfully publish a systematic review and meta-analysis in medical research. Eur. J. Epidemiol. 2020, 35, 49–60. [Google Scholar] [CrossRef]
  99. Kitchenham, B. Procedures for Performing Systematic Reviews; Keele University: Keele, UK, 2004; Volume 33, pp. 1–26. [Google Scholar]
  100. Nguyen, N.; Singh, S. A Primer on Systematic Reviews and Meta-Analyses. Semin. Liver Dis. 2018, 38, 103–111. [Google Scholar] [CrossRef]
  101. Kitchenham, B.; Pearl Brereton, O.; Budgen, D.; Turner, M.; Bailey, J.; Linkman, S. Systematic literature reviews in software engineering—A systematic literature review. Inf. Softw. Technol. 2009, 51, 7–15. [Google Scholar] [CrossRef]
  102. Yildirim, G.; Rahman, A.; Singh, V.P. A Bibliometric Analysis of Drought Indices, Risk, and Forecast as Components of Drought Early Warning Systems. Water 2022, 14, 253. [Google Scholar] [CrossRef]
Water 16 00436 g001
Figure 1. Flowchart of the proposed SLR methodology.
Figure 1. Flowchart of the proposed SLR methodology.
Water 16 00436 g001
Water 16 00436 g002
Figure 2. Number of documents found in the initial search, totaled by year.
Figure 2. Number of documents found in the initial search, totaled by year.
Water 16 00436 g002
Water 16 00436 g003aWater 16 00436 g003b
Figure 3. Reported keyword occurrences in the initial search: Upper Scopus and Lower WOS.
Figure 3. Reported keyword occurrences in the initial search: Upper Scopus and Lower WOS.
Water 16 00436 g003aWater 16 00436 g003b
Water 16 00436 g004aWater 16 00436 g004b
Figure 4. Number of documents by subject and research area obtained for each step: Upper Scopus and Lower WOS.
Figure 4. Number of documents by subject and research area obtained for each step: Upper Scopus and Lower WOS.
Water 16 00436 g004aWater 16 00436 g004b
Water 16 00436 g005aWater 16 00436 g005b
Figure 5. Number of documents (in Log10 scale) by type obtained in each step: Upper Scopus and Lower WOS.
Figure 5. Number of documents (in Log10 scale) by type obtained in each step: Upper Scopus and Lower WOS.
Water 16 00436 g005aWater 16 00436 g005b
Water 16 00436 g006aWater 16 00436 g006b
Figure 6. Number of documents by publication year obtained for each step: Upper Scopus and Lower WOS.
Figure 6. Number of documents by publication year obtained for each step: Upper Scopus and Lower WOS.
Water 16 00436 g006aWater 16 00436 g006b
Water 16 00436 g007aWater 16 00436 g007b
Figure 7. Reported keyword occurrences in the from the last filter: Upper Scopus and Lower WOS.
Figure 7. Reported keyword occurrences in the from the last filter: Upper Scopus and Lower WOS.
Water 16 00436 g007aWater 16 00436 g007b
Water 16 00436 g008aWater 16 00436 g008b
Figure 8. Number of documents included in Form 1 by publication year: Upper Scopus and Lower WOS.
Figure 8. Number of documents included in Form 1 by publication year: Upper Scopus and Lower WOS.
Water 16 00436 g008aWater 16 00436 g008b
Water 16 00436 g009
Figure 9. Document quantity consolidated for both databases in Form 1 by publication year.
Figure 9. Document quantity consolidated for both databases in Form 1 by publication year.
Water 16 00436 g009
Water 16 00436 g010
Figure 10. Number of documents by year of publication obtained in the final selection analysis from databases: Upper Filtered by Database and lower Filtered by time scale.
Figure 10. Number of documents by year of publication obtained in the final selection analysis from databases: Upper Filtered by Database and lower Filtered by time scale.
Water 16 00436 g010
Water 16 00436 g011
Figure 11. Number of documents by year of publication obtained by colleagues references.
Figure 11. Number of documents by year of publication obtained by colleagues references.
Water 16 00436 g011
Table 1. List of terms included and excluded from both databases during the initial search.
Table 1. List of terms included and excluded from both databases during the initial search.
Term to Be IncludedTerm to Be Excluded
Hydro*HydropowerPhoto*
UncertaintyHydrogen*Petro*
Forecast
Table 2. Filter 1: Publication year included in both databases.
Table 2. Filter 1: Publication year included in both databases.
Term to Be Included
2017201820192020202120222023
Table 3. Filter 2: Area included in each database.
Table 3. Filter 2: Area included in each database.
Scopus, Subject Area to Be Included
Environmental ScienceEngineeringMathematics
Earth and Planetary SciencesComputer Science
WOS, Research Area to Be Included
Meteorologic Atmospheric SciencesEngineeringMathematics
Water ResourcesComputer Science
Table 4. Filter 3: Languages included in both databases.
Table 4. Filter 3: Languages included in both databases.
Term to Be Included
EnglishSpanish
Table 5. Filter 4: Keywords included in both databases.
Table 5. Filter 4: Keywords included in both databases.
Term to Be Included
Uncertainty AnalysisHydrological Modeling
ForecastingForecasting Method
Weather ForecastingUncertainty
Table 6. Filter 5: Area included in each database.
Table 6. Filter 5: Area included in each database.
Scopus, Subject Area to Be Excluded
Economics, Econometrics, and FinanceSocial SciencesMedicine
Biochemistry, Genetics, and Molecular BiologyMaterial SciencesChemistry
Agricultural and Biological SciencesChemical EngineeringEnergy
WOS, Research Area to Be Excluded
Mathematical Methods in Social SciencesTelecommunicationsGeology
Public Environmental Occupational HealthAutomation Control SystemsEnergy Fuels
Operations Research Management ScienceGeochemistry GeophysicsAgriculture
Imaging Science Photographic TechnologyAstronomy AstrophysicsOceanography
Science Technology Other TopicsInstrument InstrumentationPhysical Geography
Life and Sciences Biomedic Other TopicsEducation Educational ResearchRemote Sensing
Mathematical Computational BiologyMarine Freshwater Biology
Table 7. Filter 6: Keywords included in both databases.
Table 7. Filter 6: Keywords included in both databases.
Term to Be Included
Uncertainty AnalysisStreamflow forecastingProbabilistic forecast
Weather ForecastingError AnalysisHydrological forecasting
ForecastingStreamflow forecastUncertainty Quantifications
Forecasting MethodEnsemble forecastHydrological forecast
UncertaintyStreamflow predictionDeterministic Forecast
PredictionPrecipitation ForecastUncertainty Quantification
HydrologyForecast uncertaintyProbability forecasting
Table 8. Sample or proposed Form 1.
Table 8. Sample or proposed Form 1.
ItemYearTitleDOIAbstractDuplicated?Selected?
…………………
1–82023The suitability of a seasonal ensemble hybrid framework including data-driven approaches for hydrological forecasting10.5194/hess-27-501-2023Hydrological forecasts are important for operational water management and near-future planning, even more so in light of the increased occurrences of extreme events such as floods and droughts… YesNo
…………………
1–492022Exploring hydrologic post-processing of ensemble streamflow forecasts based on affine kernel dressing and non-dominated sorting genetic algorithm II10.5194/hess-26-1001-2022Forecast uncertainties are unfortunately inevitable when conducting a deterministic analysis of a dynamical system. The cascade of uncertainty originates from different components of the forecasting chain, such as the chaotic nature of the atmosphere, various initial conditions and boundaries, inappropriate conceptual hydrologic modeling, and the inconsistent stationarity…YesYes
…………………
2–682021Bluecat: A Local Uncertainty Estimator for Deterministic Simulations and Predictions10.1029/2021WR031215We present a new method for simulating and predicting hydrologic variables with uncertainty assessment and provide example applications to river flows…NoYes
…………………
Table 9. Sample or proposed Form 2.
Table 9. Sample or proposed Form 2.
ItemLinkYearJournalPost
Process		StatisticMathML/IAHydrolog.Data
Sources		Input
var		Input
t		Input
x		Output
var		Out
t		ModelBench
Marking		MetricsRemarks
…………………………………………………
1–49https://...2022Hydrology and Earth System ScienceYesAffine Kernel Dressing, Ensemble Kalman FilterN/ANon-Dominated Sorting Genetic Algorithm IIBucket, GR4J, HBV, IHACRES, SAC-SMAECMWF, TIGGE at downscaling resolution of 0.5° to 0.1° and aggregated to 7 Days ahead, DEMBy modelDailyBasinsFlowDaily to 7 DaysModels + AKD, Models + NSGA-IIRaw
Forecast		MAE, MCRPS, RMSE, KGE and spreadThe performance of the AI method was superior to that of the AKD method…
…………………………………………………
2–68https://...2022Water Resources ResearchYesBlue CatN/AN/AHYMODGauges (Precip., Temp., Flow)Precip., Temp., FlowDailyBasinsFlowDaily to 100 DaysHYMOD + BlueCatObserved dataPredictive probability probability (PPP) plot, NSEThis paper presents a new method for simulating and predicting…
…………………………………………………
Table 10. Evolution of the number of documents found in each phase.
Table 10. Evolution of the number of documents found in each phase.
Search StepScopusWOS
Docs%Docs%
Initial1.9471005.598100
Filter 183442.83.07254.9
Filter 278240.22.88251.5
Filter 374438.22.86051.1
Filter 461531.61.14020.4
Filter 554427.980614.4
Filter 625613.12714.8
Table 11. Summary of steps or protocols applied in the analyzed reviews.
Table 11. Summary of steps or protocols applied in the analyzed reviews.
ReferenceResearch
Question		Protocol
Inclusion or
Exclusion Criteria		Search
Various
Sources		Applying
Incl. and
Excl. Criteria		Systematic
Data
Extraction		Evaluation
of Studies
Quality		Quantitative or
Qualitative
Summary		Narrative
Summary
[6]*------***
[29]*------***
[28]****-*-****-
[71]*------***
[46]*-----***
[21]****-***-*******
[22]***-----****
[23]*-----****
[32]*------***
[24]******--****
[29]***************
[20]*----*****
[45]**-----****
[34]*------***
[47]**-----*****
[48]*------***
[19]*-----****
[102]**************-
[18]*-******-*****
Note(s): * Poorly applied or inferred superficially from the text; ** clearly presented or easily inferred from the text; *** accurately applied; - not applied.
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

De León Pérez, D.; Acosta Vega, R.; Salazar Galán, S.; Aranda, J.Á.; Francés García, F. Toward Systematic Literature Reviews in Hydrological Sciences. Water 2024, 16, 436. https://doi.org/10.3390/w16030436

AMA Style

De León Pérez D, Acosta Vega R, Salazar Galán S, Aranda JÁ, Francés García F. Toward Systematic Literature Reviews in Hydrological Sciences. Water. 2024; 16(3):436. https://doi.org/10.3390/w16030436

Chicago/Turabian Style

De León Pérez, David, Rick Acosta Vega, Sergio Salazar Galán, José Ángel Aranda, and Félix Francés García. 2024. "Toward Systematic Literature Reviews in Hydrological Sciences" Water 16, no. 3: 436. https://doi.org/10.3390/w16030436

APA Style

De León Pérez, D., Acosta Vega, R., Salazar Galán, S., Aranda, J. Á., & Francés García, F. (2024). Toward Systematic Literature Reviews in Hydrological Sciences. Water, 16(3), 436. https://doi.org/10.3390/w16030436

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop