Next Article in Journal
Changes of Hydrological Extremes in the Center of Eastern Europe and Their Plausible Causes
Next Article in Special Issue
Decision-Making Theory and Methodology for Water, Energy and Food Security
Previous Article in Journal
Characterization of the Nitrogen Removal Potential of Two Newly Isolated Acinetobacter Strains under Low Temperature
Previous Article in Special Issue
Decision Support Strategies for Household Water Consumption Behaviors Based on Advanced Recommender Systems
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:

Water and Environmental Resources: A Multi-Criteria Assessment of Management Approaches

Departamento de Ingeniería de Procesos e Hidráulica, Ciencias Básicas e Ingeniería, Universidad Autónoma Metropolitana-Iztapalapa, Mexico City 09340, Mexico
CONAHCyT—Departamento de Ingeniería Geomática e Hidráulica, División de Ingenierías, Campus Guanajuato, Universidad de Guanajuato, Guanajuato 38096, Mexico
International Institute for Applied Systems Analysis (IIASA), 2361 Laxenburg, Austria
Departamento de Dinámica Terrestre y Superficial, Instituto de Geología, Universidad Nacional Autónoma de México, Mexico City 04510, Mexico
Department of Land, Air and Water Resources, University of California Davis, Davis, CA 95616, USA
Departamento de Ecologia y Recursos Naturales, Universidad de Guadalajara, Jalisco 48900, Mexico
Author to whom correspondence should be addressed.
Water 2023, 15(16), 2991;
Submission received: 30 June 2023 / Revised: 10 August 2023 / Accepted: 16 August 2023 / Published: 19 August 2023


The present study applied a multi-criteria analysis to evaluate the best approach among six theoretical frameworks related to the integrated management of water–environmental resources, analyzing the frequency of multiple management criteria. The literature review covers the period from 1990 to 2015, with a notable presence of the theoretical frameworks of Integrated Water Resources Management (IWRM), Ecohealth, Ecosystem Approach (EA), Water Framework Directive (WFD), and, to a lesser extent, the Watershed Governance Prism (WGP) and the Sustainability Wheel (SW). The multi-criteria decision-making (MCDM) methods applied include AHP (Analytic Hierarchy Process), TOPSIS (Technique for Order of Preference by Similarity to Ideal Solution), and PROMETHEE (Preference Ranking Organization Method for Enrichment Evaluations). Twenty-five criteria were analyzed, such as governance, participation, sustainability, decentralization, and health and well-being, among others. We started with five criteria for evaluating the hierarchy of the six theoretical frameworks using the AHP method. Subsequently, we again evaluated the five criteria using the TOPSIS and PROMETHEE methods to calibrate the results with the AHP. Then, using word counting, we evaluated the best approach, applying 10, 15, 20, and 25 more criteria. Our results indicate that the best integrated management alternative was the WFD, which fulfilled 47% of the management criteria. Second, with 45%, was the WGP, and third was IWRM, with 41%; less successful approaches to the criteria were demonstrated by the EA, SW, and Ecohealth methods. By applying this methodology, we demonstrated an excellent structured tool that can aid in the selection of the most important issue within a given sector.

1. Introduction

In 1992, the International Conference on Water and the Environment (ICWE) emphasized the development of holistic approaches to integrated water management [1] and all social, environmental, cultural, institutional, and political components related to resource management [2]. However, achieving sustainable and equitable resource management systems is not possible without a solid foundation of good governance. This requires the concerted and specific participation of local authorities, the private sector, unions, NGOs, citizens, and academia [1]. Comprehensive and collective participation will result in better management of resources at the watershed level. However, specific uses of resources (soils, forests, water, or fishing) have been privileged until these resources’ degradation and depletion [3]. Yet, today, the quest for solutions to challenges through the adoption of inter- and transdisciplinary approaches is imperative [1].
Global society requires that changes to people’s way of life occur; we must recognize the fundamental role of water in supporting ecosystems and the pressures exerted by human activities [4]. Human well-being and progress toward sustainable development depend on the proper management of ecosystems to ensure their conservation amid growing demands for ecosystem services [3]. More integrated, collaborative approaches to resource management can ensure greater sustainable development and better scientific knowledge by focusing on problem-solving in an integrated and satisfactory manner [5].
Thus, a series of water and environmental resource management frameworks is essential for ensuring the sustainable and efficient use of natural resources. This series of frameworks has different purposes, with a high degree of robustness and reliability in facing all kinds of resource management challenges and issues. Diverse water and environmental resource management approaches may include considering the entire water cycle, adapting to and mitigating the impacts of climate change on water resources, addressing challenges resulting from population growth and increased water demand, and ensuring sustainable water supply and sanitation services in cities [6,7]. Although these generalizations are applicable, each specific framework should be tailored to the local context, considering factors such as water availability, socio-economic conditions, and environmental considerations [8,9,10]. With this in mind, we aim to identify the best frame of reference for managing water and environmental resources. We must bear in mind that water and environmental issues are significant global concerns and can vary depending on the region. Water scarcity, water pollution, deteriorating water quality, inadequate sanitation, lack of access to clean water, groundwater depletion, and the impacts of climate change are some water-related problems that will require a multifaceted management approach that enables us to prioritize sustainable water use, promote water-saving technologies, invest in water treatment and sanitation infrastructure, and enhance awareness and education about water conservation and protection [11,12,13].
Below, we will reference some works that address various elements used in our study, such as management theoretical frameworks, MCDM, and the frequency of integrated management concepts, since we could not find a single comprehensive work similar to the one that we propose. Guerrero et al. [14] stated that the ecosystem approach (EA) to water management complements current thinking on IWRM, the two sets being consistent with each other; this is because EA principles have the potential to complement and enrich IWRM practice. Calizaya et al. [15] identified the best solutions for existing conflicts in a region with problems concerning water scarcity, thereby promoting interaction with the parties in conflict and widening the scope to achieve a sustainable strategy. Using the AHP method and based on the evaluation of economic, social, and environmental criteria, Chávez and Osuna [16] used the PROMETHEE method to carry out a ranking study in the countries of the European Union and determine which country is leading advances toward sustainable development; they did so by using a set of criteria that are in conflict with each other. Geng and Wardlaw [17], in the context of IWRM, included economic, social, and environmental issues, applying MCDM to integrate different objectives in the planning, management, and decision-making process. They identified a variety of criteria in terms of economic, social, and environmental dimensions for use in their MCDM analysis. The GWP [18] published a comparison of the relationship between the objectives of the WFD and those of IWRM, wherein experiences of implementation, a review of the progress made toward the adoption of IWRM in EU countries, and a review of the progress being made to comply with the WFD were collated. Widianingsih et al. [19] analyzed the scientific literature on watershed governance through a bibliometric analysis using the Scopus database and the VOSviewer software, thus highlighting the quality and quantity of articles on basin governance around the world. Regarding word frequency, De Filippo et al. [20], in their investigation of recurring key terms in IWRM and citizen science studies, found that the most used words were management, public participation and governance, sustainability, ecosystem, and conservation, all of which are related concepts within the field of water resources.
Our original idea was to consider articles from 1990 to the present (at least until 2022) in order to incorporate the highest number of references regarding management approaches. However, to our surprise, the documents consulted mainly referred to the four classic approaches (IWRM, WFD, EA, and Ecohealth) and, to a lesser extent, the WGP and SW. The search was challenging; we did not find many pertinent studies. Subsequently, the period was narrowed down to 2015 using the SW approach because we did not find many studies with similar characteristics to those shown in this work.

1.1. Motivation and Scope

The motivation for carrying out this study arose from the fact that we did not find any published works similar to the one we propose herein. We attempted to answer the question of which of the six integrated management theoretical frameworks is superior, using both qualitative and quantitative analyses of these integrated management concepts.
We were able to answer this question in two stages. The first consisted of using the AHP method, using our own experience in various sessions to qualitatively evaluate which of the six alternatives was the best. For this evaluation, five management criteria were used (governance, participation, sustainability, decentralization, and health and well-being).
Subsequently, the same question was answered again, only now we progressively increased the number of criteria from 10 to 25; having done so, it was no longer profitable to use the AHP method.
Thus, we decided to use two different multi-criteria methods, TOPSIS and PROMETHEE. The first method determines the distance between alternatives by finding the ideal solution and the non-ideal solution, and the second method determines this distance by ranking the different alternatives. For this reason, we counted the frequency of the most relevant management concepts from each theoretical framework, which served as inputs for the two MCDM methods. By evaluating the criteria, we were able to produce new results.
This work uses different water–environmental resource management approaches, the frequency of common concepts between the approaches consulted, and an MCDM evaluation. In several of the works consulted, we found that these topics were unlinked. For example, we did find related works with approach–approach, approach–MCDM, and approach–bibliometrics/frequency links.

1.2. Contributions

With our analysis, we were able to verify that the WGP and the SW are novel approaches that are superior to classical approaches. Both adopt particularities and contribute their own concepts, through which a more integrated approach is achieved.
Through principal components analysis applied to these management concepts, we observed that some concepts have low relative importance or are poorly represented.
Our analysis allowed us to confirm two groups. The first is made up of the WGP, Ecohealth, and EA, which are closely related, and the second is made up of IWRM, the SW, and the WFD. Through our analysis of the main components, we identified that both groups are independent. The former is more oriented to ecosystem management, and the latter to water resources.
According to FNWA [1], it is important that the New Water Culture adopt a holistic approach to water management and recognize its multiple dimensions (i.e., the ethical, environmental, social, economic, and emotional values incorporated in aquatic ecosystems) to build and respond to the challenges of the 21st century.

1.3. Integrating Theoretical Frameworks

The great effort that has been put into the development of the six theoretical frameworks for the integrated management of water–environmental resources is admirable. However, it is both possible and recommended that these management approaches complement each other by incorporating elements from one or several other theoretical frameworks since they can strengthen the existing frameworks. For example, IWRM, which is oriented toward water resources, can be complemented by incorporating aspects of the EA, which refers to ecosystem services, the selection of species, and the loss of biodiversity. The WGP, which is oriented toward water governance, sustainable development, ecosystems, and health and well-being, can be complemented by incorporating aspects of the SW, such as adaptive and learning capacity, as well as distributive and procedural justice. Thus, the WFD, which refers to achieving good ecological status in water bodies, may be well complemented by aspects of Ecohealth, such as socio-ecological health, trans-disciplinarity, and social equity. This will promote a more comprehensive knowledge base in management approaches and allow a better approach to the problems and resources of basins. According to FNWC [1], this will be achieved through the adoption of inter and transdisciplinary approaches. Other clear examples are the approaches proposed by Parkes et al. [21] and Schneider et al. [22].

1.4. Frameworks for Water–Environmental Resource Management

For a historical overview of the main water and environmental advances over four decades, highlighting the variety of frameworks, including Integrated Water Resources Management (IWRM) [23], the Ecosystem Approach to Health (Ecohealth) [24], the Ecosystem Approach (EA) [25], and the Water Framework Directive (WFD) [26], see Figure 1. Typically, these have been issued by national or international government agencies. Some examples of these agencies are the International Conference on Water and the Environment (ICWE), International Development Research Center of Canada (IDRC), Conference of the Parties to the Convention on Biological Diversity (COP-CBD), and Parliament and the Council of Europe, who established the Water Framework Directive. Parallel to such broader management approaches, the Watershed Governance Prism (WGP) [21] and the Sustainability Wheel (SW) [22] have primarily been developed by scientific research groups rather than governmental institutions (Health Sciences Programs, University of Northern British Columbia, Canada, and Centre for Development and Environment, University of Bern, Switzerland), highlighting their peculiarity compared to the four existing approaches; the WGP and SW are more robust and incorporate new notions (e.g., water governance and water governance systems).

1.4.1. Integrated Water Resources Management (IWRM)

In 1987, the World Commission on Environment and Development (WCED) made an urgent call for long-term environmental strategies to achieve sustainable development and effectively address environmental concerns [27]. In 1992, the four principles of the Declaration of Dublin on Water and Sustainable Development were presented, and since then, they have become the strategic basis for IWRM [23]. It is important to mention the four fundamental principles for water and sustainable development: (1) freshwater is a finite and vulnerable resource, essential to sustaining life, development, and the environment; (2) water development and management should be based on a participatory approach, involving users, planners, and policymakers at all levels; (3) women play a central part in the provision, management, and safeguarding of water; and (4) water has an economic value in all its competing uses and should be recognized as an economic good [28,29,30]. These principles laid the foundation for integrated water resource management and sustainable development approaches. Accordingly, it is important to highlight that, to this day, they continue to guide global efforts in water management and underscore the importance of considering social, economic, and environmental aspects when addressing water challenges [31]. IWRM involves a process that promotes the coordinated development and management of water, land, and related resources to maximize economic and social well-being in an equitable manner, without compromising the sustainability of vital ecosystems [32]. Subsequently, the relationship between IWRM and sustainable development could be the next step toward a more coherent integration in water resource management [5].

1.4.2. Ecosystem Approach to Human Health (Ecohealth)

The International Development Research Center (IDRC) established the Ecosystem Approaches to Human Health (Ecohealth) program in 1996–2001. The Ecohealth program was initiated by the IDRC in collaboration with several international partners [24]. Its primary goal is to promote research and action that recognizes the intricate connections between ecosystems, human health, and sustainable development. Ecohealth emphasizes interdisciplinary and participatory approaches to address complex health issues within the context of ecosystems [33]. It focuses on understanding the interactions between the ecological, social, and economic factors that influence human health and also supports research projects that integrate natural and social sciences, involving multiple stakeholders, such as researchers, communities, policymakers, and practitioners [24]. Ecohealth suggests that human health and well-being are dependent on ecosystems, as both are the result of ecosystem management [34]. For Webb et al. [35], public health was not a clear focus before Ecohealth; however, now, it has been extended to other social, economic, and environmental spheres. When scientists from various disciplines become involved and participate in a community with decision makers, they can implement transdisciplinary research [24,36]. Since its establishment, the Ecohealth program has contributed to advancing the field of ecohealth and strengthening collaboration between researchers, educators, politicians, and communities in addressing health challenges within the broader context of the ecosystem [24,37].

1.4.3. Ecosystem Approach (EA)

In 2000, during the Millennium Ecosystem Assessment (MEA) in Nairobi, Kenya, the concept of the Ecosystem Approach was conveyed at the global political level as a strategy for integrated management [25]. The MEA, a comprehensive assessment of the world’s ecosystems, aimed to provide scientific insights into the consequences of ecosystem change for human well-being. One of the key suggestions of the MEA was the adoption of the Ecosystem Approach (EA) for the integrated management of land, water, and living resources. The EA recognizes that ecosystems are complex and interconnected systems and that their management should consider the interactions between ecological, social, and economic factors. It focuses on understanding and addressing the drivers of ecosystem change, as well as the impact of these changes on human well-being and the sustainable use of resources. Moreover, it encourages interdisciplinary collaboration, stakeholder engagement, and adaptive management strategies to foster the resilience and sustainability of ecosystems. The MEA’s recommendations and the promotion of the EA have since influenced global policies and frameworks, including the Convention on Biological Diversity (CBD). The CBD recognizes the Ecosystem Approach as a key guiding principle for the conservation and sustainable use of biodiversity. As mentioned above, people, water, and nature are part of the same system; therefore, any and all water policies must be incorporated into a comprehensive and systemic vision [3,25,38,39,40,41].

1.4.4. Water Framework Directive (WFD)

In 2005, the European Declaration for a New Water Culture (NWC) declared that rivers, lakes, wetlands, and aquifers should be considered biosphere heritage sites. Their management must thus be performed by communities and public institutions in a responsible and sustainable manner. The NWC proposes the conservation of ecosystems [1] and clarifies and reinforces the proper implementation of the WFD [42]. The European Water Framework Directive (WFD), adopted in 2000, is a key legislative instrument of the European Union (EU) that aims to achieve good water status for all European waters by 2027. It emphasizes integrated river basin management, the protection of aquatic ecosystems, and community involvement in decision-making processes [26]. The objective of the WFD is that waters (rivers, lakes, estuaries, coastal waters, wetlands, aquifers) achieve an optimal ecological status through the application of hydrological basin plans (HBPs) intended to prevent water deterioration and prevent or limit the entry of pollutants into aquifers. There are six guiding principles of the WFD: (1) integrated river basin management; (2) environmental objectives and monitoring; (3) stakeholder involvement; (4) water-related planning and management measures; (5) economic analysis and cost recovery; and (6) international cooperation. This European water policy is an instrument for rational water management, establishing the natural limits of a basin and active citizen participation as a management unit [1] and fostering the joint protection of surface and groundwater, inland, transitional, and coastal waters [26,43,44,45].

1.4.5. The Watershed Governance Prism (WGP)

Parkes et al. [46,47] proposed a theoretical framework called the Prism Framework of Health and Sustainability, which depicts the relationships between governance and development, ecosystems, social systems, and health. Later, Parkes et al. [21] renamed it the Watershed Governance Prism, which integrates four different perspectives of water governance. Perspective A consists of water governance for sustainable development. Perspective B consists of water governance for ecosystems and well-being. Perspective C consists of water governance for the social determinants of health. Perspective D consists of water governance designed to promote socio-ecological health. The integration of and interaction between the four perspectives (A, B, C, and D) make up the Watershed Governance Prism (WGP), which facilitates integrated watershed governance and allows us to better understand the interactions between the four perspectives of water governance [21,48]. These perspectives aim to enhance understanding and decision making related to ecosystem health and sustainable water management. The prism encourages a transdisciplinary perspective and the integration of multiple stakeholders and knowledge domains in order to foster sustainable and resilient water management. The WGP is thus a contemporary approach that presents multiple facets of governance characterizing water resource management by linking social and environmental aspects with the determinants of health in a watershed context. The WGP is a tool for facilitating integrated watershed governance and understanding across four visions of water governance that also promote health, sustainability, and socio-ecological resilience [21]; in addition, it is a useful approach for understanding complex links, specifically land–water interactions and the driving forces of change (floods, droughts) that exert pressures on socio-environmental systems [19,48]. A case of its application can be found in the work of Armas-Vargas et al. [49].

1.4.6. The Sustainability Wheel (SW)

Schneider et al. [22] and the Center for Development and Environment (CDE) developed a framework called the Sustainability Wheel. The Sustainability Wheel is a tool designed to support sustainability assessments and decision-making processes. It aims to provide a comprehensive and integrated view of sustainability by considering various dimensions and indicators. Moreover, it combines social, environmental, and economic dimensions to evaluate the sustainability of a particular context, project, or system. This approach is often depicted as a graphical representation, resembling a wheel with multiple spokes. Each spoke represents a specific dimension or aspect of sustainability, such as biodiversity, water resources, energy, livelihoods, governance, or equity. Within each dimension, relevant indicators or sub-indicators are identified to assess the sustainability performance of a particular area. The SW combines the transparent identification of sustainability principles, their regional contextualization based on their subprinciples (indicators), and the scoring of these indicators through deliberative dialog among an interdisciplinary team of researchers, which accounts for their various qualitative and quantitative research results. The SW is advantageous in its capacity for structuring complex and heterogeneous knowledge to attain a global perspective on water sustainability [22]. A sustainable future for water is possible, although it depends largely on the social, economic, technical, and institutional reforms that stakeholders are willing to undertake [50]. This approach (principles and indicators) is also applicable to other areas of study, and a case of its application can be found in the work of Schneider et al. [51].
From this brief description of six theoretical frameworks used to address issues regarding water and environmental resources in watersheds, a natural question arises: which framework is the most effective? We were able to answer this question in two stages, as described in the following section. The first consisted of using the AHP method (i.e., the Analytic Hierarchy Process) to qualitatively evaluate which of the six alternatives was the best. For this evaluation, five management criteria were used (governance, participation, sustainability, decentralization, health and well-being). Subsequently, the same question was answered again, only we progressively increased the number of criteria from 10 to 25; in this context, it was no longer profitable to use the AHP method. Instead, we decided to use two different multi-criteria methods: TOPSIS (Technique for Order Preference by Similarity to Ideal Solution) and PROMETHEE (Preference Ranking Organization Method for Enrichment Evaluations). The first determines the distance between alternatives by finding the ideal solution and the non-ideal solution; the second does so by ranking the different alternatives.

1.5. Multi-Criteria Decision-Making (MCDM) Methods

There are different types of MCDM methods that have been developed by different authors in recent decades [52,53]. According to Pereira-Basílio et al. [54], there are five methods that are most commonly used in various areas of knowledge (AHP, TOPSIS, VIKOR, PROMETHEE, and ANP); however, they highlight the AHP method. Below is the central idea of the MCDM method that we use in this work:
The AHP is a multi-criteria decision-making (MCDM) method [55], widely used for evaluating and determining alternatives in problem-solving. It is a structured method that optimizes decision making when there are multiple attributes (criteria), doing so through the discretization of a problem. The AHP is a technique based on pairwise comparisons and is generally composed of four stages, defined below as the implementation procedure. Its application is simple; more details on this mathematical approach are provided by Saaty [55], Triantaphyllou and Mann [56], Blachowski [57], and Bivina and Parida [58].
TOPSIS is a multi-criteria method proposed by Hwang and Yoon [59], used to determine the best alternative based on the concept of the compromise solution. Its main idea is to choose the best alternative that is nearest to the positive ideal solution (the optimal solution) and furthest from the negative ideal solution (the inferior solution). Then, the superior of these two is chosen and thereby deemed the best alternative.
PROMETHEE is a multi-criteria decision technique developed by Brans et al. [60]; it is based on outranking techniques, through which we are able to study how good or bad an alternative is compared to another. It is one of the best-known and most widely applied outranking methods [61] and generally follows the sequence (1) preference function; (2) comparison between alternatives; (3) alternative comparison; (4) criteria matrix, partial rankings; and (5) final ranking of alternatives [62].

1.6. MCDM Associations

For Soares de Assis et al. [63], the rise of new techniques and their integration with prior associations and fuzzy sets mean that decision makers now have just over 30 methods at their disposal. The main discrepancies between these methods are the complexity of the algorithms, the criteria weighting methods, the way in which preference evaluation criteria are represented, the possibility of uncertain data, and the type of data aggregation [64].
Wątróbski et al. [65] proposed a new method called the Data vARIability Assessment-Technique for Order of Preference by Similarity to Ideal Solution (DARIA-TOPSIS). The TOPSIS method has shown its usefulness in solving multi-criteria decision-making problems in several domains [66]. This fact, combined with the simplicity of its algorithm, contributed to the basis for developing a new multi-criteria method that considers the variability of the efficiencies of alternatives over time. The DARIA-TOPSIS method provides aggregated efficiency results regarding the performance of the evaluated alternatives, considering the dynamics of the changes within the investigated time range. The authors provide a publicly available library, called pydaria, for use in the Python environment.
The Characteristic Objects METhod (COMET) is based on Triangular Fuzzy Numbers (TFN) [67]. A pairwise comparison of the determined Characteristic Objects (CO) enables relationships between criteria-based TFN to be established; the rule base is then defined and the preference values are calculated [68,69]. This method is completely free of the rank reversal phenomenon, much like the SPOTIS method [70]. The SAPEVO method (Simple Aggregation of Preferences Expressed by Ordinal Vectors–Multiple Decision Makers) allows for the evaluation of a single decision maker. This advanced version extends the method to multiple decision makers; in addition, it introduces a process of standardization of the evaluation matrices, which increases the consistency of the model [71]. Another method is COmplex PRoportional Assessment (COPRAS), which assumes a direct and proportional relationship between the importance of the decision variants examined [72,73]. This technique ranks alternatives based on their relative importance [74].
The PROMETHEE methods are another family of superior methods, which include PROMETHEE I, II, III, IV, V, and VI, as well as GDSS, TRI, and CLUSTER [75]. The modeling technique proposed by Moreira et al. [76] determines a new extension present in the family of PROMETHEE methods, in which the integration of the original modeling of the method with the evaluation techniques by ordinal inputs of the SAPEVO method is carried out [77]. This hybrid method represents the integration of two methodological concepts: one for cardinal evaluation (PROMETHEE) and the other for ordinal evaluation (SAPEVO) [78,79].

1.7. The Future Potential of MCDM

MCDA (Multi-Criteria Decision Analysis) methods are very useful, as they allow us to deal with multiple, often contradictory, criteria in a structured way and consider different preferences. In energy debates, there are many different, and often conflicting, sustainability criteria and viewpoints among stakeholders [80]. The field of application of multi-criteria methods is wide and diverse. For example, they can be used to select sites for the installation of solar and wind power plants, assess the impact of volcanic hazards, select hotel locations for the tourism sector, select optimal irrigation plans, classify bridges affected by corrosion, manage water resources and health, and even aid in the selection of mobile devices. There is no limit to the field of their application; however, Villa-Silva et al. [81] revealed, in their compilation from 1980 to 2018, that the sectors of greatest interest for the application of multi-criteria methods involve the selection of suppliers and projects.
Although there is extensive information on the web concerning the implementation of MCDM methods, either for self-teaching or storing in Excel sheets, Kizielewicz et al. [82] developed an open source proposal for the Python 3 library called pymcdm. It is designed to address decision making using MCDA/MCDM approaches and includes classic and new methods for decision making. The library is available through the Python Package Index (PyPi) and is easy to install and update. It has also been uploaded to the GitLab repository under the MIT license, making it even more accessible. The pymcdm library consists of several modules designed for different MCDA/MCDM problems. It includes tools related to the evaluation of alternatives, the determination of optimal solutions and the relevance of criteria, comparative analysis, and visualization. Herein, we present a brief overview of the framework of the pymcdm library:
  • Methods (fifteen MCDA/MCDM evaluations):
  • Weights (ten methods for determining the relevance of criteria):
Equal weights, entropy weights, standard deviation weights, MEREC, CRITIC, CILOS, IDOCRIW, angle weights, Gini weights, and variance weights.
  • Normalizations (eight normalization methods categorized by profit/cost criteria):
    min–max, max, sum, vector, logarithmic, linear, nonlinear, and enhanced accuracy.
  • Correlations (six correlation/similarity coefficients):
Spearman’s rank correlation coefficient, Pearson correlation coefficient, weighted Spearman’s rank correlation coefficient, rank similarity coefficient, Kendall rank correlation coefficient, and Goodman and Kruskal’s gamma coefficient.
  • Visuals (functions for visualizing important MCDA/MCDM data):
COMET charts, weight charts, PROMETHEE charts, correlation charts, and ranking charts.
The provision of a wide range of approaches, the high performance and consistency of the software, and its small size are what distinguishes the pymcdm library from other software, making it a tool with great potential for decision makers [82].

1.8. Use of Multi-Criteria Normalization Techniques

Normalization is a necessary step for the selection of an alternative or criterion; it has two attributes: cost (down) and benefit (up). Several normalization methods have been proposed, where rij is the normalized value of xij that represents the nominal value of attribute j [83]. The most recurrent normalization methods are min–max, max, sum, vector, logarithmic, linear, nonlinear, and enhanced accuracy [82,84,85]. Several normalization techniques have been proposed, and some of them are used for specific MCDM methods. For example, the vector normalization method is used within the TOPSIS method, and the sum is implemented within the AHP. It should be noted that there is no consensus among researchers on the conditions in which to use each specific normalization technique [85]. Some equations are linear, but the user is free to use other functions if they are considered to fit the problem better based on knowledge or experience [86]. For example, Jahan and Edwards [84] found just over 30 normalization methods, highlighting, in addition to the eight techniques already mentioned, the Markovic method [87], Zavadskas and Turskis normalization [88], the target-based normalization technique [89], and the distance for target criteria [90], among others.
In the study by Vafaei et al. [85], the authors stated that max–min normalization is considered the best technique. However, they proposed a conceptual model to help decision makers select the most appropriate normalization technique for their decision-making problems. This model consists of three phases that distinguish the types of data from the matrices using different methods of normalization and the application of metrics (Euclidean distance, standard deviation, mean ks (from a Pearson’s correlation), ranking consistency index, and mean squared error). Within our work, due to its scope, we used both our normalization methods and common methods used in multi-criteria methods, these being the semi-linear normalization method (vector) for TOPSIS, the linear normalization method (max–min) for PROMETHEE, and linear normalization (sum) for the AHP.

2. Materials and Methods

2.1. Main Focal Documents for Each Framework

Table 1 shows the main frameworks for management and the essential documents we used in our AHP, which were published from 1990 to 2015. In this study, we define the IDRC’s book Ecohealth, written by Lebel et al., as an original source. Concerning Parkes et al. [21] and Schneider et al. [22], although theirs are scientific articles, we consider them management approaches for the purposes of this work due to their integration of water and socio-environmental issues.

2.2. Similarities between Management Frameworks

Although the aforementioned focal approaches are distinct, we found that they share some terms and/or literary concepts in common; we did so by comparing their main characteristics, i.e., the principles, pillars, and objectives that support and govern each framework. Figure 2 is a simple diagram, wherein the terms governance, decentralization, participation, sustainability, quantity and quality, access to water, water resources, pollution, etc., are at the center. These terms are common among the six approaches. On the sides of the diagram, the specific and relevant characteristics of each approach are indicated. In the case of the WFD, some of its unique aspects are achieving the optimal ecological status of water bodies while integrating deltas, estuaries, and coastal platforms. In the WGP, the social determinants of health and the promotion of socio-ecological health are elementary concepts. The EA’s focus on understanding and managing ecosystems in an economic context and within the limits of their functions is unique. In the SW, adaptive and learning capacity, as well as distributive and procedural justice, are intrinsic elements. In Ecohealth, trans-disciplinarity and social and ethnic equity are essential and specific aspects; in IWRM, the role of women and water resources (surface and groundwater) are characteristic emphases.

2.3. Analytic Hierarchy Process

2.3.1. Definition of the Objective or Representation of the Problem

This process consists of representing the problem through a hierarchical structure (commonly) of three subordinate levels, indicating the objective to be achieved, the criteria/attributes, and the alternatives to be compared. Figure 3 shows the implementation of each stage to be developed. Its purpose is to determine which of the six management approaches (or alternatives) is the best by comparing their attributes or criteria (such as governance, participation, sustainability, decentralization, and health and well-being) with each of these six alternatives for water–environmental management.

2.3.2. Evaluation of the Criteria

Matrix A was constructed to compare the five criteria and calculate their relative importance (management criteria). For this case, a criteria matrix of five rows by five columns was constructed, as shown in Table 2. A relative-importance scale rating was assigned to each comparison to ascertain which of these two criteria is more or less important; when two have the same importance, the evaluation is 1. Table 2 shows the comparative evaluation of the five criteria; for example, to determine whether governance is more or less important than participation. It was deemed that governance is slightly less important than participation, and thus a value of 1/3 was assigned. Therefore, it is common that participation is slightly more important than governance, which is why it has a value of 3. That is to say, without participation, there is no governance. Similarly, sustainability and health and well-being lose meaning or relevance once governance becomes less important (see Figure 4a for the comparisons considered in this study).
From Matrix A (or comparison by pairs, Table 2), the normalized matrix (A’) was generated by dividing each value of Matrix A by the total of its respective column (Table 3). Of the six matrices generated, below, we indicate the procedure generated for the criterion–criteria matrix using an example. Normalization is necessary for the selection of an alternative or criterion; this is achieved using two attributes: cost (−) and benefit (+) [83]. For Matrix A, we only considered positive criteria. The normalized value rij was calculated using (Equations (1) and (2)):
r i j + = x i j i = 1 n x i j ;   i = 1 , , n ;   j = 1 , ,   m
r i j = 1 / x i j i = 1 n 1 / x i j ;   i = 1 , , n ;   j = 1 , ,   m
Subsequently, the total horizontal sum of each row of the normalized matrix was calculated to determine the relative weight of each criterion (w). This previous result is also known as a relative priority vector (Table 3). The inconsistency ratio (IR) incurred when assigning the evaluations was calculated using the consistency index (CI) and the random consistency index (RCI). The CI is a function of the size of the matrix (n) and the maximum eigenvector (λmax), and the RCI is derived from the size of the matrix (n), which is the number of criteria. Finally, having obtained the IR (%), this value should be below 10%; if a value is outside this range, the procedure should be repeated until acceptance is achieved (Table 4). Notably, this procedure can also be applied to the selection of alternatives.

2.3.3. Evaluation of the Alternatives

In addition to the criteria matrix, five matrices (order 6 × 6) were constructed, in which the six alternatives corresponding to each criterion were compared. For example, Figure 3 shows that the “governance” criterion refers to each of the six management alternatives using a continuous line. Notably, in the construction of these five matrices, we followed the procedure cited in Section 2.3.2. Table 4 summarizes the parameters evaluated using the comparison matrix, first between criteria (row I). Then, each criterion is compared with the six alternatives (rows II to VI). Hence, all the IR values are within the tolerance margin of less than 10%.

2.3.4. Hierarchy of Alternatives

Finally, to define which of the six alternatives or frameworks is the most relevant with respect to the five management criteria used, following Saaty [55], Triantaphyllou and Mann [56], Blachowski [57], and Bivina and Parida [58], we multiplied each component in the weight matrix of the criteria by the corresponding weight matrix of each of the alternative matrices. The value closest to 1 (or 100%) was chosen as the best option.

2.4. Limitations of the AHP and Selection

Regarding the application of the AHP to the five management criteria, although it is a simple technique when few criteria and alternatives are used, it demands time, as well as knowledge of the objective problem. Therefore, we did not continue using this MCDM method because we needed to evaluate twenty more criteria, making it difficult to evaluate the decision-making process in each management approach to determine the one that presents the best performance.
Accordingly, we decided to use two more MCDM methods. The first was the Technique for the Order of Preference by Similarity to Ideal Solution (TOPSIS) method, which, according to Ture et al. [91] and Tzeng and Huang [92], is a method for analyzing multiple criteria decisions via Euclidean distance. The second method was the Preference Ranking Organization Method for Enrichment Evaluations (PROMETHEE), which hierarchically orders a set of alternatives [93]. To select the best MCDM method, the following steps were necessary:
  • The five criteria and the six alternatives were evaluated again using TOPSIS and PROMETHEE.
  • The results obtained from the AHP (for the five criteria) were compared to the TOPSIS and PROMETHEE results in order to calibrate and choose the best adjustment coefficient.
  • The Pearson and Spearman correlation coefficients were evaluated using the AHP-TOPSIS and AHP-PROMETHEE methods to select the one closest to the value of 1.
  • Once the method with the highest correlation value was chosen, it was assumed that the model had been adjusted (to the AHP method). Therefore, it was used to evaluate the twenty management criteria, considering that the model was able to extrapolate the results.
  • The management criteria were contamination/pollution, access, social, water resources, resilience, economic, ecosystem, water uses, development, quality and quantity, stakeholders, scale/level, water, groundwater, human, environment, demand, issues/problems, watersheds/basin, and agriculture.
  • After counting the frequency of the twenty-five management criteria in the six documents consulted (Table 5), we evaluated them using the selected method in order to simulate or predict what would be the best approach for the 10, 15, 20, and 25 management criteria.

2.5. Inputs for TOPSIS and PROMETHEE

Table 5 presents the frequency of the terms most often cited in the body of the consulted approaches, showing their importance or irrelevance at the time of the citation. For example, in 1992, IWRM did not include the concept of governance, as such; however, the concepts of “participation and sustainability” (and their related derivations) were relevant throughout the document, followed, to a lesser extent, by “health and well-being” (or welfare), as well as the concept of “decentralization”, which was cited on one occasion. At the beginning of the current millennium, in the EA and the WFD approaches, the terms “governance”, “decentralization”, and “health and well-being” appeared to a lesser extent, in contrast to “sustainable”, which had 51 and 17 citations in the respective reference documents. Finally, the WGP and SW had greater coverage of all the cited terms, except for “decentralization”.

2.6. Entropy Method

Due to the varying importance of each criterion in an MCDM problem, the weighting process of the criteria must be carried out using the performance of the objective component [94]. This is an objective method of assigning weights (wj), which are determined based on evaluations of the decision matrix, without influencing the preferences or bias of the decision maker [95]. In Shannon and Weaver [96], Zhu et al. [97], and Moradian et al. [94], the procedure for determining the weights (wj) using the entropy method (Ej) is as follows:
(I) From the decision matrix (B), with m alternatives and n criteria, the normalized matrix rij is calculated and defined using Equation (3) (using the values from Table 5):
r i j = x i j i = 1 m x i j ;   i = 1 , , m ;   j = 1 ,   2 , n  
Table 6 shows the results of evaluating the sum (linear) normalization technique under the benefit criterion condition of use.
(II) The entropy (Ej) level of each criterion is calculated (Equation (4)):
E j = k i = 1 m ( r i j l o g r i j )   j = 1 ,   2 , n ;   0 E j 1
where k = 1 log m   , and m is the number of alternatives to be analyzed (see Table 7).
(III) The more similar the rij considered, the greater the entropy Ej of a criterion. However, it is in our interest to measure the variation (Dj) from the entropy (Ej), and then calculate the complement using Equation (5):
D j = 1 E j   j = 1 ,   2 , n  
(IV) Entropy weights (Equation (6)) are calculated using normalized values of the divergence, Dj:
w j = D j j = 1 n D j = 1 ;   j = 1 ,   2 , n  

2.7. TOPSIS Method

The multi-criteria method for decision making called TOPSIS, developed by Hwang and Yoon [59], is useful for evaluating complex systems [98,99]. TOPSIS essentially uses the principle of the shortest ideal approximation (found using closeness to the ideal), as indicated below. It determines the best alternative based on the shortest Euclidean distance from the positive ideal solution (PIS) and the furthest Euclidean distance from the negative ideal solution (NIS).
Using the information from Table 5, the TOPSIS method was evaluated. Below, we outline the procedure used to determine the best alternative, using an example.
The algorithm generally proceeds via the following steps:
(I) The construction of the decision matrix begins the process, starting from m alternatives Ai, i = 1, 2, …, m, that are evaluated based on the criteria Cj, j = 1, 2, …, n, where xij represents the valuation of the alternative Ai with respect to the criterion Cj and, W = [ w1, w2, …, wn].
(II) The normalization of the decision matrix (Equation (7)):
r i j = x i j i = 1 n x i j 2 ;   i = 1 , , n ;   j = 1 , ,   m
Table 8 shows the results of evaluating the vector (semi-linear) normalization technique under the benefit criterion condition of use.
(III) Calculation of the weighted normalized decision matrix (Equation (8)):
v i j = r i j w j ;   i = 1 , , n ;   j = 1 , ,   m
Table 9 shows the results of multiplying the normalized matrix by the weights of each criterion, wj (see Table 7).
(IV) Determination of the positive ideal solution (PIS) and the negative ideal solution (NIS):
A + = v 1 + ,   v 2 + , , v n +
A = v 1 ,   v 2 , , v n
v j + = ( m a x   v i j x ) | j j 1 ,   ( m i n   v i j x ) | j j 2 ;   i = 1 , , m
v j = ( min v i j x ) | j j 1 ,   ( m a x   v i j x ) | j j 2 ;   i = 1 , , m
where j1 and j2 denote the negative and positive criteria, respectively.
Table 10 shows the evaluation of the positive and negative ideal solutions of each management criterion.
(V) Calculation of the distance from the positive and negative ideal solutions.
The Euclidean distance from the ideal best solution (Equation (9)) is calculated as follows:
d i + = j = 1 n v i j x v j + x 2 ;   i = 1 , , m
The Euclidean distance from the ideal worst solution (Equation (10)) is then calculated as follows:
d i = j = 1 n v i j x v j x 2 ;   i = 1 , , m
Finally, Table 11 shows the results of calculating the best and worst Euclidean distances, as well as the relative proximity and the ranking of the best alternative (which is the WGP).
(VI) Calculation of the relative proximity Ri to the ideal solution (Equation (11)):
R i = d i d i + + d i ;   i = 1 , , m
As long as the relative proximity Ri is closer to the value of 1, the alternative is at a shorter distance from the positive ideal solution and at a longer distance from the negative ideal solution, leading to the final placement of the alternatives in descending order.

2.8. PROMETHEE Method

PROMETHEE is a multi-criteria decision technique that establishes a hierarchical ordering of a set A of alternatives through the evaluation of k criteria, g1, g2, …, gk. It requires that each criterion g1 be associated with a generalized criterion Pj(x), assessing the preference for an alternative with respect to an alternative b as a function of the difference between evaluations, g j a g j b [93]. It is common for the values of the weights (wj) to be known before using this method and to normalize the decision matrix using the attributes of beneficial or nonbeneficial criteria, with Equations (12) and (13), respectively.
R i j = x i j M i n ( x i j ) M a x ( x i j ) M i n ( x i j ) ;   i = 1 , , m ;   j = 1 ,   2 , n  
R i j = M a x ( x i j ) x i j   M a x ( x i j ) M i n ( x i j ) ;   i = 1 , , m ;   j = 1 ,   2 , n
where xij is the performance value of the ith alternative (m) to the jth criterion (n). Below, we indicate the procedure used in PROMETHEE to determine the best alternative (using data from Table 5).
Table 12 shows the results of evaluating the max–min (linear) normalization technique under the benefit criterion condition of use.
(I) Determination of deviation (dj) via pairwise comparison (Equation (14)):
d j a , b = g j a g j b
where dj(a, b) denotes the difference between the evaluations of a and b on each criterion.
The data in Table 13 were obtained by subtracting the data pairs from Table 12, which, for the purposes of this explanation (Table 13), only shows an extract of the complete procedure (omitting the comparisons for d2-1 to d2-6, d3-1 to d3-6, d4-1 to d4-6, d5-1 to d5-6, and d6-1 to d6-5).
(II) Definition of the preference index (P) multi-criteria (Equations (15) and (16)):
π a , b = j = 1 k P a , b w j
π b , a = j = 1 k P b , a w j
where wj > 0, (j = 1, 2, …, k) is the weight of importance associated with criterion j, where j = 1 k w j = 1 .
The data in Table 14 were obtained by multiplying the pairwise comparison deviations (dj) (from Table 13) by the importance weights (wj) (from Table 7). Similarly, only one extract is presented because it is of the same order as the previous table.
Table 15 shows an n × n matrix, which depends on the number of alternatives, which in this case was six. This aggregate preference matrix (with diagonal zeros) contains the sum of the data in Table 14, with a transposed arrangement in the IWRM row in Table 15.
(III) Determination of the flow of overcoming, or of positive and negative relevance:
+ a = 1 n 1 x ϵ A π a , x
a = 1 n 1 x ϵ A π x , a
Two management flows are considered, the outgoing and the incoming, which reflect the degree to which an alternative a dominates or is dominated; they are defined by Equations (17) and (18), where n is the number of alternatives.
(VI) Calculation of the net flow of relevance (net outranking flow) (Equation (19)):
a = + a a ,
sometimes rewritten in the following form (Equation (20)):
a = 1 n 1 j = 1 k x ϵ A P j a , x P j x , a w j
so that an alternative a will surpass another b if + a > a , and they will be indifferent if + a = a [93]. Finally, Table 16 shows a summary of the net flow of overcoming the alternatives, indicating the WGP as the best alternative and the WFD as the least effective.

3. Results and Discussion

3.1. AHP and the Best Management Alternative

According to the authors of [100], decision makers must have an excellent understanding of the selection process. Figure 4a shows the results of our comparisons of the five management criteria. The concept of “participation” had a 48% priority, which was greater than the other criteria, and thus represents the highest priority. For Andersson and Ostrom [101], success in institutions is achieved through participation across multiple scales, with participants gaining experience in the focal environment. The subsequent criteria with a high priority were “governance” and “decentralization”, with 20% and 18%, respectively. For Domínguez [2], participation, interaction, and organization among society, government, and academia contribute to improving water governance in basins; this will enable us to adequately manage water resources and provide water services to different levels of society. Water governance is implemented whenever resource management organizations establish an effective policy, together with an appropriate legal framework, to regulate and manage water to respond to environmental, economic, and socio-political needs. Within the CBD [25], management should be decentralized to the lowest appropriate level, that is, where decentralized systems can lead to greater efficiency, effectiveness, and equity. Management must involve all stakeholders and balance local interests with the public interest. The closer the managing agents are to the ecosystem, the greater their responsibility, accountability, participation, and use of local knowledge. According to Lebel [24], if people are not in good health, development cannot be sustainable. Thus, the criteria of “sustainable” and “health and well-being”, with 7% and 8% priority, respectively, depend on the three above-mentioned criteria to enhance them.
In Figure 4b–f, the results we obtained from the AHP are shown, comparing each criterion with each alternative. They present the best results (Figure 4b), with 38% and 36% for the WGP and SW, respectively, whereby the concept of “governance” was more important for both management approaches. The opposite occurs for IWRM and Ecohealth, where across the years, in the documents listed in Table 4, this concept was not discussed or cited in general. In the case of the EA and WFD, both present in the year 2000, the priority of this concept stood out, with an average between 11% and 8%, respectively; it was expressed minimally within these frameworks. Notably, our evaluations in Figure 4b–f were based on our own experiences, group sessions, and literature review. For Saldaña and Bonrostro [102], an advantage of the AHP is that it enables us to make any necessary corrections, as indicated by our validation of the consistency index (CI).
In Figure 4c, the “participation” criterion is shown to be an average trend among the six alternatives, with the highest value found for Ecohealth, at 23%; the minimum value, at 12%, was found for the SW. Therefore, “participation” was a coinciding and essential criterion in every approach; thus, it should be considered necessary. Hence, considering the importance of new (transdisciplinary) knowledge, Lebel [24] emphasized that scientists from various disciplines must contribute their knowledge and skills to both individuals in the community and decision makers. In Figure 4e, the “decentralization” criterion shows an upward and stable trend over time. Notably, it demonstrates the importance of not concentrating on decision making, and it had a 9% priority in IWRM and 20% priority in the WFD in the year 2000, and 17% for the SW in 2014. With respect to the “sustainable” criterion, similar values were observed between IWRM and the EA, with 26% and 23%, respectively. These results support the work of Guerrero et al. [14], who expressed that the EA complements the current thinking on IWRM, being both consistent and holistic, the first focused on ecosystem management and the second on water management. In Schneider et al. [22], the SW had the highest value of 28% due to research assessing the sustainability of water governance systems. In Figure 4f, the criterion “health and well-being” presents a higher priority for Ecohealth, with a priority of 29% and 42% for the WGP; these results were, however, expected, as the prism of Parkes et al. [47] is based on Ecohealth and IWRM.
Finally, Figure 4g shows the final priorities of choice, from the lowest to the highest, among the six focal alternatives. The framework that best represents or gives the most importance to the five focal management criteria was the WGP, followed by the SW; both approaches are recent (2010 and 2014). The normalized previous results [56]—with respect only to the first four approaches—in Figure 4h demonstrate that Ecohealth was the best choice or alternative due to its 69% acknowledgment of the five management criteria, followed by the EA and IWRM, with 66% and 57%, respectively; the lowest priority was shown by the WFD, with 50%. Observations from MCDM methods should be used as decision-support tools. The solutions produced by these methods should be taken as an indication of what the best course of action may be. The search to find the best MCDM method is still ongoing, and research into these methods remains essential in many scientific and engineering fields [56].

3.2. Calibration and the Best Choice of MCDM

Figure 5 shows the final comparison between the three MCDM methods used to determine which of the two methods, TOPSIS or PROMETHEE, presents the best correlation regarding the AHP method. For Pereira-Basílio [54], in the last 44 years, 33,201 authors have written 23,494 documents on multi-criteria methods. AHP, TOPSIS, and VIKOR were the methods discussed in the highest number of publications, followed by PROMETHEE, with 1382. This is because they are the methods most used by professionals to solve problems with multiple criteria. Using the AHP, we found that the best framework, based only on the five criteria (Figure 5a), according to the relative scale close to the ideal solution, was the WGP (with an evaluation of 0.79), followed by Ecohealth, with 0.69, and the EA and SW, with 0.66. Those with a lower relative degree of proximity to the ideal solution were IWRM, with 0.57, and the WFD, with 0.50. According to TOPSIS, a method based on ideal distance, the best management approach was the WGP of 2010, coinciding with the AHP. The second-best options, with evaluations of 0.33 and 0.32, were Ecohealth (from 1996) and the SW (from 2014), respectively. As the third-best management alternative, TOPSIS referred to the EA, with a result of 0.28. Finally, the lowest evaluations were attributed to IWRM and the WFD, both of which were below 15%. PROMETHEE, an outranking technique for determining how good or not one alternative is with respect to another, identified the best management frameworks; positive evaluations of the WGP of 2010 and Ecohealth of 1996 were made, as well as different negative evaluations of the SW, EA, IWRM, and WFD models. Figure 5b,c show the results of our comparisons—AHP-TOPSIS and AHP-PROMETHEE—using Pearson’s and Spearman’s correlation coefficients to measure the degree of correspondence among the results [103,104]; we obtained values of 0.951 (0.99) for TOPSIS (Figure 5b) and 0.910 (0.99) for PROMETHEE (Figure 5c). Notably, in Zavadskas et al. [105], a review of 136 publications showed that the AHP, TOPSIS, and PROMETHEE methods were present in 27%, 16%, and 5% of the total, respectively, occupying 48% of the decision-making methods applied to engineering and management problems.

3.3. The Best Approaches to Water–Environmental Management

As the best AHP-TOPSIS correlation (Figure 5b), Figure 6 shows the simulation we performed using TOPSIS to analyze 5, 10, 15, 20, and 25 terms or management criteria; it thus partially defines which of the six management frameworks (or alternatives) are the best when using such criteria. Figure 6a,b show a contrast between the comparison of 5 and 10 criteria, where IWRM increased from 11% to 49%; therefore, in IWRM, there are more significant criteria. For example, the EA increases from 28% to 32% in terms of the relative degree of significance. The opposite is the case for the Ecohealth and WGP approaches, where the relative degrees decrease from 33% to 31% and from 72% to 57%, suggesting that the new management criteria being incorporated are not as relevant or important within these frameworks. Figure 6c, with 15 evaluated criteria, thus indicates that the best option continues to be the WGP, with 51%; however, its position continues to decline as the best option. The second-best alternative is now IWRM, with 47%; despite being 30 years old, it maintains relevant terms such as economic, ecosystem, water uses, development, quality, and quantity. With the exception of the WS (35%), the rest of the alternatives oscillate, on average, around 30%; the degree of relative importance of the WFD increases by 31%, and within the WFD, the presence of the 15 criteria is both notable and indispensable.
Finally, Figure 6d,e represent similar behavior, where the percentage of relative closeness to the ideal solution increases slightly from 20 to 25 management criteria; the WFD is now shown to be the best option, with 47%, followed by the WGP and IWRM, with 45% and 41%, including the criteria of environment, demand, issues/problems, watersheds/basins, and agriculture. Ecohealth and the EA (and regarding the 10 to 25 criteria used), remain the same between 28% and 31%, respectively, on average, indicating the limited presence/relevance of focal concepts. Notably, the SW of 2014 has remained stable, with an average of 35% for all criteria. This is an expected behavior that is perhaps due to the recent formation of the SW; being newer, it incorporates aspects from all prior management approaches. Additionally, according to Schneider et al. [22], this framework is advantageous in that it enables a holistic and comprehensive perspective on water sustainability to be obtained. To Heldt et al. [106], the WFD is a potential tool to improve IWRM implementation globally in transitioning and developing countries. This can be achieved with clear guidelines and has been successful within the European Union, as well as in candidate and neighboring countries. The WFD is a comprehensive approach to water management that takes into account the various national and local political contexts in Europe.

3.4. Frequency and Importance of Concepts

Figure 7 shows the results of our analysis of the 25 criteria. Figure 7a represents the possible behavior, over time, of the five criteria for each management framework. The concept of “governance” is present mostly in the WGP of 2010 and in the SW of 2014, but not in the rest of the approaches. For the “participation” criterion, IWRM (of 1992) and Ecohealth (of 1996) emphasize the importance of mentioning this criterion more than the other approaches. The “sustainable” criterion is consistently present from 1992 to 2014. “Decentralization”, however, is a concept that is rarely found or is not given due importance. The criterion of “health and well-being” has 25, 120, and 188 mentions in IWRM (of 1992), Ecohealth (of 1996), and WGP (of 2010), respectively; however, it is not mentioned as a whole term, but rather as health or well-being individually, thereby demonstrating the importance of each at the time of the citation. This is why the WGP has a greater relevance (75%) to the mention of such criteria (see Figure 6a).
In Figure 7c, “pollution” is present in IWRM and Ecohealth and has a greater presence in the WFD, with 110 mentions. Although “social” aspects are present in all approaches, this criterion is essential in the WGP (of 2010), just as the criterion of “water resources” is in IWRM (of 1992); less mentioned are the concepts of “access” to water and “resilience”. In Figure 6b, the WGP continues to place emphasis on and address new criteria, with 57%, followed by IWRM and the SW, with 49% and 35%, respectively.
Figure 7c shows that the new focal criteria have importance in all six approaches; “quality and quantity” is more relevant in the WFD (of 2000), “ecosystems” in the EA (of 2000), and “development” in IWRM (of 1992). In Figure 7d, the “water” criterion is the most important term in the WFD (from 2000), followed by IWRM (from 1992), and, to a lesser extent, the SW (from 2014). Notably, the “groundwater” criterion is mentioned 171 times in the WFD, followed by its 29 mentions in IWRM and 5 in the SW; it is less important in the other approaches. For Howard and Merrifield [107], the protection and management of groundwater-dependent ecosystems (GDEs) is, for many, an abstract problem. For Foster et al. [108], groundwater plays an integral role by sustaining different types of aquatic, terrestrial, and coastal ecosystems, both in humid and arid regions. This is a key factor in maintaining the ecological integrity of GDEs. Regarding the last five criteria, Figure 7e shows the importance of the concepts “environment”, “issues/problems”, and “watershed/basin”. However, Figure 7a,b,d,e also reflect the criteria that are not as relevant, such as “decentralization”, “resilience”, “stakeholders”, and, to a certain degree, “demand”. We can even observe how it has decreased in importance over time, alongside “participation”, “economic”, “environment”, and “agriculture”. The aspects that have increased in importance over time can also be observed, e.g., “governance”, “ecosystem”, and “scale/level”.

3.5. Exploratory Analysis of Management Concepts

Figure 8 shows the distribution and relationships of the criteria within the six water–environmental management frameworks. According to Meglen [109], Vásquez-Hurtado [110], and Marín-Diazaraque [111], factor analysis is a statistical data reduction technique used to explain correlations between observed variables. It facilitates the identification of variables with high loads, low loads, or those related to both axes on the same factor. The axis of the principal component “F1” is composed of IWRM, the SW, and the WFD, with correlation coefficients of 0.95, 0.91, and 0.89, respectively; the axis of the main component “F2” is composed of Ecohealth, the WGP, and the EA, with correlation coefficients of 0.81, 0.70, and 0.55, respectively, revealing the high similarity between the approaches in both components. In the main component “F2”, the criteria most closely associated with Ecohealth and the EA are “ecosystem” and “health and well-being”, and the criteria that are relatively far apart are “social”, “human”, and “issues”, highlighting the strong association among the three. In the upper quadrants, the WGP has a good relationship with the criteria “environmental”, “ecosystem”, and “health and well-being”. With respect to the main component “F1”, the criterion “water” has a strong relationship with IWRM, the SW, and the WFD. Due to its proximity to the origin, there is a diminished relationship with the criteria “watersheds”, “scale/level”, and “development”. Finally, the criteria “governance”, “participation”, “economic”, “agriculture”, “stakeholders”, “pollution”, “access”, “resilience”, “decentralization”, “demand”, and “water uses” have similar characteristics because they are relatively close to each other and have a weak relationship with the management alternatives; this is because they are opposed to these (i.e., they are alternatives). Wilson et al. [112] identified that water governance too often remains depoliticized and devoid of political content or meaning. It is necessary to re-center and re-theorize “the political” in relation to decision making, use, and management, collectively. These criteria are displayed in Figure 7; most are cited only when necessary, meaning they have little relevance to these management alternatives. The criteria “quality and quantity” of water, “water resources”, and “groundwater” are more referenced and evident in the WFD, SW, and IWRM compared to Ecohealth, the EA, and the WGP. Notably, Figure 8 can thus be interpreted as a criticism; it indicates the criteria that are more important and those that are not. For example, Bunch et al. [113] discussed the integration of watershed management and public health; their results specifically concerned the biophysical determinants of health, and thus they neglected the most relevant social aspects. In the negative–negative quadrant, some social management criteria with relationships or closeness are shown.

3.6. Is the WFD the Best Management Approach Option?

The WFD can be seen as a success in Europe, thanks to its clear guidelines for all member states. The WFD may also support IWRM planning in non-EU countries [106]. Since 2001, support for Central and Eastern Europe (CEE) countries, as they seek to implement the WFD, has successfully delivered more than 30 guidance documents and policy papers [18]. Based on the previous results, the WFD appears to be the best management approach option. The five reasons outlined below justify this statement.
High Priority in Criteria: According to the results of the TOPSIS, the WFD received the highest priority among the five management criteria that were evaluated. The criteria of “water”, “environment”, and “quality and quantity” were considered important, and the WFD scored well in these aspects. The participation and decentralization aligns with the WFD’s focus on involving stakeholders and decentralized decision making for effective water resource management.
The results show that the methods used to evaluate the different management approaches, such as TOPSIS, consistently point to the WFD as one of the preferred options. These methods provide decision makers with valuable information and support to make informed decisions. Thus, the WFD is a strong candidate for effective water and environmental management due to its emphasis on groundwater, basin management, the pollution of water bodies, and adaptability to diverse contexts.
Support for ecosystems: This aligns with the idea that human well-being is intricately connected to water resources. The results imply that the WFD acknowledges the need to safeguard water quality for human consumption and ecosystem health.
Transdisciplinary Knowledge: The WFD appears to consider the importance of incorporating knowledge from various disciplines, including scientists, communities, and decision makers. This transdisciplinary approach aligns with the idea that effective water management requires collaboration and input from diverse stakeholders.
Adaptation to Changing Contexts: The WFD’s flexibility and adaptability to different political, social, and environmental contexts are highlighted as key strengths. This adaptability allows the WFD to be applied effectively in various regions, responding to specific needs and challenges.

4. Conclusions

With this work, we managed to answer the question of which of the six integrated management approaches demonstrates the best performance by analyzing the frequency of multiple basin management criteria cited in the base documents. Of the six management theoretical frameworks analyzed, four can be considered classic (IWRM, EA, Ecohealth, and WFD), whereas the remaining two are recent (WGP and SW).
  • Using the AHP method, we found that the WGP (from 2010) was one of the best integrated management alternatives evaluated, with 79%, followed by Ecohealth (from 1996) with 69%; lower priority was given to the management criteria in IWRM and the WFD. It should be noted that in the evaluation of the method, only five management criteria were considered: governance, participation, sustainability, decentralization, and health and well-being.
  • Expanding the spectrum to 25 management criteria, using the TOPSIS method, the WFD (of 2000) was found to be one of the best integrated management alternatives evaluated, with 79%, followed by the WGP (of 2010), with 45%, and IWRM, with 41%. (of 1992). The remaining approaches had a priority lower than 32% (EA, SW, and Ecohealth).
  • Of the two MCDM methods we used to calibrate the AHP (using five management criteria), TOPSIS had the best fit, with a Pearson’s coefficient of 0.951 and Spearman’s coefficient of 0.99, which were slightly above the 0.910 (0.99) achieved by PROMETHEE. Choosing to use this calibration between AHP-TOPSIS, we considered extrapolating it and applying it to the 10, 15, 20, and 25 management criteria. We considered it more practical to use the method based on the ideal distance rather than a pairwise comparison, as the latter requires more time.
  • The concept of “water” was the criterion most mentioned in the WFD, with 446 hits, followed by 349 in IWRM, and 202 hits in the SW. The following criteria had a high number of hits within all approaches: “environment”, “health and well-being”, “ecosystem”, “scale/level”, “watersheds/basin”, and “quality and quantity”. In contrast, the least-cited criteria were “resilience” and “decentralization”, with a total of 11 and 8 mentions, respectively, among the six approaches. Only the WGP mentioned the concept of “resilience” six times, and the EA mentioned the concept of “decentralization” six times.
  • These theoretical frameworks are the result of the hard work of many people and institutions, reflecting a concern for the conservation of water–environmental resources of basins. However, through an exploratory approach involving patterns shown in bi-plots and using the numbers of citations, we observed that these approaches placed little importance on concepts such as participation, decision making, resilience, water demand and contamination, and groundwater, among others. Since these criteria were located in the negative–negative quadrant, we interpreted a lack of prioritization.
  • The three best evaluated approaches were the WFD, WGP, and IWRM. The first provides a holistic approach by integrating multiple perspectives and dimensions, including ecosystem structure and function, governance and institutions, livelihoods, and well-being. The second promotes efforts to improve water quality, protect ecosystems, and involve communities in decision-making processes, sharing best practices and lessons learned to achieve sustainable water management. Additionally, the third requires political commitment, institutional capacity, stakeholder participation, and effective governance mechanisms for its implementation.
Of the six theoretical frameworks, all have their specific characteristics. However, they may be improved if they are combined or complemented, as all have aspects that are beneficial for the water resources of basins. Issues concerning water and environmental resources are complex, as they have a bearing on a wide range of areas and a large scale. Each approach has advantages and disadvantages that need to be analyzed in the context of the circumstances of each specific area. Therefore, each program’s approach should be adapted to the context in which it is going to be implemented; one size does not fit all in a complex world.

Author Contributions

Conceptualization, F.A.V., L.F.N. and E.G.R.; Methodology, F.A.V. and E.G.R.; Software, S.S.S. and D.M.-R.; Validation, F.A.V. and E.G.R.; Formal analysis, F.A.V., C.R.S. and S.O.-O.; Investigation, F.A.V., L.F.N. and S.O.-O.; Resources, C.R.S. and E.G.R.; Data Curation, S.S.S. and D.M.-R.; Writing—original draft preparation, F.A.V., L.F.N. and S.S.S.; Writing—review and editing, L.F.N., C.R.S., S.O.-O., S.S.S. and D.M.-R.; Supervision, F.A.V. and L.F.N. All authors have read and agreed to the published version of the manuscript.


This research received no external funding.

Data Availability Statement

Not applicable.


We express our gratitude to the DCBI at Metropolitan Autonomous University—Iztapalapa (UAM–I) for the support we have received.

Conflicts of Interest

The authors declare no conflict of interest.


  1. FNWA. European Declaration for a New Water Culture. Foundation for a New Water Culture. 2005. Available online: (accessed on 25 March 2023).
  2. Domínguez, S.J. Hacia una Buena Gobernanza Para la Gestión Integrada de los Recursos Hídricos (Towards Good Governance for the Integrated Water Resources Management). Proceso Regional de las Américas VI foro Mundial del Agua. 2012. Available online: (accessed on 15 October 2021).
  3. Andrade-Pérez, A. Aplicación del Enfoque Ecosistémico en Latinoamérica (Application of the Ecosystem Approach in Latin America); Commission on Ecosystem Management—International Union for Conservation of Nature, IUCN: Gland, Switzerland, 2007. [Google Scholar]
  4. Perevochtchikova, M. Cultura del Agua en México: Conceptualización y Vulnerabilidad Social (Water Culture in Mexico: Conceptualization and Social Vulnerability); México, D.F., Ed.; Universidad Nacional Autonoma de México: Miguel Ángel Porrúa, México, 2012. [Google Scholar]
  5. Cardwell, H.E.; Cole, R.A.; Cartwright, L.A.; Martin, L.A. Integrated water resources management: Definitions and conceptual musings. J. Contemp. Water Res. Educ. 2006, 135, 8–18. [Google Scholar] [CrossRef]
  6. Navarro-Ramírez, V.; Ramírez-Hernandez, J.; Gil-Samaniego, M.; Rodríguez-Burgueño, J.E. Methodological frameworks to assess sustainable water resources management in industry: A review. Ecol. Indic. 2020, 119, 106819. [Google Scholar] [CrossRef]
  7. Tsani, S.; Koundouri, P.; Akinsete, E. Resource management and sustainable development: A review of the European water policies in accordance with the United Nations’ Sustainable Development Goals. Environ. Sci. Policy 2020, 114, 570–579. [Google Scholar] [CrossRef]
  8. Alamanos, A.; Koundouri, P.; Papadaki, L.; Pliakou, T.; Toli, E. Water for Tomorrow: A Living Lab on the Creation of the Science-Policy-Stakeholder Interface. Water 2022, 14, 2879. [Google Scholar] [CrossRef]
  9. Giupponi, C.; Jakeman, A.J.; Karssenberg, D.; Hare, M.P. Sustainable Management of Water Resources; Edward Elgar Publishing: Dratham, UK, 2006. [Google Scholar]
  10. Tzanakakis, V.A.; Angelakis, A.N.; Paranychianakis, N.V.; Dialynas, Y.G.; Tchobanoglous, G. Challenges and opportunities for sustainable management of water resources in the island of Crete, Greece. Water 2020, 12, 1538. [Google Scholar] [CrossRef]
  11. Owen, L.A.; Pickering, K.T. An Introduction to Global Environmental Issues; Routledge: London, UK, 2006. [Google Scholar]
  12. Hardoy, J.E.; Satterthwaite, D. Environmental problems of Third World cities: A global issue ignored? Public Adm. Dev. 1991, 11, 341–361. [Google Scholar] [CrossRef]
  13. Umarjonovna, D.D.; Akbaralievna, Y.M. Global Environmental Problems and Their Solution. Web Semant. Univers. J. Innov. Educ. 2023, 2, 326–330. [Google Scholar]
  14. Guerrero, E.; de Keizer, O.; Córdoba, R. La Aplicación del Enfoque Ecosistémico en la Gestión de los Recursos Hídricos (The Application of the Ecosystem Approach in the Management of Water Resources); UICN: Quito, Ecuador, 2006. [Google Scholar]
  15. Calizaya, A.; Meixner, O.; Bengtsson, L.; Berndtsson, R. Multi-criteria decision analysis (MCDA) for integrated water resources management (IWRM) in the Lake Poopo Basin, Bolivia. Water Resour. Manag. 2010, 24, 2267–2289. [Google Scholar] [CrossRef]
  16. Chávez, M.V.H.; Osuna, Y.R. Metodología de análisis multicriterio aplicación al crecimiento sostenible en la Unión Europea (Multicriteria analysis methodology application to sustainable growth in the European Union). Gestión Terc. Milen. 2013, 16, 19–28. [Google Scholar] [CrossRef]
  17. Geng, G.; Wardlaw, R. Application of multi-criterion decision making analysis to integrated water resources management. Water Resour. Manag. 2013, 27, 3191–3207. [Google Scholar] [CrossRef]
  18. Global Water Partnership (GWP). Integrated Water Resources Management in Central and Eastern Europe: IWRM vs EU Water Framework Directive; GWP: Elanders, Sweden, 2015; Available online: (accessed on 3 August 2023).
  19. Widianingsih, I.; Paskarina, C.; Riswanda, R.; Putera, P.B. Evolutionary Study of watershed governance research: A bibliometric analysis. Sci. Technol. Libr. 2021, 40, 416–434. [Google Scholar] [CrossRef]
  20. De Filippo, D.; Sanz Casado, E.; Berteni, F.; Barisani, F.; Bautista Puig, N.; Grossi, G. Assessing citizen science methods in IWRM for a new science shop: A bibliometric approach. Hydrol. Sci. J. 2021, 66, 179–192. [Google Scholar] [CrossRef]
  21. Parkes, M.W.; Morrison, K.E.; Bunch, M.J.; Hallström, L.K.; Neudoerffer, R.C.; Venema, H.D.; Waltner-Toews, D. Towards integrated governance for water, health and social–ecological systems: The watershed governance prism. Glob. Environ. Chang. 2010, 20, 693–704. [Google Scholar] [CrossRef]
  22. Schneider, F.; Bonriposi, M.; Graefe, O.; Herweg, K.; Homewood, C.; Huss, M.; Kauzlaric, M.; Liniger, H.; Rey, E.; Reynard, E.; et al. Assessing the sustainability of water governance systems: The sustainability wheel. J. Environ. Plan. Manag. 2014, 58, 1577–1600. [Google Scholar] [CrossRef]
  23. International Conference on Water and the Environment (ICWE). The dublin statement and report of the conference. In Proceedings of the International Conference on Water and the Environment: Development Issues for the 21st Century, Dublin, Ireland, 26–31 January 1992; WMO: Geneva, Switzerland, 1992. [Google Scholar]
  24. Lebel, J. Health: An Ecosystem Approach; International Development Research Centre: Ottawa, ON, Canada, 2003. [Google Scholar]
  25. Convention on Biological Diversity (CBD). Convention on Biological Diversity. Ecosystem Approach. Available online: (accessed on 22 April 2022).
  26. EUR-Lex. Directive 2000/60/EC of the European Parliament, and of the Council of 23 October 2000 Establishing a Framework for Community Action in the Field of Water Policy. OJEC, L 327/1. 2000. Available online: (accessed on 3 December 2022).
  27. Brundtland, G.H. Our Common Future, Chairman’s Foreword. UN Documents: Gathering a Body of Global Agreements. UN Documents. 1987. Available online: (accessed on 11 March 2022).
  28. Gain, A.K.; Rouillard, J.J.; Benson, D. Can integrated water resources management increase adaptive capacity to climate change adaptation? A critical review. J. Water Resour. Prot. 2013, 5, 11–20. [Google Scholar] [CrossRef]
  29. Lautze, J.; De Silva, S.; Giordano, M.; Sanford, L. Putting the cart before the horse: Water governance and IWRM. In Natural Resources Forum; Blackwell Publishing Ltd.: Oxford, UK, 2011; Volume 35, pp. 1–8. [Google Scholar] [CrossRef]
  30. Xie, M. Integrated water resources management (IWRM)–introduction to principles and practices. In Africa Regional Workshop; IWRM: Nairobi, Kenya, 2006. [Google Scholar]
  31. Savenije, H.H.G.; Van Der Zaag, P. Integrated water resources management: Concepts and issues. Phys. Chem. Earth Parts A/B/C 2008, 33, 290–297. [Google Scholar] [CrossRef]
  32. Global Water Partnership (GWP). 2020. Available online: (accessed on 11 July 2022).
  33. Dakubo, C.Y. Ecosystems and Human Health: A Critical Approach to Ecohealth Research and Practice; Springer Science & Business Media: Berlin/Heidelberg, Germany, 2010. [Google Scholar]
  34. Bunch, M.J.; Morrison, K.E.; Parkes, M.W.; Venema, H.D. Promoting health and well-being by managing for social–ecological resilience: The potential of integrating ecohealth and water resources management approaches. Ecol. Soc. 2011, 16, 6. [Google Scholar] [CrossRef]
  35. Webb, J.C.; Mergler, D.; Parkes, M.W.; Saint-Charles, J.; Spiegel, J.; Waltner-Toews, D.; Yassi, A.; Woollard, R.F. Tools for thoughtful action: The role of ecosystem approaches to health in enhancing public health. Can. J. Public Health 2010, 101, 439–441. [Google Scholar] [CrossRef]
  36. Lerner, H.; Berg, C. A Comparison of Three Holistic Approaches to Health: One Health, EcoHealth, and Planetary Health. Front. Veter-Sci. 2017, 4, 163. [Google Scholar] [CrossRef] [PubMed]
  37. Charron, D.F. Investigación de Ecosalud en la Práctica: Aplicaciones Innovadoras de un Enfoque Ecosistémico para la Salud (Ecohealth Research in Practice: Innovative Applications of an Ecosystem Approach to Health); CIID: Ottawa, ON, Canada, 2014. [Google Scholar]
  38. Jian, F.; Jayas, D.S. The Ecosystem Approach to Grain Storage. Agric. Res. 2012, 1, 148–156. [Google Scholar] [CrossRef]
  39. Morishita, J. What is the ecosystem approach for fisheries management? Mar. Policy 2008, 32, 19–26. [Google Scholar] [CrossRef]
  40. Shepherd, G. The Ecosystem Approach: Learning from Experience, 5th ed.; IUCN: Gland, Switzerland, 2008. [Google Scholar]
  41. Waltner-Toews, D.; Kay, J.J.; Lister, N.M.E. (Eds.) The Ecosystem Approach: Complexity, Uncertainty, and Managing for Sustainability; Columbia University Press: New York, NY, USA, 2008. [Google Scholar]
  42. Euwater, R. Declaración Europea por una nueva cultura del agua (European Declaration for a new water culture). Cuad. Del CENDES 2005, 22, 161–163. [Google Scholar]
  43. Kallis, G.; Butler, D. The EU water framework directive: Measures and implications. Water Policy 2001, 3, 125–142. [Google Scholar] [CrossRef]
  44. Mostert, E. The European Water Framework Directive and water management research. Phys. Chem. Earth Parts A/B/C 2003, 28, 523–527. [Google Scholar] [CrossRef]
  45. Voulvoulis, N.; Arpon, K.D.; Giakoumis, T. The EU Water Framework Directive: From great expectations to problems with implementation. Sci. Total Environ. 2017, 575, 358–366. [Google Scholar] [CrossRef]
  46. Parkes, M.; Eyles, R.; Benwell, G.; Panelli, R.; Townsend, C.; Weinstein, P. Integration of ecology and health research at the catchment scale: The Taieri River catchment, New Zealand. J. Rural. Remote Environ. Health 2004, 3, 1–17. [Google Scholar]
  47. Parkes, M.W.; Morrison, K.E.; Bunch, M.J.; Venema, H.D. EcoHealth and Watersheds: Ecosystem Approaches to Re-Integrate Water Resources Management with Health and Well-Being; Network for Ecosystem Sustainability and Health (Publication Series No. 2) and the International Institute for Sustainable Development: Winnipeg, MB, Canada, 2008. [Google Scholar]
  48. Bunch, M.J.; Waltner-Toews, D. Grappling with complexity: The context for one health and the ecohealth approach. In One Health: The Theory and Practice of Integrated Health Approaches; Zinsstag, J., Schelling, E., Waltner-Toews, D., Whittaker, M., Tanner, M., Eds.; CAB International: Wallingford, UK, 2015; pp. 415–426. [Google Scholar]
  49. Armas-Vargas, F.; Escolero, O.; Sandoval-Solis, S.; Nava, L.F.; Mazari-Hiriart, M.; Rojas-Serna, C.; López-Corona, O. A Quantitative Approach to the Watershed Governance Prism: The Duero River Basin, Mexico. Water 2023, 15, 743. [Google Scholar] [CrossRef]
  50. Schneider, F. Exploring water sustainability through stakeholders’ perspectives and hybrid water in the Swiss Alps. Water Altern. 2015, 8, 280–296. [Google Scholar]
  51. Schneider, F.; Bonriposi, M.; Graefe, O.; Herweg, K.; Homewood, C.; Huss, M.; Kauzlaric, M.; Liniger, H.; Rey, E.; Reynard, E.; et al. MontanAqua: Tackling Water Stress in the Alps: Water Management Options in the Crans-Montana-Sierre Region (Valais). GAIA-Ecol. Perspect. Sci. Soc. 2016, 25, 191–193. [Google Scholar] [CrossRef]
  52. Watrobski, J.; Jankowski, J.; Ziemba, P.; Karczmarczyk, A.; Ziolo, M. Generalised framework for multi-criteria method selection. Omega 2019, 86, 107–124. [Google Scholar] [CrossRef]
  53. Taherdoost, H.; Madanchian, M. Multi-Criteria Decision Making (MCDM) Methods and Concepts. Encyclopedia 2023, 3, 77–87. [Google Scholar] [CrossRef]
  54. Pereira-Basílio, M.; Pereira, V.; Costa, H.G.; Santos, M.; Ghosh, A. A systematic review of the applications of multi-criteria decision aid methods (1977–2022). Electronics 2022, 11, 1720. [Google Scholar] [CrossRef]
  55. Saaty, T.L. How to make a decision: The analytic hierarchy process. Eur. J. Oper. Res. 1990, 48, 9–26. [Google Scholar] [CrossRef]
  56. Triantaphyllou, E.; Mann, S.H. Using the analytic hierarchy process for decision making in engineering applications: Some challenges. Int. J. Ind. Eng. Appl. Pract. 1995, 2, 35–44. [Google Scholar]
  57. Blachowski, J. Methodology for assessment of the accessibility of a brown coal deposit with analytical hierarchy process and weighted linear combination. Environ. Earth Sci. 2015, 74, 4119–4131. [Google Scholar] [CrossRef]
  58. Bivina, G.R.; Parida, M. Prioritizing pedestrian needs using a multi-criteria decision approach for a sustainable built environment in the Indian context. Environ. Dev. Sustain. 2020, 22, 4929–4950. [Google Scholar] [CrossRef]
  59. Hwang, C.L.; Yoon, K. Multiple Attribute Decision Making: Methods and Applications. A State-of-the-Art Survey; Springer: Berlin/Heidelberg, Germany, 1981. [Google Scholar]
  60. Brans, J.P.; Vincke, P.; Mareschal, B. How to select and how to rank projects: The PROMETHEE method. Eur. J. Oper. Res. 1986, 24, 228–238. [Google Scholar] [CrossRef]
  61. Georgopoulou, E.; Sarafidis, Y.; Diakoulaki, D. Design and implementation of a group DSS for sustaining renewable energies exploitation. Eur. J. Oper. Res. 1998, 109, 483–500. [Google Scholar] [CrossRef]
  62. Kolios, A.; Mytilinou, V.; Lozano-Minguez, E.; Salonitis, K. A comparative study of multiple-criteria decision-making methods under stochastic inputs. Energies 2016, 9, 566. [Google Scholar] [CrossRef]
  63. Soares de Assis, G.; dos Santos, M.; Basílio, M.P. Use of the WASPAS Method to Select Suitable Helicopters for Aerial Activity Carried Out by the Military Police of the State of Rio de Janeiro. Axioms 2023, 12, 77. [Google Scholar] [CrossRef]
  64. Bączkiewicz, A.; Wątróbski, J.; Kizielewicz, B.; Sałabun, W. Towards Objectification of Multi-Criteria Assessments: A Comparative Study on MCDA Methods. In Proceedings of the 2021 16th Conference on Computer Science and Intelligence Systems (FedCSIS), Online, 2–5 September 2021. [Google Scholar]
  65. Wątróbski, J.; Bączkiewicz, A.; Ziemba, E.; Sałabun, W. Sustainable cities and communities assessment using the DARIA-TOPSIS method. Sustain. Cities Soc. 2022, 83, 103926. [Google Scholar] [CrossRef]
  66. Behzadian, M.; Otaghsara, S.K.; Yazdani, M.; Ignatius, J. A state-of the-art survey of TOPSIS applications. Expert Syst. Appl. 2012, 39, 13051–13069. [Google Scholar] [CrossRef]
  67. Sałabun, W.; Piegat, A.; Wątróbski, J.; Karczmarczyk, A.; Jankowski, J. The COMET method: The first MCDA method completely resistant to rank reversal paradox. Eur. Work. Group Ser. 2019, 3, 10–16. [Google Scholar]
  68. Więckowski, J.; Kołodziejczyk, J. Swimming progression evaluation by assessment model based on the COMET method. Procedia Comput. Sci. 2020, 176, 3514–3523. [Google Scholar] [CrossRef]
  69. Kizielewicz, B.; Dobryakova, L. MCDA based approach to sports players’ evaluation under incomplete knowledge. Procedia Comput. Sci. 2020, 176, 3524–3535. [Google Scholar] [CrossRef]
  70. Sałabun, W.; Ziemba, P.; Wątróbski, J. The Rank Reversals Paradox in Management Decisions: The Comparison of the AHP and COMET methods, In International Conference on Intelligent Decision Technologies 2016. IDT 2016. Smart Innovation, Systems and Technologies; Czarnowski, I., Caballero, A., Howlett, R., Jain, L., Eds.; Springer: Berlin/Heidelberg, Germany, 2016; Volume 56. [Google Scholar]
  71. Teixeira, L.F.H.; Santos, M.; Gomes, C.F. Proposta e implementação em python do método Simple Aggregation of Preferences Expressed by Ordinal Vectors: Multi decision makers: Uma ferramenta web simples e intuitiva para Apoio à Decisão Multicritério. In Simpósio de Pesquisa Operacional e Logística da Marinha; Blucher: Rio de Janeiro, Brazil, 6–8 November 2019; pp. 1–17. Available online: (accessed on 29 June 2023).
  72. Bausys, R.; Zavadskas, E.K.; Kaklauskas, A. Application of neutrosophic set to multicriteria decision making by COPRAS. In Economic Computation and Economic Cybernetics Studies and Research (ECECSR); Academy of Economic Studies: Bucharest, Romania, 2015; pp. 91–106. [Google Scholar]
  73. Yazdani, M.; Jahan, A.; Zavadskas, E. Analysis in material selection: Influence of normalization tools on COPRAS-G. Econ. Comput. Econ. Cybern. Stud. Res. 2017, 51, 59–74. [Google Scholar]
  74. Oppio, A.; Bottero, M.; Arcidiacono, A. Assessing urban quality: A proposal for a MCDA evaluation framework. Ann. Oper. Res. 2018, 312, 1427–1444. [Google Scholar] [CrossRef]
  75. Behzadian, M.; Kazemzadeh, R.B.; Albadvi, A.; Aghdasi, M. PROMETHEE: A comprehensive literature review on methodologies and applications. Eur. J. Oper. Res. 2010, 200, 198–215. [Google Scholar] [CrossRef]
  76. Moreira, M.Â.L.; Dos Santos, M.; Gomes, C.F.S. Proposta De Modelagem Híbrida Promethee-Sapevo-M1: Avaliação Multicritério de Drones para Emprego na Guerra Naval. In Proceedings of the Simpósio de Pesquisa Operacional e Logística da Marinha, Rio de Janeiro, Brazil, 6–8 November 2019; pp. 2253–2271. [Google Scholar]
  77. Gomes, L.F.A.M.; Mury, A.R.; Gomes, C.F.S. Multicriteria ranking with ordinal data. Syst. Anal. Model. Simul. 1997, 27, 139–145. [Google Scholar]
  78. Kittur, J. Using the PROMETHEE and TOPSIS multi-criteria decision making methods to evaluate optimal generation. In Proceedings of the 2015 International Conference on Power and Advanced Control Engineering (ICPACE), Bengaluru, India, 12–14 August 2015; pp. 80–85. [Google Scholar] [CrossRef]
  79. De Moura, P.D.; Dos Santos, M.; Pinheiro, I.A.; Lellis, M.A.M.; Vilarinho, A.T.; De Souza, C.R.; Simoes, C.F.G. Multicriteria and Statistical Approach to Support the Outranking Analysis of the OECD Countries. IEEE Access 2022, 10, 69714–69726. [Google Scholar] [CrossRef]
  80. Santoyo-Castelazo, E.; Azapagic, A. Sustainability assessment of energy systems: Integrating environmental, economic and social aspects. J. Clean. Prod. 2014, 80, 119–138. [Google Scholar] [CrossRef]
  81. Villa-Silva, A.J.; Pérez-Domínguez, L.; Martínez-Gómez, E.; Pérez-Olguín, I.J.C.; Durán-Almeraz, S.N. Una revisión de literatura de 1980 a 2018 de los métodos Multi-criterio (A literature review from 1980 to 2018 of Multi-criteria methods). Mundo FESC 2019, 9, 89–102. [Google Scholar]
  82. Kizielewicz, B.; Shekhovtsov, A.; Sałabun, W. pymcdm—The universal library for solving multi-criteria decision-making problems. SoftwareX 2023, 22, 101368. [Google Scholar] [CrossRef]
  83. Bazrafkan, A.; Pakravan, M.R. An MADM network selection approach for next generation heterogeneous networks. In Proceedings of the Iranian Conference on Electrical Engineering (ICEE), Tehran, Iran, 2–4 May 2017; pp. 1884–1890. [Google Scholar] [CrossRef]
  84. Jahan, A.; Edwards, K.L. A state-of-the-art survey on the influence of normalization techniques in ranking: Improving the materials selection process in engineering design. Mater. Des. 2015, 65, 335–342. [Google Scholar] [CrossRef]
  85. Vafaei, N.; Ribeiro, R.A.; Camarinha-Matos, L.M. Assessing normalization techniques for simple additive weighting method. Procedia Comput. Sci. 2022, 199, 1229–1236. [Google Scholar] [CrossRef]
  86. Fontana, M. Métodos de Decisión Multicriterio AHP y PROMETHEE Aplicados a la Elección de un Dispositivo Móvil (AHP and PROMETHEE Multicriteria Decision Methods Applied to the Choice of a Mobile Device); Escuela Técnica Superior de Ingeniería Universidad de Sevilla: Sevilla, España, 2015. [Google Scholar]
  87. Marković, Z. Modification of TOPSIS method for solving of multicriteria tasks. Yugosl. J. Oper Res. 2010, 20, 117–143. [Google Scholar] [CrossRef]
  88. Zavadskas, E.K.; Turskis, Z. A new logarithmic normalization method in games theory. Informatica 2008, 19, 303–314. [Google Scholar] [CrossRef]
  89. Jahan, A.; Bahraminasab, M.; Edwards, K.L. A target-based normalization technique for materials selection. Mater. Des. 2012, 35, 647–654. [Google Scholar] [CrossRef]
  90. Zeng, Q.-L.; Li, D.-D.; Yang, Y.-B. VIKOR method with enhanced accuracy for multiple criteria decision making in healthcare management. J. Med. Syst. 2013, 37, 9908. [Google Scholar] [CrossRef]
  91. Ture, H.; Dogan, S.; Kocak, D. Assessing euro 2020 strategy using multi-criteria decision making methods: VIKOR and TOPSIS. Soc. Indic. Res. 2019, 142, 645–665. [Google Scholar] [CrossRef]
  92. Tzeng, G.H.; Huang, J.J. Multiple Attribute Decision Making: Methods and Applications; CRC Press: Boca Raton, FL, USA, 2011. [Google Scholar]
  93. Brans, J.P.; Mareschal, B. Promethee methods. In Multiple Criteria Decision Analysis: State of the Art Surveys; Figueira, J., Greco, S., Ehrogott, M., Eds.; Springer: Berlin/Heidelberg, Germany, 2005; pp. 163–186. [Google Scholar]
  94. Moradian, M.; Modanloo, V.; Aghaiee, S. Comparative analysis of multi criteria decision making techniques for material selection of brake booster valve body. J. Traffic Transp. Eng. 2018, 6, 526–534. [Google Scholar] [CrossRef]
  95. García, J.M.E. Entropía. Asignación de Pesos No Subjetivos, Para la Valuación (Entropy. Assignment of Non-Subjective Weights, for the Valuation). Tesis de Maestría; Instituto Tecnológico de la Construcción, ITC: Enschede, The Netherlands, 2002. [Google Scholar]
  96. Shannon, C.E.; Weaver, W. The Mathematical Theory of Communication; The University of Illinois Press: Champaign, IL, USA, 1964. [Google Scholar]
  97. Zhu, Y.; Tian, D.; Yan, F. Effectiveness of entropy weight method in decision-making. Math. Probl. Eng. 2020, 2020, 3564835. [Google Scholar] [CrossRef]
  98. Opricovic, S.; Tzeng, G.-H. Compromise solution by MCDM methods: A comparative analysis of VIKOR and TOPSIS. Eur. J. Oper. Res. 2004, 156, 445–455. [Google Scholar] [CrossRef]
  99. Kaya, I.; Çolak, M.; Terzi, F. A comprehensive review of fuzzy multi criteria decision making methodologies for energy policy making. Energy Strat. Rev. 2019, 24, 207–228. [Google Scholar] [CrossRef]
  100. Kusumawardani, R.P.; Agintiara, M. Application of fuzzy AHP-TOPSIS method for decision making in human resource manager selection process. Procedia Comput. Sci. 2015, 72, 638–646. [Google Scholar] [CrossRef]
  101. Andersson, K.P.; Ostrom, E. Analyzing decentralized resource regimes from a polycentric perspective. Policy Sci. 2008, 41, 71–93. [Google Scholar] [CrossRef]
  102. Saldaña, R.P.; Bonrostro, J.P. Los métodos de decisión multicriterio discretos. Un punto de vista racional aplicado a la toma de decisiones (Discrete multicriteria decision methods. A rational point of view applied to decision making). Anáhuac J. 2016, 16, 47–78. [Google Scholar]
  103. Allison, J.S.; Santana, L.; Visagie, I.J.H. A primer on simple measures of association taught at undergraduate level. Teach. Stat. 2022, 44, 96–103. [Google Scholar] [CrossRef]
  104. Gibbons, J.D.; Chakraborti, S. Nonparametric Statistical Inference, 4th ed.; The University of Alabama Tuscaloosa, Marcel Dekker, Inc.: Tuscaloosa, AL, USA, 2003. [Google Scholar]
  105. Zavadskas, E.K.; Antuchevičienė, J.; Kapliński, O. Multi-criteria decision making in civil engineering: Part I—A state-of-the-art survey. Eng. Struct. Technol. 2015, 7, 103–113. [Google Scholar] [CrossRef]
  106. Heldt, S.; Rodríguez de Francisco, J.C.; Dombrowsky, I.; Feld, C.K.; Karthe, D. Is the EU WFD suitable to support IWRM planning in non-European countries? Lessons learnt from the introduction of IWRM and River Basin Management in Mongolia. Environ. Sci. Policy 2017, 75, 28–37. [Google Scholar] [CrossRef]
  107. Howard, J.; Merrifield, M. Mapping groundwater dependent ecosystems in California. PLoS ONE 2010, 5, e11249. [Google Scholar] [CrossRef] [PubMed]
  108. Foster, S.; Koundouri, P.; Tuinhof, A.; Kemper, K.; Nanni, M.; Garduno, H. Groundwater Dependent Ecosystems: The Challenge of Balanced Assessment and Adequate Conservation; The World Bank: Washington, DC, USA, 2006. [Google Scholar]
  109. Meglen, R.R. Examining large databases: A chemometric approach using principal component analysis. Mar. Chem. 1992, 39, 217–237. [Google Scholar] [CrossRef]
  110. Vásquez-Hurtado, J.H. Study of variables that influence to achieve maximum throughput on a path of a multi-hop multi-channel wireless system. Acta Nova 2012, 5, 445–461. [Google Scholar]
  111. Marín-Diazaraque, J.M. Analisis Factorial. El procedimiento Analisis Factorial. Capítulo 20. Material docente (Factorial Analysis. The Factor Analysis Procedure. Chapter 20. Teaching Material); Universidad Carlos III de Madrid: Madrid, Spain, 2014; pp. 419–459. Available online: (accessed on 25 June 2022).
  112. Wilson, N.J.; Harris, L.M.; Nelson, J.; Shah, S.H. Re-theorizing politics in water governance. Water 2019, 11, 1470. [Google Scholar] [CrossRef]
  113. Bunch, M.J.; Parkes, M.; Zubrycki, K.; Venema, H.; Hallstrom, L.; Neudorffer, C.; Berbés-Blázquez, M.; Morrison, K. Watershed management and public health: An exploration of the intersection of two fields as reported in the literature from 2000 to 2010. Environ. Manag. 2014, 54, 240–254. [Google Scholar] [CrossRef]
Figure 1. Historical development of events related to water–environmental management [21,22]. WWF: World Water Forum. Source: own elaboration.
Figure 1. Historical development of events related to water–environmental management [21,22]. WWF: World Water Forum. Source: own elaboration.
Water 15 02991 g001
Figure 2. Diagram of the common, relevant, and intrinsic aspects of each approach (IWRM, EA, Ecohealth, WFD, WGP, SW). Source: own elaboration.
Figure 2. Diagram of the common, relevant, and intrinsic aspects of each approach (IWRM, EA, Ecohealth, WFD, WGP, SW). Source: own elaboration.
Water 15 02991 g002
Figure 3. AHP structure for choosing the best management theoretical framework. Source: own elaboration.
Figure 3. AHP structure for choosing the best management theoretical framework. Source: own elaboration.
Water 15 02991 g003
Figure 4. (a) Importance of the criteria; (bf) evaluation of each management alternative; (g) best evaluated alternative; (h) best standardized management alternative. Source: own elaboration.
Figure 4. (a) Importance of the criteria; (bf) evaluation of each management alternative; (g) best evaluated alternative; (h) best standardized management alternative. Source: own elaboration.
Water 15 02991 g004
Figure 5. (a) Comparison of multi-criteria methods for the six management alternatives and correlations between (b) AHP-TOPSIS and (c) AHP-PROMETHEE; dot means: evaluation of alternatives. Source: own elaboration.
Figure 5. (a) Comparison of multi-criteria methods for the six management alternatives and correlations between (b) AHP-TOPSIS and (c) AHP-PROMETHEE; dot means: evaluation of alternatives. Source: own elaboration.
Water 15 02991 g005
Figure 6. Simulation projected for (a) 5, (b) 10, (c) 15, (d) 20, and (e) 25 management criteria in TOPSIS. Source: own elaboration.
Figure 6. Simulation projected for (a) 5, (b) 10, (c) 15, (d) 20, and (e) 25 management criteria in TOPSIS. Source: own elaboration.
Water 15 02991 g006
Figure 7. Count and importance of each criterion by management approach: (a) governance, with more than 100 mentions in the WGP; (b) access to water nearby, with 20 mentions in IWRM; (c) quantity and quality, with more than 200 mentions in the WFD; (d) groundwater, not mentioned in Ecohealth, the EA, or the WGP; and (e) agriculture, with minor mentions in the six management theoretical frameworks. Source: own elaboration.
Figure 7. Count and importance of each criterion by management approach: (a) governance, with more than 100 mentions in the WGP; (b) access to water nearby, with 20 mentions in IWRM; (c) quantity and quality, with more than 200 mentions in the WFD; (d) groundwater, not mentioned in Ecohealth, the EA, or the WGP; and (e) agriculture, with minor mentions in the six management theoretical frameworks. Source: own elaboration.
Water 15 02991 g007
Figure 8. Exploratory analysis of the criteria involved in water–environmental management frameworks; dot means ubication of criteria. Source: own elaboration.
Figure 8. Exploratory analysis of the criteria involved in water–environmental management frameworks; dot means ubication of criteria. Source: own elaboration.
Water 15 02991 g008
Table 1. Main references for each approach.
Table 1. Main references for each approach.
Framework Source Author(s)
IWRMThe Dublin statement and report from the conference.International Conference on Water and the Environment [23]
EcohealthEcohealth, an ecosystem approach.[24]
EAConference of the parties under the Convention on Biological DiversityConvention on Biological Diversity [25]
WFDDirective 2000/60/EC of the European Parliament and of the Council establishing a community framework for action in the field of water policy.[26]
WGPToward integrated governance for water, health, and social-ecological systems: the Watershed Governance Prism.Parkes et al. [21]
SWAssessing the sustainability of water governance systems: the Sustainability Wheel.Schneider et al. [22]
Table 2. Comparison Matrix A: Defining the relative importance of the criteria ( x i j ).
Table 2. Comparison Matrix A: Defining the relative importance of the criteria ( x i j ).
CriteriaGovernanceParticipationSustainabilityDecentralizationHealth and Well-Being
Health and well-being1/31/511/21
Table 3. Normalized Matrix A’ and the relative weight of each criterion (w).
Table 3. Normalized Matrix A’ and the relative weight of each criterion (w).
CriteriaGovernanceParticipationSustainabilityDecentralizationHealth and Well-BeingΣH=ww%
Health and well-being0.
Table 4. Summary of the evaluations of each comparison matrix.
Table 4. Summary of the evaluations of each comparison matrix.
No.ComparisonnλmaxCIRCIIR < 10%
ICriterion by criterion55.
IICriterion 1 for the 6 alternatives66.450.091.247.2%
IIICriterion 2 for the 6 alternatives66.890.181.248.7%
IVCriterion 3 for the 6 alternatives66.550.111.248.8%
VCriterion 4 for the 6 alternatives66.370.071.245.9%
VICriterion 5 for the 6 alternatives66.520.101.248.4%
Table 5. Frequency of words found in the six sources consulted (Matrix B).
Table 5. Frequency of words found in the six sources consulted (Matrix B).
Approach\CriteriaGovernanceParticipationSustainabilityDecentralizationHealth and Well-Being
Total sum16611424811346
Notes: (1) The word government was not included in the documents, as it has connotations that are different from those of governance. With respect to the terms governance and governance of water, although they are concepts that differ, they were accounted for. In the cells with 0*, a value of 1 was assigned to the entropy method. (2) In the documents consulted, words with similar derivatives, such as participate and participation; sustainable and sustainability; decentralized, decentralization, and decentralize; and health, welfare, and well-being, were considered. (3) The word count essentially comprised the main content, excluding the index, abstract, prologue, references, and names of institutions.
Table 6. Normalized Matrix B using the sum (linear) technique.
Table 6. Normalized Matrix B using the sum (linear) technique.
Approach\CriteriaGovernanceParticipationSustainabilityDecentralizationHealth and Well-Being
Table 7. Summary of the entropy components: level (E), variation (D), and weight (w).
Table 7. Summary of the entropy components: level (E), variation (D), and weight (w).
ComponentGovernanceParticipationSustainabilityDecentralizationHealth and Well-Being
E j   =0.500.840.940.790.59
D j   =0.500.
w j   =0.370.
Table 8. Normalized Matrix B using the vector technique.
Table 8. Normalized Matrix B using the vector technique.
Approach\CriteriaGovernanceParticipationSustainabilityDecentralizationHealth and Well-Being
Table 9. Calculation of the ideal best and ideal worst values.
Table 9. Calculation of the ideal best and ideal worst values.
Approach\CriteriaGovernanceParticipationSustainabilityDecentralizationHealth and Well-Being
Table 10. Summary of the PIS and NIS of each criterion.
Table 10. Summary of the PIS and NIS of each criterion.
Ideal SolutionGovernanceParticipationSustainabilityDecentralizationHealth and Well-Being
A + 0.340.
Table 11. Summary of Euclidean distances, relative proximities, and rankings.
Table 11. Summary of Euclidean distances, relative proximities, and rankings.
Approach d i + d i RiRank
Table 12. Normalized Matrix B using the max–min (linear) technique.
Table 12. Normalized Matrix B using the max–min (linear) technique.
Approach\CriteriaGovernanceParticipationSustainabilityDecentralizationHealth and Well-Being
1 (1)0.000.380.710.170.12
1 = IWRM, 2 = Ecohealth, 3 = EA, 4 = WFD, 5 = WGP and 6 = SW.
Table 13. Deviation (dj) by pairwise comparison from Table 12.
Table 13. Deviation (dj) by pairwise comparison from Table 12.
GovernanceParticipationSustainabilityDecentralizationHealth and Well-Being
Table 14. Aggregate preference index (P) multi-criteria.
Table 14. Aggregate preference index (P) multi-criteria.
GovernanceParticipationSustainabilityDecentralizationHealth and Well-BeingΣH=
Table 15. Aggregate preference function matrix.
Table 15. Aggregate preference function matrix.
IWRMEcohealthEAWFDWGPSW x ϵ A π a , x =
x ϵ A π x , a =1.220.921.171.700.381.08
Table 16. The net flow of relevance (net outranking flow).
Table 16. The net flow of relevance (net outranking flow).
Approach Positive   + a Negative   a Net   Value   a Ranking
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Armas Vargas, F.; Nava, L.F.; Gómez Reyes, E.; Olea-Olea, S.; Rojas Serna, C.; Sandoval Solís, S.; Meza-Rodríguez, D. Water and Environmental Resources: A Multi-Criteria Assessment of Management Approaches. Water 2023, 15, 2991.

AMA Style

Armas Vargas F, Nava LF, Gómez Reyes E, Olea-Olea S, Rojas Serna C, Sandoval Solís S, Meza-Rodríguez D. Water and Environmental Resources: A Multi-Criteria Assessment of Management Approaches. Water. 2023; 15(16):2991.

Chicago/Turabian Style

Armas Vargas, Felipe, Luzma Fabiola Nava, Eugenio Gómez Reyes, Selene Olea-Olea, Claudia Rojas Serna, Samuel Sandoval Solís, and Demetrio Meza-Rodríguez. 2023. "Water and Environmental Resources: A Multi-Criteria Assessment of Management Approaches" Water 15, no. 16: 2991.

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop