Explore, Synthesize, and Repeat: Unraveling Complex Water Management Issues through the Stakeholder Engagement Wheel

Abstract

Effective stakeholder engagement is fundamental to water management, yet there are as many approaches to consultation as there are efforts. This paper provides an evaluation of, and lessons learned from three water management engagement processes, and uses this assessment to offer a framework for stakeholder engagement. The Stakeholder Engagement Wheel framework is centered on a bridging organization that ensures that the process continues to move forward, and a steering committee that guides and changes activities according to stakeholder interests and concerns. Around the Stakeholder Engagement Wheel are four steps designed to examine iteratively the water management issue driving the engagement process and expand the sphere of interests involved. Many engagement processes have limited effectiveness because of: (1) paucity of time; (2) complexity of water resources management; (3) difficulty of engaging diverse stakeholders; and (4) lack of methods for engagement that are centered on empowerment, equity, trust, and learning. In this study, we have encountered all four of these issues and have addressed all but the first through a deliberate, iterative, and flexible approach. By cycling through activities and actions as proposed in the Stakeholder Engagement Wheel, we can build a community of practitioners with the nuanced and shared understanding needed for cohesive action and robust decisions in the face our considerable challenges.

1. Introduction

Effective stakeholder engagement is fundamental to water management [1,2], yet there are as many approaches to consultation as there are efforts. These efforts are rarely examined for their effectiveness [3], and even less frequently analyzed to identify overarching elements of success. While general discussions of frameworks to support this engagement are limited [2,4,5], strategies are particularly lacking for engagement with stakeholders who have expertise in the subject based on their professional or personal interest. These stakeholders differ from the general public because engagement with them is ideally ongoing and often used to improve understanding of a critical issue through social learning. In contrast, engagement with the general public is not necessarily intended to be iterative, and typically is used to vet new policies or gain social license to operate [6].

Stakeholder engagement to promote social learning is distinct from engagement for policy making or obtaining social license because the overarching goal in social learning is understanding complex issues, and not necessarily resolution of conflict [6]. While engagement as a learning process may not solve the conflict per se, it is considered very important to overcome persistent norms and “difficult-to-change socio-technical systems” [7] (p. 398). Overcoming these systems and norms through a social-learning process can lead to improved water management outcomes [8].

While the methods and the definitions of social learning are debated [9], there is evidence that social learning, particularly in the adaptive natural resource management context, is enhanced by bridging organizations [10]. Bridging organizations are neutral third parties that serve to bring diverse groups together and include a “focus on trust building, sense making, learning, vertical and horizontal collaboration and conflict resolution” that is distinct from the organizations they connect [11]. Bridging organizations can also help balance power asymmetries in discussions [11], which is particularly important in engagement for water management [12]. They also play an important role in promoting social learning and provide opportunities for diverse perspectives to retain their identities while also forming productive relationships with others [13,14,15].

The success of an engagement effort can be related to who convenes the process, e.g., [16], but the approach to engagement is also important. Typically, stakeholder engagement has proceeded with a “tool-kit” approach, where engagement proceeds based on the selection of the relevant tools for the job [17]. According to Reed (2008) [17], the “tool-kit” approach is inadequate because it lacks an underpinning philosophy of how, why, and when to engage with stakeholders. Increasingly, the focus is shifting to the importance of engagement as a process that is centered on empowerment, equity, trust, and learning [17,18,19]. Antunes, et al. (2009) [2] suggest that instead of choosing a tool from the toolbox, the emphasis should be on using multiple tools within a framework.

Existing examples of process design for engagement generally follow a cycle that includes planning, identifying stakeholders, engaging with stakeholders, evaluation, and implementation, e.g., [20,21]. The existing frameworks, however, are focused on engagement with the general public to create or vet policy or to gain social license to operate. There remains a paucity of information on what processes (1) encourage social learning [22] and (2) are best for engagement with stakeholders with expertise on problem that is the focus of the engagement [3]. Notably, Mathur, et al. (2008) [6] states that “[e]xisting practices view stakeholder engagement: mostly from a management perspective; sometimes from an ethical perspective; less often as a combination of the two; and rarely have any element of the social learning perspective. There is a need for an approach that combines all the three perspectives if sustainability is to be pursued.” Stakeholders with expertise can include lay stakeholders, who are people that have a deep interest in the subject, and professional stakeholders, who are paid representatives of organized interests [23]. While there are numerous recent handbooks for engagement with the general public to improve natural resources management or encourage corporate responsibility, e.g., [20,24,25], we did not discover any frameworks tailored to designing a social-learning process to engage with stakeholders who have subject expertise.

In this paper we use experiences from three engagement processes to build a framework for effective stakeholder engagement, where the goal was to unravel complex water management problems through promoting social learning. The engagement projects occurred at three geographic scales within the U.S. state of Arizona. Key to this framework is the role of a bridging organization as the convener and promoter of a process and a steering committee, who together explore and then synthesize disparate data and perspectives to increase understanding of water resource management issues.

2. Study Areas and Key Project Aspects

The University of Arizona Water Resources Research Center (WRRC) convened three engagement processes that occurred at three different geographic and governance scales (local, regional, and state) (Figure 1). The geographic scales included the U.S. state of Arizona, the Upper Gila River Watershed, and the Town of Clarkdale. The general approach for each engagement process was to establish a project problem and goal, create a steering committee, conduct engagement activities, and determine next steps or action items. A summary of activities and issues addressed in each activity for the three projects is included in

Table 1. Engagement and outreach activities.

Project	Activity Name	Number	No. of People	Key Issues Addressed
		Scope
		Duration
Roadmap	Presentations on Water for Natural Areas	38	800+	What do we know about water for natural areas? How can the WRRC database on water needs of riparian and aquatic ecosystems be used for management and planning?
		11 counties
		1 hour
	Phase II Kick-off Meeting	2	52	What are the goals of this project? What do we hope to accomplish?
		Regional
		3 hours
	Survey	1	171	What makes a community decide to consider water natural areas? And What makes water for natural areas vulnerable?
		Statewide
		NA
	Benefits and Challenges Workshop	1	39	What are the benefits, challenges, and reasons for including natural areas in water management and planning?
		Statewide
		½ day
	Focus Groups	43	247	How should we consider water for natural areas in water management and planning and what incentives would make it more attractive to do so?
		10 Counties
		1 hour
	Action Items Workshop	4	79	What strategies and actions should be taken to improve education, facilitate collaboration, set priorities, create incentives and encourage water conservation to include natural areas in water management and planning?
		Regional
		½ day
Gila	Science Coordination Workshop	1	40	What data do you have and what data do you need? How can we better coordinate data collection and use in the watershed?
		Watershed
		1.5 day
	Shared History Timeline	1	43	What is the history of the watershed? How can we learn from that history as we move forward?
		Watershed
		3 hours
	Key Informant Interviews	8	10	What do you think the watershed will look like in the future? What were the biggest surprises from the past?
		Watershed
		1 hour
	Scenario Drivers Workshop	1	16	What are the key drivers of change in the watershed? What drivers of change are most uncertain?
		Watershed
		1 day
	Survey	1	27	What scenarios would be most useful for your management concerns? What questions do you have about the scenarios?
		Watershed
		NA
	Scenario Narrative Interviews	8	8	Questions about the future of the Gila Watershed based on scenario frameworks determined through the survey.
		Watershed
		1 hour
Clarkdale	Water 101 & 102 Public Meetings	2	52	Information sessions on current water use and management in Clarkdale and regional groundwater modeling results. Attendees were “lay stakeholders”.
		Regional
		2 hour
	Open House	1	25	What concerns do you have about water in Clarkdale? What do you think about different management options that the Town is considering?
		Town
		2 hour
	Water Experts Workshop	1	27	What should Clarkdale include or not include in its Water Resources Management Program (WRMP) and why? What management strategies will have the greatest impact?
		Statewide
		1 day
	Water Expert Interviews	8	8	What should Clarkdale include or not include in its Water Resources Management Program (WRMP) and why? What management strategies will have the greatest impact?
		Multi-State
		1 hour
	Small Town Water Forum	1	29	What should Clarkdale include in its WRMP? What challenges do small towns face in managing their water? How can towns in Arizona work together to improve water management?
		Statewide
		1.5 day

.

2.1. Statewide Geographic Scale: Arizona

Arizona is a semi-arid state in the southwestern United States (Figure 1). Annual precipitation varies across the state from a high of 914 mm in the mountainous central region of the state to a low of 76 mm in the Sonoran Desert in the southwestern portion of the state. Water demand in Arizona was estimated to be 8,634,372 million cubic meters or 7 million acre-feet in 2014. In 2014, the Colorado River met 40% of this demand, 17% was met by other instate rivers, 40% by groundwater, and the remaining 3% by reclaimed water use. Water extractions and diversions in Arizona are dominated by agriculture (73%), followed by municipal (21%) and industrial (6%) uses [26].

Governance, defined as the overarching framework laws, regulations, and customs, which includes mechanisms of stakeholder engagement [27], varies across the state. For example, the Active Management Areas (AMAs) are within the central and most populous portion of Arizona. Within the AMAs groundwater pumping is tightly regulated. In general, beyond the AMAs only a notice of intent to drill is required and groundwater pumping is unregulated. There are four types of water recognized by Arizona water law: surface water (streams and springs), groundwater, Colorado River water, and effluent [28]. Particularly notable are the management difficulties created by varied restrictions on groundwater pumping throughout the state and by limited legal recognition of the physical connection between surface water and groundwater [29].

The project at a statewide scale, Roadmap for Connecting the Environment to Arizona Water Planning (Roadmap), took place over four years (January 2010–December 2014) and actively engaged over 1000 people across all Arizona counties. The goals of this project were to (1) synthesize available information about environmental water needs and identify knowledge gaps and (2) identify voluntary, stakeholder-driven options for addressing water needs of natural areas in the context of limited water supplies and existing water rights. These two goals were met in three phases. The first phase (January 2010–January 2011) focused on compiling and analyzing 121 studies on environmental flow needs for riparian and aquatic species in Arizona. The Arizona Environmental Water Needs Assessment and Methodology Guidebook summarized the results of this analysis. In addition, a database was created as a tool for land and water managers to identify available information on riparian and aquatic species flow needs [30]. The second phase of the project (January 2011–January 2012) focused on disseminating results from the environmental water needs assessment and preparing for the engagement process in phase three. Phase three (January 2012–December 2014) built the Roadmap for Considering Water for Arizona’s Natural Areas. Key aspects of this last phase included formation of a 12 person steering committee that met every six weeks for the final two years of the project and engagement activities, including focus groups and workshops.

2.2. Watershed Scale: Upper Gila River Watershed

The Upper Gila River Watershed has an area of approximately 39,350 square kilometers that straddle the Arizona-New Mexico boundary. Forty-eight percent of the watershed is in Arizona, and the remaining 52% is in New Mexico. The engagement process described in this paper focused on the Arizona portion of the watershed. The Upper Gila River Watershed is rural. Total population of the Arizona portion is approximately 68,000. Approximately 70% of this population lives within five miles of the Gila River or its main tributary the San Francisco River. The principal industries are farming, ranching, and mining. Ninety-one percent of water use in the Arizona portion of the watershed is for agricultural purposes. Significant portions of the watershed are owned by the federal government and the San Carlos Apache Tribe. As a sovereign nation, the San Carlos Apache Tribe regulates and manages their water supply separate from any state laws [31]. The watershed is not within an AMA; however, the Arizona Water Rights Settlement Act of 2004 and the Globe Equity Decree of 1935 limit use of groundwater and surface water, respectively, near the Gila River, including portions that run through the San Carlos Apache lands [32]. Efforts to improve watershed health are led not by state or federal organizations, but by a watershed group (Gila Watershed Partnership, GWP), which does not have regulatory authority.

The Upper Gila Watershed project, Participatory Watershed Assessment for the Upper Gila (Gila), took place over two years (October 2012–September 2014) and actively engaged approximately 150 people. The goal of this project was to understand natural and social conditions in order to: (1) provide reliable, long-term water supplies for a resilient community; (2) preserve the rural, agricultural lifestyle; and (3) sustain and enhance the health of the Upper Gila Watershed in the face of uncertain physically and legally available water supplies. The Gila project included two phases, assessing baseline watershed conditions and creating stakeholder-driven scenarios for watershed planning. Both the baseline watershed assessment and scenario planning phases were designed to be stakeholder driven and were guided by a watershed planning working group. Project outputs included an Atlas of the Upper Gila River Watershed and Scenarios for the Future of the Upper Gila Watershed [33,34].

2.3. Town Scale: Town of Clarkdale

The Town of Clarkdale, with a population of 4087 in 2010, lies on high ground above the Verde River, at an elevation of 1085 meters. Several long-standing economic activities, such as farming and ranching, depend on sufficient Verde River flows. Activities tied to ecotourism have also become an ever-larger component of the area’s economy. The river itself, as one of the few rivers with dependable surface flows in this semi-arid region, has garnered increased attention for kayaking and fishing. Verde Valley river tourism events, combined with ecological, recreational, historical, and/or agricultural elements, are also rising in popularity [35]. The Town of Clarkdale is not within an AMA, however it has adopted ordinances to limit new development if water supplies are inadequate and to set standards for low-water use plants in outdoor landscaping.

The project, Recommendations for a Water Resources Management Program for the Town of Clarkdale, Arizona (Clarkdale), took place over two years (January 2013–December 2014), and included approximately 130 stakeholders. The goal of this project was to develop recommendations for a water resource management program that equitably meets the needs of residents, businesses, and Clarkdale’s natural environmental in order for it to be a robust and resilient community. These recommendations were created through research on existing water management approaches applicable to small towns, advice from a steering committee, and a series of workshops and meetings. The primary project output was the Recommendations for the Clarkdale Water Management Program publication [36].

3. The Stakeholder Engagement Wheel

Unraveling thorny water management issues requires us to move beyond the “tool-kit approach,” and to instead employ a highly iterative and collaborative process that not only increases understanding of issues but also builds personal connection and trust among stakeholders. It is accepted practice to gather and integrate iteratively water resource data to build a comprehensive understanding of physical hydrologic and water management systems. Engagement for water resource management should be no different. Cycling through engagement efforts as activities and actions that are designed and then implemented allows us to build a community of practitioners with the nuanced and shared-understanding needed for cohesive action. This community of practice can then fully comprehend data and shape perspectives to make robust decisions. Because trust and collective knowledge are key outcomes of social learning, we look to the social learning literature for guidance. To encourage social learning an engagement process should be (1) inclusive [37,38]; (2) interactive [36,37,39]; and (3) flexible [40,41]. The Stakeholder Engagement Wheel (Figure 2) shows how our understanding of water management issues can be an aggregate of engaged learning opportunities with each step offering the prospect to encourage diverse and interactive participation.

The Stakeholder Engagement Wheel moves both in a larger circle as well as in smaller, iterative circles between steps. These smaller circles are a key aspect of the Wheel because within them information is collected, synthesized, and then used to help solve water management problems. Additional stakeholders, ideas, and understanding gained or changed through each iteration are shown by the dashed lines around the numbered steps. To summarize at the outset, at the center of the Wheel are a convener, which ensures that the process continues to move forward, and a steering committee, which guides and changes the process according to stakeholder interests and concerns. To promote inclusiveness and interactiveness, the problem is iteratively examined throughout steps 1–4 along the Wheel. Multiple iterations of problem-solving throughout the process ensure that the problem and goal are robust (between 1 and 2), that ample engagement to understand the problem and to reach the most diverse set of stakeholders possible occurs (2 and 3), and that actions are developed that meet stakeholder needs (3 and 4).

3.1. Getting Started—The Importance of the Convener and the Steering Committee

At the center of the Wheel are a bridging organization or another convener and the steering committee. The role of the bridging organization is to keep the Wheel turning through facilitating examination of the problem(s) and providing a neutral convening space that balances power asymmetries and ideas, which are well-noted functions of bridging organizations [11,14,15]. Because neutrality is key to building common ground among disparate ideologies [15] and expertise is important for facilitating the examination of the problem, the group that serves as a bridging organization should be carefully considered to ensure that it reflects these qualities. While this bridging organization role can be played by entities such as non-governmental organizations, university-affiliated organizations, such as Cooperative Extension and National Institutes for Water Resources in the United States, are uniquely suited for this task because not only do they not have water rights or a particular water interest to protect, they are also recognized by local communities as providing independent expertise [14].

Representing the bridging organization, we also found that we played a key role in moving each engagement process forward, which is a role beyond just bringing people together. For example, in the Clarkdale project the Town was originally designated as the project manager, and therefore, responsible for bringing the team together and designing engagement efforts. In initial team meetings, however, it was determined that the project was better served by the WRRC managing the effort because of our subject-matter expertise and the time constraints of Town staff.

Surrounding the convener on the Wheel is the steering committee. The role of the steering committee is to guide and change the process according to stakeholder interests and concerns, thereby promoting flexibility and responsiveness. For this reason, steering committees are important to an engagement process that seeks to reach the top rungs of Arnstein’s ladder [42], or as redefined by Luyet, et al. [43], create a process that empowers stakeholders through “delegation of decision-making over project development and implementation.” Although steering committees are mentioned in case studies and practitioner manuals on stakeholder engagement, e.g., [1,20,26,44], they are infrequently noted as key to successful outcomes. An exception is Slocum [45], who states that steering committees are key components of the process and suggests that a diverse and strong steering committee helps both to establish and complete the engagement process.

The steering committee is the linchpin to taking the process beyond a “tool-box” approach because they promote diverse participation, in-depth examination of the water management questions, and ensure the process is driven by the people who will be impacted by its outcomes. When acting as a microcosm for the spectrum of interests, the steering committee serves as an initial platform for exploration of the problem, goal setting, and identifying activities to broaden engagement and people to engage. For the steering committee act as a small-scale version of the broader array of interests, members should be selected based on their representativeness of the issues central to the engagement process and their willingness to sit at the table with people whose opinions may differ from their own [15]. Based on experiences with the three steering committees associated with the projects described here, we found that it is important consistently engage with this core group of stakeholders. The steering committee should be people who are either local to the engagement effort or are significantly vested in the project’s outcomes.

Through each of the three projects, we found that the time we invested in forming and maintaining engagement with our steering committees aided our ability to reach out to additional stakeholders and develop successful engagement activities. During the Roadmap project, we made the most concerted effort to engage consistently with the steering committee. This interaction enabled us to connect agricultural and industrial water interests to the process who were initially not sufficiently represented. The Roadmap focus groups also provide evidence of the importance of a steering committee. Steering committee members recommended focus group participants through a stakeholder mapping exercise. This exercise identified 47 key organizations that should be included in the project because they would either support, be neutral to, or have reservations about allocating water to the environment. Of the 47 groups, 34% were identified by steering committee members as supporting environmental flows, 30% as neutral, and 36% as having reservations about allocating water for environmental flows. Over the course of only four months, we were able to conduct focus groups with 87% of the groups. Most importantly, 75% of the organizations identified as having reservations about water for the environment participated. The ability to have a relatively high success rate of engaging with potentially adversarial interests can be attributed to the advice and assistance we received from steering committee members in arranging focus group meetings.

3.2. Setting the Wheel in Motion—Creating a Problem Statement and Goal (Step 1)

Having created a steering committee, the Wheel can begin to move through refining a problem statement and goal. Establishing a clear goal has been noted by others as a key element of effective engagement, e.g., [23,46,47]. Without a clear problem to address and goals for the process, an otherwise effective approach will fall short [23]. Establishing a problem statement and goal are Step 1 and sit atop the Wheel because our experience confirmed the importance of goal setting. In the Gila and Clarkdale project the goals were established through workshops (Gila) and team meetings (Clarkdale). In the Roadmap project the goal was established by the objective of the grant. During the Clarkdale and Gila projects, the steering committees frequently referred to the goal when there were questions about which stakeholders to include or activities to conduct. The Gila project’s desire to “preserve the rural, agricultural lifestyle” in the Gila project led to engagement activities targeted to farmers and ranchers. The need to “equitably meet the needs of residents, businesses, and Clarkdale’s natural environment” as stated in the project goal, resulted in a Water 101 series where the Town could explain local hydrologic conditions and receive feedback from residents about their water concerns. Similarly, the WRRC used the Roadmap project goal of “identifying voluntary, stakeholder driven, options…” to both shape workshop discussions and encourage participation of water interests that may be concerned with additional regulation and laws.

3.3. Maintaining the Momentum—Engagement Activities (Step 2)

Once the initial problem and goal(s) are established, the engagement process proceeds to Step 2 —engagement activities designed to improve understanding of the problem and stakeholder perspectives. Multiple iterations of exploring the problem through an engagement activity, synthesizing what was learned from the engagement, and then designing the next engagement activity based on this understanding help ensure the diverse participation, interactiveness, and process flexibility necessary for social learning. Based on the experience of others in stakeholder engagement, e.g., [18,19,20] and our own experience, we recommend that engagement rely on input from the steering committees, the goal(s) of the project, and integration of what has been learned from previous project activities.

Synthesizing results are particularly important in Step 2 because it enables the bridging organization and steering committee to adapt their approach to ensure iterative engagement with a diverse set of stakeholders. In synthesizing results of each engagement activity, and of the project as a whole, it is important to consider how information is displayed and how stakeholders will access the information. Effective communication about results from each activity facilitates steering committee decisions about next steps in the engagement process and can empower stakeholders to use the information assembled. The display of information was particularly important in the Gila and Roadmap projects. In the Gila project, graphics of the scenarios and the key drivers of change helped participants quickly understand the findings and assisted the project team in the dissemination of results to local councilmembers and county supervisors. For Roadmap project, graphical display of the qualitative analysis findings from focus group results led to the development of key questions for the workshops to develop action items (see [15]).

Examples of the evolving engagement approach as a result of exploring the problem and then integrating this understanding can be seen in all three projects. The original Clarkdale plan included three expert workshops focused on one of three building blocks for a water management plan: demand-based, supply-based, and augmentation-based options and recommendations. After our first meeting with the steering committee, however, it became clear that these categories contained too much potential overlap of subject matter and experts. Therefore, we decided to conduct one expert meeting and a series of follow-up phone interviews during which we discussed all three categories at one time. Because of this change in approach, we were able to convene one meeting with experts instead of three and then build upon the knowledge gained from that first meeting through a very well-received small town forum on water management. In this case, adjusting the approach based on examining who attended the first engagement exercise, as well as summarizing the results of that exercise, led to a more robust set of recommendations for the Town.

The Gila project was originally designed with a series of scenario building workshops to create the future scenarios. Discussion with our steering committee at the beginning of the scenario planning process made it clear that multiple, daylong workshops over the course of four months would be difficult for them to manage with their workloads. As a result, we decided to substitute the longer workshops with monthly steering committee meetings that were open to anyone interested. In this case, shorter meetings at regularly scheduled times interspersed with initial and final half-day day workshops made it possible for regular attendance necessary to build the scenarios. During the Roadmap project, the engagement approach was shifted from an initial set of workshops throughout the state that combined multiple water interests to focus groups with individual interest groups. This change in activities was suggested by the steering committee as a way to bring water interests into the discussion that otherwise might be reluctant to participate in a workshop. As previously discussed, as a result of this change we were able to reach 87% of the stakeholder interests identified as key to meeting the goals of engagement.

3.4. Getting Traction—Developing and Implementing Action Items (Steps 3 and 4)

Step 3 on the Wheel is to develop action items or next steps. While exploring and synthesizing a problem may help to better understand it and increase the diversity of participants, at some point it becomes necessary to develop concrete actions to address the problems identified in Steps 1 and 2. Similar to the makeup of the steering committee and engagement activities, the approach to developing action depends on the project’s goals. For example, for the Clarkdale effort action items were developed through a synthesis of our research and the expert input gained throughout the project. These action items were then presented to the Town Council for adoption. The Gila project identified next steps through a synthesis of potential future conditions in the watershed based on a watershed baseline assessment and four future scenarios. Four regional workshops were used during the Roadmap project to develop action items. Using workshops as the mechanism to develop action items enabled all interested participants, not just the steering committee, to directly impact final recommendations. Roadmap recommendations (action items) were then distributed via presentations, webinars, and the report, Roadmap for Considering Water for Arizona’s Natural Areas.

Having established action items or next steps, the engagement process proceeds to Step 4—implementation of selected action items. Of the seven action items identified as part of the Clarkdale project, three have been implemented, and the Town is actively seeking funding for the others. The recommendations report itself was accepted by the Town Council, but not adopted as funding is needed to proceed on each specific recommendation (Personal Communication, Jodie Filardo, Town of Clarkdale). Using the scenarios developed to create coordinated water and land management plans across the watershed’s jurisdictions was the only action item from the Gila effort. This plan is proceeding through funding to the WRRC from the U.S. Bureau of Reclamation and has already resulted in water demand and supply estimates and identification of cross-jurisdictional opportunities for water management. Separate from the current WRRC convened process in the watershed, all of the local jurisdictions in one county have come together to discuss a standardized outdoor landscaping ordinance that could be adopted by each individual jurisdiction. Measuring implementation of the 10 action items from the Roadmap project is more difficult; however, we are aware of at least two efforts by environmental non-governmental organizations to implement action items. These early indications of implementation are encouraging, and although the projects described here have not been systematically examined for successful implementation, there are many examples of how failed engagement up to the point of implementation hampers successful implementation, e.g., [44,48,49] and some examples of how effective engagement can enable successful implementation, e.g., [18,50,51].

3.5. Coming Full Circle—Evaluation of Effectiveness

Evidence of social-learning as a result of engagement can take years to manifest, making a near-term evaluation of engagement processes difficult [11]. Near-term evaluation is, however, an important [42] and oft neglected [3] element of the study of engagement. Because the ultimate goal of this framework is to improve social learning outcomes, we evaluated the effectiveness of the Stakeholder Engagement Wheel approach against engagement elements that encourage social learning, i.e., inclusiveness, interactiveness, and flexibility of approach [19,20,23,37]. For the projects that inspired the Stakeholder Engagement Wheel, inclusiveness was examined through documenting the diversity of water interests and calculating an index of stakeholder engagement. Interactiveness was evaluated through the variety of engagement techniques used and the participation level of the stakeholders over time. The flexibility of the engagement process was determined by comparing the original project plans with the actual project activities that were implemented.

The following steps enabled evaluation of the extent to which the process was inclusive of a diversity of interests: (1) key stakeholders were determined by stakeholder mapping (Roadmap and Clarkdale) or key informant interviews (Gila); (2) stakeholder type was documented at each activity; and (3) an involvement index was calculated based on the approach used by Walter, et al. 2007. To determine the involvement index, each interactive activity was assigned points based on its length, with twice the number of points given to interactive activities. For example, a two-hour workshop would receive four points and a one-hour presentation would receive one point. The points were then summed and divided by the maximum number of possible points for each project to calculate the index. An involvement index score of 1.0 would indicate that every participant in that water interest was involved in all project activities. The average for the projects includes steering committee participation in regular project activities, but the hours they contributed for steering committee activities were not used to calculate the average. All projects engaged at least once with each of the key stakeholder groups identified through stakeholder mapping or key informant interviews. Figure 3 shows the total number of participants and average of the involvement index for each water interest. It is notable that the Roadmap project, which had the most concerted effort to engage a diverse set of interests and the most engaged steering committee, the average index of involvement is similar across different water interests.

The following steps enabled evaluation of the extent to which the process was interactive: (1) repeat and increasing engagement was measured using the same data gathered for inclusiveness on participant involvement and (2) the variety of engagement techniques was described qualitatively. Figure 4 indicates the growing reach of each of the projects and repeat engagement. We acknowledge that there could be some undercounting of repeat engagement and over counting of total participants because of anonymous participants in the Gila and Roadmap projects. Throughout the Roadmap and Gila projects, the number of people involved as well as those who participated more than once increased as the project progressed. Because the goal of the Clarkdale project was to gather as many different ideas a possible to create recommendations for the town’s water resources management program, there were no attempts to engage the same stakeholders or interests more than once. As shown in Table 1, all projects included multiple engagement techniques recognized to encourage two-way interaction or engagement in order to improve opportunities for interactiveness. The variety of techniques also provided different types of opportunities for engagement. For both the Roadmap and the Gila projects used a survey. These surveys were able to reach more people for input than could have been engaged through meetings alone. Although this engagement is not equivalent to attending a workshop, it does provide a good opportunity to broaden the scope of those involved. The shortcoming of this method, however, is the need to allow for anonymous responses in order to increase participation, which makes it difficult to track individual participants.

Finally, the flexibility of the engagement process was determined by comparing the original project plans with the actual project activities that were implemented. Process changes were made as a result of findings from formal and informal evaluations. For all projects, input from post-workshop surveys and steering committee advice changed the engagement approach. For example, suggestions from the evaluation after the science coordination workshop for the Gila project included concerns about the length of the meeting and the need for a better-defined purpose of the meeting. Because of these comments, subsequent workshops and meetings were shortened from 1.5 days to a single day, and the WRRC team worked to ensure that the purpose and outputs from each meeting were clear. Also during the workshop, participants stressed the importance of historical information on the watershed and expressed concern over the lack of data. As a result of this concern, the WRRC decided to add an activity not originally planned, and hosted shared history timeline exercise to gather historical perspectives on watershed management.

While engagement through a bridging organization provides a flexibility that may not be available to a state or federal agency, long-term engagement can be hampered by the need for continued grant funding. It is common for grants to last for only one or two years, which may hinder the ability of steering committees or broader stakeholder groups to continue their work. Engaging stakeholders on a project-by-project basis impedes the process of relationship building and reduces the likelihood of developing truly collaborative processes [45]. Issues with multi-year funding also highlight the importance of finding ways for bridging organizations to help "turn the wheel", but not necessarily be solely responsible for the forward motion of the process. In addition to other methods suggested for determining the long-term success of an approach, e.g., [3,38], another metric could be the ability for social learning and the project activities to continue without the bridging organization playing a role.

4. Discussion and Final Remarks

Effective stakeholder engagement through the process-design framework proposed here recognizes that in order to unravel complex problems we must repeatedly explore and synthesize information. Many engagement processes, particularly in water resources management, fall short of this goal [13,42,52]. The reasons for these shortcomings include: a paucity of time [20,53]; the complexity of water resources management [13,42]; the difficulty of engaging diverse stakeholders, especially those who feel disenfranchised [49,54]; and the lack of advice for an engagement process centered on empowerment, equity, trust, and learning that takes engagement beyond the “tool-kit” approach [17,18,19]. The projects that inspired and followed the Stakeholder Engagement Wheel encountered all of these challenges. Through the process described here they were able to address all but the first, namely a lack of time, through a deliberate, iterative, and flexible approach. This framework is particularly important for the many university bridging organizations, such as Regional Integrated Science Assessment Centers funded by the National Oceanic and Atmospheric Administration because they are increasingly relied upon to provide both unbiased research and stakeholder perspectives on key issues like water management and climate change.

Our findings are similar to others who have discovered that the convener of the process plays an important role in the efficacy of engagement [17,18]. Unlike most other approaches, however, we found that the people engaged through a steering committee also contribute to the efficacy of an engagement process. Because there are not many examples of an increased efficacy of engagement due to the use of steering committees, future research could focus on their role in promoting successful engagement. This research should examine best practices for informing, engaging, and maintaining these core groups of stakeholders and evaluating similar processes that include or omit a steering committee.

Water resource management is a complex endeavor, and sound management requires effective stakeholder engagement. The framework presented here provides an example of how a bridging organization can structure and evaluate the process of engagement. While the importance of bridging organizations in promoting social learning, and flexible, iterative approaches are recognized, there are few recommendations for a flexible process. As illustrated through the three projects presented here, the proposed framework provides for inclusivity, interactiveness, and flexibility in engagement through guidance by a steering committee and iteratively examining the problem. While further assessment is necessary, we suggest this framework provides a tangible approach that could aid other efforts engaging to solve complex water management problems across local, state, and regional scales. By cycling through activities and actions as proposed in the Stakeholder Engagement Wheel, we can build a community of practitioners with the nuanced and shared-understanding needed for cohesive action and robust decisions in the face our considerable challenges.