Learning to Manage Common Resources: Stakeholders Playing a Serious Game See Increased Interdependence in Groundwater Basin Management

Abstract

This paper reports an empirical evaluation of a new serious game created to foster learning about collaborative management of common pool resources. Stakeholders (n = 41) involved in the implementation of California’s Sustainable Groundwater Management Act were recruited to play a new serious game designed to illustrate how alternative water management strategies, including pumping restrictions and simple trading schemes, affect supply. In the game, a group of six players set in a groundwater basin area enact the allocation, needs, and use of water in rounds representing annual seasons. Pre-post surveys found that the gameplay increased perceived interdependence among stakeholders, and optimism about the groundwater management process. Qualitative feedback suggested that participants gained new insights into the nature of common pool resources and the needs of other stakeholders. Serious games may be useful in fostering attitudes, such as interdependence needed for successful collaborative planning and governance.

1. Introduction

Improving collaboration among stakeholders has become an increasing priority, due to the increasing need to manage common pool resources (CPR) and expand the use of collaboration-based regulatory frameworks among policymakers. To contribute to this aim, this paper reports an evaluation of a serious game developed to educate stakeholders involved in groundwater management in California. In contrast to most other serious game evaluations, we conduct a pre-post evaluation with real-world stakeholders, as players to test the game’s influence on four learning outcomes drawn from theories of collaboration and CPR management. The remainder of the introduction describes (1) the policy context of the project in California groundwater management, (2) evaluations of serious games for environmental collaboration, (3) serious games that influence participants’ beliefs and knowledge, and (4) our research questions and hypotheses.

1.1. Collaborative Management of California Groundwater

Between 2011 and 2016, the U.S. state of California experienced its worst drought in modern history. The four-year period between 2011 and 2015 was the driest observed since record-keeping began, and coincided with two of the hottest years in the state’s recorded history [1]. The drought had far-reaching consequences, including drinking water wells going dry [2], mandated reductions in urban water use, significant reductions in surface water supplies for agriculture, the death of an estimated 102 million trees [3], and stress placed on many of the state’s ecosystems [1].

Due to the lack of precipitation during this period, the state turned to groundwater to meet immediate water needs. California’s roughly 1 million groundwater wells provide up to 46% of the state’s total water supply during a dry year [4]. The recent drought accelerated the long-term decline in groundwater observed in many parts of the state since the 1960s [4]. In response, the state adopted the Sustainable Groundwater Management Act (SGMA) in 2014, which created the state’s first comprehensive mandate to change how groundwater is managed in California. The law requires the formation of local groundwater sustainability agencies (GSAs) and, for many groundwater basins, the development of groundwater sustainability plans (GSPs). The law mandates that these plans address how local stakeholders will manage groundwater to avoid the following “undesirable results”: (1) lower groundwater levels, (2) degraded water quality, (3) seawater intrusion, (4) land subsidence, (5) reduced groundwater storage, and (6) depletion of interconnected surface water. The act gives GSAs broad leeway in the specific actions they take to meet these goals, allowing them to tailor their plans to their unique local contexts [5]. Depending on local needs, management strategies may include the imposition of spacing requirements on new groundwater well construction, establishing pumping allocations, allowing for the transfer of allocations, assessing fees on pumping, and the development of agricultural land fallowing programs [6].

In environmental management terms, groundwater is a typical example of a CPR [7], which is defined as a resource that is available to many users, but whose large size makes it challenging (but not impossible) to exclude users. Indeed, early research by Elinor Ostrom on how to develop institutions to manage CPR focused on California groundwater management as an example [8,9]. Although an extensive literature by Ostrom has documented how different communities have been able to create institutions that successfully manage CPR, their creation is by no means guaranteed, and there are many examples of CPRs that have been mismanaged, even with institutional support [8,10,11]. For example, Ostrom has argued that 10 variables are associated with the sustainable management of a CPR, including the predictability of the resource system, the creation of collective-choice rules in the governance system, and the norms, knowledge, and social capital of the users [12].

The California groundwater case thus illustrates two broad, interrelated trends in the environmental management field. First, increasing anthropomorphic stressors on natural systems—chief among them climate change—are intensifying the need to improve the management of many CPRs [13,14]. The magnitude and severity of these changes—and in this case legislative mandates—are forcing shorter timelines for progress to avert severe consequences. Second, there is a growing emphasis on collaboration with beneficiaries to achieve desired management outcomes—not only the sustainable management of CPR, but also the creative, flexible, “win-win” solutions that collaboration is believed to foster [15]. SGMA therefore implements a collaborative approach applied in many policy domains, including watershed management [16,17], hydropower licensing [18,19], marine conservation [20], transportation planning [21,22,23], as forest restoration [24], and regional land-use planning [25].

Although Ostrom’s research has identified the conditions under which communities have successfully managed a CPR through a governance system, such as preventing the depletion of groundwater resources by adopting rules restricting pumping, it says little about how these governance systems are created. In addition, the norms, knowledge, and social capital of the users required for sustainable management identified by Ostrom can be deliberately cultivated. For theories which focus on how governance systems are created, and how participants can learn, we turn to three leading frameworks for collaborative planning and governance that identify common requirements for success and complement Ostrom’s ideas by specifying more specific variables and proposing a process resulting in new governance systems. Emerson, Nabatchi, and Balough provide an integrative framework identifying power relations, levels of trust, and differential incentives among stakeholders as essential contextual factors and drivers of successful collaborative governance [26]. Similarly, Ansell and Gash [27] identify asymmetries in power, resources, and knowledge among participating stakeholders, as well as their history of cooperation or conflict, as key factors that shape the incentives and constraints on stakeholder participation. A related collaboration model proposed by Booher and Innes [28] stresses the importance of involving a diverse and interdependent set of participants.

These collaboration frameworks all include extended negotiation as a critical part of a collaborative process; however, practically speaking, participants in collaborative processes (such as SGMA) face a difficult question: How to foster successful collaboration among groups of stakeholders with diverse and conflicting interests, wide power and knowledge asymmetries, and potentially low levels of perceived interdependence? For example, a recent study of collaboration in Puget Sound salmon restoration found that mandated relationships are associated with lower productivity than shared interest relationships, highlighting the benefits of genuine collaboration that arise from a shared belief in interdependence, rather than only on external mandates to collaborate [29].

If obstacles to collaboration are not addressed, collaboration may suffer, due to a low level of stakeholder motivation. Stakeholders with the most resources and political influence may be tempted to impose unilateral solutions that meet their interests, but neglect the interests of less powerful ones, such as smaller farms and rural communities. When collaboration levels are low, governance can be dominated by elite stakeholders, which raises normative concerns. In addition, a small but growing body of research suggests that poor collaboration leads to inferior policies, such as less creative plans [23], longer decision times [18], superficial or flawed use of scientific knowledge [30], and inferior results on objective goals such as water quality [31]. Even proponents of game-theoretic perspectives on conflict—which stress self-interested behavior far more than the creativity and mutual gains possible through collaboration—call attention to the importance of game fairness to explain stakeholder motivation for ongoing cooperation [32].

Therefore, this paper addresses the problem of catalyzing collaborative planning in groups through a participatory activity; specifically, a serious game for environmental management called the Groundwater Management Game. To examine the effect of playing the game on participants, we report results obtained from gameplay with stakeholders involved in groundwater management in California. Through a pre-test–post-test research design, we examine whether gameplay increases four specific outcome measures that reflect successful collaboration. Since the game and our research surveys are freely available, we invite replication of this investigation or other research which builds on our findings.

The primary contribution of this paper is to present a rigorous evaluation of cooperative game play among real-world water management stakeholders. As the following section explains, a large amount of literature on serious games points to the need for empirical studies to establish their policy-relevant value [33], but contains relatively few such evaluations. In addition, the literature provides few examples of operationalizing concepts from collaborative planning and governance using quantitative measures [18,34]. Our study’s evaluation of learning outcomes means that it may be possible that the game could influence subsequent collaborations among stakeholders, bridging the worlds of serious gaming and collaborative planning.

1.2. Evaluating the Effectiveness of Serious Games for Environmental Collaboration

The term ‘serious games’ refers to the use of games to achieve various learning objectives, either within a traditional educational setting or among adults [35]. Serious games are not meant solely for entertainment; rather, they are tools to facilitate social learning and teach decision-making skills.

Several recent studies demonstrate the potential for serious games to improve groundwater-related collaboration. A recent special issue in the journal Water offers several related studies of serious games and water governance, such as a role-play simulation in the U.S. improved a group’s ability to develop a water safety plan [36] and how role playing in Bangladesh resulted in participants learning about drinking water supply strategies [37]. Serious games have been shown to foster social learning in complex tasks, such as developing a new regional conceptualization for transboundary water management policies [38]. One study of a serious game to foster improved groundwater management in Andhra Pradesh, India, illustrates their potential as a management tool. Communities which played the game as part of a broader NGO-led educational program were more likely to adopt rules to govern groundwater than other program communities who did not play the game [39]. However, this study assessed learning by examining shifts in water-specific mental models, and did not use pre-tests–post-tests to examine changes to a wider set of learning outcomes.

Serious games and simulations in other areas of policy learning suggest the need for more rigorous evaluation studies. For example, Rumore, Schenk and Susskind [40] identify only a limited number of rigorous studies of serious games. Drawing on their own work with serious games on climate change adaptation, they suggest that such games have the potential to improve adaptation literacy, enhance collaborative capacity, and foster social learning. A review of game research by Hays [41] concludes that the evaluation of instructional games is fragmented by topic, age group, and type of game. Madani, Pierce and Mirachi [42] describe 25 serious games for environmental management, but do not evaluate them; instead, they conclude that future research should investigate how to evaluate game effectiveness: “Developing standardized methods for evaluating the effectiveness of games is a critical research frontier, recognizing that the effectiveness of games is not universal…” [42]. Connolly and colleagues reviewed the learning and behavioral outcomes of 70 higher-quality digital game studies [43], and concluded that knowledge, affective and motivational outcomes are most commonly observed. However, few papers in this review provide evidence of higher-order thinking and social skills, and few are categorized serious games.

Duke and Geurts [44] summarize the evaluation of serious games in their book, entitled Policy Games for Strategic Management. They conclude that outside of the educational field, “there is less empirical research on the conditions, utility and relevance of trajectories and tools within the field of participatory policy analysis than one would expect” [44]. They also note that the literature generally presents games without evaluation; although participant surveys can gauge satisfaction, they are not typically designed to test impact. However, Duke and Geurts [44] single out the work of Rouwette and colleagues [45] as being exemplary serious game research, proposing the following criteria for a high-quality evaluation:

• The assessment was guided by a well-established theory.
• The theory was used to generate testable hypotheses linking the game to changes in beliefs, attitudes, perceived behavior, and behavioral intentions.
• Data were collected before and after gaming.
• Measurement instruments were well-designed.
• Data were analyzed using appropriate statistics.

Other studies support these criteria; for example, Roozeboom, Visschedijk, and Oprins [46] suggest that game evaluations should use pre-test–post-test designs to examine learning outcomes. Their results show that experimental groups experienced changes to all of their learning outcome measures. Our study follows these guidelines to investigate a serious game’s impact on individual players’ attitudes and beliefs about cooperative water management.

1.3. Serious Game Evaluations Observing Belief Change

Some studies demonstrate that serious games can affect their participants’ beliefs and knowledge, which highlights their potential benefits for enhancing cooperation in groundwater management applications. The groundwater game field trial in Andhra Pradesh, India, described above documented statistical differences in water-related mental models between communities that played the game and control communities [39]. Corrigan and colleagues [47] evaluated the effectiveness of an airport operations serious game for collaborative learning through qualitative case information. Similarly, Cowley and colleagues [48] measured topic comprehension before and after participants played a single-player serious game called Peacemaker. After the game, some participants engaged in a facilitated group discussion, while others did not. Although such discussion should enhance learning, the scores of those who participated in a discussion were lower than non-participants. These results suggest that unintended effects may occur during the interaction of gameplay and reflection.

Rouwette and colleagues [45] report changes in attitudes, norms, perceived behavioral control, and intention to act following play in a game called Marco Polo. They conclude that gameplay strengthens players’ belief that responding to both consumer and market developments is beneficial. Fink [49] reports on an empirical exploration of setting characteristics (types of participants, group sizes, game length, and whether players are actual stakeholders) in a simulation of EU trade negotiations. The results show that the duration, number, and type of participants did not influence outcomes, except that including real-world European Parliament members, which resulted in different outcomes; furthermore, there were some national-level differences in negotiation style. Hummel and colleagues [50] report on an evaluation of a serious game for complex learning with limited virtual collaboration. Although the study has a small sample size (n = 12), survey data support the claim that the game improved learning outcomes, especially in students’ insights into differing perspectives. However, the limited nature of collaboration in this game resulted in satisfaction ratings.

The serious game evaluations most similar to the present study describe cognitive changes resulting from gameplay. One study evaluated a 3-h role-play simulation about sustainable development, which aimed to improve both substantive knowledge and skill outcomes [51]. This serious game resulted in higher scores on knowledge outcomes, yet somewhat lower scores for skill-based outcomes. Haug, Huitema, and Wenzler [52] also describe an evaluation of a serious game about European climate policy. They investigated cognitive, relational, and normative outcomes through surveys and found limited evidence for cognitive and relational learning, though this may be due to high levels of participant expertise. They also conducted a concept map exercise and found some evidence of cognitive changes. However, they conclude with the need for a more systematic assessment of learning effects in interactive appraisal exercises.

In sum, the evaluation of learning through serious games has resulted in few conclusions about their effectiveness and minimal evidence for improvement in participants’ collaboration intent. The use of pre-test–post-test designs, and in some cases treatment and control groups, allowed for placing the effects of games in context. For example, Rouwette and colleagues [45] use pre- and post-tests to document shifts in cognition and evaluations about market and consumer demands from playing a serious game.

1.4. Research Questions and Hypotheses

As described above in Section 1.1, several theoretical frameworks for collaborative governance which posit stakeholder interdependence arise from key qualities of these problems [26]. Diverse participants represent diverging interests in common resource use [28]. These stakeholders have different incentives to manage resources [27]; they also have different power relations, levels of trust, other resources, and in particular, knowledge about common resource management. However, if stakeholders can learn about the nature of their interdependence through exploring potential resource management strategies, will their interest in CPR management and collaboration change?

To investigate this question, we created a new serious game to capture these key qualities of CPR problems: by exposing players to principles of management, they may learn about the need for collaborative solutions. In a water management scenario, this requires gathering a group of local stakeholders within a groundwater basin, identifying differences in financial resources and power relations within members of the group, demonstrating that they have different incentives and constraints on their participation, and showing how the effectiveness of the selected management scheme can enhance ongoing cooperation [32]. If we make these differences among stakeholders explicit and visible through the game, the experience of insufficient water resources may provide participants with a compelling motivation to change their views on cooperative management.

In the evaluation of learning from the serious game, we hypothesized that playing the Groundwater Management Game will provide a base of knowledge about common pool resources and challenges to their management. In particular, by observing the effects of game play, we investigate the following hypotheses:

• Given the importance of interdependence in collaborative planning theory, our first hypothesis was that playing the Groundwater Management Game would increase players’ perceptions of interdependence among stakeholders.
• Since collaboration—and negotiation more broadly—can benefit from the stakeholders’ ability to understand others’ perspectives, our second hypothesis focused on changes in perspective-taking ability. Hummel and colleagues [50] found that serious gameplay improved students’ insights into differing perspectives. We hypothesized that playing the Groundwater Management Simulation would produce similar results with SGMA stakeholders: that stakeholders would understand and empathize with others through increased knowledge of the constraints others face, increasing players’ ability to take others’ perspectives.
• Next, since one identified outcome from serious games is attitudinal change [45], we predicted that environmental resource attitudes would generalize from groundwater resources to pro-environmental feelings about other resources, increasing players’ interest in natural resource management.
• Since the policy context for the game so strongly resembles a classical CPR problem, we sought to investigate the extent to which playing the game would result in stronger attitudes about effective CPR management components, as identified by Ostrom and colleagues [53]. We expect gameplay to increase players’ optimism in their views of collaborative CPR management.

To test these hypotheses, we examined learning from gameplay among real-world stakeholders recruited to play the Groundwater Management Game. Stakeholders from California groundwater basins were invited to participate through their previous or current engagement in management and planning activities. The study’s design included pre-test–post-test surveys, containing scales, to measure four outcomes corresponding to these hypotheses.

2. Materials and Methods

2.1. Research Design

The overall study research design utilized a pre-test–post-test survey to investigate the hypotheses conducted before and after groups of California groundwater stakeholders involved in SGMA played the Groundwater Management Game. The survey questions and game materials were tested during two sessions held with a total of 24 university students, one with undergraduates and the other with graduate students. The order of the survey questions was not varied. This research design has several limitations. The short time before the pre- and post-testing means we are not able to test for long-term learning or attitudinal changes, and respondents may recall their earlier responses. In addition, despite our efforts to recruit a representative cross-section of real-world SGMA stakeholders, these were not selected randomly, either from the SGMA process or the population at large. However, the strength of this approach is to test the game’s effectiveness among the target group for this collaboration intervention.

2.2. The Groundwater Management Game

This research evaluates a novel serious game called the Groundwater Management Game, developed over a three-year period to support stakeholder education and engagement around groundwater management challenges in California. The development of this game had three phases: (1) scoping sessions with target stakeholders featuring an older game and feedback surveys in June–December 2017, (2) developing the new game (January–May 2018), (3) and three game play events resulting in the data reported in this paper (June 2018, December 2018, and December 2019). The following section describes these phases in more detail.

The hands-on, interactive game provides players with the opportunity to experience the challenges of managing increasingly scarce groundwater when there are competing needs. The game also provides players with a greater understanding of different management tools and approaches [54]. This game was developed based on previous educational games about CPR management created by the Environmental Defense Fund (EDF), a U.S.-based environmental nonprofit. One existing game, What’s the Catch?, is used in fisheries management [55] for groups to discuss and explore alternative management strategies, such as catch-share strategies, to foster greater creativity in the management and regulation of fisheries.

The earliest iteration of the Groundwater Game was developed by EDF in partnership with a California-based consulting firm, Environmental Incentives. The multi-player digital groundwater simulation game explored the use of markets, specifically tradable water pumping credits, as a policy tool in the context of groundwater management [56]. Created to test behavioral economics research questions, the game initially featured a group of all agricultural players with differing water needs, who play under differing water availability scenarios. This game was tested with approximately 400 university students across California, to assess how playing the serious game affected preferences for different management strategies.

This existing groundwater game was transformed for use in the present study in the form of a multiplayer game, enacting water management issues. An EDF field team and researchers in collaborative planning and applied psychology worked together to create the new game following the guidelines for serious games. This game aims to teach players about CPR management more broadly, by illustrating how individual stakeholders impact collective outcomes. Stakeholders play in groups of six, and assume differing roles of individuals with water needs sharing a water basin. The development cycle included playing the earlier version of the game seven times with 44 people, held between June and December 2017. These pilot participants were in California and included members of irrigation districts, water-focused NGOs, environmental justice groups, and county farm bureaus. Players gave both verbal and written feedback on the game, such as suggestions to add roles for different agricultural growers, municipal water users, and disadvantaged communities; the need to incorporate surface water use and variations in access to surface water due to climate variability; and the flexibility for participants to collectively devise their own groundwater management strategies.

The resulting game, the Groundwater Management Game, is played by groups of six individuals. Each is assigned a stakeholder role at random (Alfalfa Grower, Almond Grower, Broccoli Grower, Rural Family, Community Water System, Urban Water Utility) for the duration of the game. Each role is identified by a specific water need (ranging from 2 to 50 beads per year), the resources available, and a water use pumping capacity (represented by measuring spoons of difference sizes). The players are told that they must try to meet their assigned water need (provided on their role description), make decisions to sell or buy needed water from each other, and consider how to balance individual and collective interests in managing the shared resource. To simulate the shared water resources, a large transparent bowl (the “reservoir”) is filled with a set number of blue glass marbles (the “water”) at the start of the game. Players are given different-sized measuring spoons (their “pump” for removing allocated water from the reservoir) based on their assigned role, and they each draw their water allocation from the reservoir to store in their collection cup. The game also incorporates surface water allocations for the Almond and Alfalfa Growers and the urban water utility, which are randomly assigned based on the roll of a die to represent a dry, average, or wet year (resulting in 0, 5 or 10 additional beads).

On each round of gameplay, the players must seek to meet their assigned water need, make decisions to sell or buy needed water from each other, and consider how to balance individual and collective interests in managing the shared resource. First, the players take turns withdrawing their water allocation using their “pump;” then, the water each player has received is evident to all, by glancing at the level of glass marbles in their “collection” cups. This visual display of resource use makes differences very explicit, helping each player to recognize the impact of their own and others’ resource use.

Next, during rounds where it is allowed, the players can discuss and negotiate individual trades to acquire or sell any additional water, as needed. This element of the game allows participants to experiment with trading water credits, one management tool available to groundwater managers under SGMA. At the end of the round, the players each count and report their water resource (number of marbles), and a coordinator tracks the water use for all on an Excel spreadsheet, recording player decisions and outcomes for the round. The spreadsheet contains charts describing the history of the aquifer and player-level water use and earnings (Figure 1). Then, the aquifer is recharged with additional beads representing annual rain, and a new round begins. This allows players to see the impact of alternative management strategies in the differing outcomes of each round.

The game continues in six rounds, each simulating one year of water use. The coordinator specifies the management strategies used in each round, so that initial rounds expose players to different management options. The first two rounds are always played with unrestricted pumping, which is used to demonstrate the need for management action by the group, to avoid using up water more quickly than it is replenished. Round three introduces the concept of sustainable yield, allocating an amount equal to the simulated rainfall in that year using proportions, based on the historical unrestricted pumping from the first two rounds. Round 4 uses a sustainable yield with a historical allocation, but also introduces the ability to trade water. The group is then asked to deliberate and decide on the management approach used for the final two rounds, which combines a choice of how water will be made available (open pumping, historical allocation, or equal allocation), with a choice about whether or not trading is allowed (Figure 2). Although the spreadsheet allows for six rounds, the game play events where data were collected only featured five rounds of play. This gameplay allows players to see and compare the impact of specific management strategies over years (rounds), and to highlight a visual comparison of comparative outcomes from allocation strategies. Game play takes between 75 and 90 min, including the debriefing of participants.

2.3. Participants

Study participants were drawn from three groundwater-focused planning venues in California, where the research team was able to obtain permission to play the game. First, 11 people in a water leadership training targeting rural and small-scale water systems were invited to participate during a training session held in El Centro, California, in June 2018. The participants included citizens concerned with how their water was being managed within their community, as well as staff members from small water utilities. The second game play event occurred in December 2018 during a community planning meeting composed of 12 stakeholders involved in the Solano Subbasin Groundwater Sustainability Agency Collaborative, an organization convened under SGMA to create a GSP. The Solano subbasin is located in the eastern portion of Solano County in the southern Sacramento Valley, and is an area pumped extensively for local agricultural and municipal uses. The third game play event was held in December 2019 in Davis, California, and co-hosted by the Army Corps of Engineers. The 18 participants included staff from the Army Corps of Engineers, U.S. Department of Agriculture, California Department of Water Resources, consultants who work on groundwater issues, legislative staff, and two graduate students studying water management.

The stakeholders’ self-reported identities are shown in

Table 1. Game participant stakeholder identity in groundwater management.

Stakeholder Role	n	%
Other	8	20
Utility water customers	8	20
Disadvantaged communities	8	20
Water and environment NGOs	7	17
Local management agencies	6	15
Community water systems	5	12
Utility companies	4	10
Small-scale farmers	4	10
Irrigation districts	3	7
Tribal interests	3	7
State water agencies	3	7
Large-scale farmers	2	5
Growers of different crop types	2	5
Agribusiness interests	2	5
Mutual water companies	1	2

, and reflect the wide range of players, from grassroots community leaders in some of the most rural parts of the state, to professional stakeholders involved in SGMA activities. The role assignments for game play were intentionally made at random to allow individuals to experience other roles. Spanish translations of all game materials, as well as the research surveys, were prepared to accommodate the bilingual audience, but all participants chose to play the game and complete the evaluation surveys in English. In order to accommodate the six-player game setup, the El Centro workshop had one member of the research team participate as a player in one group, and during another gameplay event, two participants shared a role and played together. In sum, 41 people participated in a game play session. Due to an incomplete survey with some blank questions, our sample size for two of the four scales was 40, as noted below. The average age was 47.7 (SD = 14.2), and the average self-rating on the SES measure was 6.4 (SD = 1.8) out of 10. Approximately half (51%) were female, and the majority were White (68%), with 17% Hispanic and 12% Asian American or Asian and Pacific Islander.

2.4. Measures

In line with our hypotheses, we created four outcome measures to assess learning as a result of gameplay. The same measures were presented in the same order on the pre- and post-survey, with the addition of demographics and open-ended questions at the end of the post-survey. The scale items in each measure are included in the results tables.

To measure interdependence perceptions, we revised items from Deyle and Wiedenman’s [23] study of collaborative transportation planning, to include information on groundwater basin issues. This study operationalized the concept of interdependence from the collaborative planning literature. For example, one item asked, “To what degree are the people in your groundwater basin dependent on each other to manage water resources over the long term?” These four questions, and interdependence scale, assessed increases in participants’ recognition of stakeholders’ dependence on the common resource, based on the simulation game play.

To create an outcome measure of perspective taking, we drew from the perspective-taking trait subscale of Davis’ [57] study of individual differences in empathy. We revised the wording slightly to make these nine items gender neutral (by changing “the other guy’s” to “another person’s” point of view). Two questions are reverse-scored (as in the original study); for example, the rating on, “If I’m sure I’m right about something, I don’t waste much time listening to other people’s arguments,” is reversed, so that across items, a high rating (7) always indicates high levels of taking the other’s perspective.

To measure pro-environmental attitude change, we used a 6-item scale developed by Liao and colleagues [58,59], including items about water, as well as other natural resources.

The nine scale items tested stakeholders’ optimism about resource management, including original questions generated from Ostrom’s earlier work on groundwater management [9] and from Dietz et al. [60] For example, items like, “We can come to an agreement on how to sustainably manage groundwater levels,” serve as an index of beliefs about the potential for successfully managing CPR. These items were altered to include specific references to water management.

Finally, on the post-test only, a demographic section included questions about the stakeholders’ involvement in the basin, as well as their gender, age, race, and ethnicity. In addition, we included a standard measure of socioeconomic status (SES) [61]. The question uses an image of a ladder to represent SES across 10 increasing levels and asks respondents to identify the rung of the ladder where they would place themselves.

3. Results

First, we tested the internal consistency of our four scales (Interdependence, Perspective-Taking, Pro-environmental Attitudes, and Water Management Optimism) from the survey. Cronbach’s Alpha was calculated using Stata. All four scales were found to have acceptable scale reliability coefficients: 0.67 for Interdependence, 0.80 for Perspective-Taking, 0.71 for Pro-Environmental Attitudes, and 0.75 for Optimism.

Next, we conducted two-tailed, paired T-tests in Stata for scale-level differences (see Figure 3). All scale averages (both pre-game and post-game) were above the midpoints, suggesting that participants were generally positive on the measures before the game began. Even so, two scales showed statistically significant improvement from before to after play: interdependence (p = 0.001) and optimism (p = 0.02). For interdependence, the scale average increased 21.7% after game play, and for Optimism, the already high levels pre-game increased by 6.4% post-game. The Perspective-Taking scale averages were similar, and the Pro-Environmental Management scale average decreased slightly, though both were high pre-game. All of the scales are summarized below by item.

3.1. Perceived Interdependence

The interdependence scale (

Table 2. Perceived Interdependence n = 40.

Scale Items	Pre-Game (Standard Deviation)	Post-Game (Standard Deviation)	Change	p
A1. To what degree are the people in your groundwater basin dependent on each other to manage water resources over the long term?	3.9 (1.8)	4.8 (1.5)	0.9	0.009
A2. To what degree do the people in your groundwater basin recognize their interdependence to manage water resources over the long term?	2.9 (1.5)	3.7 (1.7)	0.8	0.009
A3. To what extent do the people in your groundwater basin collaborate with each other to manage water resources?	3.07 (1.4)	3.9 (1.6)	0.9	0.009
A4. To what extent do you feel people in your groundwater basin need to collaborate with each other to manage water resources?	4.9 (1.4)	5.4 (0.8)	0.6	0.008
Total Perceived Interdependence Scale	14.6 (4.3)	17.7 (4.1)	3.2	0.001

) captures ratings from 0 (Not Interdependent at All) to 6 (Very Interdependent) on four items. The change from pre-game to post-game was statistically significant for all items, with the mean values for each interdependence item increasing from 0.6 to 0.9 (almost one point on the 7-point scale), as reported post-game.

3.2. Perspective-Taking

The nine questions on this 9-item scale were scored on a scale of 1 to 5, with 5 indicating that the given statement “Describes Me Well” and 0 indicating that it “Does Not Describe Me Well.” As

Table 3. Perspective-Taking. n = 41.

Scale Items	Pre-Game (Standard Deviation)	Post-Game (Standard Deviation)	Change	p
B1. I sometimes find it difficult to see things from another person’s point of view. (reversed)	3.9 (0.9)	3.9 (0.9)	0	0.87
B2. I try to look at everybody’s side of a disagreement before I make a decision.	4.1 (0.8)	4.1 (0.8)	0	1.0
B3. I try to understand my friends better by imagining how things look from their perspective.	4.0 (1.0)	4.1 (0.8)	0.1	0.36
B4. If I’m sure I’m right about something, I don’t waste much time listening to other people’s arguments. (reversed)	3.6 (0.8)	3.6 (0.9)	−0.1	0.62
B5. I believe that there are two sides to every question and try to look at them both.	4.1 (1.0)	4.0 (1.1)	−0.1	0.53
B6. When I’m upset at someone, I usually try to “put myself in their shoes” for a while.	3.5 (1.0)	3.6 (0.9)	0.1	0.58
B7. Before criticizing somebody, I try to imagine how I would feel if I were in their place.	3.7 (1.1)	3.8 (0.9)	0.1	0.57
Total Perspective-taking Scale	26.9 (4.6)	26.9 (4.3)	0	0.99

shows, no item changed significantly from pre- to post-game.

3.3. Pro-Environmental Attitudes

The six items in the Pro-Environmental Attitude scale were rated on a scale of 0 to 6, with 6 corresponding to “Strongly Agree” and 0 corresponding to “Strongly Disagree.” Among the questions concerning natural resource attitudes, only one—about whether climate change will have a local impact—shows a significant change at the 95% confidence level (

Table 4. Pro-environmental Attitudes. n = 41.

Scale Items	Pre-Game (Standard Deviation)	Post-Game (Standard Deviation)	Change	p
C1. I am concerned about climate change.	5.4 (1.0)	5.5 (1.0)	0.1	0.83
C2. Climate change will only affect places far away from me. (reversed)	5.6 (0.8)	5.1 (1.7)	−0.6	0.03
C3. I am concerned about global water resources.	5.3 (1.0)	5.4 (1.0)	0.1	0.52
C4. I am concerned about local groundwater resources.	5.2 (1.4)	5.5 (0.8)	0.3	0.15
C5. Environmental issues are the least of my problems. (reversed)	5.1 (1.2)	5.1 (1.0)	0.0	1.0
C6. The earth’s natural resources are limited.	5.5 (1.0)	5.4 (0.9)	−0.1	0.68
Total Pro-Environmental Attitudes Scale	32.1 (4.2)	31.7 (4.4)	−0.4	0.37

). One other item, about local water resources, shows a marginal increase post-game (p = 0.15).

3.4. Optimism

The nine questions on the Optimism for Water Management scale were scored on a rating scale of −5 to 5, with 5 corresponding to Completely Agree and −5 corresponding to Completely Disagree. Responses to questions about optimism for the water management process generally showed small increases (

Table 5. Optimism for Water Management, n = 40.

Scale Items	Pre-Game (Standard Deviation)	Post-Game (Standard Deviation)	Change	p
D1. Groundwater resources must be managed for the benefit of all	4.3 (1.2)	4.3 (1.0)	0	1
D2. We can develop ways to manage, use, and conserve groundwater.	4.2 (1.6)	4.3 (0.9)	0.1	0.63
D3. We can cooperate and work together to prevent overuse of groundwater.	4.3 (1.0)	4.3 (1.1)	0	0.84
D4. Establishing rules for how groundwater can be used is good for everyone.	4.1 (1.3)	4.3 (1.1)	0.1	0.57
D5. We can come to an agreement on how to sustainably manage groundwater levels.	3.4 (1.4)	3.8 (1.3)	0.4	0.03
D6. I am willing to follow water use rules as long as others also follow these rules.	3.1 (2.3)	3.5 (1.8)	0.5	0.13
D7. I am willing to communicate often with other groundwater users in my basin.	3.6 (1.6)	4.1 (1.1)	0.5	0.01
D8. There are many individuals who are legitimate beneficiaries in the groundwater basin.	3.5 (1.8)	3.6 (1.6)	0.1	0.51
D9. It is in my own interest to follow the rules about groundwater use	3.5 (2.2)	3.9 (1.6)	0.4	0.16
Total Water Management Optimism Scale	33.8 (8.4)	35.9 (7.4)	2.2	0.018

). Item 5 focused on groundwater management, and this direct item found significantly increased ratings post-game (p = 0.03). Another item, “I am willing to communicate often with other groundwater users in my basin”, also showed significant improvement (p = 0.01).

Reliable improvements in pro-management attitudes were observed on two of the four scales. These findings suggest that the game play had the most influence on attitudes toward water management itself, rather than a more general improvement in perspective-taking or pro-environmental attitudes; however, both of these measures were already high before participants played the game. This may have led to a ceiling effect, limiting the scales’ measurement of change.

4. Discussion

As hypothesized, playing the California Groundwater Game strongly increased participants’ perceptions of stakeholder interdependence in their groundwater basin. Interestingly, playing the game increased, not only their perception of overall interdependence, but also their insights into specific stakeholder perspectives. For instance, one participant wrote in the postgame survey, “(I) gained a perspective I did not previously hold, on the tight web interconnecting the various users of (ground) water and how powerless (relatively powerless) the less financially well off are.” (See Appendix A for all participant comments).

Likewise, another participant commented “groundwater users may not know or understand their interdependence on using this resource sustainably, collectively, until seeing the numbers. Regulation does make a difference.” Future research could further investigate the reasons behind these cognitive shifts, as well as probe different experiences among participants by gender or other aspects of identity. For example, the game may cause players to realize how different stakeholders are already collaborating, or it may cause them to revise their view of stakeholders (such as farmers), who may seem to lack motivation to collaborate. We think this is particularly significant since interdependence plays such a central role in theories of collaboration, and has been identified in recent empirical research as an important perception to develop in order to improve different forms of environmental collaboration [62].

Perspective-taking is studied mainly as an individual difference [57], and it is not necessarily easily changed by a single experience. However, Hummel and colleagues [51] did find evidence of a perspective change after a serious game. In the present study, there was no evidence of change from pre- to post-game averages. Evidence from qualitative data suggests that participants did enhance their perspective taking through the game, in ways not captured by the quantitative survey questions. For example, one participant noted, “I developed empathy for the 20 acre broccoli grower. I wouldn’t want to be in that business.” Another participant said, “The game gave me a perspective as (an) urban water (manager). Normally I look at water perspectives from a rural perspective.” These remarks suggest that more specific forms of perspective-taking (such as varied roles in water management) may be affected by gameplay.

Playing the game had only minor effects on pro-environmental attitudes. However, many participants reported highly pro-environmental attitudes before the game, with pretest means of around 5 on a 6-point scale. A participant group with a wider range of pre-game attitudes may show more improvement from game play. When asked what they learned from playing the game, one participant wrote:

It reiterated that there are many, many legitimate stakeholders when it comes to groundwater—and they all “need a piece of the pie.” I’m a water professional, so this game really just confirmed a lot of what I already knew about stakeholders and groundwater.

Concern about climate change (item C2) increased, despite no explicit mentioned in the game. However, the random assignment of annual surface water contributions (in dry, normal, and wet years) may highlight climate change as an uncertain factor affecting water supply.

Finally, the significant increase in optimism about the water management process shows that the game affected attitudes related to institutional changes in management. Successful CPR management practices, such as establishing shared roles, communicating with others, and abiding by the shared rules, appeared to impact players’ belief in the possibility of successful cooperative management. One participant reported learning, “options available for all players; and the importance to negotiate and interact with others for mutual benefit.” Another commented that, “all the hands in the bowl is a great metaphor for reality!” As players vied simultaneously for their share of the blue marbles representing water, the game vividly illustrated the limited nature of water resources, and provided a more concrete understanding of groundwater as a CPR.

We suggest several important next steps for future research. We encourage additional evaluations aligned with the collaboration frameworks introduced above [26,27,28]. The challenge is to identify the relationships between short-term changes through gameplay and long-term effectiveness in catalyzing actual collaboration. Longitudinal studies are needed to measure the persistence of game effects over time, along with direct behavioral measures of engagement in a local groundwater management process. In addition, studies showing the relevance of increased awareness of the interdependence of basin residents and increased optimism for management (regardless of whether a game is used) must be connected to successful collaboration outcomes. Such research may also identify other learning outcomes that are beneficial in serious game design.

During the game development process, we encountered several participants, often those from technical or scientific backgrounds, who desired more complexity, sophistication, and realism in the game’s representation of the groundwater system. They suggested modifying it to incorporate factors like climate change, weather patterns, water pollution, subsurface geology, and expanded management actions like groundwater recharge. Altering the game to make it more suited to specific stakeholders might be problematic, because differences in knowledge among stakeholder groups would not be addressed. It might also lead to technical discussions focused on policy or scientific minutiae, rather than allowing participants to engage more directly with the differing interests and perspectives that diverse stakeholders bring to the table. Increasing complexity and realism may work against the game’s accessibility to a wide range of stakeholders. While some players may be ready for more sophisticated simulations, observations of the gameplay suggest that the game conveyed simple but profound lessons that were appropriate for most participants. For example, one participant reported learning, “the interests of a variety of stakeholders and the need to collaborate to manage water resources for the good of all. The idea of ‘the greatest good for the greatest number.’”

For those new to groundwater management, such as the operators of community water systems for whom SGMA will be their first basin-wide collaboration, the game’s benefits include the concrete illustration of how varied management strategies play out in practice; for example, the unregulated management approach used in the early rounds demonstrates that some type of management strategy is needed. The game may therefore be a useful educational tool to improve willingness to engage with cooperative water management. The vivid visualization of differences in amounts of water use may enhance recognition of the importance of inclusion and equity in groundwater management [5]. Finally, serious games play a valuable role in allowing stakeholder groups to explore different possible outcomes of unfamiliar management strategies within a low-stakes simulation environment. Very few players asked for greater realism to depict issues that might arise during planning meetings held within their basin, suggesting less knowledge about issues arising from cooperation. By allowing stakeholders to see behaviors over several rounds under a given policy, the game might encourage participants to identify alternative strategies that might work in their basin. Of course, the value of such exploration depends on whether the stakeholders explore these strategies in greater depth in a more technical setting with access to simulations, including greater realism and specificity to their basin.

The limitations of the present study prevent us from drawing broad conclusions from these findings. While testing game play with actual groundwater basin stakeholders increased the external validity of the results, it also resulted in a smaller participant group than is desirable for comparison purposes. Though the game play events were intentionally offered in varied settings (rural, inland, etc.), a wider range of location and participant characteristics might be important in determining reactions to the game. For example, basins where management has been attempted unsuccessfully, or where management has not been considered yet, may result in different experiences and conclusions from gameplay. Finally, participants took the post-game surveys after only about 75 to 90 min of play; consequently, they may have recalled their answers to the pre-game survey, which may have resulted in a more conservative estimate of attitude change. Longer periods of play and delayed post-game testing may enhance the findings described in the study. Since the game spreadsheet, instructions, and other materials have been made public, we encourage others interested in playing the game to also consider collecting pre-test–post-test data, to replicate this study, or test additional hypotheses [54].

5. Conclusions

This article responds to a tension between two different trends. The environmental impact of climate change and other human activities are placing natural systems—and the ecosystems upon which they depend—under intense stress, necessitating quick and decisive action to address environmental problems. However, many of these same problems require intensive collaboration among affected stakeholders in local area groups, because top-down regulatory solutions are infeasible or ineffective. Groundwater management in California, the substantive context of this study, exemplifies this tension. The serious game, the Groundwater Management Game, shows that games may work to allow stakeholder groups to focus on understanding that the initial management strategies may not be sustainable given the limited CPR, and convince them of the potential value of collaboration even among stakeholders with unequal resources.

The gameplay results show that experience with alternative management strategies through the physical depiction of a serious game helps people understand the critical interdependence of stakeholders in groundwater basins. They also suggest that this experience increases optimism that collaborative planning can work. In addition, the game resulted in changes to attitudes about local impacts of climate change, and quantitative data suggested other valuable learning outcomes that may not have been revealed through the statistical analysis, due to the high level of knowledge of most players. Future research is needed to test the impact of serious games on longer-term outcomes, and investigate whether these factors can be linked to successful groundwater management outcomes. The current findings suggest that serious games hold promise for educating diverse stakeholders about common resource problems, conflicts and tensions, that can make collaboration difficult, and the potential benefits of successful cooperative management.