Next Article in Journal
Flow Regime and Nutrient-Loading Trends from the Largest South European Watersheds: Implications for the Productivity of Mediterranean and Black Sea’s Coastal Areas
Next Article in Special Issue
Conceptualizing Serious Games as a Learning-Based Intervention in the Context of Natural Resources and Environmental Governance
Previous Article in Journal
Evaluation of Evapotranspiration Estimates in the Yellow River Basin against the Water Balance Method
Previous Article in Special Issue
Serious Games as Planning Support Systems: Learning from Playing Maritime Spatial Planning Challenge 2050
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

A Serious Game Designed to Explore and Understand the Complexities of Flood Mitigation Options in Urban–Rural Catchments

1
Centre for Water Systems, University of Exeter, North Park Road, Exeter EX4 4QF, UK
2
KWR Watercycle Research Institute, 3433 PE Nieuwegein, The Netherlands
3
Department of Geography, University of Sheffield, Winter Street, Sheffield S10 2TN, UK
4
Westcountry Rivers Trust Head Office, Rain-Charm House, Kyl Cober Parc, Stoke Climsland, Callington, Cornwall PL17 8PH, UK
*
Author to whom correspondence should be addressed.
Water 2018, 10(12), 1885; https://doi.org/10.3390/w10121885
Submission received: 13 September 2018 / Revised: 7 December 2018 / Accepted: 11 December 2018 / Published: 19 December 2018

Abstract

:
Flood prevention in mixed urban–rural environments has become a greater concern due to climate change. It is a complex task requiring both efficient management of resources and the involvement of multiple stakeholders from diverse backgrounds. As Serious Games (games used for purposes other than mere entertainment) have emerged as an effective means of engaging stakeholders, this work proposes a new Serious Game applied to flood mitigation in the village of Millbrook in the UK. Results show that the game has both an informative and a transformative effect (statistical significance levels from 0.01 to 0.05), improving participants’ understanding of the problem, and helping them to find a new and improved approach to flood risk management in Millbrook, with the potential to improve resilience significantly. Furthermore, the game successfully transformed participants into “citizen scientists” in the purest sense of the term—it led them to use inductive reasoning from data produced by the game to correctly confirm or reject hypotheses and resulted in more than 70% of the participants revising their initial assumptions. Interestingly, the game instigated the formation of new local partnerships and helped to prioritize the discussion of natural flood management measures in Millbrook Parish Council meetings.

1. Introduction

Climate change is currently causing a substantial increase in flood risk across the UK [1,2]. High residual risk of flooding will remain, even with the current investment plans on flood defense [3], and hard engineering solutions will not be affordable to protect all areas. In this context, finding other ways to improve resilience to flooding by preventing or mitigating flood impacts becomes a necessity. The complexity of the problem is significant because flood prevention in an urban–rural environment requires both the management of limited resources and the involvement of multiple stakeholders with different perspectives such as residents, farmers, business owners, utilities companies and policy makers.
On the evening of the 30th of September 2016, a workshop involving comprehensive communication and engagement of stakeholders by the Westcountry Rivers Trust took place in the in the center of Millbrook, a small village in Cornwall, UK. Millbrook has had many recent problems caused by large pluvial flood events. The goal was to present causes to the Millbrook flood events as mostly pluvial, and to emphasize land use and natural flood management measures as potential solutions recommended by hydrology reports. Much of the participants’ attention ended up focusing on other relevant topics such as the presence of the reservoir in the village and the drainage infrastructure. In that context, the limitations of traditional methods as an effective conduit for translating complex data, modelled outputs and academic research findings, into clear and concise messages that can be understood by a non-technical audience became apparent. As an action research to respond to that problem, the idea of designing a Serious Game that could potentially fulfill that role was then born. This is also in line with the concept of Shared Vision Planning [4], which engages stakeholders in developing and experimenting with interactive simulation models for building consensus [5].
Serious Games (games used for purposes other than mere entertainment), as an effective means of engaging stakeholders, have been previously used for water system planning and management [5,6] with common aims of raising awareness [7,8,9,10,11], facilitating dialogue [11,12,13,14,15,16], or training for crisis response [17,18]. The Serious Game presented in this work aims to fulfill the research action goal in three ways. First, the software developed introduces novel technical elements to make it an effective Virtual Reality tool for visualizing results of simulations of 3D animated floods—a category of the Serious Game visualization system that is missing from the present body of work [6]. While existing work tends to emphasize “step-based” 3D flood visualization [19,20,21,22,23], the tool developed in this work focuses on the real-time deformation of 3D meshes representing volumes of water, with animated shaders allowing the mapping of color gradients to the mesh surface in order to emphasize flood depth. Second, to the best of our knowledge, this is the very first time that a Serious Game combining hydraulic simulation results, expert knowledge about farming systems and soil science, drainage infrastructure management, as well as representative damage and cost estimates have been put together to help players find an efficient (and previously unexplored) solution to the flood resilience problem for a real-world case study. Finally we propose a novel Serious Game intervention design inspired by the Socratic Method [23] and aim to prove that it can (a) enable participants to use inductive reasoning based upon game-derived data (hydraulic and costs) to confirm or reject hypotheses in a rational manner, and (b) encourage the players to question their initial assumptions (perception of the truth) and assist them in shifting towards a more reasoned, scientifically sound, deductive solution, i.e., closer to the evidence (“scientific truth”).
From an experimental design point of view, the Serious Game combines pre- and post-game questionnaires with a traditional interpersonal group-based intervention. The intervention is based on asking participants to collaborate and answer questions in a specific order while playing the digital game developed using the Unity® [24] game engine. It must be emphasized that the approach does not consist solely of the digital game, but also integrates the questionnaires, intervention, and software all together. In this paper, the following materials and methods section presents all the elements used to design the Serious Game. The results are then outlined in Section 3 and their implications are discussed in Section 4.

2. Materials and Methods

First, we present the design of the Serious Game as an intervention specifically focused on enabling non-scientists to understand complex scientific models. Second, we present the choices made during the game development to produce a Virtual Reality tool suitable for visualizing results of simulations of 3D animated floods. Third, we present the experimental design composed of pre- and post-game questionnaires and used to assess the intervention based on this Serious Game.

2.1. The Overall Design of the Intervention

Using criteria extracted from [5], the game design can be characterized as follows. The primary aim or outcome of the game is for the user to explore different flood mitigation actions and corresponding outcomes in order to answer five questions. Finding the answers to these questions leads to the discovery of the optimal flood outcome to the given problem while minimizing costs. During the initialization of the game, a facilitator presents the Millbrook flood problem and provides contextual information regarding the history of flooding in the village, explanations regarding how to use the interface as well as technical concepts such as terrain roughness, and the different farming systems that the players can choose from. Although the game is built as a single player experience, several participants can play it in parallel in the same room. Communication between participants is not compulsory, but is encouraged so that they can freely discuss their choice with their neighbors when answering questions. Players do not have a limited and differentiated role, and they are each encouraged to explore the whole range of possible flood mitigation choices during playtime. The user interface makes use of sophisticated 3D video technology comparable to that already available in the videos game industry so that the user is able to visualize results of simulations via a Virtual Reality tool. The simulation model used relies on cellular automata and a series of conditional statements. The level of realism of the game can be seen as moderate when considering its individual aspects, such as costs and hydraulics models, in isolation. When looking at all of them combined, the level of sophistication of the socio-technical-environmental system presented is significant. Performance feedback during gameplay is instant as the player is immediately informed of the consequences of their actions on the flood and cost outcomes. Progress monitoring, such as the “capability of saving intermediate game results for follow up analysis” [5] is ignored as the user can switch at any time the combination of measures they wish to explore. Game portability is good because it can be played off-line on most Windows computers, though we have limited this portability by delivering the Serious Game intervention in a supervised environment. The game is designed to be a challenging, utilitarian, and demanding exercise.
To fulfill our aim to help participants to explore and understand complex scientific models, the Serious Game intervention is designed using three steps as shown in Figure 1.

2.1.1. The Introductory Phase

Participants are given a general introduction (see link to additional materials at the end) on the Millbrook case study, and then asked to fill in a pre-game questionnaire (Appendix A) in which they need to confirm or validate different hypotheses, for the following reasons:
  • The evaluation of any eventual cognitive change brought about by the Serious Game. This allows us to understand the pre-game state of the initial participants’ views on the Millbrook flood problem. This includes possible pre-formed opinions (perception of the truth) on what would be the best possible decisions to achieve the balance between flood prevention and cost.
  • It alerts participants to the crucial questions that they need to keep in mind while they improve their understanding of the problem, and therefore helps them to consolidate memories on technical issues, and to validate or reject hypotheses at the end of their investigative work.
  • It creates a Serious Game equivalent of the questioning phase of the Socratic Method. The only difference is that instead of it emerging naturally through dialogue, the hypotheses are artificially created by the mix of introductory materials and the pre-game questionnaire.

2.1.2. The Exploratory Phase

During the exploratory phase of the game, players must interrogate the possible solutions and answer questions (Appendix B). The first two questions focus on finding the combinations of factors leading to the worst and best flood damage outcomes, e.g., “What is the combination of measures that leads to the worst flood outcome? Also note the damage/cost and the number of flooded houses.” The next two questions focus on finding combinations leading to outcomes with the lowest and highest associated management costs. The last question is about finding an optimal solution that minimizes both flood and cost. During this phase, the digital game provides the players with the freedom to apply their own management decisions. It gives them feedback in the form of simulation results, which they can use to check the validity of the given hypotheses.
The use of compulsory ordered questions is essential because:
  • It reduces an overwhelmingly large combination of solutions into a manageable list of five tasks that are essentially guiding participants’ explorations of the different solution scenarios.
  • It forces participants to answer questions where the solutions shown by the game might contradict what they initially thought were the best decisions, hence targeting preconceptions. For example, the exploration of answers to question 2: “What is the combination of measures that leads to the best flood outcome?” leads to the discovery that changing the farming systems around the village has a much greater impact than changing the level of investment in the drainage infrastructure. This corresponds to the Serious Game equivalent of the disproof and refutation stage of the Socratic method.

2.1.3. The Final Phase

Finally, in the third phase, players are asked to capitalize on the previous explorative work by finding the solutions that minimize both damages and management costs. They then have to fill in the post-game questionnaire (Appendix C) and apply inductive reasoning from the data revealed by their exploratory work to reject or confirm the presented hypotheses. Note that players can justify each answer in their questionnaire, giving them the opportunity to reach the Serious Game equivalent of the third step of the Socratic Method, namely to question their initial assumptions (“perception of the truth”) and replace them with what they deduced to be a more scientifically sound answer (“the truth”).

2.2. Game Development Choices

Millbrook has had many recent problems caused by large pluvial flood events. A 2 km square terrain around the village was selected, and uniform rain input of 45 mm/h for the first hour of simulation was applied, which equates to a 1 in 100-year event. In the game, players can simply change four parameters: the level of drainage infrastructure investment and the type of farming system applied in three instances (one for each sub-catchment in the village). They are then able to see flood impacts and associated management costs in numerous resulting combinations.
A 3D virtual table that comprises different informative documents and post-it notes introducing the case has been constructed as shown in Figure 2 to provide additional information to the users.
This innovative and original setting for flood visualization is quite powerful and expandable as follows:
  • This allows a wealth of information to be represented as a spatial construct of 3D meshes on a virtual table of practically infinite size (Figure 2), where the geographic location of each item on the virtual table leads to better recall, in a similar way to mnemonic “visual palaces” [25].
  • It can structure this information visually so that the user can see how components are related to each other in the case study, and allows the user to move, rotate, and zoom around components, and finally potentially display geographically distant elements as interconnected infrastructure nodes. For example, a distant power plant that influences the electricity supply of a village can be represented near the village on the virtual plan due to the cause–effect relationship between the two.
  • Furthermore, this setting is easily translatable as a Virtual Reality headset-based Serious Game without major code modification.
The Unity® rendering engine has been improved in our previous work [26] to allow the terrain/flood height to be rendered with a higher degree of precision. The height data are provided by standard images in .png format where each pixel’s red, green, blue, and alpha values encode in 32 bits the height of the terrain. From a practical point of view, these floating-point height data are rescaled into large integer numbers, between 0 and 232 (allowing the system to display the difference in height with sufficient precision). Furthermore, by using .png rather than Unity® default .raw file format, more compact images are produced that can easily be sent through the internet. Although, so far, the data are stored locally with the game, whole sequences of images could potentially be sent from a “game” server hosting many test cases to provide multiple animations steps through a flood sequence.
Terrain and flood surfaces can now be deformed in real-time using advanced Shaders [27] software that exploits the computational capabilities of graphic cards. Shaders—programs implemented in the OpenGL Shading Language to display vertices and visual fragmentation using the accelerated computational power of the graphic card—are adopted to output in real time a 3D geometric linear interpolation of the flood “height-map” between the starting and the end state by looking at the “counter” input. Flood animation is implemented by providing a series of images and “counter” float numbers between 0 and 1, expressing how far the animations are from the end stage.
The Shaders are engineered not only to deform 3D meshes efficiently, but also to change the flood mesh color gradient from clear blue to dark blue with increasing flood depth (Figure 3). This works seamlessly even on fairly modest machines or small laptops, animating meshes with more than 1,440,000 triangles in real time with ease, as long as they have a DirectX 11 compatible graphic card.
The Serious Game allows players to change (i.e., make decision about): (a) the level of investment in the drainage infrastructure, and (b) the type of farming system applied in each of the three sub-catchments in the village. They can then see the resulting flood impact and associated management costs. These inputs are integrated in the user interface as shown in Figure 4. The simplicity of the interface is the product of a compromise between computability, accessibility, and realism. Allowing the player to change the type of farming system in every individual field in Millbrook would provide a more realistic setting, but it could lead to a usability problem (manually changing each of the hundred fields before computing the consequences would detract from the games effectiveness because participants would lose sight of the problem while doing so). Furthermore, this would have been computationally impractical because it would require 2 × 6100 hydraulic modeling results to reflect the flood risk of different management scenarios. The present choice of grouping the fields into three cultivated areas allows the user to set farming systems quickly and explore different consequences of their choices, while also keeping the overall problem in mind. While the spatial resolution for targeting measures is quite low in the current model, it still provides a useful indication of which parts of the catchment provide the greatest constraints or opportunities for land use change and flood risk alleviation. It reduces the number of hydraulic computations to a manageable 432 combinations. Although it might initially look more complicated, the user interface and search space structure is quite simple because it closely follows the infiltration gradient of the farming systems as they are presented to the participants, from dark brown (system with the lowest infiltration rate) to dark green (system with the highest infiltration rate). Participants are therefore expected to find the best solution available and master the problem of minimizing both the flood damage and management cost in under an hour without having to explore all possible choices. Each farming system, as described in Table 1, comes with an associated infiltration rate, surface roughness value, sedimentation rate leading to a percentage of pipe blockage in the village downstream, and profitability (ranging from “−−“ for higher loss, “−” for average loss, “0” for null, “+” for average profit, and “++” for higher profit).
These farming systems are used as model values in the Serious Game in accordance with the latest available agri-environmental data for the Millbrook catchment and from a thorough review of relevant literature detailed in Appendix D.
To communicate effectively the consequences of the decisions made by users, three elements have been included on the interface of the Serious Game as shown in Figure 4.
  • A temporal slider at the bottom allows a user to change the timing during a selected event to see the progression of the flood. Note that the overall speed of the animation can also be paused, slowed, or accelerated.
  • A reactive flood damage information window in the top left corner shows the total number of households flooded, split by the severity of flood inundation, and tracks the costs associated with this. A further indicator shows the number of households flooded at a given point in time. When the mouse hovers on the window, a graph pops up showing the distribution of the number of flooded buildings over time to highlight the peak of the flood event as well as compare the number of occurrences of major versus minor flooding of houses.
  • A window at the top center-left of the screen shows the total cost of the selected drainage infrastructure management policy, as well as the total farming profitability of all three cultivated areas around the village.
The 3D map of Millbrook is 2000 m × 2000 m. Houses colored in orange have a water depth between 15 and 30 cm inside the building, while houses colored in red have a water depth beyond 50 cm. A player can choose between two levels of investment in the 16 km of the urban drainage infrastructure of Millbrook. The infiltration rates are shown in the context of a one in a hundred-year event of 45 mm of rainfall in one hour. The investment levels indicate the capacity of the drainage network to manage surface runoff under different maintenance conditions (additional data about drainage management costs are available in Appendix E).
  • A high-maintenance (high cost, well maintained) drainage infrastructure that can cope with surface runoff up to 26 mm/h/m2 in urbanized areas. At the high level of investment scenario, the yearly management cost of the urban drainage infrastructure is evaluated at 406k GBP and includes cleaning 10% of the length of all drainage pipes per year, repairing 3% of total length of pipes, and replacing 1%.
  • A low-maintenance (low cost, not well maintained) drainage infrastructure that can discharge surface runoff up to 13 mm/h/m2 in urbanized areas. At this low level of investment scenario, the yearly management cost of the urban drainage infrastructure is evaluated at 203k GBP and includes cleaning 5% of the length of all drainage pipes per year, repairing 1.5% of total length of pipes, and replacing 0.5%.
Flood depths encoded in images are produced using the WCA 2D flood model [29], a part of the CADDIES [30,31] modelling framework. CADDIES is an open-source framework designed to facilitate the design and deployment of Cellular Automata (CA), specifically for 2D urban flood modelling. It is ideal for fast evaluations of floods [31], and therefore allows the swift computation of multiple simulation results corresponding to various users’ choices in the context of serious gaming. In the present work, the flood models were pre-run to produce output images used to make the 3D animated mesh of the flood (more details about computation times in Appendix F).
The Millbrook example presents the users with a high-speed model, where a cell size of 2.5 m was selected for the flood modelling with WCA 2D as a compromise between accuracy and computability to produce the 432 different flood outcomes in a few days. The speed of execution makes it suitable in the context of serious gaming for quickly displaying numerous flood outputs resulting from different combination of parameters.
The existing farming situation on the ground in Millbrook (a mixture of “heavily grazed permanent pasture” and “intensive arable cultivated areas under tillage with standard rotation”) leads to the worst possible combination in the game. It has a maximum damage outcome (108 houses flooded for 2.6 million GBP of damage), independently of the management cost (high or low maintenance infrastructures both lead to the same result), and a poor overall farming profitability (−30k GBP/year).
The optimal outcomes found in the game maximize farming profitability (+109k GBP/year) and reduce the number of flooded houses by approximately 80% (20 to 22 houses for 0.68 to 0.8 million GBP of damage depending on the choice for either a high or a low maintenance drainage infrastructure). It corresponds to a mixture of the “Arable with no-till, cover crops, mixed rotation and conservation agriculture” and the “Arable with mixed rotation, cover crops and minimum tillage”. It is notable, that despite the high cost, the high maintenance drainage infrastructure seems to have a negligible influence on the flood impact. Internal testing was carried out prior to the first game event on a beta version by a group of six staff members from the Westcountry Rivers Trusts, comprising farm advisors, fisheries and river management specialists and data/communication officers. The feedback included some suggestions for improving the visual aspects or ‘usability’ of the game, and some comments about improvements to the accuracy or completeness of the model, mainly regarding the type of farming systems available.

2.3. The Experimental Design Used to Assess the Intervention

The Serious Game combines pre- and post-game questionnaires with an intervention where participants are encouraged to collaborate and asked to answer questions in a specific order while using a digital game.
The initial idea was to measure the impact of the Serious Game on the participants from the Millbrook community. However, after the first gaming session took place at the Westcountry Rivers Trusts headquarters in Cornwall in the UK, invitations were sent via email but attendance was poor (2 of 20 residents). We were left with only six valid contributions (with correctly filled forms). Only two of the participants were residents from Millbrook (non-specialists), and the other four were environmental and land management advisors. Other contributors submitted incomplete forms and therefore their participation was disregarded. To overcome this problem we had to get more people to play the game even if not linked to Millbrook, and therefore focus more on the cognitive impact of the game on participants rather than its direct effects on the Millbrook community. The second gaming session took place at the IHE Delft Institute for Water Education in the Netherlands as part of a compulsory introduction to Serious Gaming and had nine valid contributions (eight of the participants were MSc students in Hydrology, and one was a non-specialist). The last gaming session took place in Southampton in the UK, wherein neighbors from the same street were invited to participate using “door-to-door” presentations. This session had seven valid contributions from non-specialists from various backgrounds. The total number of valid participations was 22 of a total of 31 (ten were non-specialists).
Thus, our primary aim was to measure whether there was a statistically significant difference between the pre-game and post-game players’ views on the best solutions to the Millbrook flood problem. More specifically, we aimed to prove that playing the Serious Game induced a statistically significant directional change in the way participants confirmed or rejected seven different “Millbrook flood” hypotheses towards what the hydraulic and cost models of the game point to as the correct rational answers. We created seven different hypotheses (see Table 2) in order to test the limits of the Serious Game, regarding how effective it can be at making such problems accessible to the general public (non-specialists). The first hypothesis set the context to describing the extent of the flood damages caused by a one in a hundred-year event of 45 mm of rainfall in one hour in the village of Millbrook. The next hypotheses explore whether investing in drainage infrastructures or land use are the best strategies to limit risks of flood damage. Finally, the last group of hypotheses use a more specific technical language and focus on characterizing the impact of different farming systems on both flood impact and profitability.
Using a statistical directional test, we aimed to find whether playing the game would induce a change in the way players confirmed or rejected the seven different “Millbrook flood” hypotheses in the pre- and post-game questionnaires. Levels of significance were computed using a Wilcoxon Signed-Ranks One-Tailed Test [32]—see Appendix G for more details—where data are paired (between pre- and post-game answers) and answers are set on an ordinal scale (−2 = strongly disagree, −1 = disagree, 0 = neutral, 1 = agree, 2 = strongly agree). With a total of 22 participants, the standard Z-value based version of the test could not be used, and was replaced by the W-value suitable for sample sizes where N < 30. The null hypothesis was that playing the Serious Game does not change the players’ views. Levels of significance considered were: “Not Significant” (for p > 0.05), “Significant” (for p < 0.05), and “Highly Significant” (for p < 0.01).
We also intended to show to a lesser extent that playing the Serious Game was informative and significantly changed the participants’ views on the solutions to the Millbrook flood mitigation problem. In order to achieve this we sought to demonstrate that (a) there was an increase in the number of correct or relevant numerical values used in the justification of the answers, (b) there was a gain in technical vocabulary, (c) there was an increase in the use of comparative, superlative, and quantitative words that translates into a lexical field more oriented toward showing the degree of influences of factors over the flood and cost outcomes, and (d) there possibly was a growth in confidence and accuracy in the participants’ answers.
In response to the shift in the aim of the paper, showing a real impact on the Millbrook community itself became less important than demonstrating that the Serious Game could induce cognitive change in the participants and therefore be a suitable tool developed in the context of the action research.

3. Results

The differences measured between answers in the pre- and the post-game questionnaires (Appendix A and Appendix C) are used to infer the results as described in the following sections.

3.1. Transformative Nature of the Millbrook Serious Game

Table 2 shows, with the associated statistical level of significance, how playing the game induced a directional change in the way participants confirmed or rejected the seven hypotheses, while Figure 5 and Figure 6 provide radar charts showing, for all participants, where answers to hypotheses (excluding Q5 that is a different type of question) were changed to stronger agreement or disagreement respectively after playing the game.
Figure 7 shows how participants perceived in 70% of cases that playing the game made them revise their views and assumptions on what they initially thought were the causes and best solutions to the Millbrook Flood problem (see supplementary data Table S1 in “tables” table). Note that the “perceived” change seems to match the “measured” changes of views shown in Table 3.
We can report inconclusive effects when trying to select the most important flood factors in Q5, which asked: “what are the two most important factors in this list of three that influence flooding when looking at the ground cover of cultivated areas?” Participants were required to select two of the three following factors in the pre-game and the post-game questionnaire.
  • The capacity of the ground to absorb water
  • The amount of washed sediments that can block drainage pipes in the village downstream
  • The roughness of the terrain that slows the travelling speed of the water
There does not seem to be any significant difference between pre- and post-game questionnaires amongst the participants. After playing the Serious Game, the majority of participants were able to establish that the drainage system was having a limited effect on the flood level for the modeled pluvial scenarios. Using deductive reasoning, participants would then have been expected to decrease the relative importance of “the sediments and drain blockage” factor to stay consistent with their other conclusions. This did not happen. One possible explanation is that the influence of terrain roughness and sediment induced drain blockage are shown in a much subtler way than water absorption capability in the Serious Game model. As these factors were perceived as less salient, the difference in ranking between them was not the subject of the analysis. The participants did not have a prior opinion on this matter, thus they did not have pre-conceptions to revise in relation to it.
Slightly more than half the participants (12 of 22) were experts in either environmental matters or hydrology. Results seem to show that the presence of experts boosts the accuracy of the answers in the very first hypothesis only, and in fact decreases the accuracy of the answers towards neutrality in all other hypotheses, as shown in Table 3. We cannot divide our sample into two groups of size 12 and 10 and apply a Mann-Whitney U Test in order to confirm this hypothesis because the sizes of these groups are too small to achieve statistical significance. Still, simply by showing the change in mean (as opposed to a change in median, which is less meaningful) it allows us to speculate that experts act as dampeners because overall they tend to be more reserved and “neutral” in their attitude. As such, the presence of “moderating” experts amongst our participants suggests that we should obtain even stronger results with a non-specialist audience.

3.2. Informative Nature of the Millbrook Serious Game

Changes induced by the game in terms of use of numerical values, vocabulary, and confidence and accountability are listed here.

3.2.1. The Accuracy and Increasing Use of Numerical Values in Participants’ Answers

Overall, the words written by participants to justify their answers underwent a qualitative change from speculative in tone and unfocused to a greater focus on numerical values learned from the game. In Q1, the word used with the highest frequency in the pre-game responses is “Unknown”. By contrast, in the post-game responses, the word used with the highest frequency becomes “108” which is the exact maximum number of houses flooded in the worst scenario. This shows a better understanding and a more precise quantification of the magnitude of the flood issues.
From Q2 to Q7 the degree of numeracy of the answers—the number of correct or relevant figures used in the justification of the answer—is nil in the pre-game questionnaire (none of the very few numbers written in justifications of these answer were relevant as mostly they were just copied from the questions) and higher in the post-game questionnaire (10 words in 293, approximately 3.4% of words written in justifications of these answer are accurate numbers).

3.2.2. The Acquisition of a Domain Specific Vocabulary and the Increased Utilization of Technically Accurate Words.

Although the size of the lexical field (list of all the words) does not change significantly (pre-game questionnaire uses 278 words to justify their answers while the post-game questionnaire uses 293)—in the post-game questionnaire there are many more words used appropriately, with a high frequency (underlying data can be found in supplementary file in “grid data” tab and in Tables S1–S4 in the “table” tab). In the pre-game answers, “Unknown” (17 times), “neutral” (15 times), “runoff” (7 times), and “reduce” (7 times) are the four most used words. In the post-game answers, “Farming” (17 times), “Impact” (14 times), “Soil” (12 times) and “Infrastructure” (11 times) are the four most encountered words. This indicates that players have acquired a more specialized vocabulary from playing the game and that they more often tend to use domain specific and technical words to justify their answers.
Furthermore, the proportion of words expressing a comparative relationship between two elements (for example: “bigger”, “reduces”, etc.) or a relative importance (“high”, “greatly”, etc.) also changes from 18.3% (51 words out of 278) in the pre-game answers to 29.3% (86 words out of 293) in the post-game answers. This shows a better grasp of the influences of different factors associated with the flood and damages and their relative importance.

3.2.3. The Growth in Confidence and Answer Accountability Induced by the Game.

Due to the fact that justifying their answer was not compulsory during the exercise, 14 participants sometimes chose answers without justifying them in the forms (resulting in one-third of the answers lacking an explanation). In these instances, we cannot say if the participants were able to justify their answers or not. That was only possible if they stayed “neutral” on their agreement/disagreement scale—in which case we concluded that they were not able to justify their answer. Most of the participants that were not able to justify their answers before playing the game were then found able to do so after playing it. This was done in an explicit and technically detailed manner—only 2 of the 28 answers were either wrong or not explicit enough (exact details can be found in supplementary data in “grid data” tab).
When looking specifically at the users that were not able to justify the answers to the questions before playing the game (33.33% of “Unknown”), the overwhelming majority of these players were able to justify their answers after playing the Serious Game (only 1.8% of “Unknown”) (see Figure S1 in “grid data” table in data file for more details). There is only one exception, in questions 5 and 7, where the same user could justify his/her answer in the pre-game questionnaire, but failed to justify his/her answer in the post-game questionnaire—that particular player admitted having lost sight of that question while playing the game.
When directly asked if they learned anything new after playing the game, on average, the participants’ answers were mostly positive, as shown in Figure 8 (Table S2 in “tables” tab in supplementary data shows the complete list of associated answers). A couple of notable exceptions came from a land management advisor and a hydrology MSc student who felt that the game simply confirmed their pre-existing understanding.
Finally, feedback from the participants regarding the suitability of the Serious Game for helping people to understand the Millbrook flood issues was positive overall, as shown in Figure 9 (Table S3 in “tables” tab in supplementary data shows the complete list of associated answers).
When asked why they would recommend the Serious Game, participants’ answers can be divided into three broad categories:
  • It is a good informative tool that allows the user to view and understand the effects of decisions for many different scenarios.
  • It encourages thought and discussion.
  • It is good at changing attitude and promoting positive changes.
Regarding the impact of the Serious Game on the Millbrook community, eleven months after playing the Serious Game, a number of participants have communicated the outputs with other local stakeholders and have initiated the formation of an ongoing natural flood management partnership for action in Millbrook.

4. Discussion and Conclusions

In this section we discuss the cognitive changes induced by the Serious Game on players, the general impact on the Millbrook community, the limitations of this paper, and future work.

4.1. Cognitive Impact of the Serious Game

In this work we have shown that playing the Millbrook Serious Game significantly changed the participants’ views on the Millbrook flood problem. Primarily, we have demonstrated that the game was informative (Section 3.2) because it improved participants’ knowledge of the quantitative effects of flood damage, increased their use of technical vocabulary, improved their understanding of the influences and relative importance of multiple factors affecting the flood and cost outcomes, and caused a growth in confidence and accuracy in their answers. In Section 3.1 we established our most important aim whereby statistically significant changes induced by playing the game were directional and lead the participants to strengthen their confirmation and rejection of specific hypotheses. This also encouraged them to deduce rational and correct conclusions regarding fairly technical and non-trivial decisions for the Millbrook flood problem. In effect, the game briefly and successfully transformed the participants into “citizen scientists” in the purest sense of the term—it led them to use inductive reasoning from data produced during the game (hydraulic and costs) to correctly confirm or reject hypotheses in a rational manner. Feedback from participants even strengthened this claim when they admitted that the game allowed them to revise their views and assumptions on what they initially thought were the causes and best solutions to the Millbrook flood problem.

4.2. Impacts of the Serious Game on the Millbrook Community

Surprisingly, considering that we only had two residents from the Millbrook community playing the game, there nonetheless was a significant impact. These two participants did communicate with other local stakeholders and together have initiated the formation of an ongoing natural flood management partnership for action in Millbrook. The formation of partnerships is hugely beneficial for instigating action on the ground [33] because they encourage collaborative working, the sharing of resources, funds, knowledge and skills, and they can provide local groups with a clearer vision and drive for change. Westcountry Rivers Trust have stated that the Serious Game has helped to strengthen the relationship between Westcountry Rivers Trust and Millbrook residents, parish councilors, academic researchers, and local experts. They also added that this game will be a valuable tool, enabling them to communicate natural flood management concepts to a wide range of individuals across the south west of England. They concluded that the combined effect of the aforementioned benefits has increased the likelihood that natural flood management measures will be implemented in the Millbrook catchment area.

4.3. Limitations and Future Work

Possible limitations of this work regarding the efficiency of the Serious Game as a teaching and decision support tool were also found. The first concerns the barrier of technology as a limiting external factor. One participant’s unfamiliarity with computer technology interfered with the consistency of game play and thus their answers, and so had to be discarded. This experience leads us to suspect that for a mixed audience, another kind of game interface or type may be more appropriate (possibly board games or numeric games with a comprehensive training component).
A second limitation is that some results indicated limits in the understanding of the participants. As shown in Figure S2 in the supplementary data file (“grid data” tab) there were inconclusive effects when participants tested the relative importance of more subtle factors such as “the sediments and drain blockage”, rather than terrain roughness. This is likely a limitation in the depth of the cognitive effects of the Serious Game, possibly due to its relatively short duration. The game has a duration of one hour only, which is probably not enough time for participants to thoroughly investigate these subtler effects among the 432 combinations of flood management strategies. Each of these strategies results in outcomes that probably require more in-depth investigation. Eight of the participants were MSc students in hydrology, and they themselves seemed to have conflicting views on the prevalence of either roughness or sediment-induced drain blockage as the second most important factor. Potentially this could be improved by increasing the duration of the game, introducing more questions, and extending the duration of the exploration phase over multiple sessions.
A third perceived limitation was that the presence of too many participants with a high level of technical expertise might limit the generalization of the results. The Serious Game was originally built for a non-specialist audience—and more specifically for the Millbrook residents. As such, having slightly more than half the participants being experts in either environmental matters or hydrology might be seen as a limitation. At first, it would seem that this would: (a) lessen our ability to demonstrate that the Serious Game improves participants’ knowledge as they are already experts, (b) “boost” the accuracy of answers unnecessarily and, therefore, skew the results towards correct hypothesis confirmation or rejection. As far as we can see, by carefully examining the data, it seems that these reservations are unfounded. First, the fact that we can demonstrate a strong informative value in Section 3.2 despite the presence of experts suggests that the game will work even better with a non-specialist audience. Second, results in Table 3 in Section 3.1 seem to show that the presence of experts only boosts the accuracy of the answers in the very first hypothesis and in fact decreases the accuracy of the answers towards neutrality in all other six hypotheses. As such, the presence of “moderating” experts in the midst of our participants suggests that we should obtain even stronger results with a non-specialist audience. Future work will emphasize testing with a targeted audience sample.
Another limitation, from a flood modelling point of view, is that the model is relatively coarse as we only consider three large agricultural areas and two infrastructure interventions. Future development of the model could incorporate finer spatial units and the ability to model a wider range of measures at specific sites across the catchment (e.g., storage ponds or leaky dams) so as to better support land management and potentially enable farmers participation. This of course would have to be done carefully so as to limit the resulting increase in complexity by selectively emulating key features of water and the possible actions of players in catchments [34].
Whilst the small size of our study limits our ability to generalize these results, the positive outcomes we have observed so far lead us to consider this work as a template applicable to other urban–rural flood case studies throughout the world. Potential new case studies are under consideration and should add to the available information to provide a more complete picture of this novel type of teaching tool and its suitability for improving the understanding of complex issues. We also intend to add control treatment groups in future experiments. Four possible control groups come to mind: a standard workshop without a serious game; a workshop without a serious game but with an intervention that includes a dialogue with participants based on the Socratic method and the validation or rejection of hypotheses; a facilitated workshop with a serious game without the Socratic method-based intervention; and finally, a treatment using another common exiting flood visualization tool.
Finally, the design of the Serious Game presented in this work could scale very well to complex problems with thousands of variables that would require weeks of practice to master—an area of serious gaming that is presently attracting increased interest. In the context of the video game industry, modern strategy games used for pure entertainment have presently reached a staggering level of complexity. Players usually need to practice for days or even weeks to understand how to simultaneously balance dozens of variables and their combined influences on the dynamic of the systems they play. In this respect, the Serious Game described in this paper has been engineered to be a “challenging/utilitarian” exercise that scales well with complex problems that could take long periods of playing time to master. A solution could be to extend the “exploration phase” of the Socratic Method inspired process described in Section 2.1.2 to several weeks if necessary, by asking participants to answer different groups of questions per session, focusing on a specific sub-problem at a time and this way avoid the kind of problems resulting from a broad-scale approach [35]. Although useful, this design does not provide an answer to the open question of motivational affordance, and additional work is needed in this area. We intend to pursue this avenue in the context of the ongoing (2016–2020) SIM4NEXUS research project [36], with a Serious Game that has possibly the most complex underlying model ever attempted.

Supplementary Materials

The following are available online at https://www.mdpi.com/2073-4441/10/12/1885/s1. Table S1: Average response to Q10: Has this Serious Game changed any previous view you had on the Millbrook problem? Table S2: Average response to Q9: Did you learn anything new today? Table S3: Average response to Q11: Would you recommend this type of Serious Game for other people involved in Millbrook flood problem? Table S4: Average response to Q1: A one in-a-hundred years rainfall event of 45mm of water in one hour could cause serious flood problems in Millbrook, Figure S1: Justifying answers before and after playing the game, Figure S2: Prioritization of flood factors.

Author Contributions

Conceptualization, M.K. and D.S.; Investigation, M.K., H.L. and S.W.; Methodology, M.K. and D.S.; Software, M.K., M.J.G. and A.S.C.; Supervision, D.S. and L.V.-L.; Validation, M.K., M.J.G., A.S.C., L.V.-L., H.L. and S.W.; Visualisation, M.K.; Writing—original draft, M.K., M.J.G., D.S., H.L. and S.W.; Writing—review and editing, M.K., D.S., A.S.C., L.V.-L., H.L. and S.W.

Funding

The authors would like to acknowledge the funding provided by the UK Engineering and Physical Sciences Research Council, grant EP/M018865/1 (The Nexus Game), the ongoing EC H2020 EU-CIRCLE (GA 653824) project, and the ongoing e European Union’s Horizon 2020 research and innovation programme under Grant Agreement No. 689,150 SIM4NEXUS. Harry Langford acknowledges his knowledge exchange funding from the N8 AgriFood Resilience programme.

Acknowledgments

Many thanks to the Westcountry Rivers Trust for sharing data and organizing the first Serious Game meeting with Millbrook stakeholders.

Conflicts of Interest

The authors declare no conflict of interest. The founding sponsors had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, and in the decision to publish the results.

Appendix A

Millbrook Serious Game Questionnaire 1
University of Exeter—Westcountry Rivers Trust
Privacy Statement: Please complete this questionnaire. Any concerns can be communicated to Mehdi Khoury ([email protected]) at University of Exeter. Thank you for your time and cooperation.
Answer the following questions by circling the most appropriate answer.
1.A one in-a-hundred years rainfall event of 45 mm of water in one hour could cause serious flood problems in Millbrook.
Strongly Disagree   Disagree   Neutral   Agree   Strongly Agree
How many residences could potentially be flooded? Any worst-case cost approximation?

2.Improving the village sewer infrastructure that directly drains excess water could result in neutralising up to 90% of the flood resulting from such an event.
Strongly Disagree   Disagree   Neutral   Agree   Strongly Agree
Can you justify your answer?

3.Improving farming practices that change the ground cover of the cultivated areas around the village could result in neutralising up to 90% of the flood resulting from such an event.
Strongly Disagree   Disagree   Neutral   Agree   Strongly Agree
Can you justify your answer?

4.Investing money in the sewer infrastructure gives a better result for your money when fighting against flood than trying to support different farming practice.
Strongly Disagree   Disagree   Neutral   Agree   Strongly Agree
Can you justify your answer?

5.What are the two most important factors in this list of three that influence flooding when looking at the ground cover of cultivated areas?
- 
the capacity of the ground to absorb water
- 
the amount of washed sediments that can block drainage pipes in the village downstream
- 
the roughness of the terrain the slows the travelling speed of the water
Can you justify your answer?

6.Heavily grazed permanent pasture increases greatly the capacity of the soil to absorb water.
Strongly Disagree   Disagree   Neutral   Agree   Strongly Agree
Can you justify your answer?

7.Growing crops while minimising tillage increases greatly the capacity of the soil to absorb water.
Strongly Disagree   Disagree   Neutral   Agree   Strongly Agree
Can you justify your answer?

8.Intensive farming that compacts the ground and lowers the capacity of the soil to absorb water, remains the most profiTable way to exploit a cultivated area.
Strongly Disagree   Disagree   Neutral   Agree   Strongly Agree
Can you justify your answer?

Appendix B

Questions
1- 
What is the combination of measures that leads to the worst flood outcome? Also note the damage/cost and number of flood houses.
2- 
What is the combination of measures that leads to the best flood outcome? Also note the damage/cost and number of flood houses.
3- 
What is the combination of measures that leads to the worst financial outcome from both the infrastructure cost and the farming profitability?
4- 
What is the combination of measures that leads to the best financial outcome from both the infrastructure cost and the farming profitability?
5- 
Is there a satisfying solution that manages to minimise the flood and bring an acceptable financial outcome?

Appendix C

Millbrook Serious Game Questionnaire 2
University of Exeter—Westcountry Rivers Trust
Privacy Statement
Please complete this questionnaire. Any concerns can be communicated to Mehdi Khoury ([email protected]) at University of Exeter. Thank you for your time and cooperation.
Answer the following questions by circling the most appropriate answer.
1.A one in-a-hundred years rainfall event of 45 mm of water in one hour could cause serious flood problems in Millbrook.
Strongly Disagree   Disagree   Neutral   Agree   Strongly Agree
How many residences could potentially be flooded? Any worst-case cost approximation?

2.Improving the village sewer infrastructure that directly drains excess water could result in neutralising up to 90% of the flood resulting from such an event.
Strongly Disagree   Disagree   Neutral   Agree   Strongly Agree
Can you justify your answer?

3.Improving farming practices that change the ground cover of the cultivated areas around the village could result in neutralising up to 90% of the flood resulting from such an event.
Strongly Disagree   Disagree   Neutral   Agree   Strongly Agree
Can you justify your answer?

4.Investing money in the sewer infrastructure gives a better result for your money when fighting against flood than trying to support different farming practice.
Strongly Disagree   Disagree   Neutral   Agree   Strongly Agree
Can you justify your answer?

5.What are the two most important factors in this list of three that influence flooding when looking at the ground cover of cultivated areas?
- 
the amount of washed sediments that can block drainage pipes in the village downstream
- 
the capacity of the ground to absorb water
- 
the roughness of the terrain the slows the travelling speed of the water
Can you justify your answer?

6.Heavily grazed permanent pasture increases greatly the capacity of the soil to absorb water.
Strongly Disagree   Disagree   Neutral   Agree   Strongly Agree
Can you justify your answer?

7.Growing crops while minimising tillage increases greatly the capacity of the soil to absorb water.
Strongly Disagree   Disagree   Neutral   Agree   Strongly Agree
Can you justify your answer?

8.Intensive farming that compacts the ground and lowers the capacity of the soil to absorb water, remains the most profiTable way to exploit a cultivated area.
Strongly Disagree   Disagree   Neutral   Agree   Strongly Agree
Can you justify your answer?

9.Did you learn anything new today?
Strongly Disagree   Disagree   Neutral   Agree   Strongly Agree
If so, what was news to you?

10.Has this Serious Game changed any previous view you had on the Millbrook problem?
Strongly Disagree   Disagree   Neutral   Agree   Strongly Agree
If so, what was the view that was changed?

11.Would you recommend this type of Serious Game for other people involved in Millbrook flood problem?
Strongly Disagree   Disagree   Neutral   Agree   Strongly Agree
If so, why?

Appendix D. Soil Science Review behind Chosen Farming Systems Values and Parameters in the Game

1: Overview of infiltration and overland flows across different farming practice scenarios
Soil infiltration rates are fundamentally affected by soil type and specifically the texture of the soil (i.e., the ratio of sand, silt and clay) [37]. Beyond this, there are several positive impactors that can contribute to improved infiltration rate and capacity, including: crop diversity, residue maintenance, cover cropping, managed traffic (minimizing area compacted by heavy vehicles), increased organic matter content, conservation agriculture, contour farming and increased earthworm content. Conversely, some practices and outcomes have negative effects on infiltration rate and capacity, including: surface crusting, low organic matter content, compaction, high soil moisture (waterlogging) and winter break rotations.
Infiltration rate is often estimated theoretically via such standards as the Richards equation [32] or the Green and Ampt [38] method. In these methods, the existing saturation of the soil and the porosity allow for such estimation, though they often assume a homogeneous soil. Empirically, infiltration rate data is often measured in isolation, or for simple scientific comparison between two system types (e.g., organic versus conventional); there is a paucity of data contrasting the variable farm practices across a region or watershed, measuring infiltration dynamics over time, and accounting for differing soil profile compositions. As such, to compile representative values, it is necessary to interrogate the literature for suitable data with which to set generalized values appropriate for a Serious Game.
The soils in the Millbrook watershed belong to Soils cape 6 [39] (a freely draining, slightly acid loam). These shale-derived sandy loam soils are of below average fertility, with neutral to acid pastureland dominating their use. Their carbon stocks are typically low and they are prone to both soil erosion and compaction.
In conventional tilled fields in the UK, infiltration rates on sandy loam soils typically vary between 15–45 mm/h, though compaction by agricultural machinery increases bulk density, reduces porosity and rapidly reduces infiltration rates to 3 mm/h [40,41,42,43,44]. With these values in mind, and considering the likely bare infiltration rates of Soils cape 6 soils (10–20 mm/h), it is considered unlikely that arable soils in the Millbrook watershed will have an infiltration rate greater than 30 mm/h, unless conservation agriculture techniques are employed. Similarly, for grazed pasture soils, infiltration rates typically fall between 3–36 mm/h [45,46], with a mean of 9 mm/h for permanent pasture, as low as 0.1 mm/h with heavy grazing pressure. As such, it is considered unlikely that permanent pasture in the Millbrook watershed, unless regular leys or mechanical improvement have been incorporated into the management strategy, will have an infiltration rate greater than 30 mm/h.
With the above summarized data in mind, the following values (mm/h) were chosen: red (heavily grazed permanent pasture) = 10, orange (tilled arable with standard rotation and traffic) = 15, yellow (young pasture with mechanical improvement and moderate grazing) = 30, aqua (tilled arable with diverse rotation and increased organic soil amendment) = 38.
Cover crops and crop residues increase infiltration rates by breaking the kinetic energy of rainfall, increasing residence time on the soil surface, and increasing the prevalence of root holes and thus heterogeneous porosity. Data on the effects of these vegetation types is sparse, though relevant literature suggests that cover crops and crop residues can increase infiltration rates up to at least 70 mm/h [42,47]. As such, a value for farming systems (blue) in which cover crops and minimum-tillage contribute to permanent cover was set at 70 mm/h. Beyond this, a wholesale shift to a no-till, conservation agriculture approach that built soil carbon, maintained soil cover, and minimized traffic could conceivably increase infiltration rates in a similar vain to a lightly, trafficked organic field, i.e., 6–10 times greater [40].
As such, for the purpose of this Serious Game, we have settled on 90 mm/h (6× that of conventional tillage) as the fastest infiltration rate possible for this farm system.
2: Sediment fluxes
The principal factors affecting soil erosion are: (a) the amount and intensity of rainfall and wind, (b) topography, (c) soil properties, and (d) natural and managed vegetation resulting from farming practice. Positive impactors, such as the presence of year-round ground cover, high organic matter contents and good aggregate stability, protect against soil erosion, whilst negative impactors, such as surface crusting, compaction, and winter break crop rotations, promote soil erosion [40]. The largest factors contributing to erosion were found in [48] to be crop cover and valley features, with erosive rainfall events typically at >10 mm d−1 and erosion dominating bare soil environments. Beyond 15% crop cover, erosion was more typically associated with tramlines and wheel tracks and exacerbated by landscape features. Cropland soils often erode at a higher rate than comparable grassland soils, due to their reduced ground cover, with [48] finding cropland erosion rates to be between 0.5–4× greater than grassland erosion rates. The median rate of soil erosion in the UK is approximately 1.3 tones/ha, with higher mean rates (>2.6 tones/ha) being associated with sandy or silty soils ([49] and references therein). Sediment losses vary substantially with crop type and agricultural management practice, though research does suggest that traffic and tillage both increase sediment losses [47,49].
As such, for the Serious Game, greater sediment loss rates were forecast from arable fields, with changes in management practice towards less intense, conservation agriculture approaches with greater ground cover yielding lower sediment losses. We present these as a percentage of the sediment loss needed to block the local drainage systems, from 10% to 90%. Note that at particular times, i.e., just after tillage in a bare arable field, the sediment loss has the potential to be both very substantial and enough to block 100% of the local drainage system.
3: Farm cost estimates
Table A1 presents a summary of the profitability for each of the three sub-regions of the Millbrook catchment, and estimated for a pre-Brexit scenario for the year 2016–2017 using the online calculator in DEFRA’s Farm Business Survey 2016/17 [50]. Profitability for a given area is computed by multiplying either the high or the average net profitability per hectare—respectively 628.7 GBP/Hectare and 283.9 GBP/Hectare—by 85% of each region surface so as to take into account the fact that a small portion of the said “cultivated areas” around the village is not being exploited. The survey provided recent average business incomes for different types of farming.
Table A1. Profitability (in GBP) of each region in the catchment around Millbrook.
Table A1. Profitability (in GBP) of each region in the catchment around Millbrook.
Profitability ClassRegion 1
(75 Hectares)
Region 2
(123 Hectares)
Region 3
(101 Hectares)
high profit (++)40,50265,96154,476
average profit (+)18,28929,78524,599
null000
average loss (−)−18,289−29,785−24,599
high loss (−−)−40,502−65,961−54,476
When defining profitability for the Serious Game, the fact that the average cereal farm profitability was more than double the average profitability of a livestock farm determined the basic profitability scores. The average mixed farm was somewhere in between, with ‘general cropping’ (i.e., diversifying the rotation) usually leading to improved profitability. These incomes are inclusive of single farm and agri-environment payments, with the majority of farms making a loss on agricultural income alone. Compounding factors, such as the potential for longer ‘overwintering’ periods in certain livestock farming approaches, the profitability of grass leys, reduced fuel costs in minimum tillage approaches, and the profitability of diversifying the rotation, were all considered in making subtle adjustments to the in-game profitability scores.

Appendix E. The Level of Investment in the Drainage Infrastructure

We have excluded lining (involving the installation of an internal resin layer that bonds to the inside of the pipe, essentially creating a pipe within a pipe), as data on this specific procedure were not readily available. The formula applied for estimating the management costs is confidential (South West Water). However, the distribution of pipe diameters by total length used to compute these costs is presented in Table A2. It should be pointed out that any pipes of unknown diameter were assigned an estimated diameter of 150 mm for the calculations.
Table A2. Total length of pipes under 225 mm, 225 to 450 mm, and larger than 450 mm.
Table A2. Total length of pipes under 225 mm, 225 to 450 mm, and larger than 450 mm.
Pipe diameter (mm)100125150225300350380400450600unknown
Total Length (m)312817454832242427150545718221383328
Category<225 mm225–450 mm≥450 mm
Total Length (m)567764269603328

Appendix F. Computational Complexity of the Hydraulic Model and Compromises Made

A 1 m resolution cell size took approximately two hours to complete, while a 2 m resolution model took approximately 1 h. Reducing the resolution to 5 m still yield a processing time of approximately 1.5–5 min, and the 10 m resolution was completed in less than 10 s with a recent PC with a Nvidia GTX 1070 graphic card. A compromise of spatial quality over processing time is sought by selecting a resolution of 2.5 m, as the loss of precision between 2 m and 2.5 m is not really noticeable from a player point of view.
The simulation was run for twelve hours of simulation time to allow the flow to propagate after the rain events. The open boundary condition was set to allow flow to escape the terrain. The velocity between two neighboring cells with a water level difference less than 0.1 mm was assumed as zero to speed up the calculation for insignificant quantities of flows.
Different types of surfaces shown in in were associated with an appropriate infiltration rate and roughness as shown Table A3.
Table A3. Types of terrain surfaces with associated infiltration rate and roughness values.
Table A3. Types of terrain surfaces with associated infiltration rate and roughness values.
Surface TypeForestCultivated AreaUrbanUrban DrainageRoad
Infiltration rate (mm/h)120 mm/hFrom 10 to 90 mm/h—see farming systems Table 220.25 mm/hSee Equations (1) and (2) in Section 2.1.14.5 mm/h
Roughness (Manning coef)0.75From 0.3 to 0.5 Manning coef—See farming systems Table 20.250.250.05
The flood damage evaluation uses the residential sector average damage curve for short duration floods (less than 12 h) from the Multi-Colored Manual [51,52]. We simplify calculations as followed to compute damages from flood depth. Due to the average slope of the terrain and history of floods, house ground floors were slightly elevated. Assuming that the floor of each house starts 15 cm above the ground level, we subtract 15 cm from the flood depth to get the water depth measured inside the house to use it as a proxy for residential damage as plotted on the curve. Then we consider two possible levels of flood damage per building. Any flood depth between 0.3 m and 0.5 m will be considered inside a building as a 0.15 m water depth damage of 314 GBP/m2 and any flood beyond 0.5 m will be considered inside a building as a 0.35 m water depth damage of 476 GBP/m2. This simplification allows us to limit the complexity of the game visualization to two levels of flood damage while also the over-estimation that would result from not taking into account ground floor elevation. The data regarding the surface and location of each building were provided by The Westcountry Rivers Trust.

Appendix G. Detail of Wilcoxon Signed-Ranks One-Tailed Test computations

To prove that the Serious Game induces a directional change in the way participants confirm or reject each hypothesis, we calculate the level of significance as follows. We rank the differences for each pair, and affix a sign to each rank. From the sum of positive ranks W+ and the sum of negative ranks W− we compute the W test value such that W = min {W+, W−}. The number of instances where the subject’s difference score is zero is then deducted from the sample size N. The critical value T at p = 0.05 and p = 0.01 for a one-tail test is then given by the test as provided in [31], and if W ≤ T, the null hypothesis (being that that the median difference is zero) can be rejected. Details of the results are shown below for all seven hypotheses.
  • For Question 1, the W-value is 29. The critical value of W for N = 15 at p ≤ 0.05 is 30. Therefore, the result is significant at p ≤ 0.05.
  • For Question 2, the W-value is 16. The critical value of W for N = 18 at p ≤ 0.01 is 32. Therefore, the result is significant at p ≤ 0.01.
  • For Question 3, the W-value is 31. The critical value of W for N = 18 at p ≤ 0.01 is 32. Therefore, the result is significant at p ≤ 0.01.
  • For Question 4, the W-value is 6.5. The critical value of W for N = 17 at p ≤ 0.01 is 27. Therefore, the result is significant at p ≤ 0.01.
  • For Question 6, the W-value is 16.5. The critical value of W for N = 19 at p ≤ 0.01 is 37. Therefore, the result is significant at p ≤ 0.01.
  • For Question 7, the W-value is 0. The critical value of W for N = 11 at p ≤ 0.01 is 7. Therefore, the result is significant at p ≤ 0.01.
  • For Question 8, the W-value is 12.5. The critical value of W for N = 13 at p ≤ 0.05 is 21. Therefore, the result is significant at p ≤ 0.05.

References

  1. Alfieri, L.; Dottori, F.; Betts, R.; Salamon, P.; Feyen, L. Multi-Model Projections of River Flood Risk in Europe under Global Warming. Climate 2018, 6, 6. [Google Scholar] [CrossRef]
  2. Kerbs, J.; Hall, J.; Johnson, A. UK Climate Change Risk Assessment 2017-Synthesis Report: Priorities for the next Five Years; UK Committee on Climate Change: London, UK, 2016. [Google Scholar]
  3. UK Environment Agency. Flood and Coastal Erosion Risk Management Long-term Investment Scenarios (LTIS) 2014; UK Environment Agency: Bristol, UK, 2014.
  4. Palmer, R.N.; Cardwell, H.E.; Lorie, M.A.; Werick, W. Disciplined planning, structured participation, and collaborative modeling—Applying shared vision planning to water resources. J. Am. Water Resour. Assoc. 2013, 49, 614–628. [Google Scholar] [CrossRef]
  5. Savic, D.A.; Morley, M.S.; Khoury, M. Serious Gaming for water system planning and management. Water 2016, 8, 456. [Google Scholar] [CrossRef]
  6. Aubert, A.H.; Bauer, R.; Lienert, J. A review of water-related serious games to specify use in environmental Multi-Criteria Decision Analysis. Environ. Modell. Softw. 2018, 105, 64–78. [Google Scholar] [CrossRef]
  7. Rebolledo-Mendez, G.; Avramides, K.; De Freitas, S.; Memarzia, K. Societal impact of a serious game on raising public awareness: The case of FloodSim. In Proceedings of the 2009 ACM SIGGRAPH Symposium on Video Games, New Orleans, LA, USA, 4–6 August 2009; pp. 15–22. [Google Scholar]
  8. Aqua Republica. Available online: http://aquarepublica.com/ (accessed on 10 October 2016).
  9. Meera, P.; McLain, M.L.; Bijlani, K.; Jayakrishnan, R.; Rao, B.R. Serious game on flood risk management. In Emerging Research in Computing, Information, Communication and Applications; Springer: New York, NY, USA, 2016; pp. 197–206. [Google Scholar]
  10. Douven, W.; Mul, M.L.; Son, L.; Bakker, N.; Radosevich, G.; Hendriks, A. Games to create awareness and design policies for transboundary cooperation in river basins: Lessons from the shariva game of the Mekong river commission. Water Resour. Manag. 2014, 28, 1431–1447. [Google Scholar] [CrossRef]
  11. Valkering, P.; van der Brugge, R.; Offermans, A.; Haasnoot, M.; Vreugdenhil, H. A perspective-based simulation game to explore future pathways of a water-society system under climate change. Simul. Gaming 2013, 44, 366–390. [Google Scholar] [CrossRef]
  12. Van der Wal, M.M.; de Kraker, J.; Kroeze, C.; Kirschner, P.A.; Valkering, P. Can computer models be used for social learning? A Serious Game in water management. Environ. Model. Softw. 2016, 75, 119–132. [Google Scholar] [CrossRef]
  13. Haasnoot, M.; Middelkoop, H.; van Beek, E.; van Deursen, W.P.A. A method to develop sustainable water management strategies for an uncertain future. Sustain. Dev. 2011, 19, 369–381. [Google Scholar] [CrossRef]
  14. Stefanska, J.; Magnuszewski, P.; Sendzimir, J.; Romaniuk, P.; Taillieu, T.; Dubel, A.; Flachner, Z.; Balogh, P. A gaming exercise to explore problem-solving versus relational activities for river floodplain management. Environ. Policy Gov. 2011, 21, 454–471. [Google Scholar] [CrossRef]
  15. Seibert, J.; Vis, M.J.P. Irrigania—A web-based game about sharing water resources. Hydrol. Earth Syst. Sci. 2012, 16, 2523–2530. [Google Scholar] [CrossRef]
  16. Bassi, A.M.; de Rego, F.; Harrisson, J.; Lombardi, N. WATERSTORY ILE: A systemic approach to solve a long-lasting and far-reaching problem. Simul. Gaming 2015, 46, 404–429. [Google Scholar] [CrossRef]
  17. De Kleermaeker, S.; Zijderveld, A.; Thonus, B. Training for crisis response with serious games based on early warning systems. In Proceedings of the 8th International ISCRAM Conference, Lisbon, Portugal, 8–11 May 2011. [Google Scholar]
  18. De Kleermaeker, S.; Arentz, L. Serious gaming in training for crisis response. In Proceedings of the 9th International ISCRAM Conference, Vancouver, BC, Canada, 22–25 April 2012. [Google Scholar]
  19. Leskens, J.; Kehl, C.; Tutenel, T.; Kol, T.; Haan, G.; Stelling, G.; Eisemann, E. An interactive simulation and visualization tool for flood analysis usable for practitioners. Mitig. Adapt. Strateg. Glob. Chang. 2015, 22, 307–324. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  20. Kemec, S.; Zlatanova, S.; Duzgun, H. A Framework for Defining a 3D Model in Support of Risk Management. Geogr. Inf. Cartogr. Risk Cris. Manag. 2010, 69–82. [Google Scholar] [CrossRef]
  21. Mioc, D.; Anton, F.; Nickerson, B.; Santos, M.; Adda, P.; Tienaah, T.; Ahmad, A.; Mezouagh, M.; MacGillivray, M. Flood progression modelling and impact analysis. 2011. Available online: http://orbit.dtu.dk/files/51177341/InTech1.pdf (accessed on 13 June 2018).
  22. Swiss Federal Geoportal. Available online: https://map.geo.admin.ch (accessed on 14 June 2018).
  23. Hunnicutt, B.K. Leisure and play in Plato’s teaching and philosophy of learning. Leisure Sci. 1990, 12, 211–227. [Google Scholar] [CrossRef]
  24. “Unity®” (or Unity3d) Software. Available online: https://unity3d.com (accessed on 11 March 2018).
  25. Qureshi, A.; Rizvi, F.; Syed, A.; Shahid, A.; Manzoor, H. The method of loci as a mnemonic device to facilitate learning in endocrinology leads to improvement in student performance as measured by assessments. Adv. Physiol. Educ. 2014, 38, 140–144. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  26. Khoury, M.; Chen, A.; Gibson, M.; Vamvakeridou-Lyroudia, L.; Stewart, D.; Wood, M.; Savic, D.; Djordjevic, S. A Serious Game to explore different flooding scenarios and their respective effects on infrastructures. In Proceedings of the 13th International Conference on Hydroinformatics, Palermo, Italy, 1–6 July 2018. [Google Scholar]
  27. Lammers, K. Unity Shaders and Effects Cookbook; Packt Publishing: Birmingham, UK, 2013. [Google Scholar]
  28. Guidolin, M.; Chen, A.S.; Ghimire, B.; Keedwell, E.C.; Djordjević, S.; Savić, D.A. A weighted cellular automata 2D inundation model for rapid flood analysis. Environ. Modell. Softw. 2016, 84, 378–394. [Google Scholar] [CrossRef]
  29. CADDIES Framework University of Exete. 2016. Available online: emps.exeter.ac.uk/engineering/research/cws/resources/caddies-framework/ (accessed on 25 August 2018).
  30. Gibson, M.; Savic, D.; Djordjevic, S.; Chen, A.; Fraser, S.; Watson, T. Accuracy and computational efficiency of 2D urban surface flood modelling based on cellular automata. Procedia Eng. 2016, 154, 801–810. [Google Scholar] [CrossRef]
  31. Online Calculator Used for the Wilcoxon Signed-Ranks One-Tailed Test: “Social Science Statistics”. Available online: https://www.socscistatistics.com/tests/signedranks/Default2.aspx (accessed on 25 August 2018).
  32. Lankford, B.; Watson, D. Metaphor in natural resource gaming: Insights from the river basin game. Simul. Gaming 2007, 38, 421–442. [Google Scholar] [CrossRef]
  33. Van Huijstee, M.; Francken, M.; Leroy, P. Partnerships for sustainable development: A review of current literature. Environ. Sci. 2007, 4, 75–89. [Google Scholar] [CrossRef]
  34. Harteveld, C. Triadic Game Design: Balancing Reality, Meaning and Play; Springer: London, UK, 2011. [Google Scholar]
  35. Sušnik, J.; Chew, C.; Domingo, X.; Mereu, S.; Trabucco, A.; Evans, B.; Vamvakeridou-Lyroudia, L.; Savić, D.A.; Laspidou, C.; Brouwer, F. Multi-Stakeholder Development of a Serious Game to Explore the Water-Energy-Food-Land-Climate Nexus: The SIM4NEXUS Approach. Water 2018, 10, 139. [Google Scholar] [CrossRef]
  36. Foth, H.D. Fundamentals of Soil Science, 8th ed.; Wiley: New York, NY, USA, 1991; ISBN 978-0-471-52279-9. [Google Scholar]
  37. Richards, L.A. Capillary conduction of liquids through porous mediums. Physics 1931, 1, 318–333. [Google Scholar] [CrossRef]
  38. Green, W.H.; Ampt, G. Studies of soil physics. J. Agric. Sci. 1911, 4, 1–24. [Google Scholar] [CrossRef]
  39. Soilscape Soils Types Map. Available online: http://www.landis.org.uk/soilscapes/index.cfm (accessed on 8 September 2018).
  40. Kooistra, M.J.; Bouma, J.; Boersma, O.H.; Jager, A. Physical and morphological characterization of undisturbed and disturbed ploughpans in a sandy loam soil. Soil Tillage Res. 1984, 4, 405–417. [Google Scholar] [CrossRef]
  41. Meek, B.D.; Rechel, E.R.; Carter, L.M.; DeTar, W.R.; Urie, A.L. Infiltration rate of a sandy loam soil: Effects of traffic, tillage, and plant roots. Soil Sci. Soc. Am. J. 1992, 56, 908–913. [Google Scholar] [CrossRef]
  42. Stobart, R.M.; Morris, N.L. The impact of cover crops on yield and soils in the New Farming Systems programme. Asp. Appl. Biol. 2014, 127, 223–232. [Google Scholar]
  43. Chyba, J.; Kroulik, M.; Kristof, K.; Misiewicz, P.A. The influence of agricultural traffic on soil infiltration rates. Agron. Res. 2017, 15, 664–673. [Google Scholar]
  44. Fullen, M.A. Compaction, hydrological processes and soil erosion on loamy sands in east Shropshire, England. Soil Tillage Res. 1985, 6, 17–29. [Google Scholar] [CrossRef]
  45. Hathaway-Jenkins, L.J.; Sakrabani, R.; Pearce, B.; Whitmore, A.P.; Godwin, R.J. A comparison of soil and water properties in organic and conventional farming systems in England. Soil Use Manag. 2011, 27, 133–142. [Google Scholar] [CrossRef]
  46. Slaymaker, O.; Spencer, T.; Embleton-Hamann, C. Geomorphology and Global Environmental Change; Cambridge University Press: New York, NY, USA, 2009. [Google Scholar]
  47. Wang, X.; Gao, H.; Tullberg, J.N.; Li, H.; Kuhn, N.; McHugh, A.D.; Li, Y. Traffic and tillage effects on runoff and soil loss on the Loess Plateau of northern China. Soil Res. 2008, 46, 667–675. [Google Scholar] [CrossRef]
  48. Evans, R.; Collins, A.L.; Foster, I.D.; Rickson, R.J.; Anthony, S.G.; Brewer, T.; Deeks, L.; Newell-Price, J.P.; Truckell, I.G.; Zhang, Y. Extent, frequency and rate of water erosion of arable land in Britain–benefits and challenges for modelling. Soil Use Manag. 2016, 32, 149–161. [Google Scholar] [CrossRef]
  49. Rohde, K.; Yule, D. Soil compaction and controlled traffic farming research in Central Queensland. ISTRO 2003, 16, 1020–1027. [Google Scholar]
  50. Online Farming Profitability Indicators: “Farm Business Survey”. Available online: http://www.farmbusinesssurvey.co.uk/benchmarking/Default.aspx?Module=ProfitResults_ (accessed on 25 August 2017).
  51. Penning-Rowsell, E.; Priest, S.; Parker, D.; Morris, J.; Tunstall, S.; Viavattene, C.; Chatterton, J.; Owen, D. Flood and Coastal Erosion Risk Management: A Manual for Economic Appraisal; Routledge: London, UK, 2013; ISBN 9780415815154. [Google Scholar]
  52. Multi-Colored_Manual Online, Middlesex University. May 2018. Available online: https://www.mcm-online.co.uk/ (accessed on 20 August 2018).
Figure 1. Three parts of the Serious Game process.
Figure 1. Three parts of the Serious Game process.
Water 10 01885 g001
Figure 2. Overview of the 3D virtual table with narrative and interactive elements.
Figure 2. Overview of the 3D virtual table with narrative and interactive elements.
Water 10 01885 g002
Figure 3. Shaders: (a) is a close up with the vertices of the mesh made visible while (b) is an overview of the final result.
Figure 3. Shaders: (a) is a close up with the vertices of the mesh made visible while (b) is an overview of the final result.
Water 10 01885 g003
Figure 4. The user inputs as seen in the Serious Game interface.
Figure 4. The user inputs as seen in the Serious Game interface.
Water 10 01885 g004
Figure 5. Radar charts showing for all participants, where the changes induced by the games for answers to hypothesis reflects a stronger agreement. Each participant answer (−2 = strongly disagree, −1 = disagree, 0 = neutral, 1 = agree, 2 = strongly agree) is rated on one axis in yellow (pre-game) and red (post-game) areas. The questions answered are (a) Q1; (b) Q3; (c) Q7.
Figure 5. Radar charts showing for all participants, where the changes induced by the games for answers to hypothesis reflects a stronger agreement. Each participant answer (−2 = strongly disagree, −1 = disagree, 0 = neutral, 1 = agree, 2 = strongly agree) is rated on one axis in yellow (pre-game) and red (post-game) areas. The questions answered are (a) Q1; (b) Q3; (c) Q7.
Water 10 01885 g005
Figure 6. Radar charts showing for all participants, where the changes induced by the games for answers to hypothesis should reflect a stronger disagreement. Each participant answer (−2 = strongly disagree, −1 = disagree, 0 = neutral, 1 = agree, 2 = strongly agree) is rated on one axis in yellow (pre-game) and red (post-game) areas. The questions answered are (a) Q2; (b) Q4; (c) Q6; (d) Q8.
Figure 6. Radar charts showing for all participants, where the changes induced by the games for answers to hypothesis should reflect a stronger disagreement. Each participant answer (−2 = strongly disagree, −1 = disagree, 0 = neutral, 1 = agree, 2 = strongly agree) is rated on one axis in yellow (pre-game) and red (post-game) areas. The questions answered are (a) Q2; (b) Q4; (c) Q6; (d) Q8.
Water 10 01885 g006
Figure 7. Shows the average score of answers from participants when asked if the game made them revise their views on the problem—range [0,2] and standard deviation 0.71.
Figure 7. Shows the average score of answers from participants when asked if the game made them revise their views on the problem—range [0,2] and standard deviation 0.71.
Water 10 01885 g007
Figure 8. Shows the average score of answers from participants when asked if they have acquired new knowledge from playing the game—range [−1,2] and standard deviation 0.74.
Figure 8. Shows the average score of answers from participants when asked if they have acquired new knowledge from playing the game—range [−1,2] and standard deviation 0.74.
Water 10 01885 g008
Figure 9. The average score from all participants’ answers when asked if they would recommend this type of Serious Game for stakeholders—range [1,2] and standard deviation 0.46.
Figure 9. The average score from all participants’ answers when asked if they would recommend this type of Serious Game for stakeholders—range [1,2] and standard deviation 0.46.
Water 10 01885 g009
Table 1. Farming systems.
Table 1. Farming systems.
TypeInfiltration Rate (mm/h) *Surface Roughness (Manning Coefficient)Drainage Pipe Blockage (%)Profitability (−−, −, 0, +, ++)
Heavily grazed permanent pasture100.340
Intensive arable under tillage and standard rotation150.4900
Young grassland with moderate grazing and mechanical improvement300.3510−−
Arable with mixed rotation and increased soil amendment380.4570+
Arable with mixed rotation, cover crops and minimum tillage700.5050++
Arable with no-till, cover crops, mixed rotation and conservation agriculture900.5020+
* Underlying soil is Soilscape 6 [28] (freely draining, slightly acid loam); as such, a bare soil infiltration rate of between 10–20 mm/h is likely.
Table 2. Playing the game induces a directional change in the way participants confirm or reject the seven hypotheses.
Table 2. Playing the game induces a directional change in the way participants confirm or reject the seven hypotheses.
HypothesisCorrect Answer Directional Change 1Level of Significance 2 of Directional Change
Q1—A one in-a-hundred years rainfall event of 45 mm of water in one hour could cause serious flood problems in Millbrook (valid answer: true)TrueIncreased Agreement Water 10 01885 i001+
(p = 0.05)
Q2—Improving the village sewer infrastructure that directly drains excess water could result in neutralizing up to 90% of the flood resulting from such an eventFalseIncreased Disagreement Water 10 01885 i002++
(p = 0.01)
Q3—Improving farming practices that change the ground cover of the cultivated areas around the village could result in neutralizing up to 90% of the flood resulting from such an eventTrueIncreased Agreement Water 10 01885 i001++
(p = 0.01)
Q4—Investing money in the sewer infrastructure gives a better result for your money when fighting against flood than trying to support different farming practiceFalseIncreased Disagreement Water 10 01885 i002++
(p = 0.01)
Q6—Heavily grazed permanent pasture increases greatly the capacity of the soil to absorb waterFalseIncreased Disagreement Water 10 01885 i002++
(p = 0.01)
Q7—Growing crops while minimizing tillage increases greatly the capacity of the soil to absorb waterTrueIncreased Agreement Water 10 01885 i001++
(p = 0.01)
Q8—Intensive farming that compacts the ground and lowers the capacity of the soil to absorb water, remains the most profitable way to exploit a cultivated areaFalseIncreased Disagreement Water 10 01885 i002+
(p = 0.05)
1 Difference between pre- and post-game questionnaire answers (−2 = strongly disagree, −1 = disagree, 0 = neutral, 1 = agree, 2 = strongly agree). 2 Significance using a Wilcoxon signed rank one-tailed test for paired ordinal data. + means medium significance, and ++ means higher significance.
Table 3. Shows the shifting of average scores as arrows between all the participants’ answers to a hypothesis from being a non-specialist (starting point of the rectangle) to being a domain expert in either environmental science or hydrology (pointing end).
Table 3. Shows the shifting of average scores as arrows between all the participants’ answers to a hypothesis from being a non-specialist (starting point of the rectangle) to being a domain expert in either environmental science or hydrology (pointing end).
QuestionShifting of Average Scores from Non-specialist to Domain Expert
Q1 Water 10 01885 i003
Q2 Water 10 01885 i004
Q3 Water 10 01885 i005
Q4 Water 10 01885 i006
Q6 Water 10 01885 i007
Q7 Water 10 01885 i008
Q8 Water 10 01885 i009

Share and Cite

MDPI and ACS Style

Khoury, M.; Gibson, M.J.; Savic, D.; Chen, A.S.; Vamvakeridou-Lyroudia, L.; Langford, H.; Wigley, S. A Serious Game Designed to Explore and Understand the Complexities of Flood Mitigation Options in Urban–Rural Catchments. Water 2018, 10, 1885. https://doi.org/10.3390/w10121885

AMA Style

Khoury M, Gibson MJ, Savic D, Chen AS, Vamvakeridou-Lyroudia L, Langford H, Wigley S. A Serious Game Designed to Explore and Understand the Complexities of Flood Mitigation Options in Urban–Rural Catchments. Water. 2018; 10(12):1885. https://doi.org/10.3390/w10121885

Chicago/Turabian Style

Khoury, Mehdi, Michael J. Gibson, Dragan Savic, Albert S. Chen, Lydia Vamvakeridou-Lyroudia, Harry Langford, and Sarah Wigley. 2018. "A Serious Game Designed to Explore and Understand the Complexities of Flood Mitigation Options in Urban–Rural Catchments" Water 10, no. 12: 1885. https://doi.org/10.3390/w10121885

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop