A Serious Game Designed to Explore and Understand the Complexities of Flood Mitigation Options in Urban–Rural Catchments

: Flood prevention in mixed urban–rural environments has become a greater concern due to climate change. It is a complex task requiring both efﬁcient management of resources and the involvement of multiple stakeholders from diverse backgrounds. As Serious Games (games used for purposes other than mere entertainment) have emerged as an effective means of engaging stakeholders, this work proposes a new Serious Game applied to ﬂood mitigation in the village of Millbrook in the UK. Results show that the game has both an informative and a transformative effect (statistical signiﬁcance levels from 0.01 to 0.05), improving participants’ understanding of the problem, and helping them to ﬁnd a new and improved approach to ﬂood risk management in Millbrook, with the potential to improve resilience signiﬁcantly. Furthermore, the game successfully transformed participants into “citizen scientists” in the purest sense of the term—it led them to use inductive reasoning from data produced by the game to correctly conﬁrm or reject hypotheses and resulted in more than 70% of the participants revising their initial assumptions. Interestingly, the game instigated the formation of new local partnerships and helped to prioritize the discussion of natural ﬂood management measures in Millbrook Parish Council meetings.


Introduction
Climate change is currently causing a substantial increase in flood risk across the UK [1,2]. High residual risk of flooding will remain, even with the current investment plans on flood defense [3], and hard engineering solutions will not be affordable to protect all areas. In this context, finding other ways to improve resilience to flooding by preventing or mitigating flood impacts becomes a necessity. The complexity of the problem is significant because flood prevention in an urban-rural environment requires both the management of limited resources and the involvement of multiple stakeholders with different perspectives such as residents, farmers, business owners, utilities companies and policy makers.
On the evening of the 30th of September 2016, a workshop involving comprehensive communication and engagement of stakeholders by the Westcountry Rivers Trust took place in the in the center of Millbrook, a small village in Cornwall, UK. Millbrook has had many recent problems caused by large pluvial flood events. The goal was to present causes to the Millbrook flood events as mostly pluvial, and to emphasize land use and natural flood management measures as potential solutions recommended by hydrology reports. Much of the participants' attention ended up focusing on other relevant topics such as the presence of the reservoir in the village and the drainage infrastructure. In that context, the limitations of traditional methods as an effective conduit for translating complex data, modelled outputs and academic research findings, into clear and concise messages that can be understood by a non-technical audience became apparent. As an action research to respond to that problem, the idea of designing a Serious Game that could potentially fulfill that role was then born. This is also in line with the concept of Shared Vision Planning [4], which engages stakeholders in developing and experimenting with interactive simulation models for building consensus [5].
Serious Games (games used for purposes other than mere entertainment), as an effective means of engaging stakeholders, have been previously used for water system planning and management [5,6] with common aims of raising awareness [7][8][9][10][11], facilitating dialogue [11][12][13][14][15][16], or training for crisis response [17,18]. The Serious Game presented in this work aims to fulfill the research action goal in three ways. First, the software developed introduces novel technical elements to make it an effective Virtual Reality tool for visualizing results of simulations of 3D animated floods-a category of the Serious Game visualization system that is missing from the present body of work [6]. While existing work tends to emphasize "step-based" 3D flood visualization [19][20][21][22][23], the tool developed in this work focuses on the real-time deformation of 3D meshes representing volumes of water, with animated shaders allowing the mapping of color gradients to the mesh surface in order to emphasize flood depth. Second, to the best of our knowledge, this is the very first time that a Serious Game combining hydraulic simulation results, expert knowledge about farming systems and soil science, drainage infrastructure management, as well as representative damage and cost estimates have been put together to help players find an efficient (and previously unexplored) solution to the flood resilience problem for a real-world case study. Finally we propose a novel Serious Game intervention design inspired by the Socratic Method [23] and aim to prove that it can (a) enable participants to use inductive reasoning based upon game-derived data (hydraulic and costs) to confirm or reject hypotheses in a rational manner, and (b) encourage the players to question their initial assumptions (perception of the truth) and assist them in shifting towards a more reasoned, scientifically sound, deductive solution, i.e., closer to the evidence ("scientific truth").
From an experimental design point of view, the Serious Game combines pre-and post-game questionnaires with a traditional interpersonal group-based intervention. The intervention is based on asking participants to collaborate and answer questions in a specific order while playing the digital game developed using the Unity ® [24] game engine. It must be emphasized that the approach does not consist solely of the digital game, but also integrates the questionnaires, intervention, and software all together. In this paper, the following materials and methods section presents all the elements used to design the Serious Game. The results are then outlined in Section 3 and their implications are discussed in Section 4.

Materials and Methods
First, we present the design of the Serious Game as an intervention specifically focused on enabling non-scientists to understand complex scientific models. Second, we present the choices made during the game development to produce a Virtual Reality tool suitable for visualizing results of simulations of 3D animated floods. Third, we present the experimental design composed of pre-and post-game questionnaires and used to assess the intervention based on this Serious Game.

The Overall Design of the Intervention
Using criteria extracted from [5], the game design can be characterized as follows. The primary aim or outcome of the game is for the user to explore different flood mitigation actions and corresponding outcomes in order to answer five questions. Finding the answers to these questions leads to the discovery of the optimal flood outcome to the given problem while minimizing costs. During the initialization of the game, a facilitator presents the Millbrook flood problem and provides contextual information regarding the history of flooding in the village, explanations regarding how to use the interface as well as technical concepts such as terrain roughness, and the different farming systems that the players can choose from. Although the game is built as a single player experience, several participants can play it in parallel in the same room. Communication between participants is not compulsory, but is encouraged so that they can freely discuss their choice with their neighbors when answering questions. Players do not have a limited and differentiated role, and they are each encouraged to explore the whole range of possible flood mitigation choices during playtime. The user interface makes use of sophisticated 3D video technology comparable to that already available in the videos game industry so that the user is able to visualize results of simulations via a Virtual Reality tool. The simulation model used relies on cellular automata and a series of conditional statements. The level of realism of the game can be seen as moderate when considering its individual aspects, such as costs and hydraulics models, in isolation. When looking at all of them combined, the level of sophistication of the socio-technical-environmental system presented is significant. Performance feedback during gameplay is instant as the player is immediately informed of the consequences of their actions on the flood and cost outcomes. Progress monitoring, such as the "capability of saving intermediate game results for follow up analysis" [5] is ignored as the user can switch at any time the combination of measures they wish to explore. Game portability is good because it can be played off-line on most Windows computers, though we have limited this portability by delivering the Serious Game intervention in a supervised environment. The game is designed to be a challenging, utilitarian, and demanding exercise.
To fulfill our aim to help participants to explore and understand complex scientific models, the Serious Game intervention is designed using three steps as shown in Figure 1.

The Introductory Phase
Participants are given a general introduction (see link to additional materials at the end) on the Millbrook case study, and then asked to fill in a pre-game questionnaire (Appendix A) in which they need to confirm or validate different hypotheses, for the following reasons:

•
The evaluation of any eventual cognitive change brought about by the Serious Game. This allows us to understand the pre-game state of the initial participants' views on the Millbrook flood problem. This includes possible pre-formed opinions (perception of the truth) on what would be the best possible decisions to achieve the balance between flood prevention and cost.
• It alerts participants to the crucial questions that they need to keep in mind while they improve their understanding of the problem, and therefore helps them to consolidate memories on technical issues, and to validate or reject hypotheses at the end of their investigative work.
• It creates a Serious Game equivalent of the questioning phase of the Socratic Method. The only difference is that instead of it emerging naturally through dialogue, the hypotheses are artificially created by the mix of introductory materials and the pre-game questionnaire.

The Exploratory Phase
During the exploratory phase of the game, players must interrogate the possible solutions and answer questions (Appendix B). The first two questions focus on finding the combinations of factors leading to the worst and best flood damage outcomes, e.g., "What is the combination of measures that leads to the worst flood outcome? Also note the damage/cost and the number of flooded houses." The next two questions focus on finding combinations leading to outcomes with the lowest and highest associated management costs. The last question is about finding an optimal solution that minimizes both flood and cost. During this phase, the digital game provides the players with the freedom to apply their own management decisions. It gives them feedback in the form of simulation results, which they can use to check the validity of the given hypotheses.
The use of compulsory ordered questions is essential because: • It reduces an overwhelmingly large combination of solutions into a manageable list of five tasks that are essentially guiding participants' explorations of the different solution scenarios.
• It forces participants to answer questions where the solutions shown by the game might contradict what they initially thought were the best decisions, hence targeting preconceptions. For example, the exploration of answers to question 2: "What is the combination of measures that leads to the best flood outcome?" leads to the discovery that changing the farming systems around the village has a much greater impact than changing the level of investment in the drainage infrastructure. This corresponds to the Serious Game equivalent of the disproof and refutation stage of the Socratic method.

The Final Phase
Finally, in the third phase, players are asked to capitalize on the previous explorative work by finding the solutions that minimize both damages and management costs. They then have to fill in the post-game questionnaire (Appendix C) and apply inductive reasoning from the data revealed by their exploratory work to reject or confirm the presented hypotheses. Note that players can justify each answer in their questionnaire, giving them the opportunity to reach the Serious Game equivalent of the third step of the Socratic Method, namely to question their initial assumptions ("perception of the truth") and replace them with what they deduced to be a more scientifically sound answer ("the truth").

Game Development Choices
Millbrook has had many recent problems caused by large pluvial flood events. A 2 km square terrain around the village was selected, and uniform rain input of 45 mm/h for the first hour of simulation was applied, which equates to a 1 in 100-year event. In the game, players can simply change four parameters: the level of drainage infrastructure investment and the type of farming system applied in three instances (one for each sub-catchment in the village). They are then able to see flood impacts and associated management costs in numerous resulting combinations.
A 3D virtual table that comprises different informative documents and post-it notes introducing the case has been constructed as shown in Figure 2 to provide additional information to the users. This innovative and original setting for flood visualization is quite powerful and expandable as follows:

•
This allows a wealth of information to be represented as a spatial construct of 3D meshes on a virtual table of practically infinite size (Figure 2), where the geographic location of each item on the virtual table leads to better recall, in a similar way to mnemonic "visual palaces" [25].
• It can structure this information visually so that the user can see how components are related to each other in the case study, and allows the user to move, rotate, and zoom around components, and finally potentially display geographically distant elements as interconnected infrastructure nodes. For example, a distant power plant that influences the electricity supply of a village can be represented near the village on the virtual plan due to the cause-effect relationship between the two.
• Furthermore, this setting is easily translatable as a Virtual Reality headset-based Serious Game without major code modification.  The Serious Game allows players to change (i.e., make decision about): (a) the level of investment in the drainage infrastructure, and (b) the type of farming system applied in each of the three sub-catchments in the village. They can then see the resulting flood impact and associated management costs. These inputs are integrated in the user interface as shown in Figure 4. The simplicity of the interface is the product of a compromise between computability, accessibility, and realism. Allowing the player to change the type of farming system in every individual field in Millbrook would provide a more realistic setting, but it could lead to a usability problem (manually changing each of the hundred fields before computing the consequences would detract from the games effectiveness because participants would lose sight of the problem while doing so). Furthermore, this would have been computationally impractical because it would require 2 × 6 100 hydraulic modeling results to reflect the flood risk of different management scenarios. The present choice of grouping the fields into three cultivated areas allows the user to set farming systems quickly and explore different consequences of their choices, while also keeping the overall problem in mind. While the spatial resolution for targeting measures is quite low in the current model, it still provides a useful indication of which parts of the catchment provide the greatest constraints or opportunities for land use change and flood risk alleviation. It reduces the number of hydraulic computations to a manageable 432 combinations. Although it might initially look more complicated, the user interface and search space structure is quite simple because it closely follows the infiltration gradient of the farming systems as they are presented to the participants, from dark brown (system with the lowest infiltration rate) to dark green (system with the highest infiltration rate). Participants are therefore expected to find the best solution available and master the problem of minimizing both the flood damage and management cost in under an hour without having to explore all possible choices. Each farming system, as described in Table 1, comes with an associated infiltration rate, surface roughness value, sedimentation rate leading to a percentage of pipe blockage in the village downstream, and profitability (ranging from "−−" for higher loss, "−" for average loss, "0" for null, "+" for average profit, and "++" for higher profit). These farming systems are used as model values in the Serious Game in accordance with the latest available agri-environmental data for the Millbrook catchment and from a thorough review of relevant literature detailed in Appendix D. To communicate effectively the consequences of the decisions made by users, three elements have been included on the interface of the Serious Game as shown in Figure 4.

•
A temporal slider at the bottom allows a user to change the timing during a selected event to see the progression of the flood. Note that the overall speed of the animation can also be paused, slowed, or accelerated. • A high-maintenance (high cost, well maintained) drainage infrastructure that can cope with surface runoff up to 26 mm/h/m 2 in urbanized areas. At the high level of investment scenario, the yearly management cost of the urban drainage infrastructure is evaluated at 406k GBP and includes cleaning 10% of the length of all drainage pipes per year, repairing 3% of total length of pipes, and replacing 1%.
• A low-maintenance (low cost, not well maintained) drainage infrastructure that can discharge surface runoff up to 13 mm/h/m 2 in urbanized areas. At this low level of investment scenario, the yearly management cost of the urban drainage infrastructure is evaluated at 203k GBP and includes cleaning 5% of the length of all drainage pipes per year, repairing 1.5% of total length of pipes, and replacing 0.5%.
Flood depths encoded in images are produced using the WCA 2D flood model [29], a part of the CADDIES [30,31] modelling framework. CADDIES is an open-source framework designed to facilitate the design and deployment of Cellular Automata (CA), specifically for 2D urban flood modelling. It is ideal for fast evaluations of floods [31], and therefore allows the swift computation of multiple simulation results corresponding to various users' choices in the context of serious gaming. In the present work, the flood models were pre-run to produce output images used to make the 3D animated mesh of the flood (more details about computation times in Appendix F).
The Millbrook example presents the users with a high-speed model, where a cell size of 2.5 m was selected for the flood modelling with WCA 2D as a compromise between accuracy and computability to produce the 432 different flood outcomes in a few days. The speed of execution makes it suitable in the context of serious gaming for quickly displaying numerous flood outputs resulting from different combination of parameters.
The existing farming situation on the ground in Millbrook (a mixture of "heavily grazed permanent pasture" and "intensive arable cultivated areas under tillage with standard rotation") leads to the worst possible combination in the game. It has a maximum damage outcome (108 houses flooded for 2.6 million GBP of damage), independently of the management cost (high or low maintenance infrastructures both lead to the same result), and a poor overall farming profitability (−30k GBP/year).
The optimal outcomes found in the game maximize farming profitability (+109k GBP/year) and reduce the number of flooded houses by approximately 80% (20 to 22 houses for 0.68 to 0.8 million GBP of damage depending on the choice for either a high or a low maintenance drainage infrastructure). It corresponds to a mixture of the "Arable with no-till, cover crops, mixed rotation and conservation agriculture" and the "Arable with mixed rotation, cover crops and minimum tillage". It is notable, that despite the high cost, the high maintenance drainage infrastructure seems to have a negligible influence on the flood impact. Internal testing was carried out prior to the first game event on a beta version by a group of six staff members from the Westcountry Rivers Trusts, comprising farm advisors, fisheries and river management specialists and data/communication officers. The feedback included some suggestions for improving the visual aspects or 'usability' of the game, and some comments about improvements to the accuracy or completeness of the model, mainly regarding the type of farming systems available.

The Experimental Design Used to Assess the Intervention
The Serious Game combines pre-and post-game questionnaires with an intervention where participants are encouraged to collaborate and asked to answer questions in a specific order while using a digital game.
The initial idea was to measure the impact of the Serious Game on the participants from the Millbrook community. However, after the first gaming session took place at the Westcountry Rivers Trusts headquarters in Cornwall in the UK, invitations were sent via email but attendance was poor (2 of 20 residents). We were left with only six valid contributions (with correctly filled forms). Only two of the participants were residents from Millbrook (non-specialists), and the other four were environmental and land management advisors. Other contributors submitted incomplete forms and therefore their participation was disregarded. To overcome this problem we had to get more people to play the game even if not linked to Millbrook, and therefore focus more on the cognitive impact of the game on participants rather than its direct effects on the Millbrook community. The second gaming session took place at the IHE Delft Institute for Water Education in the Netherlands as part of a compulsory introduction to Serious Gaming and had nine valid contributions (eight of the participants were MSc students in Hydrology, and one was a non-specialist). The last gaming session took place in Southampton in the UK, wherein neighbors from the same street were invited to participate using "door-to-door" presentations. This session had seven valid contributions from non-specialists from various backgrounds. The total number of valid participations was 22 of a total of 31 (ten were non-specialists).
Thus, our primary aim was to measure whether there was a statistically significant difference between the pre-game and post-game players' views on the best solutions to the Millbrook flood problem. More specifically, we aimed to prove that playing the Serious Game induced a statistically significant directional change in the way participants confirmed or rejected seven different "Millbrook flood" hypotheses towards what the hydraulic and cost models of the game point to as the correct rational answers. We created seven different hypotheses (see Table 2) in order to test the limits of the Serious Game, regarding how effective it can be at making such problems accessible to the general public (non-specialists). The first hypothesis set the context to describing the extent of the flood damages caused by a one in a hundred-year event of 45 mm of rainfall in one hour in the village of Millbrook. The next hypotheses explore whether investing in drainage infrastructures or land use are the best strategies to limit risks of flood damage. Finally, the last group of hypotheses use a more specific technical language and focus on characterizing the impact of different farming systems on both flood impact and profitability.
Using a statistical directional test, we aimed to find whether playing the game would induce a change in the way players confirmed or rejected the seven different "Millbrook flood" hypotheses in the pre-and post-game questionnaires. Levels of significance were computed using a Wilcoxon Signed-Ranks One-Tailed Test [32]-see Appendix G for more details-where data are paired (between pre-and post-game answers) and answers are set on an ordinal scale (−2 = strongly disagree, −1 = disagree, 0 = neutral, 1 = agree, 2 = strongly agree). With a total of 22 participants, the standard Z-value based version of the test could not be used, and was replaced by the W-value suitable for sample sizes where N < 30. The null hypothesis was that playing the Serious Game does not change the players' views. Levels of significance considered were: "Not Significant" (for p > 0.05), "Significant" (for p < 0.05), and "Highly Significant" (for p < 0.01).
We also intended to show to a lesser extent that playing the Serious Game was informative and significantly changed the participants' views on the solutions to the Millbrook flood mitigation problem. In order to achieve this we sought to demonstrate that (a) there was an increase in the number of correct or relevant numerical values used in the justification of the answers, (b) there was a gain in technical vocabulary, (c) there was an increase in the use of comparative, superlative, and quantitative words that translates into a lexical field more oriented toward showing the degree of influences of factors over the flood and cost outcomes, and (d) there possibly was a growth in confidence and accuracy in the participants' answers.
In response to the shift in the aim of the paper, showing a real impact on the Millbrook community itself became less important than demonstrating that the Serious Game could induce cognitive change in the participants and therefore be a suitable tool developed in the context of the action research.

Results
The differences measured between answers in the pre-and the post-game questionnaires (Appendices A and C) are used to infer the results as described in the following sections. Table 2 shows, with the associated statistical level of significance, how playing the game induced a directional change in the way participants confirmed or rejected the seven hypotheses, while Figures 5 and 6 provide radar charts showing, for all participants, where answers to hypotheses (excluding Q5 that is a different type of question) were changed to stronger agreement or disagreement respectively after playing the game.

Transformative Nature of the Millbrook Serious Game
Q8-Intensive farming that compacts the ground and lowers the capacity of the soil to absorb water, remains the most profitable way to exploit a cultivated area

False Increased Disagreement
Disagreeme + (p = 0.05) 1 Difference between pre-and post-game questionnaire answers (−2 = strongly disagree, −1 = disagree, 0 = neutral, 1 = agree, 2 = strongly agree). 2 Significance using a Wilcoxon signed rank one-tailed test for paired ordinal data. + means medium significance, and ++ means higher significance.  Figure 7 shows how participants perceived in 70% of cases that playing the game made them revise their views and assumptions on what they initially thought were the causes and best solutions to the Millbrook Flood problem (see supplementary data Table S1 in "tables" table). Note that the "perceived" change seems to match the "measured" changes of views shown in Table 3. We can report inconclusive effects when trying to select the most important flood factors in Q5, which asked: "what are the two most important factors in this list of three that influence flooding when looking at the ground cover of cultivated areas?" Participants were required to select two of the three following factors in the pre-game and the post-game questionnaire.

•
The capacity of the ground to absorb water • The amount of washed sediments that can block drainage pipes in the village downstream

•
The roughness of the terrain that slows the travelling speed of the water There does not seem to be any significant difference between pre-and post-game questionnaires amongst the participants. After playing the Serious Game, the majority of participants were able to establish that the drainage system was having a limited effect on the flood level for the modeled pluvial scenarios. Using deductive reasoning, participants would then have been expected to decrease the relative importance of "the sediments and drain blockage" factor to stay consistent with their other conclusions. This did not happen. One possible explanation is that the influence of terrain roughness and sediment induced drain blockage are shown in a much subtler way than water absorption capability in the Serious Game model. As these factors were perceived as less salient, the difference in ranking between them was not the subject of the analysis. The participants did not have a prior opinion on this matter, thus they did not have pre-conceptions to revise in relation to it. Table 3. Shows the shifting of average scores as arrows between all the participants' answers to a hypothesis from being a non-specialist (starting point of the rectangle) to being a domain expert in either environmental science or hydrology (pointing end).

Question
Shifting of Average Scores from Non-specialist to Domain Expert Q1 Q2 Q3 Q4 Q6 Q7

Q8
Slightly more than half the participants (12 of 22) were experts in either environmental matters or hydrology. Results seem to show that the presence of experts boosts the accuracy of the answers in the very first hypothesis only, and in fact decreases the accuracy of the answers towards neutrality in all other hypotheses, as shown in Table 3. We cannot divide our sample into two groups of size 12 and 10 and apply a Mann-Whitney U Test in order to confirm this hypothesis because the sizes of these groups are too small to achieve statistical significance. Still, simply by showing the change in mean (as opposed to a change in median, which is less meaningful) it allows us to speculate that experts act as dampeners because overall they tend to be more reserved and "neutral" in their attitude. As such, the presence of "moderating" experts amongst our participants suggests that we should obtain even stronger results with a non-specialist audience.

Informative Nature of the Millbrook Serious Game
Changes induced by the game in terms of use of numerical values, vocabulary, and confidence and accountability are listed here.

The Accuracy and Increasing Use of Numerical Values in Participants' Answers
Overall, the words written by participants to justify their answers underwent a qualitative change from speculative in tone and unfocused to a greater focus on numerical values learned from the game. In Q1, the word used with the highest frequency in the pre-game responses is "Unknown". By contrast, in the post-game responses, the word used with the highest frequency becomes "108" which is the exact maximum number of houses flooded in the worst scenario. This shows a better understanding and a more precise quantification of the magnitude of the flood issues.
From Q2 to Q7 the degree of numeracy of the answers-the number of correct or relevant figures used in the justification of the answer-is nil in the pre-game questionnaire (none of the very few numbers written in justifications of these answer were relevant as mostly they were just copied from the questions) and higher in the post-game questionnaire (10 words in 293, approximately 3.4% of words written in justifications of these answer are accurate numbers).

The Acquisition of a Domain Specific Vocabulary and the Increased Utilization of Technically Accurate Words.
Although the size of the lexical field (list of all the words) does not change significantly (pre-game questionnaire uses 278 words to justify their answers while the post-game questionnaire uses 293)-in the post-game questionnaire there are many more words used appropriately, with a high frequency (underlying data can be found in supplementary file in "grid data" tab and in Tables S1-S4 in the "table" tab). In the pre-game answers, "Unknown" (17 times), "neutral" (15 times), "runoff" (7 times), and "reduce" (7 times) are the four most used words. In the post-game answers, "Farming" (17 times), "Impact" (14 times), "Soil" (12 times) and "Infrastructure" (11 times) are the four most encountered words. This indicates that players have acquired a more specialized vocabulary from playing the game and that they more often tend to use domain specific and technical words to justify their answers.
Furthermore, the proportion of words expressing a comparative relationship between two elements (for example: "bigger", "reduces", etc.) or a relative importance ("high", "greatly", etc.) also changes from 18.3% (51 words out of 278) in the pre-game answers to 29.3% (86 words out of 293) in the post-game answers. This shows a better grasp of the influences of different factors associated with the flood and damages and their relative importance.

The Growth in Confidence and Answer Accountability Induced by the Game.
Due to the fact that justifying their answer was not compulsory during the exercise, 14 participants sometimes chose answers without justifying them in the forms (resulting in one-third of the answers lacking an explanation). In these instances, we cannot say if the participants were able to justify their answers or not. That was only possible if they stayed "neutral" on their agreement/disagreement scale-in which case we concluded that they were not able to justify their answer. Most of the participants that were not able to justify their answers before playing the game were then found able to do so after playing it. This was done in an explicit and technically detailed manner-only 2 of the 28 answers were either wrong or not explicit enough (exact details can be found in supplementary data in "grid data" tab).
When looking specifically at the users that were not able to justify the answers to the questions before playing the game (33.33% of "Unknown"), the overwhelming majority of these players were able to justify their answers after playing the Serious Game (only 1.8% of "Unknown") (see Figure S1 in "grid data" table in data file for more details). There is only one exception, in questions 5 and 7, where the same user could justify his/her answer in the pre-game questionnaire, but failed to justify his/her answer in the post-game questionnaire-that particular player admitted having lost sight of that question while playing the game.
When directly asked if they learned anything new after playing the game, on average, the participants' answers were mostly positive, as shown in Figure 8 ( Table S2 in "tables" tab in supplementary data shows the complete list of associated answers). A couple of notable exceptions came from a land management advisor and a hydrology MSc student who felt that the game simply confirmed their pre-existing understanding. Finally, feedback from the participants regarding the suitability of the Serious Game for helping people to understand the Millbrook flood issues was positive overall, as shown in Figure 9 ( Table S3 in "tables" tab in supplementary data shows the complete list of associated answers). When asked why they would recommend the Serious Game, participants' answers can be divided into three broad categories:

•
It is a good informative tool that allows the user to view and understand the effects of decisions for many different scenarios.
• It encourages thought and discussion.
• It is good at changing attitude and promoting positive changes.
Regarding the impact of the Serious Game on the Millbrook community, eleven months after playing the Serious Game, a number of participants have communicated the outputs with other local stakeholders and have initiated the formation of an ongoing natural flood management partnership for action in Millbrook.

Discussion and Conclusions
In this section we discuss the cognitive changes induced by the Serious Game on players, the general impact on the Millbrook community, the limitations of this paper, and future work.

Cognitive Impact of the Serious Game
In this work we have shown that playing the Millbrook Serious Game significantly changed the participants' views on the Millbrook flood problem. Primarily, we have demonstrated that the game was informative (Section 3.2) because it improved participants' knowledge of the quantitative effects of flood damage, increased their use of technical vocabulary, improved their understanding of the influences and relative importance of multiple factors affecting the flood and cost outcomes, and caused a growth in confidence and accuracy in their answers. In Section 3.1 we established our most important aim whereby statistically significant changes induced by playing the game were directional and lead the participants to strengthen their confirmation and rejection of specific hypotheses. This also encouraged them to deduce rational and correct conclusions regarding fairly technical and non-trivial decisions for the Millbrook flood problem. In effect, the game briefly and successfully transformed the participants into "citizen scientists" in the purest sense of the term-it led them to use inductive reasoning from data produced during the game (hydraulic and costs) to correctly confirm or reject hypotheses in a rational manner. Feedback from participants even strengthened this claim when they admitted that the game allowed them to revise their views and assumptions on what they initially thought were the causes and best solutions to the Millbrook flood problem.

Impacts of the Serious Game on the Millbrook Community
Surprisingly, considering that we only had two residents from the Millbrook community playing the game, there nonetheless was a significant impact. These two participants did communicate with other local stakeholders and together have initiated the formation of an ongoing natural flood management partnership for action in Millbrook. The formation of partnerships is hugely beneficial for instigating action on the ground [33] because they encourage collaborative working, the sharing of resources, funds, knowledge and skills, and they can provide local groups with a clearer vision and drive for change. Westcountry Rivers Trust have stated that the Serious Game has helped to strengthen the relationship between Westcountry Rivers Trust and Millbrook residents, parish councilors, academic researchers, and local experts. They also added that this game will be a valuable tool, enabling them to communicate natural flood management concepts to a wide range of individuals across the south west of England. They concluded that the combined effect of the aforementioned benefits has increased the likelihood that natural flood management measures will be implemented in the Millbrook catchment area.

Limitations and Future Work
Possible limitations of this work regarding the efficiency of the Serious Game as a teaching and decision support tool were also found. The first concerns the barrier of technology as a limiting external factor. One participant's unfamiliarity with computer technology interfered with the consistency of game play and thus their answers, and so had to be discarded. This experience leads us to suspect that for a mixed audience, another kind of game interface or type may be more appropriate (possibly board games or numeric games with a comprehensive training component).
A second limitation is that some results indicated limits in the understanding of the participants. As shown in Figure S2 in the supplementary data file ("grid data" tab) there were inconclusive effects when participants tested the relative importance of more subtle factors such as "the sediments and drain blockage", rather than terrain roughness. This is likely a limitation in the depth of the cognitive effects of the Serious Game, possibly due to its relatively short duration. The game has a duration of one hour only, which is probably not enough time for participants to thoroughly investigate these subtler effects among the 432 combinations of flood management strategies. Each of these strategies results in outcomes that probably require more in-depth investigation. Eight of the participants were MSc students in hydrology, and they themselves seemed to have conflicting views on the prevalence of either roughness or sediment-induced drain blockage as the second most important factor. Potentially this could be improved by increasing the duration of the game, introducing more questions, and extending the duration of the exploration phase over multiple sessions.
A third perceived limitation was that the presence of too many participants with a high level of technical expertise might limit the generalization of the results. The Serious Game was originally built for a non-specialist audience-and more specifically for the Millbrook residents. As such, having slightly more than half the participants being experts in either environmental matters or hydrology might be seen as a limitation. At first, it would seem that this would: (a) lessen our ability to demonstrate that the Serious Game improves participants' knowledge as they are already experts, (b) "boost" the accuracy of answers unnecessarily and, therefore, skew the results towards correct hypothesis confirmation or rejection. As far as we can see, by carefully examining the data, it seems that these reservations are unfounded. First, the fact that we can demonstrate a strong informative value in Section 3.2 despite the presence of experts suggests that the game will work even better with a non-specialist audience. Second, results in Table 3 in Section 3.1 seem to show that the presence of experts only boosts the accuracy of the answers in the very first hypothesis and in fact decreases the accuracy of the answers towards neutrality in all other six hypotheses. As such, the presence of "moderating" experts in the midst of our participants suggests that we should obtain even stronger results with a non-specialist audience. Future work will emphasize testing with a targeted audience sample.
Another limitation, from a flood modelling point of view, is that the model is relatively coarse as we only consider three large agricultural areas and two infrastructure interventions. Future development of the model could incorporate finer spatial units and the ability to model a wider range of measures at specific sites across the catchment (e.g., storage ponds or leaky dams) so as to better support land management and potentially enable farmers participation. This of course would have to be done carefully so as to limit the resulting increase in complexity by selectively emulating key features of water and the possible actions of players in catchments [34].
Whilst the small size of our study limits our ability to generalize these results, the positive outcomes we have observed so far lead us to consider this work as a template applicable to other urban-rural flood case studies throughout the world. Potential new case studies are under consideration and should add to the available information to provide a more complete picture of this novel type of teaching tool and its suitability for improving the understanding of complex issues. We also intend to add control treatment groups in future experiments. Four possible control groups come to mind: a standard workshop without a serious game; a workshop without a serious game but with an intervention that includes a dialogue with participants based on the Socratic method and the validation or rejection of hypotheses; a facilitated workshop with a serious game without the Socratic method-based intervention; and finally, a treatment using another common exiting flood visualization tool.
Finally, the design of the Serious Game presented in this work could scale very well to complex problems with thousands of variables that would require weeks of practice to master-an area of serious gaming that is presently attracting increased interest. In the context of the video game industry, modern strategy games used for pure entertainment have presently reached a staggering level of complexity. Players usually need to practice for days or even weeks to understand how to simultaneously balance dozens of variables and their combined influences on the dynamic of the systems they play. In this respect, the Serious Game described in this paper has been engineered to be a "challenging/utilitarian" exercise that scales well with complex problems that could take long periods of playing time to master. A solution could be to extend the "exploration phase" of the Socratic Method inspired process described in Section 2.1.2 to several weeks if necessary, by asking participants to answer different groups of questions per session, focusing on a specific sub-problem at a time and this way avoid the kind of problems resulting from a broad-scale approach [35]. Although useful, this design does not provide an answer to the open question of motivational affordance, and additional work is needed in this area. We intend to pursue this avenue in the context of the ongoing (2016-2020) SIM4NEXUS research project [36], with a Serious Game that has possibly the most complex underlying model ever attempted.
Supplementary Materials: The following are available online at http://www.mdpi.com/2073-4441/10/12/1885/ s1. Table S1: Average response to Q10: Has this Serious Game changed any previous view you had on the Millbrook problem? Table S2: Average response to Q9: Did you learn anything new today? Table S3: Average response to Q11: Would you recommend this type of Serious Game for other people involved in Millbrook flood problem? Table S4: Average response to Q1: A one in-a-hundred years rainfall event of 45mm of water in one hour could cause serious flood problems in Millbrook, Figure S1: Justifying answers before and after playing the game, Figure

A one in-a-hundred years rainfall event of 45 mm of water in one hour could cause serious flood problems in Millbrook.
Strongly Disagree Disagree Neutral Agree Strongly Agree How many residences could potentially be flooded? Any worst-case cost approximation?

2.
Improving the village sewer infrastructure that directly drains excess water could result in neutralising up to 90% of the flood resulting from such an event.
Strongly Disagree Disagree Neutral Agree Strongly Agree Can you justify your answer?

3.
Improving farming practices that change the ground cover of the cultivated areas around the village could result in neutralising up to 90% of the flood resulting from such an event.
Strongly Disagree Disagree Neutral Agree Strongly Agree Can you justify your answer?

4.
Investing money in the sewer infrastructure gives a better result for your money when fighting against flood than trying to support different farming practice. Strongly Disagree Disagree Neutral Agree Strongly Agree Can you justify your answer?

5.
What are the two most important factors in this list of three that influence flooding when looking at the ground cover of cultivated areas?
the capacity of the ground to absorb water the amount of washed sediments that can block drainage pipes in the village downstream the roughness of the terrain the slows the travelling speed of the water Can you justify your answer?

6.
Heavily grazed permanent pasture increases greatly the capacity of the soil to absorb water. Strongly Disagree Disagree Neutral Agree Strongly Agree Can you justify your answer?

7.
Growing crops while minimising tillage increases greatly the capacity of the soil to absorb water. Strongly Disagree Disagree Neutral Agree Strongly Agree Can you justify your answer?

8.
Intensive farming that compacts the ground and lowers the capacity of the soil to absorb water, remains the most profiTable way to exploit a cultivated area. Strongly Disagree Disagree Neutral Agree Strongly Agree Can you justify your answer?

1-
What is the combination of measures that leads to the worst flood outcome? Also note the damage/cost and number of flood houses.

4-
What is the combination of measures that leads to the best financial outcome from both the infrastructure cost and the farming profitability?

5-
Is there a satisfying solution that manages to minimise the flood and bring an acceptable financial outcome?

Privacy Statement
Please complete this questionnaire. Any concerns can be communicated to Mehdi Khoury (m.khoury@exeter.ac.uk) at University of Exeter. Thank you for your time and cooperation. Answer the following questions by circling the most appropriate answer.

A one in-a-hundred years rainfall event of 45 mm of water in one hour could cause serious flood problems in Millbrook.
Strongly Disagree Disagree Neutral Agree Strongly Agree How many residences could potentially be flooded? Any worst-case cost approximation?

2.
Improving the village sewer infrastructure that directly drains excess water could result in neutralising up to 90% of the flood resulting from such an event.
Strongly Disagree Disagree Neutral Agree Strongly Agree Can you justify your answer?

3.
Improving farming practices that change the ground cover of the cultivated areas around the village could result in neutralising up to 90% of the flood resulting from such an event.
Strongly Disagree Disagree Neutral Agree Strongly Agree Can you justify your answer?

4.
Investing money in the sewer infrastructure gives a better result for your money when fighting against flood than trying to support different farming practice. Strongly Disagree Disagree Neutral Agree Strongly Agree Can you justify your answer?

5.
What are the two most important factors in this list of three that influence flooding when looking at the ground cover of cultivated areas?
the amount of washed sediments that can block drainage pipes in the village downstream the capacity of the ground to absorb water the roughness of the terrain the slows the travelling speed of the water Can you justify your answer?

6.
Heavily grazed permanent pasture increases greatly the capacity of the soil to absorb water. Strongly Disagree Disagree Neutral Agree Strongly Agree Can you justify your answer?

7.
Growing crops while minimising tillage increases greatly the capacity of the soil to absorb water. Strongly Disagree Disagree Neutral Agree Strongly Agree Can you justify your answer?

8.
Intensive farming that compacts the ground and lowers the capacity of the soil to absorb water, remains the most profiTable way to exploit a cultivated area.

1: Overview of infiltration and overland flows across different farming practice scenarios
Soil infiltration rates are fundamentally affected by soil type and specifically the texture of the soil (i.e., the ratio of sand, silt and clay) [37]. Beyond this, there are several positive impactors that can contribute to improved infiltration rate and capacity, including: crop diversity, residue maintenance, cover cropping, managed traffic (minimizing area compacted by heavy vehicles), increased organic matter content, conservation agriculture, contour farming and increased earthworm content. Conversely, some practices and outcomes have negative effects on infiltration rate and capacity, including: surface crusting, low organic matter content, compaction, high soil moisture (waterlogging) and winter break rotations.
Infiltration rate is often estimated theoretically via such standards as the Richards equation [32] or the Green and Ampt [38] method. In these methods, the existing saturation of the soil and the porosity allow for such estimation, though they often assume a homogeneous soil. Empirically, infiltration rate data is often measured in isolation, or for simple scientific comparison between two system types (e.g., organic versus conventional); there is a paucity of data contrasting the variable farm practices across a region or watershed, measuring infiltration dynamics over time, and accounting for differing soil profile compositions. As such, to compile representative values, it is necessary to interrogate the literature for suitable data with which to set generalized values appropriate for a Serious Game.
The soils in the Millbrook watershed belong to Soils cape 6 [39] (a freely draining, slightly acid loam). These shale-derived sandy loam soils are of below average fertility, with neutral to acid pastureland dominating their use. Their carbon stocks are typically low and they are prone to both soil erosion and compaction.
In conventional tilled fields in the UK, infiltration rates on sandy loam soils typically vary between 15-45 mm/h, though compaction by agricultural machinery increases bulk density, reduces porosity and rapidly reduces infiltration rates to 3 mm/h [40][41][42][43][44]. With these values in mind, and considering the likely bare infiltration rates of Soils cape 6 soils (10-20 mm/h), it is considered unlikely that arable soils in the Millbrook watershed will have an infiltration rate greater than 30 mm/h, unless conservation agriculture techniques are employed. Similarly, for grazed pasture soils, infiltration rates typically fall between 3-36 mm/h [45,46], with a mean of 9 mm/h for permanent pasture, as low as 0.1 mm/h with heavy grazing pressure. As such, it is considered unlikely that permanent pasture in the Millbrook watershed, unless regular leys or mechanical improvement have been incorporated into the management strategy, will have an infiltration rate greater than 30 mm/h.
With the above summarized data in mind, the following values (mm/h) were chosen: red (heavily grazed permanent pasture) = 10, orange (tilled arable with standard rotation and traffic) = 15, yellow (young pasture with mechanical improvement and moderate grazing) = 30, aqua (tilled arable with diverse rotation and increased organic soil amendment) = 38.
Cover crops and crop residues increase infiltration rates by breaking the kinetic energy of rainfall, increasing residence time on the soil surface, and increasing the prevalence of root holes and thus heterogeneous porosity. Data on the effects of these vegetation types is sparse, though relevant literature suggests that cover crops and crop residues can increase infiltration rates up to at least 70 mm/h [42,47]. As such, a value for farming systems (blue) in which cover crops and minimum-tillage contribute to permanent cover was set at 70 mm/h. Beyond this, a wholesale shift to a no-till, conservation agriculture approach that built soil carbon, maintained soil cover, and minimized traffic could conceivably increase infiltration rates in a similar vain to a lightly, trafficked organic field, i.e., 6-10 times greater [40].
As such, for the purpose of this Serious Game, we have settled on 90 mm/h (6× that of conventional tillage) as the fastest infiltration rate possible for this farm system.

2: Sediment fluxes
The principal factors affecting soil erosion are: (a) the amount and intensity of rainfall and wind, (b) topography, (c) soil properties, and (d) natural and managed vegetation resulting from farming practice. Positive impactors, such as the presence of year-round ground cover, high organic matter contents and good aggregate stability, protect against soil erosion, whilst negative impactors, such as surface crusting, compaction, and winter break crop rotations, promote soil erosion [40]. The largest factors contributing to erosion were found in [48] to be crop cover and valley features, with erosive rainfall events typically at >10 mm d −1 and erosion dominating bare soil environments. Beyond 15% crop cover, erosion was more typically associated with tramlines and wheel tracks and exacerbated by landscape features. Cropland soils often erode at a higher rate than comparable grassland soils, due to their reduced ground cover, with [48] finding cropland erosion rates to be between 0.5-4× greater than grassland erosion rates. The median rate of soil erosion in the UK is approximately 1.3 tones/ha, with higher mean rates (>2.6 tones/ha) being associated with sandy or silty soils ( [49] and references therein). Sediment losses vary substantially with crop type and agricultural management practice, though research does suggest that traffic and tillage both increase sediment losses [47,49].
As such, for the Serious Game, greater sediment loss rates were forecast from arable fields, with changes in management practice towards less intense, conservation agriculture approaches with greater ground cover yielding lower sediment losses. We present these as a percentage of the sediment loss needed to block the local drainage systems, from 10% to 90%. Note that at particular times, i.e., just after tillage in a bare arable field, the sediment loss has the potential to be both very substantial and enough to block 100% of the local drainage system. Table A1 presents a summary of the profitability for each of the three sub-regions of the Millbrook catchment, and estimated for a pre-Brexit scenario for the year 2016-2017 using the online calculator in DEFRA's Farm Business Survey 2016/17 [50]. Profitability for a given area is computed by multiplying either the high or the average net profitability per hectare-respectively 628.7 GBP/Hectare and 283.9 GBP/Hectare-by 85% of each region surface so as to take into account the fact that a small portion of the said "cultivated areas" around the village is not being exploited. The survey provided recent average business incomes for different types of farming. When defining profitability for the Serious Game, the fact that the average cereal farm profitability was more than double the average profitability of a livestock farm determined the basic profitability scores. The average mixed farm was somewhere in between, with 'general cropping' (i.e., diversifying the rotation) usually leading to improved profitability. These incomes are inclusive of single farm and agri-environment payments, with the majority of farms making a loss on agricultural income alone. Compounding factors, such as the potential for longer 'overwintering' periods in certain livestock farming approaches, the profitability of grass leys, reduced fuel costs in minimum tillage approaches, and the profitability of diversifying the rotation, were all considered in making subtle adjustments to the in-game profitability scores.

Appendix E. The Level of Investment in the Drainage Infrastructure
We have excluded lining (involving the installation of an internal resin layer that bonds to the inside of the pipe, essentially creating a pipe within a pipe), as data on this specific procedure were not readily available. The formula applied for estimating the management costs is confidential (South West Water). However, the distribution of pipe diameters by total length used to compute these costs is presented in Table A1. It should be pointed out that any pipes of unknown diameter were assigned an estimated diameter of 150 mm for the calculations.

Appendix F. Computational Complexity of the Hydraulic Model and Compromises Made
A 1 m resolution cell size took approximately two hours to complete, while a 2 m resolution model took approximately 1 h. Reducing the resolution to 5 m still yield a processing time of approximately 1.5-5 min, and the 10 m resolution was completed in less than 10 s with a recent PC with a Nvidia GTX 1070 graphic card. A compromise of spatial quality over processing time is sought by selecting a resolution of 2.5 m, as the loss of precision between 2 m and 2.5 m is not really noticeable from a player point of view.
The simulation was run for twelve hours of simulation time to allow the flow to propagate after the rain events. The open boundary condition was set to allow flow to escape the terrain. The velocity between two neighboring cells with a water level difference less than 0.1 mm was assumed as zero to speed up the calculation for insignificant quantities of flows.
Different types of surfaces shown in in were associated with an appropriate infiltration rate and roughness as shown Table A1. The flood damage evaluation uses the residential sector average damage curve for short duration floods (less than 12 h) from the Multi-Colored Manual [51,52]. We simplify calculations as followed to compute damages from flood depth. Due to the average slope of the terrain and history of floods, house ground floors were slightly elevated. Assuming that the floor of each house starts 15 cm above the ground level, we subtract 15 cm from the flood depth to get the water depth measured inside the house to use it as a proxy for residential damage as plotted on the curve. Then we consider two possible levels of flood damage per building. Any flood depth between 0.3 m and 0.5 m will be considered inside a building as a 0.15 m water depth damage of 314 GBP/m 2 and any flood beyond 0.5 m will be considered inside a building as a 0.35 m water depth damage of 476 GBP/m 2 . This simplification allows us to limit the complexity of the game visualization to two levels of flood damage while also the over-estimation that would result from not taking into account ground floor elevation. The data regarding the surface and location of each building were provided by The Westcountry Rivers Trust.

Appendix G. Detail of Wilcoxon Signed-Ranks One-Tailed Test computations
To prove that the Serious Game induces a directional change in the way participants confirm or reject each hypothesis, we calculate the level of significance as follows. We rank the differences for each pair, and affix a sign to each rank. From the sum of positive ranks W+ and the sum of negative ranks W− we compute the W test value such that W = min {W+, W−}. The number of instances where the subject's difference score is zero is then deducted from the sample size N. The critical value T at p = 0.05 and p = 0.01 for a one-tail test is then given by the test as provided in [31], and if W ≤ T, the null hypothesis (being that that the median difference is zero) can be rejected. Details of the results are shown below for all seven hypotheses.