Models, Simulations and Games for Water Management: A Comparative Q-Method Study in The Netherlands and China

Abstract

How do policy analysts perceive the various roles that Models, Simulations and Games (MSG) have, or can have in Integrated Water Resources Management (IWRM)? Fifty-five policy analysts in water management in The Netherlands and China were interviewed, following the procedure of the Q-method. Comparative analysis of the combined quantitative and qualitative data show that: (1) The debate on the role of MSG for IWRM is structured around five frames in The Netherlands and three frames in China. (2) The frames in The Netherlands and China are significantly different. (3) In China, there is a predominant frame that perceives MSG for IWRM as data driven simulation technology for rationalization of water management, which is less significant in The Netherlands. (4) The reverse is true with regard to MSG for stakeholder interaction, learning and integrated assessment, which are significant frames in The Netherlands, but not in China. The conclusion is that frame differences can easily confuse professional and academic debate about MSG for water management; within the same institutional and cultural context, but even more so in Netherlands–China co-operation projects. Frames are also relevant when designing, using or evaluating innovative methods for integrated water resources management.

1. Introduction

How do policy analysts perceive the various roles that Models, Simulations and Games (MSG) have, or can have for public policy making in general, and for Integrated Water Resources Management (IWRM) in particular? What do policy analysts in water management expect of MSG, if anything at all? These questions are relevant in light of a growing advocacy to develop and apply advanced models, simulations and (serious) games, in particular, for IWRM [1,2,3,4,5].

The professional and academic debate on the use and usefulness of MSG for policy-making and management in general has quite a tradition [6,7]; however, it is also rife with strong and diverging opinions about what MSG can or should deliver. Some expect nothing less than a digital revolution in the way we make, or will make policy decisions; a Futurium where scientific evidence and public participation are fully integrated [8]. In the near or more distant future, we may well be managing natural systems, such as oceans and river basins, through advanced and highly interactive and participatory management and control systems, built upon big data, geoinformation systems, artificial intelligence, scientific models and simulations combined with advanced game-technology, and Virtual and Augmented Reality (VR and AR, respectively) [9,10,11,12]. This is also the rationale behind “Understanding Game-based Approaches for Improving Sustainable Water Governance: The Potential of Serious Games to Solve Water Problems”, the theme of this Special Issue.

Others claim that such technological innovations are rather trivial, because they are not well aligned with the true and underlying reality of politics and management; the blunt exercise of power to protect particular interests [13]; and economic growth and the accumulation of capital. Analytical methods in the policy process, including MSG, may be seen as irrelevant [14], or as an intricate form of manipulation [15,16]. Conversely, they may be seen as a last resort to rescue evidence-based policy making from the growing influence of a “post-truth” belief system [17]. From that perspective, the advocacy for things such as serious games, gamification and virtual reality can be seen as the response of policy analysis to a growing social need to get “engaged and entertained”, also in public policy making.

Other voices start from the normative end. The urgency and complexity of contemporary problems—climate change and the rapid degradation of natural systems in particular—necessitates a radical transformation of the way we make policy decisions [18,19]. This has led to a strong call for a post-normal science [20]. Holistic interventions should drive for transformative change at the level of values and fundamental beliefs of stakeholders [21,22]. Advocates of such view commonly expect a lot of social learning with or without the support of interactive methods (serious games), immersive technologies (VR/AR, Virtual Reality/ Augmented Reality) and big data [2,23,24,25].

Amidst such diverging voices, empirical evidence about the use and usefulness of MSG for transformative policy making does not really take us much further, unfortunately. There is a considerable and fast-growing list of research publications where MSG have been used for IWRM (e.g., [3,10,12,26,27,28,29,30,31,32,33,34,35,36,37,38,39]). There are a few observations that can be taken from this literature. Most of the empirical research is done in an educational context, often with university students as players. Professionals in such game sessions are usually experts who work at the boundary of science and policy; academics, consultants, and strategy officers. Game sessions are often part of a general conference, or a dedicated seminar or workshop with reflective and social purposes. Most of the empirical studies can be classified as case-based, tool-oriented design and validation studies. Learning effects are mainly considered at the individual level, or player-group level [3,37]. Some MSG have been used effectively however as intervention method with local stakeholders at a community level, for instance with local farmers or fishermen in rural areas of developing countries. See for a few examples: [33,36,40,41].

The most significant observation however is that research on how MSG actually contributes to, or even influences, policy making and management is scarce, perhaps because it requires an evaluation type of research that is quite difficult to set up. It would need to build on a comparative analysis of a rich and varied set of cases where such innovative approaches have been used for policy making, over a longer period (for a few exceptions, see [5,6]). Furthermore, how do we observe and value the different functions of policy analysis against one another; their instrumental, communicative, enlightenment and empowerment functions [42,43]?

The objective of this paper therefore is to take a step back by reconstructing the different fundamental and coherent views, called frames, that policy-analysts and policy makers have about the use and usefulness of MSG for IWRM. To do so systematically and profoundly, we use the Q-method, explained in greater detail in the Materials and Methods Section. There are three key concepts that provide the basis for the operationalization of statement in the Q-study: (1) Policy Analysis (PA); (2) Integrated Water Resource Management (IWRM); and (3) Models, Simulations and (Serious) Games (MSG).

Policy analysis is defined by Dunn [44] as “an intellectual and practical activity aimed at creating, critically assessing, and communicating knowledge of and in the policy-making process”. Policy analysis finds itself at the heart of the Science–Policy Interface (SPI) [13,45,46,47,48]. This is the boundary sphere were scientific and other knowledge oriented institutions interact closely with political institutions and public agencies. Research has shown that the different rationalities in the world of science and the world of politics lead to many kinds of tensions, such as the risk that results of studies tend to come too late, and after the political facts. These boundary tensions need to be managed and mediated roughly by accommodating policymaking to science (speaking truth to power) or, vice versa, by accommodating science to policymaking (sciences that serve power) [49,50,51]. Models, Simulations and Games (MSG) are among the more sophisticated methods in the policy analysis’ tool box, and as such subject to many of the tensions at the SPI [45,48,52]. The boundary tensions with regard to the use and usefulness of MSG for policy making are the focus of our research.

Integrated Water Resources Management is the locus—the domain application area—of our research. IWRM has been defined by the Global Water Partnership (GWP) [53] as “a process which promotes the coordinated development and management of water, land and related resources, in order to maximize the resultant economic and social welfare in an equitable manner without compromising the sustainability of vital ecosystems”. In this research, we will consider two water management regimes, in The Netherlands and China, which may be very different in the way they view the science–policy interface in IWRM.

According to some authors, IWRM is closely connected to the transformation of a “prediction and control” regime to an “adaptive, learning” type of regime [54]. Similar to major regions in China, The Netherlands is a river delta, although of a different size than for instance the Pearl River delta. Most of The Netherlands lies below sea level and a substantial proportion of land has been won from the sea through land reclamation and the building of dikes. Since the early 1990s, however, there is an ongoing paradigm shift from engineering-driven solutions (more, higher and stronger levees) to solutions where nature, technology and human aspects are in balance. Since the “Room for the River” policy in the early 1990s, IWRM is well established in The Netherlands. Due to global environmental changes, flooding is becoming a major problem in other parts of the world too, where there is much less expertise in water management. China is one of the countries to which The Netherlands actively disseminates and exports the principles and methods of IWRM, along with many of the tools to realize it. With the increasing occurrence and scale of flooding, the central government of the Peoples Republic of China (PRC) is reforming from a highly fragmented and engineering driven water management administration, towards an IWRM approach. The gives an excellent opportunity to compare the two countries. We will therefore reconstruct the different frames of policy analysts and policy makers in the two countries with respect to IWRM on aspects such as governance, sectoral and functional integration—and examine if and how these frames influence their perceptions on the use and usefulness of MSG. We intentionally do not narrow down the understanding of IWRM—s.a. into urban/rural or water quality/quantity—as this would need to come to the surface in the interviews.

Models, Simulations and (Serious) Games is increasingly being used as a general reference term for a varied cluster of related methodologies and techniques for understanding and managing complexity in a context of decision-making. This includes systems dynamics, agent-based modelling, Geographical Information Systems (GIS) and gaming-simulation. More recent innovations in this area are (serious) gaming, Virtual Reality (VR) and Augmented Reality (AR). There is considerable overlap and cross fertilization among these methods.

The term model mainly refers to the manner in which a part of reality—called the “reference system”—is captured and represented, in a mental, physical, digital or virtual form. Simulation mainly refers to the manner in which the dynamic behavior of the reference system is represented in such a way that it enables human experimentation and manipulation over time. When one or more human actors can give input into the simulation—the user can play with certain input parameters—it is not uncommon to call it an interactive simulation. When multiple users interact with the simulation as well as with each other, it is commonly called a social simulation. When there is also a defined set of rules for human–computer or human–human interaction, in the form of play rules, player roles, player goals and a narrative, it becomes a simulation-game or a serious game. The term simulation-game or serious game generally refers to the societal utility of games, the game technology, as well as the game principles and the game formats. This utility of games is always mediated through a form of learning or change, at the individual, organizational or system level. The almost infinite number of ways how external purposes, applications domains, game technologies and game mechanics, can be combined creates an enormous variety in serious games. This also explains why they are also called under different names, such as policy exercises or gamification [55]. In this study, we will consider all kinds of genres of serious games—including interactive models, simulations and VR/AR—designed and used for IWRM, be it learning, research, validation, intervention and more.

2. Materials and Methods

2.1. Q-Method

A comparative Q-method study in The Netherlands and China was designed and executed to understand how policy analysts—e.g., the experts, policy makers, managers, etc. who operate at the science–policy interface—frame the use and usefulness of MSG for IWRM. The Q-method is particularly suited to uncover the underlying structure of human subjectivity around a certain topic [56]. It combines qualitative and quantitative data collected in personal interviews, and works as follows:

• Collection of several statements (the Q-set, usually between 30 and 40) that are representative for diverging views on a certain topic.
• Identification of respondents (P-set, usually between 25 and 40) who are representative for the diverging viewpoints.
• Personal interviews in which each respondent ranks the collection of statements (Q-sort) on a scale (Q-scale, −3 to +3, strongly disagree–strongly agree), thus laying out a quasi-normal distribution of the cards, as in Figure 1, while giving detailed explanations and arguments to the interviewer.
• Statistical analysis (an “inverted factor analysis”) of the Q-sorts (called the Q-data) and coding of the qualitative information.
• Reconstruction of a limited set of frames based on the factor analysis but interpreted and labelled with the qualitative data.

2.2. Q-Set

The set of statements—called the Q-sort—was collected by scanning the literature in the widest and most diverse coverage possible, as well as interviewing a few experts. The initial Q-sample consisted of sixty-five statements divided over the three main issues IWRM, Policy Analysis and MSG. The initial statements were improved and reduced to forty-three statements in several Q-trials with peer experts. The statements were then translated from English to Dutch and Chinese. The quality of the translation was verified in a few more trials with bilingual colleagues, with English or Chinese as a native language. The final set of Q-statements was put on laminated cards, as well as into the online Flash Q system for online use, mainly in China. To make the interviews in China shorter and more practical, it later proved necessary to reduce the number of statements to thirty-four and adapt the Q-scale accordingly. Although this affected comparability, the Q method is positioned as a flexible method that relies heavily on the explanations that the respondents give for their positioning of the Q-statements.

2.3. P-Set

The respondents in The Netherlands and China were senior professionals, mostly male, with quite a lot of work experience in the field. Both in China and in The Netherlands, we selected respondents from all relevant fields and points of view: water policy makers, water managers, water stakeholders, water researchers and consultants, MSG developers and users. In The Netherlands, we approached candidates for an interview through organizations such as the ministry, the water boards, provincial and local governments, universities, consultant and engineering companies. The long list of candidates went up to around fifty in The Netherlands, of which thirty-three were interviewed with a fully completed Q-sort.

Interviewees in China were selected through a similar procedure, making use of our professional contacts in cities of Dalian, Nanjing and Guangzhou. We contacted officials in the local municipality, water management sectors, infrastructure design institutes, water research institutes and universities. Finally, we had useful interviews with twenty-two persons from different institutions who also completed the Q-sort. All interviews were done by the first author, born and raised in China, educated in universities in The Netherlands and proficient in the Dutch and English language.

The whole interview would take between one and three hours, with the card sorting consuming about an hour. In The Netherlands, all but five respondents used the card version. In China, we only used the digital Flash Q-software version (not the cards) during live, personal interviews. At the end of the interview, respondents were asked to answer some questions related to their professional affiliation, their expertise, and knowledge about MSG and additional things if there were any. We coded all additional information and inserted the data into one database. All interviews were fully transcribed. After some introduction of interviewer, objective and technique, the respondents were asked to read (aloud) the statements on the cards and subsequently rank the cards in a quasi-normal distribution (scale from −3 to +3). Figure 1 gives an impression of a quasi-normal distribution as in this research in paper-based form.

2.4. Analysis

The results of the Q sorts were analyzed with factor analysis, supported with the software PQ Method. With the factor analysis (Varimax rotation) respondents with the same answer patterns were grouped together. The factor analysis extracts factors on two layers, a frame about IWRM and a frame about MSG, that matched together. The frames suggested by the statistical analysis were combined with coded background data (position, work field, experience with MSG, etc.), general questions (familiarity with and trust in MSG) as well as quotations from the interviews.

3. Results

3.1. The Netherlands

Twenty-three out of thirty-three respondents in The Netherlands had quite some experience with the use of MSG for policymaking. Three respondents, all in frame NL1 (see below) had “no trust at all” in MSGs and one had a “little doubt”. The respondents could reflect on the conditional usefulness of MSG, such as “when it is based on scientific data”, “when it is used with the right participants”, or “yes, but it is costly and time consuming”. From the analysis, five frames are distinguished:

• Frame NL1: Bureaucratic alignment
• Frame NL2: Stakeholder interaction
• Frame NL3: Learning
• Frame NL4: Uncertainty
• Frame NL5: Science vs. emotions.

3.1.1. Frame NL1: Bureaucratic Alignment

Frame NL1 is represented by eleven respondents, three respondents belonging to the world of science, and eight to the world of politics.

Table 1. Distinguished statements about IWRM in frame NL1.

Number	Statement	Ranking
16	Reinforcing levees (dikes) etc., is insufficient to keep The Netherlands safe from flooding in the 21st century.	+3
18	Socioeconomic developments in flood-prone areas should be mitigated through spatial planning and construction regulation.	+3
11	A strong degree of integration of water management and spatial planning at different administrative and spatial levels is crucial for water management.	+3
5	Uncertainty in water management is deepened by a lack of integration among social, political, technological, ecological, economic (etc.) knowledge.	−3
7	Water managers should set more ‘social learning’ activities on their agenda.	−3
8	A centralized form of governance, with sufficient authority and decision power at the national level, is crucial for water management.	0

shows the distinguished statements about IWRM in this frame. Distinguished statements combine extreme answers (−3; +3) with strong factor loadings.

The statements give a rather clear picture of how the respondents frame the issue of IWRM, namely as predominantly a matter of integrating engineering works and infrastructures with spatial planning, regulation and levels of governance. Water management at the different administrative levels of governance should be integrated. At the other extreme, the integration of stakeholders’ values and things like collaborative learning and stakeholder participation are most strongly rejected. We therefore label this frame “bureaucratic alignment”, because it is mainly the administrators and experts who integrate (or should integrate) their expert knowledge and authority. One interviewee who scored high on frame NL1 expressed quite honestly why he rejects learning and stakeholder interaction:

“At the national level, the spatial solution has been chosen as the necessary solution to flood management in the future. However, until now the top-down decision is not fully accepted and cooperated with, at regional and local levels. […] To deal with these issues you need to have a clear understanding of the power and interest of the stakeholders. […] When people in a group are against the spatial solution you brought, it is hard to talk to and convince them. However, if you come in the evening to communicate individually, the chance of negotiation will increase. They will ask you what your offer is.”

(Interviewee No. 16).

In other words, agreement with stakeholders can better be reached through power, by making “package deals” or by “compensation”. Top-down decisions simply need to be sold locally. There is little need for or influence of learning, since stakeholders already know what they do, and do not want. Now, the question should be asked what the role of integrated approaches and MSG methods is. Probably not a lot, except for making package deals and getting things done, preferably behind closed doors. This is reflected in the respondents’ framing of MSG in IWRM. Five respondents had no experience of MSG; one had, but does not trust it; and a few others seemed politely skeptical.

Table 2. Distinguished statements about MSG in frame NL1.

Number	Statement	Ranking
23	The key function of ‘policy analysis’ is to support the stakeholders’ learning process.	−3
31	Policy simulation does not need to be computerized. ‘Low-tech’ gaming based on human behavior is also a scientifically proven method for water policy analysis.	−2
43	Playing together in a simulation game increases the stakeholders’ willingness to cooperate in the real world.	−2
36	Visualization (e.g., by pictures, animations or 3D graphics) significantly increases the users’ understanding of models and simulations.	+3
39	Computer simulations can accommodate poorly with conflicting values and interests of stakeholders in water management and water policy.	−3

shows the distinguished statements about MSG presented in this frame.

In line with the bureaucratic tendency in this frame and the rejection of things such as learning and stakeholder participation, low-tech games and human play are regarded as not very useful. From the interviews, it becomes clear that in frame NL1 the bureaucratic process can best be supported with simulation and visualization technology. Think of models that can calculate the consequences of plans, or visualization tools to integrate plans in 3D. Such analyses are powerful for bureaucratic integration and when negotiating with stakeholders. The social learning value of MSG, however, is deemed irrelevant and impossible in this frame. The reason for this is nicely illustrated in the following quotations:

“Simulating the richness of social values is impossible because a lot of social values, individual values, are not possible to involve in the model. […] A spatial solution needs people’s property. They have different reasons to refuse to give you their property. […] How can you explore those through gaming? […] In such an environment the most important issues of integration are network cooperation and visualization technology. Visualization increases the policymakers’ understanding of technical analysis and therefore contributes to cooperation.”

(Interviewee No. 16).

“Policymakers look for excuses to not to learn from the game. Gaming is not the thing to change the behaviours of individuals. However, it can be used strategically to show the community the need to make the long-term decision and stimulate the discussion.”

(Interviewee No. 20).

3.1.2. Frame NL2: Stakeholder Interaction

Nine respondents loaded in frame NL2. We coded four respondents as belonging to the world of science (researchers, consultants) and five as belonging to the world of politics (public policymakers, water managers). All had experience with MSG, and trusted it to be useful for policy making. We label this frame “stakeholder interaction”.

Table 3. Distinguished statements about IWRM in frame NL2.

Number	Statement	Ranking
22	The key solution to the consequences of climate change lies in active public involvement and stakeholder participation. Societal interaction will provide the most significant contribution to water management and policymaking in the near future.	+3
2	The key problems in water management today are more socio-political than technological-infrastructural in nature.	+2
9	A network type of governance, with interaction between interdependent stakeholders, is crucial for water management.	+2
11	A strong degree of integration of water management and spatial planning at different administrative and spatial levels is crucial for water management.	+2
18	Socioeconomic developments in flood-prone areas should be mitigated through spatial planning and construction regulations.	0
3	The increasing complexity of society leads to a problematic compartmentalization and fragmentation in water management.	−3
8	A centralized form of governance, with sufficient authority and decision power at the national level, is crucial for water management.	−3

shows the distinguished statements about IWRM.

This frame relies more strongly than frame NL1 on a network view of society and policymaking, whereby many stakeholders are and should be involved in water management. Water management problems are more socio-political than technological–infrastructural in nature; solutions should therefore come from stakeholder interaction and public involvement. Governance should accommodate the network character of society. The centralization of governance should be strongly rejected (difference from frame NL1). Power, authority and competences are unequally distributed in the interaction between water authorities and stakeholders and other authorities: the region of a water board can be as big as ten municipalities or a province. This can cause a lack of trust and frustrate the cooperation between water boards and municipalities. This needs some time and effort to change, to build enough trust to cooperate.

Table 4. Distinguished statements about MSG in frame NL2.

Number	Statement	Ranking
23	The key function of policy analysis is to support the stakeholders’ learning process.	−2
31	Policy simulation does not need to be computerized. Low-tech gaming based on human behaviour is also a scientifically proven method for water policy analysis.	+3
28	The outcomes of computer simulation are generally more authoritative (trustworthy) for water policymakers and water managers than the outcomes of a simulation game with real stakeholders.	−3
42	Simulation gaming can effectively facilitate and support the interaction among stakeholders from different governance sectors.	+3
43	Playing’ together in a simulation game increases the stakeholders’ willingness to cooperate in the real world.	+3
37	Computer simulations for water management and water policymaking should be easy to use and understand by non-expert users.	−3

shows the distinguished statements about MSG in this frame.

As in frame NL1, the respondents in this frame disagree that the function of policy analysis is to support a learning process. However, in contrast to frame NL1, playing together in a game is seen to have effects on stakeholders’ interaction in the real world. According to frame NL2, strategic interaction in a serious game is a social intervention to further stakeholder interaction, and not a learning process (difference from frame NL3). Two quotations illustrate how respondents in this frame distinguish serious games from computer simulation, that is, they are not comparable.

3.1.3. Frame NL3: Learning

Only three respondents loaded in frame NL3, and we coded all of them as water manager. In contrast to the other frames, these three respondents put significant emphasis on learning, that is, increasing stakeholders’ understanding of the complexity of water management. As in frame NL1, but in contrast to frame NL2, the three respondents in this frame do not consider problems in water management to be mainly socio-political in nature. From the statements, it also appears that respondents are rather technology-oriented (contrast with frames NL1 and NL2), and rejecting of stakeholder interaction and network governance.

Table 5. Distinguished statements about IWRM in frame NL3.

Number	Statement	Ranking
23	The key function of policy analysis is to support the stakeholders’ learning process.	+3
2	The key problems in water management today are more socio-political than technological–infrastructural in nature.	−2
22	The key solution to the consequences of climate change lies in active public involvement and stakeholder participation. Societal interaction will provide the most significant contribution to water management and policymaking in the near future.	−3
16	Reinforcing levees (dikes) etc., is insufficient to keep The Netherlands safe from flooding in the 21st century.	−2
9	A network type of governance, with interaction between interdependent stakeholders, is crucial for water management.	−1
11	A strong degree of integration of water management and spatial planning at different administrative and spatial levels is crucial for water management.	−1
29	Rational thinking should always be combined with human emotions in policy analysis for integrated water management.	−2

shows the most distinguished statements about IWRM in this frame.

The following quotation illustrates the frame:

“The water management problem is very dependent on the local conditions. Generating the solution needs a lot of local knowledge. Water governance should be more decentralized. In The Netherlands, the surroundings of the local area is really different, both the socio-political issues and the characteristics of the water problem. The solutions must satisfy the needs of local development. For example, agriculture in greenhouses is a typical economic activity in the area around [city name]. The policy strategy needs to involve the calculation of the cost and impact of policies on this activity, which is not necessary in the other areas. Central government cannot generate solutions but only make political choices. The practitioners in the local sectors are the experts to get the job done.”

(Interviewee No. 1).

The focus on learning in a complex technological setting, rather than stakeholder interaction, seems to be why the respondents consider the combination of computer simulation and stakeholder participation important. SG is valued as a good method for innovative learning in water management. It can be effective to analyze the future, to test policy options in a safe environment, etc.

Table 6. Distinguished statements about MSG in frame NL3.

Number	Statement	Ranking
20	Methods that combine computer simulation with stakeholder participation are supportive of water management.	+2
30	A simulation game with real stakeholders as players is generally more effective to foresee and analyse what can happen in the near future than a computer simulation.	+3
24	Most computer models are not flexible enough to deal with complex water problems. Models that can be quickly developed and changed to fit the circumstances are needed.	+3
27	Gaming simulation with real stakeholders as players is a better strategy for the innovative process than using computer simulations in integrated water management.	+2
30	A simulation game with real stakeholders as players is generally more effective to foresee and analyse what can happen in the near future than a computer simulation.	+3
34	Testing various policy options in a safe environment (such as simulation gaming with real stakeholders as players) is crucial to avoid serious consequences of water policymaking to the real world.	+2
43	Playing together in a simulation game increases the stakeholders’ willingness to cooperate in the real world.	−3
37	Computer simulations for water management and water policymaking should be easy to use and understand by non-expert users.	−3

shows the most distinguished statements about MSG in this frame.

What is interesting is that the respondents strongly disagreed with the statement that gaming increases stakeholders’ cooperation in the real world, whereas this statement was strongly valued in frame NL2. Upon closer inspection, however, this is not so strange. Respondents value MSG for their capacity to learn from it, and not as socio-political intervention, as in frame NL2. We believe that frame NL3 is present among a small number of experts in water management who have a focus on engineering but are open to social innovation. They believe in interactive simulations and games for learning, but reject the negotiated nonsense and the wheeling and dealing that commonly occur in interactive stakeholder processes:

“We do not believe in the complex integrated model. You can, for example, combine the groundwater model with the surface water model. In such a model you get more parameters that can also be wrong. It will not give more certain results but create more doubt. For instance, models often give incorrect predictions of the water level rise. Simulation games should be used first to explore the possibilities. It can mean a lot at the start of a process to explore each other’s views and understand the opportunities and constraints analysis. Computer simulation can be used after a game to analyse the best option.”

(Interviewee No. 33).

3.1.4. Frame NL4: Uncertainty

Six respondents loaded in frame NL4. We coded two respondents as belonging to the world of science (researchers) and four as belonging to the world of politics (public policymakers, water manager). Frames NL4 and NL2 have in common a strong preference for network governance, cooperation and integrated policymaking among administrative levels. Like frame NL1, frame NL4 does not agree that water management problems are more socio-political than technological-infrastructural in nature.

Table 7. Distinguished statements about IWRM in frame NL4.

Number	Statement	Ranking
23	The key problems in water management today are more socio-political than technological–infrastructural in nature.	−1
4	There are significant uncertainties about the local and regional impacts of global climate change.	+3
17	There is a need to collaboratively find local solutions to water problems (flooding, draughts, pollution, etc.).	+2
9	A network type of governance, with interaction between interdependent stakeholders, is crucial for water management.	+3
11	A strong degree of integration of water management and spatial planning at different administrative and spatial levels is crucial for water management.	+3
12	A strong degree of cooperation among public water management authorities is crucial for water management.	+3

shows the most distinguished statements about IWRM in this frame.

The main distinction between frame NL4 and the other frames lies in the emphasis put upon uncertainty and the interactions between the global and the local system. The respondents in frame NL4 markedly agree that the local and regional impacts of climate change are very uncertain and that this makes integration between planning scales and water management authorities necessary. This becomes clear in the interviews:

“There are a lot of technical uncertainties and they are rarely communicated to policymakers. At the same time, decision makers don’t like to take uncertainty into their policy. This brings the risk that we spend a lot of money on analysing the measures, which may not be as useful as we think. More effort should be made to increase the communication of uncertainty to decision makers. In this way, decisions can be made in a more robust and flexible way to deal with uncertain situations, instead of aiming to reach the number that indicates the coming water level, a goal that can be both unrealistic and risky.”

(Interviewee No. 15).

Respondents in this frame have a strong systems orientation towards policy analysis. They like to see the big picture and the longer term future. In contrast to frame NL3, they do not attach much value to learning from MSG, but they do seem to approach MSG as a kind of “integrated assessment”. The respondents are very aware that politicians have a limited capacity to incorporate scientific information in policymaking, and that this is a problem. Enhanced cooperation and communication between the world of science and the world of politics is necessary.

Table 8. Distinguished statements about MSG in frame NL4.

Number	Statement	Ranking
6	A system approach is useful for water management only when it addresses the techno-physical and socio-political aspects in an integrated fashion.	+2
23	The key function of policy analysis is to support the stakeholders’ learning process.	−3
27	Simulation gaming with real stakeholders as players is a better strategy for the innovative process than using computer simulations in integrated water management.	−2
30	A simulation game with real stakeholders as players is generally more effective to foresee and analyse what can happen in the near future than a computer simulation.	−2
39	Computer simulations can accommodate poorly with conflicting values and interests of stakeholders in water management and water policy.	−2
36	Visualization (e.g., by pictures, animations or 3D graphics) significantly increases the users’ understanding of models and simulations.	+1

shows the most distinguished statements about MSG in this frame.

Quotations about MSG in frame NL4:

“Simulation should not be only technical, but also involve socio-political aspects. Computer simulation can shine a light on the conflicts. If you have a clear view on the social conflicts and values you should be able to put them in the computer simulation as well, in graphics or in other forms. But it has not been done very well yet. A lot of experience of technological development has been gained. However, social simulation is very hard because reading the exact interests of stakeholders is difficult. We don’t know how far computer simulation can go, but we think technologies for such analysis have improved. But there is a lot of room to improve them further.”

(Interviewee No. 4).

“I do think it’s useful to talk to each other and share information and ideas. But it’s only good when you have a good start, already have the information and foundation. For example, the model can show which approach is more promising and do the analysis. In many cases, the information is available. You just need to study more to get it. However, the situation in The Netherlands is that in some areas they really talk too much. They have so many workshops to talk about things that are easier to study by water modelling and analysis. We think they should study more before doing the workshops, do more of the analysis.”

(Interviewee No. 6).

3.1.5. Frame NL5: Science vs. Emotions

Four respondents loaded in frame NL5. We coded three respondents as belonging to the world of science (researchers, consultants) and one as belonging to the world of politics (public policymaker).

Table 9. Distinguished statements about IWRM in frame 5.

Number	Statement	Ranking
2	The key problems in water management today are more socio-political than technological–infrastructural in nature.	+3
12	A strong degree of cooperation among public water management authorities is crucial for water management.	+3
29	Rational thinking should always be combined with human emotions in policy analysis for integrated water management.	+2
16	Reinforcing levees (dikes) etc. is insufficient to keep The Netherlands safe from flooding in the 21st century.	−3

shows the most distinguished statements about IWRM in this frame.

A somewhat cynical attitude towards the science–policy interface and the value of SG emerges in frame NL5. Based upon the interviews, it appears that some of the statements were answered with a kind of alternative interpretation (see quotation below). First, frame NL5 most strongly believes in “reinforcing levees” as the solution to keep The Netherlands from flooding. Secondly, and equally strongly, the frame agrees that problems in water management are more socio-political than techno-infrastructural. It becomes clear from the interviews that respondents in this frame believe that the technical solutions are available and that science has provided most of the answers, but that politicians and societal stakeholders do not listen: they should trust water experts to get the job done, but unfortunately emotions and irrationalities play too big a role. It seems that the respondents have come to accept it:

“Science and knowledge generation are not the problem in the current water decision-making process. In The Netherlands a lot of investigations have been made on scientific research for the long-term water management. The result is based on very good investigation and therefore does not need to be doubted. But on the other hand, the lack of communication of management sectors is the big problem in The Netherlands. A lot of failures to make a decision on a development plan happened due to the lack of willingness to cooperate. It is very often that sectors make plans by themselves; there is not so much communication.”

(Interviewee No. 19).

“Science is no longer taken seriously enough in decision making. Emotion and power dominate the decision-making process. The politicians are not interested in rational evidence. The priority of interest and power determines what will happen. Scientific evidence can help, but it depends on the political situation. It can be easily denied if it does not match the political interest in the problem. We should move back to the situation that the socio-political power does not constrain the technical power”.

(Interviewee No. 11).

If this is the case, what is the role of MSG? In the interviews, some respondents indicated that it could be useful to find a balance between science and emotions. However, it is not clear whether MSG can help to find a balance. For the most part, the respondents seem to have mixed feelings about MSG. They disagree strongly that computer simulations are difficult for policy stakeholders to use and understand. They also highlight the importance of visualization to increase policymakers’ understanding of models and simulation. Low-technology games are not scientifically valued (see

Table 10. Distinguished statements about MSG in frame NL5.

Number	Statement	Ranking
28	The outcomes of computer simulation are generally more authoritative (trustworthy) for water policymakers and water managers than the outcomes of a simulation game with real stakeholders.	−3
31	Policy simulation does not need to be computerized. Low-tech gaming based on human behaviour is also a scientifically proven method for water policy analysis.	−1
33	The process of decision making simulated by human players in a gaming environment is generally more useful for learning than for real policy analysis.	−2
36	Visualization (e.g., by pictures, animations or 3D graphics) significantly increases the users’ understanding of models and simulations.	+3
38	Computer simulations in water management are generally difficult to use and understand by policy stakeholders.	−3

). From the discussions, it seems that gaming is regarded as useful to reflect the human emotions (but not to change them) and it needs the support of technology to gain the trust of players if it tries to simulate the reality (see quotations below).

Quotations about MSG in frame NL5:

“For me, the concept of gaming means computer model based, role playing, rules and group activity. […] We never use a game in a real decision-making process. We use games in academic exercises. With the students the experience often shows the non-rational outcome, which is not what we expected. The decision always depends on the political and social power of some of the roles. We think the reason behind it is that people are selfish. If they are powerful enough they will push their selfish interest. In such a situation, gaming does not help at all. So we think gaming can help to make a quicker decision, but it does not help to make a better decision.”

(Interviewee No. 11).

“The dilemma is that gaming works with respondents who are willing to be involved and communicate. But if the respondents are already open and willing to interact, is the value of gaming still significant, considering the time and money consuming process to organize it?”

(Interviewee No. 9).

3.1.6. Intermediate Conclusions

Through the analysis of data, we found some rationality between how the respondents view IWRM and the role that MSG can or should play. The familiarity, experience with and trust in MSG was generally high.

Table A1. The frames in key words.

The Netherlands
Frame	Bureaucratic Alignment	Stakeholder Interaction	Learning	Uncertainty	Science vs. Emotions
Policy-science interface	Power, authority, multi-level public governance.	Policy networks, Stakeholder interdependency.	Social-technical complexity; social dimension of technology.	Future and complex system orientation.	Engineering and science.
IWRM	Engineering solutions into (local, regional) planning; Alignment between administrative authorities and levels.	Stakeholder views, interests; interdependencies.	Social and technical dimensions of water management.	Local and global integrated future perspectives (climate change);	Science has to deal with irrationality, and emotions.
MSG	Minor relevance; Moderate trust in games; Only as supportive simulation and visualization technology.	Moderate trust and relevance of social games and simulations for better network interaction, trust, collaboration.	High trust and relevance of interactive, social-technical simulations and games for social learning.	Moderate trust and relevance of MSG as integrated assessment, analysis of possible futures.	Not much relevance of MSG; perhaps as a way to deal with public emotions.
Respondents (33)	33% (11 persons)	27% (9 persons)	9% (3 persons)	18% (6 persons)	12% (4 persons)
China
Frame	Doctrine of the Mean	Modern and Rational Governors	Open Minded Reformers
Policy-science interface	Centralization, hierarchy; policy analysis not very relevant.	Network governance; Rational, evidence-based decision-making.	Network type of governance; Science-based but open for social dimensions.
IWRM	Better co-operation with regional and local authorities.	Sectoral and expert integration.	Integration of more perspectives, disciplines, possibly stakeholders.
MSG	No trust and relevance in MSG. Support and legitimation of policy with data driven scientific computer models and simulations.	Low trust and relevance of MSG for different purposes. May be good for experts, not public or stakeholders.	Moderate trust and relevance of MSG. May be useful for broad array of uses, possibly social learning.
Respondents (22)	54% (12 persons)	18% (4 respondents)	27% (6 persons)

present an overview of the five frames. They are separated in their views on governance (hierarchy, power, and public participation?), IWRM (what is integrated? administration, sectors, disciplines, and stakeholders?), the role of science in policy making (science or evidence based, political power play, social and human factors, and role of emotions). These factors correspond with different views on the role of MSG: (1) strengthen the evidence and science base of decision-making; (2) strengthen the interaction and involvement of stakeholders; (3) for learning; (4) for integrated, long term assessment; and (5) for dealing with public emotions.

3.2. Frames in China

The twenty-two respondents in China were quite unfamiliar with MSG and therefore either curious, or skeptical about the use of MSG for IWRM. To facilitate the Q-method, we decided to show a few good international MSG examples before the interview. We identified three Chinese frames:

• Frame CH1: Doctrine of the mean
• Frame CH2: Modern and rational governors
• Frame CH3: The open-minded reformer

3.2.1. Frame CH1: The Doctrine of the Mean

This frame is shared by 12 respondents, the biggest group among the four frames. We coded all of them as belonging to the world of politics (provincial and local water managers). The attitude in this frame provides quite a good understanding of the values in the view of the doctrine of the mean: “Say as little as possible while knowing perfectly well what is wrong, to be worldly wise and play it safe”.

Table 11. Distinguished statements about IWRM in frame CH1.

Number	Statement	Ranking
2	The key problems in water management today are more socio-political than technological-infrastructural in nature.	+2
6	A centralized form of governance, with sufficient authority and decision power at the national level, is crucial for water management.	+2
7	A network type of governance, with interaction between interdependent stakeholders, is crucial for water management.	+1
8	There is a need for methods that can enhance the cooperation among different sectors and levels of governance in water management.	+2
9	There is a need for methods that can analyse the conflicts and cooperation among different regions in river basins.	0

shows the most distinguished statements about IWRM in this frame. The best form of governance is for central government to have a strong leading position, and more interaction and cooperation among different levels of government sectors. Methods to enhance cooperation are needed, but methods to analyze conflicts are not needed. The topics related to the science–policy interface, such as the impact of uncertainty and fragmentation, and matters like stakeholder participation and collaborative learning for policy analysis, are regarded as unimportant or are strongly rejected (see Table 11).

Another interviewee in this frame admitted quite honestly that the interaction among government sectors is more of a network routine than for meaningful policy analysis (see quotation below).

“Interactive, participatory policy analysis in China is still more of a ‘lip service’.”

(Interviewee No. 11).

“The focus of water management and flood control in China is still on the development of infrastructure. […] With such a focus, a centralized government is efficient. Enhancing the cooperation among sectors increases the efficiency of management. However, the advanced development in Western countries is dependent not on participation and social interaction, but on the standardization of rules. The standardization in China is still at a low level. This is the critical reason for the problems in water management.”

(Interviewee No. 7).

Table 12. Distinguished statements about MSG in frame CH1.

Number	Statement	Ranking
19	By letting stakeholders play their own role (interests, behaviour, etc.) in a gaming environment, we can simulate real problems and solutions in water management and derive valuable insights for water policymaking.	−2
25	Policy simulation does not need to be computerized. Low-tech gaming based on human behaviour is also a scientifically proven method for water policy analysis.	−2
34	Playing together in a simulation game increases the stakeholders’ willingness to cooperate in the real world.	−2
33	Simulation gaming can effectively facilitate and support the interaction among stakeholders from different governance sectors.	−2
22	The outcomes of computer simulation are generally more authoritative (trustworthy) for water policymakers and water managers than the outcomes of a simulation game with real stakeholders.	+1

presents the distinguished statements about MSG. It is quite obvious that in this frame MSG is neither trusted nor preferred. All the key statements about the values of MSG for policymaking, whether for providing real-life insights or for increasing cooperation, are strongly rejected. Low-tech games are certainly not a scientific method. An integrated computer model—data driven—is more trustworthy for analysis.

MSG is regarded as too Western or too early for the complex Chinese politics. This is especially the situation in the regional and local areas. The culture gap can also explain the distrust of MSG, as expressed by two of the interviewees (see quotation below).

Quotations about MSG in frame CH1

“The situation of ‘treatment after pollution’ is not avoidable in the developing process, which the developed countries also experienced. […] However, whether the Western method of such participatory role playing game is also useful for the ‘Chinese solution’ is still too early to see.”

(Interviewee No. 18).

“The power relation and strategic game in China’s policy environment is deeply embedded in its routine. Chinese politicians follow ‘the doctrine of the mean’ to be able to survive in the environment, which makes it impossible for them to articulate their needs and interests, and express their emotions. The Chinese political game contains many uncertain and un-parameterized variables to design a game for.”

(Interviewee No. 22).

3.2.2. Frame CH2: Modern and Rational Governors

Five respondents loaded in frame CH3; two of them are coded as belonging to the world of science (researchers), while the other two belong to the world of politics (regional and local policymakers). In contrast to the other frames, the five respondents in this frame gave the impression that they appreciate a network type of governance. We label this frame the “modern and rational governors”: they modestly agree on the importance of socio-political problems in water management, which reflects their modern outlook. They are also rational governors, because they rely on science and strongly disagree with the contribution of public participation to solving the problem and consequences of climate change. They reject the suggestion that policy analysis should involve human and social aspects.

Table 13. Distinguished statements about IWRM in frame CH3.

Number	Statement	Ranking
6	A centralized form of governance, with sufficient authority and decision power at the national level, is crucial for water management.	−2
2	The key problems in water management today are more socio-political than technological-infrastructural in nature.	+1
7	A network type of governance, with interaction between interdependent stakeholders, is crucial for water management.	+1
8	There is a need for methods that can enhance the cooperation among different sectors and levels of governance in water management.	+2
16	The key solution to the consequences of climate change lies in active public involvement and stakeholder participation. Societal interaction will provide the most significant contribution to water management and policymaking in the near future.	−2
23	Rational thinking should always be combined with human emotions in policy analysis for integrated water management.	−2

shows the most distinguished statements about IWRM in this frame.

Quotations below gives a respondent’s thought on the urgent needs to develop a rational, science-based cooperative government.

“Rational, science-based governance is urgently needed to improve the efficiency of management. However, the bottom-up type of social participation is not a suitable method due to the very complex social situation in China. It will lead to a big crisis and loss of control when too much emotion is allowed in the policy analysis process. A good governmental regulation system based on rational priorities is the proper way to achieve better water management.”

(Interviewee No. 4; water manager).

Frame CH2 also agrees on such things as the need for social simulation, the limitation of using computer modelling alone, the importance of visualization and using MSG for learning rather than for policy analysis (see

Table 14. Distinguished statements about MSG in frame CH2.

Number	Statement	Ranking
15	There is a need for socio-political simulations that provide valuable insights into the multi-actor complexity of water management.	+2
18	Most computer models are not flexible enough to deal with complex water problems. Models that can be quickly developed and changed to fit the circumstances are needed.	+2
25	Policy simulation does not need to be computerized. Low-tech gaming based on human behaviour is also a scientifically proven method for water policy analysis.	+1
30	Computer simulations for water management and water policymaking should be easy to use and understand by non-expert users.	+1
26	It is not enough to rely on computer simulation for the exploration of policy problems and the testing of policy options (even they have been developed on the basis of best-available scientific knowledge).	+1
29	Visualization (e.g., by pictures, animations or 3D graphics) significantly increases the users’ understanding of models and simulations.	+1
27	The process of decision making simulated by human players in a gaming environment is generally more useful for learning than for real policy analysis.	+2

).

Quotations about MSG in frame CH2

“The development and application of integrated models is quite advanced due to the large investment from the national government. However, developing socio-political simulation is a different topic. In developed countries such as The Netherlands, they are interested because the development of infrastructures is completed. Water management can now focus more on the small-scale, ‘soft’ issues and use the more ‘soft integrated’ approaches such as gaming for less urgent issues in a long-term perspective. It is important to address the long-term planning in water management, but in China the more urgent issue is developing infrastructures, especially in the northwest area. Gaming will not be considered in these tasks. It is useful to learn new perspectives in policy analysis, but only after the fundamental structure has been completed.”

(Interviewee No. 8).

3.2.3. Frame CH3: The Open-Minded Reformers

Six respondents loaded in frame CH3; two of them also loaded on frame CH1. We coded two respondent as belonging to the world of science (researcher) and four as belonging to the world of politics (public policymaker).

Table 15. Distinguished statements about IWRM in frame CH3.

Number	Statement	Ranking
4	Uncertainty in water management is deepened by a lack of integration among social, political, technological, ecological, economic, etc. knowledge.	−2
6	A centralized form of governance, with sufficient authority and decision power at the national level, is crucial for water management.	0
7	A network type of governance, with interaction between interdependent stakeholders, is crucial for water management.	+1
2	The key problems in water management today are more socio-political than technological-infrastructural in nature.	−1

shows the most distinguished statements about IWRM in this frame. The views of these open-minded reformers.

This frame shows quite strong agreement with the use and usefulness of MSG for IWRM in China. Games are regarded a more social-innovative strategy for IWRM than computer simulation, and it can bring valuable insight for real-life policymaking. Besides the scientific value, MSG is also regarded useful for learning, increasing the real-life communication and willingness of collaboration among government sectors (see

Table 16. Distinguished statements about MSG in frame CH3.

Number	Statement	Ranking
19	By letting stakeholders play their own role (interests, behaviour, etc.) in a gaming environment, we can simulate real problems and solutions in water management and derive valuable insights for water policymaking.	+2
21	Simulation gaming with real stakeholders as players is a better strategy for the innovative process than using computer simulations in integrated water management.	+2
22	The outcomes of computer simulations are generally more authoritative (trustworthy) for water policymakers and water managers than the outcomes of a simulation game with real stakeholders.	−2
25	Policy simulation does not need to be computerized. Low-tech gaming based on human behaviour is also a scientifically proven method for water policy analysis.	+1
32	Simulation gaming with real stakeholders as players integrates ‘soft knowledge’ from stakeholders with ‘hard knowledge’ from scientific research.	+2
34	Playing together in a simulation game increases the stakeholders’ willingness to cooperate in the real world.	+2
33	Simulation gaming can effectively facilitate and support the interaction among stakeholders from different governance sectors.	+2
27	The process of decision making simulated by human players in a gaming environment is generally more useful for learning than for real policy analysis.	−2

).

Quotations about MSG in frame CH3

“Developing methods and technology for socio-technical integration in China is only a matter of time. […] In China there are already some demonstration projects going on at the national level, big institutions. However it is still quite new and needs more time to be introduced to the local level […]. The technology in China has been developing quite rapidly in recent years. We now have a lot of advanced 3D visualization technology and integrated simulation models. They are used successfully in technological control and management in large-scale infrastructures. So far, however, there has not been much convincing evidence […]”

(Interviewee No. 5; senior researcher).

4. Conclusions and Discussion

The empirical studies in this manuscript provide insights into the use and usefulness of MSG from a macro level. Not from one case or one experiment, but from the generalized overviews at the higher level of institutions and political system, where knowledge, expertise, interests, power and stakes are interwoven at the science–policy interface.

Comparative analysis of the combined quantitative and qualitative data show that the debate on MSG for IWRM is structured around five frames in The Netherlands and three frames in China. Table A1 gives an overview of the frames in key words. The frames look at IWRM regimes—control vs. adaptive—through different lenses that are internally consistent in the way they perceive MSG at the science–policy interface. In and between the frames, there are significant differences. The words that are used—such as integration, stakeholder involvement, visualization, game, and evidence-based—might be the same, but they have very different meanings and connotations—even within the same country.

It is difficult to compare the frames in The Netherlands and China. Many respondents, also in China, agree that IWRM is a general trend in both countries and also globally. Water management is moving from its hard-core engineering domain to become an inherent part of integrated spatial development in order to achieve the ecology-based, sustainable, risk-involved management of water resource. However, the pathway to IWRM in both countries will be curved differently. There is also a similar “skeptic” tendency in both countries which has to do with the respondents’ view on whether political reality can, and should, be influenced by science, evidence, participation and learning. In The Netherlands, there is a growing frame that perceives MSG for water management as “data driven simulation and visualization technology” especially in light of emerging trends in systems science and AI. A similar trend is noticeable in China, however there it also seems to have a function as political legitimation—the appearance of being modern, science-based. However, the use of MSG for understanding and managing complex systems, social learning or as persuasive technology, are significant frames in The Netherlands, but not as yet in China.

The findings and conclusions have scientific and practical relevance. First, the Q-method was found to be a valuable method for comparative, cross cultural research around topics that show a diversity of understandings, opinions and values. In countries such as China, officials are not very inclined to give interviews and speak openly. The Q method proved practical and effective, because it was novel, gave just enough structure and soon proved fun to do. The definition of a stable Q-set of statements that can be used in multiple contexts, requires pre-study validation efforts to avoid that the statement set needs intermediate revisions.

Second, many Dutch companies are now taking their advanced IWRM approaches and MSG methods across the globe, especially to Asia, and into China. Frame differences can easily confuse professional and academic debate about MSG for policy making and IWRM; within the same institutional and cultural context, but even more so in Netherlands–China water management co-operation projects. Frames are relevant when designing, using or evaluating innovative methods for IWRM. MSG are now increasingly advocated for social learning in a context of natural resource management. Learning to understand, design and work with multiple frames is a prerequisite for the relevance and impact of innovations in IWRM and MSG.