Use and Utility: Exploring the Diversity and Design of Water Models at the Science-Policy Interface

Abstract

Effort to narrow the gap between the production and use of scientific knowledge for environmental decision-making is gaining traction, yet in practice, supply and demand remains largely unbalanced. A qualitative study based on empirical analysis offers a novel approach to exploring key factors, focussing on seven water models in the context of two organisations at the science-policy interface: the PIREN-Seine in France and the CRC for Water Sensitive Cities in Australia. Tentative linkages drawn from these examples identify: (1) objective and expertise; (2) knowledge and tools; and (3) support structures as main drivers influencing the production of scientific knowledge which, in turn, affect the use and utility of modelling tools. Further insight is gained by highlighting the wide spectrum of uses and utilities existing in practice, suggesting that such ‘boundary organisations’ facilitate interactions and exchanges that give added value to scientific knowledge. Coordinated strategies that integrate inter-, extra-, and intra-boundary activities, framed through collaborative scenario building and the use of interactive modelling platforms, may offer ways to enhance the use and utility of scientific knowledge (and its tools) to better support water resources management, policy and planning decisions, thus promoting a more cohesive relationship between science and policy.

1. Introduction

The trade-off between scientific complexity and ‘usability’ of scientific knowledge and tools to support management, policy and planning decisions is a fundamental question at the heart of the science-policy interface. Similar to all areas of environmental decision-making, water resources managers must make decisions under high system complexity and uncertainty, which demands effective integration of useful and relevant scientific information [1]. In this context, ‘useful’ scientific knowledge possesses a utilitarian function by clarifying and expanding different options for decision makers to achieve desired outcomes [2,3] and must also be perceived as credible (reliable and of high quality), relevant (context-specific) and legitimate (transparent and objective) [4,5,6,7,8].

The myriad of challenges and opportunities affecting usability have been well documented in the literature, summarized by Lemos et al. [9] as a function of the interconnected factors of fit, interplay and interaction. Elucidating the complexity of these dynamics requires a departure from the traditional ‘linear’ model of research use, where scientists produce scientific information, viewed as objective or neutral facts, which are then transmitted to a passive audience [10]. Instead, use and utility should be understood as the product of a complex and nuanced relationship comprised of mediated interactions between the various actors involved. Accordingly, the science-policy interface represents a set of social processes between scientists and decision makers, which facilitates the exchange and co-construction of knowledge to support decision-making [11], while also taking into account the complex, iterative, and selective nature of the decision making process [12].

A growing body of work dedicated to the subject defines the science-policy interface in terms of ‘boundaries’ [6,13,14], which ‘demarcate the socially constructed and negotiated borders between science and policy, between disciplines, across nations and across multiple levels’ ([5] (p.1)). ‘Boundary organisations,’ are intermediary organisations straddling the frontiers of science and policy through the co-production of shared interests, knowledge and tools [6], that can facilitate and/or hinder communication, collaboration and collective action [5]. Touted by some as promoting the best of both worlds, others remain cautious of how we distinguish ‘science’ from ‘non-science’ within these arrangements, otherwise known as the ‘boundary problem’ [14]. In the same vein, Jasanoff [13,15] argues that scientific claims are socially constructed through various social influences and constraints, which can place unusual strains on science when applied to real-world situations. While concerns over the bureaucratisation or standardisation of science are certainly valid, Guston [6] maintains that boundary organisations can help to avoid these issues, by having one foot in science and the other in policy, thereby keeping one another in check.

Within this discourse, modelling tools can be considered ‘boundary objects’ [16,17], which serve to deepen scientific understanding, while concurrently supporting key management, policy and planning decisions [1,18]. Their dual function as a research and an operational tool has enabled practitioners to navigate the complexities of water resources management and planning, which demands not only a nuanced understanding of dynamic environmental processes but also the ability to negotiate trade-offs between a multitude of social, economic, political and ecological interests among competing stakeholders. On the other hand, models have different forms and functions, not all of which are equal in terms of: (1) their use, i.e., ‘the method or manner of employing or applying something’; and (2) their utility, i.e., its ‘fitness for some purpose or worth to some end’ or ‘something useful or designed for use’ [19].

While considerable efforts have been made to bridge the gap between the production and use of scientific knowledge in decision making [5,9,20,21,22,23,24,25,26], many authors continue to highlight a mismatch between supply and demand [1,2,10,12,20,27,28,29], suggesting the need for further insight into the production as well as the use and utility of such knowledge and tools in practice. Much of the existing literature on creating ‘usable’ science focuses on opportunities and challenges without delving into what exactly this information is used or useful for in the context of environmental decision making. To date, discussion on model complexity vs. usability has been largely based on the notion that a model’s use and utility is contingent upon its ‘usability’ (e.g., user-friendly interface, simplified processes and outputs, etc.). However, this overlooks the multitude of uses (ranging from direct to indirect), which exist in practice. Here, we distinguish ‘utility’ from what others have referred to as ‘usability’ [9] in an effort to incorporate this diversity. Within this literature, boundary organisations have been identified as an effective strategy for producing knowledge that is both useful and usable for decision making [9,27,30,31,32], yet there is still a lack of empirical data to reinforce this hypothesis. In an effort to address these gaps, this paper aims to provide further insight by using a novel approach based on empirical analysis to explore the boundary organisation hypothesis: the way an organisation or a (set of) tools is structured can help or hinder the production of scientific knowledge that is perceived as valuable for the implementation (or elaboration) of public policies. We explore this hypothesis, focussing on the use and utility of modelling tools within the context of two interdisciplinary research programs whose core activities are rooted in research-industry collaboration (public or private): the PIREN-Seine (Programme Interdisciplinaire de Recherche sur l’Environnement de la Seine) in France and the CRC (Cooperative Research Centre) for Water Sensitive Cities in Australia.

The choice of these examples derived from a desire to compare two exemplary experiences, which share the overall objective of improving collaboration and exchange at the science-policy interface. Specifically, both aim to address challenges of water resources management, policy and planning through the advancement of scientific knowledge and the development of modelling tools in partnership with various stakeholders. These challenges include technical factors, such as model complexity, uncertainty and the availability and reliability of data, as well as socio-economic factors such as institutional barriers and paradigms, competing objectives, time and resource constraints and lack of effective communication and understanding. However, they approach these challenges using strategies that are fundamentally different: one being more ‘research-oriented,’ while the other is more ‘industry-oriented.’ On one hand, the PIREN-Seine in France favours models with more scientific rigour at the cost of usability for industry partners. On the other, the CRC for Water Sensitive Cities in Australia is developing modelling tools designed for industry use, though it remains to be seen whether they will be readily adopted. The breadth and diversity of modelling tools represented in both examples provides a sufficient dataset with which to draw from, while the openness and transparency of these programs allowed for the collection of empirical data, which can be considered an adequate representation of reality. As both programs use modelling tools developed (or partially developed) outside of their defined ‘boundaries,’ we are also able to go beyond the two case studies to explore the legacy of seven water models across two countries. Finally, the diversity of modelling tools found in both examples represents different stages of model development and use, thereby giving further insight into current and potential use and utility.

Through an empirical analysis of the PIREN-Seine in France and the CRC for Water Sensitive Cities in Australia, this paper aims to narrow the gap between the production and use of scientific knowledge by exploring the nuanced relationship between the use and utility of modelling tools within boundary organisations at the science-policy interface. Section 2 presents the methods and materials used to inform this analysis, as well as the framework for discussion. Based on Grounded Theory (GT) [33], our approach is an exploration of the factors influencing the use and utility of modelling tools, using empirical data as a starting point. Through an historical perspective, Section 3 offers a comprehensive characterisation of the different strategies implemented by the two organisations in order to enhance utility. Brief descriptions of seven water models will be presented to provide context for the discussion that follows. Section 4 explores the links between the respective strategies and their effect on the use and utility of these models for decision-making. We deepen this discussion in Section 5, by characterising the different types of use and utility represented in the two examples. By delving into these specificities, we highlight the influence of model use (direct or indirect) on its utility and vice versa. Moving past the assumption that knowledge is useful only when it is used, we posit that the social value of this knowledge is also derived from the different types of interactions and exchanges existing between the complex, dynamic web of science-policy boundaries. Finally, we arrive at the conclusion that the use and utility of scientific knowledge (and its tools) could be enhanced through coordinated strategies which frame these inter-, intra- and extra-boundary exchanges and interactions through the co-construction of scenarios and the use of interactive modelling platforms.

2. Materials and Methods

We conducted a qualitative study using an approach based on Grounded Theory (GT), a general research methodology that derives theory through the systematic collection and analysis of data [33,34,35]. Rather than having an established framework or theory from the outset with which to test against research data, this method offers a more flexible, adaptive approach through an iterative process that involves: raising generative (but not static or confining) questions to guide research, identifying core theoretical concepts through the systematic collection and analysis of data, and developing tentative linkages between core concepts and data [35]. This approach allows for an exploration (and subsequent identification) of the factors influencing the use and utility of modelling tools through:

• A characterisation of the strategies implemented by the two organisations and a description of the different types of modelling tools, which is used to explore the influence of these strategies and the potential use and utility embedded in the structure of the model (Section 3);
• Systematic observation and analysis of the interactions and perceptions of the different producers and users of modelling tools, which allows us to form tentative linkages (Section 4) and;
• A characterisation of the different model uses (ranging from direct to indirect), which is shown to inform their utility (and vice versa) (Section 5).

Our analysis draws primarily on systematic document analysis (e.g., activity reports, scientific literature produced by both programs), formal semi-structured interviews with researchers and practitioners from both countries, and observations during science-practice engagement activities. This provided a rich data set for comparing PIREN-Seine in France and CRC for Water Sensitive Cities in Australia.

2.1. Document Analysis

Document analysis focused on the work produced by the PIREN-Seine and the CRC for Water Sensitive Cities throughout the duration of each program, which included hundreds of peer-reviewed journal articles as well as grey literature such as periodic activity reports (over 700 reports from the PIREN-Seine and over 150 from the CRC for Water Sensitive Cities), synthesis documents and other communications. As a lot of the modelling in the Australian urban water sector also emerged from a long legacy of research and industry collaboration dating back to the 1990s, we were also cognisant of older documents prior to the commencement of the CRC for Water Sensitive Cities research program including those from its predecessors, the CRC for Catchment Hydrology, the CRC for Freshwater Ecology and the eWater CRC. Pertinent documents were identified by searching different combinations of the following keywords: ‘decision making’; ‘exploratory modelling’; ‘management’; ‘model’; ‘modelling’; ‘planning’; ‘policy’; ‘strategic planning’; ‘water sensitive cities’ and ‘water sensitive urban design’ on each program’s website in addition to major search engines (Scopus, Web of Science, Google Scholar). Keywords were selected and narrowed down from initial searches based on relevance to the respective research program and included the names of specific ‘operational’ or industry partners (practitioners) and known modelling tools in order to obtain information about their use and application in practice.

Though this process was systematic, the permeable nature of the ‘boundaries’ between science and policy limited our ability to adequately define the models represented in this study. First, there is no clear consensus regarding ownership. While for some, it is a question of licencing and rights, for others, the model developer is considered the ‘owner,’ since they have the ability to change the code. Second, model development is typically a long process, where different actors may be involved in some capacity at various stages, contributing to its overall development and evolution. This work can be carried out under the auspices of the boundary organisation or it can be done through external contracts or exchanges. Third, these models are not entirely independent. That is to say, they often include modules or sub-modules that were developed outside of the program. In some instances, models were created in other research contexts and were subsequently developed, further elaborated and maintained by the program. With these limitations in mind, we refer to ‘PIREN support tools’ or ‘Water Sensitive City (WSC) tools’ to distinguish modelling tools that were developed, used and supported within these two contexts to conduct research associated with the respective program. To capture (as much as possible) the breadth and diversity of modelling tools represented in both cases, we took a broad definition of ‘model’ to mean any model, modelling tool, or part of a modelling tool mentioned in the documents produced by either program, that was either developed or used at one time or another by a researcher of that program. Under this definition, a model can also refer to a sub-model or module that can simulate biophysical or chemical processes using mathematical equations and numeric calculations.

2.2. Semi-Structured Interviews and Observation of Engagement

Since what is written and officially communicated is not necessarily what is said and done in practice; observation and semi-structured exploratory interviews were implemented to support initial findings in both France and Australia. A total of 36 and 21 interviews were conducted in France and Australia respectively with researchers (including modellers and non-modellers) and practitioners (including modellers, water authorities, consultants, regulating authorities and government officials) who were either previously or are currently involved (both directly or indirectly) with modelling activities within these two contexts. Interviews were semi-structured, based on a general question guide (provided in Annex A) that focussed on themes relating to: (1) the development and use of modelling tools; (2) the relationship between researchers and partners; (3) the regional context; as well as (4) the objectives and themes of the respective research program. Questions were adapted to individual participants according to their role and involvement in modelling activities, the program, or their position. Interviews were open-ended and lasted anywhere from 1 to 4 h with an average duration of 1.5 h. Interviews were transcribed and coded according to the four themes listed above. Anecdotal observations were used as secondary data, which was collected throughout 2015–2017, during numerous meetings, seminars and conferences organised by the PIREN-Seine in France. This included two general assembly and annual planning meetings organised to reflect on the year’s work and co-define upcoming program objectives. In Australia, anecdotal observation was limited to seminars and conferences organized by the CRC for Water Sensitive Cities from May to August 2017, which included one major national conference in Perth and two workshops.

3. Retracing the History of (Co-) Production in France and Australia

The PIREN-Seine and the CRC for Water Sensitive Cities provide a platform for researchers and practitioners to collectively address some of the key issues of water resources management, policy and planning, using different strategies to achieve a common objective. As its name suggests, the PIREN-Seine focuses on the Seine River basin in France, while the CRC for Water Sensitive Cities extends its focus across cities to include the Yarra, Swan-Canning and Brisbane river basins in Australia, which represent notable examples of historically significant catchments facing serious issues of water quality and quantity due to increasing anthropogenic pressures caused by rapid urbanisation, population growth and climate change.

The PIREN-Seine has adopted a territorial perspective of the Seine River basin, with a desire to understand the ecological functioning of the entire watershed in relation to human activities [36,37]. Most of the research is centred on issues of water quality, though water quantity concerns are also explored (mostly from a quality perspective), particularly in light of recent major flood events. While industry collaboration is considered an essential part of the program, the intrinsic desire to maintain scientific integrity is reflected in the knowledge and tools produced, which have traditionally leaned towards academic pursuits as a primary function and responding to operational demands as secondary. As a result, modelling tools are primarily seen as ‘research tools,’ which have been used to support management and planning decisions, though the tools themselves have only been adopted by industry partners in exceptional cases of mutual interest and investment.

In contrast, the CRC for Water Sensitive Cities focuses on issues of urban water management in cities throughout Australia and abroad in pursuit of sustainability, resilience and liveability [38,39]. This has been partially motivated by extreme weather conditions experienced within the region, such as the Millennium Drought [40,41,42], which lasted more than a decade and has shifted the primary focus towards issues of water supply security (e.g., seawater desalination, rain water harvesting) even though water quality remains a serious concern, particularly for recreation and consumption [43,44]. Direct uptake of research into practice being the main objective, a large part of this work has been devoted to adoption pathways and socio-technical transitions, resulting in tools that lean towards practical application as a primary function. While this has proved successful in some cases, leading to wide-scale adoption of one example (i.e., the MUSIC model) that we feature in this study, it remains to be seen whether the new generation of modelling tools will be able to generate the same appeal. The contrasting strategies and diversity of models at different stages of development represented by these examples makes for a fruitful comparison for exploring how organisational configurations and context-specific drivers may influence the production of knowledge and tools within these spaces and what that means in terms of use and utility. A summary of the two research programs is presented in

Table 1. Summary of Research Programs.

	PIREN-Seine	CRC for Water Sensitive Cities
Duration	(1989–)	(2012–2021)
Level/Scale	Territory; Basin	Urban; City
Interest	Seine River basin	Cities in Australia and abroad
Research Priority	Quality/Quantity	Quantity/Quality
Main Objective	To produce research to better understand river system functioning that can also support decisions	To produce research and tools for industry use to achieve water sensitive cities
Types of Actors	National research institutes, universities, mixed research groups, research laboratories, public institutions, regulating authorities	Universities, public utilities, governments (local, state), regulating authorities, capacity-building organisations, consulting companies, software companies

below.

As the PIREN-Seine and the CRC for Water Sensitive Cities both position themselves at the science-policy interface, a comparison between the two presents a mutual learning opportunity: the CRC for Water Sensitive Cities can benefit from the nearly 30 years of experience from the PIREN-Seine, while the PIREN-Seine can gain insight from an international perspective. Additionally, this analysis can provide guidance for similar examples on a wider scale: The Seine River basin is facing strong anthropogenic pressures that are characteristic of many large watersheds, while Australia can be considered a ‘litmus test’ for other countries as it continues to face extreme weather conditions that may soon become the norm under climate change.

3.1. The PIREN-Seine, France

Established in 1989, the PIREN-Seine (PIREN) is an interdisciplinary research program in France comprised of 22 research teams and 140 researchers from a range of academic backgrounds. The majority are rooted in the fields of hydrology, biology, chemistry, or engineering, while a growing number of geographers, agronomists, political economists, political scientists and sociologists have become involved. The main types of actors in relation to modelling activities are represented in Table 1. Notable industry partners include the Syndicat Interdépartemental pour l’Assainissement de l’Agglomération Parisienne (SIAAP), a public institution responsible for wastewater treatment and sanitation in the Paris region and the Agence de l’Eau Seine-Normandie (AESN), a public institution responsible for the management of water resources in the Seine-Normandy watershed, both of whom, are heavily involved in modelling activities within the PIREN-Seine.

Partnerships between universities, research units and research institutions not only provide a pool of expertise, they can also be a source of funding, either through specific projects that directly or indirectly contribute to the work of the PIREN-Seine or through in-kind contributions in the form of researchers who are paid by their own institutions or doctoral students and post-doctoral researchers who support them. As for industry partners, relationships are largely financial, allowing them direct access to the knowledge and tools produced by the PIREN-Seine. They also play an active role in the elaboration of the program’s research objectives and, in some cases, the modelling tools as well. In the case of the regulating authority—the Direction Régionale et Interdépartementale de l’Environnement et de l’Énergie (DRIEE)—the relationship has an added regulatory element. While each organisation has a defined role within the basin, individual relationships are not clearly defined, as many researchers and industry partners have formal and informal relationships that extend beyond the ‘borders’ of the program. For example, several individuals who have previously obtained their doctoral degree under the supervision of PIREN-Seine researchers now represent industry partners. Furthermore, a model that may have been developed within the context of the PIREN-Seine may see further development outside of the program through external contracts with individual researchers, research teams or even external consultancies.

Over the past three decades, the objectives and research themes of PIREN have evolved in response to changing research and operational needs and emerging trends, while gradually incorporating new disciplines and perspectives [45], which also went hand-in-hand with the development and evolution of modelling tools. Phase 1 (1989–1992), emerged from the need to create dialogue and fundamental partnerships between researchers and water actors as a prerequisite for mobilising research that could address specific water quality concerns at a territorial scale. Initial objectives soon evolved towards obtaining a more global vision that encompasses the entire river basin, a mentality that echoed the 1992 Water Act [46] and the Master Plan for Water Development and Management (SDAGE) [47]. Whereas Phase 1 looked at the longitudinal dimension of the aquatic continuum (upstream–downstream), Phase 2 (1992–1996) turned its attention to transverse interactions between watercourses and riparian zones such as wetlands, as well as the urban water cycle and the fate of pollutants in the river system. It is within this phase where the perception of models began to change from being seen as strictly research tools to their consideration for decision support.

From 1998 to 2006, work in Phases 3 and 4 aimed to contextualize the hydrographic network within the different interactions and anthropogenic influences occurring within the watershed. A retrospective outlook was used to consider the historic and dynamic nature of the hydrological system, which in turn increased the capability of models to simulate and test prospective management and planning scenarios. Phase 5 (2007–2010) integrated public health risks posed by emerging micropollutants such as new molecules with little known effects, pharmaceuticals and pathogens. Territorial studies also investigated the impact of ecological engineering and the reform of the Common Agricultural Policy (CAP). Phase 6 (2011–2014) further expanded into 5 main research axes, which reflected the concerns and challenges jointly identified by researchers and industry partners. These include: (1) creation of agricultural scenarios according to water quality requirements; (2) identification of the role of wetlands; (3) a deepened understanding of water quality in the current climate; (4) a better understanding of the relationships between chemical pressures and ecological states; and (5) understanding dynamics of chemical pressure over a long duration.

The current phase, Phase 7 (2015–2020), focuses on gaining an in-depth understanding of the mechanisms that regulate water resources and climate change scenarios to support management strategies that are more adapted to the agricultural, environmental and urban issues facing the region. Scenario building has become increasingly popular, allowing researchers and industry partners to collectively envision and anticipate possible futures. This outlook is reflected in official discourse, which promotes a science-policy transfer through a newly dedicated transfer unit (‘cellule de transfert’). At the same time, a shared mentality insists upon its foundation in research, aiming to provide knowledge and expertise that helps inform management and policy decisions without directly implicating itself in the role of a policy maker.

3.2. PIREN-Seine Models: From Aggregation to Integration

PIREN-Seine modelling tools have evolved in parallel to its research objectives, adapting to suit changing demands and/or being used with other models to answer specific questions or to provide a more global view of the functioning of the system. This has produced a variety of models, including hydrologic models, biogeochemical models, hydraulic models, agronomic models, economic models and a model that simulates the environmental impact on fish populations. The majority of these models address issues of water quality, particularly the transfer of nutrients or pollutants through different parts of the system. However, within a large river system such as the Seine, individual models are only capable of telling ‘part of the story,’ limited to a specific temporal and spatial scale. At the same time, increasingly strict requirements from regulations such as the European Water Framework Directive (EU-WFD) [48] are placing increasing pressure on researchers and decision makers to restore water bodies to ‘good ecological status’ [49,50,51], which demands a global vision of the system. These trends have resulted in change in trajectory from individual models responding to specific questions, to the adaptation or coupling of models to answer bigger questions, towards modelling chains and/or platforms that can be applied to the entire Seine system. Here, we present four main models (see

Table 2. Overview of PIREN-Seine Support Tools *.

Model	Type	Key References
ProSe	River quality model	Even et al. [52]; Garnier and Mouchel [53]
Seneque	Catchment quality model	Garnier and Mouchel [53]; Billen et al. [54]
MODCOU	Surface-groundwater model	Ledoux [55]; Ledoux et al. [56]
STICS	Agronomic model	Brisson et al. [57,58]

* Limited to the models presented in this paper.

) based on their history of development and use (directly and indirectly) by industry partners: ProSe, Seneque, MODCOU and STICS.

3.2.1. ProSe

Short for ‘Projet Seine,’ the model ProSe was developed by researchers at École des Mines ParisTech in collaboration with PIREN-Seine research teams, research institutions, universities and industry partners [52,59] within the context of the PIREN-Seine. Originally designed to study problems of water quality and chronic deoxygenation related to effluent discharges from wastewater treatment plants on downstream sectors of the river and accidental overflow of sewage networks during rainy events [60,61], it has also been applied to hydraulic problems and questions associated with the transport of particles [60].

The modular structure of ProSe allows for greater adaptability in simulating different scenarios, therefore its applicability is widespread. In recent years, ProSe has undergone several revisions (producing versions 1 to 4), increasing previous functionality in terms of knowledge gained as well as the ability to be coupled with other models. Although it is neither a standardized nor commercial model, simulations using ProSe are requested and sometimes required by regulating authorities such as the DRIEE to justify project proposals (SIAAP representative, 27 June 2016), and it is now widely considered a reference model for water quality of the Seine. The development and evolution of ProSe has been partially motivated by special interest from the SIAAP, who uses ProSe as a medium- to long-term management and planning tool (SIAAP representative, 29 November 2016). This has resulted in additional investment (time and resources), which extends outside of PIREN-Seine, either through ARMINES, a consultancy arm of École des Mines engineering school, or through a working group involving researchers and practitioners interested in adapting ProSe to meet operational demands (PIREN researcher, 28 April 2016). As such, ProSe is considered both a research and operational tool, even though the tool itself is one and the same (PIREN researcher, 16 June 2016). However, despite being frequently cited as an example of this dual functionality, the future of ProSe remains uncertain. Many original developers have either retired or expressed interest in moving on to other research projects (PIREN researcher, 16 June 2016), while its only current operational user (SIAAP) is moving towards artificial intelligence and real-time control methods and is considering replacing the model with statistical techniques for daily operations (SIAAP representative, 10 March 2016).

3.2.2. Seneque

Seneque, which stands for ‘Seine en equation’, was developed by the research team METIS—an interdisciplinary research unit at the University of Pierre and Marie Curie (UPMC)—in the context of the PIREN-Seine, though some of its components (i.e., RIVE) were developed outside of the program. Based on the concept of stream-order, Seneque simulates the transport of nutrients and the biogeochemical functioning of the hydrographic network using a simplified and idealised conceptualisation of the drainage network of large regional basins with a refined representation of in stream microbiological processes using the RIVE model [62,63]. Also referred to as Riverstrahler, Seneque is essentially the same model applied to the Seine River basin and coupled with a GIS interface [64]. The added functionality of a user-friendly interface has enhanced the user’s ability to visualize and explore results in a way that is more accessible to non-specialist users. Since its creation, Seneque has undergone several revisions and has been applied to different situations in combination with other models [52,62,65,66].

Also considered to be an ‘operational’ model, Seneque has been appropriated directly by the AESN as a medium- to long-term planning tool, used for example, to evaluate the ecological state of the basin by amalgamating different datasets to construct ‘snapshots’ at different spatial and temporal scales. The model has since reverted back to a ‘research’ tool mostly due to a loss in internal expertise at the AESN (AESN representative, 8 June 2016). The most recent incarnation of the model, Pynuts, has allowed researchers more flexibility in terms of model development, to explore new research questions using updated technology without having to invest time and resources on interfacing. However, plans to add an interface are in the works to, once again, allow it to be appropriated directly by industry partners in the future.

3.2.3. MODCOU

The MODCOU model was developed by researchers at École des Mines ParisTech [55,56,67,68] to simulate the movement and circulation of surface and groundwater. MODCOU describes surface and groundwater flow at a daily time step: the surface model calculates the water balance between evaporation, runoff and infiltration, while the underground model calculates the transfer of water in aquifers and surface-groundwater exchanges [67,69].

Much of the work on MODCOU is concentrated on its integration with other models. For example, it is often coupled with other models such as STICS (presented next) [67,70] in order to obtain a more complete understanding of nitrate contamination and the influence of agricultural activity on surface and groundwater. To date, MODCOU has been effectively applied to predict surface and groundwater flows in many French basins with varying scales and hydrogeological settings [67,71]. Though it has remained as a research tool, studies requested by partners such as the AESN to assess the impact of climate change on water resources have used MODCOU to evaluate groundwater levels and monitor trends in nitrate and pesticide content.

3.2.4. STICS

The model STICS has been developed by the Institut National de Recherche Agronimique (INRA) since 1996 [57] in collaboration with large research and professional institutes [58]. It was not developed in the context of the PIREN-Seine but is considered here as a PIREN support tool since it is often used to conduct research within the context of the program. STICS (Simulateur mulTIdisciplinaire pour les Cultures Standard) is an agronomic model that simulates crop growth, soil water and nitrogen balances driven by daily climatic data [57,58,72]. Intended to simulate the evolution of water, carbon and nitrogen in the soil-plant system over one or more years successively [57,73], STICS was designed and developed with the dual objective of calculating agronomic variables (e.g., plant biomass, harvested yield, protein content of the grain, nitrogen balances of the crop) and environmental variables (e.g., flow of water and nitrate out of the root zone) [58,72]. Crop generality allows for adaptation to various crops, whereas robustness in the model allows the user to simulate various soil-climate conditions without considerable bias in the outputs.

Development of the model has focused on usability through collaboration between model developers and users in a way that allows users to participate in its evolution. Mostly considered a research tool, its conceptual modularity has allowed STICS to be chained with other models in order to understand the transfer of nitrates and pesticides into surface and groundwater [74]. These types of studies are often requested by partners such as the AESN, who are interested in monitoring the impact of agriculture on water quality. In this way, it can also be considered a decision-support tool, although researchers are charged with running the model and scenarios.

3.3. The CRC for Water Sensitive Cities, Australia

Established in 2012, the CRC for Water Sensitive Cities (CRCWSC) [75] is one of many Cooperative Research Centres in Australia, which are part of a government initiative to fund innovative research that can directly meet the needs of industry. CRCWSC involves over 200 researchers from various backgrounds (hydrology, biology, chemistry, engineering, economics and social sciences), from national and international universities and research institutions. Setting itself apart from other on-going CRCs, the CRC for Water Sensitive Cities builds upon the research base of previous CRCs (the CRC for Catchment Hydrology from 1992 to 2005, CRC for Freshwater Ecology from 1993 to 2005 and eWater CRC from 2005 to 2008) and focuses specifically on creating water sensitive cities [76,77], or sponge cities [78,79,80], guided primarily by three main principles: (1) Cities as water supply catchments; (2) Cities providing ecosystem services; and (3) Cities comprising water sensitive communities [77].

Main actors in relation to modelling activities are represented in Table 1. Some of these partnerships are financial in nature, either through direct funding to the program, funding for specific projects which contributes to the work of the CRC for Water Sensitive Cities, or through in-kind contributions of researchers paid by their home institutions. Most partners are directly involved in research support, either as researchers themselves, or ‘beta-testers,’ who test, apply, provide feedback, and play an essential role in disseminating the knowledge and tools on the ground. This network also includes associate partners, who may access the knowledge or tools and help test, apply, and disseminate this research without direct investment, and who may also contribute to capacity-building activities.

Whilst PIREN is a research program that is renegotiated every 4–5 years, CRCWSC runs for 9 years (2012–2021), as opposed to the average 5 years of other CRCs. Its research program comprises two parts: Tranche 1 (2012–2016), focused on research and Tranche 2 (2016–2021), focuses on adoption pathways and implementation of the research produced in addition to building new knowledge. Within the first tranche, four diverse programs in the areas of Society (Program A), Water Sensitive Urbanism (Program B), Future Technologies (Program C) and Adoption Pathways (Program D), have produced research outputs that have either fed directly into the development of new modelling tools or have applied, adopted, and expanded existing industry standard models in new contexts. In particular, Program D focussed on developing partnerships between relevant actors at all levels (from community to government), capacity building, and holistic decision-support tools. With the first tranche completed, this program has continued in an evolved form in Tranche 2.

3.4. Water Sensitive City (WSC) Models: New Tools for New Strategies

WSC (models or tools) have moved away from decision support based on deterministic or stochastic models towards integrated modelling platforms and visualisation—an evolution in strategic planning within a new era of ‘deep uncertainty’ [81,82] and greater collaboration [83,84]. Whereas running models individually can support management and policy decisions on a short- to medium-term, an integrated modelling approach allows for exploratory modelling and adaptive planning for an uncertain future [81]. In the context of CRCWSC research, models are complementary, meant for use at different parts of the workflow. Here, we focus on three models (see

Table 3. Overview of Water Sensitive City (WSC) Tools *.

Model	Type	Key References
MUSIC	Stormwater quality model	Wong et al. [85]; http://www.ewater.org.au/products/music/
Water Sensitive Toolkit	Infrastructure planning tool	https://watersensitivecities.org.au/solutions/water-sensitive-cities-toolkit/
DAnCE4Water	Cloud-based city modelling platform	Rauch et al. [86]; www.dance4water.org

* Limited to the models presented in this paper.

): MUSIC, WSC Toolkit, and DAnCE4Water. Two of these models began development well before the CRC for Water Sensitive City program began, but have since been extended or upgraded based on the latest research resulting from Tranche 1 and are currently used—or intend to be used—by industry partners.

3.4.1. MUSIC

The Model for Urban Stormwater Improvement Conceptualisation (MUSIC) was developed in 2001 by the CRC for Catchment Hydrology (1992–2005), involving many past and current researchers of CRCWSC. This work continued after it merged with the CRC for Freshwater Ecology (1993–2005) to eventually form eWater, a government owned non-profit organisation (and CRCWSC industry partner) offering capacity building, technical support services and modelling tools to support integrated water resources management and governance. Developed with the objective of synthesizing research into an easy-to-use tool, MUSIC is a decision support system that allows water managers to evaluate stormwater management systems based on specific water quality objectives, as well as determine appropriate sizing of stormwater treatment facilities and associated infrastructure [85]. Its core feature is how it describes water quality behaviour through a first-order kinetic decay model (K-C* Model) of three key pollutants (suspended solids, phosphorus, nitrogen), and hydrodynamic behaviour within a stormwater treatment device through the continuously stirred tank reactor (CSTR) concept [85,87]. The current version of MUSIC (v6) has expanded and updated initial capabilities to a wider range of stormwater treatment devices and new performance indicators [88]. Through on-going research efforts and communication between eWater and CRCWSC, many improvements to MUSIC’s capabilities and functionality have been made and its applicability to non-Australian cities like Singapore is being assessed.

As one of eWater’s most widely adopted models, MUSIC has since become the industry standard across Australia for stormwater quality management and Water Sensitive Urban Design (WSUD). Early endorsement from two key industry partners in Melbourne and Brisbane, who were investigating ways of protecting receiving waters from urban stormwater pollution, heavily contributed to rapid adoption across many municipalities in Australian’s east, particularly in the states of Victoria and Queensland [89]. Practitioners use MUSIC to design integrated stormwater management plans based on a specific catchment and to demonstrate compliance to local standards. It has also been used for CRCWSC research, contributing to the development of other tools such as the WSC Toolkit.

3.4.2. Water Sensitive Cities (WSC) Toolkit

Developed in Tranche 1 of the CRCWSC program, the WSC Toolkit synthesises key research outcomes into easy-to-use modules for assessing the benefits of WSUD. The model aims at supporting strategic planning, by focussing on evidence-based quantification of the benefits of urban green infrastructure (GI) initiatives in order to develop business cases that are both robust and water sensitive [90]. The model is capable of: (1) improving stream health impacts based on the effectiveness of WSUD in mitigating runoff volumes, frequency and pollutant concentrations [91,92,93,94]; (2) assessing changes in flow frequency and reduction of geomorphic impact on streams based on the stream erosion index [95] and; (3) mitigating the urban heat island effect through urban greening and retaining water in the landscape [96]. Other modules are still under development including a future climate module, which will draw from a database of future rainfall projections for major Australian cities and can be used independently or as input data for future climate scenarios [97,98]. An economic valuation module is also planned, to consider the likely willingness-to-pay of community members based on various improvements made to liveability and sustainability of the catchment.

The WSC Toolkit is currently in closed ‘beta-testing’ mode, with its adoption slowly taking place in select municipalities across Australia. Much of its momentum is currently driven by the need for quick and easy microclimate assessment tools that enable local municipalities to formulate a business case for funding more WSUD and green infrastructure projects. The ability of the WSC Toolkit to communicate directly with MUSIC is also a strategic choice and leverages the familiarity of an existing large user base.

3.4.3. DAnCE4Water

The DAnCE4Water model (Dynamic Adaptation for eNabling City Evolution for Water) began as part of the European Framework Program 7—‘PREPARED enabling change’ (www.prepared-fp7.eu) prior to the CRC for Water Sensitive Cities [86,99]. It was then adopted within Program A (Society) of the CRC for Water Sensitive Cities, where it evolved into a cloud-based city modelling platform. Aspiring to be an interactive, ‘user-friendly’ decision support tool for different water actors to explore future scenarios and evaluate different policy and action strategies, DAnCE4Water takes into account the interactions between urban water infrastructure, the urban environment, and social dynamics [86]. This is represented by three modules rooted in a central unit, or ‘conductor,’ which runs each scenario by storing, managing and providing required data to the relevant modules [100]. Formerly driven by a societal transitions model [101], DAnCE4Water now relies on the interplay between urban development and societal dynamics influenced by an economic willingness-to-pay framework. The urban development module, in particular, projects the changes of the urban environment down to the household level [102]. Various biophysical modules are used to simulate the impact of urban development on infrastructure, and include well-known hydraulic models such as EPANET [103] and EPA SWMM [104], as well as a link with MIKE URBAN for flood risk assessment [105].

While this modelling tool has great potential for strategic planning and adaption, its use and utility remain undetermined for the moment, as (at the time of writing) it is still under development and not yet fully operational due to its scale and broad city-scale scope. The underlying computational and web-based framework has, however, paved the way for smaller tools that are currently being trialled across Australia, such as the Water Sensitive Cities Index, which enables municipalities to benchmark how ‘water sensitive’ their local area is compared to their peers and the overarching vision of CRCWSC [106].

4. Influence of Organisational Configurations and Context-Specific Drivers

Our exploration of the strategies and modelling tools in the context of the PIREN-Seine in France and the CRC for Water Sensitive Cities in Australia provides insight into their effect on the use and utility of modelling tools in each example. Both PIREN and CRCWSC fit the criteria for ‘boundary organisations’: they straddle the boundary between two distinct worlds (i.e., science and policy) but are accountable to both, provide opportunity and sometimes incentives for the development and use of shared objects or ‘boundary objects’ [6,16,17] (e.g., modelling tools), and involve participation of actors from both sides of the boundary, as well as actors who play a mediating role [6]. In this way, they not only mobilise various stakeholders but also orient research and available tools towards achieving common goals, which in turn, informs the potential and/or intended use and utility of their scientific knowledge and tools. Here, we draw from systematic observation and analysis to explore tentative linkages, highlighting the role of organisational configurations and context-specific drivers on model use and utility in practice. These can be classified into three main categories: (1) objectives and expertise; (2) knowledge and tools; and (3) supporting structures.

4.1. Objective and Expertise

Empirical data suggests that the objective(s) of the program and the expertise of the individuals involved have a large influence on the scientific knowledge that is produced and subsequently, how that knowledge is used (if at all). On one hand, PIREN has a territorial focus with expertise on the Seine River basin, although some of the knowledge and tools have been applied to other basins (PIREN researcher, 2 May 2016). On the other, CRCWSC has an urban focus, which began in Australia in the early days of its Melbourne-based predecessor (the ‘Cities as Water Supply Catchments’ Project) and has now expanded abroad through the involvement of international partners. Although PIREN engages researchers from different disciplines, most have a background in natural sciences or engineering with a focus on water quality. Other than annual conferences and planning sessions, research teams mostly keep to themselves (PIREN researcher, 1 December 2016). The representation of social sciences is small but growing, moving from quantitative studies to more qualitative studies, which include historical trends, social dynamics and the production of science. Likewise, CRCWSC involves an interdisciplinary team, though there is a greater balance between the natural and social sciences, which is seen as both necessary and inseparable (CRCWSC researcher, 22 June 2017).

Differing perspectives on the relationship between science and policy is perhaps the biggest difference between the two programs: PIREN tends to favour research over policy, while CRCWSC specifically orients its research towards use and adoption. On one hand, PIREN prefers a more marked distinction, with the objective of providing expertise and support without taking an active role in policy (PIREN researcher, 29 June 2016), although this perspective is not necessarily shared among all individuals and the mentality is generally becoming more open. Even if researchers would like their work to be applicable in practice, policy issues are commonly perceived as something beyond their role and responsibility (PIREN researcher, 12 January 2017). While this allows them to maintain scientific objectivity, it may also limit their impact in terms of knowledge dissemination and practical application, or at least render it more difficult to ascertain. On the other hand, CRCWSC has a clear objective: to promote sustainability, resilience and liveability through WSUD and the water sensitive cities by directly engaging with local councils, regional and national governments and citizens. Taking an active role in connecting science and policy and specifically organising its research around its use and transfer in practice has resulted in direct impacts on policy and planning (e.g., regulation standards set by MUSIC) (CRCWSC researcher, 9 June 2017).

Increasingly blurred borders and long collaborative relationships (official and unofficial) have likely contributed to building trust, credibility and legitimacy; a sentiment that was expressed in some form or another by all interview participants. In both examples, the science-policy interface resembled the web of interactions described by Vogel et al. [107]: in PIREN, many practitioners came from the same academic training as researchers (AESN representative, 8 June 2016), while in CRCWSC, it was common for researchers and practitioners to have held positions on both sides of the boundary at different stages in their career (CRCWSC researcher/industry partner representative, 21 June 2017). While this also occurs in PIREN (some industry partners were previous students of PIREN researchers), the lines between research and practice in this example have traditionally been more distinct. Collaboration, co-production and co-development resulting from the multitude of official and unofficial interactions and exchanges (inter-, intra- and extra-boundary) create mutual understanding and communication, which subsequently promote feelings of trust among different actors. In both examples, all interview participants expressed ‘trust’ in the models, as far as models can be trusted, knowing they are only a representation of reality. Confidence is fostered through official interactions such as conferences, working groups, planning sessions and workshops, as well as unofficial interactions where practitioners can consult researchers even when they are ‘off-the-clock’ (SIAAP representative, 10 March 2017). Whereas blurring the borders may foster collaboration, understanding and trust, maintaining legitimacy may, in some cases, require the borders to be restored (even if only temporarily) in order to clearly distinguish science from policy. This allows scientific knowledge (e.g., model outputs) to maintain scientific objectivity, since it is produced by researchers using scientific tools, and is therefore presumed to be free from political bias (SIAAP representative, 29 November 2016).

4.2. Knowledge and Tools

One of the biggest differences between knowledge and tools that have emerged from the two examples is their definition of purpose. Whereas PIREN support tools tend to place research as their primary objective and (indirectly) policy and planning as secondary, WSC tools are designed to make the underpinning research available and actionable for practitioners to demonstrate compliance and show the multiple benefits of local water sensitive solutions to regulators, authorities and communities. On one hand, a wide range of PIREN support tools are considered useful for practitioners, yet these tools tend to be highly academic and sometimes difficult to translate directly into action. On the other, the ‘user-friendly’ design of WSC tools is meant to promote adoption by industry partners, though some are still too new to be fully evaluated for use and utility.

In some cases, models may be improperly used or stretched beyond their capabilities to answer questions that they were not designed to answer (Australian water utility representative, 8 August 2017). While this is a general concern among model developers (CRCWSC researcher, 20 June 2017), there is a general feeling of trust among water actors that models will not be intentionally abused (CRCWSC researcher, 25 July 2017). For PIREN, a higher level of trust is felt among practitioners who have modelling expertise or who were involved in the development process, owing to a better understanding of the objectives and limitations of the model (AESN representative, 8 June 2016). For the most part, uncertainties were not explicitly discussed between researchers and industry partners in either case; the onus is therefore placed on experts and technicians to transmit relevant information (PIREN industry partner representative, 7 March 2017). Industry partners who have internal modelling expertise may also run their own uncertainty analyses, motivated by the direct consequences of such uncertainties on their work (SIAAP representative, 3 March 2017). ‘Acceptance’ or explicit concerns over uncertainty is therefore linked to potential consequences (social, economic, environmental) of management and planning decisions that were based on modelling results.

Other tools might have to be simplified to enhance their use and utility. For example the Water Sensitive Cities Index [106], which is less of a model and more of a benchmarking tool (CRCWSC researcher, 20 June 2017) has found opportunities for application due to its simplicity. Conversely, a more critical view was expressed for some of the larger-scale strategic planning tools, which may be considered ‘helpful but unnecessary’, as it was opined that conventional methods such as cost-benefit analyses or SWOT analyses could deliver the same results (Government Representative, 31 July 2017). It is important to highlight that this view may stem from a previous controversial experience that the state of Victoria has had with the use of such large-scale ‘black boxes’ [108]. Although this case was frequently cited, interview participants in Australia still generally expressed high levels of trust in models due to the demand for greater transparency and communication following this incident (CRCWSC researcher, 20 June 2017).

In the case of PIREN, the lack of ‘operational’ models that partners can use themselves is a strategic choice, not only for reasons of objectivity but also due to time and resource constraints:

“Tools are available if [partners] want to use them as is but they don’t have the human resources and they don’t finance the interfacing either…We think more in terms of services, where the user defines what they want to do or what they want to evaluate and we [researchers] will perform the simulations and deliver the results”.

(PIREN researcher, 29 June 2017)

In this way, providing services are considered to be a more efficient use of resources for both researchers and industry partners, none of whom are prepared to invest time and human resources for a model they may only require on occasion. However, there may be less of an incentive to provide these services in cases where industry demand does not pique scientific interest.

4.3. Support Structures

Support structures refer to the different configurations that can promote or reinforce scientific knowledge or tools. This includes financial structures, organisational configurations, technical support and regulatory measures. Lemos et al. [9] suggest that usability can be improved through strategies of value-adding, retailing, wholesaling and customisation. While these may exist to some extent in both examples, the limitations posed by their respective ‘boundaries’ (in objectives and expertise, knowledge and tools and support structures) may not leave enough room to fully incorporate these strategies unless it is made to be a deliberate aim. For PIREN, this necessitated external contracts and support structures through the creation of ARMINES, the consulting arm of École des Mines ParisTech (PIREN researcher, 29 June 2016). While ARMINES provides a lucrative side line activity, which tailors research to specific industry demands, it is usually the research (scientific knowledge) itself that is customised, rather than the tools. For example, an industry partner such as AESN may request a specific study to be conducted and only require the results. In France, retailing, wholesaling and customisation of modelling tools is often perceived as the work of consultants, not researchers. For CRCWSC, modelling work was also outsourced with the MUSIC model through support from eWater (CRCWSC researcher, 9 June 2017). The structure of eWater is more aligned to strategies of retailing, wholesaling and customisation of tools, resulting in higher adoption of their tools. On one hand, boundary organisations play an important role in putting key players together with support and tools oriented towards a common objective and on a much wider scale than other science-policy partnerships. On the other, their ‘boundaries’ may limit their ability to fully support effective strategies that promote use and utility alone. The ‘best of both worlds’ may, in fact, be found in coordinated strategies that combine interactions and exchanges inside, outside and between these ‘boundaries.’

Within these structures, financing often plays a large role on what is or can be done. On one hand, PIREN benefits from an extended and, for the moment, indefinite duration, allowing them more freedom to explore a wider range of research questions over a longer time period. However, their research actions are limited by a fixed amount of public funding from industry partners, an amount that has not seen much increase over the years despite a growing number of researchers who are involved in the program. Additionally, the autonomy of researchers is also subject to external funding sources that may come from universities, national research projects, or European projects, which allows certain freedoms while posing other constraints. On the other hand, CRCWSC is working on a 9-year timeline with a fixed budget of public and private funding from industry partners, governments and companies. Compared to PIREN, they are working with a bigger budget on a smaller time frame, which has allowed them to focus on specific goals and meet targeted objectives. In-kind support is also a major contributor in both examples, by way of researchers and doctoral students. At the same time, CRCWSC could face major challenges on the impact and sustainability of their work, particularly regarding the refinement, maintenance and adoption of modelling tools once the program ends. This can partly be addressed with technical support structures, which include user guidelines, technical manuals, training workshops, capacity building, and user support in the form of collaboration between researchers and industry partners, which fosters mutual understanding, transparency, and trust. The WSC Toolkit, for example, has initiated some of these structures including a user manual and a series of national training workshops for some of its operational features, building upon the experience learnt in the development and adoption of MUSIC (CRCWSC researcher, 20 June 2017). Technical support also exists in PIREN, though more through official or unofficial collaboration between researchers and partners. In the case of ProSe, for example, a technical working group was created in parallel to PIREN, involving some of the same researchers and partners while remaining outside of its boundaries.

Another important supporting structure is regulation, as illustrated by the examples of MUSIC in Australia and ProSe in France, both of which are required (even if unofficially) by regulating authorities. As an industry recognized tool, MUSIC has helped standardize regulations (e.g., [109,110]) across different territories with shared water networks (Government representative, 26 June 2017). The use of MUSIC as a compliance tool has also supported its legitimacy, since it ‘helps speed up the process’ for project proposals (Australian water utility representative, 8 August 2017). Additionally, models that are used nation-wide undergo a government-recognised accreditation process, which enhances the perception of its validity (Government representative, 26 June 2017). However, despite accreditation and validity, cost can be a limiting factor, with licences ranging from AU$0 for a 21-day limited trial version to prices starting at AU$5000 for a multiple user licence [111] (www.ewater.org.au). In the case of ProSe, the fact that the SIAAP is the only operating partner capable of running the model independently gives them a better bargaining position, however; the requirement to use ProSe also limits their ability to explore other models that may be better adapted to their needs (SIAAP representative, 3 March 2017). Regulations and the demand for evidence-based decisions may also place pressure on science to answer non-scientific questions. For example, since Paris won the bid to host the summer Olympics in 2024, there has been increased pressure for scientists to improve the water quality in the Seine in order to make it swimmable. Although issues of water quality are of scientific interest, particularly for PIREN, some may consider specific requirements for recreational use (e.g., faecal contamination levels) to be outside of the interest or expertise of PIREN researchers.

5. Moving Beyond the ‘Usability Approach’

Technological advancement, coupled with the production of expertise, has led to the development of a large number of modelling tools [1,49,112], which aim to address specific environmental questions at different temporal and spatial scales. In parallel, practitioners face increasing pressure to base management and policy decisions on scientific evidence and data [18,113,114]. In this context, it would seem natural for modelling tools to be adopted by managers and decision makers, yet this is still far from the norm [115]. While challenges posed by the lack of communication or expertise are often cited among the main driving factors influencing the adoption of models [1,49,112,116,117], much of the literature is based on the dichotomy of ‘use’ vs. ‘non-use.’ However, the tentative linkages explored in the previous section suggest a more complex and nuanced relationship between use and utility that stretch beyond the common understanding of ‘usability,’ where the value of scientific knowledge and tools is tied to its ability to be applied (or directly used) in practice [9]. Proponents of, what we refer to as the ‘usability approach,’ often speak about ‘usability’ without detailing how scientific knowledge is actually used and what it is used for in practice, which we argue, have consequences on its use and utility. Building on previous research and aiming to deepen ‘usability approach’ thinking, this section explores the myriad of uses and utilities represented in our two examples.

5.1. Use vs. Utility

The major utilities for WSC tools and PIREN-Seine support tools generally fall under three main categories: (1) Enlightenment; (2) Decision support; and (3) Negotiation support, which reinforces previous findings [113,118,119]. Enlightenment can refer to a general contribution to overall understanding, specific information used for daily management or medium to long-term planning, or to monitoring trends and emerging issues. Decision support refers to daily management, medium to long-term planning, or evaluating actions taken, as well as to anticipating future trends. Negotiation support can refer to justifying a project or proposal, a way of asserting a certain role or position among a network of actors, or a way of acquiring or maintaining bargaining power. These categories are typically not independent and often coincide. The utility of a model is further influenced by three factors: objective, relevance and knowledge/expertise [120]. Objective refers to the set of priorities that the user seeks to be satisfied by the model. In other words, what is asked of the model, what purpose will it serve and what can be done with the model or its results? Relevance refers to how closely the model simulations correspond to the issues at stake for the user. In other words, the capability of the model to respond to the specific needs of the user, as well as the importance given to what is modelled. Finally, knowledge/expertise relates to the background or training of the user and their experience with modelling activities. This includes their capacity to run the model independently, add or modify components, understand its functions and limitations, know what data is required, and effectively translate and/or interpret the results.

5.2. User Involvement

In addition to the various utilities listed above, model use was found to be better represented as a spectrum based on four levels of user involvement [120] ranging from:

• Direct++, which indicates total mastery of the model;
• Direct+, which refers to independent model use without being able to change the model itself;
• Direct, which refers to a good understanding of what is being modelled while retaining limited involvement in the modelling process; to
• Non-Direct, which refers to complete detachment from modelling activities.

In Direct++, users can run the model independently, have access to input data, run simulations and are capable of making changes to the model itself (to the code, parameters, etc.). Next, Direct+ users understand how the model works; they can run simulations by themselves and may participate in the development of a model but are not able to make changes to it themselves. Direct use refers to users who have a good understanding of what is modelled and may participate in the elaboration of scenarios but are not involved in the modelling process itself. This type of user typically requests studies from experts and prefers to use the results instead of investing in in-house modelling expertise. Finally, there is Non-Direct use, where users are removed from the modelling process but can still benefit indirectly, as the knowledge produced by models is diffused into the global domain. A general framework outlining the relationships between use and utility from our two examples is found in Figure 1 below.

5.3. Integration and Application of Concepts

Of the numerous modelling tools that were either developed and/or used by PIREN over the past few decades, only two models (Seneque and ProSe) were identified as being used directly (at one time or another) by an operational partner, while one of the two (ProSe) is still in regular use today, suggesting greater ‘non-use’ of PIREN support tools. In retrospect, we could say that this is due to the fact that PIREN models are too academic and not ‘user-friendly’, rendering them less usable and therefore less useful. However, while this may be true in some cases, many of the partners interviewed maintained that the knowledge and tools produced by PIREN were integral to their work. We can explain this discrepancy by combining use, utility and user involvement into a general framework (Figure 1), which represents empirical findings from both examples. Within this framework, most users tend to fall on opposite ends of the spectrum: the majority of researchers involved in modelling activities are considered Direct++ users, while most operational partners are considered Non-Direct users, with the exception of the SIAAP who is a Direct+ user of the model ProSe. Although some models are occasionally used for decision and negotiation support, the main utility of PIREN support tools is for enlightenment, which explains why most partners find the tools useful even if they do not use them (directly). Although enlightenment is a fundamental utility of all types of uses, more prominent examples are found at the opposite ends of the spectrum in Direct++ and Non-Direct uses. For example, researchers make simulations with models (Direct++) to gain a deeper understanding of the transfer of micropollutants in the basin. While this information is relevant to operational partners, the science may not be at the point where it can be translated into action, or, similarly, the regulations may not have caught up with the science. Monitoring these research activities (Non-Direct) in the meantime will help to guide future planning by anticipating these emerging trends. On the flipside, CRCWSC aims to produce modelling tools that are adopted (directly) by water managers and decision makers. Using the general framework, we can say that most of the researchers are Direct++ users, while most industry partners are (or aim to be) Direct+ or Direct users. While models such as MUSIC have achieved this objective, it is too early to say whether newer tools such as DAnCE4Water or the WSC Toolkit will share the same success.

Compared to CRCWSC, the uses and utilities found within PIREN appear to be more varied. In both examples, Direct++ users tend to be researchers or model developers, while the knowledge they produce can be useful for researchers and industry partners of all user types for enlightenment. For example, in the case of PIREN, MODCOU is considered a research model (mostly Direct++ and Direct+ uses), yet the results are used by the AESN (mostly Direct or Non-Direct uses) to monitor and identify trends, which allows them to develop more adaptive climate change strategies (AESN representative, 8 June 2016). While most of the WSC models are aimed at Direct and Direct+ uses by industry partners, there is only one current instance of a Direct+ use within PIREN (case of the SIAAP who uses ProSe for enlightenment, decision and negotiation support). While most models serve an enlightenment function, the SIAAP also uses ProSe to support decisions (e.g., when sizing infrastructure and implementing new projects) as well as negotiation support, since they are required to justify proposals to the regulating authority using ProSe (SIAAP representative, 27 June 2016). Despite having in-house capacity to run the model independently and contributing to model development and data collection, practitioners are not able to change the code and must turn to researchers for specific requests (PIREN-Seine researcher, 28 April 2016). In Australia, MUSIC is a similar example of a Direct+ use by industry partners. As it has become the industry standard, using MUSIC to support decisions and justify proposals, though not always required, is beneficial (CRCWSC researcher, 9 June 2017). Direct uses are also common within the PIREN, in cases where partners ask for a specific study to be conducted. For example, when the AESN uses STICS-MODCOU to evaluate nitrates and pesticide flows in a specific aquifer (AESN representative, 8 June 2016).

While uncertainty related to modelling was rarely explicitly discussed, findings in both examples suggested that the ‘acceptability’ of uncertainty was implicitly informed by its use and utility. Direct and Direct+ users in PIREN were more concerned with quantifying uncertainty, as the stakes were relatively higher. An underestimation of pipe sizing by the SIAAP could, for example, directly contribute to major flooding in dense urban areas resulting in high economic, social and environmental costs. Failure to account for model uncertainty in these cases could also undermine project proposals based on modelling results, which in turn, undermines their negotiating power as it calls into question the expertise. On the other hand, the technical expertise required of these user types allows them to maintain trust in the model, by knowing what you can and cannot trust (SIAAP representative, 29 November 2016). Conversely, Non-Direct users may also maintain a high level of trust in the models despite a lack of technical expertise. In this case, trust is not in knowing what to trust (in the model) but rather, whom you can trust (experts) (DRIEE representative, 12 May 2016). For CRCWSC, uncertainty was considered ‘more acceptable’ (implicitly) in strategic planning tools such as DAnCE4Water. Since its intended use is to explore a range of possible future scenarios, the high level of associated uncertainty is a given (CRCWSC researcher, 14 June 2017).

Despite research and practice becoming increasingly collaborative processes, several studies continue to highlight the weak correlation between scientific production and use in practice. For example, through an empirical analysis of 20 scientific assessments co-produced by researchers and decision makers, Weichselgartner and Kasperson [27] revealed that decision makers did not sufficiently draw from available research-based knowledge, while at the same time, the knowledge produced by researchers was not sufficiently usable (directly). In another example, Holmes and Clark [28] analysed the studies conducted by the Environment Research Funders’ Forum (ERFF) in the United Kingdom, pointing out that there was still significant lag time between current practice and guidance. Similarly, in their assessment of management practices in the Columbia River Basin, Callahan et al. [29] found that climate forecasts were significantly underutilised by managers despite their potential to support their ability to manage water resources in the face of increased climate variability.

The general framework of use and utility provided in Figure 1 extends the concept of use and utility from the strict dichotomy common to ‘usability approach’ thinking to a spectrum of uses and utilities that are found in examples such as PIREN and CRCWSC. Maintaining this dichotomy could lead some to develop solutions that are counteractive to their objective of increasing the adoption of modelling tools by practitioners and decision-makers. For example, a simplified model with a user-friendly interface may seem like a logical solution to overcome issues of communication and lack of expertise between researchers and practitioners. However, it may be of little use to a practitioner who requires a complex model to answer specific questions, but does not want to invest the time and resources towards in-house expertise. A better understanding of the nuanced relationship between use and utility can therefore support the development of tools that are more adapted to the needs of practitioners and decision-makers, according to what is needed (the model itself or the results), how they are used (level of user involvement), and what they are used for (justify proposals, monitor trends, etc.). While there is no one-size-fits-all solution (nor do we advocate for one), our analysis may help identify key points to consider when assessing the use and utility of modelling tools to better support water resources management, policy, and planning decisions.

Furthermore, discussion on how to produce ‘usable’ science could benefit from more in-depth analyses of specific examples. The fundamental difference between the CRC for Water Sensitive Cities and the PIREN-Seine is their objective and approach, which has resulted in different tools with different purposes. On one hand, the CRC for Water Sensitive Cities has taken a more market or policy driven approach, resulting in the production of more ‘operational’ tools, as well as active involvement from developers, water actors, local councils, and state governments (CRCWSC researcher, 9 June 2017). Not only does this promote research that is directly ‘usable’ for policy, it also establishes a target audience and a built-in user base (Australian water utility representative, interview 27 July 2017). In addition to decision support, WSC tools are designed with the specific (and arguably political) objective of achieving water sensitive cities in mind. In the case of MUSIC, its development and use as a compliance tool further entrenches the intimate relationship between science and policy, by creating both supply and demand (CRCWSC researcher, 9 June 2017). On the other hand, the PIREN-Seine has traditionally focused on the production of research and research tools as a primary objective to enlighten policy and planning decisions (PIREN researcher, 29 June 2016). Whereas CRCWSC takes an active role in policy, PIREN prefers the role of policy supporter rather than direct advisor, resulting in mostly ‘research’ tools and knowledge that is often difficult to translate to action and with an impact on policy that is not as easily quantifiable. However, the example of ProSe illustrates how a ‘research’ tool can also be ‘operational’ when mutual interest and supporting structures are strategically aligned (SIAAP representative, 27 June 2016).

While some commonalities can be extrapolated, our analysis of the specific organisational configurations and context-dependent drivers supports findings of previous authors [10,12] who stress the importance of moving beyond the traditional ‘linear’ model of research use, and advocate for a better account of the complex and nuanced interactions which take place at the science-policy interface. Vogel et al. [107] suggest we begin by reimagining these relationships in terms of ‘spider webs’, which are ‘composed of nodes and a multitude of ephemeral linkages’ (p. 360). Commonly held perceptions concerning the production of ‘usable’ knowledge and tools for management and policy tends to oversimplify the problem [2,10,12,107], which, in turn, limits opportunities for overcoming this fundamental challenge. Attempts to tackle this issue would therefore benefit from reframing the discussion to include and embrace the diversity that exists in modelling, which will not only provide a more informed understanding, but also help guide the development of knowledge and tools that are more adapted to different user needs. While the debate over scientific complexity vs. usability is still valid for specific models, it does not always need to be a trade-off. Instead, we can think of models as having different forms and functions, which can be used to complement one another or at different stages of the workflow to support different levels of planning and action (CRCWSC researcher, 6 June 2017; CRCWSC researcher, 20 June 2017). For example, deterministic models or real-time control for short to medium term management and planning and modelling platforms for longer-term planning and strategic thinking.

While each program uses a different approach, both are moving towards the idea of co-construction through collaborative scenario building and the use of modelling chains and/or integrated modelling platforms to address industry demands. On one hand, this is a logical choice, as scenario building and strategic modelling can support more robust and adaptive strategies (CRCWSC researcher, 14 June 2017). On the other, focus on ‘co-construction’ over ‘co-production’ may be considered a strategic choice, since a strict focus on the co-production of modelling tools requiring researchers and practitioners to invest heavily in time and resources, may not end up being very productive (PIREN researcher, 29 June 2016). Therefore, changing the discourse to the concept of co-construction of scenarios rather than co-production of models may allow for a more effective collaborative exchange as well as a more efficient use of resources. Partners may still be involved in the development of modelling tools, by providing feedback or as ‘beta-testers’ but the technical development (e.g., changing the code, adding parameters) resides with the researchers, who have the technical expertise. This way, each side plays to its strengths, while enhanced communication and understanding can be facilitated through interactive spaces such as workshops, seminars and working groups [10,107].

Finally, a more efficient science-policy relationship may benefit from a shifted focus from knowledge transfer to knowledge brokering [121,122,123], which helps ensure appropriate translation of research findings and facilitates the creation, sharing, and use of knowledge [124]. Knowledge brokers have played a key role in the dissemination of the work of the CRC for Water Sensitive Cities, helping to bring together different stakeholders towards the same objectives and increasing their impact on policy (CRCWSC researcher, 25 June 2017). In both cases, knowledge brokering would enhance the use and utility of modelling tools by helping developers understand user needs and helping users understand the objectives and limitations of the model. Modelling chains and platforms may be considered effective ‘boundary objects,’ by linking different modules together to tackle questions that are relevant to both research and policy. The same can be said of scenario building through strategic thinking exercises facilitated by these tools. In this context, the use and utility of the model itself is less of a concern, since the purpose is not to produce a specific outcome but rather to co-conceptualize and envision a range of possible outcomes. This allows different actors to come together and explore different strategies in a more neutral setting. Whether it is used directly or indirectly, collaborative scenario building and the use of modelling chains and/or platforms may prove to be a more effective path to enhancing the use and usability of scientific knowledge in practice.

6. Conclusions

Science and policy have become increasingly interdependent and science-policy collaborations more common, yet clear pathways for producing ‘usable’ scientific knowledge and tools remain uncertain. A novel approach based on an empirical analysis was used in the context of two boundary organisations in France and Australia to explore the tentative links between program strategy and the use and utility of modelling tools. Organisational configurations and context-specific drivers of: (1) objective and expertise; (2) knowledge and tools and; (3) support structures were identified as primary factors. Empirical findings highlighted a complex and nuanced relationship between use and utility, which suggests the need to go beyond ‘usability approach’ thinking. Further insight was also given into the role played by boundary organisations in bringing together relevant actors, facilitating formal and informal exchanges and building capacity, credibility, salience and legitimacy, suggesting that knowledge brokering and coordinated strategies which effectively integrate inter-, extra-, and intra-boundary activities would likely enhance use and utility. An exploration of the layered complexities between use and utility also suggests that added social value is created through mediated interactions and exchanges, which are facilitated by boundary organisations. The trend towards collaborative scenario building and the use of modelling chains and/or interactive modelling platforms offers ways of framing these interactions to better support management, policy and planning decisions. In this way, models may become a tool for communication and mediation between various actors, serving as a common reference point for co-conceptualising robust and adaptive strategies towards a shared vision of water resources management.