How Cities Think : Knowledge Co-Production for Urban Sustainability and Resilience

Understanding and transforming how cities think is a crucial part of developing effective knowledge infrastructures for the Anthropocene. In this article, we review knowledge co-production as a popular approach in environmental and sustainability science communities to the generation of useable knowledge for sustainability and resilience. We present knowledge systems analysis as a conceptual and empirical framework for understanding existing co-production processes as preconditions to the design of new knowledge infrastructures in cities. Knowledge systems are the organizational practices and routines that make, validate, communicate, and apply knowledge. The knowledge systems analysis framework examines both the workings of these practices and routines and their interplay with the visions, values, social relations, and power dynamics embedded in the governance of building sustainable cities. The framework can be useful in uncovering hidden relations and highlighting the societal foundations that shape what is (and what is not) known by cities and how cities can co-produce new knowledge with meaningful sustainability and resilience actions and transformations. We highlight key innovations and design philosophies that we think can advance research and practice on knowledge co-production for urban sustainability and resilience.


Introduction
Cities are increasingly leaders in the creation and transition to more sustainable and resilient pathways.From more efficient transportation and building technologies to green infrastructure solutions that protect people from flood hazards, cities are on the front line of implementing sustainable strategies and building new infrastructures to enhance resilience to climate change [1,2].Yet, cities also face great challenges to sustainability transformations.Cities exhibit obduracy because of existing social, economic, political, and physical structures that are difficult to change, even when the vision and actions needed are known [3].Why is it that, even when agreeing on what needs to be done, city institutions and infrastructure are resistant to change towards more sustainable pathways?
We believe that part of the answer lies in the way that urban knowledge systems-the social practices through which knowledge, ideas, and beliefs are produced, circulated, and put into action-keep certain patterns of thinking in place.Events like Hurricane Katrina and Superstorm Sandy, for instance, have exposed failures in the knowledge systems that engineers, designers, and decision-makers used to design hurricane protection infrastructures and limited the abilities of cities like New Orleans and New York to reduce the vulnerability of their populations to various stresses and shocks, including extreme climate and weather variability [4,5].Addressing the changing conditions of the Anthropocene will thus require innovations in not only how we design cities' built infrastructures but also in how we upgrade and design their knowledge infrastructures as well [6].In other words, sustainability demands transformations in ways of thinking-or how cities think.
This article examines knowledge co-production, an idea which is increasingly popular within the environmental and sustainability research communities, as a promising approach to generate and apply usable knowledge for complex sustainability challenges [7][8][9][10][11][12].In its most robust form, knowledge co-production refers to linked practices of knowledge production and application where diverse science, practice, and policy actors collectively identify problems, produce knowledge, and put that knowledge into action through collaboration, integration, and learning processes [13][14][15].Knowledge co-production re-thinks the relationship between knowledge and decision-making beyond conventional notions of the 'science-policy interface' that assume that knowledge production and decision-making happen independently from one another [16,17].This approach is deemed promising for building knowledge systems for cities because it acknowledges the diversity of actors, knowledge systems, social relations and networks involved in creating and applying knowledge relevant to sustainability [18,19].
Too often, however, experiments in knowledge co-production suppose that the construction and use of new knowledge can simply happen de novo, independent of what has come before.Yet, as illuminated in detail by Sheila Jasanoff [20], in regulatory settings, the construction and use of knowledge is deeply intertwined with arrangements and practices of governance-and cities are no different [21].How cities know and how they design social and policy arrangements go hand-in-hand; they get made and produced together.Knowledge both is an outcome of governance and creates the conditions for it.It contributes to, comes to be embedded in, and helps to construct shared beliefs, discourse, practices, policies, and visions.Thus, the city transformations envisioned by advocates of knowledge co-production cannot be understood as mere exercises in creating and applying knowledge, however broadly sourced across diverse participants; rather, they are exercises in reconfiguring the relationships between and institutional configurations of both how cities think and how they act.They are thus social and political exercises at least as much as they are epistemic ones.
We propose in this article that an analysis of the co-production of existing knowledge-governance dynamics and conditions, as defined by Jasanoff, can help cities to understand and improve their ability to create and deploy new knowledge effectively in service of sustainability and resilience.Large investments are currently being directed towards knowledge co-production experiments in support for sustainability and resilience in cities.The project we are currently involved in, for instance, the Urban Resilience to Extreme Weather-related Events Sustainability Research Network (UREx SRN), is a $12 million dollar investment by the National Science Foundation to co-produce new knowledge and new strategies to improve the resilience of urban infrastructures among researchers, cities, and urban stakeholders.This effort engages urban governance institutions that already know in well-defined ways-and through well-defined practices and routines-that shape how they design and implement infrastructure projects and plans.Understanding how city knowledge systems and dynamics construct and shape what decision-makers already know and wish to know, vis-à-vis infrastructure in their cities, is thus a crucial prior step to investing in new organizations and policy arrangements for knowledge co-production in cities.To put it differently, analyzing how cities think is a necessary precondition to building capacities and designing institutions for knowledge co-production for sustainability.
We present knowledge systems analysis as a conceptual and empirical framework to understand how cities think.Following a review of the definitions of and approaches to co-production found in the literature, we describe knowledge systems analysis and how it can be used by both researchers and practitioners to analyze the contexts in which new efforts to co-produce sustainability and resilience knowledge and action are situated.In particular, knowledge systems analysis emphasizes the structured social and institutional processes within which knowledge and information are produced, evaluated, circulated, and applied in governance and decision-making [22,23].We then highlight key innovations and design philosophies that we think can advance efforts to co-produce knowledge and action for urban sustainability and resilience.We conclude with suggestions for future research directions for analyzing urban knowledge systems and applying these to improving future knowledge co-production efforts.

Knowledge Co-Production for Sustainability and Resilience
There are two main interpretations and uses for the term "co-production" [9,21,24].Within the sustainability science community, knowledge co-production, as van Kerkhoff and Lebel [9] define it, is a prescriptive and instrumental form as it invokes an agenda where relationships can and should be deliberatively designed and managed for improving the scientific basis of decision-making at the project and program scale.This instrumental use of the concept involves shared or collaborative knowledge production to link knowledge to action.Specifically, this literature focuses on how to make knowledge systems-or the institutions to harness science and technology for sustainability-more effective [15].A key finding of this line of research has shown that knowledge systems are most likely to be effective in influencing action if they are perceived to be salient, credible and legitimate by the larger stakeholder community [15].This idea of knowledge co-production has taken hold most notably in the contemporary literature exploring science-policy interactions in part as a response to failed conventional science-policy models that assumed that if you get the science right and put it in the hands of the right people, it will be used automatically to inform decision-making.Examples of these conventional models include the loading dock model, where science is transferred to the policy 'dock' through a one-way loading truck, or the bridge model, wherein academia and policy engage in a two-way interaction by building bridges between the two [17].By giving a new look into how science-policy interfaces are organized, the literature is moving away from looking at the relationship between science and society as a one-or two-way interaction to more of a complex relationship in terms of multiple actors and knowledges, multiple interactions, and multiple mechanisms (see for instance [10][11][12][13][14][15][16][17][18][19][24][25][26]).
The recent popularity of organized arrangements, such as "boundary organizations" in sustainability science [26], reflects the growing importance and social investment given to these institutional approaches as a way to effectively link knowledge systems with user demands [27].Other examples of knowledge co-production ideas put into practice include joint knowledge production [13], collaborative adaptive management [14], transdisciplinary research [28], and communities of practice [12].Throughout each of these flavors of knowledge co-production there are several common themes.Building trust between and amongst both researchers and stakeholders and developing a common sense of project goals is fundamental to the process.Collaborating with a broad and relevant range of stakeholder groups [29] with different skills and assets (e.g., knowledge brokers, assessment teams, implementers, and bridging agents) across project elements, including the articulation and identification of knowledge needs and questions, is also crucial to maximize knowledge co-production [30].These practices and an open, deliberative, transparent setting that promotes trust help to promote mutual, social learning-a goal as important as more specific project specific deliverables [10].
The other form of the concept of co-production has a long lineage as an analytical lens in the fields of history of science and science and technology studies (STS), particularly through the work by Sheila Jasanoff on the dynamic interaction between the production of knowledge and social order [20].According to Jasanoff, the idiom of co-production highlights the mutually constitutive, interactive, and influential arrangements of knowledge-making and decision-making in various aspects of political life-knowledge both shapes and is shaped by social processes.In Jasanoff's words, "the ways in which we know and represent the world (both nature and society) are inseparable from the ways in which we choose to live in it" [20] (p.2).Therefore, the production and use of knowledge is deeply embedded in all kinds of social, cultural, and political dynamics, such that what we know cannot be separated from how we act and organize the world.
This version of co-production brings into focus underlying knowledge-power dynamics and social practices that can help to explain both how worldviews and ways of thinking remain in place and difficult to change and how they change over time.For Jasanoff [20], co-production emerges from the constant interplay of different cultural domains, including the cognitive, the material, the social, and the normative.Jasanoff and Wynne [31] further argue that these cultural domains can vary across different policy cultures-bureaucratic, civic, economic, scientific (Figure 1).These policy cultures have different knowledge-governance formulations such that they share practices for producing knowledge that also align with how they view and understand how the world works, and more importantly, how it should work.These policy cultures are constantly interacting with each other, but they are also competing forms of rationality that shape social order, within their own domains and across the collective whole.
the constant interplay of different cultural domains, including the cognitive, the material, the social, and the normative.Jasanoff and Wynne [31] further argue that these cultural domains can vary across different policy cultures-bureaucratic, civic, economic, scientific (Figure 1).These policy cultures have different knowledge-governance formulations such that they share practices for producing knowledge that also align with how they view and understand how the world works, and more importantly, how it should work.These policy cultures are constantly interacting with each other, but they are also competing forms of rationality that shape social order, within their own domains and across the collective whole.Research by Carina Wyborn on what she terms "connectivity conservation" offers an empirical example of the application of co-production as analytical lens.Wyborn [24] operationalized Jasanoff's categories of co-production-context (material), knowledge (cognitive), process (social), vision (normative)-as a lens to empirically examine co-production processes in two cases of connectivity conservation in the US (Yellowstone to Yukon Region) and Australia (Habitat 141°).In both cases, researchers and practitioners were attempting to establish knowledge co-production efforts to facilitate the link between conservation science and governance.Wyborn found that, while both cases had similar propositions of the relationship between science and governance, the ways in which the work played out in each case to co-produce context, knowledge, process, and vision of governance determined the different framings and outcomes.In other words, on-the-ground knowledge-power dynamics played out differently in each case, and in each case the dynamics did not correlate with the design principles for linking conservation science and action.Wyborn suggests that highlighting Research by Carina Wyborn on what she terms "connectivity conservation" offers an empirical example of the application of co-production as analytical lens.Wyborn [24] operationalized Jasanoff's categories of co-production-context (material), knowledge (cognitive), process (social), vision (normative)-as a lens to empirically examine co-production processes in two cases of connectivity conservation in the US (Yellowstone to Yukon Region) and Australia (Habitat 141 • ).In both cases, researchers and practitioners were attempting to establish knowledge co-production efforts to facilitate the link between conservation science and governance.Wyborn found that, while both cases had similar propositions of the relationship between science and governance, the ways in which the work played out in each case to co-produce context, knowledge, process, and vision of governance determined the different framings and outcomes.In other words, on-the-ground knowledge-power dynamics played out differently in each case, and in each case the dynamics did not correlate with the design principles for linking conservation science and action.Wyborn suggests that highlighting how co-production shapes the relationship between science and governance can be a fruitful contribution to the design of efforts to advance knowledge for adaptive governance.
The analytical form of co-production resonates with the concept of "knowledge governance" that is developing in the sustainability literature.While this concept has a distinct interpretation in organizational economics as an approach to maximizing knowledge transactions to improve organizational efficiency [32], the analytical form of co-production we are discussing here is more closely aligned with the critical lens of socio-political approaches described in van Kerkhoff [33].Specifically, like knowledge governance approaches, co-production analyzes direct attention to the formal and informal rules, conventions, and networks of actors that shape the ways we approach knowledge processes, such as creating, sharing, accessing, and using knowledge [33][34][35][36].Similarly, knowledge governance focuses on a broader level than the project-based use of knowledge co-production through joint knowledge efforts or boundary management, to what van Kerkhoff describes as the middle layer where the institutional 'rules of the game' shape the possibilities and choices available to decision-makers and organizations.A key distinction, however, is that, by examining how these knowledge governance dynamics are embedded in broader social, political, and cultural dynamics, Jasanoff's co-production goes further to describe the macro-social processes that link how we govern knowledge with how we govern society [20].At the same time, knowledge governance, like knowledge co-production, tends to focus more on how knowledge gets made and less on the organization of decision-making as an instantiation of particular ways of knowing.Still, there are important similarities, and the co-production and the knowledge systems analysis framework we present in the next section lend themselves to examining existing knowledge governance dynamics and conditions that may enhance or constrain cities' knowledge processes in cities.

Knowledge Systems Analysis: A Framework to Design Knowledge Co-Production in Cities
Both variants of the concept of co-production we have discussed are important for urban sustainability.Together, they present a more sophisticated and nuanced view of the relationship between knowledge and action.No longer is the relationship between knowledge and policy seen as a one-way or two-way interaction where knowledge is generated on one side, (the 'knowledge' side of scientists and/or experts that is then transferred to the other side), and 'policy' on the other side (where decision-making bodies use the knowledge).Rather, the interactions of knowledge and decision-making in governance processes are much more complex, especially as we seek to transform both how institutions think and act in pursuit of greater sustainability.Knowledge is rarely singular, for example, in sustainability problems, nor is governance; instead, multiple knowledge institutions intersect across a multiplicity of governing sites that transcend traditional institutional and jurisdictional boundaries [37].
Ideas of co-production particularly highlight the challenges to changing how cities think.They show that the social organization of cities is closely coupled with how cities organize knowledge, such that to re-organize and transform cities requires simultaneously changing how they organize knowledge production and how they put that knowledge to use in formulating policy.At the same time, to re-organize knowledge requires understanding how urban governance and life function socially, politically, and economically, including the factors that enable and constrain the possibility of change in urban knowledge systems.Therefore, in efforts to create and apply new knowledge for urban sustainability and resilience (knowledge co-production), a crucial first step is to understand the complex ways in which epistemic and governance practices are already interlaced across diverse city processes and institutions (the co-production of knowledge and governance).
Cities present a great challenge to the design of knowledge co-production approaches for sustainability.We are concerned that efforts to engage in knowledge co-production in support of urban sustainability and resilience generally lack a thorough examination of how cities think-what local people know about the city, how they know and experience the city, how they envision the city.Cities are more than the physical and institutional infrastructures that service an urban population.Cities are also spaces where a high diversity of organizations and their knowledge systems can come together in networks to catalyze or oppose new ideas and innovations.While urban governance scholars recognize the importance of multiple knowledges or expertise in researching and developing strategies toward the sustainable city [38,39], they may lack a critical analysis of the politics and power dynamics surrounding expertise, of the institutional practices that shape what knowledge is produced and how cities are envisioned, and whether capacities are present to rethink and reconfigure the linkages between knowledge and action.
We present knowledge systems analysis as a framework to describe and analyze existing knowledge and governance interactions as pre-conditions to designing knowledge co-production efforts for urban sustainability and resilience (Figure 2).We define knowledge systems as the organizational practices and routines that generate, validate, communicate, and apply knowledge [22,23].We consider knowledge systems as more than sites where research, data, and information are produced and used in decision-making.They are also where imaginations, ideals, and beliefs of social order are being forged by different social groups [37].Knowledge systems frame which questions get asked, and which don't, and determine the methods used to answer those questions.They define assumptions, establish burdens of proof, and decide who does the review and how.They lay out how to decide when knowledge is uncertain and what to do if it is and they set limits on the boundaries of relevant expertise.They also set priorities for investments in new knowledge.Another important distinction in the way that we view and use knowledge systems is how we define knowledge.We define knowledge as a claim or an idea or belief that someone, whether an individual or a community, takes to be true, or at least relatively more true than other kinds of statements, and therefore of sufficient merit to guide his, her, or their reasoning or, especially for our purposes here, action.This definition of knowledge stems from a sociological perspective that acknowledges the complex judgments, ideas, framings, tacit skills and values that shape what knowledge is, rather than viewing it as just simple statements of truth or fact [31].As Jasanoff [40] argues, to understand knowledge requires understanding knowledge-in-the-making. Dynamic social processes are involved in the making of knowledge such that its production is a result of the articulation, deliberation, negotiation, and valorization of particular knowledge claims.The structure and dynamics of these social processes determine, in turn, whose knowledge claims matter and how claims are constructed, evaluated, contested, and sanctioned as knowledge [41].This view of knowledge is a fundamental basis for the "idiom of co-production" that informs our knowledge systems analysis framework.
Our approach to analyzing knowledge systems begins with an examination of the elements of knowledge systems, including the content and types of knowledge being produced, the values, standards, and epistemologies (or ways of knowing) that guide its work, and the social practices and structures involved in creating and applying knowledge.Knowledge systems analysis describes what the knowledge system knows as well as what it doesn't know (e.g., the tacit and explicit uncertainties that surround knowledge claims and the kinds and varieties of knowledge claims that the knowledge system might produce but doesn't for one reason or another), as well as the values, methods, and epistemologies that organize how the system knows what it does (Table 1).In addition, knowledge systems analysis focuses on the people and social practices that make knowledge.The practices used to make knowledge are often hidden from plain view, even to those who are producing or using this knowledge.Even less obvious are the cognitive and cultural dispositions that shape how groups and institutions think.More often than not, organizations take for granted how they know what they know.Much like journalists use a variety of sources to put together a story, or an archeologist uses material and textual tools to 'dig up' evidence from the past, knowledge systems analysis uses different conceptual lenses and approaches to map and describe where within a knowledge system a particular knowledge is located: who knows what; where data is generated and stored; and how, where, and to whom it flows as it is processed, handled, shared, and used.At the same time, knowledge system mapping requires understanding how the people involved in knowledge systems are organized, trained, evaluated, and rewarded for their work [22,23].

Knowledge Claims
Statements or propositions about the world whose relationship to truth cannot be easily or directly ascertained.Whether they are correct or not is always uncertain, at least to some degree.
The statement that "the 2010 Census enumerated 308,745,538 people in the US" is a claim, since the Census cannot obtain an exact count of every single person in the country.

Values and Standards
Define the foundation of knowledge production in the system through a process of simplification, or creating simplified representations of complex social and/or natural processes.Knowledge systems analysis also examines the tasks or functions of knowledge systems, with an emphasis on four key functional areas: knowledge generation, validation, circulation, and application ( [22,23]; Table 2).Knowledge generation refers to the act of generating knowledge through research, for instance, whether it is scientific research, market research, or journalism.This process involves problem formulation, data collection and analysis, and reporting of information.The second function of knowledge systems is knowledge validation and involves the practices by which knowledge is subject to review, critique, assessment, and check.A common example is the review process used to publish scientific papers or expectations that journalists check the facts of their story before publishing it.Rules and expectations for reviewing and judging the validity of the information can vary from scientific journals and media outlets, so part of analyzing knowledge systems involves figuring out what the expectations are, who determines them, and how are they put into practice.Knowledge circulation refers to the practices of communicating, exchanging, transmitting, or translating knowledge from one person or organization to another.Other ways people often refer to this activity are knowledge exchange or information flows.Describing this activity involves sorting out who has access to new knowledge claims, through what channels, and what forms of communication are used and whether these are properly communicating knowledge.The Census data collection process involves significant fieldwork (e.g., surveyors that travel around communities knocking on doors for people to fill out their forms), but also legal and political work that govern knowledge generation (e.g., Congress writes laws specifying how the Census will be conducted).Agencies must also develop regulatory processes to determine exactly what data to collect and which methods to use.

Validation
The practices, processes, and routines by which knowledge claims are subject to review, critique, assessment, or check.Includes who in a knowledge system is assessing, reviewing, testing, or otherwise checking the knowledge that is being generated.
The National Science Foundation peer review process is known for the rigor of its procedures and the caliber of the scientists that it brings together to evaluate the quality of the research generated by the agency's funding.

Circulation or Communication
The practices by which knowledge claims are exchanged, transmitted, or translated from one location to another.Involves sorting out who has access to new knowledge claims, through what channels, whether those are the right people, whether the forms of communication are properly communicating enough additional information to judge a knowledge claim and its value.
Nutritional labeling in the US is an explicit effort to ensure that knowledge claims are circulated to a wide array of citizens.The standardization of food packaging labels enhances consumer decision-making by making knowledge available and easy to read at the time of purchase.

Application
The social and institutional practices by which knowledge is factored into decisions.This phase is often also referred to as the use, uptake, or consumption of knowledge.
Regulatory agencies, like the EPA, have internal and external processes, such as administrative hearings, to present and review relevant scientific research when constructing a new regulatory rule.The agency must decide how to put the knowledge collected and reviewed to use, typically through formal and informal conversations and deliberations, an official judgment and then formal statement by the Administrator.
A final function of knowledge systems is knowledge application.This phase is usually where most of the literature on linking knowledge to action and knowledge co-production focuses, as it refers to the social and institutional practices by which knowledge is factored into decisions, or put into action by decision-makers and stakeholders.In other words, this is the phase related to users and consumers of knowledge, or knowledge users.For instance, we know from previous research that knowledge systems tend to be more effective when the knowledge is viewed as credible, legitimate, and salient by multiple stakeholders (e.g., [15]).We know less, however, about the nuances of how exactly knowledge is acted upon and how this use of knowledge feeds back into the other functions of knowledge systems (generation, validation, and circulation).Who acts on particular kinds of knowledge?What other knowledge systems do stakeholders already rely on to make their decisions?What expectations do they have about the knowledge system?How is uncertainty about the knowledge being communicated?What do users know about how the knowledge was generated, validated, and circulated in order to evaluate whether the knowledge expressed is credible, legitimate, and salient?These questions raise the point that the functions of knowledge systems are not independent of one another, but rather are tightly coupled, with each facet of the system reinforcing the others.
More often than not, the co-production of knowledge, decisions, and actions around sustainability and resilience involve many diverse institutions.The functions and tasks of knowledge systems are thus often distributed across multiple organizations with varying structures, goals, and degrees of accountability.In boundary organizations, for instance, Guston describes multiple lines of accountability to both scientific and political organizations [26].In addition, because knowledge co-production efforts attempt to bring together different types of knowledge and expertise, the intertwining of multiple epistemic cultures will likely bring to the fore both epistemic conflict over different assumptions about how the world works and political conflict over whose expertise should count in decision-making [42,43].
In our framework we describe these dynamics in terms of three layers of complexities in knowledge systems: organizational, operational, and political ( [22,23]; Table 3).Organizational complexity arises when multiple organizations or networks are involved in knowledge production.Operational complexity refers to instances when the goals and values underlying the collection of knowledge, and the processes needed to generate that knowledge, are not obviously aligned; thus considerable effort needs to be placed to coordinating activities and routines, such as standardizing research protocols, to ensure the credibility of the system.Political complexity arises when the work and products of knowledge systems become entangled with politics or conflicts within or between organizations.The case studies examined by Wyborn, which we reviewed briefly above, offer a good illustration of these complexities [24].Both cases showed significant organizational and political complexity as they engaged multiple science, management and policy organizations to outline strategies for connectivity conservation based on conservation science, yet neither was effective at actively connecting science with governance.In the US case, the Y2Y conservation proposals in Yellowstone experienced backlash from the local community because the proposals used science to justify a narrow vision of appropriate land-use that did not line up with local normative visions of how the landscape should be managed.While the Habitat 141 • 's science vision wasn't in conflict with local governance goals, project leaders couldn't re-organize themselves appropriately because of disagreements over where decision-making power for conservation actions should be located.The leadership was not able to coordinate the organizational and operational complexity involved in a large-scale conservation project involving multiple institutional levels.These examples again highlight the importance of paying close attention to how key actors and stakeholders formulate and re-organize themselves to reconcile tensions between science and governance.
Another illustrative example specifically related to knowledge systems dynamics in cities is the case of land use planning in San Juan, Puerto Rico, described by Muñoz-Erickson [21,44,45].In 2009, increasing development of the city's green areas, especially in the upper headwaters of the main watershed of the city, exposed many residents to river and urban flood risks.While the Municipality's land use regulatory framework included protection of these green areas as part of the sustainable development of the city, projects were still permitted.In her analysis of the land use governance landscape in San Juan, Muñoz-Erickson [21] found that, in addition to economic and political interests, knowledge systems also played an important role in shaping outcomes.Relevant factors included a lack of organizational capacity to generate and validate site-specific knowledge about proposed projects and power dynamics within the state's planning agency.The latter was key because the state continued to make decisions on land use in San Juan (based on their own knowledge systems and not the Municipality's) even though the Municipality had gained autonomy in 2003.Muñoz-Erickson applied the knowledge-action systems analysis (KASA) framework, a type of knowledge systems analysis that uses social network analysis to map and analyze co-production processes that link knowledge to action [39,45].She mapped and analyzed the network of organizations producing knowledge on land use, what frames and epistemologies where circulating across the network and how, and which organizations had greater influence over how that knowledge was applied.

Organizational Complexity
When knowledge systems are in a complex decision-making landscape that involves a multiplicity of interacting actors and viewpoints, and complicated rules of procedure.Oftentimes knowledge and decision-making become tightly coupled to one another, such that integrating new knowledge into this form of closed system can be a very difficult undertaking.
Decisions involving ecosystem services typically involve trade-offs among ecosystem services and multiple stakeholders and organizations.Knowledge of the trade-offs among ecosystem services is often absent from or neglected within disconnected decision-making processes, leading to decisions that have unexpected or problematic outcomes.

Operational Complexity
Conditions under which highly dynamic social work is necessary to carry out the core functions of knowledge systems, involving diverse participants and organizations, and requiring careful coordination across the system's many organizational components.
The UN Framework Convention on Climate Change coordinates across multiple experts and organizations the various tasks of emissions inventories, including defining which emissions to count and allocate to responsible parties, the standardization of those methods, and the review processes by independent experts from other countries to ensure transparency.Boundary work and orchestration are also crucial functions to ensure legitimacy and credibility across multiple institutions and forms of expertise.

Political Complexity
Conditions of high interconnection between knowledge production and the exercise of political power, especially in the presence of conflicts within or between organizations.In the adversarial political context of the US, in particular, the connection of science and expert advice within many facets of decision-making in the US federal government is an illustration of the political complexity of knowledge systems.
The knowledge claims underpinning EPA regulatory decisions have been widely contested by both industry groups and environmental organizations, depending on which group perceived an interest in undermining EPA credibility on any given policy issue.Further layers of organizational complexity, e.g., the presence of the EPA Science Advisory Board, often exacerbate knowledge conflicts rather than mitigate them by presenting another opportunity for divergent views of the proper use of scientific evidence to arise and become subject to critical commentary by policy actors.
The application of knowledge systems analysis revealed various complex dynamics in San Juan's co-production processes that could serve as barriers to the design of knowledge co-production efforts to build urban sustainability and resilience pathways.For instance, while a diverse network of organizations existed to generate, exchange, and use knowledge informing Municipal land use practices, including non-governmental organizations, a significant breakdown in knowledge flow between the Municipality's office of territorial ordinance and the state's planning agency created barriers to communicating knowledge of local conditions to the state agency [23].In addition, political complexities created distinct power asymmetries that impacted the ways in which diverse knowledge systems and visions were able to inform planning processes.The Municipality's ideas and epistemic cultures, which included social dimensions of urban planning such as quality of life and equity considerations, conflicted with (and often lost out to) the state's hegemonic ideas of the city as a node for regional economic power [44].
The case of San Juan highlights the forms of organizational, operational, and political complexity that knowledge systems can experience.The knowledge systems' tasks and functions around land use planning and decisions in San Juan were carried out by multiple organizations in competition with each other.Still today, although the Municipality has sketched out a pathway towards more sustainable futures through a vision of a Livable City [44], knowledge-power dynamics may keep these ideas from moving into action.In this respect, knowledge systems analysis is useful as a diagnostic tool to examine and make explicit the interplay of values, knowledge, and power that enable and constrain research and decision-making processes underpinning elements of societal stability and transformation.With this context, one can see multiple definitions and applications of the concept of knowledge systems.We interpret these as variations across a knowledge systems spectrum that ranges from specific and tightly closed knowledge systems, such as the US Census, to more complex knowledge-action systems where multiple knowledge systems and organizations interact fluidly with one another across complex social and physical landscapes.Knowledge systems analysis is a powerful framework to uncover hidden relations and highlight the societal foundations that shape what is (and is not) possible (or, arguably more appropriately, what can be easily accomplished and what will require extensive work) in the creation and application of new knowledge to advance sustainability.The framework can help identify and make explicit the tensions and assumptions informing efforts to design and implement new knowledge-making arrangements such that they can work within, or transform, existing knowledge-power structures, thus increasing the likelihood of knowledge leading to action.In the next section we present general guidelines or design criteria that we view useful for designing knowledge co-production processes in cities from a knowledge systems lens.

Design Philosophies for Knowledge Co-Production for Urban Sustainability and Resilience
Understanding the way cities think is necessary to building knowledge infrastructures that transform cultural and institutional barriers to building sustainable and resilient pathways to sustainability.Because every context will present particular barriers and opportunities to linking knowledge and action, analyzing and evaluating existing knowledge-power dynamics can help in designing appropriate architectures for knowledge systems.Simply put, one size does not fit all in the design of knowledge systems.Simplistic assumptions about how knowledge systems work in the real world have led to a plethora of lists of ingredients for 'science-policy interfaces' with outcomes that remain unexamined.Thus, the following are not meant to serve as a 'blueprint', but rather normative and organizational elements of the design of co-production-what we might call design philosophies-that need close attention to ensure the success of knowledge co-production initiatives.Following each, we provide a set of questions and strategies to aid the design and practice of knowledge co-production.

Context and Inclusiveness
Building knowledge systems that align with the local context entails the use of more inclusive definitions and approaches for defining knowledge and the actors that produce and use it.Breaking down knowledge stereotypes is necessary, removing a priori assumptions about who produces and uses knowledge.For instance, analyzing and evaluating the local epistemic context in San Juan revealed and helped explain not only the knowledge produced and the needs of knowledge users (and gaps between them) but also distributions of power and expertise and perceptions of credibility and legitimacy across actors in the local political context.The investigation showed, for instance, a heterogeneous network of land use and green area knowledge with a variety of sources of knowledge, including organizations not traditionally perceived as experts (i.e., civic groups) [21].This may be indicative that credibility and legitimacy in San Juan is more widely distributed among a more diverse set of actors than commonly considered in US policymaking (where academic, scientific, or technical government institutions commonly predominate).Researchers and practitioners engaged in knowledge co-production processes should be exposed to and experience the complex social and institutional dynamics shaping knowledge and governance in a place.
In conducting these analyses, focus should be put on the "interactional" elements of the co-production of knowledge and governance.As described above, for Jasanoff, co-production occurs through interactions among diverse elements of and participants within a given political culture.These interactions both maintain stability but also create the potential for structural change.Except in rare circumstances, research on co-production suggests that transformational change occurs more frequently through reconfigurations of existing knowledge and political arrangements than through their replacement with entirely novel alternatives.Thus, understanding the dynamics and structures of existing knowledge systems and the ways that they contribute to larger processes in the co-production of knowledge and society-and situating new knowledge-making initiatives within this context-can help open up the potential for the kinds of major changes in cities necessary to achieve sustainability and resilience.Key questions and strategies for building context and inclusiveness in knowledge co-production.

•
Analyze existing knowledge systems; do not make assumptions about how they work in the city: What do people know or need to know about the city?Who are the key actors producing and using knowledge for urban planning and sustainability?How are their knowledge systems structured and functioning?What epistemic practices inform their visions and expectations of the city?How is their network constituted?How do the credibility and legitimacy of science and other knowledge play out in this context?What actors are perceived as credible and legitimate, and why or why not?

•
Expose researchers to these conditions and the complex social-ecological realities of the place.Ethnographic research approaches, such as field work, observations and unstructured interviews can be useful tools to build epistemic context and initiate rapport, and hence trust, with local stakeholders.

•
Identify all knowledge-relevant stakeholders (including marginal actors) and engage early to assess their needs, priorities, and existing knowledge systems.Develop trust by engaging in multiple ways, formally and informally, and continuously follow-up and communicate with stakeholders.

Adaptability and Reflexivity
Building institutional reflexivity is crucial to avoid failures in the future and build more adaptive knowledge systems.Reflexivity is the idea that those who produce and use knowledge are aware of and reflective about how they do so [23].It implies that the assumptions, framings, values, and practices underpinning knowledge production and use for sustainability be open to scrutiny [46].In other words, reflexivity calls for knowledge-producing institutions to be self-critical and routinely reflect on how they build knowledge about cities, the assumptions they make about how cities work, and their normative premises for how urban development pathways should be steered in the future.Reflexivity is related to adaptability in that the approach demands awareness of system uncertainty and unintended consequences.It goes further, however, to consider the effects that such reflection has on how we produce or change the production of knowledge, as producers and users come to terms with the impossibility of having full and complete knowledge of system dynamics [47].
From the standpoint of practice, reflexivity involves 'opening up' knowledge production processes for review and critique.In other words, it involves developing institutional mechanisms that allow outside actors, including non-scientists, to be part of the design and review of the research process [48].Much like the peer review process in science, knowledge systems need an external review body, such as extended peer communities [49] or advisory committees, to provide context and critical assessment of the assumptions, methods, and direction of research in relation to city needs, changes, and expectations.These bodies should not only bring accountability to the knowledge system by integrating various stakeholder or actor groups involved in governance but must also be inclusive of the various ideas, knowledge, and values needed to address and be congruent with the system.A reflexive approach to improving a knowledge production process, however, brings up an 'efficiency paradox' as it implies a balance between opening up and closing it down [50].Closing down is necessary to do the work and have the ability to act, but the timing of closing may cause rigidity.Voss and Kemp argue that the issue is not a matter of either/or but of doing both throughout the knowledge co-production process [50].The key to this balancing act is the timing and structure of mechanisms to open up using an iterative process.For instance, broad inclusiveness is crucial in the beginning and final phases of a project, therefore using methods that allow greater representation and deliberation of ideas, viewpoints, and epistemologies.Other points in the stage are more technical and may require a narrower and more specific set of expertise to review and provide critique (but be wary of too glibly assuming this; even minimal checking in with stakeholders can help spot problems early).Finally, mechanisms for monitoring and evaluating the knowledge production process are crucial to assess whether learning is occurring and if both ecological and social outcomes are being met.Key questions and strategies for building adaptability and reflexivity in knowledge co-production

•
Institute an advisory review body, in which both political interests and epistemologies (ways of knowing) are represented, and build accountability in the knowledge production process.

•
Be flexible with engagement methods-use a variety of methods with varying frequencies, including consultative (e.g., surveys, rapid appraisals), informal meetings (e.g., office visits, fields trips), and active participation (e.g., engagement in decisions on research) to develop an appropriate framework that fits local context and the diversity of ways that researchers and practitioners are able to engage given different reasoning styles, time, and other capacities.

•
Iteratively frame research agenda and process; approach knowledge systems as experiments; evaluate and adapt.

•
Monitor knowledge systems through learning indicators and knowledge system analysis and evaluation.

•
Account for the 'intangibles', or non-quantifiable elements, of quality of life in a city.

Knowledge-Action Networks
While the previous two design philosophies related more to the dynamics and functions of knowledge systems, attention to knowledge-action networks focuses on the structure, or architecture, of efforts to design new strategies for creating and applying knowledge for advancing sustainability.We use the term knowledge-action networks to refer to the multiplicity of spaces (i.e., nodes), both physical and organizationally, where knowledge and action interact frequently.In San Juan, for instance, this happens not only in expert organizations that produce knowledge and link it to action through various means for circulating and applying it but also to places where a diversity of ideas about urban sustainability are being constructed and deliberated, such as community meetings, coffee shops, and even churches.As we suggested above, in the first design philosophy, the architecture of new knowledge systems and knowledge co-production processes needs to engage with existing knowledge systems and their relationships to the ecological and political landscape of the city, to be most effective.In this way, the new interactions stimulated by knowledge co-production initiatives can help catalyze the transformations necessary for sustainability and resilience.Network theory reinforces this perspective, observing that creativity and innovation are best fostered by diverse and polycentric networks, as opposed to isolated networks composed of siloed entities with similar views and perspectives.A polycentric design entails strengthening existing capacities and connections where there are weak links and building new ones where they are absent.Interventions, such as establishing new knowledge co-production efforts, should take these local network properties into consideration and build on them, enhancing polycentricity and opening up possibilities for change [18].
Following the adaptive and reflexive approach proposed here, this structure needs to reflect the knowledge-power relationships in these networked and complex contexts, while at the same time be adaptive and recognize when re-organization or new institutional arrangements are needed for knowledge production.The structure also should be flexible enough to help link existing knowledges together (and to action) and facilitate knowledge flows where needed, thus allowing local stakeholders to feel ownership of the knowledge co-production process.Monitoring and evaluation of knowledge systems functions and performance is part of designing a reflexive structure.Strong leadership is needed to manage knowledge systems complexities (e.g., organizational, operational, political) and to work with existing capacities/projects so as to not compete or be redundant.Developing and maintaining a network imaginary, as Goldstein and Butler [51] have proposed for the US Fire Learning Network (FLN), is an approach that can provide the cultural and organizational 'glue' that helps balance the social cohesion, yet flexibility, of a distributed knowledge-action network.The authors describe that the FLN is able to maintain an extensive network of research nodes across the US without the need for a hierarchical authority structure, by articulating a network imaginary through technologies, planning guidelines and media.Put differently, a shared-mental schema of a community of diverse interests and knowledge but with a common goal (i.e., manage fire) was created and perpetuated through the communication and research practices of the network such that people working at different locations feel part of this imagined community.
Key questions and strategies for designing knowledge co-production.

•
Evaluate and invest existing institutional structure and capacities for co-production: do not assume capacity is already there.Where capacities do exist, work or help transform them, instead of automatically building new structures (e.g., new organization).

•
Recognize that in an increasingly networked society, power and knowledge are distributed, thus the knowledge-action networks need to be cognizant of the distribution of expertise in the governance space and the inevitable political, organizational, and operational complexity that this creates.

•
Develop epistemic or transdisciplinary consortiums: instead of looking for uniformity or consensus, foster diversity and pluralism of ideas, knowledge and ways of reasoning.Individuals trusted and deemed credible by researchers and stakeholders alike can serve as the 'mediators' between knowledge and action.

•
Create a variety of spaces and/or activities or support others in leading them (i.e., field trips, seminars, workshops, retreats, office visits, etc.) to deliberate research questions and outputs such that stakeholders feel ownership of the process.

•
Develop a network imaginary as the cultural glue to keep the network together and thus allow actors to have ownership of the process and outcomes of the networked structure.
As we mentioned earlier in this section, these design philosophies are meant to highlight key normative and organizational dimensions of co-production that require close attention.More empirical research is needed to explore how these philosophies can guide innovations in practice and to evaluate the results in advancing urban sustainability and resilience.The set of questions and strategies we present here offer a starting point.

Conclusions
Co-production requires a fundamental transformation of both knowledge and governance toward more critical, inclusive and reflexive practices.The social, institutional, and ecological complexities of cities defy simple arrangements that link knowledge producers on one side and knowledge users on the other.Instead, institutional arrangements that are able to meaningfully engage the institutional and ecological complexity and dynamism of cities are more likely to be effective in generating useful and innovative strategies for sustainability and putting them to work to create long-term transformation.A lack of awareness of how these existing knowledge systems work can have unforeseen consequences on the resilience of cities.
In this study we discussed knowledge systems analysis as a conceptual and empirical framework to understand how cities think.This framework is useful to both scientists and practitioners interested in designing knowledge co-production efforts to produce better knowledge and facilitate successful implementation of sustainable outcomes.It provides a way to understand existing institutional conditions, as well as to build reflexivity and change through its long-term application to evaluate how existing and new knowledge co-production processes perform over time.Future research should apply this framework to understand co-production in multiple cities and for multiple resource domains (e.g., water, energy, etc.) to develop more robust assessments of how these systems work in multiple sustainability contexts.Experimenting with different institutional configurations could also provide a way to test the design propositions recommended here.Doing so will create new insights into the arrangements and stakeholder engagement processes most useful to tackle urban sustainability issues.
We also hope that future research in this area can broaden the scope of how knowledge systems are addressed in sustainability science and science and technology studies (STS) by acknowledging the complexity of these systems, especially in cities, and presenting ways to tackle this complexity analytically.The use of multiple, interdisciplinary concepts and methods can highlight important institutional and epistemological aspects of knowledge systems that are more difficult to assess through a single analytical approach.From a practical perspective, understanding the complex workings of knowledge systems has important implications for how we design and build them in practice.Thus, linking knowledge to action is not as simple as building 'interfaces' or other institutional arrangements drawn from theoretical designs.Rather, it requires that we first assess how knowledge gets made, vetted, circulated, and applied within complex political and institutional terrains, such that whatever intervention we design not only makes sense within that place but also has the interactional capabilities to create necessary change.The knowledge systems analysis framework challenges researchers and practitioners in cities to ask themselves: are the social and institutional conditions of the system they are working in conducive to knowledge co-production efforts?If not, why not?What needs to change to build an urban knowledge infrastructure for sustainability and resilience?If yes, what kinds of capabilities are necessary to transform knowledge co-production from a new way of thinking about knowledge to a force for effective change?Ultimately, the goal of understanding how cities think is not only to help produce better outcomes for knowledge co-production efforts but also to provide a window into the adaptive capacity and transformation potential of cities.

Figure 1 .
Figure 1.Illustration of the model of co-production of knowledge and society.Derived from Jasanoff and Wynne 1998 (Battelle Press: Columbus, OH, USA)

Figure 1 .
Figure 1.Illustration of the model of co-production of knowledge and society.Derived from Jasanoff and Wynne 1998 (Battelle Press: Columbus, OH, USA).

Figure 2 .
Figure 2. Main components of the knowledge systems analysis framework.

Table 1 .
Elements of knowledge systems.

Table 2 .
Functions of knowledge systems.

Table 3 .
Complexities of knowledge systems.