Scientist Engagement with Boundary Organizations and Knowledge Coproduction: A Case Study of the Southwest Fire Science Consortium

Abstract

Knowledge coproduction is increasingly advocated as a way to address complex socioecological issues, such as catastrophic wildfires. In turn, attention has been paid to boundary organizations to foster knowledge coproduction. Despite this growing interest, little research has examined the interplay between knowledge coproduction, boundary organizations, and scientists. We interviewed scientists involved with the Southwest Fire Science Consortium (SWFSC) to examine (1) relationships between their engagement with SWFSC and knowledge coproduction in their own work and (2) SWFSC’s role in fostering participation in knowledge coproduction. Overall, scientists more engaged with SWFSC reported involvement in a wider variety of knowledge coproduction activities. However, some knowledge coproduction activities, especially those requiring greater time investment or facing institutional barriers (e.g., research collaboration) were less common among all participants. Most scientists involved in knowledge coproduction believed that SWFSC increased their participation in these activities outside the boundary organization context, in part because SWFSC provided opportunities to interact with and understand the needs of managers/practitioners, as well as build research collaborations. Findings indicate that boundary organizations, such as SWFSC, can foster knowledge coproduction, but that they may need to further explore ways to address challenges for knowledge coproduction activities that involve greater time commitment or institutional challenges.

1. Introduction

Over the past decade, there has been an increased focus on knowledge coproduction as a means to generate workable solutions to socioecological challenges, such as mega-wildfires and climate change [1,2]. (Several similar terms for coproduced knowledge exist, as different sectors and disciplines use varying terminology and meanings may have changed over time [3,4]. For clarity, we use “coproduced knowledge” and “knowledge coproduction”, but refer to literature employing similar concepts (e.g., translational ecology)). Knowledge coproduction has been defined as “the process of producing usable, or actionable, science through collaboration between scientists and those who use science to make policy and management decisions” [5] (p. 179). However, despite a focus on increasing knowledge coproduction, questions remain as to how and the best methods to conduct knowledge coproduction, as well as its potential weaknesses. Boundary organizations, or organizations that play a bridging role between producers and users of science, have often fostered knowledge coproduction by increasing interactions and communication between these groups [4,5,6]. Recently, there has been an increased desire to understand the ways in which scientists can include knowledge coproduction and related practices in their research, with or without the involvement of a boundary organization (see the 2017 Frontiers in Ecology and Environment special issue on Translational Ecology). Several recent papers have laid out methods and approaches to assist scientists with conducting knowledge coproduction [7,8,9].

Despite growing interest in coproduced knowledge, scientists, and boundary organizations, the relationship among the three has been less examined [6,10]. To build on existing research examining interactions between knowledge coproduction, scientists, and boundary organizations, we explored the level of scientist engagement in a boundary organization, the Southwest Fire Science Consortium, and how that relates to involvement in knowledge coproduction activities in their work outside the boundary organization context. As the need to address socioecological challenges grows, calls for scientists, managers, and practitioners to work together increases [5]. Understanding the relationship between scientists and boundary organizations could help both parties increase and foster such opportunities.

1.1. Knowledge Coproduction

Coproduced knowledge differs from many other more traditional scientific approaches. Some of these approaches fall under what has been described as Mode 1 knowledge production [11,12], which occurs in an academic context and involves disciplinarity, homogeneity, autonomy, and traditional quality control. Mode 2 knowledge production (i.e., knowledge coproduction) can supplement Mode 1 by expanding beyond scientific institutions and specific disciplines to involve transdisciplinary collaborations, reflexivity, social accountability, and novel quality control [12]. Coproduced knowledge is a “collaborative research approach that explicitly involves end users in all stages of the knowledge development and dissemination to varying degrees, but which maintains an emphasis on two-way, iterative, and sustained interactions and the production of actionable science” [4] (p. 919). These stages include problem framing, research design, implementation, and exploring impacts of the process [13,14]. Trust building is also an important part in the process of knowledge coproduction [13,15].

With coproduced knowledge, methods of science delivery can also differ from more conventional mechanisms, such as peer review papers, reports, or presentations. While Mode 1 knowledge production can result in practical applications, it usually requires knowledge transfer to cross the gap [12], which is often one-way, from scientist to decision maker. Cash et al. [16] compared this process to a loading dock, where scientists conduct research, publish peer-reviewed papers, and leave the papers for managers to find, interpret, and use in decision-making. In contrast, although products that are coproduced might be less tangible (e.g., new processes, events, or traditions) than scientific papers, they might be considered more legitimate because they bring together expertise from many perspectives, allowing better access to the knowledge-production process for all partners involved [6,17]. However, it is also possible and important for the coproduction process to result in peer-reviewed products, given the higher level of legitimacy ascribed to these products [16]; these papers would also include non-academic authors involved in the process.

Despite knowledge coproduction’s growing popularity [1,3], it has not escaped critique. Bremer and Meisch [1] revealed that, in many articles, the concept of coproduction is “highly overlapping, messy, and in flux, not unlike other popular conceptual meeting points around ‘resilience’ or ‘sustainability’” (p. 6). Challenges can arise because definitions of and approaches to knowledge coproduction differ depending on the field (e.g., sustainability science, public administration), researcher focus and interest, and scope (e.g., project-level vs. policy changes) [14,17,18]. Wyborn et al. [19] noted that knowledge coproduction risks trying to accomplish too much, and other researchers have discussed challenges of engaging in knowledge coproduction given what the entire process may entail (i.e., stakeholder trust; commitment; and interaction at multiple stages, including, amongst other factors, research development, dissemination, and evaluation [5,20]). Other barriers include fears of maintaining scientific credibility and objectivity; professional, personal, and scholarship risks; diversity of partners; time and practice in learning how to frame research to address management needs; and institutional cultures [8,18,20,21].

In addition, Meadow et al. [5] pointed out that while there has been greater recognition for what coproduced knowledge could look like, far less work has examined the mechanisms for achieving these goals. Lack of a “standardized or precise empirical framework” for knowledge coproduction can result in a great variety of practices, attitudes, and expectations, which could undermine the approach (p. 886). This variety can be seen in the proliferation of how-to guides [22], which not only illustrate lack of standardization, but can perpetuate inequities and existing power structures (e.g., whose knowledge, who benefits, who decides), especially if coproduction is seen primarily as a means of ‘solving’ scientific problems [19,23].

1.2. Boundary Organizations and Coproduced Knowledge

One approach to facilitating knowledge coproduction is through boundary work, boundary spanners, and boundary organizations [4,5,6,24]. Boundary work is described as the act of working across the boundary between science research and science use [2], while boundary organizations are organizations that operate between the worlds of science production and science use to carry out boundary work [15]. Individuals working at the interface of science and management are considered boundary spanners [25]; they may or may not work within a boundary organization [6]. This paper focuses on boundary organizations and the role they may play in facilitating knowledge coproduction activities among the scientists with whom they engage.

Although boundary organizations do not necessarily address the problems described in the previous section, they may foster the process given their aim to bridge the gap between producers and users of science to enable knowledge exchange [4,26]. Recent research has focused on the role boundary organizations and spanners can play in facilitating coproduced, useable knowledge and science delivery, as well as how this might lead to more productive decision making by fostering interactions and two-way, sustained knowledge exchange between groups (e.g., scientists and non-scientists) with potentially strained relationships [2,4,6,27,28,29]. In addition, boundary organizations can reduce the costs of coproduction, “especially during the long period required to build trust and legitimacy” [22] (p. 723).

Several papers have examined mechanisms that can aid boundary organizations in the facilitation of coproduced knowledge, as well as challenges boundary organizations may face. To be successful, boundary organizations and information produced and disseminated must be recognized as scientifically credible, relevant to the needs of decision makers, and legitimate to both social worlds [30]. Safford et al. [6] claimed that “effectiveness in translating ecological information depends on (1) commitment to a well-planned system for boundary-spanning activities; (2) development of useful products by the boundary-spanning unit and its partners; (3) the existence of an accountability framework that includes both science providers and users; and (4) the traits of individual boundary spanners”. Dilling and Lemos [15] focused their suggestions on outcomes, such as scientist–stakeholder relationships, accessibility of information, and progress on societal outcomes. However, Colavito et al. [4] revealed that in boundary organizations there is often a continuum of boundary spanning activities from coproduction to science delivery, and “as you move along the continuum… there are tradeoffs associated with the number of individuals served, level of engagement and time required” (p. 928).

1.3. Scientists, Knowledge Coproduction, and Boundary Organizations

With increased attention on knowledge coproduction, there has also been growing interest in the ways scientists can engage in it. In 2017, Frontiers in Ecology and Environment dedicated an issue to translational ecology (TE), or the effective translation between good science and informed practice [31]. Enquist et al. [8] presented a framework and identified six principles that typify TE practices: collaboration (co-developed knowledge), engagement (diverse relationships and partnerships), commitment (long-term trust), communication (knowledge exchange and learning), process (buy-in, co-ownership), and decision-framing context (decision-relevant outcomes). Also in this issue, Wall et al. [29] focused on “understanding the more intangible input to research projects, stakeholder engagement, the role of social capital, and evaluating the outputs, outcomes, and impacts of translational science projects and initiatives”. Beier et al. [7] also outlined guiding principles and recommendations for all parties involved: understanding manager needs before suggesting projects, honestly conveying uncertainty and the application of results, accepting challenges by managers, evaluating work, and revisiting based on recommendations. To address the gap in practical “know-how” of coproduction work, Djenontin and Meadow [32] reviewed nine transdisciplinary and collaborative research projects and categorized components and processes for engaging in knowledge coproduction into four categories: setting up, development and design, implementation (methods, stakeholder engagement, communication), and output management and dissemination. In doing so, they revealed that projects most often involved coproduction during the implementation process (research methods, stakeholder engagement practices, and communication strategies), while the setting-up stage was most often overlooked.

However, despite interest in scientists and knowledge coproduction, as mentioned above, few researchers have examined the relationship among knowledge coproduction, scientists, and boundary organizations [6,10]. Those who have, most often have focused on the role that boundary organizations or spanners can play in helping scientists. Cvitanovich et al. [33] proposed that knowledge brokers (i.e., those whose role it is to facilitate the exchange of knowledge between various stakeholders) could help address scientists’ challenges to engaging in two-way knowledge exchange. Others have explored how boundary spanners can empower scientists, leading to more deliberate engagement with policy makers and increased spaces and roles for scientists and policy actors to coproduce research [10,25,34]. Although Wall et al. [9] focused on the role that boundary organizations might play in helping scientists negotiate knowledge coproduction, they stated that a research team itself can carry out boundary work. Tedim et al. [35] similarly described wildfire scientists with translational mindsets as ones who themselves are boundary crossers and engage with practitioners to conduct outcome-oriented research on real-world problems.

By building on the little existing research that has examined interactions among knowledge coproduction, scientists, and boundary organizations, we explored the level of scientist engagement in a boundary organization and how that relates to their own involvement in knowledge coproduction activities in their work. We focus on scientists working towards inclusion of some knowledge coproduction activities rather than knowledge coproduction as a whole process. Engaging in the whole suite of possible knowledge coproduction activities is challenging, and given that knowledge production exists on a continuum, more often people are involved in certain activities or stages in the process [4,18,32]. Our research questions were:

(1)

What, if any, is the relationship between scientists’ engagement in a boundary organization and their involvement in knowledge coproduction activities in their own work outside the boundary organization context?

(2)

Do scientists with differing levels of engagement in a boundary organization participate in similar or different knowledge coproduction activities?

(3)

Do scientists who are involved in knowledge coproduction activities feel that the boundary organization increased their use of such approaches in their own work?

To examine these questions, we focused on scientists involved with the boundary organization the Southwest Fire Science Consortium (SWFSC).

2. Materials and Methods

2.1. Southwest Fire Science Consortium

The SWFSC is one of 15 Fire Science Exchanges (FSEs) established by the Joint Fire Science Program (JFSP) to improve science delivery and knowledge exchange (Figure 1).

JFSP was developed in 1998 to fund scientific research on fire that could then be applied to management on the ground [36]. Its mission includes increasing access to scientific information, providing a knowledge base for managers, and providing credible and usable science for planning and on-the-ground application [37]. However, because the use and delivery of science was found to often be hindered by institutional, cultural, and communication barriers [38], in 2011, JFSP established the Fire Science Exchange Network (previously, Fire Science Consortia) with eight regional FSEs; they later added an additional seven. The main purpose of the FSEs is to provide and facilitate relevant information between fire researchers, managers, and practitioners [39]. Through FSEs, scientists gain a venue to communicate research findings [36]. Within the fire management community, FSEs have improved the access, usability, and application of fire science in land management and decision-making and have provided opportunities to participate in developing research agendas [36,40,41,42]. Additionally, as part of a network, the FSEs regularly communicate among themselves to improve their effectiveness.

The SWFSC’s 2016 logic model (a practical tool mapping how program activities translate into short-, medium-, and long-term outcomes to achieve large goals [41]) called for increased knowledge exchange. SWFSC proposed that this would be accomplished through several coproduced methods, including collaborative work, valuing scientific data and on-the-ground experience and knowledge, and implementation of practitioner- and manager-driven research. Additionally, in-person interactions (e.g., forums, roundtables, workshops, webinars, field trips) were anticipated to facilitate two-way communication, build professional connections between scientists and managers, and provide opportunities to collaborate and develop applicable research that is implemented on the ground. As seen in these activities, the SWFSC’s role is to bridge producers and users of science and they themselves do not conduct scientific research; their funding is outreach-based and only extends to direct programmatic research. Therefore, it provides a good case study to examine the role of knowledge coproduction in scientists’ work outside the boundary organization as there is no chance for our interviewees to conflate scientific research within and outside the boundary organization.

SWFSC communicates with a large number of users who engage with the organization to varying degrees. The contact list includes, but is not limited to, people who attended conferences co-sponsored by SWFSC, went on field trips, participated in webinars, and/or served on its governing board. All individuals on the contact list signed up themselves or asked to be signed up during an event. Given that many scientists connected to SWFSC are involved at varying levels, it offers a good case study for examining the relationship between levels of engagement with a boundary organization and scientist participation in knowledge coproduction activities in their work outside the boundary organization context.

2.2. Data Collection and Analysis

To answer our research questions, we received Institutional Review Board approval from Northern Arizona University (1033100-1) to interview scientists connected to SWFSC. In this paper, “scientist” refers to producers of scientific information, typically, those in the research community from disciplines including but not limited to climate sciences, ecology, social sciences, and hydrology. Our sample comprised scientists from the overall user group and currently/formerly on SWFSC’s governing board. Governing board members included principal and co-principal investigators, voting and non-voting members, substitute members for absences, and past board members. Most governing board members were previously general members of the SWFSC who, through increased engagement with the boundary organization, decided to join the board; therefore, although it might be expected that they now have high engagement with the SWFSC, their involvement likely grew over time. To ensure that we also included in our sample those with potentially lower levels of engagement with SWFSC, we also randomly selected non-governing board members from the contact list. We limited our sampling frame to those working in the region covered by SWFSC: Arizona, New Mexico, southern Utah, southwest Texas, and a small section of southwest Colorado (Figure 1). After deleting individuals outside these parameters, we generated a final list of 639 individuals who were classified as either scientists or managers/practitioners based on their position title and affiliation; our larger study also included interviews with 30 managers/practitioners, but these are not included in this paper’s analysis. Using Excel’s random number generator, we selected 30 scientists and emailed an invitation to participate. For those who did not respond to the first invitation, we sent a follow-up email and then telephoned. After that, it was assumed that the individual was unavailable or not interested. We then generated another list of scientists to contact until we reached our target sample size of 20 non-governing board scientists.

Between March and September 2017, we conducted semi-structured interviews in person or by phone, depending on a person’s location, availability, and preference. Semi-structured interviews follow an interview guide that allows the researcher to address the same topics with all respondents while also accommodating changes in question sequence, follow-up questions, and the addition of new topics to subsequent interviews [43]. Although we categorized individuals as managers or scientists, we asked respondents how they self-identified and used the corresponding interview guide. We asked users and producers of science similar but slightly different questions. This paper draws on scientists’ responses about science delivery and development as well as their collaboration with managers and practitioners. Interviews were audio-recorded and generally lasted 30–60 min. Between the governing board (n = 10) and non-governing board scientists (n = 20), we interviewed a total of 30 scientists, but only include analysis of 28 scientists in this paper (one scientist conducted research outside the study parameters, but that was unclear until the interview, and the SWFSC program coordinator does not produce scientific information outside the boundary organization). We assigned codes to individuals indicating their identifying number (1, 2, 3…) and category: S = scientist.

We transcribed the interviews and uploaded transcriptions to NVivo qualitative data analysis software [44], which allows researchers to develop codes, categorize sections of transcripts under those codes to identify trends, and easily retrieve quotes relevant to each code. We created a code book based on initial themes [45]. Through an iterative process of analysis, we added or combined codes. We also broke down the code related to knowledge coproduction using a priori codes detailing knowledge coproduction activities in which scientists can engage. To establish a priori codes, we combined and modified recommendations set out in Beier et al., Enquist et al. Wall et al., and Djenontin and Meadow [7,8,9,32] to group coproduced actions together under broader headings (

Table 1. Knowledge coproduction categories and activities—a priori codes. Categories are bolded and activities within the category are bulleted. A priori codes are a combination of categories and elements (e.g., Research Collaboration on: Research Design).

Decision Framing Context	Engagement
Understanding needs, values, and time frame of user	Diverse relationship and partnerships
Consideration of broader social context	Regular interactive meetings with stakeholders
Research Collaboration on:	Communication & Knowledge
Definition of problem Research question development	Multidirectional knowledge exchange and learning
Research design Data collection	Recognition of manager knowledge/scientist humbleness
Data analysis and meaning making
Development, management, and/or dissemination of research outputs
Commitment	Evaluation
Buy-in, co-ownership of project	Evaluate coproduction products, process, and/or actionability of science
Long-term trust built	Revise based on evaluation

). We recognize that there might be additional knowledge coproduction activities, but these captured many found in the literature. We focused on these methods and approaches instead of societal transformation in governance, power, and politics [17,23], because our research aimed to see if scientists were engaging in any knowledge coproduction and these more transformational aspects of knowledge production are less common [17]. Responses were primarily to the broad question “Do you work with stakeholders outside the research community on science delivery and/or research?” Follow-up questions asked for more details, such as who was included and why, what was done, outputs produced, whether the process was evaluated, successes/challenges. Informant quotes were organized into the appropriate codes to allow for analysis of themes and easy retrieval of quotes representing key themes [43,46]. We were then able to compare responses, allowing us to uncover similarities and differences.

3. Results

3.1. Participant Demographics and Engagement with SWFSC

Scientists worked at universities (n = 15), primarily in Arizona and New Mexico; federal and state agencies (n = 6, n = 1); non-profit organizations (n = 3); and consulting firms (n = 3). Scientists had many research foci, including fire ecology, forest ecology and restoration, forest regeneration, climate science, hydrology, archaeology, social science and policy, wildlife, modeling, GIS and remote sensing, fire history (although history is not usually considered science, this respondent was a producer of fire science history that could be used to inform current management practices). Throughout this paper, we use “they” so as not to disclose gender, potentially revealing participant identity.

We designated each respondent’s level of engagement with the SWFSC on a scale of 1–3 (no–low, medium, and high, respectively); throughout the paper, we refer to them as L1, L2, and L3. To determine level of engagement, we looked at responses to questions asking how much they interacted with SWFSC and how their interactions changed over time. Given that these questions allowed individuals to self-report level of engagement and one person’s perception of high interaction might be what another considers medium, we also examined involvement in and interaction with SWFSC events and communications: emails, newsletters, conferences, workshops, field trips, the website, SWFSC publications (fact sheets, summaries), using SWFSC to locate other types of publications, social media, board member conference calls, and personal interactions with SWFSC. Even if scientists might not be as engaged with SWFSC currently—given time constraints or different positions—if they had previously been highly involved, we categorized them as L3.

We determined level of engagement using a combination of quantitative and qualitive analyses. First, we tallied the number of activities in which they participated and noted the frequency for each (e.g., once, occasionally, often). However, we could not rely only on this quantification to compare individuals’ engagement in activities as some activities require more involvement than others. For example, although conferences could be considered high-engagement activities, in some cases individuals only attended a conference to present their research or were unaware of SWFSC’s involvement with the conference. Given this, an individual who attended one conference with little to no other forms of engagement ranked low. In contrast, SWFSC field trips and workshops illustrated higher levels of engagement as participants were in closer interaction with those directly involved in SWFSC. No–low engagement also included those who did not engage with SWFSC, such as respondents whose only connection was being on a mailing list or attending a SWFSC co-hosted conference but who were unaware what SWFSC was. Although the designation process involves subjective evaluation, given that individuals were compared against each other using the same metrics, we believe that it provides an accurate comparison of varying levels of involvement.

We categorized the 28 scientists into 9 L1 (32%), 4 L2 (14%), and 16 L3 (54%) (Figure 2). All of those on the governing board had a high level of engagement in SWFSC. Non-governing board scientists tended most to be L1 (n = 9), followed by L3 (n = 6) and L2 (n = 4). Given that we had representatives at all levels of engagement, especially the upper and lower ends, we could make observations about how level of engagement relates to participation in knowledge coproduction activities. Throughout this paper, we use numbers rather than percentages to describe frequencies; percentages can be misleading in samples less than 25, especially when making comparisons between groups with different sample sizes, and it is more accurate to provide the actual number [47]. Below are representative quotes illustrating engagement levels.

High Engagement (L3):

I’ve been to 3 or 4 of [SWFSC’s] conferences, workshops. I’ve participated in a few of their field trips. I’ve even led a couple of them. I’ve participated in the Fires of Change. We hosted one of the artists and I led one of the field trips on the Slide Fire for that. I’ve served on at least one…committee for the Consortium. I work semi-frequently with the director of it and I’ve participated in webinars. I’ve helped out with a few people for… papers [syntheses/summaries] that they put out.

S11

Medium Engagement (L2):

I participated in a field trip previously, but I would say the biggest way I have participated in and will continue to participate in is the webinars… It’s easy to dedicate time to.

S21

No–Low Engagement (L1):

I don’t interact with them much at all, except to go to conferences.

S23

3.2. How Engagement with SWFSC Relates to Coproduction of Knowledge in Scientists’ Work

Overall, knowledge coproduction activities with managers/practitioners were more commonly conducted by L3 than L1 scientists in their own work, both in terms of the number of activities and diversity of categories (

Table 2. Number of L1, L2, and L3 scientists involved in knowledge coproduction categories and activities.

Knowledge Coproduction Activities		Level of Engagement
		L1 n = 9 (32%)	L2 n = 4 (14%)	L3 n = 15 (54%)
Decision-framing context
	Understand needs, values, and time frame of user	6	2	14
	Consideration of broader social context	0	1	8
Research Collaboration
	Definition of problem	0	1	5
	Research question dev.	0	0	2
	Research design	1	0	2
	Data collection	0	0	3
	Data analysis and meaning making	1	0	1
	Development, management, dissemination of research outputs	1	0	2
Commitment
	Buy-in, co-ownership of project	0	0	6
	Long term trust built	0	1	8
Engagement
	Diverse relationship and partnerships	2	1	11
	Regular interactive meetings with stakeholders	1	0	8
Communication and knowledge
	Multidirectional knowledge exchange and learning	0	2	6
	Recognition of manager knowledge/scientist humbleness	2	3	9
Evaluation
	Evaluation of product, process, and/or actionability of science	1	0	7
	Revise based on evaluation	0	0	4

). For example, 12/16 L3 scientists were involved in at least three activities spread across three categories (thereby demonstrating diversity in approaches and activities), whereas only one out of nine L1 scientists participated in this range (

Table A1. Respondent participation in knowledge coproduction categories and activities. Colors indicate level of engagement (blue = L1, green = L2, salmon = L3).

	S10	S14	S15	S16	S18	S20	S23	S24	S27	S13	S21	S22	S28	S1 *	S2 *	S3 *	S4 *	S5 *	S6 *	S7 *	S8 *	S9 *	S11	S12	S17	S19	S25	S26
Decision Framing Context (2)	1	1	0	1	0	1	1	1	0	0	2	1	0	2	2	2	2	2	1	1	1	1	2	2	1	1	1	1
Understand needs, values, and time frame of user	x	x		x		x	x	x			x	x		x	x	x	x	x	x	x		x	x	x	x	x	x	x
Consideration of broader social context											x			x	x	x	x	x			x		x	x
Research Collaboration on: (6)	1	0	0	0	0	0	0	0	2	0	0	1	0	5	1	0	4	0	2	0	0	0	1	0	1	0	0	0
Definition of problem												x		x	x		x		x						x
Research question development														x			x
Research design	x													x			x
Data collection																	x		x				x
Data analysis and meaning making									x					x
Development, management, and/or dissemination of research outputs									x					x									x
Commitment (2)	0	0	0	0	0	0	0	0	0	0	1	0	0	2	2	0	1	0	2	1	1	0	1	2	1	1	0	0
Buy-in, co-ownership of project														x	x		x		x					x	x
Long-term trust built											x			x	x				x	x	x		x	x		x
Engagement (2)	1	0	0	0	0	0	0	2	0	0	1	0	0	2	1	1	2	0	0	0	1	1	2	2	2	1	2	2
Diverse relationship and partnerships	x							x			x			x	x	x	x					x	x	x	x	x	x	x
Regular interactive meetings with stakeholders								x						x			x				x		x	x	x		x	x
Communication & Knowledge (2)	1	0	0	0	0	0	0	0	1	0	2	2	1	2	2	2	2	0	2	1	1	0	0	1	0	0	0	2
Multidirectional knowledge exchange and learning											x	x		x	x	x	x		x									x
Recognition of manager knowledge/scientist humbleness	x								x		x	x	x	x	x	x	x		x	x	x			x				x
Evaluation (2)	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	2	0	2	1	0	0	2	2	1	0	0	1
Evaluate coproduction products, process, and/or actionability of science									x								x		x	x			x	x	x			x
Revise based on evaluation																	x		x				x	x
Total (16)	4	1	0	2	0	1	1	3	3	0	6	4	1	13	8	5	13	2	9	4	4	2	8	9	7	3	3	6

* Indicates member of the governing board.

). Five L1 and two L2 scientists mentioned involvement in 0–1 coproduction activities. We realize that engagement in only a couple of activities does not necessarily indicate knowledge coproduction, especially ones that do not require interaction with a user (e.g., recognizing manager needs and knowledge); however, given the continuum concept, it would be arbitrary to create a metric as a cutoff for knowledge coproduction. Instead, we provide counts for all activities (Table 2) and specifics for each participant (Table A1) to allow comparisons.

3.3. Impact of Engagement Level on Types of Knowledge Coproduction Activities

Differences existed in which specific knowledge coproduction activities scientists were more likely to engage, and several activities were mentioned by only two or three scientists (e.g., research question development, research design). The most common categories included decision-framing context, engagement, and communication and knowledge; far fewer, mostly L3 scientists, discussed research collaboration (Table 2). Most L1 and L2 scientists who participated in knowledge coproduction activities described the importance of understanding needs, values, and time frame of user (n = 6/9, n = 2/4). In the following sections, we discuss in greater detail the coproduction activities listed in Table 2.

3.3.1. Decision-Framing Context

The most common coproduction activity in which scientists engaged (n = 22) was understanding needs, values, and time frame of the user. Less common was consideration of broader social context. Half of L3 scientists still mentioned this activity (n = 8) in contrast to one L2 and no L1 scientists. This activity involves recognizing the broader dynamics at play, such as understanding that managers cannot always easily implement some actions. For example, although prescribed burning scientifically makes sense for many ecosystems, managers must balance it with burn windows, availability of personnel, return on investment, points of concern (e.g., health impacts from smoke), current policy, and public opinion.

3.3.2. Research Collaboration

Scientists were least involved with collaboration on research, and within this category, the most common activity was working with managers/practitioners on brainstorming and definition of the problem, but even that activity was only mentioned by five L3 scientists and one L2 scientist. S17 explained that, for one project, “[I] had conference calls with five or six people [mostly managers and stakeholders] at a time where we were all trying to come up with ideas of how to make a project”. More often, it was a less formal process, involving casual conversation. S6 described how a research idea developed during a discussion with a fire management officer after a large fire in the Sky Islands: “What came up for him was ‘Well, what’s this system? What’s this landscape going to look like now? What’s going to happen to it?’ and that was very stimulating”.

Less frequently mentioned activities, mostly carried out by L3 scientists, included research question development, research design, data collection and analysis, and development and delivery of research outputs. Two L3 scientists described working with managers to formulate research questions and two L3 scientists and one L1 scientist worked with managers/practitioners on research design. Data collection with managers was conducted by three L3 scientists who described implementation by managers/practitioners that involved data collection or allowed for data to be collected. S12 explained that their organization looked for “innovative technologies and ways to develop data collection efforts that we can provide to the stakeholder groups to do the job of monitoring faster”. Even though this quote focused on the role a researcher might play in developing equipment, their organization works with stakeholders to conduct implementation and monitoring. These examples demonstrate that, in knowledge coproduction, data collection might look different to methods traditionally used by scientists. One L3 and one L1 scientist mentioned data analysis and meaning making. Illustrating managers’ role in meaning making, S1 said that some of their partners “have great modeling capabilities and other pieces that advise or help my students do the modeling”. One L1 and two L3 scientists mentioned working with managers on the development, management, and/or dissemination of results. When asked if they collaborated on science delivery with managers, other scientists either said no or provided an example not representative of collaboration (e.g., presenting findings at a meeting). This is discussed in greater detail in the communication subsection.

3.3.3. Commitment

While no L1 or L2 scientists mentioned co-ownership/buy-in, six L3 scientists explained how they have been involved in co-owned projects or how managers/practitioners have had buy-in. In many cases, this involved bringing stakeholders in from the beginning. S1 said that “Pretty much every grant that I write, I work with managers…and also non-profits… if the managers don’t want to be a [co-principal investigator], I usually just put them on as collaborators and get their input on our ideas to see what makes sense, what’s needed by them, and what’s useful”. Many times, buy-in and co-ownership illustrated a long-term commitment. S12 explained the multi-year planning process with one Collaborative Forest Landscape Restoration Program (CFLRP) project and S2 described seeing policy and management changes ten years after starting the project. Despite the length of time, these individuals believed that buy-in contributed to project success.

Long-term trust was mentioned by one L2 scientist and eight L3 scientists. Several interviewees, such as S19, mentioned that it takes a long time to build that trust and come to an agreement, but it is worth it for project success: “They really have to trust you… and that doesn’t come overnight. So, it’s been a long process doing the restoration program that the clients I work with have built over the past 15 years”. Interviewees also explained that trust helped improve projects. In some cases, it led to more collaborative processes and projects, as well as new projects. S7 explained, “They contact us, we provide them information, and typically it launches a new relationship. So, you keep up with those folks and end up working with them more down the road once they sort of have a familiarity with you and comfort and trust and that goes both ways”. Informants also explained that trust allows them to better address problems that can arise, such as working with individuals who have pre-conceived ideas about researchers and distrust of the organization, project, and/or process. Given these challenges to building long-term trust, many informants described various mechanisms that foster trust building: maintaining communication after the project, thanking partners, sharing manager/practitioner success stories and ideas at events, being flexible, engaging in informal interactions that build rapport, and working together in the field.

3.3.4. Engagement

Engagement activities were also more common among L3 scientists. Diverse relationships and partnerships was mentioned by 11 L3, one L2, and one L1 informants. These interviewees worked with an array of individuals and groups, including tribes, NGOs, the public, private landowners, and government agencies. S1 explained that it was important to take “time to engage in the spectrum of partners”. These informants listed benefits that involved various ways of improving the research, such as providing a means for sharing and learning new knowledge, which would make it more likely that research is used and in turn, result in better solutions. Informants also mentioned benefits that extended beyond research, such as facilitating social acceptance and consensus on projects and approaches. S4 explained, “We rely on our fire management relationships to even maintain our experimental designs. If we’re crazy scientists trying to burn a small chunk of land every five years…you need fire management officers to be on your side”. Three L1 scientists who did not work with diverse partners expressed a desire to do so.

Less commonly mentioned was regular interactive meetings with these stakeholders (eight L3, one L1). In three of these instances, individuals were part of a CFLRP project, which required regular stakeholder meetings. Two individuals focused on other shorter-term projects that required regular meetings; for example, S11 described collaboratively writing a General Technical Report with a wide range of US Forest Service employees. Informal lunch meetings with managers, where they would discuss work and projects among other things, was mentioned by two people. S12 summed up, “Without having that really frequent interaction with those partners on either side, I don’t think we would ever come to some of the solutions that we came to”. This informant also explained that frequent interactions with their partners led to increased receptiveness to new information and changes.

3.3.5. Communication and Knowledge

Communication and knowledge was one category in which scientists more frequently engaged, but they primarily mentioned that they recognized that managers hold valuable knowledge (n = 2 L1, 3 L2, 9 L3). Most people who discussed knowledge emphasized that managers have on-the-ground expertise that can assist scientists’ work. For instance, S2 stated, “Where the science is sort of an emerging topic [and] there’s not enough research out there…so [it helps] having peer-to-peer sessions where managers can say ‘Look, I haven’t done an academic study on this but this is what I’ve seen on my landscape when we tried x, y, or z’”. In interacting with managers, some scientists shared that they had gained a different perspective and/or learned more about a topic. Also mentioned by some informants was a recognition that they (i.e., scientists) could be wrong.

Many, regardless of engagement level, discussed the importance of being able to effectively communicate results to their audience, such as using language that is understandable and connecting the message in a meaningful way to the audience. However, a key component of coproduced knowledge is that information is not just delivered to managers and other decision makers via one-way transmission but is instead multidirectional. Six L3, two L2, and no L1 scientists said that they engaged in multidirectional knowledge exchange and learning. S3 described the process of this two-way learning: “Those are useful lessons for scientists—to learn about how the operational world works and to communicate well with managers. And conversely, it’s important for managers to learn from the scientists that there are reasons why you have to do experiments and why you have to have random controls”.

However, when asked if they ever engaged in multidirectional knowledge sharing in more formalized science delivery, many individuals said no. For example, S23 stated, “We usually present [the report] at a conference and then try to publish it somewhere. I don’t take it to the land managers myself at all. I don’t really have any interaction with them”. Yet, in other cases, some individuals who did not engage in two-way communication expressed a desire to do so given its importance.

3.3.6. Evaluation

If respondents said that they collaborated on research or science delivery, we asked them if there was any evaluation of the product or process. Seven L3 scientists and one L1 scientist discussed experiences of evaluating the product, process, or actionability of the science. In some cases, evaluation was formal, such as an external review, after action review, or monitoring protocol built into a CFLRP project, but in other cases the evaluation was informal, such as satisfied collaborators who wanted to continue working together. S6 was one of the four L3 scientists who mentioned that they revise projects based on evaluation: “Whether it’s in the planning process or in the actual implementation…we’re always constantly being adaptive and that involves, by definition, evaluation”. However, in most instances, informants described a one-time evaluation process, as opposed to an iterative process that led to revisions and reevaluation.

3.4. SWFSC’s Role in Fostering Knowledge Coproduction

Scientists more engaged with SWFSC were involved in a greater number and variety of knowledge coproduction activities (Table 2 and Table A1). Therefore, we examined whether SWFSC had any influence on their participation in knowledge coproduction activities. Although we did not ask this directly, we asked if and how SWFSC influenced their relationship and communication with managers and how SWFSC programming influenced their scientific research. In addition, when describing involvement in knowledge coproduction activities, some scientists also described the role that SWFSC had played. Of both governing board and general members who participated in at least one knowledge coproduction activity, 2/7 L1, 2/3 L2, and 11/15 L3 informants said that the SWFSC increased their participation in these activities, especially understanding needs of managers. However, expanding beyond recognizing manager needs, only 1/3 L1 and 2/2 L2 said that the SWFSC has played a role. Three L3 respondents involved in knowledge coproduction activities said that the SWFSC did not play a role because their engagement in such activities was more due to their jobs that required such interactions (e.g., extension agent, employee in another boundary organization); they noted that their work dovetailed and complemented that of the SWFSC. S8, a former wildland firefighter, explained that the position afforded her contacts and an understanding of managers prior to working with the SWFSC, but that the SWFSC provided additional opportunities to interact with managers.

Scientists detailed several ways that the SWFSC fostered increased knowledge coproduction activities that could then extend beyond the boundary organization. These included bringing people together to gain different perspectives, which increased their recognition of manager knowledge, understanding the needs of managers, and in some cases, trust building. S8 described a bimonthly luncheon that the director of the SWFSC organized with fire managers and scientists: “It’s not formal. It’s just, ‘Hey, what have you been up to these past couple months?’ and see how life is and then talk about work and build trust… and relationships… and rapport”. Workshops, conferences, and field trips also offered opportunities for two-way interactions between scientists and managers; these were viewed as a space for managers and scientists to feel comfortable and that their knowledge was valued, which some said carried over to their interactions with managers in other contexts. Scientists also explained that the SWFSC provided an opportunity for them to understand the relevance of research to managers and to know what research questions to ask. Finally, these SWFSC activities allowed a venue for developing research collaborations between managers and scientists that led to knowledge coproduction in their work outside the boundary organization context.

4. Discussion

4.1. Relationship between Scientists’ Engagement in a Boundary Organization and Their Involvement in Knowledge Coproduction

Before discussing the potential role SWFSC played in fostering coproduction activities in scientists’ work, we first discuss the overall relationship between the level of scientists’ engagement in the SWFSC and their involvement in knowledge coproduction activities. L3 scientists tended to be more likely to participate in knowledge coproduction activities and a wider diversity of activities, both in terms of number of individual activities and broader categories. As we discuss in greater detail in Section 4.3, this relationship could have been because those already interested in knowledge coproduction were drawn to and motivated to be involved with a boundary organization aimed at increasing knowledge coproduction. In contrast, those who had low engagement with the SWFSC were also less likely to participate in knowledge coproduction activities, and when they did, they participated in fewer activities. Given that there were only four L2 scientists, it is harder to draw conclusions about this mid-level engagement group; if one or two informants mentioned participating in an activity, we cannot determine whether it was just that individual or a general trend for the group. It is interesting to note, however, that most scientists who participated in our interviews were either high or low engagement, with fewer in the middle. Future research can survey a larger population to assess whether these trends continue and, if so, explore reasons why scientists tend to remain at more extreme levels.

4.2. Do Differing Levels of Engagement in a Boundary Organization Result in Similar or Different Knowledge Coproduction Activities?

Regarding our second research question, involvement in some activities differed based on scientists’ level of engagement with the SWFSC. Since L3 scientists were involved in a greater variety of activities, it is unsurprising that we would find activities only conducted by L3 scientists (e.g., research question development, revision based on evaluation). Some activities (e.g., understanding needs, values, and time frame of user; recognizing manager knowledge) were discussed by scientists at all engagement levels, but still a greater number of L3 scientists mentioned them, even when adjusting for differences in sample size. In addition, one of the most frequently mentioned activities, understanding needs, values, and time frame of user, is a first and crucial step in knowledge coproduction, but it is only a first step. While this activity involves asking managers what they need or understanding the constraints on a manager’s time, scientists can still be the ones to formulate questions and design research; by itself, this activity does not necessarily require knowledge coproduction.

Despite some differences, we also found similarities between scientists at different engagement levels, especially in terms of lack of participation in certain activities. For example, only nine scientists mentioned activities within the category, research collaboration (e.g., definition of problem, research question development, research design, data collection, data analysis, research outputs), three of whom solely referred to defining the problem. Research collaboration activities tend to require larger investments in terms of time, trust, and institutional change related to how research is usually carried out, which scientists have noted are challenges [48]. Jagannathan et al.’s [17] review of 21 projects revealed that most coproduction took place through workshops or consultations rather than extended collaboration on projects. In addition, trust is needed not only between partners to accomplish the work. Scientists also need to trust managers and practitioners to conduct work that is traditionally part of the scientists’ realm and that the science will remain objective and credible [5,8,29].

Some research has called for knowledge coproduction to go beyond the project level and instead work to bring about societal transformative change and changes in governance. Norström et al. [14] explained that “co-production processes produce more than just knowledge; they develop capacity, build networks, foster social capital, and implement actions that contribute to sustainability”. These are goals towards which to strive, but, as seen above, there are challenges to such transformative approaches, both at an institutional and individual level. Partly for these reasons and because of existing institutional challenges and risks (e.g., academia’s focus on publication, professional risks, risks to fields of scholarship [48]), some scholars have questioned whether knowledge coproduction is a desirable or feasible path [20,21]. However, the scientists we interviewed who engaged in knowledge coproduction believed there was value to these activities; additionally, several not involved in knowledge coproduction expressed a desire to do so more as they saw potential benefits. Although involvement in only some coproduction activities or more encompassing knowledge coproduction around a specific project might not bring about dramatic shifts in society, they can still generate positive results [8]. Enquist et al. [8] explained that “smaller translational efforts can be viewed as measures (or steps) in the short run, thereby making progress toward successful outcomes in the longer term”. For example, developing relationships now can foster future collaborations that eventually result in action [8].

In addition, those who study knowledge coproduction may be more nuanced in their conceptualization than many scientists and managers who include it in their work. Individuals might just be starting to use some of these approaches, and they may be relatively unfamiliar with the overall concept or theory behind knowledge coproduction. Mach et al. [18] noted that “co-production has not been a single approach, but instead a diversity of forms of engaged research” and “the jargon may both obfuscate governance dimensions and limit understanding of what works” (p. 30). They call for a recognition of the diversity of ways that stakeholders can interact, which can foster effective engagement, and, in turn, advance actionable science.

At the other end of the spectrum, how-to guides produced to teach people how to engage in knowledge coproduction risk being too prescriptive [22]; by focusing entirely on process, people may believe that all steps must be conducted to engage in knowledge coproduction and larger structural issues that can lead to failure, such as power and politics, may be overlooked [19,23,27]. In addition, as has already been discussed, completing every step can be challenging or even impossible due to individual or institutional barriers. An overemphasis on specific activities or the order of such activities could lead some individuals to give up, or not even attempt, knowledge coproduction activities. By instead emphasizing knowledge coproduction as a suite or continuum of activities in which to participate [18], scientists who are interested but might be entering at different points or only able to achieve certain aspects due to barriers, could be more likely to try some activities. Again, this might not result in transformation or what people might envision in knowledge coproduction, but it could initiate a journey to more visionary outcomes.

4.3. Role of Boundary Organizations in Fostering Knowledge Coproduction

In answer to our final research question, most scientists involved in knowledge coproduction activities believed that the boundary organization increased their use of such approaches. Specifically, the boundary organization provided opportunities for groups to come together to share diverse perspectives and knowledge, for scientists to learn more about manager needs, and to develop potential research ideas. This was especially the case with L3 scientists and those who were engaged in more than one knowledge coproduction activity. Unsurprisingly, those who were less involved in the SWFSC (L1) said it had less of an impact on their involvement in such activities; however, as can also be seen, L1 scientists were less involved in such knowledge coproduction activities overall.

Our findings support those of other researchers who found that boundary organizations can bring groups together [2,4,5,6]. In our study, scientists recognized and valued these opportunities; those who actively participated in several SWFSC events described that their involvement in knowledge coproduction activities in their own work also increased. However, we did not measure the extent to which these scientists had previously been involved or interested in knowledge coproduction activities, or if this only resulted from engaging with SWFSC. It could be that those interested in knowledge coproduction were drawn to a boundary organization that lists as one of its goals increasing knowledge coproduction.

We are not claiming that coproduced knowledge has to be the way forward and there are many critiques of this approach and discussions of barriers [1,18,19,20,21]. However, several scientists interviewed, both who are currently involved in knowledge coproduction activities and those who are not, expressed a desire to increase their participation in knowledge coproduction. Several scientists explained that they would like more guidance on how to engage in these activities. Boundary organizations, such as SWFSC, could explore ways to not only provide more passive opportunities for this engagement (e.g., webinars, field trips) but also more active opportunities. JFSP has explored this in the past but could continue expanding opportunities to hold knowledge coproduction workshops that explain what it is, detail types of activities in which one could engage and how, and provide opportunities to brainstorm ways to incorporate it into researchers’ own work. As knowledge coproduction is not the standard for scientific institutions and barriers exist [48], focused workshops on knowledge coproduction could help those scientists who are struggling to begin or who want to increase their current involvement in knowledge coproduction. In particular, workshops could explore ways to expand scientists’ involvement in less common activities, such as those involving research collaboration and evaluation, and address potential reasons for hesitation, such as trust in managers to conduct research, time investment, and institutional barriers.

4.4. Future Research and Directions

Given that our study did not directly measure involvement in knowledge coproduction activities before and after becoming part of a boundary organization, future research could explicitly explore this. In another study, the lead author surveyed several FSEs asking individuals to reflect on involvement in knowledge coproduction activities before and after engaging with the boundary organization. Research could also survey new members about their involvement in knowledge coproduction activities and then reevaluate after a certain amount of time; a large event where several new people are added to a contact list, such as a conference, could be a good source of comparative data. It would be expected that some of these people would become more actively engaged with the organization, while for others involvement would remain limited. Another approach could be to explore the competencies, skills, and attitudes of scientists involved in knowledge coproduction, ideally capturing a snapshot of these traits before and after involvement with a boundary organization. Doing so could help increase understanding as to whether scientists with certain traits are drawn to boundary organizations that conduct this work, whether they changed their views and gained skills since being involved, or a combination.

Additionally, our study focused on scientists, not on managers or practitioners. Although research has examined how managers use information and the challenges they encounter [49,50,51], more work could explore manager and practitioner involvement in knowledge coproduction activities and their engagement with various boundary organizations. Dubois et al. [51] claimed that boundary spanners could help strengthen the usefulness and accessibility of implementation data for research. Currently, many practitioners are not yet fully involved in the implementation process and lack means to push their data into the research space, thereby leaving data largely fragmented, inaccessible, and not informing practice [51]. Research could also expand on Dubois et al.’s [51] findings to see whether boundary organizations not only help managers and practitioners by facilitating knowledge exchange between groups but also whether engagement in a boundary organization fosters increases in knowledge coproduction activities among managers and practitioners. In addition, extending research to include managers and practitioners allows for comparisons among different actor groups (i.e., scientists, managers, practitioners). If certain types of actors are more or less likely to engage in knowledge coproduction activities, identifying reasons could allow boundary organizations to help address potential barriers, which may differ among groups.

Finally, we only examined one boundary organization serving a particular audience and focus. Additional work should attempt to ascertain whether these findings apply to other types of boundary organizations that address different issues (e.g., Regional Integrated Sciences and Assessments Programs (RISAs)) or cross-cultural/country boundary organizations. The latter is especially important, as recent scholarship on knowledge coproduction has highlighted that Indigenous and other minority group voices might be left out of knowledge coproduction activities [1,19]. Turnhout et al. [23] also discussed challenges and pitfalls of knowledge coproduction activities between actors from the Global North and South, where power dimensions might be retained. Future research examining boundary organizations working at these intersections should explore if and how boundary organizations can facilitate more equitable and just involvement in knowledge coproduction activities, if the relevant groups desire. In addition, it is likely that such research can expand potential knowledge coproduction activities from those on which researchers have focused and which we summarized in Table 1 to also include approaches from a greater diversity of individuals impacted by and working towards solutions to socioecological issues.

5. Conclusions

Overall, scientists who engaged more with SWFSC reported greater participation in a variety of knowledge coproduction activities. However, some knowledge coproduction activities, especially those requiring greater time investments or encountering institutional barriers, were less commonly engaged in among all participants. These included many activities within research collaboration, such as research question development; research design; data collection; data analysis and meaning making; and development, management, and/or dissemination of research outputs; as well as revise based on evaluation. More frequent were activities involving the recognition of managers’ needs and knowledge or engaging with partners through diverse relationships and meetings.

Most scientists involved in knowledge coproduction believed that SWFSC increased their participation in these activities outside the boundary organization context. Several other scientists indicated that they would like to include knowledge coproduction in their future work. Boundary organizations can play a critical role in fostering and assisting interested scientists with knowledge coproduction. As found in this study, being part of the SWFSC provided opportunities to interact with and understand the needs of managers/practitioners, as well as build research collaborations. If boundary organizations increase communication between scientists and managers/practitioners initially, they can facilitate different types of future collaborations, such as those where both groups work together from the beginning to co-develop projects. Additionally, since scientists worked less with managers on actual research, boundary organizations could help identify reasons and work to mitigate barriers for those interested in engaging in these activities. For example, showcasing examples where managers and scientists coproduced rigorous scientific findings could alleviate potential concerns over credibility and demonstrate the strength of coproduced knowledge.