Institutional Approaches for Studying System-Oriented Networks

Abstract

Institutional, policy, and management scholars and practitioners are increasingly interested in leveraging network perspectives, methods, and data to understand complex social phenomena, including the various stages of the policy process, community mobilization, and coupled natural and human systems. Viewing these phenomena through the lens of system-oriented networks can be valuable for understanding and intervening within complex policy arenas. However, currently, there is no clear consensus on who and what constitutes a relevant actor in a system-oriented network. Furthermore, numerous conceptual and methodological traditions for conceptualizing, measuring, and analyzing system-oriented networks have arisen, and each is linked to different disciplinary traditions. In this paper, we showcase six approaches from the public policy and public management literature for conceptualizing and analyzing system-oriented networks. We offer a conceptual framework for characterizing different approaches which considers differences in their focal system of interest, analytical focus, theoretical orientation, and approach for determining network boundaries. We review these elements with an eye toward helping scholars and practitioners interested in system-oriented networks to make informed decisions about the array of available approaches.

1. Introduction

Institutional scholars have a long and rich history of employing a network perspective to theorize about, and gain insight into, complex problems, policy issues, and processes. Networks in this pursuit are conceived of by an analyst based on the network’s relationship to some perceived institutional system of interest. If you have ever watched a news report discussing the challenge of growing homelessness in a local community; thought about all the different programs, policies, services, organizations, and agencies that serve the homeless population; and questioned why this system seems to be failing, you have taken the first steps to conceptualizing a system-oriented network (SON). Once you begin thinking about system-oriented networks, you may find that you see them everywhere. They hold the explanation for how the avocado in your shopping basket got from a farm in Southern California to a grocery store in New Jersey. They can shed light on what led to the Paris Climate Accords or help us understand what enabled Silicon Valley to become an international powerhouse for technological innovation. Anywhere a given outcome can be conceived of as resulting from the combined interactions of different elements that are controlled or influenced by three or more interdependent actors, a system-oriented network can be found. Thus, we argue that studying system-oriented networks in a systematic way can be useful to both scholars and practitioners seeking to address collective action problems and improve public outcomes.

However, despite their intuitive appeal and ubiquitous nature, the theoretical and analytical challenges associated with the study of system-oriented networks are complex. These challenges are further exacerbated by the nature of the literature associated with this class of networks, which is both diverse and fragmented, offering a bewildering array of different terminology, theoretical orientations, and approaches that may be daunting to the uninitiated. When conducting institutional research on system-oriented networks, various analytical approaches can be utilized to analyze and understand these networks. In order to understand which analytical approaches are most aligned with certain theoretical orientations, network contexts, and research interests, it is important to discuss and evaluate these approaches.

The purpose of this article is to provide an introduction to some of the different analytical approaches available for studying system-oriented networks from an institutional perspective. Thus, our focus is oriented toward public policy and public management contexts. As illustrated below, these approaches have been informed by diverse disciplinary perspectives, including policy process, institutional theory, and the broader systems literature (e.g., systems of systems and systems engineering) that is rich and full of insights that lie outside of the scope of this article. Yet we encourage future scholars to integrate this literature as much can be gained from the cross-pollination of the different disciplines.1 Additionally, this showcase of diverse approaches to studying system-oriented networks aims to illustrate the breadth of theories, network contexts, and research interests represented across different approaches. By doing so, researchers can determine which analytical approach best suits their research aims and context. In this article, we first discuss key features of system-oriented networks and discuss cross-cutting challenges associated with the study of this class of networks. We then compare six different analytical approaches to studying system-oriented networks and discuss the types of network contexts and research interests for which they are well-suited.

Key Features of System-Oriented Networks. Scholarship reflecting a system-oriented network orientation dates back to the early 1900s as scholars began to advance theories of the policy process, which viewed policy outcomes as the product of interactions among an array of formal and informal actors [2]. In the 1960s, a system-oriented network perspective also rose to prominence in organizational, administrative, and sociological theory as scholars began to leverage concepts from the advent of system-thinking to conceptualize problems and to consider organizations as embedded within—and enabled and constrained by—complex networks [3,4]. These ideas inspired new theoretical development across a number of disciplines. However, disciplinary fragmentation hindered cross-pollination among scholars from different traditions, leading to a rich but relatively disconnected set of literatures [5].

In 2022, Nowell and Milward introduced the term “system-oriented networks”2 to refer to a taxonomic class of networks that spanned across various disciplinary and theoretical traditions but shared in common several key features [5]. First, system-oriented networks are conceptualized by an analyst. Second, the conceptualization of a system-oriented network is linked to some perceived system of interest. A system is understood to be a set of interacting or interdependent processes that collectively co-produce a given (or unintended) outcome [6]. In other words, the system of interest is conceptualized in relation to some higher-order outcome or outcomes of interest. For example, a policy analyst may seek to understand the system of factors and associated actors that explain a set of policy actions related to a watershed. An organizational scientist may seek to understand the system of factors and associated actors that explain incident-level outcomes in a disaster response. A sociologist may seek to understand the system of factors and associated actors that explain community-level outcomes related to the recidivism of juvenile offenders. Therefore, while the foundations of many system-oriented networks draw heavily on social network analysis, studies of system-oriented networks transcend graph theory.

Third, scholarship on system-oriented networks seeks to explain system and/or network-level phenomena. While it is common that studies of system-oriented networks consider outcomes at multiple levels of analysis, some level of interest at the system or network level is inherent to this class of scholarship. Fourth, the network associated with this system of interest is conceptualized as the actors that directly or indirectly have an influence on one or more system elements or processes. These actors are interdependent because the elements of the system they are affiliated with are interdependent. Finally, studies of system-oriented networks are dominated by a relational perspective that seeks to (1) explain system-level outcomes as a function of the pattern and nature of institutional relationships among relevant actors within the system, (2) understand why certain patterns emerge in the system, and (3) to understand the processes underlying tie formation.

2. Cross-Cutting Methodological Challenges in Studying System-Oriented Networks

Those interested in the study of system-oriented networks have several options to consider in terms of methodological approaches that stem from multiple disciplines, including public management, public policy, and environmental resource management. In this paper, we systematically review six such approaches with an eye toward understanding cross-cutting challenges and offering a framework for characterizing their diversity. Our comparison reveals that different approaches seek to illuminate different phenomena associated with system-oriented networks and are intellectually rooted in different theoretical traditions. However, all approaches share in common the goal of providing a theoretically and empirically robust representation and analysis of the system of interest and its corresponding network. The approaches also share in common a general set of methodological tasks that the analyst must perform in order to study a system-oriented network. Each of these tasks are accompanied by a common set of cross-cutting challenges.

Task 1: Conceptualizing a System-Oriented Network. System-oriented networks, by definition, are conceived of by an analyst. This is not to suggest that these networks are not sociologically real in the sense of being empirically observable and verifiable. Rather, this simply means that the network must first be defined by the analyst before it can be described. This requires the analyst to articulate some system of interest anchored to some outcome of interest. The second step requires the analyst to identify relevant network actors associated with relevant system elements. For example, if the outcome of interest is air quality, the system of interest may be conceptualized as the pollutants that contribute to poor air quality, and the network may be conceived of as actors who have direct influence over those pollutants, such as farmers, ranchers, manufacturing entities, and land agencies. Alternatively, the system of interest could be the processes that mitigate air pollution, and the network could be viewed as the actors that seek to influence these processes, such as air quality boards, federal regulators, and advocacy groups. A third alternative might be to conceptualize the system of interest as encompassing both processes of pollution and mitigation simultaneously, resulting in a larger system-oriented network. Decisions about how to conceptualize a system and its associated network are always subjective based on the analyst’s theoretical lens, research interests, and understanding of the targeted system.

The network conceptualization stage has several inherent challenges. The first is the challenge of finding parsimony between holistic and comprehensive representations of a given system and its associated network with the need to draw realistic boundaries that are feasible for study. Drawing a boundary that is too large, such as a country, for example, would pose major challenges for the analyst. Systems are frequently complex and may be poorly understood while empirical scrutiny generally necessitates simplification [7,8]. This requires the analyst to make judgment calls about what to include and exclude from consideration in attempting to model a system-oriented network [9,10,11]. Decisions about who and what is included and excluded from consideration can dramatically influence what an analyst “finds” [12].

A second related challenge concerns the subjective nature of the network itself. Frequently, scholars utilize key informants to identify relevant system elements and actors within the network for a given context. However, the concept of relevance is itself subjective and often subject to pluralism if different interest groups have competing mental models of who and what constitutes a relevant stakeholder. In other words, who or what is deemed relevant for a given outcome will likely look different depending on whom you ask.

Task 2: Operationalize Decision Rules to Define Network Boundaries. In the analysis of system-oriented networks, a key task involves translating conceptual boundaries into decision rules that clearly define what constitutes “the network” and what does not. These decision rules are crucial for the identification of relevant actors within the network. All analytical approaches for studying system-oriented networks seek to make empirically robust statements about the system of interest and its corresponding network. However, without clear decision rules to define the network, the relationship of the theorized system-oriented network to the resulting data can be dubious.

A related challenge is the need to obtain sufficient representation of a given network to justify a network or system-level conclusion. Since most descriptive network analysis techniques do not rely on sampling theory, comprehensive information about network components, akin to near population-level data, is often required. Like the parable of the blind men and the elephant, network-level analysis and descriptions based on incomplete data can produce a distorted representation [13]. This challenge is particularly pronounced in complex networks where obtaining comprehensive data can be quite difficult. Finally, analysts must be wary to ensure the methods used to identify relevant actors do not bias the data. For example, snowball sampling can systematically exclude node isolates or disconnected sub-groups [12].

Task 3: Analysis of System-Oriented Networks. The final task for the system-oriented network analyst is to identify an analysis approach that adequately addresses the research questions of interest. One of the main challenges in the analysis of system-oriented networks is the nature of these networks. The interdependent nature of any class of networks has long been identified as problematic for violating traditional statistical assumptions related to the independence of observations [9] (pp. 80–81). Further, system-oriented networks are often conceptualized as dynamic, non-linear, multi-level, multi-mode, cross-scale, multiplex, and composed of diverse types of actors [7,8]. As such, these approaches are amenable to a variety of analytic techniques. For example, scholars applying these approaches have relied on descriptive social network analysis (SNA), exponential random graph models (ERGMs), temporal exponential random graph models (TERGMs), case studies, and others [8,9,11,12,14,15,16]. These distinctive characteristics present further analytical difficulties, making system-oriented networks perhaps the most analytically challenging of all the network classes.

3. Comparative Analysis of Six Institutional Approaches for Studying System-Oriented Networks

System-oriented networks, as defined by Nowell and Milward [5], represent a taxonomic class of institutional networks. As such, an institutional perspective is inherent to their nature and study. While methods for studying system-oriented networks share in common several cross-cutting requirements and associated challenges, the literature offers a number of different analytical approaches that researchers can implement to gain insight into a system-oriented network of interest. In the remainder of this paper, we offer a systematic review and comparison of six different approaches for conceptualizing and studying system-oriented networks available to scholars. These approaches were selected as information-rich examples that appear in the institutional, public policy, and public management literature and illustrate the diversity that exists across approaches of this network class. For each approach, we systematically reviewed both the seminal as well as subsequent empirical literature associated with the respective approach with a particular eye toward understanding the theoretical foundations that define the approach as well as how the approach has been operationalized by scholars to define system-oriented networks. Using an inductive comparative case analysis methodology [17,18], we then developed a framework for characterizing each approach across four dimensions: (1) focal system of interest, (2) dominant analytical emphasis, (3) underlying theoretical orientation, and (4) and relevant network boundaries (for summary, see

Table 1. Analytical approaches for understanding system-oriented networks.

Approach	Focal System of Interest	Dominant Analytical Emphasis	Underlying Theoretical Orientation	Focus of Network Boundaries
Governance Networks	Policy stream or specific policy function	How do networks influence the policy process?	Policy process—particularly multiple steams theory, institutional analysis and development (IAD), and complexity theory	What are the patterns of resource exchanges and coordination involved in [policy function]? Who are the actors involved in these exchanges?
Ecology of Games	Collective action problem	How are polycentric systems structured around a given collective action problem and what are the consequences of this structure?	Policy process—particularly IAD and polycentric systems	Who is important to [collective action problem?]. What are the forums where collective problem solving occurs?
Networks of Networks	Network domain	How are purpose-oriented networks (PONs) and their members affected by population dynamics over time and what are the consequences for system level outcomes?	Organizational theory—particularly population ecology, resource dependency, and institutional theory	What is the population of purpose-oriented networks active in a given policy and geographic area? Who are the members affiliated with these purpose-oriented networks?
PARTNER	Cross-sector community collaboration around targeted issue	How can communities realize collaborative advantage?	Social capital theory	What are the ego-networks associated with key actors in a community?
Social–Ecological Networks	Social–ecological system associated with an outcome of concern	What is the nature and consequences of interdependence and interaction between/among social and ecological system elements?	Social ecology systems theory	What are the key ecological elements associated with a given outcome of concern and what are the social elements that are interdependent with those ecological elements?
STEW-MAP	Environmental stewardship associated with a focal ecological topic	How does environmental stewardship activity influence ecological outcomes?	Social–ecological systems theory	Who is active in environmental stewardship within a given community and who do they partner with?

). Finally, we developed profiles of each approach, guided by this framework. In order to enhance the rigor of our analysis and resulting profiles, member-checks [19] were conducted. Profiles were shared with the seminal authors of each approach for feedback and refinement. We found that cross-comparison of different approaches presents an opportunity for integration with each other or with approaches from other fields.

Governance Networks. One conceptualization of system-oriented networks is known as governance networks, sometimes referred to as complex governance networks (CGNs) [8,14,20]. Morçöl [8,20], Koliba et al. [14], Klijn and Koppenjan, [21,22], as well as others, have recognized that the social phenomenon of “governance” generally requires the engagement of diverse actors which can be meaningfully understood through a complexity theory and network lens [8,14]. As such, governance networks have been defined as networks with “relatively stable patterns of coordinated action and resource exchanges; involving policy actors crossing different social scales, drawn from the public, private, or nonprofit sectors and across geographic levels; who interact through a variety of competitive, command and control, cooperative, and negotiated arrangements; for purposes anchored in one or more facets of the policy stream…and can be found within or across different policy domains” [14]. Moreover, Morçöl [8] notes that because governance network processes and outcomes are nonlinear in nature, to understand governance networks, one must seek to understand the components of complexity theory such as emergence, nonlinearity, feedback mechanisms, micro-to-macro and macro-to-micro processes, and the nuances of accountability relationships. Klijn and Koppenjan [21] argue that there are multiple types of complexity surrounding the study of governance networks that must be understood and coped with. First is substantive complexity, which is caused by a lack of information and disagreements about the information that is available [21]. Second, strategic complexity is the complexity that arises from the strategic choices made by autonomous actors addressing complex problems [21]. Lastly, institutional complexity arises due to the rules that exist within different networks, which in some instances can reduce complexity, but in others, it can contribute to complexity [21].

The study of governance networks or CGNs is oriented around questions such as how governance networks form and change over time. How do governance networks influence the policy process? What are the success factors for network performance? How do we manage and sustain governance networks that span space and time [15] (p. 425)? How can actors manage complexity within networks to address wicked problems [21,22]? How do individual-level phenomena influence macro-level norms, culture, policies, etc.? And, how do macro-level phenomena shape micro-level behavior and beliefs [8]? To answer these questions, Morçöl and Koliba et al. argue that scholars must rely heavily on theories of the policy process, complexity theory, as well as concepts from network studies, new public management, and new public governance [8,14].

There are many instances of scholars leveraging perspectives from complexity and systems theory in the study of governance networks. For example, Unlu et al. [23] used complexity theory to understand how rule structures influence interactions between actors in a network surrounding the implementation of drug consumption rooms (DCRs) in Finland. To identify actors in the DCR policy system, Unlu et al. [23] first analyzed the research context itself which led them to recognize that DCR policy was largely a local-level phenomenon (with a few exceptions); as such, the authors launched a review of artifacts from various forums: websites, city council meetings, law enforcement reports, newspaper articles, etc. Using these topic-related sources, the authors were able to identify those involved in the DCR policy arena. In another application, Yi [24] found that, at least for self-organizing green energy networks operating in the U.S., social capital and policy entrepreneurs positively relate to overall network performance in terms of green job growth and renewable energy production capacity. To identify network membership, Yi [24] first defined what it meant to be an actor within the green energy policy domain based on a hyperlink network methodology (essentially, an organization’s webpage serves as a node and on that webpage, links to another organization’s website serves as a tie), then searched various websites (largely governmental sources) to identify network members based on an organizational website’s referencing of another organization’s web address.3

While neither Morçöl [8,20] nor Koliba et al. [14] offer a true theoretical framework of complex governance networks, they leverage existing theories to provide a conceptual understanding of how to think about governance networks and offer insight into identifying some of their components [9] (p. 188). For instance, Koliba and colleagues [14] argue that governance network arrangements themselves should be viewed as complex adaptive systems (CAS) because governance entails ongoing interdependent relationships consisting of game-like interactions between actors; therefore, to understand complex adaptive governance networks, analysts need to integrate concepts from systems dynamics and network architecture. Additionally, Koliba et al. [14] and Morçöl [8] highlight that governance networks can operate at different levels of geographic and social scales and that this has implications for researchers attempting to identify network boundaries and membership [8,14]. Additionally, the governance networks literature guides the analyst to conceptualize nodes within a network as representing individual people (such as policy entrepreneurs), small groups or teams, entire departments, or organizations [15] (pp. 82–88). Moreover, these actors (nodes) can be considered to behave differently from one another, as they can be reactive agents, cognitive agents, or something in between the two [9] (p. 93). Ties within this tradition, conversely, can represent simple or multiplex relationships between nodes. Analysts are encouraged to consider that nodes may or may not have different sets of resources and, thus, varying power relationships with one another [15] (pp. 91–111). These relationships can be either formal or informal and can be further characterized by their quality and the type of resource exchanges they represent, such as financial resources, natural resources, physical resources, human resources, social resources, political resources, cultural resources, and knowledge resources [15] (pp. 97–130). Another critical aspect of the governance network approach is the emphasis on processes of emergence [26] meaning that the actions and relationships of individual actors are viewed as drivers of macro-level phenomena (changes/patterns in the network structure) [9] (p. 94). These macro-level events can then be investigated through systems dynamics models, agent-based models, or social network analysis [9] (pp. 95–109).

Spatially speaking, Koliba et al. [14] argue for consideration of four nested levels of “concrete spatial scopes”–local, regional, federal, and international [14] (pp. 80–89). It is also essential for researchers to recognize that because governance networks are viewed as polycentric, complex adaptive systems, the geographic boundaries of a network can be subjective, difficult to identify, and may change over time [14]. Therefore, to determine the geographic and membership bounds of a network, Koliba et al. [14] recommend using “boundary objects, such as [but not limited to] artifacts, documents, terms, concepts, and other forms of reification that exist in social systems” [14] (p. 219). This could include information obtained from grants, contracts, regulations, websites, maps, etc. [14] (p. 219). The information obtained from boundary objects can provide the analyst with the names of actors in the system and the locations of their activities.

Koliba et al. [14] also describe various methodologies commonly used to understand governance networks, which range from thick descriptive case studies to more generalizable social network and simulation methods, like agent-based modeling and systems dynamics modeling [14] (p. 427). Moreover, Koliba et al. [14] argue that each of these methodologies provides valuable insight to researchers and has implications for managers and policymakers, thus calling for continued methodological pluralism to uncover the nuances of governance networks [14] (pp. 419–436). However, according to Morçöl [8], there is still room for improvement in methodological approaches to model and analyze these complexities and nuances [8] (pp. 92–109).

Ecology of Games. The concept of the ecology of games was originally developed by Norton Long [27] to describe urban governance systems and the interrelationships of the actors involved in local policy arenas or games [10,27]. The ecology of games framework (EGF) was built off of concepts from the original ecology of games theory as well as several additional theoretical perspectives, including polycentric systems, policy process theories, and complexity theory, to provide a better understanding of the relationships of actors engaging in policy systems, bound by institutional rules to address collective action problems [7,10]. The EGF was developed by Lubell [7] to address several issues in the earlier polycentric governance literature. For example, polycentric systems were criticized for the tendency of scholars to prescribe them as solutions for all policy problems without regard to underlying conditions or context [10,28]. Furthermore, there was a lack of attention being paid to temporal effects on polycentric systems and variation between and within systems [10]. Thus, Lubell [7] developed the EGF to provide a stronger theoretical framing for analyzing and hypothesizing how complex, polycentric systems are structured, how they govern collective action problems, and how they change over time [10]. The EGF places particular theoretical emphasis on three attributes of polycentric systems—cooperation, learning, and distribution of gains—as key to understanding system-level outcomes [28].

The EGF has been used to investigate various questions in diverse settings. For example, Hileman and Bodin [29] applied the EGF to test hypotheses on the tradeoffs between expanding relationships within polycentric environmental governance systems and effectiveness. In a separate study, Bjørndal et al. [30] used the EGF to understand how the characteristics of the Norwegian handball organizational structures influence talent development processes. Furthermore, Scott and Thomas [31] analyzed the role that structural position within an environmental policy network plays in access to resources. While based on a variety of theoretical foundations, Lubell describes EGF as being aimed at providing a framework for gaining insight into collective action problems [7].

EGF scholars tend to approach their research from a mixed-methods standpoint [7,9]. The first step in applying the EGF is for the researcher to identify the system of interest, which is a geographically defined area that contains policy issues that are engaged by actors in multiple forums [9]. The system can be selected based on convenience or some other theoretical criteria revolving around the research question [9]. Once a system is selected, the researchers must identify its relevant components—actors, issues, and forums for participation—typically through observation, interviews, document analysis, natural language processing, online searches, etc. [9].

Once relevant actors have been identified, it is common for researchers to survey the actors and ask them to identify collaborative relationships from a roster of names or to add partners that are missing from the roster [9]. For example, Scott and Thomas [31] utilized EGF as a conceptual lens to investigate how network position and attributes of collaborative groups influence resource access in the context of environmental conservancy in the Puget Sound. To bound this network, they started with a state agency to identify 57 collaborative groups working in environmental work near Puget Sound. Members of these groups were surveyed via their coordinators and asked to identify other collaborative groups they were involved in as well as identify up to five organizations that interacted with them in joint projects, coordination, and informal consultation. In another example, Angst and Hirschi [32] were able to create a network boundary regarding natural resource governance in a regional park in Switzerland that spanned two time periods. In the first period of observation, the authors identified network actors by defined positions or roles and formal group membership based on a documents review and exploratory interviews. This approach was then repeated in the second observation. In addition, the authors conducted a member check of their network boundaries by asking two key network actors to review the final list of network actors in the system for accuracy. The final list of actors in the second period of observation was then reviewed by two key actors involved in the system for accuracy [32].

While representations of the system as a social network or set of networks are common to EGF, this approach is not always the case [9,32]. The EGF is amenable to a variety of analytical approaches. For example, the EGF has been used to conceptualize case studies [9,30]. Other applications include linear modeling, social network analysis, multilevel networks, exponential random graph models (ERGMS), and related techniques (e.g., Separable Temporal ERGMs also known as STERGMs) [32], and agent-based modeling [9,32,33].

With regards to network boundaries, while the focus is on network actors associated with collective action problems, the EGF (as with most other approaches) does not specify rules for what should be considered within the system or external to it. More explication is required to set standards, particularly to inform survey respondents of what collaboration and participation mean in terms of relationships between actors, organizations, policy issues, and forums. The boundary decision for the EGF is not just set to identify the components of the system but also its geographic scale. The EGF lends itself well to incorporating and investigating the effects of time and space; however, it is recognized that smaller, local systems are far more convenient to study than larger systems that are geographically further away from the researcher [7]. This tradeoff creates a challenge for researchers not interested in local systems, as distance limits the researcher’s ability to obtain certain types of data, such as direct observation. Therefore, unless the researcher has the resources to engage in and sustain the data collection process over multiple periods, the EGF may pose significant logistical challenges. Because the EGF emphasizes system-level analysis of collective action decision-making within a given policy area, greater explication of criteria for determining sufficient qualitative saturation would provide scholars guidance about when system-level conclusions are justified.

Networks of Networks. Networks of networks is an analytical approach developed by Nowell and colleagues [12,34] to investigate and adapt theories of organizational ecology to understand network domains. Network domains are conceptualized as populations of purpose-oriented networks (PONs)4 working in a common geographic area and shared policy or problem arena [12]. This approach was developed to complement extant studies of PONs dominated by an intra-network orientation such as a focus on network resources, membership, leadership, and governance to explain network effectiveness [35]. The networks of networks approach leverages organizational theory to conceptualize PONs as whole entities operating in and interdependent with an environment comprised of other PONs working in the same issue and geographic space. Micro-processes associated with resource dependency theory, transaction costs economics, social capital, and institutional theory are theorized to drive macro-ecological effects within the network domain as well as shape the actions and outcomes of actors operating within a domain.

While this is a relatively new approach to studying networks, the methodology has been applied in several settings at varying levels of analysis. Nowell and Albrecht [34] conducted a longitudinal study of three different network domains consisting of over 70 different community health PONs over a 5-year period using stochastic actor oriented modeling (SAOMs). They conceptualized and bounded each domain based on collaborative groups fitting the definition of a PON with a stated mission related to improving some aspect of human health and wellness within a specific county. They found that the domain-level change patterns were largely consistent with the predications of organizational ecology. Van den Oord et al. [36] used the networks of networks method to understand the Antwerp (Belgium) Port Authority’s response to the COVID-19 pandemic and how they integrated other networks into their efforts. McCartha [37] examined network domain patterns of how organizations manage their portfolio of engagement to different PONs in the domain over time. Yang and Nowell [38] utilized the networks of networks methodology to investigate the role of mimetic isomorphism and strategic management initiatives in shaping organizational design and perceptions of network effectiveness. Albrecht [39] used the networks of networks approach to understand patterns of transformation from hierarchies to networks and sometimes back to hierarchies. The networks of networks methodology allows scholars to advance organizational and management theories using whole networks as the unit of analysis [38]. Additionally, because the approach results in new insights into the network environment’s role in shaping collaboration and performance, it is also highly valuable to practitioners managing these networks in the broader community.

The methods for bounding the network using a networks of networks approach begins with an initial list of PONs active within an issue or policy space within a targeted region [40]. Membership lists are then obtained from each PON within the domain and cross-referenced at both the individual and organizational units of analysis to identify membership interlocks between PONs. This process is repeated as specific time intervals for longitudinal studies. While most data needed for a networks of networks study can be obtained from archival documents, follow-up interviews and surveys can be used to obtain additional information about members and domain design and functioning. Because the networks of networks method can be used to analyze multiple levels of analysis, data are multi-mode in nature and nodes can be individuals, organizations, PONs, or any combination. Ties represent relationships and the strength of those relationships between PONs [12]. Notably, the networks of networks method emphasizes the role that membership interlocks play in enhancing social capital or working as a constraint [12].

The networks of networks methodology provides a procedure for mapping networks within a domain [12]. After specifying the rules of network membership, data are collected first via online searches to identify known PONs operating within the area of interest. After developing a membership list, the members of the networks are surveyed to identify other related networks that may not have been captured in the online search. Then, in a second phase, identified network members are asked to verify or add to the membership list. Finally, once PONs are included in the sample, PON leaders are asked to verify the level of engagement for each member as well as to provide information about each PONs. Because data collection relies heavily on archival/existing data sources and key informants for member-level data, this method has the advantage of potentially offering more comprehensive datasets of targeted system-oriented network(s); however, these data may also lack the depth of information that can be obtained through member interviews and surveys.

Like other system-oriented network approaches, the networks of networks methodology is subject to the challenges revolving around geographic boundaries. For one, the researcher determines the geographic setting, guided by theory and practicality. A larger geographic area will likely result in more networks operating in defined space, thus posing increasing response rate challenges. These challenges are heightened with longitudinal designs. However, these challenges are manageable with the appropriate level of positive relationships with network leaders. Data from this methodology have been analyzed using descriptive social network analysis and regression techniques as well as dynamic network modeling techniques (such as SAOMs) [12,39]. However, as this is a new methodology, the literature using the networks of networks approach is small, thus leaving room for integration with other analytical approaches in future studies.

PARTNER. Varda and colleagues developed a program to analyze, record, and track networks to enhance relationships (PARTNER) tool to systematically analyze cross-sector networks [11,41]. Using a survey methodology, the PARTNER tool captures data on the structure of networks as well as on four dimensions within a network that serve as a measure of quality: attribution, perceptions, agreement, and interrelationships [11,41]. These dimensions are used to develop a better understanding of network structure and effectiveness and inform decision-making on how to move the network from where it is to where the members of the network want it to be [11]. Moreover, the PARTNER methodology is heavily based on Granovetter’s theory on the strength of weak ties [42] and Burt’s [43] structural holes theory; however, the tool can also be used to advance other theoretical perspectives, such as social capital theory [11,44].

The PARTNER tool is a proprietary software, yet despite its cost, the PARTNER tool has been used in over 4000 communities in the U.S. and more than 40 countries [11]. Bohnett et al. [45] used data collected through PARTNER with an ERGM to understand the role of organizational resource contributions in creating and maintaining inter-organizational ties within Florida healthcare collaboratives. Similarly, Blebu et al. [41] used the data collected through PARTNER to identify factors that promote or hinder partnerships between healthcare workers and community-serving organizations. Rimkunas and Mellin [44] argue that the PARTNER tool may be used to apply and test components of social capital theory, at least within the context of interprofessional collaborations. As such, the tool provides an opportunity to inform both theory and practice in decision-making and network sciences.

The PARTNER tool can be used to map either entire networks or ego networks; however, as with many system-oriented network methods, the researcher must consider the adequacy of survey response rates when using the tool to understand system-level outcomes. PARTNER follows typical social network analysis practices and represents nodes as people, places, organizations, etc., and ties as the relationship between them [11]. There are four steps for utilizing PARTNER. The first of these is to bound the network. To do this, its designers recommend using a key informant method in which the researcher identifies an initial list of known network members and then collaborates with this group to identify other relevant network members [11]. The second step is to either develop a survey or use the existing PARTNER Network Survey, which collects information on four network dimensions: attribution, perceptions, agreement, and interrelationships [11]. The attribution dimension refers to how network relationships started and have changed over time [11]. Perception alludes to the views and opinions that the network members have regarding each other and the network as a whole [11]. Agreement measures the degree of consensus on the ways in which the network is functioning [11]. Interrelationships measures the “intensity, quality, and content” of relationships between members in the network [11]. After the survey is developed (or the existing PARTNER survey is chosen), it is emailed to respondents [11]. Finally, once the survey is closed, the PARTNER dashboard can be used to analyze the results, which can also be downloaded for other statistical analysis packages [11].

Bounding the network can be a challenge for any approach to network analysis. Visible Network Labs, which supports the PARTNER platform, suggests working with network members who are already known to be key informants to develop a list of network members [46]. However, they do not offer insight as to what it means to be a member of the network other than that membership is best established collaboratively, suggesting that different network members could be using different criteria to nominate other network members. This could create challenges in defining what the resulting “network” actually represents if different decision rules are applied in a non-systematic fashion. For example, Ely et al. [47] used a well-known organization to develop an initial list of network members and then collected feedback from those members to refine and bind the network member list further. Ultimately, however, the bounding decisions and approaches in the PARTNER methodology are up to the individual researcher [46]. As such, the PARTNER methodology and other approaches would benefit from further clarifying what it means to be a member of a network and clearly outlining steps and questions for identifying network partners to ensure consistency across applications.

It is unclear whether PARTNER offers any specific insight as to how to deal with geographic space in terms of bounding the network; the PARTNER literature typically refers to a community or state in terms of geographic boundaries; however, the tool can be used to collect information about where organizations are operating, and this data can then be overlaid on mapping software such as geographic information systems (GIS) [11,45,47,48]. PARTNER also has the capability to capture changes over time by implementing the survey over several periods. However, like all approaches that rely on survey data, the response rates tend to fall well below the threshold for being representative of the network [11]. As such, while the PARTNER methodology can lend itself to a longitudinal research design to track temporal changes in a network, attrition would omit a large component of the network with each period and make longitudinal comparisons problematic [11].

Typical applications of the PARTNER tool incorporate a mixed-methods analytical approach, usually combining qualitative interviews with descriptive social network measures such as density and centralization; as an added bonus, the PARTNER tool also enables the researcher to measure factors such as trust within the network through the survey instrument [41,49,50,51]. Other quantitative approaches have also been used along with the data collected through PARTNER. For example, Bohnett et al. [45] analyzed the data collected through PARTNER with an exponential random graph model. Thus, the data from PARTNER can essentially be used for analysis within the PARTNER software or as inputs for other analytical approaches [45,48].

Social–Ecological Networks. The social–ecological networks (SEN) approach developed by Bodin and colleagues [15,52] is based heavily on the natural sciences and social sciences, such as Elinor Ostrom’s social–ecological systems (SES) to understand the importance and interdependencies of biophysical, social, and institutional properties on policy and to diagnose challenges within a system [15,52,53]. Through the lens of the SEN, social–ecological systems are interconnected, multi-level arrangements comprised of governance systems, resource systems, resource units, social settings, economic settings, political settings, and ecosystems [15,54]. As such, applications of the SEN aim to understand the interactions and outcomes between actors, resources, the environment, and systems that may span across geographic boundaries and to understand three “core governance challenges”: scale sensitivity/fit, resource competition, and sequential sensitivity [15,52,55].5 Identifying the components that define a social–ecological system allows for examination and comparability among systems. Thus, the SEN can be used to assess how changes in the system’s components affect the overall system [15,55]. As such, the SEN seeks to identify better institutional arrangements within a social–ecological system [15,52,55].

Applications of the SEN revolve around environmental conservation and stewardship [52,55]. For example, Hamilton et al. [56] applied the SEN to explore factors contributing to social–ecological fit between interdependent actors connected by ecological ties in the context of wildfire mitigation. Likewise, Baggio and Hillis [57] used an agent-based model to understand how learning and idea adoption within social–ecological networks may affect the management of ecological disturbances. The SEN aims to identify and diagnose problems within polycentric environmental governance systems. However, it is unique in its efforts to integrate ecological as well as social elements into an overarching network framework [52]. The SEN is a tool that can be used to determine how and under what conditions collaborative systems improve governance of complex systems and their environmental components; thus, the SEN can be used as a tool to further theory development [15,52]. Furthermore, some scholars have found the opportunity to integrate social–ecological network research with theoretical insights and hypotheses from the EGF [9,58].

Data in the SEN approach are represented in the forms of nodes and ties. However, the choice of a node and a tie will vary depending on the phenomenon of interest to the researcher [52]. Moreover, within the SEN, it is essential to recognize that there are two distinct types of nodes—social nodes and ecological nodes. Social nodes tend to be individuals (resource owners/users), and organizations. The second type of node—ecological nodes—are typically representative of animal populations, habitats, or a phenomenon such as climate change, droughts, wildfires, etc. [15,52,55]. Because there are two types of nodes within SEN research, three types of interactions exist—social node to social node, social node to ecological node, and ecological node to ecological node [59]. The ties between nodes within the SEN indicate a relationship between two actors, competition (between species or organizations), or resource use [15,52,55]. The data used to create nodes and ties can be through various approaches: field data can be collected through interviews, surveys, experiments, etc. [15,52,55].

The SEN has no hard and fast decision rules for bounding the system. Typical applications of the SEN bound those within systems from data generated by social and ecological entities [55]. However, the SEN currently could benefit from a systematized protocol for determining who and what should be included in the system. For example, until recently, there has been little guidance for empirically explicating nodes and ties in a way that could be comparable across studies [15]. However, Bodin et al. [15] have taken steps to begin addressing this concern with the development of their comparative heuristic for SEN studies. The SEN would thus benefit from continued development regarding the description and definition of nodes and ties so that researchers can apply the SEN consistently and promote comparability between studies [15].

The bounding challenge also extends to geographic and temporal bounding decisions, typically the SEN geographically bounds systems applications to a biophysical area, socio-political units, or social and ecological entities [52,55]. Temporal applications of the SEN can be challenging as the system tends to be large spatially, thus making it difficult to collect consistent data over multiple periods [55]. However, some researchers have successfully applied time series models of the SEN. For example, Barnes et al. [60] collected data over multiple time periods and used a temporal exponential random graph model (TERGM) to study the social and environmental changes of a fishing community that was largely responsible for managing its own common-pool marine resources in Papua New Guinea. To highlight the challenge of longitudinal network studies, it should be noted that Barnes and colleagues [60] began their research in 2002 and collected data periodically until 2018, thus reflecting a sustained period of engagement in the community. Over this period, both social and ecological surveys were conducted to capture metrics and changes over time [60]. During the last two years of observation, interview data were collected from a nearly complete network of all of the households in the community. Their longitudinal analysis provides insight into the types of relationships that actors form with each other in light of resource scarcity [60].

The SEN lends itself to various analytical approaches; standard methodologies include motif frequency counts, exponential random graph modeling (ERGM), agent-based modeling, etc. [55]. Bodin et al. [15] suggest that methodological approaches to analyzing the SES are contingent upon the context of what the researcher is interested in, noting, however, that there is a need for more longitudinal SEN research [60]. It is also worth noting that modeling the social–ecological system as a network (as well as other approaches) requires one to choose how to model it, i.e., single layer, multilayer, multidimensional, etc. [55]. The SEN is a malleable framework that can be useful in various settings if the phenomenon of interest involves a meaningful interaction between social and bio-physical components [52]. However, applying the SEN, as with any methodological approach, requires informed choices regarding geography, time, data generation, and analysis that are dependent on the research question.

STEW-MAP. The Stewardship Mapping and Assessment Project (STEW-MAP) is an approach developed by the U.S. Forest Service and university researchers to aid communities in conceptualizing and analyzing environmental stewardship networks in urban areas [16]. Environmental stewardship networks in this approach are conceived of as part of a social–ecological system. As such, this approach is informed by social–ecological systems or human ecosystems theories. The aim of STEW-MAP is to provide decision-makers with an understanding of the civic capacity of organizations that occupy the same social–ecological system—i.e., the city, to identify stewardship gaps and understand the role of natural ecosystems within a city [61,62].

STEW-MAP views the city as a social–ecological system and is primarily applied in urban contexts [16,62]. Although typically applied in urban settings, STEW-MAP has been deployed in a wide, diverse range of cities around the globe. For example, the STEW-MAP program has been used in New York City, Baltimore, San Juan, Atlanta, Paris, and Valledupar [16,62]. In the case of Chicago, STEW-MAP was used to find collaborative potential for conserving land and water resources and to identify “stewardship deserts” in the city [16]. In Baltimore on the other hand, STEW-MAP was deployed to analyze the relationship between differences in the level of tree canopy and the number of (and structure of) stewardship organizations located in neighborhoods throughout the city [16,63]. One study examined the relationships between stewardship groups and neighborhood organizational, socio-economic, and environmental conditions across multiple cities using STEW-MAP data [64]. Thus, STEW-MAP is a helpful tool for public managers and decision-makers seeking to understand more about the urban environment’s complex nature and comparative analysis.

One of the strengths of STEW-MAP is the documented ways data are collected. STEW-MAP can be broken down into four phases: inventory of organizations, survey of the network, analysis, and dissemination [62]. After organizations and their partners have been identified, a survey is deployed online to gather descriptive information on groups, organizational structure, and activities; geographic information is also collected, along with social relational information on connections via funding, information, or partnerships to represent ties within the network [16]. STEW-MAP typically applies a social network method representing organizations and actors as nodes and using network descriptives, which can then be fed into a regression analysis [16]. However, one challenge is that while STEW-MAP is conceptualized as a tool to understand social–ecological systems, it is not clear if STEW-MAP networks account for the various types of nodes that are typical in the social–ecological systems networks research—i.e., both social and resource nodes. Thus, while STEW-MAP may survey groups using social–ecologically informed questions, the existence, and the nature of cross-level ties (social–ecological node connections) may be omitted in its application.

Unlike the EGF or the SEN, STEW-MAP has a delineated process for bounding the network, both geographically and nodally. First and foremost, the system is geographically bound to a city. Second, organizations within the system (the city) are asked for a list of their partners; this can be performed via snowball sampling or announcements for participation via known forums; it is suggested that this information be verified with multiple sources, such as organizational websites or other archival sources [16,62]. However, researchers have found that according to stewardship group representatives, STEW-MAP suffers from ambiguity problems; thus, the definition of partnerships may be inconsistent due to respondent subjectivity [62].

While STEW-MAP may offer a simple way to bind the system geographically, such bounding is counterintuitive to the very concept of social–ecological systems. For example, many researchers argue that social–ecological systems are comprised of interactions between actors, resources, the environment, and other systems, thus implying that they can wholly or somewhat transcend geographic boundaries [52,54,55]. Confining the systems to city limits, while simple, can lead the researcher to neglect boundary-spanning interactions with counties, other cities, regional groups, etc. However, this confinement has not always occurred in STEW-MAP applications; the City of Baltimore, for example, chose to include interaction with organizations outside the city within their stewardship network [65].

Temporally speaking, STEW-MAP offers another relatively simple approach to view changes within the system over time. The documented procedures for deploying STEW-MAP make the process repeatable, and some cities have done just that over the years [16,65]. However, because STEW-MAP survey response rates tend to fall below 50%, the results are only fractionally representative of the entire network, and the likelihood of collecting data from the same respondents over the years is low [16]. However, it should be noted that some researchers have worked to overcome the missing data challenge through imputation and network reconstruction methods [66].

Analytic approaches to STEW-MAP also tend to be rather simplistic compared to other network mapping approaches (EGF or SEN), although no systematic review has been conducted to confirm this. Many applications of STEW-MAP have used simple social network descriptives to describe the network and then used other data sources (surveys, archives, etc.) to place stewardship organizations and their network connections on an interactive map [16,66]. Others, such as in the cases of Baltimore and Seattle, have combined social network analysis with spatial regression techniques [16,63]. While the analytical approaches have typically been relatively simple to implement, possibly due to the emphasis on practitioners, the STEW-MAP data collection process would likely lend itself well to other forms of analysis, such as agent-based modeling, TERGMs, or other spatial analytic techniques [16,67].

4. Discussion

In the modern era, public policy and administration revolve around networks in some form or fashion [14]. Many of these networks reflect system-oriented networks, meaning they are defined by an analyst to represent a network of actors associated with some system-level outcome of interest based on a series of theoretically grounded decision rules [5]. The approaches compared above reveal a rich tapestry of both theoretical perspectives and methodological options for gaining insight into these complex social phenomena. The approaches discussed here reveal how versatile the concept of system-oriented networks can be. All reviewed approaches seek to identify and analyze a system-oriented network; however, each approach emphasizes a different theoretical focus. Approaches also differ considerably in the extent to which both theoretical and methodological elements have been explicated. While many approaches exist that can be used to understand system-oriented networks, little work has been done to compare them so scholars can choose when one framework or methodology is the most appropriate for their research question and setting.

Choosing the Right Approach. Perhaps the most obvious difference between the various approaches is the researcher’s theoretical orientation and the nature of questions that are asked prior to picking an approach. For example, if the researcher seeks to apply theories of policy process to understand a collective action problem and their governance solutions, or the evolution of polycentric systems, an application of the EGF or governance networks approach may be the most appropriate. Alternatively, if the researcher is interested in extending organizational theories to networks as the primary unit of analysis, then a networks of networks approach might be the preferred choice. Community planning scholars interested in how communities can engage in and enhance cross-sector collaboration to fulfill a public need may align well with the PARTNER tool. Finally, if the concern centers around the relationships between ecological and social systems or environmental stewardship then the SEN or STEW-MAP, respectively, are options researchers can utilize.

In deciding which approach is the most appropriate, researchers should also consider that some approaches have differing data requirements. For example, what can be represented as a node or a tie in the network can vary depending on the framework that is used. For example, to our knowledge, the SEN is the only approach discussed that explicitly calls for representing both ecological nodes such as animals, habitats, climate change, droughts, and wildfires along with social nodes—resource users [15,55]. Additionally, within the networks of networks method, the data focuses explicitly on geographically linked communities of purpose-oriented networks. Moreover, several approaches provide a procedure for identifying network members, although the definition of membership could be refined. For example, multiple sources are used to triangulate membership in a governance networks approach, an EGF approach, an SEN approach, and a networks of networks approach. Other approaches, such as the PARTNER tool heavily rely on known network members to self-define what it means to be a network member [47].

While approaches have historically tended to be siloed into fragmented literatures, it is possible that the right approach for a given study may be an integration of approaches and their associated theoretical perspectives. This reflects a key opportunity for future research and for cross-pollination from other disciplines, such as systems science or systems engineering. For example, while the SEN approach is the only approach discussed in this showcase of approaches that explicitly represents ecological components as nodes, new theoretical insights might be revealed from an integration of SEN and EGF perspectives such as in recent work by Hedlund and colleagues [58]. Such analysis would require software such as NEO4J6 or other social network graphing tools that allow for multiple node types within the same graph. Similarly, EGF and governance networks both seek to understand opportunities and tradeoffs associated with different forms of polycentric governance. There is, perhaps, an opportunity to integrate the hypotheses and theoretical refinement of the EGF with greater attention to the more detailed conceptualization of complex governance networks. For example, the governance networks literature discusses the multi-dimensionality of accountability relationships and how they can influence system dynamics. A careful integration may provide insight into EGF hypotheses regarding when and under what conditions political actors will use their power in one way or another and when that use of political power may be harmful to the system [7,8]. Finally, there is a significant opportunity for future research to consider new insights that may be gained through the integration of perspectives from organizational theory and policy process theory in our study of system-oriented networks.

Advancing Methods for Studying System-Oriented Networks. In addition to advancing the theoretical development of our understanding of system-oriented networks, there is also opportunity to advance our methodological understanding of best practices in the study of system-oriented networks, regardless of the approach selected. In each approach we highlighted the importance of clarifying decision rules and associated procedures for defining and justifying the boundaries of the network and its relevant actors and relations. As system-oriented networks are complex phenomena, researchers should consider that data may need to be generated from various sources while at the same time, recognizing the potential for methods themselves to lead to very different representations of “the network” [12]. As with any research, the scholar’s decisions at each phase of research can influence the findings. This is particularly true in research on system-oriented networks, where the researcher must make a series of choices to define the system of interest. The decision rules revolving around a system’s geographic and membership bounds should be well described in system-oriented network research, as should the procedures used to analyze the data or test hypotheses so that readers can adequately understand and critique.

Systematic triangulation is important for the empirically robust representation of any system-oriented network. Triangulation can be accomplished through multiple sources and multiple source types. For example, scholars can use sources such as meeting minutes, collaborative agreements, planning documents, websites, maps, interviews, etc. [7,9,12,14,52,68]. Triangulation of the data will lead to more robust insights into appropriate system boundaries, membership, activities themselves, and the locations of activities that would be omitted from any single source.

Furthermore, because they are complex systems, a mixed-methods approach is often key to studying and understanding system-oriented networks. Mixed-methods studies employ the insights that can be gained from both inductive and deductive designs; thus, a mixed-methods approach would contribute to both the breadth (generalizability) and depth (nuanced understanding of a phenomenon) of studies of system-oriented networks [69] (pp. 11–20). Mixed-methods designs are necessary whenever quantitative or qualitative designs alone cannot address a research problem [69] (p. 19). Social network analysis, for example, may reveal that some component of network structure relates to some outcome of interest, but the insight gained from a qualitative complement in the study may reveal why this is the case [70]. This, in turn, can aid the analyst in deciding how to incorporate advanced systems dynamics or agent-based modeling techniques that require an in-depth knowledge of the actors and system being modeled [8,71,72]. In this way, studies of system-oriented networks should be viewed as abductive endeavors, meaning that the researcher must balance the ebb and flow of inductive and deductive reasoning throughout the entire research process [73].

This leads to this final point: studies of system-oriented networks are typically interpretive and abductive in nature. While system-oriented networks can be used as a sampling frame for understanding more micro-dynamics using traditional sampling theory and deductive methods, extrapolation of these dynamics to the system as a whole should be done with care and viewed through an interpretive lens. Again, we argue that an inductive or abductive approach is inherent in this type of research. As such, the criteria for evaluating system-oriented network studies must extend beyond traditional standards of validity, reliability, and generalizability. It is important to note that a recent study of mixed-methods research in public policy and public administration found that most mixed-methods studies were dominated by their quantitative components with little attention to and few attempts to adequately integrate their qualitative counterparts [70]. While these validity, reliability, and generalizability criteria may be appropriate for the deductive elements of these studies, they are not appropriate for making inductive or abductive arguments about the nature of a given system-oriented network [19,74].

For inductive, interpretive analysis, methodological rigor is demonstrated through credibility, transferability, dependability, and confirmability [19,75]. Credibility can be exemplified through prolonged engagement in the field, persistent observations, triangulation of the data, and member checks [75]. Transferability can be shown through consistent and thick descriptions of the research context and the qualitative findings that allow for the comparability of future studies [70]. As such, researchers should take special care to ensure that they do not fail to adequately integrate meaningful qualitative findings [70]. Dependability can be demonstrated through well-documented and transparent procedures [75]; unfortunately, Hendren et al. [70] found that reporting on methodological decisions tends to be lacking in mixed-methods research, and many studies were missing reflexive statements on their choice of the research design itself, all of which should serve as a caution to future scholars hoping to publish mixed-methods studies that are dependable. Furthermore, mixed-methods research tends to fall into three overarching categories, convergent mix-methods, where both qualitative and quantitative data are collected and analyzed simultaneously [69] (p. 15). Explanatory sequential mixed-methods, where the researcher uses quantitative findings and adds nuance by incorporating a follow-up qualitative component that is guided by the quantitative results [69] (p. 15). And, exploratory sequential mixed-methods, which begin with qualitative research, and the resultant findings are used to develop a quantitative component [69] (p. 15). In any of these mixed-methods design types, it is critical for the researcher to document and provide their reasoning in their design choice and how data collection and analysis in one stage of the research fed into other decisions throughout the remainder of the research process [70].

5. Conclusions

This article provides a brief overview and analysis of popular approaches to understanding system-oriented networks, their foundations and theoretical underpinnings, focuses, challenges, and use case examples; we hope this will allow future scholars to make informed choices regarding their use of an approach or framework in their research designs. While this showcase covers a broad range of information relating to the six approaches to studying system-oriented networks, it should only serve as one of many resources for scholars seeking to use any of the discussed approaches. While our framework for comparing approaches to studying system-oriented networks comes from an institutional perspective (public policy and public management), future scholars should consider more comparisons to bridge the disparate literature or other ways in which these or other approaches can be refined or integrated. The approaches discussed above are each supported by a rich body of work such as empirical investigations, in some cases, approach-specific systematic reviews, methodological texts, books, etc. In the case of PARTNER and STEW-MAP, which were developed with a focus on practitioners, there are many case examples of how these tools have been deployed to assist practitioners in their problem-solving endeavors. We argue that by offering a framework (Table 1) through which to compare these approaches, we have begun to bridge the gap between the disciplines and offer a model to compare other similar approaches not discussed in this work. Furthermore, by indicating areas for future integration, we hope to spur advancement in the science of studying system-oriented networks and practice.