Holistic Infrastructure Resilience Research Requires Multiple Perspectives, Not Just Multiple Disciplines

Resilience research includes multiple definitions, concepts, perspectives, and applications across a broad range of academic disciplines. While experts, policy-makers, and practitioners assert that resilience requires holism, what is considered holistic is rarely discussed. The traditional scientific approach to holism is to engage multiple disciplines. However, this review studies an alternative approach to holism that engages multiple perspectives, as suggested by integral theory. An integral approach requires consideration of at least four irreducible domains: (1) subjective experience, (2) intersubjective culture, (3) objective behavior, and (4) interobjective systems. This way of approaching holism both engages multiple disciplines and reveals important gaps in the popular understanding of resilient infrastructure. For example, organizing the 20 most highly cited resilience research articles from all disciplines according to the Integral Map reveals that most articles in the sample set are distributed among three of the four perspectives corresponding to experience, behavior, and systems. None of the most popular articles studies resilience through the lens of culture. Thus, the importance of factors such as organizational values and group intentionality may be underappreciated in the scholarly literature.


Introduction
In applications such as infrastructure, which include multiple interconnected and interdependent social, ecological, and technological systems, the need for the integration of diverse concepts, definitions, and perspectives of people and coupled systems is particularly acute. The reason is because infrastructure contains an ensemble of interdependent systems such as power, water, transportation, and cyber security [1]. These coupled systems interact with one another on multiple levels to deliver products and services that are deemed essential to public health, safety, and well-being [2]. For example, power requires water for cooling, and water needs power to operate pumps and filtration systems [3]. Both systems rely on security and communication networks across heterogeneous spatial and temporal scales. This presents a problem because protocols for communication across operational boundaries are typically absent [4,5]. Interdependencies among coupled infrastructure systems are especially apparent in disaster scenarios such as the widespread power failure in 2013 that impacted parts of the northeastern United States and Canada. Disruptions due to the loss of power led to cascading failures across New York City that included communication systems, emergency services, transportation, water supply, and water treatment [6]. Moreover, in addition to the physical and functional connections and interdependencies, people are involved with every part of the power, water, and communication systems, from theory and conceptual development to design, operation, and management.
The growing interest in resilience research reflects a disparate landscape of definitions, concepts, and applications across a broad range of academic disciplines. Resilience concepts and applications have expanded since the mid-1800s, when "resilience" was first used to describe the engineering properties of materials [7]. Psychology and human development research began using "resilience" in the 1960s [8] to describe children's health and social adaptation in response to loss or adversity. Academic interest in resilience is often attributed to research by Holling [9] in the 1970s that extended resilience concepts from physics and engineering to ecological systems theory. Resilience research has since grown to include a range of disciplines to describe the ability to respond to unpredicted interruptions or shocks. Resilience in psychology demonstrates ways to improve how individual people and social groups such as family respond to catastrophe in their lives [10], mental and physical disabilities [11], and normal physiological processes such as aging [12]. Ecological resilience describes how ecological cycles of growth and collapse corresponding to dynamic changes in ecosystem stability can influence environmental states [13][14][15][16][17]. Socio-ecological resilience extends ecological concepts to include interactions between humans and environments, and provides a basis for ecosystem management strategies [15,[18][19][20]. Resilience in engineering systems orient designers, operators, and crisis managers to better plan and prepare, absorb, recover, and adapt to unforeseen threats [21][22][23] and propose methods for measurement and analytical quantification for infrastructure assets and systems [24][25][26]. In addition, resilience is now a common term in the public realm among federal and international agencies, particularly those focused on managing major crises caused by natural and man-made disasters [2, 23,27,28], and emerging long-term threats such as climate change [29].
Amid the surge of resilience research in diverse disciplines, researchers and practitioners alike recommend taking a holistic approach that bridges interpretations across disciplines to support the resilience of complex systems [30]. Yet policy documents such as Presidential Policy Directive 21 that call for a "holistic" approach to building resilient infrastructure [31] offer little guidance on the breadth of the boundaries that are necessary to constitute holism. As a result, researchers have little to guide their holistic intentions, and references to the term holistic or holism lack definition. While concepts of holism vary, in a basic sense, holistic research seeks to incorporate knowledge from multiple sources. Traditionally, this knowledge is organized into scientific disciplines, so holism typically means multidisciplinary or interdisciplinary knowledge that is organized around a complex problem and cannot be subsumed by a single disciplinary domain.
The dominant metaphor for the organization of knowledge has been, at least since Aristotle, the tree, in which foundational knowledge represented by the trunk is divided up into increasingly specialized limbs and branches until a single specialist might be considered the metaphorical equivalent of a leaf at the end of the subdisciplinary twig. Nonetheless, information communication technology is changing the ways that scientific knowledge is organized. The emerging metaphor for organizing knowledge is now the web, as exemplified by the worldwide web [32]. In a web topology, all knowledge can be connected to all other knowledge without first reducing it to the most general common principles (i.e., traversing down to a single common limb or trunk).
The new web metaphor is already ubiquitous in the way scientists search for knowledge. Before computing became ubiquitous, scholars would typically visit the disciplinary branch of their campus library (e.g., the law library, or the engineering library) to search through card catalogues, microfiche, and closed stacks. The underlying assumption was that nearly everything relevant to their field could be contained in their specialized building. Now, scholars search online using citation databases that span a broad swath of disciplinary domains connected through keywords, text, and bibliographic analyses. The change in metaphor from tree to web demands a new type of scientist who is capable of operating more as a spider weaving connection between disciplinary branches, instead of operating as a leaf that extends a single subdisciplinary branch [33].
Nonetheless, the spider-like approach to the organization of knowledge remains so new that there is little guidance with regard to what knowledge should be joined in any particular web. That is, the complex and multifaceted aspects of a complex system such as infrastructure resilience are so extensive that seemingly any knowledge might be connected to a specific problem. This work describes a novel approach to ensuring a holistic representation of knowledge perspectives without exhausting researchers by necessitating examination of every potential knowledge connection. To establish this new understanding of holism, we adapt the "Integral Map" [34] that has previously been applied to other complex systems research problems, including ecology [35], sustainability [36], and climate change [37]. Additionally, we demonstrate how the Integral Map supports holistic research by organizing a set of 20 highly cited resilience research articles. The Integral Map reveals commonalities, differences, and potential gaps among different interpretations of resilience, and provides a logical and coherent way to integrate knowledge from diverse academic disciplines and perspectives. Finally, we discover that the quadrant of the Map that corresponds to organizations and culture (the intersubjective) is underrepresented among the most highly cited articles reviewed, suggesting a gap in scholarly attention that is relevant to resilient infrastructure research.

Infrastructure, Holism, and Holistic Resilience Research
Controversy over the term resilience in the context of infrastructure is likely to persist. Like all essentially contested concepts (e.g., fairness, justice, sustainability-Seager et al. [38]), the abstract concept of resilience has broad appeal, despite engendering disagreement on specifics. For example, Woods [39] identified four different conceptualizations of resilience: robustness, rebound, graceful extensibility, and sustained adaptability. Within the context of rebound from disaster, Seager et al. [40] identified at least three different ways to think about resilience metrics: resources, processes, or outcomes. While resource and outcome-based approaches lend themselves to measurement (e.g., Petit et al. [41]), quantitative analyses of process-based perspectives are problematic, partly because the stressors that reveal adaptive capacities are difficult to simulate. When resilience is viewed as an emergent property of a complex system, it cannot be assessed a priori [42]. Hollnagel et al. [43] described four essential abilities, including: (1) knowing what to look for-monitoring the critical, (2) knowing what to expect-anticipating the potential, (3) knowing what to do-responding to the actual, and (4) knowing what happened-learning from the factual. Park et al. (2012) restated and simplified these into four essential processes: sensing, anticipating, adapting, and learning (SAAL). Righi et al. [44] reviewed 237 papers related to resilience engineering and described a mesh of definitions and frameworks with many of the core concepts drawn from other fields. While each of these prior works may be described as inclusive of multiple dimensions, none of them necessarily meet a standard of holism.

Holism in Infrastructure Resilience Research
In infrastructure, a holistic perspective is a comprehensive consideration of systems that includes physical interdependences as well as governing rules or institutions [45,46]. Holism in infrastructure must reach beyond engineered interdependencies to incorporate human relationships that include psychological and behavioral perspectives and human perspectives such as shared values, ethics, cultural beliefs, and worldviews [47]. Accounting for people requires infrastructure resilience to consider interdisciplinary perspectives, yet many existing perspectives are only superficially more holistic than research from a single discipline. True interdisciplinary research considers multiple ways of measuring and knowing resilience (i.e., resilience epistemologies) equally [48]. This is because different disciplines such as engineering and social science rely on different means to acquire knowledge e.g., qualitative versus quantitative research. Miller et al. [48] argued that research initiatives accommodating multiple epistemologies are needed to achieve an integrated approach to the study of complex socio-ecological systems. This concern is notably amplified in applications such as climate change, as experts such as O'Brien and Hochachka [37] noted that most interdisciplinary frameworks based on single conceptual models or methods are inadequate to the study of climate. A lack of holistic framing makes it difficult to examine relationships and interdependencies among coupled systems driving the myriad of climate change issues ranging from carbon emissions and clean energy resources to poverty and social justice [49]. When confronted with complex problems, available examples of multidisciplinary and interdisciplinary frameworks do not support holistic research by failing to include multiple ways of understanding resilience.
Limited examples of holistic infrastructure literature expand beyond technology-focused and disciplinary perspectives. Bruneau et al. [24] and Hossieni et al. [50] suggested that a comprehensive approach to infrastructure resilience includes technical, organizational, economic, and social domains. Labaka [30] argued that a holistic approach to critical infrastructure is one that includes the same four domains, in addition to the external agents influencing them. The domains help clarify boundary conditions of the units or systems of consideration, and determine what variables are included. Thus, the holistic framework proposed by Labaka, in particular, offers a thorough and comprehensive consideration of the properties, processes, and systems impacting critical infrastructure in addition to a wide range of stakeholders. However, the social domain is focused on operating policy without consideration of value systems, culture, or intersubjective meaning-making. Even expanded perspectives that include a broader range of social and technological systems do not consider values and culture. For example, Lauge [45] presented a holistic approach to understanding critical infrastructure dependencies, by collecting comprehensive surveys representing 11 system operators (e.g., energy, water, and transport) across Europe, North America, and Asia). The holistic approach described in the study considers the impacts on each system in response to its potential impacts on the other systems. However, none of the 11 infrastructures represented people or social systems. Yet people are embedded in each, and differences among responses reflect the complex dynamics of individuals and social systems interacting with technological systems. Renschler et al. [51] built upon Bruneau et al. to include social/cultural and lifestyle domains, yet these domains are only captured by services (e.g., education services) rather than cultural. In a sense, existing research cannot be considered holistic because it has not considered the critical factors that dictate human action.

Advancing Holistic Infrastructure Resilience Research with Broader Perspectives
Outside infrastructure research, there are definitions and concepts of holism that can bolster existing perspectives on resilience. Overton [52] described holism as the inclusion of all subjects, objects, and events in addition to the relationships among them. That is, holism includes the interconnectedness and interrelatedness among people, systems, and the environmental contexts within which they are embedded. Although this description lacks detail about the nature of the relationships, others are more specific. Cardona [53] argued that a holistic understanding of risk and vulnerability in relation to disaster events includes multiple interrelated subjective and objective factors among social, economic, and environmental impacts. In contrast to Overton, Cardona argued that a holistic perspective must also include internal (i.e., human interior) properties, capacities, and vulnerabilities (e.g., values, ethics, and worldviews) to increase the effectiveness of risk management [53]. Thus, a holistic understanding elucidates how risks and vulnerability can be unevenly distributed over a region or a population according to factors related to the individual and group interiors of people. An individual refers to a human subject such as an operator or a maintenance worker, and a group refers to multiple human subjects organized around a purpose, such as a control room team at a power plant.
Koestler [54] suggested that holism considers how each part of a system exists in relation to the other parts, both as a whole, and as a part of a larger whole. Koestler used the term 'holon' as a unit of holistic analysis characterized by its relationships among other parts. The metaphor of Russian dolls nested one inside of the other and forming hierarchal structures of increasing complexity is sometimes used to describe the concept of holons [54]. In other words, holons may be viewed as nested structures of complexity representing distinct "wholes" that are made up of equally whole parts [35]. As a unit of analysis, a holon can be an object such as a power plant, a person such as an operator, or a research topic.
Moreover, there are social holons (e.g., families, communities, region, state, and nation), organismic holons (e.g., atoms, molecules, cells, and organisms), ecological holons (e.g., soil, foliage, insects, and birds), and technical holons (e.g., microprocessors, circuit boards, computers, and networks). Although the above definitions and concepts of holism are helpful, Koestler's definitions lack guidance for enacting a holistic approach to research that includes human values, feelings, and experiences.
Holistic approaches for infrastructure resilience must expand to include these missing human dimensions. Subjective factors such as values, ethics, and culture dictate human decision-making before, during, and after catastrophic failures [53]. For example, explicit consideration of values in infrastructure resilience studies may reveal trade-offs in planning and preparations and post-catastrophe recovery that were otherwise unstated. An individual's personal experience, which cannot be directly observed or measured [55], further influences their adaptive capacity [56], and by virtue, the adaptive capacity of the infrastructure system with which they interact. Disasters such as the Tohoku earthquake and Fukushima power plant meltdown show how basic assumptions about safety (e.g., tacit cultural beliefs, attitudes, and perceptions) held by workers and government officials impacted disaster recovery [57]. The safety culture before the disaster, which was based on inaccurate assumptions, led to diminished authority and poor decision-making that delayed critical disclosures and compromised safety and recovery efforts [58]. In contrast, the Christchurch earthquake disaster exemplified how community members experiencing shared vulnerabilities responded with a collective capacity to innovate and self-organize because of shared values in community support [59]. For example, a group of college students used social media to form a volunteer force of thousands to coordinate support amid multiple infrastructure failures [59].

An Integral Approach to Holism
Given the limitations of existing interdisciplinary and multidisciplinary models for the holistic organization of knowledge relevant to resilient infrastructure, there is a need for a new approach to ensuring holism without necessitating an impossible standard of including everything. Integral theory may be the most promising approach, as it presents a comprehensive holistic framework for organizing a diversity of knowledge in a meaningful and coherent manner [60] that may be useful for organizing holistic resilience research for applications such as infrastructure. The theory posits that there are four fundamental human perspectives that are always present, and cannot be reduced any further [34]. These perspectives embody two epistemological distinctions of how knowledge may be considered. The first distinction is between interior and exterior, and the second distinction is between singular and plural. The four perspectives are: individual interior ("I"), group interior ("we"), individual exterior ("it"), and group exterior ("its"). The interior refers to the intangible properties and processes that cannot be identified by the physical senses [37] such as psychological and emotional capacities. The exterior refers to the objective, physical properties, processes, and interactions that can be observed and measured such as social structures, technical systems, and the natural environment. One of the critical differences between the multidisciplinary and integral approaches is the treatment of human subjects-i.e., "I" and "we". Whereas disciplinary boundaries make distinctions between the study of behavior in humans (e.g., social science) and technical systems (e.g., engineering), integral theory recognizes that studies of human behavior and experience come from exterior and interior perspectives, respectively. For example, the description of human behavior as phenomena exhibited by objective agents is a third person exterior perspective-i.e., "it"-without regard for the interior experience of feelings or meaning-making. In contrast, descriptions of the human experience, including feelings, values, and meaning, take an interior perspective. Thus, integral theory accommodates both the physical systems perspectives (e.g., exemplified by operations research), social sciences perspectives (e.g., economics) that dominate infrastructure literature, alongside interior human dimensions (such as psychology, sociology, and anthropology).

The Integral Map
Integral theory argues that the four irreducible perspectives, summarized as 'I', 'we', 'it', and 'its', are fundamental to any inquiry [34], and a holistic understanding must include representative knowledge from each perspective without favoring one over another [60]. Therefore, an integral approach to research seeks to identify what perspectives are included, or not, in a given research investigation by organizing knowledge, epistemologies, and methods [61]. Numerous researchers have applied integral theory to other holistic research programs. Examples include developmental psychology [61], ecology [35], education [62], and sustainability [63]. Although a holistic approach to research includes both epistemological and ontological diversity, this work is focused on epistemological perspectives of resilience. The characteristic method for a holistic organization of knowledge with an integral approach uses an Integral Map, as shown in Figure 1. Despite the Integral framework's separation between quadrants, a holistic approach can combine different perspectives or ways of knowing by incorporating multiple epistemologies in relation to a common research question or unit of investigation. This is because the quadrants on the Integral Map representing experience, culture, behavior, and systems are concomitant perspectives, which means that they occur together in mutuality. This includes the relationships and interdependencies among knowledge claims within and among quadrants. Thus, the framework provides a means of investigating the interrelatedness among perspectives, which can support comparison, correlation, and the potential linking of knowledge claims investigating the same or similar resilience phenomena. Concepts once considered as conflicting may be complimentary or even mutually informing when viewed with a holistic approach through an Integral lens. Moreover, conflicts may be better understood when different concepts and definitions are considered with regard to the interior and exterior of individual and collective perspectives corresponding to the quadrants of an Integral Map. For example, whereas child psychology describes resilience as observable processes representing positive adaptation amid adversity [67], some researchers view human resilience in terms of interior characteristics or properties such as self-esteem or the ability to cope [10,68]. The distinction between interior qualitative and exterior quantitative properties and processes has been the subject of debate in the psychology resilience literature for many years [7,67]. With a holistic approach, both perspectives contribute valuable information about individual human resilience and indicate how the interior properties of individual experience or group culture may relate to or influence exterior behavior and dynamic interactions among coupled complex systems.
As another example, consider how human interiority is interconnected to climate change. Subjective beliefs, ethical standards, and cultural norms can influence assumptions about how The quadrants of the Integral Map represent the interior and exterior of individual and collective human perspectives [64]. An Integral Map can be applied to any phenomena or unit of investigation (e.g., resilience research) whereby related knowledge is assembled in a structure of four domains of perspectives corresponding to experience (subjective 'I'), culture (intersubjective 'we'), behavior (objective 'it'), and systems (interobjective 'its') [34]. The framework provides a holistic structure for research by delineating among different ways of knowing, methods, and tools from each of the four domains. This means that an Integral Map can help differentiate among research perspectives [65] and provide a clear and systematic method of determining what knowledge is present in a given investigation [61]. Thus, an integral approach can be helpful in identifying gaps among diverse perspectives in existing research. Moreover, the Integral Map is content-free, i.e., it does not add any new information to the research question or initiative in consideration. Instead, it distinguishes among perspectives and identifies which ones are represented or not. Taken together, the four quadrants of the Integral Map combine to support a mixed-methods approach to research [66] by incorporating multiple ways of knowing to interrogate a research question or investigation. A holistic approach to research will include perspectives from all quadrants, thereby representing each of the four fundamental domains corresponding to different ways of knowing. With an integral approach to holism, no research knowledge or method is given preference over another, and all of the perspectives are allowed equal consideration. The delineation among perspectives corresponding to the four quadrants shows how disparate disciplines and resilience concepts can be viewed as complimentary contributors toward a larger more comprehensive understanding of the resilience of whole systems.
An Integral Map may be used, for example, to assess an individual's different ways of knowing, interacting with, and experiencing a given phenomena. The experience quadrant in Figure 1 (upper left) identifies interior awareness represented by a subjective 'I' perspective. Knowledge in the experience quadrant includes factors such as cognition, affect, and psychological maturity. The behavior quadrant (upper right) identifies exterior awareness represented by an objective 'it' perspective. In the behavior quadrant, knowledge includes individual actions, physical properties, and artifacts. The culture quadrant (lower left) is a collective interior of shared human awareness represented by the intersubjective 'we' perspective. Knowledge in the culture quadrant includes factors such as shared values, ethics, and worldviews. The systems quadrant (lower right) identifies a collective exterior awareness. Sample knowledge in the systems quadrant includes dynamic interactions between and among complex social, ecological, and technological systems, which includes dynamic relationships and interdependencies among infrastructure. Thus, an Integral Map offers a structure and process to organize a holistic approach to resilience research.
Despite the Integral framework's separation between quadrants, a holistic approach can combine different perspectives or ways of knowing by incorporating multiple epistemologies in relation to a common research question or unit of investigation. This is because the quadrants on the Integral Map representing experience, culture, behavior, and systems are concomitant perspectives, which means that they occur together in mutuality. This includes the relationships and interdependencies among knowledge claims within and among quadrants. Thus, the framework provides a means of investigating the interrelatedness among perspectives, which can support comparison, correlation, and the potential linking of knowledge claims investigating the same or similar resilience phenomena. Concepts once considered as conflicting may be complimentary or even mutually informing when viewed with a holistic approach through an Integral lens. Moreover, conflicts may be better understood when different concepts and definitions are considered with regard to the interior and exterior of individual and collective perspectives corresponding to the quadrants of an Integral Map. For example, whereas child psychology describes resilience as observable processes representing positive adaptation amid adversity [67], some researchers view human resilience in terms of interior characteristics or properties such as self-esteem or the ability to cope [10,68]. The distinction between interior qualitative and exterior quantitative properties and processes has been the subject of debate in the psychology resilience literature for many years [7,67]. With a holistic approach, both perspectives contribute valuable information about individual human resilience and indicate how the interior properties of individual experience or group culture may relate to or influence exterior behavior and dynamic interactions among coupled complex systems.
As another example, consider how human interiority is interconnected to climate change. Subjective beliefs, ethical standards, and cultural norms can influence assumptions about how climate change problems are defined and how roles, responsibilities, and solutions are conceived and proposed [37]. Moreover, subjective issues can impact how actions taken by different groups can cause an uneven distribution of resources, risk, and vulnerability in response to climate change [69]. This means that an integrated framework for climate change must accommodate both subjective and objective research perspectives. Although research based on objective criteria can study the dynamic relationships and interdependencies among complex systems, it cannot account for the influence of subjective criteria such as beliefs and worldviews that define climate change problems and constrain solutions. O'Brien and Hochachka [37] suggested that an Integral Map offers a framework for incorporating both subjective and objective perspectives of climate change along with the relationships between them. This is because an Integral approach to research accommodates both the interior and exterior dimensions of human awareness [66]. The interior dimensions refer to the subjective and intentional nature of individuals and groups, and the exterior dimensions refer to the objective physical and systemic phenomena. Thus, an Integral Map provides a holistic framework for understanding how subjective and objective perspectives of climate change are interconnected, and how they can influence one another [37].
The recognition that each quadrant of the Integral Map contributes irreducible information to holism [35] may help resolve differences between perspectives arguing one view over another in resilience research. This is because each of the four perspectives co-exist in mutuality within a holistic framework without a need to reduce or marginalize other perspectives in order to establish its claims. Although a holistic approach incorporates perspectives from all four quadrants, it does not mean that all research pursuits must consider all four quadrants. For example, infrastructure resilience literature that emphasizes a single quadrant (i.e., systems) offers a distinct way of knowing and interpreting resilience that can contribute toward a holistic view. However, resilience research is broader than this single perspective, and it is important to understand how resilience concepts and definitions across multiple paradigms align within the Integral Map. Starting more broadly and framing infrastructure literature with respect to other influential work, this approach can guide researchers to ascertain how their perspective contributes to advancing theory and applications across otherwise distinct resilience perspectives. Moreover, this approach can also guide researchers to ascertain what perspectives are included or excluded with a given investigation. Thus, a holistic approach with the Integral Map helps ensure multiple epistemologies are considered to provide the most complete, comprehensive, and holistic understanding of resilience possible. This could inform resilience research initiatives seeking a comprehensive approach to integrating perspectives from multiple disciplines.

An Integral Approach to Holistic Resilience Research
The absence of a common interdisciplinary framework for organizing and linking subjective and objective research [37,69] limits opportunities to extend learning about resilience beyond isolated academic boundaries. As a result, disparate disciplines that both contribute to resilience literature such as engineering and psychology lack a structure or means of informing one another's concepts and findings. This, in turn, precludes a larger, more holistic, transdisciplinary understanding of resilience by privileging some perspectives while marginalizing others. We demonstrate one way to overcome this gap by organizing a select group of resilience research articles by using an Integral Map. The unit of investigation applied to the Integral Map is the dominant epistemological orientation for each article corresponding to the quadrants: subjective (experience), intersubjective (culture), objective (behavior), and interobjective (systems). Another key objective is to show how an Integral Map can serve as a guide for assessing holism, and design a holistic approach to resilience research.

Holistic Resilience Research Assessment-Assumptions and Methods
We identify and map 20 highly cited resilience research articles to demonstrate how different literature can contribute to a holistic resilience perspective, and identify how infrastructure research relates to the broader resilience field. Using the Web of Science (October 2015), a literature search to identify research articles with the terms "resilient", "resilience", or "resiliency" in the title produces a total of 15,574 publications spanning 115 years . No other search terms or field limitations were applied to capture publications from a wide range of subjects and a variety of sources, including peer-reviewed journal publications and conference proceedings. To demonstrate both the breadth of resilience research and how infrastructure literature fits within a holistic perspective, it is unnecessary to map this entire set of publications. Instead, we selected the 20 articles from the search with the largest number of citations as a representative sample of broad research perspectives that also include infrastructure related subdisciplines. Specifically, literature spanned the disciplines of adult psychology, child psychology, neuroscience, ecology, socio-ecological systems, and technology. Although this approach limits our interpretations and conclusions to the articles reviewed, it demonstrates how the Integral Map can be used to assess holism. We also find that the inclusion of a greater number of publications would not invalidate or substantially change publication mapping. Moreover, by focusing on only 20 highly cited papers, we explain what this mapping may imply for holism and infrastructure resilience research trends.
To map publications, we reviewed each article to assess how it aligned with knowledge perspectives and examples associated with the four quadrants of the Integral Map. The unit of investigation is the dominant epistemological orientation for each research article corresponding to each quadrant. To accomplish this, we adapted the heuristic shown in Figure 2. The heuristic is a synthesis of multiple works, including applications of the Integral Map to ecology [35], sustainability [36], and climate change [37]. In addition to the perspectives, focus, and examples in the heuristic, which are based on the Integral Map [60,61], we included representative properties of resilience corresponding to quadrants based on experience [70], culture [71], behavior [67], and systems [22] from the resilience literature. The three authors reached consensus on the assignments by discussion. A similar process can be followed to develop a heuristic for any field or application by adhering to the perspective associated with each quadrant of the Integral Map.  To map publications, we reviewed each article to assess how it aligned with knowledge perspectives and examples associated with the four quadrants of the Integral Map. The unit of investigation is the dominant epistemological orientation for each research article corresponding to each quadrant. To accomplish this, we adapted the heuristic shown in Figure 2. The heuristic is a synthesis of multiple works, including applications of the Integral Map to ecology [35], sustainability

Holistic Resilience Research Assessment-Mapping and Results
We applied the heuristic (Figure 2) to each article to identify the perspectives, arguments, research methods, and claims that align with the items listed in the experience, behavior, culture, and systems quadrants. The process of reviewing the articles and assessing the dominant perspectives of each was based on the opinion of the three co-authors. Although other readers may disagree with our assignments, the results ( Figure 3) will nevertheless reveal that intersubjectivity is underrepresented in the top 20 most highly cited resilience articles. Intersubjectivity refers to the social interior (e.g., shared values, social cohesion, and mutual understanding) among groups such as families, working teams, and organizations.

Experience
Articles assigned to the experience quadrant of the Integral Map are focused on the interior characteristics of individual people. Thus, in the experience quadrant, the holon is an individual human being. Perspectives in this quadrant align with factors such as cognitive capacity, affect, moral maturity, psychological development, and individual beliefs and attitudes. The interior factors related to human resilience, including constructs such as self-esteem, locus of control, stress response, and emotional adaptation, represent perspectives characterized by the experience quadrant of the Integral Map. Articles from two research areas were assigned to this quadrant, as shown in Figure 3. In the area of adult psychology, four articles represent perspectives associated with the experience quadrant [10,70,72,73]. These articles identify the qualitative properties of constructs such as hardiness and self-enhancement that correlate with the resilience of people faced with adversity. Likewise, three child psychology articles [74][75][76] represent perspectives of individual properties-a.k.a. variables or characteristics-related to human resilience such as self-esteem and self-efficacy. A total of seven articles were assigned to the experience quadrant.

Behavior
The articles that we assess to align with the behavior quadrant represent perspectives that emphasize physical and empirical concepts. The holon in this quadrant could be a person (if examined exclusively as an object, such as an economic agent), a technical device, or an element of the environment. Other characteristics of this quadrant include the actions, behaviors, structures, and functions of individual holons. Articles from three research areas were assigned to the behavior quadrant. First, we assessed one article in adult psychology [72] to align with both the experience and behavior quadrants of the Integral Map, as shown in Figure 3. This could include, for example, psychological characteristics corresponding to the experience quadrant and individual actions related to the behavior quadrant. Likewise, three articles in child psychology are assigned to both the experience and behavior quadrants. Luthar et al. [75] and Rutter [74,76] each describe variations of behavioral processes (mechanisms) such as age-salient tasks completion, or positive adaptation to social and environmental conditions, such as the early loss of parents. The fourth child psychology article [67] represents a perspective focused on objective properties and behavioral processes, which aligns with the behavior quadrant. A single article in the physical sciences presents a detailed study of neurological networks [77], which aligns with the behavior quadrant.

Systems
Articles assigned to the systems quadrant of the Integral Map align with perspectives related to physical and functional interactions among two or more of the holons identified in the behavior quadrant. Example perspectives include social, ecological, and technological system functions, interactions, and structures in addition to economic and geo-political systems. Perspectives in this quadrant describe the properties and processes of coupled systems that enhance their capacity to adapt to unexpected changes, which includes the dynamic relationships and interdependencies within and among systems. Eleven of the 20 articles reviewed represent the properties and processes corresponding to systems perspectives found in the systems quadrant. Here, there are four articles in the area of ecology [14][15][16][17], five articles in the area of social-ecological systems [18][19][20]78,79], and two articles in the area of technology [80,81]. Thus, in contrast to child and adult psychology, which includes articles representing both interior and exterior individual perspectives, resilience research that is focused on systems does not include perspectives corresponding to the culture quadrants among the sample of articles that were reviewed. Although limited to the 20 top cited articles, the sample demonstrates how the Integral Map may be applied to assess holism.

Culture
There were no articles assigned to the culture quadrant. Perspectives that align with this quadrant describe a group or collective interior. This refers to the intersubjective experiences of groups such as working teams and families, organizations that operate and manage infrastructure, and institutions that set policy and oversee regulations. Perspectives assigned to the culture quadrant correspond to factors such as shared values, beliefs, cultural norms, ethics, religious views, and worldviews. Together, these properties represent the collective interior that shapes intentionality and motivation, influencing collective exterior actions and behavior in the systems quadrant. However, the absence of a single article in the culture quadrant indicates that papers addressing group interior perspectives of resilience may be cited less often. Thus, although research suggest a linkage between collective interior factors influencing resilience (e.g., values, ethics, and culture) and ecological [84], social-ecological [56], and technological [85] systems, resilience perspectives related to cultural factors are not represented among the top 20 search articles. Nonetheless, factors related to a collective interior of a group, community, or urban region may inform perspectives on resilience and adaptive capacity [69,82] in response to large-scale disasters such as Hurricane Katrina or the Fukushima power plant disaster.

Discussion
The application of the Integral Map to resilience literature provides a simple tool to organize a holistic perspective across prominent research articles and inform applications such as infrastructure on how to be more holistic. For example, infrastructure subdisciplines such as socio-ecological [82] and socio-technical [86] systems include irreducible knowledge representing both subjective and objective human perspectives. However, this literature focuses on a single quadrant (i.e., "its" or systems), where a holistic approach to resilience research in infrastructure or otherwise would extend beyond a single quadrant to encompass all four. Influential articles that are not normally associated with infrastructure research but may provide relevant perspectives for individual interiors and exteriors are found in the adult psychology, neuroscience, and child psychology disciplines. The inclusion of this literature when developing resilience research programs may help overcome narrow perspectives that dominate "holistic" research and ignore the impact of human interior on infrastructure resilience. Moreover, the Integral Map shows that if infrastructure researchers seek a holistic understanding of resilience but only focus on highly cited works, they may overlook intersubjective perspectives from the culture ("we") quadrant. Without consideration of a group's interior perspectives, information that could contribute to a more comprehensive and holistic understanding of infrastructure resilience, as called for in Presidential Policy Directive 21 [31], could be missed. Examples of group interior perspectives that may contribute to infrastructure resilience include research areas such as community resilience and social vulnerability in response to natural or man-made disasters.

Community Resilience
Norris and Stevens [87] considered hope and shared subjective interpretations of health and safety as important factors that are related to community resilience in a disaster scenario. These factors can influence infrastructure resilience, because the people involved with the design, operation, and management of critical infrastructure are members of working groups, communities, and organizations. Whereas the built environment contributes to public health, safety, and well-being [2], community resilience and infrastructure are interrelated [88]. Norris et al. [71] argued that community resilience emerges from linking the adaptive capacities of community members and resources, and includes factors such as collective efficacy and empowerment. A sense of connectedness among family members, partners, and close attachment groups can also influence social responses such as positive collective action and community restoration in response to mass trauma [87]. In addition, concepts of community resilience are relevant to system shocks on both short and long-term time scales. Masten [83] and Ungar [89] suggested that understanding psychological resilience in a cultural context may help explain differences in adaptive capacity among diverse populations that could have important implications for short-term events such as disaster management and recovery. Adger et al. [56] argued that the potential limitations of a society's ability to adapt to long-term events such as climate change are due to factors related to the social interior such as ethics, beliefs, attitudes, and culture. Moreover, the limits are viewed as fundamentally subjective in nature that can change with location, time, and context. This means cultural assets may have a unique place in shaping attitudes and values inside of social systems, and may thereby influence how shocks such as climate change are experienced among diverse populations [56]. How people interpret and assign meaning to experience of disaster events will partially determine how risks and vulnerability are distributed among populations that are embedded within, and interdependent upon, critical infrastructure.

Social Vulnerability
Numerous researchers refer to the subjective properties of risks, vulnerability, and the resilience of people in relation to factors such as climate change [56,82] and disaster risk management [53,90]. Adger [56] argued that the underlying social values, ethics, and cultural interpretations about risks and vulnerability among populations impact how people respond to climate change. This means that the actions and behaviors of individuals, groups, and even whole societies are influenced by factors such as beliefs, perceptions, and shared meanings. Although the capacity to adapt to uncertain conditions is partly dependent on technological systems and human behaviors, the ethics of how vulnerable people are impacted and influenced by social structures that are responsible for decision-making sets limits on adaptation. Thus, vulnerability and adaptive capacity are determined in part by the subjective characteristics of the social systems to which they belong [56]. Additionally, Brown and Westway [82] argued that understanding how people impact and respond to environmental change requires consideration of the subjective human characteristics that influence behavior. These factors include the psychosocial properties affecting human agency, resilience, and the ability to cope with uncertainty amid disruptive change. The variation of societal factors among populations means that risks, vulnerability, and adaptive capacity are often unevenly distributed across regions and social groups. Cutter and Emrich [91] suggested that social vulnerability is based on the characteristics of the people embedded within a given region or population, which may vary according a variety of indicators such as poverty, race, and social inequality. This means that applications such as climate change and critical infrastructure resilience can be influenced by factors such as social coherence and cultural interpretations of risks and vulnerability, which can lead to different experiences by different social groups. Hurricane Katrina was a vivid example of how social vulnerability to hazards can be unevenly distributed among population groups and across spatial regions, and how inequalities can amplify impacts [91]. This was evident in that many individuals with access to resources were able to mitigate losses and evacuate the region before the hurricane hit. Others who had no option to leave and minimal resources faced dire circumstances. Many of the differences in exposure, impact, and recovery pathways were directly related to social inequalities [91], which are characterized by the culture quadrant of the Integral Map.
Cardona [53] proposed a holistic perspective of risks and vulnerability, which he defined as "internal" risk factors that are partly determined by the subjective characteristics of people and social groups. By this definition, vulnerability can vary according to the collective understanding and interpretation of risks by different people and organizations that could either enhance or diminish potential mitigation efforts and disaster management strategies. Thus, similar to perspectives described above on climate change, the capacity of a person or group to adapt to a sudden or unexpected change in their environment represents a subjective context that can influence how disaster events are experienced and managed [53]. This means that the interior characteristics of people can impact how they interact with technology, and therefore influence the resilience of coupled complex systems such as infrastructure. Finally, Cardona [53] emphasized that risk management is dependent on how risks and vulnerability are perceived and interpreted by society and by social groups. That is, how people define their worldview and make meaning of experience and environmental conditions could be an important consideration in disaster management scenarios. In addition to the above, research in other areas including psychology [83,92], ecology [35], socio-ecological systems [93], and socio-technical systems [86] provide examples of why cultural perspectives representing the collective interior of people and environments are relevant to a holistic approach to resilience research.

Conclusions
Resilience is relevant to a wide variety of applications ranging from psychological health and well-being [10] to regime shifts in ecological systems [15] and critical infrastructure operations [1]. However, the heterogeneous nature of resilience knowledge among academic disciplines means that definitions and concepts are often incongruent with one another, which can lead to partial solutions to complex resilience problems. Moreover, without a framework for integrating disparate resilience perspectives across broad disciplines, a holistic understanding of resilience within a single discipline such as infrastructure will remain elusive. Given existing trends in the organization of knowledge away from disciplinary tree structures and toward webs, new approaches are required to guide infrastructure resilience researchers toward holistic investigations.
The Integral Map is just such an approach. By organizing knowledge according to epistemological perspective rather than by discipline, the Integral Map allows holistic completion without necessitating the inclusion of everything. Application of the Integral Map to a select set of 20 highly cited resilience research articles demonstrates how it may be used to assess holism and contribute to a more holistic understanding of infrastructure resilience. Three important lessons were gained.
(1) Infrastructure-related subdisciplines largely contribute to new knowledge within the systems ("we") quadrant.
(2) Prominent research in adult psychology, child psychology, and neuroscience may support holistic infrastructure resilience research by providing interior and exterior individual perspectives.
(3) Lastly, an absence of collective interior perspectives among highly cited work may lead researchers to overlook literature that examines culture reflecting the social interior. When considering all of the perspectives in the group of articles reviewed, the results suggest that research including factors such as cultural beliefs, shared values, and ethics would contribute to a more holistic understanding of infrastructure resilience. In sum, a holistic approach to resilience research must include both objective and subjective perspectives of the properties and processes that enable people and coupled complex systems to cope with unanticipated disruptions and adapt to change.