A Meta-Level Framework for Evaluating Resilience in Net-Zero Carbon Power Systems with Extreme Weather Events in the United States

Abstract

Important changes are underway in the U.S. power industry in the way that electricity is sourced, transported, and utilized. Disruption from extreme weather events and cybersecurity events is bringing new scrutiny to power-system resilience. Recognizing the complex social and technical aspects that are involved, this article provides a meta-level framework for coherently evaluating and making decisions about power-system resilience. It does so by examining net-zero carbon strategies with quantitative, qualitative, and integrative dimensions across discrete location-specific systems and timescales. The generalizable framework is designed with a flexibility and logic that allows for refinement to accompany stakeholder review processes and highly localized decision-making. To highlight the framework’s applicability across multiple timescales, processes, and types of knowledge, power system outages are reviewed for extreme weather events, including 2021 and 2011 winter storms that impacted Texas, the 2017 Hurricane Maria that affected Puerto Rico, and a heatwave/wildfire event in California in August 2020. By design, the meta-level framework enables utility decision-makers, regulators, insurers, and communities to analyze and track levels of resilience safeguards for a given system. Future directions to advance an integrated science of resilience in net-zero power systems and the use of this framework are also discussed.

1. Introduction

Important changes are underway for the United States (U.S.) power system in terms of decarbonization, new technological options, and shifting patterns of power consumption [1]. Changes that were not anticipated 10 to 15 years ago are occurring: in the way the system is owned and operated; how its architecture functions; the manner in which generation technologies, costs, and fuel prices interact; as well as how the markets and regulatory environment have evolved [1].

Natural and anthropogenic threats to the electricity grid have also been increasing in frequency and magnitude [2,3,4]. Extreme weather events, such as heatwaves and wildfires in the western United States, hurricanes in the Gulf of Mexico and eastern coastal region, plus extreme cold throughout the United States, have contributed to increases in the frequency and duration of power outages in the United States. [3,5] Between 2001 and 2010, 65 extreme weather events occurred in the U.S. that were estimated to cost $1 billion or more in damages (CPI-adjusted [6]. By contrast, the total increased to 135 for the period between 2011 and 2020 [6]). In 2020, a new annual record was set with 22 extreme weather events that cost a billion U.S. dollars each. This shattered the previous annual record of 16 events that occurred in 2011 and 2017 [6]. More recently in February 2021, a winter storm in North America broke 2000 records for low temperatures [7].

In response to these challenges, resilience is seen as increasingly prioritized in utility plans, yet there is no standard definition for resilience with energy systems in the United States (see

Table A1. Definitions of resilience and reliability by authoritative sources.

	Resilience	Reliability
NERC	Infrastructure resilience is the ability to reduce the magnitude and/or duration of disruptive events [106].	Reliability consists of two concepts: (1) Adequacy: the ability of the electric system to supply the aggregate electrical demand and energy requirements of the end-use customers at all times, taking into account scheduled and reasonably expected unscheduled outages of system elements [107]; (2) Operating reliability or Reliable operation - Operating reliability was replaced with security in 2001 when security became synonymous with critical infrastructure protection [107,108]. - Reliable operation is operating the elements of the bulk-power system within equipment and electric system with thermal, voltage, and stability limits, so instability, uncontrolled separation, or cascading failures of such system will not occur as a result of a sudden disturbance including a cybersecurity incident, or unanticipated failure of system elements [107].
NARUC	Resilience “addresses high-impact events” that “can be geographically and temporally widespread” [109].	Reliability is about preventing disruptions that are “more common, local, and smaller” [109].
DOE	“The ability of a power system and its components to withstand and adapt to disruptions and rapidly recover from them” [37].	“… maintaining the delivery of electric power when there is routine uncertainty in operating conditions” [37].
FERC	See IEEE.
IEEE	“The ability to withstand and reduce the magnitude and/or duration of disruptive events, which includes the capability to anticipate, absorb, adapt to, and/or rapidly recover from such an event” [110].	This is the probability a system will perform its intended functions without failure, within design parameters, under specific operating conditions, and for a specific period of time [111].
Broader Definitions	Resilience is the capacity of social, economic, and environmental systems to cope with a hazardous event, trend, or disturbance, responding or reorganizing in ways that maintain their essential function, identity, and structure while also maintaining the capacity for adaptation, learning, and transformation [112]; Resilience is the ability to prepare and plan for, absorb, recover from, or more successfully adapt to actual or potential adverse events [113]; Resilience of the energy sector refers to the capacity of the energy system its components to cope with a hazardous event or trend responding in ways that maintain their essential function, identity, and structure while also maintaining the capacity for adaptation, learning, and transformation [114].

in Appendix A). For the purposes here, resilience is defined as the ability of a system to withstand or recover from low-frequency, high-impact events.

The U.S. Department of Energy recently estimated power outages are costing American businesses roughly $150 billion a year [8]. From the perspective of consumers, the Government Accounting Office estimates outage costs could total nearly $500 billion annually by the end of the century [9]. When risks from cyber, physical, and electromagnetic attacks are also factored in, the demands on electricity planning expand by orders of magnitude and complexity [10,11,12]. With such conditions, utilities may invest billions in resilience. In addition to there not being a standard definition for resilience, there is no agreed-upon approach among utilities and regulators to characterize risk in resilience planning and investment choices [13]. The unpredictability of events, the difficulty in representing impacts, and the diversity of the locational profiles add to the complexity of operationalizing system measures for resilience objectives. Combining the advancing commitments to net-zero carbon with resilience considerations and other evolving conditions translates to power-system decision-making looking quite different from just a few years ago.

Recognizing the above complexities, the aim of this article is to assist utility decision-makers, regulators, and others to more fully scope, evaluate, and monitor resilience of power systems in the context of decarbonization. The article supports analysis and decision-making by putting forward a meta-level framework for tracking and integrating assessments in a systematic, yet flexible way.

This article is structured as follows. Recent developments in the United States relating to decarbonization and resilience are reviewed, with an emphasis on power systems. Next, the way in which resilience is overseen through key U.S. energy regulation or policy, and planning is examined to highlight interjurisdictional complexity and how there is no one authority in this domain for the U.S. power systems. Common indicators that are used to firm up the operationalization of resilience are then considered. A meta-level framework is subsequently outlined, integrating qualitative, quantitative, and geospatial factors to evaluate and monitor knowledge and complexity for a system’s resilience. The framework accounts for a temporal-spatial process in decision-making and stakeholder engagement that includes the consideration of key independent variables and potential preferences or sensitivity to these. Examples of recent, extreme weather events in Texas, Puerto Rico, and California power markets are reviewed to highlight the applicability of the framework. Key considerations for further use and future questions for additional study are then discussed. It is worth noting the framework may be used for power systems with any fuel mix as well as other critical infrastructure, systems, and sectors.

2. Decarbonization in Energy Systems: Technology and Policy Conditions

In 2020, an unprecedented $500 billion was invested worldwide in low carbon assets, despite the economic disruption caused by the COVID 19 pandemic [14] Within the U.S. energy system, studies indicate deep decarbonization is feasible by or before the mid-century in power, transportation, and heating and cooling [15,16,17,18,19,20,21]. Industrial priorities in energy research and development, such as with advanced nuclear technology, present potentially pivotal ways to enable long-term net-zero carbon energy resource mixes. In the meantime, falling prices for distributed energy resources (DER), including residential solar and wind power, electric vehicles (EV), and storage can help attain near-term decarbonization [22]. Levelized energy costs for wind and solar power, for example, have decreased on average by 70% and 90%, respectively, between 2009 and 2020 [23]. Similarly, costs for electric vehicle batteries have fallen 15% annually from 2010 to 2020 with EVs projected to reach parity with internal combustion engine vehicles by 2025 [24].

In line with the above, robust new federal policies are being outlined in the U.S. to shift the economy to a 50–52% reduction of greenhouse gases from numbers in 2005 by 2030 to attain 100% no-carbon pollution electricity by 2035, electrify the federal fleet, and improve the grid [25,26,27]. To date, these policies are aspirational until enshrined in enabling legislation, such as negotiated legislation or congressional budget reconciliation, among other options. These federal measures build on state and local policies including zero-emission credits which value the low carbon baseload premium that nuclear plants bring to regional power mixes [28] and renewable portfolio standards (RPS) which specify the percentage of electricity that is supplied by renewable energy. Thirty states plus Washington, D.C. and three territories have adopted RPS policies. Another seven states plus one territory have renewable energy goals [29]. Most electric utilities have pledged to shift to zero-carbon systems by mid-century [30]. With the U.S. federal momentum in 2021 emphasizing decarbonization, utilities are asked to deliver decarbonized power earlier by 2030 or 2035 and to absorb parts of an electrified fleet amidst questions about cost and who pays.

Accompanying the above shifts, more DER that are often non-dispatchable are being added to the U.S. power system; meanwhile, some coal and nuclear power plants are being retired before their planned lifespan is reached Approximately 95 Gigawatts (GW) of coal capacity has been retired since 2011 [31]. In 2021, 9.1 GW of electricity capacity is due to be retired in the United States. Nuclear will account for 54%, and coal will account for 30% [32]. As the share of non-dispatchable energy sources, such as wind and solar photovoltaic power, increase in the power mix, fewer dispatchable and baseload technologies are available to provide system stability and inertia. There are several mechanisms to mitigate this effect through smoothing across regional balancing areas, demand side management, energy storage, smart grids, etc. Among these options, measures such as storage may represent additional costs to the system while increasing the system resilience across all energy technologies. Some may argue that integrating new technological features introduces uncertainty to systems performance, requires a different form of active control, and could increase the cyber-attack surface area.

3. The U.S. Power System and Resilience Oversight

Today’s power system in the U.S. is a complex and interdependent ecosystem consisting of more than 10,000 power plants, 642,000+ miles of high-voltage transmission lines, roughly 56,000 substations, and over 6.3 million miles of local distribution lines [3,9]. An estimated 70% of the grid’s transmission lines and power transformers are 25+ years old, 60% of circuit breakers are 30+ years old, and the average age of power plants is 30 years old [3,9,33,34]. This system varies considerably across the country as a result of factors, such as the scale of the customer base, regional demography and topography, fuel resource availability, and relations between neighboring countries or jurisdictions [1].

Broad agreement exists on the importance of protecting this system. However, electric system planners, operators, and decision-makers often need to gain fuller clarity on the scope of resilience in relation to reliability [1] (Table A1 in Appendix A). Such system considerations frequently cut across jurisdictional boundaries of regulatory and policy actors (Box 1). With ambiguous accountability for resilience and reliability plus jurisdictional overlap, power-system actors may encounter situations in which there is no clear guide or standard in place and muddle through [35].

Box 1. Oversight of U.S. grid resilience.

No single entity in the U.S. has the authority to implement a comprehensive approach to grid resilience [4]. The U.S. Department of Energy is the lead agency for federal grid resilience efforts, including conducting R&D on technology options and providing technical and related guidance to industry and stakeholders [9]. The North American Electric Reliability Corporation (NERC) is responsible for the effective and efficient reduction of risks to the reliability and security of the North American grid. NERC “develops and enforces reliability standards; annually assesses seasonal and long-term reliability; monitors the bulk power system through system awareness; and educates, trains, and certifies industry personnel…. NERC’s jurisdiction includes users, owners, and operators of the bulk power system, which serves nearly 400 million people” [36]. The Federal Energy Regulatory Commission (FERC) by contrast regulates wholesale electricity markets and interstate electricity transmission, reviews, approves grid reliability standards put forward by NERC, and issues licenses for constructing hydropower dams among its responsibilities [9].

Individual states have jurisdiction over the retail sale of electricity plus the reliability of the investor-owned utility distribution systems through standard setting/investment oversight for distribution networks [1]. The growth of distributed energy resources and controllable loads complicates the reliability of local systems with the regulatory boundaries of FERC and the states now also overlapping [1].

A National Academy report characterizes the federal-state-local scopes by indicating federal oversight and regulation of resilience pertains to analysis of resilience risk; whereas, the state and local analog relates to the authority to support utilities’ direct investment in resilience [1].

Generally, there is a shared view that reliability relates to local, more common, and smaller disruption; whereas, resilience refers to the ability to withstand or recover from a high-impact event geographically and temporally widespread [13,37]. Conventional metrics for reliability are fairly accepted within the U.S. as with System Average Interruption Duration Index (SAIDI) and System Average Interruption Frequency Index (SAIFI) (Box 2). However, they are not consistently measured. Some jurisdictions exclude unusual events, like storm-related outages, when reporting power outage statistics [38]. It has been argued “any definition or metric that is based on measuring outage frequencies, times, extents, or impacts on customers or systems does not get at the essence of resilience.…” [13].

According to the North American Electric Reliability Corporation (NERC), there is sufficient resource adequacy across most of the electric grid in North America for the next decade [39]. The changing power-system profile, however, brings new opportunities and challenges for resilience planning. Microgrids, for instance, provide critical stability and restart capabilities during disruptive events, as was seen with Hurricane Sandy [2]. The increased use of inverter-based DER, such as solar, wind, and storage, also translates to less system inertia, low-fault currents, and a reduction of other grid services currently used to provide system reliability without system design change [2,40,41]. A more integrated approach could facilitate decarbonization plus system strengthening against disruptive events.

Box 2. Reliability and resilience metrics.

• Reliability
  • Transmission [2,42,43]:
    • Loss of Load Expectation (LOLE) calculates the amount of capacity that needs to be installed to meet the desired reliability target;
    • Loss of Load Probability (LOLP) measures the probability that a system’s load will exceed the generation and firm power contracts available to meet that load;
    • N-1 indicates a system is able to withstand at all times an unexpected failure or outage of a single system component (i.e., a single contingency situation) such as the failure of a transformer or a lightning strike that causes a transmission line outage.
  • Distribution [44,45,46]:
    • System Average Interruption Duration Index (SAIDI) is the system-wide total number of minutes per year of sustained outage per customer served;
    • System Average Interruption Frequency Index (SAIFI) measures how often the system-wide average customer was interrupted in the reporting year;
    • Customer Average Interruption Duration Index (CAIDI) tracks the total duration of an interruption for the average customer during a given time period;
    • Momentary Average Interruption Frequency Index (MAIFI) is the number of momentary outages per customer system-wide per year;
    • Average Service Availability Index (ASAI), or the service reliability index, is the ratio of the total number of customer hours that service was available during a given time period to the total customer hours demanded.
• Resilience [13,47]:
  • • Interruption Costs compare the cost of kilowatts (kW) during business as usual versus when kWs are not delivered;
    • Total Resources Costs value proposed utility investment in energy efficiency;
    • System Hardening Costs represent the costs for strengthening a system with redundancies, additional layers, or alternative configurations;
    • Social Costs assess customer benefits and related community benefits, such as ecological impacts, jobs, and/or health effects.

4. Key Contemporary Approaches for Evaluating Resilience

Resilience planning requires an understanding of critical assets within the context of their broader systems including the people and capabilities to carry out the core functions. Such planning also requires recognition of interdependencies, such as those between the power system, transport, natural gas heating, and water. If fuel station pumps and distribution systems are limited by a power outage, for instance, cascading impacts can occur across multiple systems. Electricity, like communications, provides an enabling function for other critical infrastructure [48].

With resilience planning and investment, there is a need to robustly evaluate and plan in ways that support risk characterization, allow tradeoffs to be identified and weighed, highlight impact sensitivities of different stakeholders, and inform investment decisions. Locational priorities, capabilities, resources, and conditions can frame very distinct resilience decisions, so utilities need the flexibility of an approach that can be applied across diverse service areas. Such versatility may lose robustness in terms of locationally specific depth or relevance of assumptions and priorities. Here, the key will be for the analysts and decision-makers to tailor the specifics of these dimensions.

Approaches that aim to represent resilience in power systems may include:

• Interruption costs—typically contrast the cost of kilowatts (kW) during standard use versus an outlier event when kWs cannot be delivered. These may be represented as estimates of cost per interruption event, per average kW, and per unserved kWh, as well as the total cost of sustained electric power interruptions. Real value is difficult to calculate in advance and is unlikely to represent all the benefits [13];
• Total resource costs—are used in regulatory proceedings to value proposed utility energy efficiency investment. If all customer and related community benefits were covered, this could be a reasonable gauge for resilience [13,47];
• Social cost—could substitute for the total resource costs indicated above by including customer benefits [13] and related community benefits. Used in some regulatory proceedings, this indicator may entail dimensions like ecological impacts, jobs, and/or health effects that are quantified yet have inherently qualitative aspects. Since social cost may be seen by some as outside the scope of utility responsibility, it is not used universally and may be reflected in related analysis, such as environmental impact assessments. Importantly, locationally based priorities can vary, so a utility with different regional service areas may have distinctly different social costs, even if the total resource costs are the same;
• Costs of system hardening—represent the costs for strengthening a system with redundancies, additional layers, or alternative configurations. Power systems may employ measures that: put electric distribution systems underground; place switchyards above floodplains; utilize gravity-fed rather than pumped potable water supplies; provide freeze protection for natural gas supply systems; etc.

Following the Fukushima Daiichi accident in 2011, the U.S. nuclear industry and Nuclear Regulatory Commission considered, for example, system hardening which: included revised evaluation approaches for seismic and flooding events; included new equipment to more effectively handle potential reactor core damage; and centered on strengthening emergency preparedness capabilities [49]. Specific adaptations are evident now, for instance, in greater use of portable onsite equipment and regional emergency response capabilities.

The above approaches reflect ways to quantify resilience, yet other aspects of decision-making require judgment that goes beyond simple quantitative metrics. For instance, good placement is a strategic capability that does not lend itself to quantitative metrics Tradeoffs about locational aspects can have critical implications for emergency response or sensitive groups that are not typically well-reflected in quantitative metrics. (Common practices for fortifying the power system include: standards for distribution lines and structures; redundancy; segmentation; tree resistant conductors/vegetation management; fiberglass cross-arms; flood protection; substation firewalls; recovery back-up equipment, etc. [2]. These can be uniformly monetized, but their locational strength may vary by placement/geography).

5. Meta-Level Framework for Analysis and Decision-Making

A meta-level framework is put forward for assessing the resilience of locationally specific power systems that are transitioning with low carbon aims. It serves as a logical structure for conceptualizing complex layers of information relating to resilience by defining the cumulative knowledge and gaps of understanding for a given system’s resilience (A framework is “a logical structure for classifying and organizing complex information” [50,51]. It allows for additional practices and tools to be incorporated and provides the process necessary for evaluation. By contrast, a methodology is “a documented approach for performing activities in a coherent, consistent, accountable, and repeatable manner” [51]. A methodology may be less flexible and is based on core principles. Both assessment constructs apply to what is proposed in this paper. To account for the rigor and flexibility for varied regional contexts, the term ‘framework’ is used).

As with other areas of basic and applied science that bridge multiple scientific disciplines and, in some cases, sectors [52,53,54,55,56], this framework recognizes methods that integrate capabilities and knowledge from different domains. It is done to support decision-making and tracking of a complex system in a rational way, in certain respects like multi-objective analysis or fuzzy logic approaches with flexibility that can be applied with artificial intelligence [57,58,59,60,61,62,63].

The framework is based on design principles of built systems. These include principles such as: resilience transcends scale; diverse and redundant systems are inherently more resilient; resilience anticipates interruption and a dynamic future; community contributes to resilience; and resilience is not absolute [64]. For power systems, coverage includes the baseload versus non-dispatchable balance, supply security of fuel, system agility, weatherization, cyber-attack surface, etc. and will formally characterize the types and tiers of specialized and integrated knowledge from diverse perspectives. It builds on an approach to assess the current level of understanding of nuclear fuels and materials for use in nuclear environments [65]. Similar to the technology readiness levels used by the U.S. Department of Defense, this framework provides a shorthand logic and formal means for evaluating a system’s resilience with local specificity. In conjunction with the assessment, bi-directional stakeholder engagement is envisioned throughout the process informing the assessment tiers and being informed by them.

As shown in Figure 1, the framework represents an evaluation process that increases the specialization and integration of knowledge of system resilience while accounting for energy system shifts from the current, business as usual status to increasingly more aggressive carbon mitigation actions in 2030, 2040, and 2050.

With this framework, the tiers of specialized knowledge and integration of system resilience dimensions are reflected by the following symbolic representations:

^ZU_Y is where the value U denotes the tier of resilience characterization (A to C):

○

Tier C defines the initial order and foundation, based on a qualitative review of best practices, plus expert and stakeholder elicitation, as appropriate;

○

Tier B incorporates the knowledge gained from Tier C into assumptions and refined options for quantitative analysis;

○

Tier A represents fuller integrated analysis with more specialized focus on local considerations;

○

“Superscript” Z parameter is employed to describe resiliency dimensions;

○

“Subscript” Y denotes the time step (year) that the hypothetical system is deployed (e.g., C₃₀ is for 2030, C₄₀ is for 2040, C₅₀ is for 2050).

All studies begin with (¹) the technology or pathway assessment with additional assessments, namely (²) economic, (³) social/institutional, (⁴) ecological, and (⁵) infrastructural) following, as an example ¹B₃₀ (Technology) or ²B₃₀ (Economic) or ³B₃₀ (Social/Institutional) or ⁴B₃₀ (Ecological) or ⁵B₃₀ (Infrastructural). Technology refers, here, to a singular option or mix of options. It can include a potential pathway, such as with demand response, that involves coordinated change in practices at specific times, which requires advanced management techniques. For a given process, the technology pathways would be outlined in the supporting material. More than one resiliency dimension may be covered at a given point in the process, and could be denoted as ^3–4B₃₀, for instance, for (³) social/institutional and (⁴) ecological dimensions being addressed.

Multiple paths can be taken to advance the analysis through qualitative, quantitative, and integrated assessment tiers (from broad to highly specialized and locationally specific detail). The framework accounts for energy systems with increasing shares of low carbon energy technologies (Current Baseline → 2030 → 2040 → 2050) with the goal of advancing the complex characterization of the resilience properties of future energy systems as they transition toward low or zero carbon by 2050.

As indicated in the paragraph above, the level of resilience characterization is increased through a progression of three tiers of analysis, starting from C to A:

○

(C) Qualitative Analysis—This tier is generally the starting point in the analysis and includes a review of standards and practices accounting for current and anticipated regulations, industry-community standards, and expert input from relevant fields (e.g., low carbon energy technologies, energy system dependencies, economic, social/institutional, ecology, etc.). It is based on general social and market conditions for a region and defines low-high importance plus sensitivity to resilience, etc.

Qualitative analysis (e.g., ¹C₃₀) includes preliminary prioritization and exclusions accounting for technical expertise and general community preferences and known regulatory constraints. Preliminary dependencies/interdependencies are identified with other sectors or systems, including supply chain constraints based on competing demand across power and gas markets. The balance of baseload versus non-dispatchable power is represented here.

Scenarios of primary interest are also formulated here. These should account for desired paths and conditions against which to protect. Rather than a rote process, this is highly strategic in terms of preliminary goals and threat characterizations (more coverage of scenarios may be found below). Evaluative methods may include case analysis, stakeholder engagement, Delphi ranking/matrix scoring, expert elicitation, interviews, surveys, and analytic hierarchy process, among options. Preliminary filtering eliminates early no-go options.

○

(B) Quantitative—The second tier entails quantitative analysis including modeling of non-location specific profile markets. Profile markets exhibit the “market attributes” characteristic of markets and energy systems behaviors in transition. They help to define deployment boundary conditions, differentiate the importance of energy system attributes, and can reflect the needs of underrepresented markets that are economically and/or socially marginalized. Study of the profile markets offers lessons and provides a basis for energy system deployment in various markets, domestic and foreign. Scenarios of primary interest are refined based on iterative analysis.

This level of analysis (e.g., ¹B₃₀) incorporates insights gained from Tier C then characterizes system performance numerically. It may be used to estimate/test sample projects’ effectiveness or to compare levels across different systems. Scoring reflects the variance in magnitude and duration from a target level [2]. Analysis may include approaches that are performance-based, event-specific, and accounting for uncertainty to inform decision-making.

○

(A) Integrated Analysis—The third tier, covering integrated analysis, includes more nuanced and increasingly specialized assessments of a specific location that incorporate and build on results from Tiers C and B. It includes advanced quantitative, qualitative, and geospatial assessments that mutually inform. The synthesis of multiple methods is completed in other domains [66,67,68]. If done well, it allows for the strengths of the different methods to complement and/or amplify the value of the process and findings. Scenarios of primary interest are refined, here, based on iterative analysis.

Deeper quantitative modeling is completed for specific locationally based markets (e.g., centralized energy hub, remote location, and islands). The tier of analysis (e.g., ¹A₃₀) could be carried out with a study that evaluates the dynamic effects from high-penetration rates of individual technologies in an integrated energy system consisting of a combination of energy sources and infrastructure conditions. Examples of modeling tools include MARKAL and the International Atomic Energy Agency’s (IAEA) Wien Automatic System Planning (WASP) as well as other modeling tools. Various shocks to the energy system can be used to perturbate the modeling to study the range of outcomes. These shocks may include technical equipment failure, weather-related risks, volatility energy prices, interruption of a major resource, or attacks on key energy infrastructure [69]. Modeling for uncertainty can be useful in understanding sensitivities of variables to system behavior, such as increases in the probabilities of disturbances occurring over time (e.g., from severe weather- and climate-related events).

Deeper qualitative analysis and geospatial analysis are completed for the specific locationally based region of study. Coverage includes a more comprehensive review of capabilities such as expertise and preparedness (i.e., the existence of emergency plans, personnel training, repair crew availability, and other similar measures) to mitigate, respond, and recover while also accounting for geospatial particularities. An example of a geospatial consideration would be the siting of backup power for a power plant at a higher elevation and/or separate location to minimize risk from a disturbance, such as with flooding. Additional considerations in this tier would include more comprehensive review of dependencies/interdependencies associated with paths that are deemed more favorable in Tier B and C and in the corresponding quantitative analysis of the current tier. Other conditions to evaluate include the sufficiency and geospatial access of capabilities for the period of study under different constraints. The degree of appropriate weatherization for the region in another assessment condition. As the number and types of extreme weather events occur with greater frequency, planning that assumes probability of 100-year high-impact events should regularly be reevaluated. Recent trends may not be sufficiently representative.

In addition to the above factors, jurisdictions, and layered forms of decision-making should be considered, such as with determinations from a public meeting, an advisory board, or regulatory authority. Outcomes of the framework’s earlier tiered findings may guide advanced deliberations which may in turn inform the later analysis. Ownership and operational oversight should also be factored if the resilience evaluation aims to account for system agility in technical and organizational terms. Resilience during a disturbance may hinge on clear channels of command and rapid response.

Integrated analysis will ultimately entail prioritizing/ranking and filtering for no-go options based on more specialized and cross-referenced findings from technical experts, stakeholder/advisor input, model-qualitative-geospatial informed characterization in other parts of the framework, and the concurrent tier of the evaluative process.

When combined, the three-tiered analysis formulates an integrated approach with a progression that accounts for increased specialization, place-based precision, and complexity in relation to a given location. The analysis accounts for the system configuration, networks, and necessary expertise/capabilities to manage critical assets, essential services, inextricable links/interconnectedness, interdependencies, and potential for cascade effects or multiple events. This approach identifies critical assets and their functional relationships within systems. It prioritizes the consideration of physical and cyber threats, vulnerabilities, and consequences.

Use of this framework is designed to be done in conjunction with an ongoing process of resilience monitoring that aims to ask the right questions while using the appropriate data to answer the questions and interpreting the answer for optimal applicability. This recognizes situational awareness of the known unknowns and ongoing vigilance with respect to the unknown unknowns should be part of the process (see Scenarios below).

5.1. Step-by-Step Review of the Framework

To study the resilience of future energy systems, a range of decarbonization energy strategies are developed that describe the energy transition from the current system to future configurations in 2030, 2040, and 2050. The strategies outline changes to the energy-technology mix, system resources (e.g., energy storage), policy-regulatory environment, and market and community priorities. Strategies are then assessed with the framework. As an example, the level of specialized knowledge on resilience performance as the energy system transitions to low carbon by 2050 could be described by the path:

[^1–5C₂₁ → ^1–2C₃₀ → ¹B₃₀ → ¹B₄₀ → ¹A₄₀ → ^1–5A₅₀]

(1)

○

The starting point, ¹C₂₁, is based on prevailing practices with the current technologies in the energy mix with technical, economic, social, ecological, and infrastructural assessments to provide a full baseline producing ^1–5C₂₁;

○

The next step, ^1–5C₂₁ → ¹C₃₀, continues with best practices/expert assessment/general geospatial profiling to evaluate resilience with an emergent low carbon energy system in 2030. It may for a variety of reasons only initially cover technical and economic dimensions producing ^1–2C₃₀;

○

The following step, ^1–2C₃₀ → ¹B₃₀, adds quantitative analysis of a profile market considering scenarios up to 2030;

○

The next step ¹B₃₀ → ¹B₄₀ continues the quantitative analysis up to 2040;

○

The fourth step ¹B₄₀ → ^1–5A₅₀, develops into an integrated analysis (including both quantitative, qualitative, and geospatial elements) of a proposed system’s resilience in a specific region and market in 2050. It covers technical, economic, social, ecological, and infrastructural assessments in Tier A to provide a full profile.

Ideally, all dimensions are covered for each tier and time period. However, that does not always occur in analysis and planning. This framework shows where the fuller assessment takes place and, importantly, accounts for areas where gaps exist.

In practice, an expert assessment (¹C₂₁) could begin by comparing technology options using specific intrinsic technology measures, as illustrated in

Table 1. Intrinsic measures of energy technologies.

Intrinsic Measures	Open Cycle Turbines	Wind/Solar	Nuclear Large Plants	Nuclear SMRs
Maintenance Requirements	Moderate	Low	High	Moderate
Island-Mode Operation	High	High	Low	High
Geographic Dispersion	Moderate	High	Low	Moderate
Modular Structure	Moderate	High	Low	High
Real-Time Responsiveness	High	Low	Low	Moderate
Ramping Capabilities	High	Low	Low	Moderate
Capacity Factors and Duration	Moderate	Low	High	High
Need for Refueling with Regular Use	High	Low	Low	Moderate

. These measures are qualitative that is they describe certain resilience characteristics of the technology. Analysis at this level allows us to understand a technology’s resilience baseline. As an example of real-time responsiveness, open cycle turbines (OCT) are rated high, nuclear is moderate to high, and wind/solar are rated low; it is visa-versa with fuel security for wind/solar is high, whereas OCT is low. This suggests the system resilience is improved through an offsetting mix of variable renewables, OCT, and nuclear technologies. Intrinsic measures become more pronounced when stresses are applied by increasing levels of penetration. For example, at high-penetration rates (e.g., 50–75%), the OCT Fuel Security measure of “Low” suggests a system would become less resilient due to a greater likelihood of a fuel shortage.

Using the above framework, the characterization and local knowledge accumulates as analysis progresses from C to B to A. The collected knowledge includes qualitative, quantitative, and integrated location-specific results over the study period. The framework can be used to identify potential analysis gaps, such as when quantitative modeling is performed (Tier B) but requires a specialist’s review (Tier C) to define the system uncertainties associated with novel technology systems or configurations.

A more holistic characterization of resilience using the framework should factor for the following resiliency dimensions under dynamic conditions and across multiple timeframes:

• Technical Resilience—The technical resilience of an energy system focuses on the potential disruption to the hardware and software plus energy/power inputs and outputs. In the case of a power system example, this accounts for factors including generation, transmission, and distribution by supplementing traditionally static system performance measures to factor behaviors under changing contexts. It simulates complex interactions incorporating additional resilience dimensions detailed below. This would principally be evaluated in quantitative and geospatial assessments of Tiers A and B and factor for conditions like weatherization;
• Economic Recovery Resilience—The economic recovery resilience of an energy system focuses on the potential disruption to the area economy and its capacity to recover. This may be measured in terms of the impacts on varied sectoral areas of production, supply, demand, and delivery as well as employment and post-disruption recovery efforts. It defines the minimum level of recovery investments required to restore production and delivery levels so that total economic impacts are deemed acceptable over a stipulated post-disruption duration [70]. This would be primarily evaluated in Tiers A, B, and C across all methods;
• Social and Institutional Resilience—The social and institutional resilience of the energy system focuses on the disruption to society, its capacity and social ecosystem, as well as its ability to mobilize to recover from a shock. This encompasses people, plus organizations, rules, and resources. It accounts for regulated versus deregulated markets. Readiness and adaptive capacity are key, including the community’s ability to learn, problem-solve, self-organize, and govern with institutions that can partner and adjust. This would be primarily evaluated in Tiers A and C and would generally be qualitative in form;
• Ecological Resilience—The ecological or environmental resilience of an energy system centers on the natural system and its ability to recover to a former or new steady state. The concept of adaptive capacity that is indicated with social and institutional resilience would apply here as well. This may encompass water, air/emissions, land/soil, forests/agriculture/biodiversity, etc. This area of focus would be primarily evaluated in Tiers A, B, and C with all methods;
• Infrastructural Resilience—The infrastructural resilience of an energy system refers to the built environment that goes beyond what is covered by technical resilience. In the case of a power-system resilience study, this would encompass other critical infrastructure such as communications and transportation systems as well as gasoline fueling stations—all of which typically require power to function. This would be primarily evaluated in Tiers A, B, and C across all methods.

5.2. Scenarios

Robust scenario characterization is essential in resilience scoping and risk evaluation for the framework. A ‘design basis event’ approach, for example, as is done by the nuclear industry, entails the identifying postulated events that set performance requirements for the technology, components, and system. Such scenario planning should include regular updates and a practice of revising, whenever a critical development occurs, or new condition is recognized. This could include changes to infrastructure or situational awareness and shifts in decision-making, jurisdiction, or operational control, among factors. The Fukushima Daiichi accident in Japan, for example, highlighted the importance of system hardening for multiple extreme weather events, such as a dual earthquake-tsunami occurrence, as well as for multiple reactor units being impacted concurrently. Design basis requirements for U.S. nuclear plants were updated to reflect insights gained from this event [71].

The process should also account for decision-making on acceptability boundaries and tier ranking. This framework can define specific threshold conditions linked to lower or higher observable levels of resilience.

Key scenarios for consideration include extreme weather events, such as severe cold or heat, wildfires, and hurricanes as well as cyber and physical attacks. Additional design elements include: sufficient interconnection (with other sources); flexibility of heterogeneous (diverse) systems with differing components; hierarchical embedding; reliable shutdown/startup cycles; supply chain reshoring; backup components, monitoring and warning systems for detecting precursors; minimum sync-to-grid capacity and load following; distributed, equipment design modification; and flexible generations limits on demand for social stability. Best practices also suggest analysis accounts for the strengths and weaknesses of standardization, stability, simplicity, accessibility, reproducibility, preventative measures, and resiliency to natural events.

As the share of non-dispatchable energy increases in the energy mix, the impacts on resilience will need to be better understood. For example, with the retirement of baseload generators, replacement generation can be scaled; so the new generation unit may be equal to or less than the retired generation capacity, where the load may be covered, but the resilience of the system may be altered. Using the Meta-Level Framework described above, strategies can account for improved technical resilience (e.g., hardening the technology, increasing operational flexibility, and adding system interconnects) while also evaluating the economic and societal tradeoffs for the given system conditions (e.g., buried power lines may protect from hurricanes, but not floods). Energy planners can benefit from understanding the resilience attributes of low carbon technologies and the balancing measures that may be needed to maintain the stability as the shares are increased. Simultaneously achieving low carbon goals while increasing system resilience becomes a balancing act requiring the valuation of technological, economic, social, ecological, and infrastructural attributes. Utilization of the Meta-Level Framework provides a key step towards better characterization and tracking of resilience complexities in future low carbon energy systems. The ultimate objective of this approach is to develop insight into the resilience of long-term strategies and the tradeoffs needed (e.g., how much high-carbon activities should continue versus. be replaced by low carbon technologies and how scalable are the strategies).

5.3. Examples of Extreme Weather-Power Outage Events

A number of power-system disruptions are detailed to highlight resilience challenges. Forward-looking applicability of the framework is also discussed.

5.3.1. Winter Storms: Texas (ERCOT) in 2021 and 2011

On February 2021, a winter storm in North America (referenced earlier for breaking 2000 low-temperature records) [7] provides insight into resilience planning for extreme winter cold. With this event, more than 4.5 million customers were reported as being without power, and 133 deaths have been linked to date [72,73]. In the Electric Reliability Council of Texas (ERCOT) market during the period from 14–19 February 2021, sub-zero temperatures were associated with increased power demand which exceeded the power supply. Nearly half the region’s power generation went offline with the highest amount of unavailable capacity during the period equaling 51.2 GW of the 105.7 GW installed capacity on 16 February 2021 [74] (Figure 2). Thermal capacity, consisting of natural gas, coal, and nuclear power generation, reflected the largest share of unavailable power with the majority sourced from natural gas. Renewable generation from wind and solar power also went offline. Dangerously low frequencies were observed at 59.3 hertz (Hz) for 4 min and 23 s and could have left Texas with a multi-week black start event (A safe system frequency level should be around 60 hertz. If the frequency is less than 59.4 hertz for 9+ min, generation will start tripping offline, and there is a risk of the whole system blacking out [75]). Prior to shedding load, energy prices equaled or surpassed a systemwide offer cap of $9000/megawatt hours (MWh) (compared to a more standard price of $22/MWh) [76]. Without sufficient electricity or heat, water pipes froze and ruptured adding to widespread disruption.

In addition to unavailable generation and a surge in power demand, insufficient weatherization was the other initially stated reason for outages [77]. Fuel supplies, namely natural gas and coal were not able to be delivered, and equipment/instruments froze. The ERCOT system’s winter peak of 66 GW had roughly 30 MW of thermal plants offline [77]. Scenario planning for a worst-case winter example anticipated 14 GW of thermal power being unavailable, but more than double was the case in February 2021 [77]. ERCOT had factored for variable renewable electricity (VRE) outages in its planning, as VREs function differently in the system [78]. Scenario planning was based on a winter storm in Texas in 2011.

The referenced 2011 storm affected 4.4 million electric customers in the southwestern U.S. including customers in Texas that experienced below freezing temperatures and unexpected rolling blackouts [78,79]. Over a million Texas customers lost service, some for extended periods with 193 generators going offline in ERCOT due to ice loads and complications of cold temperatures, including frozen coal piles at generating stations [75,79]. At the storm’s peak, 14.7 GW of generation was offline (lowest frequency was 59.58 hertz) [75]. Analysis by FERC and NERC highlighted the importance of weatherization and accounting for interdependencies of electricity and natural gas systems. Specific to reserves, the report indicated:

…the massive amount of generator failures that were experienced raises the question whether it would have been helpful to increase reserve levels going into the event. This action would have brought more units online earlier, might have prevented some of the freezing problems the generators experienced, and could have exposed operational problems in time to implement corrections before the units were needed to meet customer demand.

[78]

After the 2011 outages, guidelines were put forward on weatherization. An aspirational reserve margin was left in place, and there was no regulator-enforced mandate in order to maintain low-consumer prices [75]. Primary points of recommended change across the 2011 and 2021 storm outage include weatherization and adequate interconnections with neighboring markets to import electricity and a capacity market.

Table 2. Comparison of ERCOT-related storms [72,75].

Indicators	2011	2021
Generators offline	193	356
Duration of outage	7.5 h	70+ h
Lowest frequency	59.58 Hz	59.3 Hz
Maximum load shed	4000 MW	20,000 MW
Generation unavailable	14,702 MW	51,173 MW
Customers offline	1,000,000+	4,500,000+

highlights the differences between the two storms with ERCOT’s power system.

The above ERCOT winter storm events provide a basis for considering how the Meta-Level Framework could be applied. Drawing from the insights of the February 2021 storm, ERCOT should undertake a deep and holistic review of its system resilience for 2021 as new baseline. Given the scale of damages and number of people who were impacted, integrated resilience analysis should firm up a systematic understanding of the current base year (C₂₁ → B₂₁ → A₂₁) before expanding future assessments. A deliberative process that includes customer and expert input is advisable throughout the assessment. This includes choices about interconnection and weatherization in the infrastructural assessments (⁵C₂₁) and decisions about whether the deregulated ERCOT market structure should be revised (³C₂₁). Local preferences/sensitivities could be identified to narrow the range of acceptable options:

[^1–5C₂₁ →^1–5B₂₁ →^1–5A₂₁] [^1–5C₃₀ → ¹B₃₀ → ¹B₄₀ → ^1–5A₄₀ → ^1–5A₅₀]

(2)

Tier B should then assess a profile market with grid and fuel distribution dynamics that include scenarios with considerable fuel supply disruptions and variations in weatherization. The presence and absence of a capacity market can be factored. Tier A should then incorporate the findings from the prior two tiers and extend them with interdependency analysis, a more in-depth localized focus, and other forms of integration. Interdependency analysis should assess the level of reliance that water, heating, telecommunications, and other critical infrastructures have on power and vice versa.

A review of ERCOT’s Report on Existing and Potential Electric System’s Constraints and Needs from December 2020 indicated that high-industrial load growth, thermal plant retirements, existing constraints, and the increase in wind, solar, and combined cycle generation were key points of recent scrutiny [80]. Winter storms do not appear to have received recent attention. (Additional review of ERCOT planning meeting documents on the grid operator’s website shows resilience-relevant groups, such as the Grid Resilience Working Group, focusing on “risks that have a low probability of occurrence but potential high consequence of impact to the ERCOT System if they were to occur … [and] considering and evaluating practices that may address these risks” [81]. Inspection of the group’s existing and planned coverage indicates “understanding design events; mitigation practices; cost recovery mechanisms; interaction with other critical infrastructure; recovery from conditions like hurricanes; High impact/Low Frequency Events, Black Start and the Integrated Nature of the Power System, Testing and Hardening Techniques, plus electromagnetic pulses” [81,82]. Recognizing the natural limits of using meeting agendas for content analysis, the predominant focus of GRWG meetings appears to have been electromagnetic pulses). In five scenarios of varying resource mixes and variable renewable energy penetrations through to 2030, the analysis found system stresses have changed from historical instances of summer afternoons to alternate times of day and year. The report also notes the importance of evaluating system conditions other than peak load, indicating three of the top 10 constraints on the ERCOT system in 2020 were outages tied to hurricane storms [80]. Based on the experience of two extreme winter storms in 10 years, scenarios need to more fully account for such weather stresses, and infrastructure needs to account for fuller weatherization.

Especially for customers who may have questions about decision-making and are not privy to the more confidential aspects of planning, the Meta-Level Framework may be used as a tool to coordinate and summarize where the process is currently headed. Likewise, it can represent the kinds of analysis and complex local knowledge that is evaluated. It is not a replacement for tough choices, but it can reflect gaps and more systematically classify what may be siloed areas of focus.

5.3.2. Hurricane Maria: Puerto Rico in 2017

In September of 2017, Hurricane Maria struck the Commonwealth of Puerto Rico (PR) at a Category 5 strength [83]. Dumping more than 30 inches of rain on the island, the storm destroyed the island’s power infrastructure with 100% of customers losing service [83,84]. In what has been deemed the worst blackout in U.S. history (and the world’s second largest blackout), 3.4 billion customer-hours were lost in electricity service with more than 3000 deaths attributed to the lack of electricity and basic services [84]. In response efforts, the Puerto Rico Electric Power Authority (PREPA) partnered with the U.S. Department of Energy, Department of Defense, and Federal Emergency Management Agency. Potable water, electricity, and cell phone services were restored many months later [84].

Focusing on building and strengthening a new power system (the PREPA power plants are 28 years older and experience outage rates 12 times higher than the U.S. average [85]), PREPA’s 2019 Integrated Resource Plan (IRP) recommended segmenting the Puerto Rican grid into mini-grids (hundreds of MW) and microgrids (1 to 20 MW) with more storage to increase system reliability and resilience [86,87]. In 2020, a preliminary feasibility study was also completed to evaluate the potential for small modular reactors (SMRs) and microreactors [88]. Although nuclear plants were not previously considered for IRP analysis of the PR generating resource mix, the siting and operation of advanced nuclear plants, particularly under the auspices of regional or localized grid support and services, were evaluated to address aspects of resource adequacy, availability, and system performance. The study identified grid resilience benefits with advanced nuclear generation including integration with renewables. Reporting indicated that if the grid infrastructure is available, SMRs and microreactors could operate within a mini-grid or microgrid. However, the technology would need to be scaled appropriately (i.e., on mini-grids); the reactors should be limited to a capacity of 100 MW or smaller as well as designing operational strategies for flexibly bringing online and offline individual SMR units or modules (such as 40–50 MW modules). Reporting also found many SMRs (up to 700 MW) may be too large for microgrids [88].

The scope of the above, Nuclear Alternative Project (NAP) study was much broader than energy system resilience; however, it included grid resilience as an objective and contained several elements consistent with the Meta-Level Framework described in this paper. The study was designed to increase the level of understanding of the energy system in Puerto Rico, based on current standards and best practices, expert assessment, and thorough modeling of future energy scenarios, including nuclear plants. It described Puerto Rico’s current legal and regulatory framework which places a significant weight on concerns over climate change [89] and technology-neutral policy. The study also outlined best practices for designing resilient energy systems that resist hurricane exposure, evaluating finance options, performing public surveys on nuclear energy, and for conducting outreach and educational engagements.

Consistent with the Meta-Level Framework Tiers B and C, several scenarios for future generation were analyzed including an energy system modernization scenario that included natural gas, solar, and energy storage based on the Puerto Rico Integrated Resource Plan. The NAP evaluated additional scenarios including SMRs and microreactors. In one scenario, a 600 MW SMR was used to replace the retiring 602 MW heavy fuel oil plant at Palo Seco [88]. Scenarios required meeting the renewable portfolio standard goal of achieving 40% of electricity from renewables by 2025, 60% by 2040, and 100% by 2050 [88]. Quantitative analysis was performed using a socio-economic model to evaluate severe weather and seismic resiliency of advanced nuclear reactors and the economic impact on communities with technical details of SMR resiliency characteristics based on NuScale studies [90].

The analysis determined that proposed energy mixes for Puerto Rico would need to be evaluated and judged as an integrated system of how the proposed energy mix impacts the greatest number of needs across society. The exercise illustrated the importance of evaluating energy projects for the island from the perspective of the project contribution to broader overall priorities (e.g., infrastructure resiliency, economy, ecological, and energy). The NAP recognized a systematic approach is needed to evaluate public opinion of advanced reactors. As a result, the NAP recommended a detailed dynamic stability study to evaluate the suitability of SMRs for mini-grid operation and, particularly, the effect of SMR capacity on mini-grid stability with mini-grid stability criteria. Further, mini-grid stability criteria may need to be established and stability studies performed [88]. The future level of analysis described by the NAP resonates with aspects of the “Tier A” described in this paper.

An illustrative characterization of the PR assessment could reflect:

[^1–5C₁₉ →^1–5B₂₀ →^1,3,5A₂₀] [^1–5C₂₅ → ^1–3B₂₅→ ¹B₄₀ → ¹A₄₀ → ¹A₅₀]

(3)

When communicating with government regulators, insurers, communities, and members of industry or making investment choices, the above representation provides a discernable snapshot of the focus (knowledge types and level of local specialization), temporal dimensions, and by extension areas to be evaluated.

5.3.3. Heatwave/Wildfire: California in 2020

In mid-August 2020, an intense and prolonged heat wave impacted the western United States with temperatures 15–30 °F above normal, resulting in demand exceeding electricity resource adequacy and planning targets [91,92]. During this extreme weather event, ten Western Interconnection balancing authorities declared energy emergencies, including the California Independent System Operator (CAISO), which ordered the first rolling outages in 20 years [92,93,94,95] (The Western Interconnection is a wide area synchronous grid plus a major alternating current grid in the United States, spanning from western Canada to Baja California, Mexico to the Great Plains. In normal operations, power utilities within this Interconnection are electrically joined together, operating at an average, synchronized frequency of 60 Hz [96]).

The high and widespread electricity demand across the western United States limited CAISO’s ability to import from neighboring areas as their balancing authorities served native loads [94,95]. The supply was also below preseason forecasts for nearly all resource types, including natural gas, hydropower, wind, and solar [96]. The high heat reduced thermal generation, as thermal facilities do not tend to function as efficiently in extreme temperatures [94]. Below average hydropower availability and diminished solar generation due to wildfire smoke and cloud cover were also among the confounding factors affecting operations [94]. CAISO’s controlled load shedding of roughly 1800 MW was done 14–15 August 2020. Rotating outages lasted roughly 8–150 min and impacted approximately 800,000 customers served by the utilities responding to the CAISO directive [92,94].

Prior to the above event, NERC evaluations indicated issues with energy sufficiency and flexibility for the region. NERC’s 2019 Summer Reliability Assessment stated, “Extreme outages may result in insufficient resources at peak load” [97]. In its 2020 Summer Reliability Assessment for the region, NERC’s high-risk scenario predicted, “Operating mitigations and EEAs [Energy Emergency Alerts] may be needed under extreme demand and extreme resource derated conditions” [98].

Post-event analysis indicated California had a number of pre-existing conditions that required address: (1) lack of clear accountability for having the resources to power the grid (similar to the Texas conditions); (2) lack of resources, such as gas-fired plants, pumped hydro or battery storage, hydropower, or demand side management, to balance solar and wind power; (3) closure of disfavored resources before new ones are brought online, such as with battery storage replacements for natural gas plants; and (4) siloed operations [99]. To compound the conditions further, California has had 13 of its 20 most destructive wildfires, and seven of its 20 deadliest ones since 2017 [100,101,102]. Investor-owned utilities in the state are projected to spend more than $21.7 billion through 2022 on wildfire mitigation plans alone [102].

In preparation for the 2021 summer season and in response to a California Public Utility Directive, California utilities procured additional generating capacity [97,103]. Most additions are solar photovoltaic generation [95]. Additional resources in the form of storage are also being integrated, with roughly 600 MW planned to be on-line by summer and an additional 800 MW planned by 1 August 2021 [95,104]. NERC’s 2021 Summer Reliability Assessment for the region indicated Western Electricity Coordinating Council (WECC) risk scenarios identifying the continued risk of energy shortfalls for the WECC California-Mexico region on the order of 10,180 MWh and “the potential for above-normal peak demand and resource outage scenarios, similar to those seen in 2020, to result in operating emergencies in all WECC assessment areas with the exception of the winter-peaking Canadian province” [95].

Against the above backdrop, California has robust decarbonization aims including a mandate for 60% of the state’s energy to be sourced from renewable energy by 2030 and 100% of its energy to be carbon emissions free by 2045 [105]. These conditions provide useful points of reference to illustrate the framework more fully.

Assigning the timeframes of interest to be 2025, 2030, and 2045, with 2021 as the base year in Figure 3, an assessment is proposed for 2021 with a hypothetical investor-owned utility in California. A full evaluation of all resiliency parameters is desirable for 2021 and the subsequent milestone years. However, circumstances do not always allow such coverage.

Table 3. Sample tracking in more depth—Framework sections.

Studies	Resilience Parameter(s)	Research Highlights
Generator readiness	1C21	Qualitative and mechanistic understanding of resilience behavior and weatherization readiness as a function of ranked sensitivity and duration of generator outages
Regional priorities and constraints	3–4C21	Qualitative assessment of local preferences, ecological stewardship objectives and market capabilities
Cascading Failures	1–5B21 → 1–3A21	Integrated modeling of the effect of the power grid’s structural failure
Environmental Analysis (air quality)	4A25	Resiliency impacts to air/emissions associated with long-term changes to ambient temperatures by region
Critical Infrastructure (communications)	5A25	Integrated assessment of location and sensitivity levels of communication outages relative to stress points

Notes: Resiliency Dimensions: ^1. Technical; ^2. Economic Recovery; ^3. Social and Institutional; ^4. Ecological; ^5. Infrastructural. Tiers: A. Integrated, including Geospatial studies; B. Quantitative, and C. Qualitative.

details a potential evaluation strategy with sample qualitative, quantitative, and integrative areas. It is advisable to complete these framework studies with a review process that may bear similarity to what is used for integrated resource planning, but which prioritizes resilience and involves actors like local emergency planners, members of critical lifeline areas/functions, participants from dependent sectors, etc. By planning the analyses and scenarios, completing and refining the analyses and scenarios as more information becomes available, and tracking progress of resilience studies using the framework, the utility can evaluate resilience more systematically and convey progress or gaps.

6. Limits and Advantages of the Framework

In terms of limits to the framework, it identifies gaps in the research but does not explicitly identify or rank high priority research areas. However, by applying the framework methodically, the user can gain insights to the key resilience vulnerabilities in the system and the sensitivity of variables. Advantages of the framework are that it more systematically accounts for complex and varied dimensions, timescales, and geographic units of analysis with a flexible approach that can be used in conjunction with decision-making and stakeholder review processes.

7. Discussion and Conclusions

The importance of advanced resilience analysis is becoming increasingly evident as energy systems transition with low carbon aims while experiencing increased stress from severe weather and other disruptors. System vulnerabilities and uncertainties need to be assessed through a structured analytical framework that conveys levels of integration and local specificity as well as temporal aspects to understand how to operationalize and gauge resilience. Holistically understanding resilience can be a major step towards creating an adaptive system that can minimize or eliminate the most harmful impacts from future disruptions.

The outlined Meta-Level Framework aims to expand the capacity to study, communicate, and advance understanding of resilience in conjunction with ongoing vigilance toward resilience risks. Use of the Meta-Level Framework can improve understanding of the fuller nature of resilience in future low carbon energy systems by defining the:

• Role of resilience in relation to reliability;
• Standardized logic for communicating the depth of knowledge in terms of analytical rigor and dimensions of time and place (location specific);
• Gaps in understanding a system’s resilience;
• Critical assets within the context of their broader systems including the people and capabilities to carry out the core functions;
• Sensitivities of variables to system behavior;
• Dimensionality and interplay between technical, economic, social/institutional, ecological, and infrastructural resilience;
• Resilience qualities of low carbon technologies and the necessary balancing measures to maintain the stability with increasing shares;
• Early-stage strategies (e.g., scaling technologies) and their impact on achieving long-term objectives should be factored;
• Value of flexible energy technologies in the energy mix;
• Conditions under which analysis should be updated to address design basis changes, shifts in decision-making, jurisdiction, and operational control.

This paper described aspects of the framework’s applicability to a number of system disruptions using examples from Texas winter storms, Puerto Rico following Hurricane Maria, and California following a heatwave-wildfire event. These studies provided insights into lessons learned from system failures that can be used to inform researchers in the design of future systems with more robust mitigation strategies.

The framework was designed to allow flexible application across different scales with users equipped to build in uniquely localized considerations. Several uses include:

• Utilities communicating with regulators and insurance companies (and vice versa) about the current resilience posture and the risks and opportunities moving forward to 2050 as they decarbonize their energy systems;
• National energy analysts creating the analytical basis for informed decision-making based on a comprehensive understanding of the impacts over a range of mitigation options for location-specific energy systems;
• Researchers evaluating profile markets to understand and design resilient systems.

It is important to recognize electric utilities’ behavior toward resilience can be largely determined by regulatory requirements and market design. These may be evaluated under the localized focus of institutional considerations within the various tiers in the Meta-Level Framework. An important step in the research and analysis will be perfecting ways to more fully represent the weighting of complex determinants and framing the resilience decision-making. Another important takeaway is that the regulatory playing field could be improved with greater clarity on how resilience is operationalized versus reliability. Moreover, there could be a one-stop authority rather than many. Additionally, who pays for the improvements? Strengthening resilience in some respects is a form of insurance. Utility owners in private sector expect some form of financial remuneration for attained resilience.

As the framework is tested with advancing assessments in varied markets, geographies, and community profiles, questions for additional study and consideration should also include: how could more dimensionality be introduced while maintaining the formal logic of the framework; how could the framework be most effectively used in tandem with public/stakeholder planning; and how could resilience risks, such as those associated with cyber and physical attacks, be sufficiently operationalized in the analysis.

There is a need for versatility, continuing situational awareness, and some simplicity of the approach, as regulators, such as FERC, NERC, and public utility commissioners; as well as utilities and energy planners weigh tough choices about resilience and decarbonization priorities. Costs do not capture the value of complex tradeoffs. Likewise, dependencies and place-based limits require sufficient attention. The opportunities and challenges at the nexus of energy system resilience and decarbonization can become even more amplified when seen in light of cross-cutting implications in other critical systems, such as water, agriculture, and transport. Whether focusing on the resilience of energy in relation to all strategic national assets or strictly within its system, the value of an agile tool for more complex location-specific understanding cannot be overstated.