Estimated Time to Restoration of Hurricane Sandy in a Future Climate

: Power outage restoration following extreme storms is a complicated process that couples engineering processes and human decisions. Emergency managers typically rely on past experiences and have limited access to computer simulations to aid in decision-making. Climate scientists predict that although hurricane frequency may decrease, the intensity of storms may increase. Increased damage from hurricanes will result in new restoration challenges that emergency managers may not have experience solving. Our study uses agent-based modeling (ABM) to determine how restoration might have been impacted for 30 di ﬀ erent scenarios of Hurricane Sandy for a climate in 2112 (Sandy2112). These Sandy2112 scenarios were obtained from a previous study that modeled how outages from Hurricane Sandy in 2012 might have been a ﬀ ected in the future as climate change intensiﬁed both wind and precipitation hazards. As the number of outages increases, so does the expected estimated time to restoration for each storm. The impact of increasing crews is also studied to determine the relationship between the number of crews and outage durations (or restoration curves). Both the number of outages and the number of crews impact the variability in time to restoration. Our results can help emergency managers and policy makers plan for future hurricanes that are likely to become stronger and more impactful to critical infrastructure. This study utilizes the outage predictions from Wanik et al.’s 2018 study. Wanik used six numerical weather simulations from Weather Research and Forecasting model [11] based on di ﬀ erent convective parameterization schemes (Goddard, Morris, NOTCFLX, WDM6, CNTRL, and ENS) and three machine-learning algorithms (random forest, gradient boosted trees, and Bayesian additive regression trees) to predict the number of outages per 2 km grid across the state of Connecticut for one utility company. Each town is comprised of multiple 2-km grid cells, so outages were aggregated for each town and the total outages are randomly placed in each town. Two sets of input variables were used to develop the Sandy2112 scenarios: One with full weather variables including wind and precipitation and a second using only the wind variables. The precipitation increases were substantial in the Sandy2112 predictions. In case the precipitation variable was overwhelming the machine learning prediction due to the extreme values, the outage prediction models were run again without the precipitation variables. Because there were di ﬀ erences in the number of outages predicted, both sets were used as inputs for the ABM to represent multiple extreme event outage scenarios. It is unknown which, if any, Sandy2112 scenario would happen. Using the range of predictions gives emergency managers a range of scenarios to be prepared for because each scenario would require di ﬀ erent resources. In 2012, there were 16,460 outages for this utility company. Although the OPM results include the CNTRL weather simulations, CNTRL has been excluded as an input for the ABM, reducing the six weather simulations down to ﬁve. With ﬁve WRF simulation models, three machine-learning algorithms, and two sets of input variables, 30 di ﬀ erent Sandy2112 scenarios were used, exhibiting total outages ranging from 13,372 to 34,630. Using three di ﬀ erent crew search strategies, 90 di ﬀ erent ABM scenarios were run. Using the ABM with 90 di ﬀ erent parameter combinations is similar to experiencing 90 di ﬀ erent storms. Each simulation will have di ﬀ erent results, all which can be used to build knowledge on the impacts storm size and crew allocation have on the restoration time. The direct input have the separate search strategies that were tested in the ABM. The crew counts in this section were “ideal”, as compared to a true restoration where crew access is much more limited. Reintroducing the variation in crew counts creates more noise in the correlation between ETR and number of outages, which better represents the variability found in historic restorations. This study one speciﬁc crew increase strategy, but the model can be used to run many di ﬀ erent strategies and reallocations of crews. The addition of an agent-based model to weather forecasts and power outage prediction models allow for a complete simulation of extreme storms and restoration all before the storm makes landfall. This complete modeling system can be beneﬁcial as storm intensities exceed the experience of emergency managers. Storm restoration is a complex system, but added resources such as a coupled system of outage prediction and agent-based models can better prepare decision makers for a changing climate.


Introduction
Utility companies generally rely on emergency managers' experience when making decisions for storm restoration. Although this is helpful when new storms are similar to those of the past, this practice can be limiting when managers are faced with storms of new magnitudes. For example, in 2005, Mississippi Power, a utility company in Mississippi, was faced with extensive damages following Hurricane Katrina. During preparation for the storm, Mississippi Power received the lessons learned from Gulf Power during the Hurricane Ivan restoration from 2004. However, damage assessment conducted by Mississippi Power following Hurricane Katrina showed that the actual damage to the system exceeded even the "worst-case scenario" of damage similar to Hurricane Camille from 1969. Mississippi Power was tasked with restoring the grid with damages that exceeded their experience and was beyond what they were prepared for. In total, the restoration process for Mississippi Power took 12 days [1]. However, this example is similar to restorations experienced by utility companies all over the world and raises a few questions: How can emergency managers prepare for restorations that exceed their experience? Could a computer simulation, along with utilities' past crew allocations and preparedness levels provide estimates for the restoration?

Impact of Climate Change on Storms
Many climate scientists have been studying the impacts that climate change can have on extreme storms. Several major themes include the amount of precipitation carried by each storm, the paths that storms will travel, and the frequency these storms will occur. Climate change can have impacts on both the precipitation levels of extreme storms, and the paths that extreme storms take. Lackmann [4] discusses how changes in weather patterns could change the path of storms. In the case of Hurricane Sandy, climate warming could increase the strength of the westerly jet stream and push the storm eastward-possibly completely missing the East Coast [4]. Increased temperatures will increase the vapor capacity of storms, which will lead to increased potential precipitation [5]. Studies predict a decrease in hurricane frequency [6,7], but an increase in the rainfall per storm [8]. Bender et al. [6] predicts an increase in Category 4 and 5 hurricanes, but an overall decrease in hurricane activity. If these predictions stand true, emergency managers will be faced with new restoration challenges as storm intensities increase.

Resiliency
Utility companies are tasked with returning the grid to full function as soon as possible following extreme storms. Resiliency can have many definitions based on the context it is in. In terms of the electric grid, resiliency can be described as the ability of a system to recover from a disrupted state. In this paper, resiliency will mean the ability of the grid to be returned to full function, or no outages remaining. Henry and Ramirez-Marquez [3] state "quick and effective restoration following the damage is key to resilience." The intensity of extreme storms is expected to increase [6], which will lead to more widespread damage requiring extensive repairs. Although some utilities may take action towards strengthening their grid to prevent some damages, not all weather-related damages can be avoided. From 1984From -2006.2% of outage events were caused by hurricanes and tropical storms. Although the total percentage was low, the mean number of customers affected was largest at 782,695 compared to all other causes and mean size in megawatts was second highest to earthquakes at 1309 MW [9]. Enhanced tree trimming is one method taken to decrease the severity of damages caused to the electric grid [10], but this action cannot avoid all tree-related outages during storms. Increasing the computer Sustainability 2020, 12, 6502 3 of 27 simulation tools available to emergency managers as they prepare for storms may be another way to increase the resiliency of systems.
Henry and Ramirez-Marquez [3] present a framework of the system during the resilience (or restoration) process. This framework is broken into five zones, each relating to different times during the storm. Zone 0 is before the storm is predicted; the system is operating as usual, no storm is on the horizon, and this is where utilities typically begin implementing grid-strengthening mechanisms (such as replacing lines, trimming trees, etc.). Zone 1 is as the storm is predicted; the system is still operating as usual, but there is a storm predicted where the arrival time is well understood. Zone 1 is where short-term planning begins-the storm has yet to hit but some preparations and planning can be done. Zone 2 begins as the storm arrives and continues until recovery action can begin. Safety regulations prevent crews from working while winds exceed 30 mph and oftentimes roads are impassible due to downed trees. Zone 3 includes the restoration and continues until the system is back to its recovered state. The recovered state is at least the same as before the disruption occurred. Zone 4 is the time for utilities to reflect on the restoration, implement lessons learned, and make long-term improvements [3].
Computer models and simulations provide key insights as storms are forecasted. Power outage estimation models exist for tropical cyclones and hurricanes [11][12][13][14], thunderstorms [15], and various weather and vegetation inputs [16][17][18]. Han et al. [12] recognized that accurate storm estimates are necessary to have the proper crews in place to make restoration as efficient as possible. However, for the vast amount of outage prediction modeling, models pertaining to the power outage restoration process are limited. Previous models have taken a stochastic approach to model power outages, determine a pre-hurricane crew allocation, and use updated damage assessment to revise the crew mobilization plan [19,20]. A mathematical model was developed to optimally locate repair vehicles and crews [21]. Yao and Min state that the high correlation between extreme weather conditions and power failures along with reliable weather forecasts make the demands for repair crews more predictable than emergency response of firetrucks and ambulance [21]. The cost of the restoration was a driving factor in acquiring additional crews [21]. Other restoration models consider the stockpile locations of replacement parts to be used in the restoration process [22]. Brown et al. [23] developed a Monte Carlo Simulation that was useful in determining system sensitivity to design improvements but could not be used in a predictive manner.
The process of power outage restoration is a complex system that couples both human decisions and engineering processes. Restoration also has many variables and moving parts, which makes modeling the process difficult. Although mathematical models such as [19][20][21][22][23] can be used to model power outage restoration, they are limited in their ability to capture the human decisions. This study presents the use of an agent-based model (ABM) to study the impact human decisions have on storm-induced power outages. Those human decisions include how crews are dispatched, the number of crews, and where those crews are positioned. Unlike previous studies [23], the ABM can be used in a predictive manner using outage estimations and expected crew resources to estimate the restoration time as a storm is forecasted. This case study uses 30 scenarios of Hurricane Sandy in a climate-enhanced year 2112. The geospatial features of the ABM allow for outages to be placed on a town-level scale and limit utility crews to the same work areas as they would during a real storm restoration. Additionally, the ABM can be used as a planning tool. Emergency managers can move or add crews in the simulation as outage predictions increase. As emergency managers are tasked with determining the need of Mutual Assistance Agreements [24], the impact of additional crews and their arrival times can be tested. The use of an ABM over a mathematical model allows for the storm to be modeled in a virtual environment that mirrors the expected damages. Emergency managers can then test their strategies and resources to determine their optimal restoration plan. The ABM gives the expected restoration time for the State of Connecticut as a whole, but still allows for the specific assignment of crews to smaller areas. This sub grouping allows emergency managers to make the same small-scale decisions as they would in practice while still getting the same large-scale output. Our novel approach to power outage restoration modeling makes further use of outage prediction models and provides emergency managers with another tool to investigate a storm while it is still in the forecasting stage or after the restoration to test whether other methods could have improved the restoration time. In this study, the use of 30 different outage prediction scenarios presents a range of damage, and corresponding restoration time, that could be expected in a climate-enhanced future.
Agent-based models (ABM) have been shown to predict the overall restoration time of storm-induced power outages for the electric utility grid in the State of Connecticut [25]. Unlike previous models, the ABM can be used as a predictive tool to give emergency managers a complete picture of a storm as the weather system begins developing in the forecast. The ABM presented in [25] can be utilized, as shown in this study, to aid emergency managers in Resiliency Zones 1-3. As the storm is predicted in Zone 1, the weather forecasts can be utilized to predict the expected outages [11,15,17,18,26], which can then allow emergency managers to begin understanding the necessary resources. As the storm is better understood in Zones 2 and 3, the ABM can be run with the updated outage predictions to see how the predicted restoration time would change with available resources. The case study presented here investigates using a baseline of Hurricane Sandy from 2012 to estimate the restoration times of Hurricane Sandy under a changing climate in the year 2100 utilizing the same resources as Hurricane Sandy in 2012. Although the data from this storm happen to be from the next century, it should not be used as any confidence that this storm would even occur in 2100. Our purpose in this paper is to leverage future climate data to motivate better preparedness during extreme events. In this example, Hurricane Sandy caused significant damage, so revisiting the storm in a future climate regime may help utilities appreciate what a new reality may look like. Previous studies have been conducted analyzing the weather patterns and damage caused by Hurricane Sandy, which makes it an ideal storm to study the restoration process. The study proposed here is intended to be an extension of previous work conducted by Wanik et al. [11,18,26] to use past databases of outages and weather to build decision-support tools for utilities. The outage prediction model answers the question of how much the damage could be and the agent-based model investigates the question of how long the restoration could be. To our knowledge, no other studies have directly used power outage predictions to estimate the time to restoration based on predicted outages, expected crews, and with different crew work methods.

Case Study on Power Outage Impacts from Future Hurricane Sandy Scenarios
Wanik et al. [11] studied how climate change could impact the power outages associated with Hurricane Sandy in a future climate using the IPCC AR4 A2 emissions scenario to track outage changes in the year 2112 (100 years after Hurricane Sandy made landfall). The study used six different Weather Research and Forecasting (WRF) model simulations: Control, Goddard scheme, Morris scheme, WRF double-moment 6-class microphysics scheme, "no TC flux", and the ensemble for weather inputs (hereafter referred to as CNTRL, Goddard, Morris, WDM6, NOTCFLX, ENS). Additionally, three machine-learning algorithms were used to predict the outages: Random forest, gradient boosted trees, and Bayesian additive regression trees. Although this is only one of many outage prediction models [11][12][13][14][15][16][17][18], the results from Wanik et al. [11] will be used as the inputs for the study presented here.
This study is a continuation and utilization of the information learned from Wanik et al. [11]. Using direct inputs from an outage prediction model to investigate impacts on the estimated restoration time makes this study a novel approach. The previous research was conducted to answer the question of how much damage could occur from a storm similar to Hurricane Sandy in a future climate. The study presented here uses the estimated damage, measured in outages, to determine the new expected restoration times. We expect to see an increase in restoration time when the number of outages increase but the crews are held constant. Therefore, the estimated restoration times are then used to determine an increase in crew counts per area work center in order to reduce the estimated restoration time back to the historic restoration. In this study, we are evaluating the restoration times of the different Future Sandy scenarios presented in Wanik et al. [11]. Having a better understanding of the changes of the predicted outages and their locations, utilities can test how long the restoration may take using a baseline scenario of past storms. Walsh et al. [25] presented an agent-based model (ABM) to estimate the time to restoration of storm induced power outages. In the study presented

Case Study on Power Outage Impacts from Future Hurricane Sandy Scenarios
Wanik et al. [11] studied how climate change could impact the power outages associated with Hurricane Sandy in a future climate using the IPCC AR4 A2 emissions scenario to track outage changes in the year 2112 (100 years after Hurricane Sandy made landfall). The study used six different Weather Research and Forecasting (WRF) model simulations: Control, Goddard scheme, Morris scheme, WRF double-moment 6-class microphysics scheme, "no TC flux", and the ensemble for weather inputs (hereafter referred to as CNTRL, Goddard, Morris, WDM6, NOTCFLX, ENS). Additionally, three machine-learning algorithms were used to predict the outages: Random forest, gradient boosted trees, and Bayesian additive regression trees. Although this is only one of many outage prediction models [11][12][13][14][15][16][17][18], the results from Wanik et al. [11] will be used as the inputs for the study presented here.
This study is a continuation and utilization of the information learned from Wanik et al. [11]. Using direct inputs from an outage prediction model to investigate impacts on the estimated restoration time makes this study a novel approach. The previous research was conducted to answer the question of how much damage could occur from a storm similar to Hurricane Sandy in a future climate. The study presented here uses the estimated damage, measured in outages, to determine the new expected restoration times. We expect to see an increase in restoration time when the number of outages increase but the crews are held constant. Therefore, the estimated restoration times are then used to determine an increase in crew counts per area work center in order to reduce the estimated restoration time back to the historic restoration. In this study, we are evaluating the restoration times of the different Future Sandy scenarios presented in Wanik et al. [11]. Having a better understanding of the changes of the predicted outages and their locations, utilities can test how long the restoration may take using a baseline scenario of past storms. Walsh et al. [25] presented an agent-based model (ABM) to estimate the time to restoration of storm induced power outages. In the study presented here, the ABM is calibrated for Hurricane Sandy under 2012 conditions (hereafter referred to as Sandy2012) and then coupled with Wanik et al.'s predicted power outage locations for 2112 scenarios (hereafter referred to as Sandy2112). Information extracted from this study can improve utility resiliency following extreme storms by allowing emergency managers to test past methods and resources for current storms as the storm is predicted and then determine whether additional support or different allocations would be needed. We hypothesize that (1) an increase in the number of outages will lead to a linear increase in estimated time to restoration (ETR) when the crew counts are held constant, (2) with adequate crew increases we expect to be able to reduce the ETR back down to the historic levels, and (3) the large number of outages will make restoration strategies more salient compared to smaller storms. Figure 2 highlights the workflow of this study. Section 2 focuses on utilizing Wanik et al.'s [11] Sandy2112 predicted power outage locations with Sandy2012 crew counts to estimate the total restoration time. Section 3 proposes a method to increase the crew counts to study the changes in the total restoration time for each Sandy2112 scenario. This crew increase method is intended to simulate emergency managers using the ABM to determine resources needed to reach a goal restoration time. Section 4 presents other metrics captured by the ABM that can add insight for emergency managers during restoration. Sections 2-4 show how the ABM can use information from any outage prediction model to simulate the full restoration process and assessment to determine whether allocated resources would be adequate. here, the ABM is calibrated for Hurricane Sandy under 2012 conditions (hereafter referred to as Sandy2012) and then coupled with Wanik et al.'s predicted power outage locations for 2112 scenarios (hereafter referred to as Sandy2112). Information extracted from this study can improve utility resiliency following extreme storms by allowing emergency managers to test past methods and resources for current storms as the storm is predicted and then determine whether additional support or different allocations would be needed. We hypothesize that (1) an increase in the number of outages will lead to a linear increase in estimated time to restoration (ETR) when the crew counts are held constant, (2) with adequate crew increases we expect to be able to reduce the ETR back down to the historic levels, and (3) the large number of outages will make restoration strategies more salient compared to smaller storms. Figure 2 highlights the workflow of this study. Section 2 focuses on utilizing Wanik et al.'s [11] Sandy2112 predicted power outage locations with Sandy2012 crew counts to estimate the total restoration time. Section 3 proposes a method to increase the crew counts to study the changes in the total restoration time for each Sandy2112 scenario. This crew increase method is intended to simulate emergency managers using the ABM to determine resources needed to reach a goal restoration time. Section 4 presents other metrics captured by the ABM that can add insight for emergency managers during restoration. Sections 2-4 show how the ABM can use information from any outage prediction model to simulate the full restoration process and assessment to determine whether allocated resources would be adequate.  Wanik et al. (2018) and is used as an input for this study. Section 2 uses the outage predictions and applies the agent-based model (ABM) to predict the estimated restoration time of each Sandy2112 scenario. Section 3 uses the output of the ABM from Section 2 to determine the rate at which to increase the crew counts. Section 3 then reruns the ABM with the same outages to determine new (reduced) estimated restoration times. Section 4 uses the results of Sections 2 and 3 to compare other metrics such as outage durations and the time to reach target restoration percentiles.

Methods
This study utilized the ABM detailed in Walsh et al. [25] to derive restoration scenarios for the different Sandy2112 outage predictions presented in Wanik et al. [11]. Several adaptations have been added to the ABM to better represent the restoration process. First, the work zones for crews was updated to a higher spatial resolution from state-wide to area work center (AWC). Crews are assigned an AWC during the model setup process. Crews stay in that AWC until all of the outages there have been repaired. Once their AWC is complete, if any outages remain, crews will then be assigned a new AWC based on the search strategy used. For example, if the strategy is set to nearest  Wanik et al. (2018) and is used as an input for this study. Section 2 uses the outage predictions and applies the agent-based model (ABM) to predict the estimated restoration time of each Sandy2112 scenario. Section 3 uses the output of the ABM from Section 2 to determine the rate at which to increase the crew counts. Section 3 then reruns the ABM with the same outages to determine new (reduced) estimated restoration times. Section 4 uses the results of Sections 2 and 3 to compare other metrics such as outage durations and the time to reach target restoration percentiles.

Methods
This study utilized the ABM detailed in Walsh et al. [25] to derive restoration scenarios for the different Sandy2112 outage predictions presented in Wanik et al. [11]. Several adaptations have been added to the ABM to better represent the restoration process. First, the work zones for crews was updated to a higher spatial resolution from state-wide to area work center (AWC). Crews are assigned an AWC during the model setup process. Crews stay in that AWC until all of the outages there have been repaired. Once their AWC is complete, if any outages remain, crews will then be assigned a new AWC based on the search strategy used. For example, if the strategy is set to nearest outage, a crew in a completed AWC will find the nearest outage in a different AWC and then set that as their AWC to complete restoration. Crews will remain in the newly assigned AWC and not "return" to their original AWC until statewide restoration is completed. Daily crew counts by AWC were provided from a local utility company that has been used as the input for this study, as previously detailed in Table 1. The crew counts were from the actual crews allocated for the restoration of Sandy2012-caused outages in Connecticut but have been kept the same for the initial test of Sandy2112 scenarios. Furthermore, about 75% of crews are simulated to be working during the daytime hours and 25% of crews are working during the overnight hours.
The ABM was calibrated based on utility data from Sandy2012 to get a baseline for Sandy2112 experiments. The outage locations, crews working, and historic restoration curve were used. The individual repair time for each outage location was unknown. A series of repair time ranges from a minimum of one hour to a defined maximum was tested and compared to the historic restoration curve of Sandy2012. It was found that a range of repair times of one to eight hours in 15-minute increments with uniform distribution best fit Sandy2012. We have applied that same range to Sandy2112; outages are randomly assigned a repair time between one and eight hours during the model setup process. Repairs for different infrastructure require different time allocations. For example, on average, it takes 2 h to fix a wire, 4 h to fix a transformer, and 8 h to fix a downed pole. Applying a range of repair times implicitly assigns different fault types to the system.
During calibration, we used four different search strategies for outage assignment to crews: Nearest outage, most customers affected, most customers affected within a radius of the nearest outage (referred to as nearest within radius), and fastest repair time within radius of the nearest outage (referred to as fastest within radius). The same methods could not be applied directly to Sandy2112 scenarios because there are no data on predicted number of customers affected. The number of customers affected per outage depends on the infrastructure and the population density. The same infrastructure could have more customers affected in a city location compared to a rural location. Therefore, in this study, we are limited to running strategies that only include location or repair time. The use of different search strategies is what separates the ABM from other linear models. The ABM allows input of spatial data and models individual crew behavior, which adds complexity that linear models cannot capture. The Sandy2012 calibration showed that the shape of the restoration curve between strategies was different, but the overall estimated restoration time was not significantly different, as shown in Figure 3.
To account for all possible restoration curve scenarios, Sandy2112 simulations will be run with three search strategies: Nearest outage, fastest repair time, and fastest-within-radius. Search strategies can also be a way to optimize the restoration, which is another novel approach in modeling storm restoration. The restoration curves in Figure 3 highlight how different strategies reduce the customers without power faster than others. In this scenario, although the final restoration time for all four strategies are close, the number of customers without power varies significantly at different time steps. However, in practice, safety is a limitation of this. Especially in the beginning of the storm, there are some outages that must be addressed before others due to safety concerns. Modeling different search strategies allows emergency managers to investigate different approaches to storm restoration. The ABM allows managers to see the direct impact their decisions can have on both the restoration curve and the overall restoration time.
One point to highlight regarding the historic restoration curve (black line in Figure 3) is the periodic increases in customers remaining without power. Throughout the restoration, locations with power are sometimes shut off in order to make a safe repair. Once the power is restored, there is a greater decrease in customers without power-the repair returned all those who temporarily lost power along with those still waiting for restoration. The restoration curves of the ABM are smooth because the simulation does not account for more outages, whether intentional or unintentional, except for at the start.
Another way to compare the results of the ABM to the historic restoration is to look at the repair rates. Compared to Table 3, the repair rates for the historic restoration of Sandy2012, the ABM repair rates for replicated Sandy2012 are higher, as seen in Table 4. Part of this could be the repair times assigned to the outages. Although the location of each outage is known, the repair time of each outage is unknown, but the ABM assigns a repair time in model setup using uniform distribution. Therefore, the ABM may have more outages with faster repair times than in the historic restoration. The nearest outage strategies are consistent with the historic restoration, but the fastest repair time strategies result in higher rates early in restoration and lower rates later in restoration. The daily average across all strategies results in higher rates early and lower rates after day two or three. Additionally, the ABM gives a "best case" scenario. Crews are assumed to have all materials, knowledge, and skills to make a repair. Each outage is assumed to be ready for crews to begin work as soon as they arrive. Therefore, there is no lost time to acquire additional materials, change crew assignments based on fault type, or wait for a tree crew to clear the area. These three factors would also reduce the repair rate by introducing less efficiency. Although having more differentiation between crew types, materials on hand, and road clearing would make the model more realistic, it would also make the model much more complex. ABMs are designed to be a bottom-up approach [27]. The developer continues to add complexity without over constraining the model. The goal of an ABM is to keep it as simple as possible to investigate the impacts individual parameters can have. Results from [25] showed that the ABM was able to recreate the validation storms without the added levels of agent-type, and that simplification was carried over to this model version. One point to highlight regarding the historic restoration curve (black line in Figure 3) is the periodic increases in customers remaining without power. Throughout the restoration, locations with power are sometimes shut off in order to make a safe repair. Once the power is restored, there is a greater decrease in customers without power-the repair returned all those who temporarily lost power along with those still waiting for restoration. The restoration curves of the ABM are smooth because the simulation does not account for more outages, whether intentional or unintentional, except for at the start.
Another way to compare the results of the ABM to the historic restoration is to look at the repair rates. Compared to Table 3, the repair rates for the historic restoration of Sandy2012, the ABM repair rates for replicated Sandy2012 are higher, as seen in Table 4. Part of this could be the repair times assigned to the outages. Although the location of each outage is known, the repair time of each outage is unknown, but the ABM assigns a repair time in model setup using uniform distribution. Therefore, the ABM may have more outages with faster repair times than in the historic restoration. The nearest outage strategies are consistent with the historic restoration, but the fastest repair time strategies result in higher rates early in restoration and lower rates later in restoration. The daily average across all strategies results in higher rates early and lower rates after day two or three. Additionally, the ABM gives a "best case" scenario. Crews are assumed to have all materials, knowledge, and skills to make a repair. Each outage is assumed to be ready for crews to begin work as soon as they arrive. Therefore, there is no lost time to acquire additional materials, change crew assignments based on fault type, or wait for a tree crew to clear the area. These three factors would also reduce the repair rate by introducing less efficiency. Although having more differentiation between crew types, materials on hand, and road clearing would make the model more realistic, it would also make the model much more complex. ABMs are designed to be a bottom-up approach [27]. The developer continues to add complexity without over constraining the model. The goal of an ABM is to keep it as simple as possible to investigate the impacts individual parameters can have. Results from [25] [11] based on different convective parameterization schemes (Goddard, Morris, NOTCFLX, WDM6, CNTRL, and ENS) and three machine-learning algorithms (random forest, gradient boosted trees, and Bayesian additive regression trees) to predict the number of outages per 2 km grid across the state of Connecticut for one utility company. Each town is comprised of multiple 2-km grid cells, so outages were aggregated for each town and the total outages are randomly placed in each town. Two sets of input variables were used to develop the Sandy2112 scenarios: One with full weather variables including wind and precipitation and a second using only the wind variables. The precipitation increases were substantial in the Sandy2112 predictions. In case the precipitation variable was overwhelming the machine learning prediction due to the extreme values, the outage prediction models were run again without the precipitation variables. Because there were differences in the number of outages predicted, both sets were used as inputs for the ABM to represent multiple extreme event outage scenarios. It is unknown which, if any, Sandy2112 scenario would happen. Using the range of predictions gives emergency managers a range of scenarios to be prepared for because each scenario would require different resources. In 2012, there were 16,460 outages for this utility company. Although the OPM results include the CNTRL weather simulations, CNTRL has been excluded as an input for the ABM, reducing the six weather simulations down to five. With five WRF simulation models, three machine-learning algorithms, and two sets of input variables, 30 different Sandy2112 scenarios were used, exhibiting total outages ranging from 13,372 to 34,630. Using three different crew search strategies, 90 different ABM scenarios were run. Using the ABM with 90 different parameter combinations is similar to experiencing 90 different storms. Each simulation will have different results, all which can be used to build knowledge on the impacts storm size and crew allocation have on the restoration time. The direct input of outage prediction model results is a novel approach that allows emergency managers to test different decisions and investigate the impacts those decisions have on the restoration.
Wanik et al. [11] notes that each Sandy2112 scenario resulted in different storm paths and different magnitudes of weather variables. The different storm tracks are accounted for in the ABM by the number of outages predicted per town. The outage prediction model works at a more granular scale (2-km grids), but it is assumed that the most directly hit areas will experience more outages. Although the ABM assigns outages on a town level and crews on an area work center level, the final output is the statewide ETR. One key finding from Wanik et al. [11] was that most Sandy2112 scenarios showed an increase in the number of outages, but there were four storm/machine learning model combinations that had a decrease in the number of outages predicted: (a) Goddard scheme with boosted trees and full weather variables, (b) Goddard scheme with boosted trees and wind variables only, (c) Goddard scheme with Bayesian additive regression trees and wind variables only, and (d) Morris scheme with boosted additive regression trees and full weather variables. These four scenarios with reduced outages show that changes in weather in the future may change the path of the storm or the storm intensity, resulting in less outages in the study area. Wanik found that random forest had the highest change in outages, followed by Bayesian additive regression trees and then by boosted trees. It is expected that the models with the most outages will have the longest ETRs and the least outages will have the shortest ETRs. Again, the use of the 30 different scenarios is intended to develop a range of estimated time to restoration based on the range of predicted outages from the different models used.

Results
In this part of the study, we explore how the changes in the number of outages will change the overall restoration time for the storm. The resources were held constant across the ABM simulations-the number of crews working does not increase. As expected, the estimated time to restoration increased with an increase in the number of outages and as crew counts were held constant. The only variables changing in each run is the number and location of outages. The storm track changes are represented by the changes in the location of the outages. Outages are summed per town and randomly located on roadways within that town. Crews are set to specific area work centers and the number of crews working corresponds to the historic restoration of Sandy2012, shown in Table 1. Crew counts are forward filled (continuing indefinitely) from the last value per AWC in cases where ABM simulations extend beyond the 11-day crew counts from Sandy2012. Panels A, B, and C in Figure 4 show the variability of predicted outages by the different WRF convective parameterizations, machine-learning algorithms, and weather inputs used in outage prediction modeling, and the red horizontal line shows the Sandy2012 outages. Panels D, E, and F show the estimated ETR for each number of outages, grouped by the same WRF models, machine learning algorithms and weather parameters as panels A, B, and C. The boxplots for number of outages and the corresponding predicted ETR are similarly shaped and show the same pattern. Because of the corresponding shapes between the two boxplots, we hypothesize that the number of outages and ETR are linearly related. Table A1 in Appendix A summarizes the number of predicted outages and ETR for each Sandy2112 scenario. ck changes are represented by the changes in the location of the outages. Outages are summed p wn and randomly located on roadways within that town. Crews are set to specific area work cente d the number of crews working corresponds to the historic restoration of Sandy2012, shown ble 1. Crew counts are forward filled (continuing indefinitely) from the last value per AWC in cas here ABM simulations extend beyond the 11-day crew counts from Sandy2012. Panels A, B, and C in Figure 4 show the variability of predicted outages by the different WR nvective parameterizations, machine-learning algorithms, and weather inputs used in outag ediction modeling, and the red horizontal line shows the Sandy2012 outages. Panels D, E, and ow the estimated ETR for each number of outages, grouped by the same WRF models, machin arning algorithms and weather parameters as panels A, B, and C. The boxplots for number tages and the corresponding predicted ETR are similarly shaped and show the same patter cause of the corresponding shapes between the two boxplots, we hypothesize that the number tages and ETR are linearly related. Table A1 in Appendix A summarizes the number of predicte tages and ETR for each Sandy2112 scenario.  To test our hypothesis of linearity, each number of predicted outages corresponds to one of the 30 Sandy2112 scenarios (summarized in Table A1 in Appendix A). Each scenario has three different predicted ETRs, one for each search strategy: Nearest outage, fastest repair time, and fastest repair time within a radius of the nearest outage. Figure 5 highlights how the number of outages and the chosen search strategy affects the final ETR. The fastest repair time tends to have a longer ETR than nearest outage and fastest within radius. The Kendall Rank correlation coefficient for nearest outage is 0.9448, fastest repair time is 0.9206, and fastest within radius is 0.9700, and the regression equation for each strategy is shown below the legend of Figure 5. Each of the Kendall's correlation coefficients represent a high correlation and linear trend between the number of outages and the predicted restoration time. Kendall Rank correlation was used because the fastest within radius strategy indicated heteroscedasticity, requiring use of a nonparametric test. Additionally, the small sample size was another reason to use the Kendall's Rank correlation. The black dot, which represents the Sandy2012 outages and restoration time, lies below the ABM ETR estimates indicating that the ABM slightly overestimates the predicted ETR.
is 0.9448, fastest repair time is 0.9206, and fastest within radius is 0.9700, and the regression equation for each strategy is shown below the legend of Figure 5. Each of the Kendall's correlation coefficients represent a high correlation and linear trend between the number of outages and the predicted restoration time. Kendall Rank correlation was used because the fastest within radius strategy indicated heteroscedasticity, requiring use of a nonparametric test. Additionally, the small sample size was another reason to use the Kendall's Rank correlation. The black dot, which represents the Sandy2012 outages and restoration time, lies below the ABM ETR estimates indicating that the ABM slightly overestimates the predicted ETR.
Previous studies from Hines et al. [9,28] and Carreras et al. [29] found insignificant correlation between blackout size and duration. However, previous studies are observations of different storms that have many different variables, including both the number of outages and the number of crews working. Using the ABM allows us to take away one of those dimensions, the number of crews, to find a simplification and now a linear relationship in the number of outages (representing size of the storm) and restoration duration. Our assumption of equal crew resources per scenario reduces one aspect of the variability seen in practice as previous studies compared only blackout durations and size without considering crew counts for each restoration. This linear relationship shows that the number of crews must change as the number of outages changes to prevent a change in the ETR. The simplification of the ABM allows us to study the impacts of one parameter before adding variability of multiple parameters.  Previous studies from Hines et al. [9,28] and Carreras et al. [29] found insignificant correlation between blackout size and duration. However, previous studies are observations of different storms that have many different variables, including both the number of outages and the number of crews working. Using the ABM allows us to take away one of those dimensions, the number of crews, to find a simplification and now a linear relationship in the number of outages (representing size of the storm) and restoration duration. Our assumption of equal crew resources per scenario reduces one aspect of the variability seen in practice as previous studies compared only blackout durations and size without considering crew counts for each restoration. This linear relationship shows that the number of crews must change as the number of outages changes to prevent a change in the ETR. The simplification of the ABM allows us to study the impacts of one parameter before adding variability of multiple parameters.
In addition to comparing the final ETR, the restoration curve for each strategy can be investigated. Figure 3 showed the curve in terms of thousands of customers remaining without power for Sandy2012 whereas Figure 6 shows the number of outages remaining for the WDM6 WRF simulation and boosted tree machine-learning algorithm with precipitation and wind input variables for Sandy2112. The ABM does not include customer predictions, therefore the restoration curve in terms of customers effected cannot be compared. Figure 6 shows the difference in the search strategies and the average of all Sustainability 2020, 12, 6502 13 of 27 three throughout restoration. Results prove our third hypothesis and shows differences among the search strategies. Specifically, crews going to the fastest repair time outage within a radius of the nearest outage (fastest-within-radius) and the nearest outage strategies consistently have the shortest ETRs. Crews going to the fastest repair time resulted in the longest overall ETR. By using the fastest repair time strategy, crews may spend more time travelling, but they also save the long duration outages until the end. When the repair time is not included in the prioritization, such as in the nearest outage strategy, there can be more overlap in the order. Some crews will have quick repairs and others will have longer. The differences in those repair times will prevent a step-like curve from forming, which can be seen in the fastest repair time strategy in Figure 6. The fastest repair time search strategy tends to result in a long tail at the end of the restoration where there are only a few outages remaining, but those few outages have long repair times. The fastest within radius strategy has the shortest ETR and the steepest slope. The fastest strategy starts close to the fastest within radius strategy but has a longer tail at the end, resulting in a longer overall ETR. The nearest outage strategy reduces the number of outages the slowest at the beginning of a storm but begins to drop off about halfway through restoration and finishes with a restoration time close to the fastest within radius strategy. Figure 6 shows the restoration curve for the Sandy2112 scenario with WDM6 WRF simulation and boosted tree machine learning algorithm with wind and precipitation input variables, but the overall shape of the three search strategies is consistent across all Sandy2112 scenarios. In practice, the restoration process does not have clearly defined search strategies like what was used in the ABM. The definition of search strategies in the ABM can be one way to study optimization of the restoration process and is what separates the ABM from other stochastic approaches. If there are significant differences between strategies, emergency managers can target that strategy once outages are made safe and priority locations have been restored.
In addition to comparing the final ETR, the restoration curve for each strategy can be investigated. Figure 3 showed the curve in terms of thousands of customers remaining without power for Sandy2012 whereas Figure 6 shows the number of outages remaining for the WDM6 WRF simulation and boosted tree machine-learning algorithm with precipitation and wind input variables for Sandy2112. The ABM does not include customer predictions, therefore the restoration curve in terms of customers effected cannot be compared. Figure 6 shows the difference in the search strategies and the average of all three throughout restoration. Results prove our third hypothesis and shows differences among the search strategies. Specifically, crews going to the fastest repair time outage within a radius of the nearest outage (fastest-within-radius) and the nearest outage strategies consistently have the shortest ETRs. Crews going to the fastest repair time resulted in the longest overall ETR. By using the fastest repair time strategy, crews may spend more time travelling, but they also save the long duration outages until the end. When the repair time is not included in the prioritization, such as in the nearest outage strategy, there can be more overlap in the order. Some crews will have quick repairs and others will have longer. The differences in those repair times will prevent a step-like curve from forming, which can be seen in the fastest repair time strategy in Figure  6. The fastest repair time search strategy tends to result in a long tail at the end of the restoration where there are only a few outages remaining, but those few outages have long repair times. The fastest within radius strategy has the shortest ETR and the steepest slope. The fastest strategy starts close to the fastest within radius strategy but has a longer tail at the end, resulting in a longer overall ETR. The nearest outage strategy reduces the number of outages the slowest at the beginning of a storm but begins to drop off about halfway through restoration and finishes with a restoration time close to the fastest within radius strategy. Figure 6 shows the restoration curve for the Sandy2112 scenario with WDM6 WRF simulation and boosted tree machine learning algorithm with wind and precipitation input variables, but the overall shape of the three search strategies is consistent across all Sandy2112 scenarios. In practice, the restoration process does not have clearly defined search strategies like what was used in the ABM. The definition of search strategies in the ABM can be one way to study optimization of the restoration process and is what separates the ABM from other stochastic approaches. If there are significant differences between strategies, emergency managers can target that strategy once outages are made safe and priority locations have been restored.   A Shapiro-Wilkes test for each strategy determined the range of ETRs to be normally distributed, but due to the small sample size, a nonparametric test will be used. A Kruskal-Wallis test resulted in a p value < 0.001, indicating differences between the medians of the search strategies and historic value (169 h). To determine which groups were different, a pairwise Wilcoxon Signed Rank test was used. Table 5 presents the results. The small p values (p < 0.05) led to rejecting the null hypothesis and concluding that the medians of each group is different. In this experiment, the number of crews were held constant for all strategies and all storms. The significant difference between strategies exemplifies the value of using an ABM over a linear model and proves our third hypothesis correct. However, restoration in practice does not strictly follow one strategy vs. another. There is a mix of strategies along with several safety protocols in place. The use of the strategies in the ABM can guide emergency managers in the range of time to expect for restoration. For example, if crews were to follow more closely the fastest within radius strategy, the ETR would be the shortest. If crews went by the fastest outage strategy, the ETR would be the longest. Furthermore, if managers were to take an average of all three strategy ETRs, the modeled time would be somewhere in the middle. Beyond using the ABM as a predictive tool, the ABM can be utilized after a storm to test how the restoration could have been impacted by using different strategies. Implementing the ABM in this way allows for the use of the actual outage locations and emergency managers can determine if other decisions could have had better restoration results. Again, this use of the ABM can allow emergency managers to adapt to climate change as storm intensities and patterns change. Figure 7 compares the range of ETRs for each search strategy, along with the average of the three. A Shapiro-Wilkes test for each strategy determined the range of ETRs to be normally distributed, but due to the small sample size, a nonparametric test will be used. A Kruskal-Wallis test resulted in a p value < 0.001, indicating differences between the medians of the search strategies and historic value (169 h). To determine which groups were different, a pairwise Wilcoxon Signed Rank test was used. Table 5 presents the results. The small p values (p < 0.05) led to rejecting the null hypothesis and concluding that the medians of each group is different. In this experiment, the number of crews were held constant for all strategies and all storms. The significant difference between strategies exemplifies the value of using an ABM over a linear model and proves our third hypothesis correct. However, restoration in practice does not strictly follow one strategy vs. another. There is a mix of strategies along with several safety protocols in place. The use of the strategies in the ABM can guide emergency managers in the range of time to expect for restoration. For example, if crews were to follow more closely the fastest within radius strategy, the ETR would be the shortest. If crews went by the fastest outage strategy, the ETR would be the longest. Furthermore, if managers were to take an average of all three strategy ETRs, the modeled time would be somewhere in the middle. Beyond using the ABM as a predictive tool, the ABM can be utilized after a storm to test how the restoration could have been impacted by using different strategies. Implementing the ABM in this way allows for the use of the actual outage locations and emergency managers can determine if other decisions could have had better restoration results. Again, this use of the ABM can allow emergency managers to adapt to climate change as storm intensities and patterns change.   ABM predicted ETRs for Sandy2112 simulations with Sandy2012 crew counts. Each box represents different search strategies and the average of those three. FWR represents fastest-within-radius. Pairwise Wilcoxon Signed Rank Test determined statistically significant differences between all four groups and the historic value.

Using the Predicted Estimated Time to Restoration to Increase Crews, Recalculate Estimated Restoration Times, and Compare Restoration Rates
Section 3 will increase the crew counts by the percent change of the initial ETR over the historic/goal ETR. This part of the study reintroduces the variability of crew counts that was removed in Section 2.
In practice, an emergency manager could set a goal ETR and continue to increase crews by the percent change over the desired ETR to determine the number of crews required. This dynamic increase is one of many possible methods to increase crews. Previous work by Walsh et al. [25] used a sensitivity analysis to determine the number of crews where additional crews would no longer improve the estimated restoration time. The study presented here took the percent change approach as opposed to a full sensitivity analysis in order to test a more dynamic approach based on the results from Section 2. A full sensitivity analysis would take many model runs, which would greatly increase the time and computational demands. In a storm scenario, a more targeted crew increase method could be more beneficial to emergency managers. Available crews are one of the most limited factors during storm restoration. Utilities have a set number of crews and all are working during the restoration following extreme storms. Utilities also have access to mutual assistance crews that can travel from other areas to assist in restoration. The problem with widespread extreme storms is that mutual assistance crews may need to travel very far distances, sometimes travelling across the country. The method proposed in this paper assumes all desired crews are available when needed.

Methods
This study looks at increasing the number of crews during the Sandy2112 restoration process as if there is no limit on the number of crews available. Daily crew counts per AWC were known for Sandy2012. For each Sandy2112 scenario, the percent change in the estimated ETR of the average of all three search strategies from Section 2 was used as the percent change for the daily crew counts per AWC. Table 1 shows the initial crew counts and Table 6 shows the increased crew counts for the WDM6 WRF simulation and BART machine learning with wind and precipitation inputs, which had the greatest average percent change in ETR of 125.7% for the average of the three search strategies. Similar tables were created for each of the 30 different Sandy2112 scenarios. Figure 8 illustrates the increased crew counts for the same WDM6 WRF simulation and BART machine learning with wind and precipitation inputs. Initial values were based on Table 1 and increased by 125% per AWC per day, as shown in Table 6. Crew counts after day 11 were forward filled (repeating a value indefinitely into the future). By using the average of the three search strategies, it is expected that the ETR will not be perfectly reduced to the Sandy2012 value but should be close to the original. From reintroducing the crew count variability, we expect to see less of a linear correlation between number of outages and ETR.  AWC1  110  106  104  93  95  83  89  53  22  43  49  AWC2  62  46  32  32  96  105  66  32  25  33  27  AWC3  74  81  84  84  111  74  29  25  28  71  103  AWC4  1  9  16  16  16  16  11  5  5  6  5  AWC5  29  20  17  17  13  6  6  6  6  19  21  AWC6  59  68  78  127  182  270  326  128  34  53  103  AWC7  96  84  108  135  189  258  249  144  65  108  156  AWC8  64  87  102  331  296  355  336  351  348  222  109  AWC9  36  68  106  177  328  448  485  584  712  461  431  AWC10  34  37  37  37  42  29  18  16  16  39  52  AWC11  57  80  82  104  370  493  613  870  902  636  682  AWC12  95  84  90  91  116  66  45  40  29  31  40  AWC13  89  59  44  56  61  61  32  21  21  22  24  Total  808  830  900  1299  1917  2266  2305  2276  2214  1742  1802 and precipitation inputs. Initial values were based on Table 1 and increased by 125% per AWC per day, as shown in Table 6. Crew counts after day 11 were forward filled (repeating a value indefinitely into the future). By using the average of the three search strategies, it is expected that the ETR will not be perfectly reduced to the Sandy2012 value but should be close to the original. From reintroducing the crew count variability, we expect to see less of a linear correlation between number of outages and ETR. Figure 8. Total crew counts for Sandy2012 (black) and Sandy2112 with increased crews (red). The crew counts per AWC per day were dynamically increased for each storm scenario by the percent change of the predicted ETR over the goal ETR (169 h). This set represents the Sandy2112 scenario of Figure 8. Total crew counts for Sandy2012 (black) and Sandy2112 with increased crews (red). The crew counts per AWC per day were dynamically increased for each storm scenario by the percent change of the predicted ETR over the goal ETR (169 h). This set represents the Sandy2112 scenario of WDM6 WRF simulation with BART machine learning algorithm and weather inputs of wind and precipitation. This scenario had a predicted ETR increase of 125% over the historic 169-h restoration, resulting in crews being increased by 125% per AWC per day. Exact values can be seen in Table 6. The values after the 11th day were forward filled until ABM replicated restoration was complete.

Results
As storms become stronger, there may be uncertainty in the restoration process, especially as the number of outages exceeds the experience of emergency managers. Utilizing the known information from past storms, such as best-fit repair time range and the number of crews per area work center, emergency managers can run the ABM to find the estimated time to restoration for increased outages (as shown in Section 2). Once that time is known based on the past resources, the number of crews can be increased by the percent change from the desired ETR and the ABM can be run again. The study showed that by increasing crews by the same percentage over the desired ETR results in an ETR closer to the goal. Although this crew increase method is not completely feasible in practice given limitations on available crews, it can give managers a place to start from. If crew counts are known for incoming crews throughout the restoration, managers can run the ABM with the added information to get more situation-specific results.
The goal of the increased crew counts was to reduce the predicted ETR back to the historic 169 h restoration of Sandy 2012. Figure 9 shows that the predicted ETRs have been decreased from Figure 7. Table A2 in Appendix B lists the new predicted ETR for each scenario. A Shapiro-Wilkes test determined the groups of search strategies to not be normally distributed, along with the small sample size. Therefore, a Kruskal-Wallis test resulted in a p value < 0.001, indicating significant differences in the median between groups. A two-tailed pairwise Wilcoxon Signed Rank Test comparing the three search strategies and average of the three vs. the historic restoration was conducted ( Table 7). The results show that the median of the nearest and fastest within radius search strategies are not statistically significantly different than the historic ETR, which means the increase in crews would be able to restore power by 169 h. This reduction in ETR shows how the ABM can be used to adjust crew allocations in order to reach a goal restoration time. In contrast, the medians of the fastest search strategy and average of all three strategies are still significantly different than the historic restoration. The p value of 0.71987 between the nearest outage strategy and the fastest within radius strategy prove our hypothesis of differences in ETR for different search strategies false. In this case, having adequate crews reduces the differences in ETR across search strategies. However, the increase in crew counts was not enough to result in no significant differences for the fastest outage strategy. In this case, our hypothesis is still correct that there is a difference in ETR between search strategy. The hypothesis of search strategies resulting in different restoration times is true when resources are not adequate (Section 2), but false when there are adequate resources (Section 3). strategy. In this case, our hypothesis is still correct that there is a difference in ETR between search strategy. The hypothesis of search strategies resulting in different restoration times is true when resources are not adequate (Section 2), but false when there are adequate resources (Section 3).   Now, comparing the number of outages and ETR for each of the 90 ABM model runs results in Figure 10. By reintroducing variability in the number of crews, the strong linear correlation is lost. However, in this case, the number of crews indirectly depends on the size of the storm and were intentionally chosen to result in an ETR close to the historic 169 h. The previous studies conducted by Hines et al. [9,28] and Carreras et al. [29] did not include any information on number of crews, but the data was direct observations from historic storms. The results from Figure 9 and Table 7 showed no significant difference between the nearest and fastest within radius search strategies, which can also be seen in Figure 10 by the overlap of those two strategies whereas the fastest strategy is still separated. The Kendall's Rank correlation coefficients were 0.472 for the nearest strategy, 0.393 for the fastest within radius strategy, and 0.439 for the fastest repair time strategies. Although the p value still indicates a significant correlation, the p values are reduced from the original values of 0.9448, Figure 9. ABM predicted ETRs for Sandy2112 simulations with increased crew counts. Each box represents different search strategies and the average of all three strategies. Pairwise Wilcoxon Signed Rank Test determined the nearest and fastest within radius (FWR) strategies to not be statistically different than Sandy 2012 (169 h, red line), which means the scaling of crews during Sandy 2112 would be enough to restore power. The other two methods were significantly different than Sandy 2012-indicating that more crews would be needed to restore power. Table 7. Results of pairwise Wilcoxon Signed Rank Test for the three search strategies, the average, and the historic restoration for Sandy2112 with increased crews. A Kruskal-Wallis test resulted in a p value < 0.001, indicating statistically significant differences between groups. The pairwise Wilcoxon Signed Rank Test indicates significant differences in the medians of each group (all p values < 0.05) except for fastest with radius/historic, fastest with radius/nearest, and historic/nearest.

Average
Fastest Fastest with Radius Historic Now, comparing the number of outages and ETR for each of the 90 ABM model runs results in Figure 10. By reintroducing variability in the number of crews, the strong linear correlation is lost. However, in this case, the number of crews indirectly depends on the size of the storm and were intentionally chosen to result in an ETR close to the historic 169 h. The previous studies conducted by Hines et al. [9,28] and Carreras et al. [29] did not include any information on number of crews, but the data was direct observations from historic storms. The results from Figure 9 and Table 7 showed no significant difference between the nearest and fastest within radius search strategies, which can also be seen in Figure 10 by the overlap of those two strategies whereas the fastest strategy is still separated. The Kendall's Rank correlation coefficients were 0.472 for the nearest strategy, 0.393 for the fastest within radius strategy, and 0.439 for the fastest repair time strategies. Although the p value still indicates a significant correlation, the p values are reduced from the original values of 0.9448, 0.9700, and 0.9206, respectively, from Section 2. In this example, the variability in crew counts decreased the p value for each search strategy. This method used the number of outages to determine the crew counts, but in practice there is more variability, which most likely causes the lack of correlation between blackout size and duration found by [9,28,29]. Some of the variability could be from limited access to crews and having to work only with what is available. Additionally, a historic restoration would not have the separate search strategies that were tested in the ABM. The crew counts in this section were "ideal", as compared to a true restoration where crew access is much more limited. Reintroducing the variation in crew counts creates more noise in the correlation between ETR and number of outages, which better represents the variability found in historic restorations. 0.9700, and 0.9206, respectively, from Section 2. In this example, the variability in crew counts decreased the p value for each search strategy. This method used the number of outages to determine the crew counts, but in practice there is more variability, which most likely causes the lack of correlation between blackout size and duration found by [9,28,29]. Some of the variability could be from limited access to crews and having to work only with what is available. Additionally, a historic restoration would not have the separate search strategies that were tested in the ABM. The crew counts in this section were "ideal", as compared to a true restoration where crew access is much more limited. Reintroducing the variation in crew counts creates more noise in the correlation between ETR and number of outages, which better represents the variability found in historic restorations. Additionally, the range of predicted ETRs for each strategy was reduced for the increased crew experiments. Table 8 shows the mean, standard deviation, median, and interquartile range for each strategy and the average of all three for Sandy2112 with Sandy2012 crews and Sandy2112 with increased crews. Increasing crew counts lowered the mean and medians from the initial ETR predictions down closer to the historic. The order of the groups remained the same, but the increase of crew counts resulted in no significant difference between the medians of fastest within radius/historic, fastest within radius/nearest, and nearest/historic. The increased crew counts were enough to reduce nearest and fastest within radius to the historic values and each other, but the fastest strategy and average of all three are still statistically significantly different from the historic. In addition, the standard deviation and IQR for each group was reduced. In Section 2 (Sandy2012 crew counts), the standard deviations ranged from 53.9 to 71.7, but in Section 3 (increased crews) the standard deviations were reduced to a range of 11.9 to 18.3. This reduction brought the standard deviation from two to three days down to less than one day. This reduction in standard deviation (and Interquartile range) shows that the ABM results can vary significantly based on the number of crews available compared to the number of outages predicted. With less available crews and more outages, the standard deviation in predicted ETRs will be greater. However, although the crew counts were consistent across all Sandy2112 scenarios in Section 2, the crew counts varied in Section 3. The crew counts indirectly depended on the number of outages predicted for each scenario. If the crew Additionally, the range of predicted ETRs for each strategy was reduced for the increased crew experiments. Table 8 shows the mean, standard deviation, median, and interquartile range for each strategy and the average of all three for Sandy2112 with Sandy2012 crews and Sandy2112 with increased crews. Increasing crew counts lowered the mean and medians from the initial ETR predictions down closer to the historic. The order of the groups remained the same, but the increase of crew counts resulted in no significant difference between the medians of fastest within radius/historic, fastest within radius/nearest, and nearest/historic. The increased crew counts were enough to reduce nearest and fastest within radius to the historic values and each other, but the fastest strategy and average of all three are still statistically significantly different from the historic. In addition, the standard deviation and IQR for each group was reduced. In Section 2 (Sandy2012 crew counts), the standard deviations ranged from 53.9 to 71.7, but in Section 3 (increased crews) the standard deviations were reduced to a range of 11.9 to 18.3. This reduction brought the standard deviation from two to three days down to less than one day. This reduction in standard deviation (and Interquartile range) shows that the ABM results can vary significantly based on the number of crews available compared to the number of outages predicted. With less available crews and more outages, the standard deviation in predicted ETRs will be greater. However, although the crew counts were consistent across all Sandy2112 scenarios in Section 2, the crew counts varied in Section 3. The crew counts indirectly depended on the number of outages predicted for each scenario. If the crew counts were the same for all Sandy2112 increased crew experiments, there may be more variability in ETRs, similar to Section 2.

Metrics beyond ETR: Outage Durations and Time to Reach Percentiles
Emergency managers are interested in restoration metrics beyond just the statewide ETR. Outage durations and restoration milestones can give managers another checkpoint during restoration. The distribution of outage durations could be another interesting outcome to look at. Outage durations for Historic Sandy2012 and Sandy2112 with increased crews will be compared. Sandy2012 durations are based only on the recorded outages and durations from the utility company. Figure 11 shows the frequency of outage durations for historic Sandy2012 and crew increased Sandy2112. Historic Sandy2012 had a high number of outages with very short durations. This could be influenced by intentional outages to make safe repairs where customers that had not lost power from the storm were temporarily shut off, a repair made, and then power turned back on. The ABM does not capture any outages that occur after the start of the storm-all outages are modeled as occurring at the start of restoration. The overall shape of the durations looks to be cyclical. More crews are working during the day and less crews at night, resulting in more outages ending during daytime hours and less during nighttime. Sandy2112 with increased crews had a range of durations similar to historic Sandy2012, but the frequency of duration times increased. Again, this is because Sandy2112 had increased outage numbers over Sandy2012. The outage durations are not equal to the repair time. The outage duration is from the outage report time until restoration completion. In reality, outages are called in by customers or discovered by utility crews and can be added to the count at any time, resulting in staggered outage start times. However, the ABM outages are only added at the beginning: Therefore, the outage duration is from the start of the simulation (time = 0) until the outage is repaired and may be longer than what's recorded in the Outage Management System. The historic Sandy2012 start times were normalized to the start of the storm to match the results from the ABM simulations. The durations shown here are the sum of the start time plus the outage duration.
Another way to look at the results is to compare the time to reach different milestones along restoration. For example, we were interested in whether the 25th, 50th, 75th, or final restoration times would shift for different scenarios. However, the time it takes to reach these milestones are usually measured in number of customers without power. Currently, the ABM does not incorporate estimated customers affected per outage because the outages are predicted at 2 km or town resolution and depending on location of the outage, a different number of customers would be affected. Figure 12 shows the time it takes to reach each of those percentiles in terms of outages repaired for the ABM replicated Sandy2012, Sandy2112 with Sandy2012 crews, and Sandy2112 with increased crews. As expected, the time to reach each percentile is lower for the increased crew counts and higher for the Sandy2012 crew counts. The difference between the three ABM results are smaller for the 25th percentile and the differences between groups increases more with each percentile. ABM replicated Sandy2012 and Sandy2112 with Sandy2012 crews have similar ranges for the 25th percentile and the mean, but then the 75th percentile and total restoration time for Sandy2012 are lower than Sandy2112 with Sandy2012 crews. Early in restoration crews are inundated with outages to repair. ABM replicated Sandy2012 and Sandy2112 with Sandy2012 crew counts have the same number of crews working at each timestep. It is expected that Sandy2112 with increased crews has a lower anticipated time to reach each milestone Sustainability 2020, 12, 6502 20 of 27 because the additional crews will be able to repair more outages. Tracking the time it takes to reach each percentile goal can give emergency managers "check-in" points during the restoration process. If those percentiles are not within a desired range, the number of crews can be increased and run again.
Sustainability 2020, 12, x FOR PEER REVIEW 20 of 27 Figure 11. Histograms of outage durations for Sandy2012 historic restoration, Sandy2012 ABM replicated restoration, Sandy2112 with Sandy2012 crews, and Sandy2112 with increased crews. The cyclical patterns represent that more crews are working during the day than nighttime-therefore, more outages are repaired during the daytime than nighttime.
Another way to look at the results is to compare the time to reach different milestones along restoration. For example, we were interested in whether the 25th, 50th, 75th, or final restoration times would shift for different scenarios. However, the time it takes to reach these milestones are usually measured in number of customers without power. Currently, the ABM does not incorporate estimated customers affected per outage because the outages are predicted at 2 km or town resolution and depending on location of the outage, a different number of customers would be affected. Figure  12 shows the time it takes to reach each of those percentiles in terms of outages repaired for the ABM replicated Sandy2012, Sandy2112 with Sandy2012 crews, and Sandy2112 with increased crews. As expected, the time to reach each percentile is lower for the increased crew counts and higher for the Sandy2012 crew counts. The difference between the three ABM results are smaller for the 25th percentile and the differences between groups increases more with each percentile. ABM replicated Sandy2012 and Sandy2112 with Sandy2012 crews have similar ranges for the 25th percentile and the mean, but then the 75th percentile and total restoration time for Sandy2012 are lower than Sandy2112 with Sandy2012 crews. Early in restoration crews are inundated with outages to repair. ABM replicated Sandy2012 and Sandy2112 with Sandy2012 crew counts have the same number of crews working at each timestep. It is expected that Sandy2112 with increased crews has a lower anticipated time to reach each milestone because the additional crews will be able to repair more outages. Tracking the time it takes to reach each percentile goal can give emergency managers "check-in" points during the restoration process. If those percentiles are not within a desired range, the number of crews can be increased and run again. Figure 11. Histograms of outage durations for Sandy2012 historic restoration, Sandy2012 ABM replicated restoration, Sandy2112 with Sandy2012 crews, and Sandy2112 with increased crews. The cyclical patterns represent that more crews are working during the day than nighttime-therefore, more outages are repaired during the daytime than nighttime.
Sustainability 2020, 12, x FOR PEER REVIEW 21 of 27 Figure 12. Comparison of the restoration time for the 25th, mean, 75th, and total outage percentiles for the ABM replicated Sandy2012, Sandy2112 with Sandy2012 crew counts, and Sandy2112 with increased crews. The increased crews for Sandy2112 reduced the ETRs, as expected.

Discussion
This study was conducted as a continuation of the proof-of-concept presented by Wanik et al. [11]. Previous power outage restoration models are limited in the parameters that can be varied, but the ABM allows several parameters to be varied to determine the impact on the ETR. Furthermore, previous restoration models were not combined with a power outage prediction model. The combination of predicted outages and simulated restoration can provide emergency managers with new tools as they prepare for, and learn from, climate-enhanced storms. The ABM uses simplifications to make the simulation usable without over constraining. Therefore, there are some decisions in the ABM that do not fully reflect the current practice in utility restoration. The goal was to study the impacts of increased outages that a Sandy2112 scenario could have on the restoration Figure 12. Comparison of the restoration time for the 25th, mean, 75th, and total outage percentiles for the ABM replicated Sandy2012, Sandy2112 with Sandy2012 crew counts, and Sandy2112 with increased crews. The increased crews for Sandy2112 reduced the ETRs, as expected.

Discussion
This study was conducted as a continuation of the proof-of-concept presented by Wanik et al. [11]. Previous power outage restoration models are limited in the parameters that can be varied, but the ABM allows several parameters to be varied to determine the impact on the ETR. Furthermore, previous restoration models were not combined with a power outage prediction model. The combination of predicted outages and simulated restoration can provide emergency managers with new tools as they prepare for, and learn from, climate-enhanced storms. The ABM uses simplifications to make the simulation usable without over constraining. Therefore, there are some decisions in the ABM that do not fully reflect the current practice in utility restoration. The goal was to study the impacts of increased outages that a Sandy2112 scenario could have on the restoration time if resources were not changed. This framework can be generalized to any future storm for planning purposes. The intention was to show that coupled systems of outage prediction and agent-based modeling could increase the simulation resources available to emergency managers as they prepare for increased storm intensity. As power outage prediction is more common and widely used, strategic models such as an ABM can utilize those predictions to test the restoration process with anticipated crew counts. The driving factors in restoration are the number of outages and the number of crews available. Access to an ABM could allow emergency managers to test their proposed resources when faced with storm damage unlike what they have seen in the past. Additionally, utilities can adjust outage repair times based on their past data. We used a uniform distribution of 1 to 8 h repair times.
The ABM initiates all outages at the start of the simulation instead of having differing start times as customers report new outages. This difference may result in seemingly longer outage durations whereas in reality, there are differences in start times as well as restored times. Restoration curves typically have a ramp up phase as outages are discovered and then a peak number of customers affected within the first day after restoration begins. The ABM starts restoration at that peak number. As mentioned previously, in restoration, there is sometimes an electricity shutoff to allow crews to make safe repairs, which is not captured by the ABM. Those shutoffs cause a temporary rise in customers without power, but then there is a quick decrease in customers affected once the power is safely restored. Those temporary outages would have short outage durations compared to the outages that began at the onset of restoration and those short outage durations will cause the high spike in frequencies seen in Figure 10 for historic Sandy2012. Another limitation in the ABM is the exclusion of priority restoration locations. Utilities have specific locations, such as hospitals, that must be prioritized if power is lost. The version of the model used in this study does not include those points. However, in the model update described in Section 2.1, a sensitivity test was conducted. Five randomly placed "priority" locations were initiated in setup. If any of those locations were deemed an outage, crews must repair that outage before using the regular search strategy. As with any storm, the priority points were not guaranteed outages. Outages are randomly located in the model, so the setup identifies outages that have been placed on those priority locations. Results showed that the five outage locations per town had minimal impact on the estimated time to restoration and restoration curve. Future work could investigate using the full list of priority locations to make the simulation more realistic.
Due to the high granularity of the outage prediction model, Wanik et al. [11] investigated changes in outages by location. In this study, we chose to keep the distribution of crews the same as Sandy2012. The ABM could be utilized by emergency managers to study changes in crew locations. If counties are expecting more damage, crews could be moved from an area that is expected to see less damage. However, utilities are hesitant to move crews before a storm occurs, but the changes could be studied in the ABM with moving the crews one day after restoration begins. We chose one simple crew increase strategy to present in this study, but the ABM could be run to test a wide range of changes in crews. The relocation of crews to other area work centers could be adjusted based on changes in the weather forecast.
The increase in crew counts was conducted for the average percent change of the three search strategies for Section 2. As seen in Figures 4-6, the fastest outages search strategy had the longest predicted ETR compared to the nearest and fastest within radius search strategy. The percent change from the historic ETR represented the average of all three search strategies. Therefore, the change would be more than 1:1 for nearest and fastest within radius, but less than 1:1 for the fastest outage strategy. Continuing with the example of the WDM6 WRF model with BART machine learning algorithm and both wind and precipitation inputs, the individual percent changes for each search strategy were 105, 106, and 165 percent for the fastest within radius, nearest outage, and fastest repair time, respectively. Crew counts were increased by the average of 125%. These numbers, along with Figure 9, also show some non-linearity in increased crews compared to the ETR. Both the nearest and fastest within radius still had medians that fell slightly above the historic restoration time but were not statistically significantly different from the historic restoration. Reaching those ETRs required the use of greater than a 1:1 ratio of crew increase. Future experiments could investigate using different crew counts for each strategy. Maybe using the exact 1:1 ratio could keep the ETR statistically the same as the historic, or maybe it would be too high. Using the average of the three strategies above desired ETR was used to simplify the model inputs, as well as account for differences of restoration in practice. As stated, the ABM gives the user the ability to study one strategy explicitly, but this is unrealistic in practice. The average was proposed as a "middle ground" of the three different strategies.
Wanik et al. [11] found that the different combinations of machine-learning algorithms, WRF simulation, and weather inputs used resulted in varied outage predictions. While there is uncertainty in which scenario will be most likely in the future, using this range of predictions can be beneficial to emergency managers to increase resiliency in extreme storms. The predicted ETR is strongly correlated to the number of outages. The impact of increased crew counts was tested to determine the effect on final ETR. The percent change of ETR over the historic 169 h was used to calculate the percent change of crews to use for each scenario. Thirty different Sandy2112 scenarios were tested and 30 different crew allocation simulations were used (one for each scenario, based on the average percent change in ETR of the three search strategies). The goal of the increased crew experiment was to investigate whether the ETR could be reduced back to the historic ETR. The nearest outage and fastest with radius search strategies produced a range of ETRs where the medians were not statistically different from the historic value. However, fastest outages strategy and average of all three strategies were still significantly different than the historic 169 h. Although the fastest outage and average of all three strategies were still statistically significantly different, the medians were reduced from 293 and 252 h down to 202 and 179 h, respectively. Future work could explore whether more variation in the increased crew counts could reduce the differences in medians among the ABM search strategies more to result in the fastest strategy no longer being statistically different from the goal ETR. Differences in crew requirements for each strategy to reach the goal ETR could give emergency managers a range of desired resources.
As mentioned in Section 1.1, the coupled system of outage prediction modeling and agent-based modeling for estimating the time to restoration can aid emergency managers as they face new restoration challenges. Specifically, the ABM can be beneficial to emergency managers in resiliency Zones 1 through 4 (resiliency zones proposed by [3]). In Zone 1, emergency managers can use the ABM with the predicted outages to test plans as the storm system is predicted. During Zones 2 and 3, emergency managers can update both the outage counts and crew counts in the ABM to calculate new estimates as both damages and resources are changing. Lastly, the ABM can be utilized in Zone 4 to test other scenarios or to prepare for future storms. This study is one example of utilizing the ABM in Zone 4. Although Hurricane Sandy occurred eight years ago, the damages and resources are being used to continue preparing for a changing environment. The ABM has the ability to mirror actual storm damage. This virtual environment can produce limitless combinations of outages and damage. Additionally, the ABM can then be used with many different strategies to repair each of those combinations. A coupled system such as this outage prediction and power outage restoration model can become a training tool for managers as storm damage exceeds their experience. This study highlights the novelty of using an ABM for power outage restoration over stochastic models. The ability to study individual parameters or multiple parameters at a time can provide insight for emergency managers on the interactions of multiple decisions.

Conclusions
Resiliency, or the ability of an electrical system to return to its fully functional state, is key to timely restoration following extreme storms. Power outage models allow the expected damage to be modeled as the storm is in its early forecast stages. Agent based models can be coupled with outage prediction models to estimate the restoration time based on predicted outages and available resources. Using the ABM while the storm is still being forecasted can aid emergency managers in increasing resiliency of the system. Early estimations of the necessary crews and resources can justify the use of mutual assistance crews and having them in route if necessary.
This study found a linear relationship between the number of outages and the estimated time to restoration when the number of crews is held constant. This finding, along with the ability of the ABM to vary individual parameters, can allow emergency managers to isolate individual decisions in order to study the impact of each decision on ETR. Using the same crew counts as Hurricane Sandy from 2012 for 30 different storm scenarios in the year 2112 resulted in differences among search strategies and the historic goal restoration. Increasing the crew counts based on the ABM predicted ETR resulted in two of the three strategies no longer being statistically different from the historic restoration. The ability to model explicit search strategies separates the ABM from previous research and makes this a novel study. The strategies can allow for restoration optimization if utilities were to follow one specific strategy more closely after outages are made safe and priority locations are restored. The study presented highlights the use of the ABM as coupled with an outage prediction model during forecasting, but the ABM could also be used for training and for understanding past storms. The ability of the ABM to study the impacts that human decisions have on the restoration cannot be captured in other stochastic restoration models.
Climate scientists predict stronger storms, which could leave utilities underprepared when using their experience from past storms to make decisions for current storms. Agent-based models coupled with outage prediction models can allow emergency managers to test resource combinations prior to storm restoration. The geospatial nature of the ABM allows emergency managers to run the simulation on the same scale they would make decisions in practice. Crews can be assigned to specific work areas. This study highlighted one specific crew increase strategy, but the model can be used to run many different strategies and reallocations of crews. The addition of an agent-based model to weather forecasts and power outage prediction models allow for a complete simulation of extreme storms and restoration all before the storm makes landfall. This complete modeling system can be beneficial as storm intensities exceed the experience of emergency managers. Storm restoration is a complex system, but added resources such as a coupled system of outage prediction and agent-based models can better prepare decision makers for a changing climate.

Conflicts of Interest:
The authors declare no conflict of interest.