A Quantitative Analysis of Fuel Break Effectiveness Drivers in Southern California National Forests

Abstract

Fuel and wildfire management decisions related to fuel break construction, maintenance, and use in fire suppression suffer from limited information on fuel break success rates and drivers of effectiveness. We built a dataset of fuel break encounters with recent large wildfires in Southern California and their associated biophysical, suppression, weather, and fire behavior characteristics to develop statistical models of fuel break effectiveness with boosted regression. Our results suggest that the dominant influences on fuel break effectiveness are suppression, weather, and fire behavior. Variables related to fuel break placement, design, and maintenance were less important but aligned with manager expectations for higher success with wider and better maintained fuel breaks, and prior research findings that fuel break success increases with accessibility. Fuel breaks also held more often if burned by a wildfire during the previous decade, supporting the idea that fuel breaks may be most effective if combined with broader fuel reduction efforts.

1. Introduction

In response to escalating wildfire damage to natural resources and human assets in the Western US, there is pressure for land and fire managers to increase their investment in pre-fire fuels reduction, including linear fuel breaks intended to help firefighters safely control fires at smaller sizes ([1]; Infrastructure Investment and Jobs Act, 117-58 U.S.C. § Section 40806, 2021). As with any pre-fire fuels treatment, fuel breaks have uncertain benefits owing to imperfect knowledge of if, when, and under what conditions they will be used for fire suppression. Various forms of historical fire frequency analysis and simulation modeling of fire occurrence and spread already exist to help managers construct or prioritize the maintenance of fuel breaks in locations with the highest likelihood of intercepting damaging fires [2,3,4,5], but there are few quantitative models to estimate the probability of an engaged fuel break holding based on characteristics of the fuel break, fire, and suppression response [2,6,7,8], making it difficult to assess the potential benefit of existing and planned fuel breaks and fuel break systems.

Fuel breaks are roughly linear features designed to improve firefighting access, safety, and effectiveness by reducing fuel loads or altering fuel type or arrangement [7,9]. Fuel breaks are often associated with fire breaks, which are narrower strips with no fuel, such as a road or a dozer line specifically maintained for fire control [10,11]. Fuel break construction and maintenance methods and anticipated benefits to suppression can vary based on ecosystem and fire management system contexts. The desired effects on fire behavior may include reduced fire intensity, crown fire potential, spotting, ignition probability, and fire residence times [7,9]. Commonly sought-after benefits for suppression include: improved access, visibility, and safety; reduced resistance to control (greater fireline production); reduced mop up effort; better aerial retardant function; and improved conditions for suppression firing [7,9,10,12,13]. Ultimately, the purpose of constructing a fuel break or fire break is to improve control likelihood, so effectiveness is usually judged by whether the feature holds when engaged by fire [2,8,14,15].

Our current understanding of fuel break effectiveness comes from a mix of field experiments, physical modeling, and observational studies. Case studies, anecdotal observations, and professional experience document how fuel breaks improve control potential and the conditions that may cause them to fail, such as exposure to wind-driven head fires with spotting [7,9,13,16]. The pioneering fire break experiments of Wilson [6] provide empirical evidence that increasing fire break width and decreasing fireline intensity and ember sources in adjacent fuels (i.e., by creating a fuel break) should increase control probability. Physical modeling of fire propagation across grassland fuel breaks confirms that the optimal fuel break width increases with fireline intensity [17]. Unplanned observations of wildfires engaging with fuel treatments are commonly plagued by incomplete data to characterize the fuel treatment, fire environment, and suppression response [15,18,19], but they still provide useful insights about the effects of fuel break treatment methods, age, width, and size inferred from visible indicators of fire intensity [20,21], remotely sensed burn severity [22,23], or control outcomes [2,8,24].

Here, we build on the fuel break effectiveness research of Syphard et al. [2,8] with multivariate statistical techniques that are increasingly used to model fire control likelihood from large observational datasets of fire extents [25,26,27,28]. We investigate a variety of predictor variables including suppression effort and daily fire weather and behavior metrics to identify key controls on fuel break effectiveness.

2. Materials and Methods

We focused our analysis on the portions of fuel breaks that engaged with wildfires larger than 200 ha during the period 2017–2020 and within a 1 km buffer around the administrative boundaries of the Angeles, Cleveland, Los Padres, and San Bernardino National Forests in Southern California (Figure 1). The 200 ha minimum size was chosen to concentrate on fires that resisted an initial attack and were mapped with sufficient accuracy for evaluating if fuel breaks held or burned over. We limited our study to the period 2017–2020 so that we could account for suppression influences using spatial datasets of ground and aerial firefighting operations. We focused on fuel breaks associated with National Forest System lands in Southern California because of fuel break location data availability and the presence of fuel treatment records to characterize maintenance history. Southern California is unique among the mountainous regions of the Western US for its deep history of linear fuel break use [16] and its current fuel break abundance.

The four Southern California National Forests cross three ecoregions, including the Southern California Mountains, the Southern California/Northern Baja Coast, and the Central California Foothills and Coastal Mountains (level III ecoregions from [29]). The climate of the region is Mediterranean with hot dry summers and comparatively cooler and wetter winters. The region has two distinct fire seasons—a summer fire season characterized by high aridity and many ignitions, and a fall fire season with fewer ignitions but larger fires driven primarily by downslope gravity winds such as the Santa Ana and Sundowner winds [30,31]. Vegetation type varies by elevation and aspect with chaparral dominating the lowest elevations and south facing slopes, grading into mixed oak and evergreen woodlands at middle elevations, and transitioning into conifer forests at the highest elevations and north facing slopes (biophysical setting from [32]).

2.1. Wildfires

We compiled a GIS polygon dataset of final fire extents from multiple sources, including Monitoring Trends in Burn Severity ([33], years 2017–2018), the Geospatial Multi-Agency Coordination Group ([34], GeoMAC, years 2017–2019), the Wildland Fire Decision Support System ([35], WFDSS, years 2017–2020), and Wildland Fire Interagency Geospatial Services ([36], only 2020). For each fire, we selected the polygon that best aligned with the burned area visible in post-fire National Agriculture Imagery Program ([37], NAIP) aerial photography. All 22 fires that were subsequently identified to have fuel break interactions (

Table 1. Characteristics of the large wildfires in our analysis of fuel break effectiveness, including fire year, name, area, the number of point samples of fuel break–wildfire interactions, fuel break success rate (% of point samples that held), availability of fireline data (see suppression section), and availability of fire detection data (see fire behavior section).

Year	Name	Area (ha)	Sample Points	Success Rate (%)	Fireline	Fire Detections
2017	Canyon 1	1077	277	0.51	Yes	Yes
2017	Creek	6321	2685	0.09	Yes	Yes
2017	Holcomb	609	84	0.37	Yes	Yes
2017	Lake	297	307	0.74	Yes	No
2017	Thomas	114,079	11,802	0.20	Yes	Yes
2017	Whittier	7451	971	0.16	Yes	Yes
2017	Wildomar	351	100	0.60	Yes	Yes
2018	Charlie	1356	1073	0.46	No	No
2018	Cranston	5395	2006	0.47	Yes	Yes
2018	Holy	9318	935	0.32	Yes	Yes
2018	Stone	547	110	0.19	No	Yes
2019	Cave	1265	371	0.36	No	Yes
2019	Saddle Ridge	3561	493	0.32	Yes	Yes
2020	Apple	13,450	1142	0.15	Yes	Yes
2020	Bobcat	46,943	6437	0.34	Yes	Yes
2020	Bond	2703	868	0.08	Yes	Yes
2020	Dolan	50,395	2201	0.17	Yes	Yes
2020	El Dorado	9204	1635	0.30	Yes	Yes
2020	Lake	12,545	3918	0.31	Yes	Yes
2020	Ranch 2	1667	780	0.82	No	Yes
2020	Rowher	262	307	0.72	Yes	No
2020	Valley	6633	451	0.44	Yes	Yes

) came from either GeoMAC or WFDSS, which are typically derived from high-resolution aerial infrared mapping or field-based mapping by local managers. Start and end dates for each fire were compiled from the previously mentioned spatial data sources, incident status summary reports (also known as “ICS-209s”) for all years [38], and the Fire Program Analysis Fire Occurrence Dataset [39] for the period of overlapping coverage (years 2017–2018).

2.2. Fuel Break Polylines

We developed a GIS polyline dataset representing fuel break centerlines by acquiring and cleaning a previous fuel break dataset for the entire study area, then intersecting these fuel break polylines with candidate wildfires, followed by the manual review and editing of the fuel breaks that interacted with wildfires. The fuel break dataset we started from represents the approximate centerlines of fuel breaks originally developed by Brennan and Keeley [40]. It was updated in 2015 by the USDA Forest Service (USFS) for a fuel break evaluation project [41] to include planned and potential fuel break locations and to remove legacy features from the original dataset that had not been maintained in recent decades. Some features added by the USFS did not conform to our spatial data standards or fuel break definition. We simplified any fuel break perimeters to centerlines and removed lines associated with the boundaries of several area-wide prescribed fires.

We then iteratively intersected the cleaned fuel break polylines with each candidate wildfire extent buffered by 400 m to identify fuel breaks with potential wildfire interactions. Potential fuel break–wildfire interactions were then manually reviewed using the most recent NAIP aerial photography prior to the wildfire to correct gross spatial alignment issues and digitize any missing fuel breaks. Fuel breaks were split into segments if the width, condition, or fire break status varied for more than 800 m in length (see the predictor variables section for definitions). We removed any features that were not visible in the pre-fire aerial photography, including legacy fuel breaks from the Brennan and Keeley data set [40] that reverted to natural vegetation and any potential or planned fuel breaks from the USFS that were not implemented before the fire.

2.3. Data Model

We used a dense sampling approach to generate observations of fuel break–wildfire interaction outcomes and associated predictor variables (Figure 2). By fire, the cleaned fuel break polylines were first rasterized at 30 m resolution to match the scale of our fuel and topography predictor variables, and then converted to points representing the cell centers. Fuel break sample points were considered held (success) if they fell within 100 m of the final fire perimeter plus half the fuel break width. The 100 m distance is to account for imprecision in the fuel break and fire perimeter mapping. The additional distance of half the fuel break width is to deal with situations where the fire stopped at the leading edge of a wide fuel break. The remaining samples that intersected the fire were considered burned over (failure).

The gridded sampling strategy was chosen to avoid generalizations that are required to analyze outcomes for the original polylines. Several challenges of working with the polylines as the observation units include: (1) deciding where one fuel break starts and the other begins; (2) dealing with length-dependent influences on exposure to fire and outcomes; (3) deciding how to attribute outcomes for lines that only partially engage with the fire; and (4) designing appropriate summary metrics for predictor variables that differ along the length of the fuel break. Our sampling approach captures the diversity of conditions along held and burned-over fuel breaks while avoiding difficult scale-related decisions necessary to attribute polylines with outcomes and representative predictor variables.

2.4. Predictor Variables

We considered a wide range of predictor variables related to fuel break accessibility, fire behavior, condition and design, suppression, topography, surrounding vegetation/fuels, and weather (

Table 2. Predictors of fuel break effectiveness sorted by data category (see text for complete descriptions and data sources) and specification of models evaluated in this paper (* for inclusion in model).

			Model
Category	Predictor Variable	Units	1	2	3	4	5
Accessibility	Road proximity	Continuous: 1 on road declining to 0 at 1000 m	*	*	*	*	*
Fire behavior	Daily area burned	Hectare (ha)	*	*	*	*
	Encounter type	Categorical: heading, flanking, and backing	*	*	*	*
	Fire radiative power	Megawatt (MW)	*
Fuel break	Condition	Ordinal: 1 for poor to 5 for excellent	*	*	*	*	*
	Fire break	Binary: 0/1 for absent/present	*	*	*	*	*
	Width	Meters (m)	*	*	*	*	*
Suppression	Aerial drop	Binary: 0/1 for absent/present	*	*	*	*	*
	Fireline	Binary: 0/1 for absent/present	*	*	*	*	*
Topography	Slope	Percent	*	*	*	*	*
	Topographic Position Index	Meters (m) above or below neighborhood mean	*	*	*	*	*
Veg/fuels	Canopy cover	Percent	*	*	*	*	*
	Fire Behavior Fuel Model	Categorical: Anderson 13 + non-burnable	*	*	*	*	*
	Recent wildfire	Binary: 0/1 for absent/present in prior 10 years	*	*	*	*	*
	Recent treatment	Binary: 0/1 for absent/present in prior 10 years	*	*	*	*	*
Weather	Burning Index	Continuous: positively associated with fire			*
	Energy Release Component	Continuous: positively associated with fire			*
	100 h fuel moisture	Percent	*	*			*
	Maximum relative humidity	Percent	*	*			*
	Vapor Pressure Deficit	Continuous: positively associated with fire				*
	Wind speed	Meters per second (m/s)	*	*		*	*

; Figure S1). Predictor variables were selected to capture key drivers identified by previous studies [2,8] and to address manager questions about fuel break placement, design, and maintenance.

2.4.1. Accessibility

An earlier analysis of fuel break outcomes in this same region [2,8] found that fuel breaks were more likely to hold if accessible to firefighters. We represented accessibility in our model using a relative metric of road proximity. First, we calculated the Euclidean distance from the nearest road [42,43] in meters. Second, we converted to a relative measure of road proximity that declines linearly from a value for 1 on a road to a value of 0 at a distance of 1000 m. This reflects our expectation that roads will have a stronger positive influence on probability of control at shorter distances and it prevents models from identifying spurious statistical relationships at very long road distances.

2.4.2. Fire Behavior

Fire managers understand that fuel break success declines with increasing fire behavior whether represented by rate of spread, intensity, transition to crown fire, or initiation of spotting [7,9,16], but quantitative relationships have only been developed experimentally in limited settings [6]. Given the difficulty of characterizing fire behavior after an event, previous research used final fire size as a proxy for fire behavior, finding a strong negative relationship between fire size and fuel break success [2,8]. Here, we make use of satellite fire detections and final fire perimeters to characterize fire growth, intensity, and spread patterns.

We estimated the daily area burned (ha) by applying the Parks [44] raster interpolation method at 30 m resolution to fire detection points from MODIS and VIIRS [45] that occurred within 1000 m and +/− 5 days of the reported date range of each fire (Figure 3a). The estimated date of burning from this analysis was also extracted to the sample points to make associations with daily weather data (described in the weather section). Three of our study fires (2017 Lake, 2018 Charlie, and 2020 Rowher) did not have enough fire detections to model the date of burning. We assumed that these fires had sparse detections because most of the area burned quickly, so the final fire sizes (262–1356 ha) were assigned as the daily area burned for the earliest reported date for each fire.

We also attributed each sample point with the total fire radiative power (FRP) in MW from MODIS and VIIRS fire detections summed within a 1 km radius neighborhood of the point over the entire duration of the incident (Figure 3b). FRP represents the rate of radiative energy release detected within each satellite pixel extent. The 1 km neighborhood size allowed us to attribute most of our samples with FRP values (94.1%) while preserving much of the spatial variability in FRP across the larger fires.

Finally, we estimated the spread type associated with the fuel break encounters in categories of backing, flanking, and heading by first approximating the final fire shape as an ellipse and then dividing the ellipse into fireline intensity zones using Scott’s proportion of head fireline intensity equation [46] based on [47]. To accomplish this, we first identified the centroid of the fire detections from the first day of each fire to approximate the ignition point. If this ignition point fell outside the final fire extent, it was moved to the closest point along the perimeter. We then identified the centroid of the final fire extent. A minimum bounding rectangle of the final fire extent was then developed in alignment with the axis between the ignition point and final centroid. The ellipse major and minor axis lengths in Scott’s equation were estimated as one half the minimum bounding rectangle’s length and width, respectively. We then divided the ellipse into zones of backing (<25% of head fire intensity), flanking (25–50% of head fire intensity), and heading (≥50% of head fire intensity) radiating from the ignition point (Figure 3c).

2.4.3. Condition and Design

We manually attributed fuel break condition and design attributes based on the most recent NAIP aerial photography prior to each fire [37]. Fuel break condition was assigned an ordinal rating in five classes, ranging from one for barely discernable from the surrounding vegetation to five for freshly cleared. We treated the fuel break condition as a continuous variable in our statistical analysis. We assigned a separate binary indicator for fuel break association with a fire break (0 = no, 1 = yes), which we expect to increase fuel break success (roads were considered fire breaks). Each fuel break was attributed with its average width in meters in 5 m increments for widths up to 50 m and 10 m increments for widths greater than 50 m. The increasing width increment beyond 50 m is to reflect the difficulty of precisely measuring a representative segment width from aerial imagery for fuel breaks in poor condition or with irregular shapes. Fuel break width includes the width of the fire break if present. If a fuel break was recently burned by a wildfire, its condition was judged relative to the pre-fire vegetation, and we referenced earlier imagery for width indicators if needed; this accounts for the maintenance benefit of wildfire while not overstating the width of the fuel break. In some cases, the original data sources mapped fuel breaks along roads without recent evidence of adjacent fuels management. We opted to retain these roads as fuel breaks because they often serve as important links between maintained fuel breaks and their inclusion in the earlier datasets implies a perceived importance for fire management. We attributed these roads as fuel breaks with a high condition rating, narrow width, and fire break present.

2.4.4. Suppression

Fuel breaks are tools for suppression, not features expected to passively stop wildfires [2,9,16]. We captured fuel break use in suppression with operational data on aerial drop and fireline locations. Aerial drop data representing individual drop paths as polylines were initially collected and cleaned to support Aviation Use Summaries [48]. Stonesifer [49] provided us an aggregated dataset that includes all the aerial drops made by the fleet of federal fixed-wing aircraft during years 2017–2020. This dataset does not include drops made by state or local aircraft, so we may underestimate the fuel breaks that received aerial drops on some fires. We attributed a binary indicator for association with an aerial retardant or water drop (0 = no, 1 = yes) if a sample fell within 100 m of an aerial drop line during the year of the fire. Fireline data came from the National Incident Feature Service (NIFS) event line dataset for the years 2017–2020 [50]. The event line dataset was filtered to feature categories indicating fireline construction or use during the incident. Data gaps in the 2017 NIFS data were minimized by adding fireline data extracted from incident geospatial databases on the National Interagency Fire Center (NIFC) server [51]. We attributed a binary indicator for association with a fireline (0 = no, 1 = yes) if a sample fell within 100 m of a fireline during the year of the fire. The fireline status for samples from fires without NIFS data were attributed with null values instead of zeros.

2.4.5. Topography

Fuel breaks in Southern California are often located along major ridges or valleys to take advantage of moderated fire behavior from reduced slope and aspect changes [10,16]. We characterized topography using terrain analysis of the 30 m National Digital Elevation Model [52] to calculate the percent slope and Topographic Position Index (TPI [53]). TPI is calculated as the difference in meters between each pixel’s elevation and the neighborhood mean elevation. We calculated TPI using a 200 m circular neighborhood based on previous success characterizing landform influence on control probability at this scale [26]. Highly positive TPI values indicate ridges and highly negative values indicate valleys.

2.4.6. Vegetation/Fuels

Fuel conditions around fuel breaks may influence the likelihood of control through their effects on fire behavior and suppression difficulty. We used two LANDFIRE [32] layers to characterize pre-fire fuel conditions within a 100 m circular neighborhood around each sample. The first is the mean forest canopy cover in percent. The second is the modal fire behavior fuel model (hereafter “fuel model”) in the Anderson [54] thirteen class system, which provides an indication of surface fuel load and type. Samples without a clear mode (i.e., multiple fuel models tied for most abundant) were assigned a null value. We also assigned each sample a binary indicator of burning (0 = no, 1 = yes) in a recent wildfire (≤10 yrs) based on spatial intersection with the same interagency historical records described in the previous wildfire section (years 2007–2020). Finally, we assigned a binary indicator of recent (≤10 yrs) fuel treatment (0 = no, 1 = yes) within 100 m of the sample, as reported in the hazardous fuel treatment polygon layer from the Forest Service Activity Tracking System [55].

2.4.7. Weather

We associated each sample with daily gridded weather estimates from gridMET ([56]; https://www.climatologylab.org/gridmet.html accessed on 10 September 2021) for the previously described estimated date of burning. The gridMET spatial resolution is approximately 4 km, and all weather metrics are daily averages except where otherwise noted. The variables we investigated include the Burning Index (BI), Energy Release Component (ERC), fuel moisture for the 100 h fuel size class (FM100; percent), maximum relative humidity (RH; percent), Vapor Pressure Deficit (VPD), and wind speed at 10 m above ground level (m/s). We considered this wide selection of weather variables because of varying manager preference for basic and derived weather indices, but several of these weather variables are highly correlated (e.g., ERC and 100 h fuel moisture, BI and wind speed, etc.). Instead of constructing models from the best performing subsets, we tested the performance of three commonly used weather variables sets: (1) the basic variables including maximum RH, FM100, and wind speed; (2) the National Fire Danger Rating System [57] components BI and ERC; and (3) the Hot-Dry-Windy Index [58] components VPD and wind speed.

2.5. Statistical Model of Fuel Break Effectiveness

We used the Elith et al. [59] gbm.step routine with gbm package version 2.1.8 [60] in R version 4.1.0 [61] to model the relationship between predictor variables and fuel break outcome (failed or held) with gradient boosted regression trees. The final dataset had 38,953 observations of fuel break pixel encounters with wildfire, consisting of 10,886 held observations (27.9%) and 28,067 failed observations (72.1%). Our core models were developed using 3-fold cross-validation, a bagging fraction of 0.5, a learning rate of 0.005, a tree complexity of 10, and a maximum tree limit of 2500 to standardize comparisons and avoid excessive overfitting. The tree limit was informed by an additional cross-validation procedure described below.

We tested five models constructed from the full training dataset (Table 2) to evaluate model performance with different weather variable sets and fire behavior variables that are difficult to predict and potentially conflated with outcomes (e.g., daily area burned and FRP). Model 1 includes all fire behavior variables and the three primary weather variables—100 h fuel moisture, maximum relative humidity, and wind speed. Model 2 includes the same variables except FRP because it is not currently well known to managers nor easily predicted. Model 3 also excludes FRP and represents weather with BI and ERC. Model 4 excludes FRP and represents weather with VPD and wind speed. Model 5 excludes all fire behavior variables and represents weather with the three primary variables.

We conducted two additional analyses to characterize Model 2 performance sensitivity to varied training data. For the first analysis, we iteratively trained Model 2 with the gbm function, withholding one fire at a time (i.e., cross-validation by fire) for tree numbers of 500 to 10,000. We used the results to estimate the predictive performance of the model with independent data and to identify an appropriate tree number to avoid overfitting. For the second analysis, we trained Model 2 with the gbm function on 20 random subsets of the full training dataset with minimum spacing distances of 50, 100, 200, and 500 m. Tree number was fixed at 2500. We used the second analysis to estimate the effect of sample point spacing and sample size on model performance. For both analyses, performance was evaluated for each model with all withheld observations using the receiver operating characteristics diagram area under the curve (AUC) and overall accuracy (classifying probabilities ≥ 0.5 as predicted successes). We also evaluated model consistency by examining variation in variable importance and the predicted probabilities between Model 2 trained with the full and subset data.

3. Results

Our five candidate models achieved similarly high levels of predictive performance (

Table 3. Relative variable importance by model for included variables (top). The three most important variables in each model are in bold. Excluded variables are marked with X. Model performance metrics (bottom). * denotes accuracy metrics based on success predicted for probabilities ≥ 0.5 with the full model training data.

Category	Predictor Variable	Model 1	Model 2	Model 3	Model 4	Model 5
Accessibility	Road proximity	3.5	3.7	3.6	3.6	3.9
Fire behavior	Daily area burned	8.8	10.6	20.3	20.6	X
	Encounter type	1.4	1.3	1.4	2.1	X
	Fire radiative power	14.4	X	X	X	X
Fuel break	Condition	2.5	3.7	3.4	3.2	4.0
	Fire break	0.4	1.2	1.3	1.5	1.4
	Width	5.8	6.5	7.9	7.9	8.1
Suppression	Aerial drop	5.7	5.8	6.4	6.4	6.5
	Fireline	17.3	18.7	18.8	18.9	19.6
Topography	Slope	0.4	0.5	0.5	0.6	0.6
	Topographic Position Index	2.1	2.6	2.9	2.8	2.8
Veg/fuels	Canopy cover	1.5	2.0	2.1	2.1	2.4
	Fire Behavior Fuel Model	3.0	3.8	3.5	3.1	4.0
	Recent wildfire	2.8	4.2	4.2	4.7	4.4
	Recent treatment	3.1	3.3	4.3	5.1	3.5
Weather	Burning Index	X	X	7.4	X	X
	Energy Release Component	X	X	11.9	X	X
	100 h fuel moisture	6.4	8.6	X	X	11.3
	Maximum relative humidity	15.5	17.4	X	X	19.4
	Vapor Pressure Deficit	X	X	X	7.8	X
	Wind speed	5.5	6.1	X	9.5	7.9
Performance	AUC (training)	0.976	0.967	0.966	0.965	0.963
metrics	AUC (3-fold)	0.971	0.962	0.961	0.960	0.958
	Overall accuracy (%) *	92.9	91.9	91.7	91.6	91.5
	Obs. failure & Pred. failure (n) *	27,275	27,188	27,191	27,257	27,167
	Obs. failure & Pred. success (n) *	792	879	876	810	900
	Obs. success & Pred. failure (n) *	1969	2263	2340	2477	2418
	Obs. success & Pred. success (n) *	8917	8623	8546	8409	8468

) with a slight decline in AUC and overall accuracy from Model 1 with all variables to Model 5 with no fire behavior variables. There were no meaningful differences in performance among models constructed with the three different weather variable sets (Models 2–4; Table 3) due to their similar representations of dryness and wind speed. Across all models, the most important variables were consistently related to suppression, weather, and fire behavior, if included (Table 3), and most models captured comparable relationships between the predictor variables and the probability of control (Figure 4 and Figures S2–S5). We focus on Model 2 as the preferred model for the remainder of the paper because it performed almost as well as Model 1 but without the need for managers to estimate FRP to make predictions.

The most important predictor variables in the preferred model are dynamic characteristics like suppression, weather, and fire behavior (Figure 5). Among the remaining variables, fuel break width, recent wildfire, fuel model, road proximity, and fuel break condition are the most important. Partial dependence plots (Figure 4) demonstrate that the model aligns well with the current understanding of fuel break effectiveness drivers. Fuel break success is positively associated with fireline and aerial drop indicators of suppression effort. Effectiveness is expected to increase with increasing maximum relative humidity and 100 h fuel moisture and decrease with increasing wind speed. The likelihood of control declines with increasing daily area burned. Fuel break success is highest when fire is backing and much lower with flanking or heading spread. Fuel break width and condition are both positively associated with control. Both fire break presence and high road proximity are associated with improved success. Despite their lesser importance, the veg/fuel and topography variables align with expectations. Fuel break success is higher if recently burned or treated, or canopy cover is lower. Fuel break effectiveness is highest at middle and higher topographic positions and lower slopes.

The additional cross-validation by fire for Model 2 showed that AUC peaks when models are fit with between 1500 and 3000 trees and then slowly declines with an increasing number of trees (indicating overfitting [59]), although the exact response varies based on the fire withheld from the model (Figure 6). In contrast, the gbm.step procedure suggested that the optimal number of trees for Model 2 is above 10,000. AUC assessed with the training data for the full model increases unimodally from 0.913 at 500 trees to 0.993 at 10,000 trees. For the 2500-tree model, the validation AUCs vary between 0.275 and 0.954 with a simple mean of 0.732 and an observation-weighted mean of 0.753. The validation AUCs are all above 0.7 for the six fires with more than 2000 observations that collectively account for 74.6% of the total observations. The validation overall accuracies range between 19.1 and 92.0% with a simple mean of 66.1% and an observation-weighted mean of 75.8%.

The models created in the cross-validation by fire procedure retain the core characteristics of the full model. The variable importance measures do not differ drastically from those of the full model, suggesting that the core drivers of effectiveness are consistent across fires (Figure 5). The predicted probabilities from the validation models are also close to those of the full model. The mean difference in the predicted probabilities for the full observation set vary between less than 0.001 to 0.062, and the Spearman’s rank correlations are all high, ranging between 0.81 and 0.99. The most influential fire on the final model (highest mean difference and lowest Spearman rank correlation) is the 2017 Thomas Fire, which accounts for approximately a third of all the sample points. The predicted probabilities for the validation models have fair alignment with the spatial patterns of outcomes for the fires with the most wildfire–fuel break observations (Figure 7).

The predictive performance of Model 2 remained high when trained on subsets of the full training dataset with minimum spacing distances of 50 to 500 m and sample sizes that include between 32.1 and 2.9% of the full training dataset (

Table 4. Mean results for Model 2 trained with 20 random subsets of the full training dataset with minimum spacing distances of 50 to 500 m. AUC and overall accuracy were calculated with all samples not used for model training. The Spearman rank correlation is calculated by comparing the full and subset model predictions for the entire training dataset.

Minimum Spacing (m)	Sample Points	Percent of Full Sample Points	AUC	Overall Accuracy (%)	Spearman Corr. with Full Model Predictions
50	12,500	32.1	0.963	91.2	0.994
100	6152	15.8	0.959	90.7	0.984
200	3114	8.0	0.950	89.7	0.963
500	1146	2.9	0.925	87.2	0.909

). Compared to Model 2 trained with the full training dataset, which has a cross validated AUC of 0.962 and an overall accuracy of 91.9, the subset model performance declines slightly with increasing minimum spacing and shrinking sample size. Models trained on subsets with 50 and 100 m minimum spacings have the largest sample sizes and perform almost as well as the full model with AUCs of 0.963 and 0.959 and overall accuracies of 91.2 and 90.7%, respectively. Subset model structure and predictions are also similar to the full model; variable importance measures of the subset models are close to those of the full model (Figure S6) and Spearman rank correlations between the full and subset data predictions average 0.994 and 0.984 for 50 and 100 m spacings, respectively. Performance declines more for models trained on the subsets with 200 and 500 m minimum spacing distances (Table 4), but AUCs are still above 0.9 and overall accuracies are above 87%. The 200 and 500 m spacing model predictions remain similar to those of the full model (Spearman rank correlation > 0.9), but the underlying model structure has several differences from the full model. Suppression, weather, and fire behavior variables are still the most important predictor variables, but there is a shift towards higher importance of fuels, topography, and road proximity variables (Figure S6).

4. Discussion

Our analysis reaffirms previous work on fuel break success rates and drivers of effectiveness. The 27.9% success rate by length for our training data falls in the 22–47% range reported for these same National Forests by Syphard et al. [2] for the period 1980–2007. The high importance of suppression, weather, and fire behavior variables in our model also agrees with the Syphard et al. [2] findings that fuel break accessibility, suppression resource availability, and fire size (proxy for weather and fire behavior) are key drivers of fuel break effectiveness. Our accounting of daily weather and fire behavior helps better define the range of conditions that fuel breaks perform well in (Figure 4) compared to the more qualitative observations that fuel breaks often fail in dry, windy weather or extreme fire behavior with spotting [16]. Despite their lesser importance in the model, variables related to fuel break design and maintenance function as expected; wider fuel breaks kept in better condition with frequent fuel treatments should improve fuel break outcomes. Our model approaches the level of realism and predictive performance necessary to inform suppression decisions, and it can also be used in a pre-fire context to assess the effects of fuel break design choices.

4.1. Progress, Limitations, and Future Directions for Modeling Fuel Break Effectiveness

Our dense point sampling design and the associated determination of fuel break outcomes (Figure 2) provides a flexible data model for linking fuel break segments and their outcomes to predictor variables with different spatial and temporal resolutions. We opted to generate samples at the resolution of our most detailed spatial data, but the tight spacing of points may cause pseudo-replication issues as fuel break outcomes and several of our predictor variables may have dependencies at coarser spatial and temporal scales, like most observational fire data [23,62]. Higher resolution data streams for dynamic variables like weather, fire behavior, and suppression should mitigate much of this issue in the future, but fuel break control outcomes will always suffer from some degree of spatial autocorrelation. Figure 2 and Figure 7 illustrate representative spatial patterns of fuel break outcomes on large fires with successes and failures clustering at scales ranging from several hundred meters to several kilometers (see Supplementary Materials for additional analysis of spatial autocorrelation). Our sensitivity analysis shows that Model 2 trained with wider spaced random subsets (minimum 50–500 m) of the full training dataset perform almost as well as the same model trained with the full training dataset and captured similar relationships between predictor variables and fuel break outcomes. Future work with larger datasets could focus on decoupling sample point spacing and sample size impacts and testing sensitivity to even wider minimum sample spacings. Sampling designs could also be informed by surveying fuels managers and firefighters on what they view as the appropriate spatial scale to characterize fuel break attributes and outcomes.

Despite our accounting of many weather, fire behavior, and suppression factors, the lower AUC from the cross-validation by fire procedure also suggests that there are unique fire characteristics that are not accounted for by our predictor variables. Cross-validation by fire indicates that the actual predictive performance of our model (~0.75 AUC) is substantially lower than assessed with random fold cross-validation (~0.96; Table 3), but is still high enough to make useful predictions. We suspect that suppression strategy may further explain fuel break outcomes across fires, as fuel breaks may lose their strategic purpose, despite their potential to hold, based on neighboring fire spread that forces managers to identify a bigger box and use suppression firing to tie distant lines together. Suppression firing is currently poorly documented, but anecdotal reports from the United States [63] and reconstructions of its use in Australia suggest it may contribute considerably to the area burned by large wildfires [64]. Long range spotting may also dramatically alter the fuel break outcomes on some fires with little to no influence of the fuel break characteristics. Spotting is likely captured indirectly in our model through its association with weather variables, daily area burned, and FRP. Our simplified accounting of suppression with binary indicators of activity does not account for potentially important characteristics like fireline type, width, quality, timing of construction, and whether it was used for suppression firing or staffed for holding operations.

Fuel break control outcomes evaluated using the final fire perimeter are relevant for evaluating fuel break strategies aimed at reducing fire sizes and impacts across broad landscapes, but there are other relevant fuel break outcomes that we did not consider in this study. For example, we did not address whether fuel breaks altered fire behavior, delayed spread, increased suppression efficiency, facilitated access, or provided safer working environments for firefighters. We also did not address the role of fuel breaks in protecting community assets or controlling small fires. Presumably, fuel breaks should be more effective when they are engaged by small fires because they will cover proportionally more of the fire perimeter. Accounting for the benefits of controlling small fires at fuel breaks is challenging because the counterfactual outcome is difficult to accurately define (i.e., where the fire would have spread if not controlled at the fuel break).

Daily area burned (Models 1–4) and FRP (only Model 1) were strong predictors of fuel break success, but it is important to recognize that both can be conflated with control outcomes as fuel break failures lead to increased area burned and radiation as fuels behind the fuel break combust. Conflation between predictor variables and observed outcomes can lead to overestimated model performance. We think that area burned and FRP are reasonable to include as predictor variables given that faster spreading and more intense fires should be more challenging to safely control due to the need to access, prepare, fire, hold, and mop up more line. We opted to include daily area burned in our preferred model because most fire organizations are already making the necessary fire growth predictions to apply the model and it is consistent with previous fuel break effectiveness models for the region [2,8] except for the improved temporal resolution (daily vs. incident). In contrast, FRP is a relatively new metric that few managers can translate into practical terms, and it is not an output of well-established fire models. The real-world application of our model should also account for uncertainty from the associated fire behavior modeling.

Major issues facing future empirical fuel break research are defining consistent criteria of what qualifies as a fuel break and maintaining datasets on fuel break location and characteristics. The Brennan and Keeley [40] fuel break dataset we leveraged in our analysis includes diverse features that may not all align with the modern vision of a strategic pre-fire fuel break. Some fuel breaks were constructed opportunistically during fire suppression and may persist long enough to be useful on subsequent fires, even if they were not planned land management actions. Similarly, many of the original fuel break features appear to be roads with varying levels of shoulder maintenance, which may function like fuel breaks despite different funding sources and motivations. Only 19.6% of our samples had evidence of hazardous fuel treatment activities in the ten years before engaging with the study fires. Thus, we considered attributes reflective of treatment in our model to contrast fuel breaks of varying qualities. Fuel breaks with recent treatment held 38.5% of the time compared to 25.4% without recent treatment, confirming that either the maintenance or correlated characteristics of these fuel breaks boosts effectiveness. Our post hoc evaluation of fuel break width and condition with aerial imagery seems to have captured useful information (Figure 4 and Figure 5), but field inventories would improve their accuracy and allow for more detailed description of fuel type, amount, and arrangement.

4.2. Management Implications

The strong influence of weather and fire behavior factors on our model make it potentially useful for identifying when and where to allocate suppression effort. Fuel break effectiveness has a threshold-like response to several of the weather and fire behavior variables, suggesting that control likelihood substantially declines when fuel breaks are exposed to fast-growing head fire under dry and windy weather (Figure 4). Staffing fuel breaks under these conditions also risks firefighter safety [65], so it may make sense to delay suppression efforts until the weather moderates, or to focus on fuel breaks near the heel of the fire where lower intensity backing fire improves the likelihood of control (Figure 4 [10,66]). Suppression greatly increases the odds of fuel break success; the mean success rate for our observations without fireline and aerial drops was only 12.7% compared to 68.1% for observations with both fireline and aerial drops. Suppression clearly has strong effects, but some of this difference is likely due to the strategic use of suppression under more favorable weather conditions, which is captured through the variable interactions in our boosted regression model. Still, our estimated success rate for fuel breaks with fireline and aerial drops is close to the probability of success reported for halting fire advance with airtanker drops [67].

Wider and better maintained fuel breaks should perform better than narrow and overgrown ones, but fuel break width, condition, and prior treatment only account for 13.5% of the variable importance in our preferred model (Figure 5), so improved fuel break design and maintenance are not expected to drastically improve fuel break outcomes on the types of large fires we studied. The highest success is expected for fuel breaks freshly cleared of native vegetation to greater than 300 m wide (Figure 4). Firefighters in Southern California are highly proficient at constructing ridgetop dozer lines, so the pre-fire condition of the fuel break may not be critical on larger fires as long as firefighters have the time and resources to improve them during the incident. Like Syphard et al. [2], we found greater success for fuel breaks close to roads, so it makes sense to prioritize fuel break construction and maintenance where they are most accessible to firefighters. Additionally, the presence of a road or other fire break should further improve fuel break effectiveness by aiding suppression firing [24,64,68]. We also found evidence that fuel breaks are more effective if associated with areas recently burned by a wildfire (Figure 4 and Figure 5) consistent with reports from the Whittier Fire of moderated fire behavior in previously burned areas that improved suppression effectiveness and dramatically reduced suppression needs [13], and post hoc evaluation of the Las Conchas Fire in New Mexico that found improved fireline effectiveness in areas previously burned by the Cerro Grande Fire [69]. This finding is in line with earlier suggestions to maximize the effectiveness of fuel breaks by combining them with broader landscape fuel treatments, especially prescribed fire [7].