How Information Source and User Attributes Affect Use of Fire Management Information

Abstract

This study examines how information source and fire manager attributes affect the use of 33 different information sources used for fire management. The approach is like that of recreation demand models that predict an individual’s travel to recreation sites based on individual and site characteristics. Here, we predict “visits” to websites rather than campsites. The study develops and estimates a random utility model, using survey data from a representative sample of US Southwest fire managers. Results were consistent with predictions of economic value of information models. Significant predictors included the agency that a manager worked for, a manager‘s self-reported role within the agency, the interagency dispatch centers where they worked, the total number of fire management decisions, and the geographic and administrative scope of the dispatch center management area. Manager personal attributes (education, age, experience) only minutely improved model fit. Information use varied significantly by type of information source. The probability of use was greater for general weather or climate websites/portals than for specialized sources (such as drought, ENSO, or fire decision support tools (DSTs)). Fire management-specific sources (excluding fire DSTs) had a greater probability of use than general sources. Manager reliance on different sources of information shifted when moving from before to during the fire season. Future research could explore the internal dynamics of agencies and dispatch centers affecting information use, why fire managers do not use decision support systems more to support decisions, and the role of different types (and not just years) of experience.

1. Introduction

The National Institute of Standards and Technology has estimated the economic burden of wildland fire to range from $71 billion to $348 billion ($2016 US) annually [1]. Economic burden includes fire prevention and suppression costs and economic losses from destruction of property and loss of life. Given such losses, it is unsurprising that there is interest in providing information to aid in fire management. Multiple U.S. federal agencies have data products directly related to fire management. Others, providing more general climate, weather, and environmental information, emphasize its usefulness for fire management. Much emphasis has been placed on the supply of fire management information. This has led to a large number of information sources, data platforms, and fire decision support tools (DSTs). Agencies have paid less attention to the demand for information. A growing body of research, however, has found that many data products developed for fire management are either not used, or not used as intended [2,3,4,5], and have identified various barriers to their use [2,3,4,5,6,7,8].

What are the key factors driving fire manager use and non-use of information for fire management? This study introduces a value of information (VOI) framework to address this question. A VOI framework starts from the premise that information has value because it can improve decisions and thus increase expected utility (or payoff). The value of information is defined as the difference in expected outcomes with and without that information. We use this framework to develop a random utility model to estimate the relative influence of different factors on fire manager use and non-use of different data and information sources. The approach is like that of recreation demand models that predict an individual’s travel to recreation sites based on individual and site characteristics [9,10,11]. Here, we predict “visits” to websites rather than campsites. Factors considered include the manager’s personal attributes, type of job, and job context (agency, dispatch center, season, number of decisions). Another important factor is the type of data/information source. These vary from general climate and weather data to more specific fire-related sources, including DSTs.

This study builds on previous research that has relied on qualitative empirical methods to explore fire manager information use. The work has been critical in sharpening thinking about motivations for and barriers to information use and is important for guiding survey methods for statistics-based research on these topics. Studies, however, have relied on a small number of respondents or informants (<50), sometimes from a single agency [2,3,4,6,12,13]. This raises questions about how generalizable results actually are [6]. This study also draws from value of information studies applied to agricultural decision-making. These include studies on the use of economic data [14,15,16] and weather and climate data [17,18,19,20,21,22,23]. Recent work has introduced the value of information framework to fire management conceptually [24] and empirically to explain information use [25]. We extend the methods and insights from this literature to statistical analysis of the role of fire manager personal attributes, job environment, and information source attributes to explain their information use choices.

We make use of a representative sample of Southwest U.S. fire managers, conducting statistical tests for how well the sample represents the total target population of fire mangers. The survey data is used to estimate the random utility model and conduct formal statistical tests of hypotheses that arise out of the value of information literature and qualitative studies of fire manager information use. Because we examine 175 fire managers’ choices to use or not use 33 information sources, both before and during fire season, we are evaluating literally thousands of information use decisions. Examining thousands of decisions rather than a small number of key informant perceptions greatly adds to the statistical power of our analysis. Many of our findings are consistent with the value of the information literature and reinforce and support findings of fire management studies using qualitative empirical methods. Our study illustrates that there is not a simple either/or choice between qualitative and more statistical empirical methods. Rather, our results suggest strong complementarity between the approaches, each reinforcing the other.

Section 2 discusses data and methods used, including the survey and the representativeness of our sample data, the theoretical value of information model, and the random utility model. Section 3 reports regression results from the random utility model, testing hypotheses about drivers of information use and assessing the relative importance of fire manager personal attributes, job attributes, and data source attributes in predicting information use decisions. Section 4 discusses the main findings in light of value of information theory and earlier empirical findings on fire manger information use. We conclude by discussing some remaining questions our findings raise but do not answer. We identify knowledge and data gaps and discuss their implications for future research.

2. Materials and Methods

This study examines how the use or non-use of individual fire management information sources is affected by fire manager attributes as well as the attributes of the information sources themselves. Manager attributes include personal attributes (e.g., age, experience, and education) and attributes of their job (type of job, agency they work for, interagency dispatch center where they work). Next, we develop a random utility model to predict use of information source j by fire manager i. We estimate the model via logistic regression, using survey data from a representative sample of fire managers in the Southwest United States [25,26].

2.1. A Conceptual Value of Information Model

Fire managers each have an expected utility function V_i that depends on the expected outcomes of a vector x_n of n different decisions or actions. Each manager can consult j different information or data sources. These are represented by a vector s_j. The term b [x_n (s_j)] represents the expected benefit of using information source j to make decision n, while b₀ [x_n (s_j0)] is the expected benefit of making the nth decision without consulting information source j. V_i, the value of using information for making the ith decision, is

V_i {b_i [k, ρ, α, δ, x (s_i)] − b_i0 (k, ρ, a, x_i0)} A − c_i (s_i, k, A, ρ, α, δ)

(1)

where

• b [x (s_i)] = benefit when information source i is used (per unit of land managed);
• b₀ (x₀) = benefit when information source i is not used (per unit of land managed);
• x_i (s_i) = decision made or action taken when information source i is used;
• x_i0 = decision made or action taken when information source i is not used;
• s_i = information sources used to make decision i;
  A = land area supervised or managed;
• c_i = cost of using information to make decision (or take action) I; may include costs of delay;
• k = a measure of the manager’s knowledge or technical capacity;
• ρ = the manager’s job or role within the fire management system;
• α = the agency that the manager works for;
• δ = the dispatch center where the manager is assigned.

Fire managers can make multiple decisions. Their optimization problem is

max ΣV_i {b_i [k, ρ, α, δ, x(s_i)] − b_i (k, ρ, a, x_i0)} A − Σ c_i (s_i, k, A, ρ, α, δ)

(2)

with respect to s_i for i = 1, … n decisions, and subject to t_i (s_i) ≤ t_i where t_i is some constraint on the decision maker. This could be a budget constraint. Or it could represent a time constraint. The t_i (s_i) term could represent the time needed to access and interpret information from a given information source i. This time needed may depend on a manager’s technical capacity. This capacity could depend on a manager’s age, education, or experience. It could also depend on how much formal training is needed to use the information source and how much training the manager has. Fire managers must make decisions or take actions over a limited time interval.

For the fire manager’s constrained optimization problem, the Lagrangian function is

max L = ΣV_i {b_i [k, ρ, α, δ, x_i (s_i)] − b_0i (k, ρ, a, x_i0)} A − Σ c_i (s_i, k, A, ρ, α, δ)
+ Σ λi (t_i − tⁱ (s_i))

(3)

where λ_i represents the shadow costs of the decision maker’s time constraint for decision i. The first-order conditions for optimally acquiring information are

(∂V_i/∂b_i) (∂b_i/∂x_i) (∂x_i/∂s_i) = ∂c_i/∂s_i + λ_i (∂t_i/∂s_i) for i = 1, …n

(4)

The fire manager will consult information sources up to the point where the marginal benefit meets the marginal cost of acquiring more information or the time constraint becomes binding. Individual information sources could complement each other or be substitutes for each other. With a binding time constraint (λ_i > 0) a fire manager must decide or take an action without consulting additional information sources.

2.2. Random Utility Model of Information Use

We model fire manager use of information in a Random Utility Model (RUM) framework based on Value of Information (VOI) theory. Information is valuable to the extent it can improve expected outcomes of decisions [27,28]. Managers choose among information sources that differ in attributes that affect how much they can improve fire management decisions. The deterministic component of indirect utility from source j is specified as the net expected value of information—i.e., the expected increase in decision payoffs conditional on updated beliefs, and less the generalized costs of accessing and processing information. This yields a standard representation U_ij = V(X_i, Z_j) + ε_ij, where X_i and Z_j denote individual and source attributes [29]. This approach provides the theoretical basis for applying a recreation demand-type discrete choice framework to information use. As in recreation site choice models, individuals select among differentiated alternatives characterized by observable attributes and incur generalized access costs, with preferences defined over attributes [30]. While recreation sites generate direct consumption utility, information sources yield indirect utility through improved decision outcomes. The notion of “travel cost” in recreation models generalizes to a shadow price of information acquisition that includes time, search effort, and cognitive constraints, consistent with economic models of time allocation and information search [31,32].

Our VOI model above can thus be expressed as a reduced-form RUM, which can be used to assess the factors that influence fire manager i’s choice of whether or not to use information source j. Fire manager utility when an information source is not used is

V₀ = V {b_i0 [k_i, r_i, a_i, x_n (s_j0), z_j]} + ε [k_i, r_i, a_i, x_n (s_j0), z_j, e_ij0]

(5)

where a fire manager’s utility when information source j is used is

V₁ = V{b_i [k_i, r_i, a_i, δ_i, x_n (s_j), z_j] − c_in (s_j, k_i, r_i, a_i, δ_i, z_j)} + ε [k_i, r_i, a_i, x_n (s_j), z_j, e_ij]

(6)

The terms e_ij0 and e_ij are vectors representing unmeasured attributes of the fire manager, the system where the manager operates, or the information source itself.

If the population is drawn from a random sample with common attributes, then e_ij0 and e_ij will be random variables, and the utility function value will be stochastic [33]. One can express these random components as ε(e_ij) and ε(e_ij0). The VOI, V*, is

V* = {b_i [k_i, ρ_i, a_i, δ_i, x_n (s_j), z_j] − c_in (s_j, k_i, ρ_i, a_i, δ_i, z_j)} + ε (e_ij) − {b_i0 [k_i, ρ_i, a_i, x_n (s_j0), z_j] + ε (e_ij0)}

(7)

A utility-maximizing fire manager will use information source j if the expected net benefit is positive (i.e., if V* > 0),

V* = {b_i [k_i, ρ_i, a_i, δ_i, x_n (s_j), z_j] − c_in (s_j, k_i, ρ_i, a_i, δ_i, z_j)}
− {b_i0 [k_i, ρ_i, a_i, x_n (s_j0), z_j] + [ε (e_ij) − ε (e_ij0)]} > 0

(8)

and will not if V* ≤ 0.

We do not observe the expected value to manager i of using information source j. We do, though, observe whether or not manager i uses information source j. If ε(e_ij) and ε(e_ij0) have Gumbel distributions, their difference η = ε(e_ij) − ε(e_ij0) will have a logistic distribution [34]. We can express V* as a linear function

V_ij* = α + k′β_k + a′β_a + r′β_r + δ′β_δ + nβ_n + z′ β_z + dβ_d + dz′β_zd + η

(9)

where terms β_k, β_a, β_r, β_δ, β_n, β_z, β_d, and β_zd are regression coefficients to be estimated and

• α = a regression constant term;
• k = [0, 1] dummy variables for age, education, and experience categories that affect the benefits and costs of using a given information source;
• a = [0, 1] dummy variables denoting the agency the fire manager works for;
• r = [0, 1] dummy variables for denoting different jobs or roles in the fire management system a fire manager may have;
• δ = [0, 1] dummy variables denoting the dispatch center where a fire manager operates;
• n = the total number of decisions a fire manager makes;
• z = [0, 1] dummy variables denoting attributes of information sources that may affect their use;
• d = a dummy variable that equals zero if the choice of using or not using an information source is made before the fire season and equals one if the choice is made during the fire season;
• dz = interaction terms between the fire season variable d and information source variable z.

The dz terms account for the possibility that different types of information will have different utilities for the fire manager at different times of the year. For example, before fire season, seasonal climate or drought forecasts may be relatively useful. During fire season, fire management-specific information sources and fire decision support tools (DSTs) may be in greater demand.

The variable V*—the expected benefit of using an information source—is a latent variable. That is, it is not directly observable. Economic theory suggests managers would not use an information source if they did not expect the benefits to outweigh the costs. Our approach models actual information use as determined by this unobserved variable. Other studies have attempted to estimate potential benefits from information use in fire management [24,35,36]. These, however, are ex-ante studies, considering hypothetical or potential use of information. As such, they do not measure economic values of actual improved decision-making. Studies that attempt to estimate the economic value of actual information use are more rare and focus on agricultural applications [17,37]. The present study is more in line with studies that attempt to predict use of information based on inferred benefits [22,23,38].

2.3. Survey Methods and Data

An internet survey instrument was developed using Qualtrics to collect data. Our target survey population comes from The Southwest Area Interagency Fire, Aviation, and Dispatch Directory. It is produced by the Southwest Coordination Center (SWCC) with input from local interagency dispatch centers and the agencies that they serve. The Directory represents a comprehensive list of fire managers in the Southwest. The SWCC is a Geographic Area Coordination Center (GACC) serving the Southwest (primarily Arizona and New Mexico). GACCs are responsible for coordinating wildland fire and other incident resources within defined geographic areas of the United States. Personnel in the Directory include the dispatch workforce; those who participate in interagency coordination, including incident support and logistics professionals; and fire and aviation program managers.

Most of the population participates in making or supporting strategic and/or tactical wildland fire management decisions. The Directory includes potential respondent names and job titles, along with the agencies they work for and the dispatch centers to which they are attached. Federal agencies include the Bureau of Indian Affairs, Bureau of Land Management, Fish & Wildlife Service, Forest Service, National Oceanic and Atmospheric Administration (NOAA), and the National Park Service. State agencies include the Arizona Department of Environmental Quality (ADEQ); Arizona Department of Forestry and Fire Management; and New Mexico Energy, Minerals and Natural Resources Department Forestry Division. In addition, there were a small number of tribal fire management and forestry agencies. Positions listed in the Directory that were not directly related to fire management decisions were removed from our target population. These included accountants, budget analysts, and clerical staff. After these removals, we had email information for a target population of 485 Southwest wildland fire management professionals.

A draft survey instrument was pre-tested by colleagues in the University of Arizona Cooperative Extension and affiliated with the University of Arizona’s Climate Assessment for the Southwest (CLIMAS). Some survey questions were revised in response to comments and suggestions. An initial email was sent to target respondents that informed them of the survey’s purpose. Potential respondents were then sent a follow-up email with a survey link in the second week of October 2021. Respondents were sent up to eight subsequent reminders. The survey was conducted for five weeks, ending after the third week of November.

Of the 485 people contacted, four responded that they did not use weather and climate information as part of their jobs, while another four responded that they were no longer in jobs where that was the case. These eight were not part of our intended target population. This reduced the relevant population to 477. Of these, 206 people responded about their use of climate and weather information for fire management, representing an overall response rate of 206/477 = 43.2%. This was a higher response rate than for other surveys of wildland fire managers: [39], 34%; [40], 28%; [41], 24%; and [42], 17%.

While our target population is well-defined, it is also small. With a small population size, the response rate needs to increase to guard against nonresponse bias [43,44,45]. Particularly with internet surveys, there is a risk that the sample population will not be representative of the target population. We were, however, able to use information about our target population to evaluate the representativeness of our sample. From the Directory, there was information about the agencies where our entire target population worked. It was thus possible to assess whether the distribution of respondents by agency matched that of the entire target population.

While 206 respondents answered questions about their use of information sources, only 175 of these provided answers to questions about their personal attributes. The regression analysis of this study relied on data from this sub-sample of 175 respondents. There are two potential sources of nonresponse bias. The full sample may not be representative of the target population, or the regression sub-sample may also be non-representative. The distributions of respondents by agency in both the sample and sub-sample were quite close to the distribution for the target population as a whole (

Table 1. Distribution by agency of the total Southwest fire manager population, sample population, and regression sub-sample population with Chi² tests for differences in proportions between the target and sample populations.

Agency	Total Population	Sample Population	Sub-Sample Population
Forest Service	49.5%	47.6%	45.1%
Bureau of Land Management	15.9%	13.1%	14.3%
Bureau of Indian Affairs	12.2%	10.7%	12.6%
ADFFM	7.8%	9.7%	10.9%
National Park Service	5.0%	5.3%	5.7%
NM Forestry Division	2.9%	3.9%	2.9%
NOAA	1.5%	2.4%	1.7%
ADEQ	0.6%	1.5%	1.7%
Other	4.6%	5.9%	5.1%
Critical Value of Chi2 test		15.51	15.51
Chi2 statistic (eight degrees of freedom)		7.90	6.99
p-value		0.44	0.54

). While response rates were higher from agencies with fewer people in the Directory, absolute differences were small. Conversely, among non-respondents, agencies with fewer people in the Directory were somewhat underrepresented, while absolute differences were again small. The rank order of the agencies in the total population, sample, and sub-sample were the same.

To assess the representativeness of the survey data, Chi² goodness-of-fit tests were conducted comparing the agency distribution of the sample (n = 206) and the sub-sample (n = 175) against our target population of Southwestern fire managers (n = 477). The null hypothesis is that the agency proportions in the samples are the same as for the total population. For the full sample, the distribution did not differ significantly from the population: Chi² (8 d.f.) = 7.90 and p = 0.44. This was also true for the sub-sample Chi² (8 d.f.) = 6.99 and p = 0.54. Both values are well below the critical value of 15.51 for the test. We fail to reject the null hypothesis that the sample proportions are no different from the total population and that both the sample and regression sub-sample are representative of the Southwestern fire manager population, at least with respect to the agencies where respondents work.

2.4. Variable Description

Table 2. Descriptive statistics for variables used in regression analysis.

Variable	Proportion a	Variable	Proportion
Data/information source used	0.441	Work/environmental contexts
Age variables		During fire season	0.500
Age (<30)	0.006	Total decisions (mean)	4.223
Age (30–39)	0.171	Total decisions (standard dev.)	1.796
Age (40–49) b	0.451	Total decisions [min., max.]	[1, 7]
Age (50–59)	0.269	Data/information source type
Age (≥60)	0.103	General website/data portal b	0.152
Experience variables		Drought	0.182
Experience (<5 years)	0.029	ENSO	0.091
Experience (5–9 years)	0.057	Other outlook/forecast	0.152
Experience (10–14 years)	0.109	Fire decision support tool (DST)	0.121
Experience (15–19 years)	0.177	Other fire information source	0.303
Experience (20–29 years) b	0.486	Agency
Experience (≥30 years)	0.143	Forest Service b	0.451
Education variables		Bureau of Land Management	0.143
High school graduate	0.069	Bureau of Indian Affairs	0.126
Some college b	0.337	National Park Service	0.057
College degree	0.451	AZ Dept. of Forestry & Fire Mgmt.	0.109
Master’s/professional degree	0.126	NM Forestry Division	0.029
Doctoral degree	0.017	Other Agency	0.051
Self-reported job/role		NOAA	0.017
Administrator b	0.034	AZ Dept. of Env. Quality	0.017
Fire Manager (fuels and fire)	0.417	Dispatch Center variables
Fire Manager (suppression)	0.274	AZ—Dispatch Center (DC) c	0.109
Other job	0.274	AZ—Flagstaff (DC)	0.080
Researcher-defined job type		AZ—Phoenix DC	0.063
Modeler b	0.114	AZ—Prescott DC	0.046
Implementation	0.366	AZ—Springerville DC	0.063
Environmental Specialist	0.040	AZ—Tucson DC	0.091
Aviation	0.097	AZ—Williams DC	0.011
Logistics	0.131	NM—Alamogordo DC	0.086
Field Operations	0.177	NM—Albuquerque DC	0.074
Higher-Level Decision Maker	0.074	NM—Santa Fe DC	0.091
		NM—Silver City DC	0.097
		NM—Taos DC	0.046
		Other DC	0.023
		Southwest Coordination Center b	0.120

^a. All variables except total types of fire management decisions made are 0–1 binary variables. ^b. Default dummy variables omitted from regression equation to prevent perfect multicollinearity. ^c. Omitted from regression equation because of perfect multicollinearity.

shows descriptive statistics for variables used in the regression analysis along with those for omitted, default variables. Except for the total decisions made by a manager, all other variables are binary, 0–1 categorical variables. Respondents’ categorical selections were automatically coded as categorical variables in Qualtrics. To avoid perfect multicollinearity, one of the categories must be left out. Regression coefficients in the model show effects of difference from these reference, default categories. Respondents were asked about their use of 33 different data/information products separately for two parts of the year, before fire season and during fire season. Fire managers make different types of decisions before and during fire season. Before fire season there are more resource planning and hiring decisions, while during fire season there are more decisions regarding fire suppression. Data products also differ, with some focusing on seasonal climate or weather conditions and others providing decision support directly for fire suppression. We hypothesize that these latter sources will be in greater demand during fire season. Previous research suggests information use varies across seasons [25,26].

The information sources chosen were based on multiple factors. Several were specific data products or information sources from federal agencies that explicitly stated that their purpose was to facilitate fire management. Other federal or university sites that focus on weather and climate also note their relevance to fire management. Other sources were mentioned in the academic literature on information used for wildland fire management or were provided in passing during focus group discussions with fire managers [26].

Survey respondents were asked about their use of (i) general websites and portals, (ii) forecasts and outlooks, (iii) situation reports and information products, and (iv) decision support tools. For each of these four categories, respondents could select among a set of survey-provided sources. For each of the four types, respondents could list up to two “other sources.” Respondents could potentially name other sources up to 3296 times (2 “other” categories × 4 information source types × 2 seasons (before and during fire season) × 206 respondents). Other sources were noted only 21 times in total by respondents, 0.64% of the potential maximum. These other sources tended to be one-off responses with no single “other source” standing out as an important, missed source. These other responses represented less than 0.2% of the total information source hits in our data. These results suggest that our coverage of information sources was comprehensive. Of the 33 possible survey-provided choices, 79% were chosen by 20% or more respondents before fire season, while 82% were chosen by 20% or more respondents during fire season. This suggests that respondents were not provided with superfluous options.

Respondents were asked to “select which fire management decisions you either make, help to make, or provide information for (check all that apply).” Choices included:

• Hiring or extending fire crew personnel;
• Allocating personnel, tanker planes, or equipment;
• Requesting additional resources (severity requests);
• Making prescribed fire or wildland fire use decisions;
• Public awareness and issuing public notices;
• Briefing administrators, congressional staff, etc.;
• Other decisions.

The number of decisions made in the sample ranged from one to seven, with an average of 4.2 and a standard deviation of 1.8 (Table 2). We hypothesize that managers who make more decisions have a higher probability of accessing information sources. From a value of information perspective, with each additional decision to be made, there is potential benefit of seeking information to support that decision [25,26]. In the context of our model, in Equation (3), more decisions means that there are more opportunities for any single information source to be of value.

In the sample, there were at total of 11,550 possible “hits”—opportunities for a fire manager to use a specific information source (potential of 33 sources × 175 fire managers × 2 seasons). The actual percentage of hits was 44.1% (Table 2). Data were collected on fire manager personal characteristics (age, experience, education) as categorical variables reporting ranges for these values (Table 2). About 63% of respondents had 20 or more years of experience. Less than one percent were less than 30 years old. Education varied widely, while 45% had bachelor’s degrees, nearly 41% had either a high school diploma or some college, while the remaining 14% had higher degrees.

The value of information literature posits that the value of information will depend on the overall system where a decision maker operates [14,15,16]. This is captured in our study by two variables: the agency that the individual works for, and dispatch center where they work. Individual fire managers work for different agencies but are stationed at a particular dispatch center with authority over fire management in specific regions. So, managers from different agencies may work together at a common dispatch center. In the sample, fire managers operated out of dispatch centers in Arizona in Flagstaff, Phoenix, Prescott, Springerville, Tucson, and Williams and in New Mexico in Alamogordo, Albuquerque, Santa Fe, Silver City, and Taos. A small number had primary assignments at dispatch centers in adjoining states.

Fire managers can work across dispatch center boundary areas or states to reallocate and pool resources to manage fire. The Southwest Coordination Center (SWCC) coordinates activities among the other, smaller dispatch centers. The statewide Arizona Dispatch Center is responsible for state and private lands across the entire state. We discovered in preliminary regression analysis that this variable was perfectly collinear with other variables. Both the Arizona Dispatch Center and the SWCC were treated as a default dummy variable and excluded from regression equations. Both of these centers are responsible for broader geographical areas. From a value of information perspective, there may be economies of scale. Information from a specific source may be applied over a wider geographic area to greater benefit [25,26]. In the context of our model, in Equation (4), the marginal value of accessing a given source is an increasing function of the area over which that information would be applied.

The SWCC is responsible for coordination across the states of Arizona and New Mexico. More local dispatch centers still cover wide areas ranging between 6000 and 30,000 square miles. For context, average county size is 7600 square miles in Arizona and 3700 square miles in New Mexico. There is a parallel phenomenon with use of weather and climate information for agricultural decision-making. There is evidence that larger farms use more information as the costs of applying information gained over a larger area are small, while the benefits are greater [22,23]. Respondents were asked, “How would you describe your job within your agency or organization?” Choices were Agency Administrator, Fire Manager (fuels and fire), Fire Manager (suppression), and Other. A relatively large share (27.4%) responded, “Other.” We experimented with creating other job categories based on respondent job titles reported in the Dispatch Directory. Respondents were grouped into seven job types: Modeler, Implementation, Environmental Specialist, Aviation, Logistics, Field Operations, and Higher-Level Decision Maker (

Table 3. Researcher-defined job categories for regression analysis.

Job Category	Description	Job Examples
Modelers	Subject matter experts (SMEs) who contribute to a framework of integrated spatial fire modeling and fire science application. Human components of the modeling system. More indirect communication (via the system).	Fire Planner; Fire Prevention Specialist, Fire Ecologist
Implementation	Interface between high-level decision makers and the field where most decisions are made. (Incident Commanders are delegated to this role re: specific incidents.)	Fire Management Officer
(Fire) Environmental Specialists	SMEs who provide targeted fire environment expertise that may contribute to the larger “modeling system” but may also stand alone for certain decision purposes. Includes direct, targeted communication to decision makers.	Air Quality Forecaster; Smoke Management; Fire Weather Program Manager
Aviation	Strategic and tactical management of fire management’s “air force”.	Helicopter Coordinator; Aviation Operations Specialist
Logistics/ Dispatch Support	Work as part of the National Coordination System (NCS) to mobilize incident resources, both in preparation for and response to wildland fire; 80% reactive (i.e., decisions already made).	Dispatcher
Field Operators	The “boots on the ground.” Those who implement wildland fire management in the wildlands themselves. All mandatory early retirement job classes.	Fuels Specialist; Crew Coordinator
Higher-Level Decision Makers	“Line officer” or top of the decision chain in an agency or bureau. They are typically not experts in fire management, so information understanding is different—but the stakes are higher. Could be district, forest, state, or regional level.	Regional Fire Management Officer; Fire Management Coordinator

).

Different sources of information were grouped into six categories: (1) general websites and portals, (2) ENSO-specific products, (3) drought-specific products, (4) fire-specific sources (excluding fire decision support tools (DSTs), (5) fire DSTs, and (6) other forecasts and outlooks. The percentage of sources used varies considerably, ranging from 4% to 97% of respondents (

Table 4. Percentage of wildland fire managers surveyed using different information sources for fire management decisions by source type before and during fire season.

Information Source	% Using Source
General Websites/Portals	Before	During
Climate Prediction Center (CPC), National Weather Service	77	78
IRI (Int. Research Inst. for Climate & Society, Columbia Univ.)	5	4
Western Regional Climate Center	38	38
The Weather Channel|weather.com	36	42
National Weather Service	90	97
Average	49	52
Decision Support Tools (DSTs)
Interagency Fuel Treatment Decision Support System (IFTDSS)	26	28
Integrated Reporting of Wildland-Fire Information (IRWIN)	30	42
LANDFIRE—Landscape Fire & Resource Mgmt. Planning Tools	18	18
Wildland Fire Decision Support System (WFDSS)	46	71
Average	30	40
Drought
National Drought Mitigation Center	39	35
NIDIS (Drought.GOV)	26	24
US Drought Portal|US Climate Resiliency Toolkit	28	23
U.S Seasonal Drought Outlook	75	70
NOAA Evaporative Demand Drought Index (EDDI)	11	13
US Drought Monitor	61	63
Average	40	38
ENSO
NWS El Niño/La Niña Information|NWS	49	40
CPC El Niño/Southern Oscillation (ENSO) Diagnostic Discussion	30	24
IRI ENSO Forecast	10	7
Average	29	24
Fire-Specific
National Interagency Fire Center (NIFC) | Predictive Services	77	83
Department of the Interior of Wildland Fire	26	27
US Geological Survey Fire Danger Forecast	42	49
Southwest Coord. Center (SWCC)|SW Fire Potential Outlooks	83	91
National Significant Wildland Fire Potential Outlook (NIFC)	75	80
USFS Wildland Fire Assess. System Nat. Fuel Moisture Database	46	51
Wildfires Near Me	17	27
Incident Management Situation Report (NICC)	61	81
Fire Weather Briefing Page|NWS	61	79
InciWeb: the Incident Information System	47	75
Average	53	65
Other Outlooks/Forecasts
CPC Seasonal Temperature and Precipitation Outlooks	61	63
CPC Soil Moisture Outlooks	18	17
CLIMAS Southwest Climate Outlook	31	33
NWS Quantitative Precip. Forecasts Weather Pred. Center	38	42
IRI Seasonal Climate Forecasts	11	9
Average	32	33

). The sources vary depending on whether they are directly specific to fire management or indirectly related (e.g., via weather patterns). Information sources also differ in terms of the costs of their use (including time costs). General websites may only provide basic data, but this information can be accessed quickly. Conversely, DSTs may provide more tailored information but may require more formal training or may require additional time to input information. The National Weather Service general site was the most used information source for decision-making. On average, fire-specific sites had higher use rates, especially during fire season. No fire DST was used by more than half of managers before fire season. This is not that surprising. During fire season, 71% of managers used WFDSS. This also means that during fire season, 29% of managers did not use WFDSS. No other fire DSTs were used by more than 42% of managers during fire season. During fire season, the Weather Channel was right at the median of use at 42%.

Our regression specification interacts information source type variables with season to formally test whether the type of information used varies between seasons. We also measure how the probability of using any particular source type varies between seasons. In Arizona and New Mexico, the fire season has historically been defined as beginning in April (in southwestern low desert areas) and May (to the northeast) and running through October. For respondents, operationally the main difference is between a period of preparation and fire fuel management and a period of fire suppression.

Equation (9) is estimated as a pooled logistic regression. The data-generating process involves 175 fire managers deciding whether or not to use 33 information sources across two time periods. A standard logit model assumes all observations are independent and identically distributed. Here, this assumption is violated because choices made by an individual manager are likely correlated due to unobserved personal characteristics, organizational culture, or job attributes. Failing to account for this intra-cluster correlation would lead to a downward bias in the standard errors and inflating measures of statistical significance [46,47,48]. To account for the non-independence of observations, we used a pooled logit model with cluster-robust standard errors (using the vce(cluster) option in Stata 16). In the regressions, odds ratios (ORs) for variables and their significance levels were reported. The OR is the probability of an outcome occurring (here, the use of information), p, divided by the probability of it not occurring (here, not using an information source), 1 − p [49].

The OR measures how a respondent being in a particular category (e.g., age, education, or experience category) changes their odds of using an information source relative to a reference category. If the OR equals one, that category has no effect on the odds of use relative to the reference category. If the OR equals two, being in the category doubles the odds of use. If the OR is 0.5, being in the category cuts the odds of use in half. Outputs from the logistic regressions predict use or non-use of each information source j by each fire manager i. The adjusted count R² (ACR²) is

$${\mathrm{ACR}}^2 = {\displaystyle \frac{P_{correct}- P_{mode}}{1\mathrm{ }- P_{mode}}}$$

(10)

where

• Pcorrect = the proportion of correct predictions;
• Pmode = proportion of observations with the mode (most common) response.

The ACR² compares the logistic regression predictions to a “naive” model where one predicts that all responses were the same as the most common response. The regression model correctly predicted when manager i would use (or not use) a source 64.9% of the time. Sources were not chosen 55.9% of the time, however. If someone predicted that no sources would ever be selected, that would be correct 55.9% of the time. The ACR² measures the percentage reduction in prediction error from using the regression model relative to this naive model. If the regression predictions were no better than the naive model, then ACR² = 0. If the regression predicted outcomes perfectly, then ACR² = 1.

3. Results

Results for variables significant at the 5% level (or less) are shown in boldface (

Table 5. Logistic regression of fire manager use of individual information sources before and during fire season (variables with odds ratios significant at p < 0.05 shown in bold face).

Number of observations = 11,550; adjusted count R2 = 0.204
Percent correctly predicted = 64.9%
Regression Variables	Odds Ratio	Std. Error	p-Value
During fire season	1.142	0.063	0.017
Total decisions	1.161	0.045	0.000
Drought	0.679	0.058	0.000
During X Drought	0.804	0.053	0.001
ENSO	0.394	0.048	0.000
During X ENSO	0.705	0.058	0.000
Other outlook/forecasts	0.468	0.040	0.000
During X Other outlook/forecasts	0.906	0.055	0.104
Fire decision support tool (DST)	0.424	0.049	0.000
During X Fire DST	1.385	0.113	0.000
Other fire information source	1.208	0.096	0.017
During X Other fire information source	1.441	0.079	0.000
Bureau of Land Management	0.886	0.164	0.515
Bureau of Indian Affairs	1.374	0.288	0.129
National Park Service	0.768	0.196	0.302
AZ Dept. of Forestry & Fire Mgmt.	0.524	0.159	0.033
NM Forestry Division	1.612	0.631	0.222
Other agency	1.147	0.176	0.371
NOAA	3.037	1.288	0.009
AZ Dept. of Env. Quality	1.832	0.887	0.211
Age (<30)	5.725	3.280	0.002
Age (30–39)	1.162	0.218	0.424
Age (50–59)	0.989	0.157	0.946
Age (≥60)	1.161	0.302	0.567
Experience (<5 years)	0.771	0.327	0.540
Experience (5–9 years)	1.016	0.311	0.958
Experience (10–14 years)	0.947	0.189	0.785
Experience (15–19 years)	1.342	0.206	0.055
Experience (>30 years)	1.240	0.220	0.226
High school graduate	0.549	0.151	0.029
College degree	0.793	0.100	0.065
Masters/professional degree	0.956	0.186	0.815
Doctoral degree	0.828	0.326	0.631
AZ—Flagstaff Dispatch Center (DC)	0.621	0.178	0.096
AZ—Phoenix DC	0.500	0.135	0.010
AZ—Prescott DC	1.134	0.301	0.635
AZ—Springerville DC	1.022	0.346	0.948
AZ—Tucson DC	0.461	0.127	0.005
AZ—Williams DC	0.457	0.125	0.004
NM—Alamogordo DC	0.669	0.190	0.157
NM—Albuquerque DC	1.213	0.345	0.498
NM—Santa Fe DC	0.954	0.232	0.846
NM—Silver City DC	0.703	0.183	0.175
NM—Taos DC	0.456	0.132	0.007
Other DC	0.234	0.093	0.000
Fire Manager (fuels and fire)	1.854	0.479	0.017
Fire Manager (suppression)	2.131	0.613	0.009
Other job	1.844	0.459	0.014
Implementation	0.705	0.140	0.078
Environmental Specialist	0.605	0.228	0.183
Aviation	0.727	0.210	0.271
Logistics	0.827	0.179	0.380
Field Operations	0.802	0.177	0.319
Higher-Level Decision Maker	0.811	0.201	0.396
Constant	0.486	0.165	0.000

). The ACR² of 0.2 means that by using the regression to predict outcomes, one would reduce prediction errors by 20%. The odds of using a given information source during fire season are nearly 14% greater than using it before (OR = 1.142). Every additional decision a fire manager makes increases the odds of using an information source by 16% (OR = 1.16). All the variables for information source type were highly significant. The terms interacting information source with season were also significant, save for the during the fire season interaction with other forecasts/outlooks. These results suggest that source types strongly influence information use and that this influence varies across seasons. Fire managers in the Arizona Department of Forestry and Fire Management had a much lower odds of using information sources, while those at NOAA had much higher odds. NOAA respondents tended to be IMETs (Incident Meteorologists), who are intensive information users. This source [50] showing an IMET with four laptops and two desktop screens open illustrates the point. Managers in their 20s had much higher odds of information use than older managers, while those with only high school education had lower odds than those at other education levels. Only one experience variable was even marginally significant (p = 0.055 for experience of 15–19 years). Each of the self-reported jobs/roles was significantly different, with greater odds than the default category, Administrator. For the researcher-defined job titles, only Implementation was marginally significant (p = 0.078), with odds lower than the default category of Modeler.

3.1. Effects on Probabilities of Information Source Use

For select variables (below) we consider factors that change the probability of information source use. Viewing the results in terms of probabilities is more intuitive than referring to odds. Using the delta method [51], one may use regression results to calculate how the predicted probabilities of using a given information source increases with increases in the total types of decisions a fire manager makes (Figure 1).

The baseline probability of using a given information source starts at 34% for a fire manager making a single type of decision. This probability rises roughly 3% per decision up to our maximum of seven possible in the survey. So, moving from the minimum to the maximum number of decisions raises the probability of source use by 19%. Moving from the median number of decisions made (4) to the maximum (7), would increase the probability of source use by 10%.

Figure 2 considers difference in information source use by type, controlling for other factors. Figure 2 reports results for both before and during fire season. For all predicted probabilities, p < 0.001. Predicted probabilities were highest for other fire-specific sources, followed by general websites and data portals. The probability of using a fire DST was only 40% during fire season. Figure 2 shows shifts in the relative reliance on specific source types moving from before to during fire season. Figure 3 considers the size and statistical significance of these shifts. Figure 3 shows the marginal effects (change in predicted probabilities of using a particular information source type) of the change from before fire season to during fire season. Shifting to during fire season, the probability of using specific ENSO sources falls 4% and use of drought-specific sources falls roughly 2%. The probability of using general websites/data portals increases 3%. The probability of using fire DSTs increases 9.7% and fire-specific sources increases by more than 11%. There is no significant change in use of other forecasts or outlooks.

Figure 4 reports the predicted probabilities of local dispatch centers relative to the SWCC and the Arizona Dispatch Center on the probability of using a given information source. Results range from significantly lower probabilities—for four local dispatch centers, the probabilities are about 15% lower—to no effect. The results generally support the hypothesis that dispatch centers with a larger geographical purview have higher probabilities of information use.

3.2. Hypothesis Tests of Variable Groups

Table 6. Joint hypothesis tests for groups of variable coefficients measuring combined effects.

Null Joint Hypothesis (Wald Test)	p-Value	Hypothesis Test Result (p = 0.05)
No season–information source type interaction effects	0.0000	Rejected
No education effects	0.1293	Failed to reject
No age effects	0.0491	Rejected
No experience effects	0.1910	Failed to reject
No dispatch center effects	0.0000	Rejected
No agency effects	0.0010	Rejected
No self-reported job/role effects	0.0543	Rejected at 6% level Failed to reject at 5% level
No researcher-defined job type effects	0.7062	Rejected

reports on the joint hypothesis tests of the joint significance of groups of variables. The hypothesis that the probability of choosing particular information source types does not vary across seasons is strongly rejected. While the hypothesis of no age effects is rejected, we fail to reject the hypothesis of no experience or education effects. Recall from Table 4 that the only significant (negative) education variable was high school education. There were no significant differences across the other education levels. While self-reported job/role was marginally significant (p = 0.0543), our researcher-defined job types were not. There were strong dispatch center and agency effects, however.

3.3. Robustness Checks

We experimented with various alternative model specifications. While results supported our baseline model that interacted source type with season, interactions of season with other variable groups (e.g., personal or job characteristics) were not statistically significant. We also considered a restricted model where variables for personal characteristics (age, education, experience) and job type (self-reported and researcher-defined) were omitted, while variables for season, information source type, season–information source type interactions, agency, dispatch center, and total number of decisions were included.

Based on a likelihood ratio test, the hypothesis test for the restricted model (excluding manager personal characteristics and job type variables) is strongly rejected (p < 0.0001) (

Table 7. Comparisons of model selection criteria for the full model and a restricted model excluding fire manager personal characteristics and job type variables.

	Full Model	Restricted Model
Akaike Information Criterion (AIC)	1.27	1.28
Bayesian information criterion (BIC)	−92,542.13	−92,777.11
Percent correctly predicted	64.9	64.3
Adjusted count R2	0.20	0.19
Likelihood ratio test results
Log likelihood function	−7198.00	−7287.23
Chi2 test statistic (20 df)	178.46
Likelihood ratio test: reject null hypothesis (restricted model) p < 0.0001

). Results based on information criteria are mixed. For both the Akaike Information Criterion (AIC) and the Bayesian information criterion (BIC), a lower value is preferred. While the AIC favors the full model, the BIC favors the restricted model. Both AIC and BIC select models based on fit but impose a penalty for adding variables (complexity). BIC imposes a higher penalty on adding variables and so favors simpler models more than AIC. While the personal characteristic and job variables are jointly significant, their inclusion does not greatly improve the model’s predictive power. The percentage of outcomes predicted correctly increases, but only from 64.3% to 64.9%.

Finally, we experimented with a random effects model to account for unobserved manager-specific effects. It did not, however, improve the prediction accuracy of our baseline model or alter baseline regression findings. In addition, random effects require that unobserved manager-specific differences be uncorrelated with observed manager attributes such as age, experience, and education. Otherwise, estimates will be biased and statistically inconsistent. Our pooled logit with cluster-robust standard errors avoids this risk.

4. Discussion

Much emphasis has been placed on the supply of fire management information. This has led to a large number of information sources, data platforms, and fire decision support tools (DSTs) [52]. Agencies have paid relatively less attention to the demand for information. Here, we focus explicitly on the demand side to consider what influences information use across multiple source types. Recent research has argued that fire manager use (or non-use) of decision tools may be explained in terms of the characteristics of the tool itself, of individual fire managers, of their organizations, and of the broader institutional and environmental settings where they operate [7]. Individual characteristics include expertise or capacity to use tools and their receptivity to new modes of operation. Organization characteristics include agency “culture” of information use. The approach applied in [7] is quite similar to ours, at least in terms of identifying key groups of factors driving information use. Expanding beyond decision support tools to weather, climate, and fire information sources more generally, we find evidence that the characteristics of the information source do indeed have a strong influence on use.

4.1. Study Limitations

In our sample, there were 11,550 possible “hits”—opportunities for a fire manager to use a given information source. Our statistical model correctly predicted the outcome (whether manager i used source j) 64.9% of the time. The adjusted count R² of the model was 0.2, meaning that using the regression to predict outcomes reduced prediction errors by 20% compared to assuming all responses were the mode response. In a statistical analysis predicting the total number of information sources used by fire managers, [25] report an adjusted R² from standard linear regressions ranging between 0.28 and 0.30. Studies of fire manager information use are dominated by those employing qualitative empirical methods with sample sizes too small for regression analysis [2,3,4,6,12,13]. In a study of use of economic information by different agricultural groups, [15] report adjusted R² values ranging from 0.11 to 0.23 in models predicting use of economic information. In a study of farmer use of weather and climate information, [23] report adjusted count R² values ranging between 0.11 and 0.38 across different types of information. Thus, while our results are on par with proximate studies, there remains significant room for improvement in model-based prediction.

To address perfect collinearity problems, we combined the Southwest Coordination Center (SWCC) and the statewide Arizona Dispatch Center as a combined default variable in measuring dispatch center effects. These two centers covered the largest geographical area across all centers. Regression coefficients for the dispatch variables measure differences from these two centers combined. We, thus, did not measure potential differences between the SWCC and the statewide Arizona Dispatch Center.

A fire manager’s probability of using decision support tools (DSTs) was lower than using other fire-specific information sources and general weather and climate websites and portals. Studies using qualitative research methods have identified higher training requirements and time constraints as barriers to DST use [2,3,4,5,7,8,13,53,54]. In a value of information framework, [25] introduce time costs and short time windows for decision-making as constraining information use. This study did not have detailed measures of training, usability, or time cost associated with different information sources. Future research, discussed below, would benefit from expanding measures beyond the simple categorical variables used here.

We failed to reject the null hypothesis that experience variables, as a whole, did not significantly influence information use. The role of experience is complex. On one hand, it could increase the ease of accessing and interpreting information. On the other hand, studies have found that fire manager experience can be a substitute for additional information acquisition [2,39,55,56]. Positive age effects on information use from the baseline only appeared for fire managers in their 20s. There were negative use effects on managers with only a high school diploma. Beyond that, there were no education effects. Age, experience, and education, generally defined, are common measures of “human capital” in economic analyses. These generic variables had only a limited impact on our model’s predictive ability. Again, insights from qualitative studies suggest that more fire management-specific measures of experience, training, and technical capacity may be needed. While we used a value of information framework to explain information use and non-use, our analysis did not assess how or how well information was used. Other research has used a value of information framework to examine intensity of information use [25].

4.2. Directions for Future Research

We found strong agency and dispatch center effects, suggesting the importance of organizations and organizational structure, reinforcing earlier research findings [5,7,57,58]. While we have found that information use differs across agencies and dispatch centers where fire managers work, our framework does not allow us to determine why there are differences. We found evidence that information use is greater with centers covering larger areas, but other factors are at play. Previous research has emphasized the fact that fire management decisions are often collaborative, group decisions [13,57,59]. Future research could explore this collaborative process further. Some qualitative studies have confined themselves to fire managers in the Forest Service alone [3,4,58]. Our results suggest interagency comparisons would be useful. Our analysis not only included different federal agencies, but also state agencies and tribal entities. Our coverage of tribal fire management entities was sparse. Future research could pursue better coverage here.

We found that fire managers used different types of information sources at quite different rates. Previous research suggests that lack of training, especially in use of decision support tools (DSTs), can be a barrier to use [2,3,4,5,13,58]. While we found managers were less likely to use DSTs than general information sources and other fire-specific sources, we did not have data on respondent levels of training that could affect use patterns. Given fluctuations in the types of information systems across the year, training provision could be targeted to times when fire managers have relatively more time to participate.

Recent research categorized DSTs by eight application metrics: Usability, User Support, Geospatial, Scalability, Accessibility, Time Investment for Output, Data Flexibility, and Collaborative Potential [59]. Tools are numerically measured based on a five-point ranking system. This opens up the possibility of expanding such a numerical system to a broader array of information sources. This would facilitate considering more specific attributes of information sources in statistical analysis.

Both this study and other studies have found DSTs are not the most frequently used sources of information for fire management. Much of the literature has been devoted to understanding and overcoming adoption barriers. This amounts to studying why fire managers do not use less-used platforms. This, however, ironically misses analysis of the most-used sources. A fruitful alternative would be to also study what sources fire managers are actually using (for example, [60]), to better understand the benefits of such sources.

Our human capital variables—age, experience, and formal education—while jointly significant statistically, did not add much to the predictive power of our statistical model. Future research could deeply explore the types of experiences fire managers have and more specific training they have had, rather than general education. This could include their personal histories in a location and experiences with past fires.

5. Conclusions

Because we examine 175 fire managers’ choices to use or not use 33 information sources, both before and during fire season, we are evaluating literally thousands of information use decisions. Examining 11,550 decisions rather than a small number of key informant perceptions adds to the statistical power of our analysis. Moreover, qualitative studies tend to emphasize dominant themes and overall patterns. Our statistical approach is better suited to examining the role of heterogeneity across fire managers. While our model correctly predicted two-thirds of the outcomes of these thousands of choices, there is still much our model does not explain. Many of our findings are consistent with the value of information literature and reinforce and support the findings of studies using qualitative empirical methods to understand fire manager information use. Our study illustrates that there is not a simple either/or choice between qualitative and more statistical empirical methods. Rather, our results suggest strong complementarity between the two approaches, each reinforcing the other.