Comparative Assessment of Wildland Fire Rate of Spread Models: Effects of Wind Velocity

Abstract

Wildland fire rate of spread prediction models are important tools for the effective coordination of resident evacuation and fire suppression efforts. A comparative assessment of ten empirical and semi-empirical rate of spread prediction models is performed, using a selection of 203 laboratory experiments of surface spreading fires; special emphasis is given to the effects of external wind velocity. Five of the evaluated models have been developed using measurements obtained in laboratory-scale tests; these models are combined with two supplementary sub-models that account for the effects of wind. In addition, a selection of five empirical models that have been developed using large-scale field tests are also assessed. The performance of the ten prediction models is evaluated, both qualitatively and quantitatively, by employing a range of statistical error metrics. The laboratory-developed models are found to exhibit high sensitivity on low fuel load values, when no external wind is present, as well as on low packing ratios and high wind velocity values. The field-developed models exhibit significant discrepancies against the experimental data, due to the use of specific parameters regarding fuel type, scale and wind velocity.

1. Introduction

Wildland fires may have a devastating impact on society and the natural environment. In this context, the study of wildland fires has the aim of providing detailed information on the physical and chemical phenomena, spanning multiple spatial and temporal scales, that characterise the respective initiation, spreading and suppression mechanisms. The prediction of the wildland fire rate of spread (ROS) is important, since knowledge of the spatial–temporal evolution of a wildland fire front provides valuable information for a broad range of activities, ranging from fire defence to civilian evacuation. Due to the complexity of the phenomena involved, along with the requirements for faster-than-real-time predictions, various approaches [1,2,3] have been developed to model and predict the wildland fire ROS.

The two main modelling methodologies that are extensively used in operational environments are the “empirical” and the “semi-empirical” approach. These models comprise simple mathematical equations that yield a “quasi-steady state” value of ROS (R), as a function of relevant parameters for fuel, atmosphere and topography characteristics. Both approaches utilise empirical data, originating from either small-scale laboratory experiments or large-scale field tests, such as experiments, prescribed burns or actual wildland fire events. Semi-empirical models are based upon an energy balance equilibrium; however, they do not distinguish among the different modes of heat transfer, as “physical” models do. On the other hand, empirical models are essentially statistical correlations, connecting the empirical data to the ROS. Nevertheless, physical reasoning may not be absent from empirical modelling, since the selection of the functional forms that mathematically describe a parameter’s effect on ROS is commonly based on previous theoretical analyses. It has been shown [4] that the appropriate selection of functional forms can result in successful model extrapolation beyond the range of the specific empirical data used for model development. Both empirical and semi-empirical approaches present, due to their intended simplicity, a range of inherent limitations. One of the most important limitations is the fact that they commonly neglect the dynamic behaviour of the fire, which is known to change with time, even when the boundary conditions regarding the fuel, atmosphere and topography parameters remain the same [5].

Due to the various macroscopic forms that a natural environment can manifest, e.g., forests, grasslands, woodlands, etc., specific empirical and semi-empirical models have been developed for different environments. In this context, “surface” ROS prediction models are utilised in environments where aerial fuel is absent; the existence of a canopy adds complexity to the behaviour of the fire. Furthermore, even among the broad range of surface fuels available, the respective models may be focused on either “generic” surface fuel types or a “specific” fuel type. Generic surface-fuel-type models are usually developed using laboratory data, which are characterised by a well-defined and controllable experimentation process but represent the wildland fire phenomena at a reduced scale. On the contrary, specific-fuel models are commonly developed using field data, which represent the phenomenon at its real scale, while lacking the experimental control of laboratory tests.

Among the various atmospheric properties that influence a wildland fire’s ROS, wind velocity is of particular significance. This is mainly due to its dynamic behaviour, since the magnitude or direction of the wind may change constantly. Wind velocity is incorporated into the majority of ROS models, either directly, as a specific parameter, or indirectly, through a “wind correction” sub-model that specifically accounts for the effects of wind and is used to “modify” the predicted ROS value for quiescent conditions. It is common for field-developed models to directly incorporate the effects of wind velocity, since quiescent wildland fires are generally characterised by relatively low ROS values, compared to fires spreading under external wind conditions.

In this study, a selection of laboratory and field-developed models, as well as wind correction sub-models, are evaluated against laboratory data of surface fires found in the open literature. The main purpose of this work is to comparatively assess the performance of the laboratory-developed models combined with the supplementary wind models, as well as the field-developed models, against well-controlled laboratory experiments, aiming to evaluate their ability to yield ROS values with acceptable accuracy. Similar validation studies have been performed in the past, such as a qualitative comparison of laboratory- and field-developed models against laboratory data with the effect of wind and slope combined [6]; a quantitative comparison of laboratory-based models against laboratory data on live vegetation fuel, again, with the aforementioned effects combined [7]; a broad collection of validation studies of field-developed models against field data [8]; as well as a quantitative comparison of laboratory- and field-developed models against laboratory data under the presence of wind [9]. In this context, the current work is mainly focused on the quantitative comparison of several contemporary laboratory- and field-developed models against a broad range of laboratory-scale surface fires, with a special emphasis on the effect of wind velocity on their prediction quality.

2. Materials and Methods

In this work, a selection of five laboratory-developed, generic surface fuel ROS prediction models, along with two wind correction sub-models, and a selection of five field-developed fuel-specific models are assessed against laboratory experimental data found in the open literature. The laboratory-developed models are the well-known semi-empirical models of Rothermel [10], its re-examination from Wilson [11] and the model of Catchpole et al. [12], as well as two empirical models of Rossa and Fernandes [13]. The wind correction sub-models are the empirical correction factor of Rothermel [10] and the empirical wind model of Rossa and Fernandes [14]. The investigated field-developed empirical models are the model of Burrows et al. [15], the models of Anderson et al. [16], the model of Fernandes et al. [17], as well as the Canadian Forest Fire Behaviour Prediction System (CFFBPS) [18]. The general form of each model is presented in

Table 1. Name, type, general formula and reference of the selected models.

Model Type	Model	General Form	Ref.
ROS models developed using laboratory tests	L1	$R_0=\frac{I_{R,1}\left(\sigma ,\beta ,h,M,{m^{″}}_n,S_e\right){\xi }_1\left(\sigma ,\beta \right)}{L_1(\sigma ,{\rm M},{\rho }_b)}$	[10]
	L2	$R_0=\frac{I_{R,2}\left(\sigma ,\beta ,\delta ,h_v,M,{m^{″}}_d,Q_{p,}{Q}_w\right){\xi }_2\left(\sigma ,\beta \right)}{L_2(\sigma ,{\rm M},{\rho }_b,Q_{p,}{Q}_w)}$	[11]
	L3	$R=\frac{I_{P,3}\left(\sigma ,\beta ,M,U\right)}{L_2(\sigma ,M,{\rho }_b,Q_{p,}{Q}_w)}$	[12]
	L4	$R_0=A_4{M}^{B_4}{\delta }^{C_4}$	[13]
	L5	$R_0=A_5{M}^{B_5}{\delta }^{C_5}\ln \left(D_{5}s\right)$	[13]
“Wind correction” sub-models	W1	$R=R_0\left(1+{\mathsf{\Phi }}_W\right),{\mathsf{\Phi }}_W=A_6\left(\sigma ,\beta \right)U^{B_6\left(\sigma \right)}$	[10]
	W2	$R=f_{il}{R}_0{R}_u,R_u=1+A_7{U}^{B_7}{\dot{m}}^{\prime C_7}{\left(\frac{\delta }{H_f}\right)}_0^{D_7}$	[14]
ROS models developed using field tests	F1	$R=A_8\frac{U_z^{B_8}F{C}^{C_8}}{M^{D_8}}$	[15]
	F2	$R=\frac{A_9{U}_z^{B_9}{\delta }^{C_9}{e}^{D_{9}M}}{1+E_9{e}^{F_9{W}^2}}$	[16]
	F3	$R=\frac{A_{10}{U}_z^{B_{10}}{\rho }_b{}^{C_{10}}{e}^{D_{10}M}}{1+E_{10}{e}^{F_{10}{W}^2}}$	[16]
	F4	$R=A_{11}{U}_z^{B_{11}}{e}^{C_{11}M}{\delta }^{D_{11}}$	[17]
	F5	$R=a{(1-e^{-b\cdot ISI})}^c$	[18]

, while the respective detailed equations, values of the various parameters and a description of all symbols are given in the Appendix A and Appendix B.

2.1. Semi-Empirical, Laboratory-Developed ROS Models

The semi-empirical models of Rothermel (L1), Wilson (L2) and Catchpole et al. (L3) are based on the theoretical work of Frandsen [19], presented in 1971. In Frandsen’s work, the steady-state ROS (R) of an (infinite-width) advancing flaming front was expressed (Equation (1)) as the ratio of the heat flux transferred from the burning front to the unburned fuel, i.e., the propagating heat flux (I_P), over the heat per unit volume required by the fuel volume to ignite (L_ig).

$$R=\frac{I_P}{L_{ig}}$$

(1)

In 1972, Rothermel [10] advanced Frandsen’s concept to a fully operational model; the proposed model is considered a “milestone” in the field of wildland fire science. Rothermel’s model focuses primarily on the “fundamental” physical mechanisms of fire spread due to the fuel characteristics, while the effects of wind and slope are considered to be “secondary”, treating them in a purely empirical manner. More specifically, the model introduces the concept of reaction intensity (I_R) which corresponds to the heat flux generated from the fire front; in this case, the propagating heat flux (I_P) is a fraction of I_R, defined as the propagating heat flux ratio (ξ). Through mathematical manipulation, assumptions and theoretical reasoning, the model expresses the ROS under quiescent and horizontal conditions (R₀) as a function of physical properties that can be measured, calculated or correlated to measured fuel parameters through laboratory experimentation (c.f.

Table A1. Detailed form of L1 model functions.

Function	Units
$I_{R,1}={\mathsf{\Gamma }}^{\prime }{m^{″}}_{n}h{\eta }_{M,1}{\eta }_S$	(kJ/min·m2)
${\mathsf{\Gamma }}^{\prime }=\frac{{\left[\frac{\beta }{0.20395{\sigma }^{-0.8189}}\exp \left(1-\frac{\beta }{0.20395{\sigma }^{-0.8189}}\right)\right]}^{\frac{1}{6.7229{\sigma }^{0.1}-7.27}}}{0.0591+2.926{\sigma }^{-1.5}}$	(min−1)
${m^{″}}_n=m^{″}\left(1-S_T\right)$	(kg/m2)
${\eta }_{M,1}=1-2.59\frac{M}{M_X}+5.11{\left(\frac{M}{M_X}\right)}^2-3.52{\left(\frac{M}{M_X}\right)}^3$
${\eta }_S=0.174S_e^{-0.19}$
${\xi }_1=\frac{{\mathrm{e}}^{\left(0.792+3.7597{\sigma }^{0.5}\right)\left(\beta +0.1\right)}}{192+7.9095\sigma }$
$L_1={\rho }_b\epsilon \left(581+2594M\right)$	(kJ/m3)
$\epsilon ={\mathrm{e}}^{-\frac{4.528}{\sigma }}$
σ	(cm−1)

).

In 1990, Wilson [11] proposed a reformulation of Rothermel’s model, based on an extended set of laboratory experimental data and using different assumptions and theoretical reasoning compared to the original model. The two main new concepts of Wilson’s model are that, on the one hand, the reaction intensity (I_R) only originates from the flaming combustion of the fire front, and, on the other hand, fuel ignition initiates when the fuel pyrolysis is completed. In addition, Wilson proposed different functional forms and property dependencies on I_R and ξ (c.f.

Table A2. Detailed form of L2 and L3 model functions.

Function	Units
${\xi }_2=1-{\mathrm{e}}^{-0.17\sigma \beta }$
$L_2={\rho }_b\epsilon \left(Q_p+M{Q}_w\right)$	(kJ/m3)
$I_{R,2}={{\mathsf{\Gamma }}^{\prime }}_2{m}^{″}{h}_v{\eta }_{M,2}$	(kJ/min·m2)
${{\mathsf{\Gamma }}^{\prime }}_2=0.34\sigma {\left(\sigma \beta \delta \right)}^{-\frac{1}{2}}{\mathrm{e}}^{-\frac{\sigma \beta }{3}}{P}_f\left(n_x\right)$	(min−1)
$P_f\left(n_x\right)=\frac{1}{1+e^{-\pi \frac{n_x-{\overline{n}}_x}{1.2\sqrt{3}}}}$
$n_x=\frac{\ln \left(\sigma \beta \delta h_v/Q_w\right)}{M+\frac{Q_p}{Q_w}}$
${\eta }_{M,2}={\mathrm{e}}^{-kM}$
$I_{P,3}=\left(495.5+1934U^{0.91}\right){\mathrm{e}}^{-\frac{800}{\sigma }}{\beta }^{0.501}{\eta }_{{\rm M},2}$	(kW/m2)
σ	(cm−1)

).

The model of Catchpole et al. [12] is a semi-empirical model, also based in the original work of Frandsen, developed from wind-tunnel experimentation. In contrast to the previously presented models, this model does not consider the effect of wind as a “secondary” effect, but instead, it incorporates wind effects directly to the propagation heat flux (I_P). The latter, in this case, is not expressed as the product of I_R·ξ, but it is directly empirically correlated to the fuel characteristics, as well as to wind velocity. It is noted that the functional form for the wind velocity correction can also yield results for the case of quiescent conditions. Moreover, the L3 model keeps the proposal of Wilson [11] on the onset of fuel ignition after the completion of pyrolysis. Finally, this model considers the effect of fuel bed width (W) by means of a proportional multiplier of R for fuel beds that have a width lower than 1 m.

2.2. Empirical, Laboratory-Developed ROS Models

The two laboratory-developed, empirical models investigated in this work are the models proposed by Rossa and Fernandes [13]. The authors presented a couple of empirical models for the prediction of the “base” (i.e., quiescent conditions and flame propagation on a horizontal terrain) ROS on dead and alive generic surface fuels. Both models (L4 and L5) consider the effects of fuel moisture content (M) and fuel bed depth (δ), while the second model (L5) also accounts for the effect of specific surface (s), a quantity that corresponds to the ratio of surface area-to-volume ratio (σ) over the fuel element particle density (ρ_p).

2.3. Wind Correction Empirical Sub-Models

Wind correction sub-models are essentially supplementary models that specifically account for the additional effects of wind; they are commonly applied to the ”base” ROS (R₀) models, aiming to “correct” their values for wind effects. The wind correction sub-model of Rothermel [10] (W1) makes use of a correction factor (Φ_W), which is estimated from laboratory experiments and is empirically correlated to the wind velocity (U) and to other fuel and fuel bed parameters (c.f.

Table A3. Detailed form of W1 model function.

Function	Units
${\mathsf{\Phi }}_W=7.47{\mathrm{e}}^{-0.8711{\sigma }^{0.55}}{3.281}^{0.15988{\sigma }^{0.54}}{\left(\frac{\beta }{0.20395{\sigma }^{-0.8189}}\right)}^{-0.715{\mathrm{e}}^{-0.01094\sigma }}{U}^{0.15988{\sigma }^{0.54}}$

). The dependence of Φ_W on the fuel characteristics is associated with the physical observation that when the fuel bed is less compact, heat transfer from the burning front to the fuel bed is more efficient and vice versa in the case of a closely packed fuel bed.

Rossa and Fernandes developed an empirical sub-model (W2) to account for the effects of wind [14]; despite its empirical nature, it is based on physical reasoning in an attempt to describe the change in fire spread mechanisms under the effect of wind. The model recognises that one of the major effects of external wind on the flame front is the modification of the angle between the flame and the unburned fuel bed. For quiescent conditions, the flame tends to tilt backwards (upstream) due to increased air entrainment from the downstream direction, induced by the asymmetric temperature field, upstream and downstream of the flame front. In this case, fuel heating due to flame radiation is less important compared to radiation from the burning fuel bed. When external wind is present, the flame front is “pushed” towards the unburnt fuel, decreasing the angle between them, thus resulting in an increased contribution of the flame radiation heating. Radiation heating depends on the emissivity and the temperature of the flame envelope, as well as the envelope’s height and relative angle with the unburnt fuel bed. Thus, an increase in wind velocity results in an increase in the heat transfer rate from the flame to the unburnt fuel, and by extension, an increase in the observed ROS (c.f. Equation (1)). However, according to Byram [20], and as shown in Equation (2), an increase in ROS results in an increase in the fireline intensity (I_B), which is an alternative expression for the energy generated, similar to I_R. Subsequently, an increase in I_B implies the increasing momentum of the buoyant plume behind the fire front, which tends to pull the flame away from the unburned fuel bed, thus decreasing the above-mentioned flame geometry characteristics (height and angle) and the contribution of flame radiation heating. Thus, external wind initiates a “competition” between the pushing of the flaming front towards the fuel bed and the pulling of the flaming front by the buoyant plume whose strength is increased, indirectly, by the wind. By assuming that the momentum of the buoyant plume is analogous to the rate that the fuel is added to the flame (${\dot{m}}^{\prime }$), the W2 model introduces a correction factor (R_U) that is empirically correlated to the wind velocity (U), ${\dot{m}}^{\prime }$ and the ratio of fuel bed to flame height (δ/H_f)₀; the latter parameter quantifies the flame extension above the fuel bed. The model assumes that ${\dot{m}}^{\prime }$= m^″·R₀, while the (δ/H_f)₀ parameter is empirically estimated as a function of FMC (c.f.

Table A4. Detailed form of W2 model function.

Function	Units
${\left(\frac{\delta }{H_f}\right)}_0=0.1779+3.713\cdot {10}^{-3}M$

), based on a previous work of Rossa and Fernandes [21].

I_B = h·m^″·R

(2)

2.4. Empirical, Field-Based Models and Wind Adjustment Factor

The remaining five empirical models (F1–F5) have been developed using field experiments or observational studies of actual wildland fires on specific flora environments. Burrows et al. [15] modelled the flame ROS on spinifex grassland, a horizontally discontinuous surface fuel, yielding a “parsimonious” three-parameter model (F1). Anderson et al. [16] presented a group of empirical models (F2 and F3) based on a large database of shrubland fires. Fernandes et al. [17] developed a model for predicting the flame ROS on maritime pine stands (F4). Lastly, the Forestry Canada Fire Danger Group [18] developed the Fire Behaviour Prediction (FBP) System from various types of field experiments and observational studies (F5). This model has been developed to accept fuel-specific parameters from various fuel models of Canadian flora environments (c.f.

Table A5. Detailed form of F5 model function.

Function	Units
$ISI=A_{12}{e}^{B_{12}{U}_z}\left\{C_{12}\cdot e^{D_{12}M}\left[1+E_{12}{M}^{F_{12}}\right]\right\}$
Uz	(km/h)

).

The nominal velocity required as an input by the various field-developed ROS prediction models may differ due to the wind velocity profile developing in the Atmospheric Boundary Layer (ABL). In the ABL, the axial wind velocity decreases with decreasing height. As a result, the wind velocity measured at a given height above the flame may be significantly higher than the wind velocity close to the flame, which is the region where the external wind directly affects the flame front. On the other hand, in laboratory experiments, the velocity profile in a wind tunnel depends on the characteristics of the wind tunnel itself; in this case, the nominal velocity usually refers either to the “free stream velocity” of a developing flow or the “mean velocity” of a fully developed flow. Due to the limited wind tunnel dimensions, the reference velocity is a better approximation of the wind velocity that, being close to the fire front, is directly affecting it. Therefore, in order to use the field-developed models to predict ROS values obtained in laboratory tests, the wind tunnel velocity needs to be adjusted to an “equivalent” velocity that corresponds to the actual height that atmospheric wind velocity is measured as for each model. This wind velocity “adjustment” is performed using the methodology proposed by Albini and Baughman [22], where the Wind Adjustment Factor (WAF) is introduced. The latter parameter corresponds to the ratio of the wind velocity at mid-flame height ($\overline{\overline{U}}$) to the atmospheric velocity measured at height z (U_z). The Wind Adjustment Factor is estimated using Equation (3) [23] and is formulated under the assumption that the flame height above the fuel bed is considered to be approximately equal to the height of the fuel bed [24].

$$WAF=\frac{\overline{\overline{U}}}{U_z}=\frac{1.83}{\ln \left(\frac{z+0.36\delta }{0.13\delta }\right)}$$

(3)

The adjustment of the wind tunnel velocity (U) to the atmospheric velocity measured at height z (U_z) is based on the additional assumption that the wind tunnel nominal velocity, due to the scale of the laboratory fires, is approximately equal to $\overline{\overline{U}}$, an assumption that is commonly made in studies of similar nature [6].

2.5. Laboratory Experimental Data

The ten investigated ROS models (L1–L5 and F1–F5) and the two wind correction sub-models (W1 and W2) were evaluated against a broad set of fuel bed surface fire laboratory experiments, available in the open literature [25,26,27,28,29]. More specifically, 203 individual fire tests were used to assess the prediction accuracy of the investigated models and model combinations; 166 of these fire tests were performed under the effect of external wind, whereas the remaining 37 fire tests corresponded to quiescent wind conditions. The specific set of experimental data used in this study was selected due to the fact that the respective research works either present all the parameters required by the investigated ROS models or allow for an estimation of these parameters through inferences and reasonable assumptions. A brief summary of the main characteristics of the fire tests used for the validation study is given in

Table 2. Summary of the experimental studies used for model assessment.

Ref.	Fire Tests	No-Wind/Wind Tests	Fuel Type
[25]	9	2/7	Pine needles (Pinus Pinaster)
[26]	6	0/6	Bamboo sticks
[27]	163	30/133	Pine needles (Pinus Ponderosa)/Excelsior
[28]	7	1/6	Pine needles (Pinus Sibirica)
[29]	18	4/14	Pine needles (Pinus Sibirica)

.

Due to inconsistencies between the required model input parameters and the information available on the laboratory tests, a number of assumptions had to be made. The fuel chemical properties, such as fuel heat content (h) and the heat content of the volatile gases (h_v) needed in the L1 and L2 models, respectively, were assumed to be 18,608 and 12,306 kJ/kg, respectively, based on information presented in references [6,10]. The heat of pyrolysis (Q_P), the moisture damping constant (k) and the equivalent characteristic moisture content (M_c) required for models L2 and L3 were taken from [12], based on the specific fuel type of each fire test. The moisture of extinction (M_x), the total mineral content (S_T) and the effective mineral content (S_e), needed for the calculation of I_R,1 of model L1, were assumed to be equal to 0.3, 0.0555 and 0.01, respectively, based on [10]. The parameter ${\overline{n}}_x$ required for the L2 model was assumed to be equal to 3, according to [11]. For the wind correction model W2, (f_il) was assumed to be 2.42 [14]. In terms of the F1 model, the fuel cover parameter (CF) was assumed to be equal to 1, since the fuel beds used in the experiments had a “continuous” fuel distribution. For certain fuel properties, i.e., h, S_t, S_e, M_x, h_v and f_il, a sensitivity analysis was performed, aiming to investigate the impact of their assumed values on the respective model performance.

Finally, for the F5 model, the C-6 fuel type was selected [18]. In terms of the FMC and the fuel load values reported in the respective experimental works, they were assumed to be on a “dead” basis, unless otherwise reported or inferred. Finally, due to the fact that the σ and ρ_p parameters of the Pinus Pinaster needles that were used in [25] were not reported, they were assumed to be equal to 3057 m⁻¹ and 511 kg/m³, respectively; these values were taken from laboratory tests performed on the same fuel [30,31]. The assumptions that were made in order to be able to use all the investigated models for all the considered fire tests (c.f.

Table A6. Values of empirical parameters.

Model	Parameter	Value	Model	Parameter	Value
L4, L5	A4	0.2859	F1	A8	40.982
	A5	0.1557		B8	1.399
	B4,5	−0.7734		C8	1.201
	C4,5	0.9440		D8	1.699
	D5	0.8173	F2, F3	A9	5.6715
W2	A7	2.143 × 10−5		B9	0.9102
	B7	1.710		C9	0.2227
	C7	−1.169		D9	0.0762
	D7	−1.166		A10	3.8320
F5	α	45		B10	1.0927
	b	0.0305		C10	−0.2098
	c	2		D10	0.0721
	A12	0.208		E9,10	9
	B12	0.05039		F9,10	0.00316
	C12	91.9	F4	A11	0.773
	D12	−0.1386		B11	0.707
	E12	4.93 × 10−7		C11	−0.039
	F12	5.31		D11	0.188

) may have affected, to a certain degree, the accuracy of the predicted ROS values.

3. Results

The performance of the investigated models against the selected set of laboratory experiments is assessed both qualitatively and quantitatively, by comparing the level of agreement of the “predicted” ROS values with the respective “measured” ROS values. The results are quantitatively evaluated by calculating the values of several statistical error metrics. The error metrics selected are the Mean Biased Error (MBE), the Mean Absolute Percentage Error (MAPE) and the Root Mean Square Error (RMSE) [17]. The magnitude and sign of the MBE indicate the tendency of the model to under- or over-predict the experimental data. The magnitude of the MAPE indicates the tendency of the model to diverge from the observed values, while being a metric of the scattering of the results. The magnitude of the RMSE provides similar information to that of the MAPE but shows higher sensitivity to large discrepancies. The formulas of the above error metrics are presented in Equations (4)–(6), where R_pred and R_exp are the predicted and measured ROS values, respectively, and n is the total number of experiments.

$$\mathrm{RMSE}={\left[\frac{{\displaystyle \sum _1^n{\left(R_{\exp }-R_{pred}\right)}^2}}{n}\right]}^{\frac{1}{2}}$$

(4)

$$\mathrm{MAPE}=\frac{{\displaystyle \sum _1^n\frac{\left|R_{\exp }-R_{pred}\right|}{R_{\exp }}}}{n}100\%$$

(5)

$$\mathrm{MBE}=\frac{{\displaystyle \sum _1^n\left(R_{pred}-R_{\exp }\right)}}{n}$$

(6)

Initially, the performance of the five laboratory-developed models (L1–L5) against the 37 no-wind laboratory tests is presented, followed by the performance of the ten laboratory-developed model/wind correction sub-model combinations (L-W) against the 166 fire tests with external wind. Then, the performance of the five field-developed models (F1–F5) against the external wind fire tests is presented; since the equations of these models cannot provide a ROS value for zero wind velocity, it was not possible to assess their performance against the no-wind fire tests. Finally, a sensitivity analysis focused on the assumed values for six characteristic model fuel properties is presented, aiming to investigate their impact on the prediction accuracy of the respective models.

3.1. Laboratory-Developed Models in Quiescent Conditions

In Figure 1, ROS predictions of the five laboratory-developed models are compared against the respective measured values, obtained in the 37 fire tests performed in quiescent conditions; the bold dashed line corresponds to “perfect agreement”, whereas the normal dashed lines represent the ±35% “acceptable” error intervals, as proposed in [4]. In general, the under-prediction of the measured ROS values is undesirable, especially when the respective model is used in actual operational environments. It is evident that the obtained predictions exhibit significant scatter; however, the majority of the predicted values lie within the ±35% error region. The L1 model shows a tendency for under-prediction, while the L2 model exhibits a slight tendency for over-prediction. The L3 model under-predicts the measured ROS values in the low region (R < 2 mm/s). Models L4 and L5 demonstrate relatively good prediction qualities.

The corresponding error metric values for all 37 no-wind fire tests considered are presented in

Table 3. Error metric values of the laboratory-developed ROS model predictions for the 37 no-wind fire tests.

Model	RMSE	MAPE (%)	MBE
L1	3.5	42.8	−2.8
L2	3.2	59.3	1.0
L3	4.7	71.6	1.5
L4	3.1	45.8	−0.2
L5	2.9	50.9	0.3

; text in bold corresponds to the minimum values for each error metric. The calculated error metrics agree with the aforementioned qualitative observations. For all the 37 no-wind test cases, the L1 model exhibits the larger negative MBE, while the L3 model exhibits the highest positive MBE and the highest RMSE and MAPE; this is mainly attributed to a group of highly over-predicted values (c.f. Figure 1). The L4 and L5 models present the lowest RMSE and MBE values, whereas their MAPE values are quite close to that of L1. Overall, based on the error metrics matrix, both L4 and L5 models exhibit the best prediction quality among the five investigated models.

Aiming to investigate the potential impact of certain physical parameters on the prediction quality of the investigated ROS models, the relative error of the predicted values is presented in Figure 2, as a function of the (dry) fuel load (m^″) for all the considered test cases. It is evident that the scatter of the predicted values is reduced with increasing fuel load, thus suggesting high sensitivity of the ROS models to relatively low fuel load values. Among the five ROS models, the L3 model exhibits higher sensitivity in low m^″ values, thus resulting in the relatively high error values reported in Table 1.

3.2. Laboratory-Developed Models, Combined with Wind-Correction Sub-Models, against External Wind Conditions

In Figure 3, predictions of the laboratory-developed models (L1–L5) combined with the two wind correction sub-models (W1 and W2) are compared against the respective measured values in the 166 fire tests with external wind.

The two wind correction models exhibit relatively different behaviours. The performance of W1 depends strongly on the ROS model that it is applied to. More specifically, model W1 has a tendency for under-prediction when applied to the L1 and L4 models, while, when applied to the L2 and L3 models, the W1 model tends to yield results that over-predict the measurements. When applied to model L5, the W1 correction yields the most accurate results. On the contrary, the W2 wind correction model does not exhibit a strong dependence on the underlying ROS model. It is evident that the W2 model yields highly over-predicted values, especially in the region of high ROS values (R > 100 mm/s).

In

Table 4. Error metric values of predictions using the laboratory-developed ROS models/wind correction combinations for the 166 external wind fire tests.

Models	RMSE	MAPE (%)	MBE
L1-W1	40.1	46.5	−25.3
L2-W1	51.5	59.2	14.8
L3-W1	91.8	49.3	30.8
L4-W1	40.2	52.0	−6.7
L5-W1	42.1	43.1	1.2
L1-W2	143.8	78.0	34.4
L2-W2	127.8	105.3	35.2
L3-W2	101.0	67.1	26.8
L4-W2	131.3	94.9	33.7
L5-W2	124.5	90.1	33.0

, the error metrics for all model combinations are presented. The absolute values of all calculated error metrics are generally higher than the respective values obtained for the no-wind cases (c.f. Table 3), thus echoing the increased difficulty in accurately predicting the ROS of a wildland fire in the presence of external wind. It is evident that the W1 wind correction model performs, in all cases, better than the W2 model. In general, lower error values are observed when the W1 sub-model is combined with models L1, L4 and L5; this is expected, since these specific models also yield the best results in the no-wind test cases (c.f. Table 3). Overall, the combination with the lowest error metrics is L5-W1.

Aiming to further investigate the impact of certain physical parameters on the prediction accuracy of the investigated models, the calculated relative error for all ten model combinations is presented in Figure 4 as a function of wind velocity (Figure 4, top) and packing ratio (Figure 4, bottom). It is evident that an increase in wind velocity results in an overall decrease in the prediction accuracy of model W2, while W1 seems to be unaffected. Moreover, the accuracy of W2 is decreased at very low wind velocities. Regarding the effect of packing ratio, it seems to have no effect in W1, while it tends to decrease the accuracy of W2 with decreasing β.

3.3. Field-Developed Models against External Wind Conditions

In Figure 5, ROS predictions using the field-developed models are compared to the respective measured values for the 166 test cases where external wind was present. In general, the predictions are significantly more scattered compared to the respective results of the laboratory-developed models obtained for the same fire tests (c.f. Figure 3). This is expected, since the field-developed models are strongly dependent on the specific fuel type used in the corresponding field tests, thus making their good performance in fire tests in which different fuel types are used quite a challenge. The F1 model significantly under-predicts the measured values, whereas the F5 model exhibits the strongest over-prediction. Model F4 is also slightly over-predicting. Model F2 shows a slight tendency for under-prediction, compared to that of model F1, while the F3 model results exhibit relatively smaller discrepancies.

The calculated error metrics for the five field-developed models against all 166 wind fire tests are presented in

Table 5. Error metric values of the field-developed ROS model predictions for the 166 wind fire tests.

Model	RMSE	MAPE (%)	MBE
F1	59.2	119.0	−24.0
F2	44.4	67.5	−21.2
F3	30.7	63.1	−3.9
F4	51.5	231.3	35.2
F5	280.0	1278.3	251.6

. The significant over-prediction tendency of model F5 is evident, since the respective error metric values are very high. The lowest error metrics are obtained when the F3 model is used; however, these values are generally higher than the respective values obtained using the laboratory-developed models (c.f. Table 4). The large scatter of the majority of the field-developed models (c.f. Figure 5) is reflected in the generally large MBE values.

3.4. Sensitivity Analysis

A sensitivity analysis was performed, aiming to investigate the impact of the selected values for certain model parameters on the prediction accuracy of the respective models. Six model parameters were considered in this study, namely the fuel heat content (h), the total (S_t) and effective (S_e) mineral content and the moisture of extinction (M_x) of model L1; the fuel’s pyrolysis gas lower calorific value (h_v) for model L2; and the ignition line length factor (f_il) used in wind correction W2. Towards this end, the value of each parameter was varied by ±25%; these variation limits were selected as an adequate representation of the “common” value range of the corresponding specific fuel properties. The results of the sensitivity analysis are presented in

Table 6. Absolute and relative change in the RMSE values, resulting from the ±25% relative change in the values of six model parameters.

Parameter	Model	−25% Change	Assumed Value	+25% Change
h	L1	4.42 (25.9%)	3.51	2.67 (−23.8%)
St	L1	3.46 (−1.5%)	3.51	3.56 (1.5%)
Se	L1	3.31 (−5.6%)	3.51	3.65 (4.2%)
Mx	L1	3.80 (8.3%)	3.51	3.31 (−5.7%)
hv	L2	2.82 (−13.0%)	3.24	4.70 (45.3%)
fil	W2 (L5-W2)	85.70 (−31.2%)	124.50	165.84 (33.2%)

, in the form of the change in the overall RMSE values when each parameter value was changed by ±25%. It is evident that variations in the fuel heat content (h) used in model L1, the fuel’s pyrolysis gas lower calorific value (h_v), used in model L2, as well as the ignition line length factor (f_il) used in model W2 (the model combination L5-W2 was used for the sensitivity analysis), result in moderate sensitivity of the obtained prediction accuracy, whereas the rest of the fuel parameters have less significant impacts. The relatively higher sensitivity on the h, h_v and f_il parameter values is expected, since the h and f_il parameters are essentially “multipliers” in the respective ROS equations (c.f. Table 1), whereas h_v is an additional input through a more complicated functional form. Overall, the calculated sensitivity of the investigated parameters is considered to be reasonable, since their assumed values do not have an unexpected strong influence on the obtained results.

4. Discussion

The overall quality of the obtained predictions suggests the inherent complexities associated with the accurate description of wildland fire ROS, due to the interacting complex physical phenomena that span a broad range of spatial and temporal scales.

In the absence of wind, the main characteristics affecting the ROS of a surface fire seem to be sufficiently described by available ROS models, since the majority of the investigated models yielded relatively accurate results. The best prediction quality was achieved by model L5; notably, this is the most recently developed model, having been introduced in 2018 [13]. This suggests the importance of continuously updating ROS models by constantly incorporating the latest research findings, with the aim of improving the available operational tools. The empirical model of Rossa and Fernandes (L5) suggests that, in the absence of wind, the effects of fuel bed characteristics can be better described by the fuel bed height (δ), whereas the effects of the fuel particles are better predicted by means of the specific surface parameter (s). In general, empirical models do not provide an in-depth insight into the nature of the occurring phenomena. On the contrary, the importance of the three semi-empirical models (L1, L2 and L3) rests in the fact that they mathematically describe the concepts of energy required for fuel ignition, energy generation from the fire front and energy transferred to the unburnt fuel, which have been shown to be the main physical/chemical processes that govern the wildland fire dynamics [32], and that the complexity of the wildland fire behaviour eventually lies in their interaction as well as their interdependence. In any case, due to the relatively small number of fire tests obtained in quiescent conditions (37), it is believed that a larger pool of experimental data would allow for an improved assessment of the relevant performance of the investigated models.

When external wind was present, the wind correction sub-model W1 yielded, in general, lower metric errors than sub-model W2. This is attributed to the different physical reasoning that the two wind correction models are based upon, regarding the effects of wind velocity and packing ratio. Sub-model W2 is mainly focused on the changes in the mechanism of radiative heat transfer under the effect of wind. In low-wind-velocity conditions, the effects of radiative heating are more pronounced than convective heating; however, at higher wind velocities, it has been proposed [33] that the mechanisms of fire spread change and the effects of convective heating become significant. In the W2 sub-model, the effects of convection are incorporated by means of the empirically fitted parameters A₇ or B₇ (c.f. Table 1); thus, there is less focus on the potential changes in convective heat transfer. This may be one reason for the increasing discrepancies exhibited by W2’s predictions with increasing wind velocity values (c.f. Figure 4). On the other hand, in the very low wind velocity region, the observed W2 discrepancies may be attributed to the fact that when the wind correction parameter R_U tends to zero, the predicted ROS value is largely affected by the parameter f_il (c.f. Table 1), thus leading to the potential over-prediction of the experimentally determined values. The effects of packing ratio are only taken into account by sub-model W1. The packing ratio value affects both radiation and convection; closed-packed beds are both blocking the radiation to the fuel bed’s bottom layer and decreasing the ventilation through the fuel bed. Thus, the packing ratio may be an important parameter that has to be included in generic ROS models that account for the effects of wind.

The field-developed ROS models exhibited rather divergent performances. The majority of field models showed strong tendencies for the over-prediction or under-prediction of the measured values, with the exception of the F3 model which yielded results of comparable accuracy with the most accurate laboratory-developed model-wind correction sub-model combinations. A similar observation was reported by Cruz et al. [8], where the model of Anderson et al. was also found to exhibit relatively low MBE and MAPE values against field data. It is noted that the F3 model is the only field-developed model that accounts for the effect of fuel bed width; this may have affected the achieved levels of prediction accuracy, since the laboratory tests used for the comparative study were all performed in fuel beds that exhibited typical width values significantly lower than the large-scale fires used for the development of the F models. Furthermore, F3 is the only field-developed model that includes ρ_b as an input. It is interesting to note that ρ_b (or, more specifically, its dimensionless form (β)) was shown to be affecting the prediction accuracy of the L-W model combinations. Thus, it may be the case that the effect of the fuel bed compactness, described by means of the ρ_b or β parameters, is an important descriptive quantity that can be closely connected to the impact of external wind on fire spreading.

In any case, the difficulty of using the field-based models to accurately predict the experimentally determined ROS values may be attributed to the following four characteristics. Firstly, field-developed models are fuel specific, which means that few fuel-related parameters are required as an input, while the rest of the parameters are integrated in various statistically fitted coefficient values. For example, the F1 model was developed using measurements obtained in spinifex grassland, a fuel that exhibits horizontal discontinuity; this fact may explain the strong tendency of this specific model to under-predict the laboratory measurements obtained in continuous fuel beds. Secondly, the fact that field experiments almost always employ “live” as well as “dead” fuels may not allow one to properly consider the effects of FMC. Thirdly, field experiments usually employ fuel areas exhibiting fuel bed width at least an order of magnitude higher compared to laboratory tests. Under external wind conditions, ROS depends on the flame front’s width, which can significantly change with time, due to the large dimensions of the fuel area in conjunction with the wind that can change both in magnitude and direction, as was observed and inferred by [34,35]. From the selected field-developed models, only the models of Anderson et al. (F2 and F3) incorporated the effects of width, and their good performance, compared with the rest of the F models, might be attributed to this fact. Finally, the assumption used to determine the WAF, i.e., that the height of the flame above the fuel bed is considered to be equal to the height of the fuel bed, is usually not valid in small-scale laboratory fires, a fact that might induce further deviations from the observed values. It is noted that similar arguments regarding the potential reasons for the observed over-predictions are reported by Weise and Biging [6] in the frame of a comparative evaluation of field-developed models tested against laboratory data.

5. Conclusions

A validation study of a selection of laboratory- and field-developed ROS models and wind corrections was performed against a set of laboratory experimental data. For the models that were developed using well-controlled laboratory experiments, it was shown that while new empirical models yield better results, the semi-empirical models provide insight into the physical mechanisms that govern the wildland fire spread phenomena. The impact of external wind velocity on ROS values was found to be affected by the fuel bed’s packing ratio, since the wind correction sub-model W1, which incorporates the packing ratio, was found to yield overall better results. The ROS models developed using large-scale field tests were, mainly, fuel-specific; therefore, they were generally not able to yield accurate results when tested against laboratory-scale fire tests. This was due to the differences in scale regarding both fuel and wind characteristics, in addition to the nature of the specific fuel type used in each test that affected the model’s empirically fitted parameters.