Validation of NWCG Wildfire Directional Indicators in Test Burns in Coastal California

Abstract

One of the primary tools used for determining the origin of a wildfire is analyzing burn patterns formed during the fire progression. These patterns, called fire pattern indicators, are interpreted and used to document the direction of fire movement at specific points, creating a directional map back to the specific area of origin. This concept was first set forth in 1978 by a U.S. governmental organization, the National Wildfire Coordinating Group (NWCG). Their recommendations are currently (2016) in the third edition, and in our study, we examine these indicators. Specifically, the objective was to perform a validation exercise where controlled burns were conducted of natural vegetation plots but augmented with 32 identical sets of staged artifacts which would provide additional opportunities for fire movement to create observable directional fire pattern indicators. Three adjacent plots were burned, each using a single point ignition, all located on level, scrubland terrain. The burns were conducted in the fall season, under low to moderate burning conditions. The research was structured as a preliminary study, since only mild terrain and weather conditions were encompassed. The actual fire movements were documented by drone videos, additional ground-based videos, and still photography. Within the three burn plots, a total of 12 data sites out of 32 data sites were selected: each one containing 7 to 12 individual artifacts. Each artifact was photographically documented post-fire, and the actual fire movement direction at that location was established. Assessment entailed the use of four experienced wildland fire investigators, with each one independently assessing the direction and type of fire spread at each artifact using the photographic site evidence. An analysis was then conducted to make a statistical comparison between the actual fire movement direction and the direction estimates provided by the experts analyzing the photographic evidence and the limited information on conditions provided. The results indicate an average error of 103°. These results indicate that additional efforts are needed to study the scientific basis of the indicators and to evolve improvements in both the indicators and in the accompanying guidance to investigators.

1. Introduction

One of the pivotal tasks in fire investigation is to determine the origin of the fire. According to the Guide for Fire and Explosion Investigations [1] (NFPA 921, Sec. 18.1.2), this may be carried out by means which include witness information, electronic data (e.g., videos), fire patterns, and fire dynamics (e.g., computer fire modeling). Generally, the most conclusive determination will be obtained if there is direct eyewitness testimony or photographic evidence of the earliest stages of the fire. In many cases, however, such information may be unavailable. In such cases, fire investigators must use other types of information, typically by carefully observing the burned areas and endeavoring to identify fire patterns. According to NFPA 921, fire patterns are defined as “The visible or measurable physical changes, or identifiable shapes, formed by a fire effect or group of fire effects”. In turn, fire effects are defined as “The observable or measurable changes in or on a material as a result of a fire”. Additionally, according to NFPA 921, Sec. 6.4.1, fire patterns are subdivided into two categories: (1) intensity patterns, and (2) movement, or directional, patterns.

The above general guidance is true irrespective of what type of fire is being investigated. But details of fire patterns will obviously depend on the type of fuel which is burning. For vegetation fires, a system of directional fire indicators was first proposed in 1978 in the first edition of the NWCG Wildfire Cause Determination Handbook [2]. This brief document proposed eight types of directional indicators:

• Grass stem indicators (drooping onto direction from which fire is coming);
• Protected-fuel indicators (large fuel beds look darker when looking towards the origin, and lighter when looking in opposite direction);
• Cupping indicators (more burnout on side of fuel facing into wind);
• Char pattern indicators (more charring on back side of trees, except on level ground);
• Alligatoring (more deep, alligatored char on heavy fuels in direction facing the flames);
• Freezing of branches (in the direction that the wind blows);
• Staining of noncombustible objects (in the direction facing flames);
• Soot on noncombustible objects (more in the direction facing flames).

There is no record of how these eight indicator types were established, nor has any material been published providing commentary on the Handbook. However, a few years later, Bourhill [3], a forester at the Oregon State Department of Forestry, published a much larger, highly illustrated document describing eight types of directional indicators. These eight indicators, however, were not the same as in the NWCG document. Bourhill omitted two NWCG indicators (char pattern indicators; soot indicators) and introduced two new ones (vertical fuel indicators; ash indicators). Some 27 years after the first edition, NWCG published the second edition of the NWCG Handbook [4] in 2005. In this edition, the list of indicators grew from 8 to 14. One indicator (alligatoring) was eliminated. One indicator (char pattern indicator) was expanded into two (angle of char; depth of char), while six new indicators were added (spalling; curling; white ash; V patterns; degree of damage; die-out).

The third edition of the NWCG document was published in 2016, renamed as Guide to Wildland Fire Origin and Cause Determination [5]. This current edition provides 11 indicators:

(1)

Protection (where a noncombustible object or the fuel itself shields the unexposed side from damage, especially applicable to larger objects);

(2)

Grass stem (charred remains of grass stems droop the direction facing the flames; generally only applicable to backing fires);

(3)

Foliage freeze (softened heated leaves and branches are bent in the direction of the wind flow—not flame direction—then cool down and stiffen in the same position);

(4)

Sooting (soot deposit from combustion, preferentially on the flame-exposed side);

(5)

Staining (varnish-type deposits from pyrolysates, preferentially on the fire-exposed side of noncombustible items);

(6)

Cupping (concave, or cup-shaped char pattern on small vegetation elements, preferentially on the flame-exposed side; not reliable for larger stumps);

(7)

Angle of char (on tree trunks and standing objects; generally follows the terrain except when strongly wind-aided or spreading fire up a slope);

(8)

Spalling (breaking chunks of rock off due to heat, preferentially on the flame-exposed side; typically seen only for advancing fire);

(9)

Curling (curling of leaves towards the flame due to heating);

(10)

White ash (the mineral remains of combustion, preferentially seen on the flame-exposed side; more white will be seen when facing the direction of fire travel);

(11)

V and U fire patterns (a V- or U-shaped spreading out of the fire from the area of initial ignition).

Thus, it can be seen that three indicators (depth of char; degree of damage; die-out) were eliminated. The current validation study was focused solely on the 11 indicators currently published by the NWCG.

Additional information on these indicators is provided in the parallel NWCG course curriculum for the course FI-210, Wildland Fire Origin and Cause Determination [6]. Reference here is made solely to the current edition, 2016, of this document. The course material presents the same indicators but includes some different narratives and illustrations than does the Guide. The two documents are developed in parallel; thus, the prior edition of both was the 2005 edition.

1.1. Prior Validation Studies

Up to the present time, no commentary has ever been published on the development of the NWCG indicators. But of more concern is the absence of published validation. The NWCG collection of indicators comprises a scientific theory. Yet, none of the three editions were accompanied by any validation efforts. In 2017, the first paper was published endeavoring to validate the NWCG indicators, authored by Simeoni and ten coauthors [7]. The authors conducted test burns in the Pine Barrens of New Jersey and, as part of the exercise, undertook to validate the NWCG indicators. Their published conclusions were “that the indicators are useful tools for the wildland fire investigator to determine the direction of fire spread to pinpoint the area of origin. However, some discrepancies between indicators and preliminary study of their reliability highlight the fact that the indicators must always be interpreted in the frame of a global analysis of the fire”. A detailed numerical analysis was not given; thus, the present authors undertook to provide a summary. A total of 17 artifacts were examined by the authors, who showed the visible directional indication and the corresponding fire travel direction on a map.

Table 1. Validation results obtained by Simeoni et al. [7] as summarized by the present authors.

Validation of Direction	Number of Artifacts
45° or less	5
about 90°	5
135–180°	3
indeterminate	4

shows these results. If, for this purpose, success of validation is defined as a deviation of 45°, or less, then, excluding the indeterminate indicators, 5/13 or 38.5% of the indicators were found to be successful. These results suggested that additional validation could fruitfully be undertaken. Of even more importance is that the study by Simeoni et al. [7] utilized a line ignition source. Line ignition sources are used in prescribed burns and in back-firing operations, but actual wildland fires will almost always undergo a single-point ignition, not a line source. There have been no validation studies where a single point-source ignition was used; thus, the present study is the first such effort.

1.2. The Science Basis of Directional Indicators

In the most general context, wildland fire directional indicators are based on relative heating damage. In certain situations, artifacts may sustain preferential thermal damage and the nature of this preferential damage is used to infer a direction of fire propagation [8]. Matters become complicated since there are two primary heat transfer modes, convection, and radiation (radiant heat transfer), and these are affected differently by geometric and fluid dynamic (e.g., wind speed) factors. A substantial amount of research exists with regard to directional indicators in structure fires [8]; but for wildland fires, the literature is much more limited.

The analysis must proceed from an understanding that heat flux (that is, the energy, per unit time, per unit area; kW/m²) is the driving force for creating thermal effects or thermal damages. Protection patterns (indicator) is perhaps the simplest to understand. Physical objects may be present which block the effects of heat flux. Thus, thermal damage to areas beyond the blocking object may be attenuated, and this may, in certain cases, allow a direction of fire propagation to be inferred.

But, in many cases, matters may be complicated. In the case of convective heat transfer, at low wind speeds, the heat transfer is greatest at the zone of impingement, then progressively lower further away [9]. Yet, high wind conditions can create anomalous patterns [9,10]. Now, of course, heat transfer is not wholly convective, and radiative flame heating is also necessarily present. In simple cases, this can be related to the radiative view factor. The flames, when viewed from the side facing the direction from which the fire is coming will generally subtend a larger solid angle than will be flames viewed from the side facing the direction towards which the fire is moving. The radiant heat flux increases with increasing fuel load density [11]; this is because larger flames are created, and larger flames increase the optical thickness (mean beam length) of the flames, which is the driving force providing the radiant heat flux.

Meanwhile, increasing the wind speed or increasing the slope of the terrain causes flames to progressively bend over, and thereby become closer to the fuel. Research indicates, however, that the effect of this on the radiant heat flux towards the fuel is not consistent, and it may cause the heat flux to increase [12] or decrease [13].

For fuels comprising shrubs (and smaller fuels), heat transfer from convection is generally larger than from radiation [14]. Generalizations about heat flux effects have largely been based on fairly limited studies, and complexities and contradictions can be expected in some cases, especially with regard to slope effects and the relative proportions of radiative versus convective heating [15]. Larger upslope angles are likely to result in a greatly increased convective fraction [16]. This rule, however, is not always obeyed. Under certain conditions in strong winds, however, research indicates that flame radiation may be minimized on the windward side and increased on the leeward side [17]. In such cases, thermal damage will be greater on the leeward side.

Forestry researchers have endeavored for quite a few years to model the charring produced on tree trunks from wildland fires [18]. Yet, these efforts have been primarily focused on broader concerns and studies to correlate relative (front/back) trunk damage with fire spread direction and terrain slope have not been reported.

As noted above, in the simplest case, greater thermal damage can be expected on the windward side of an object than on its leeward side. But, a major exception to a simple behavior is when strong vortices are created on the leeward side of a blunt object (Figure 1). The vortices formed on the leeward side may entail both a high convective heat flux (due to high local velocity) and high radiative heat flux (due to concerted burning in the direct vicinity of the leeward side of the object). Both the NWCG Guide [5] and Simeoni et al. [7] consider that this is likely to occur on the leeward side of tall, thin tree trunks. However, neither reference offers any technical guidance to allow the user to determine when this is likely to occur, or not to occur. Since the indicator direction is effectively reversed for the one case, versus the other, this is a point of serious concern and uncertainty. Flame stabilization in the flow field behind a bluff body is a well-known problem in combustion, and Hertzberg et al. [19] studied it for the case of a rod-shaped body. While a rod-shaped body would be a viable schematic representation of a tree, it would have to be oriented vertically, while these researchers studied the horizontal orientation. Another reason why the classical literature [20] does not provide much guidance is that, even with a geometric simplification, wildland fuels are solid fuels dispersed along a 2-d surface, a geometry which is generally only of interest in the context of vegetation fires. A large team of wildfire researchers (Finney et al. [21]) recently did publish a review paper on studies of buoyant flame dynamics in wildfires. This study examined flame vortices in certain geometries, but not the flame wrap vortex behind a stalk plant. Perhaps the reason for so little study is the observation of Beer [22] that 2-d models will give the wrong results and that full 3-d modeling is needed to represent the vortices properly. Historically, a number of investigators discussed the phenomena of vortices in wildland fires, but only qualitatively. These include Byram and Martin [23], Church et al. [24], Emori and Saito [25], Forthofer and Goodrick [26], Gutsell and Johnson [27], and Kleynhans et al. [28]. Unfortunately, as regards useful guidance for the fire investigator, even a recent (Tohidi et al. [29], 2018) assessment concludes that “there are presently no unique parameters that can quantify and describe the necessary formation conditions”. Such research was not within the scope of the present study, but deserves high priority. The NWCG Guide [5] suggests that, on sloping terrain, the char pattern on trees will reach higher on the upslope, than on the downslope side. But, in studying char patterns in burned areas, Smith and Sutherland [30] found numerous cases of more extensive charring on the downslope side than on the upslope side. This was presumably due to wind direction having been downslope, but the possibility exists of anomalous outcomes due to variations in ground fuel distribution around a tree base.

All of the references cited above are from the general scientific literature. There have been no scientific publications of the NWCG which would provide any background research on the indicators nor give explanations of their development.

2. Methods and Materials

2.1. The Plots and Their Artifacts

Three adjacent burn plots, Plot A, Plot B, and Plot C (Figure 2) (Appendix A), were established on the grounds of Camp San Luis Obispo, a former U.S. Army base, now a facility of the California National Guard. The site is located about 5 miles NW of the City of San Luis Obispo.

The three (roughly rectangular) plots were established with containment lines around the perimeter consisting of a roadway and constructed handlines. Hoselines were in place to ensure the burns stayed within their established boundaries. Plots A and B contained 11 data sites, while plot C had 10. Three data sites were placed in a row running north to south at the west end of the plots. Row two was roughly 6.1 m east, consisted of two data points. Row three was placed roughly 6.1 m east of row two with three data points, and row four another 6.1 m to the east, identical to rows one and three (plot C only had two data sites in row four). The plot layouts were designed to ensure consistent advancing, lateral, and backing fire spread was captured on identical artifacts as the fire moved past the sites. For visual reference, an origin marker was established using a 1″ × 2″ by 2.4 m high wood post with white flagging tape attached to the top; it was placed in the middle position of row two (see Figure 3).

The data sites were laid out using a 1.83 m × 0.91 m template to provide uniformity. Each data site consisted of nine items placed in a similar pattern. The artifacts consisted of a 3″ round wood post, a pressure treated 4″ × 4″ post, two 355 mL brown, long-neck bottles, two aluminum cans, and three Trinity River rocks (large, medium, and small). The posts were secured by drilling a 25 mm hole in one end, 0.305 m deep, and then placed over a foundation stake implanted in the ground. This method prevented disturbing the dirt and fuel at the bottom of the posts when installing. The bottles and cans were placed perpendicular to each other (see Appendix B). Some preexisting artifacts were identified after the burns and were included in the assessment. After the burns, it was discovered that more than half of the aluminum cans had been moved from their original pre-fire locations by strong winds that developed the evening after the burns were conducted. These shifted artifacts were excluded from evaluation.

2.2. The Burns Conducted

The study burns were conducted on 22 October 2022, starting at 11:30 a.m. A fog overcast layer moved into the valley late on the 21st but had burned off by 10:30 the morning of the 22nd. Ten-hour fuel moisture was 22% at 12:01 p.m., when Plot A was burned. Plot B was burned at 1:48 p.m., while plot C was burned at 2:35 p.m., by which time the 10 h fuel moisture had dropped to 19%. Relative humidity remained in the high-60% range throughout the day, wind was light and variable during the Plot A burn, then became westerly 0.45 to 1.79 m/s for Plot B and C burns. Each burn was ignited at a single point in the vicinity of the origin marker using an open-flame device (Figure 3).

2.3. Post-Burn Documentation

The following day, 23 October 2022, the data sites were documented. Identification numbers were placed, and GPS location and magnetic north were identified and documented. Multiple photographs were taken in a 360° circle surrounding each data point, with additional closeup photographs being taken of specific fire effects on the various artifacts. Photographs were taken throughout the day to ensure all areas were photographed in sunlight and not blocked by shadows.

As stated above, a maximum of 32 data sites could have been utilized, but not all would have been suitable for analysis due to how the fires developed and spread. Once field data collection was completed, the next step was to select data sites for evaluation meeting the following criteria:

• The type and direction of fire spread remained consistent as the entire fire front passed the data site. This was evaluated using the pertinent drone video.
• The post-fire photographs were adequate to establish a 360° view of all the artifacts.
• Four sites each were selected to encompass areas where the fire was backing, moving laterally, or advancing. Evaluators were not advised the data sites were equally divided among the type of fire spread.
• The site numbers in the evaluation packet were assigned randomly and have no correlation to the specific burn plot or location within the plot.

The final selection consisted of three sites from Plot A, four from Plot B and five from Plot C. The selected sites were limited so that fire effects on the artifacts located within a data site would display consistent fire effects (e.g., all being backing-fire effects). Photographs of the twelve selected sites were sorted and cropped to remove what lay beyond the data site. This was carried out to ensure that only the fire patterns on the artifacts within the subject site would be used. Evaluators were provided between 34 and 58 photographs for each data site. Two versions of each photograph were provided, an original cropped JPG file version and a JPF file version of the same photo with the artifacts numbered and a 12 h clock face imposed for directional reference. The 12-noon direction remained constant on all the photographs within each data site regardless of the angle or location the photo was taken. The 12 o’clock position was not consistent from site to site to avoid the evaluators using observations external to the site under assessment (Appendix B).

2.4. The Evaluators

Four highly experienced wildland fire investigators with more than 130 years of combined fire service experience were recruited to perform the evaluations. The evaluators have professional experience in wildfire suppression operations, fire investigation (including primary responsibility to investigate wildland fires), law enforcement, and private fire investigation. All the investigators have testified in court as experts. A summary of their qualifications is given in Appendix C.

2.5. Material Supplied to the Evaluators

Each of the four evaluators were mailed a box of materials containing instructions, a thumb drive with the photo panels of the twelve data sites, twelve evaluation forms to fill out, one for each of the data sites, and a return envelope. An individual video meeting was held with each evaluator to go over the instruction sheet, walk through an example data site that was not part of the evaluation process, watch a selection of the drone videos of Plots B and C burning, and provide fire weather and 10 h fuel moisture information. It was also explained to them that the objective of this study was to validate fire spread patterns on artifacts using multiple photographs in the absence of data outside the artifact cluster or data site.

The numbered assessment packets were placed in random order into four boxes by an intern and mailed out. Evaluators were asked to not make any identifying marks or labels on the data packets they received. Once the packages were returned, they were opened by an intern and the four packets placed into a bin and the return boxes discarded. This process was to ensure that the specific identify of the evaluator identified by the number on the assessment packets would remain unknown throughout the course of this study. The evaluators were the only persons who knew the number associated with their name. A summary of the materials provided to the evaluators is provided in Appendix B.

2.6. The Analysis Work Carried Out by the Evaluators

The evaluators were tasked to first determine if the visible fire effects on each artifact are sufficient to indicate the direction the fire came from. If the answer was yes, they were asked to identify the direction the fire came from based on the imposed clock, such as the fire came from a 2-to-4-o’clock vector, the indicator category observed, and the type of fire spread—advancing, lateral, backing, or undetermined. After evaluating all the artifacts individually, they were asked to consider all the patterns observed at each site as a “pattern cluster” and determine if there is a consistent pattern among the artifacts depicting a fire spread vector. If the evaluator felt there was a pattern from the cluster, they were asked to record the vector using the clock direction, type of fire spread, and their confidence in the assessment: low, medium, or high.

3. Analysis

The evaluator packets contained a cover page listing all the artifacts on the site, then a list of questions for each artifact, and ended with a list of questions evaluating the site indicators in their totality as a cluster. The mean clock direction was recorded to the nearest 30 min or 15° reading, the correct type of fire spread, and the true vector direction (in degrees) was entered for each data site. The raw data from the evaluator analysis packets were entered into a spreadsheet. The first entry entered was the question: “A fire pattern depicting a fire spread vector exists, Yes/No”. If the answer was No, no further data other than a comment, if any, for that artifact were entered. When the answer was Yes, the clock vector, indicator category, and type of fire spread was entered, if provided. The mean vector of the clock range determined by the evaluators was converted to compass degrees, then this vector was compared to the true vector to determine the error. The error found for each location was tabulated as the absolute value of the difference between the actual compass direction and the assessor’s determination of the compass direction calculated using the mean of the clock vector. In other words, each result is taken to be a positive number. For perfect agreement, the error would be 0°. Conversely, the maximum error that can exist is 180°. All the data were entered and double-checked by an intern for accuracy prior to further analysis.

The true fire spread direction was established by analyzing the drone video of the fire from just prior to the flame front entering the data site until all visible flame had moved past the site. The direction the fire spread was coming from was imposed over a Google Earth Pro map of the plots with the data site coordinates marked as waypoints using GPS locations collected at the same location at each data site. The true north direction was documented for each site. The 12 o’clock position (an arbitrary, random assignment for each site) based on true north was determined for each site. Magnetic deviation at the burn site is +12.26°; in other words, the compass north is 12.26° to the east of true north.

Once the analysis of the overall data began, concern was raised about the unexpectedly high error rate and wide variation between the evaluators on a number of data points. Interpretations for all the direction calculations and drone videos were re-examined, conversions of degrees from true north to clock direction, mean vector and error were double-checked. No errors in any of these calculations were discovered. In general, the error ranges for the evaluators were similar. Each evaluator recorded some sites with high accuracy and others with very poor accuracy, and—surprisingly—no patterns of accuracy could be identified. In an abundance of caution, we considered the possibility that one or more evaluators may have been entering the direction the fire was spreading to, rather the direction the fire was coming from. Review of the site forms was conducted by assuming that this evaluator presented a spread direction off by 180°. This process increased the error rate more than it reduced it, and was thus rejected as a potential source of systematic error.

This study consisted of 101 artifacts that were presented to each of the four assessors, resulting in 404 total assessments. The evaluator cadre reported that fire effects on 332 artifacts indicated valid fire direction indicators. The error for each artifact assessment was calculated using the mean vector reported for each artifact against the true vector as determined primarily by drone video. The average directional vector error was found to be 103°, see

Table 2. Breakdown by vector type and evaluator number.

Spread Type	First	Second	Third	Fourth	Average
Advancing	129°	128°	71°	81°	102°
Lateral	92°	123°	111°	93°	105°
Backing	110°	121°	104°	76°	103°
Average					103°

. Details are discussed below. Further analysis began to attempt to identify the reason behind this unexpectedly high average directional error. As intended, the detailed analysis of the data was guided by NWCG PMS 412 Chapters 1A and 1B and the FI-210 Course curriculum Lesson 1A and 1B covering Fire Pattern Indicator Categories and Fire Pattern Indicator Categories, respectively.

3.1. Analysis by Spread Type

Fire spread vectors can be of three types: advancing, backing, or lateral. An analysis was made to determine if one fire spread type provided more reliable directional fire pattern indicators, compared to the other two. Table 2 shows the comparison between the error recorded for each data site, versus the error when results are limited only to artifacts where the evaluator correctly identified the fire spread type. The results were statically identical, as seen in

Table 3. Comparison of overall error to error when only considering answers where the correct fire spread type was identified.

Site	Type of Spread	Average Error (Degrees)	Average Error, When Correct Spread Type
1	Backing	110	110
2	Lateral	92	71
3	Advancing	129	152
4	Advancing	128	--
5	Backing	121	--
6	Lateral	123	129
7	Advancing	71	57
8	Lateral	111	118
9	Advancing	81	68
10	Backing	104	9
11	Lateral	93	50
12	Backing	76	61
Average		103	103

. Separating out the sites where a high number of assessments, 9 or more, correctly identified the type of fire spread, the vector error rate improves slightly but is still statically insignificant.

3.2. Analysis as Clusters

In addition to considering the three types of fire spread vectors, both the NWCG Guide and the FI-210 Course also point out the value in using multiple indicators of different categories in close proximity known as clusters. Evaluators were asked to assess all the artifacts at each data site as a cluster and determine if the cluster is depicting a fire spread direction pattern, see

Table 4. Errors determined as mean individual errors, versus errors evaluating sites as a cluster.

Site	Mean Overall Error (Degrees)	Mean Error Analyzed as Cluster (Degrees)
1	110	124
2	92	85
3	129	154
4	128	123
5	121	128
6	123	109
7	71	83
8	111	143
9	81	57
10	104	120
11	93	116
12	76	70
Average	103	109

. No statistically significant difference was found between the average individual errors and the errors when evaluated as clusters.

3.3. Analysis by Indicator Type

One of the objectives of this study was to determine if the current list of indicators presented by the NWCG should be amended. In other words, is the reliability for any indicators significantly inferior than for others?

The evaluators were asked to list the indicator type, in each case where they found a valid directional fire effect. In some cases, the assessors listed more than one indicator category for a particular artifact. For those cases, only the first-listed indicator was utilized. Reasonable interpretation was made concerning language differences. For instance, “stain” was taken to be identical to “staining”. Also, “flame wrap” was taken to mean “angle of char”. The results of the analysis are given in

Table 5. Mean errors found for each indicator category.

Indicator	Mean Error (Degrees)	Number of Data Points
Protection	81	39
Grass stem	98	7
Foliage freeze	--	2
Sooting	97	20
Staining	106	133
Cupping	--	2
Angle of char	98	89
Spalling	--	1
Curling	--	0
White ash	81	6
V or U pattern	--	1

. A decision was made that six data points will be taken to be the minimum of data points for a valid statistical analysis. Using this criterion, the basic results obtained were as follows:

• Data points sufficient for analysis:

• Protection;
• Grass stem;
• Sooting;
• Staining;
• Angle of char;
• White ash.

• Data points insufficient for analysis:

• Foliage freeze;
• Cupping;
• Spalling;
• Curling;
• V or U pattern.

Thus, of the 11 NWCG indicators, six had sufficient data, while five lacked sufficient data. For the indicators with sufficient data, the error ranged from 81° to 106°. These results were clustered rather tightly, and it would be reasonable to conclude that the mean error observed was approximately 90°. There is a slight positive correlation between the number of data points and the size of the error, but this is not a large effect.

4. Discussion

The outcome of this study was not what might have been anticipated. It shows that in a detailed photographic analysis of fire effects, the artifacts presented did not provide reliable fire direction indication for protection, grass stem, sooting, staining, angle of char, and white ash indicators (with there being insufficient data to make any conclusions with regard to the remaining NWCG indicators). It was not expected that, given the simple topography, fire spread conditions and ample artifacts, four highly experienced investigators, intimately familiar with fire indicator patterns and methodologies described in PMS 412 and FI-210 would be unable to review comprehensive photo panels and reliably determine the fire spread direction from the photographic evidence. We specifically note that the instructions to fire investigators given in PMS 412 and FI-210 are on the basis of directional indicator photographs and that photographic evidence is deemed adequate and sufficient.

This study was designed and expected to produce clear, definitive directional and vector (advancing, lateral and backing) patterns on individual artifacts and clusters. Contextual information was provided except for wind direction. The investigators viewed several drone video segments of the fires burning (without the data sites identified) that provided clear view of the fuels that were consumed and fire behavior. Evaluation was limited to visible fire pattern indicators captured on detailed photographic panels. The area captured in the photographs was limited to only the specific site area containing the artifacts and nothing beyond. The location of the 12 data sites relative to plot and location on the plot was not provided. Three types of artifacts were identical for all sites (posts, bottles, and cans), while the rocks were of the same type and similar shape and size. The other artifacts that were pre-existing varied from site to site. Most importantly, the direction and type of fire spread remained consistent through each data site from the time the fuel began to heat from the oncoming fire through the fuel bed flameout. Additionally, the data sites were designed to provide indicator clusters consisting of four types of artifacts that were expected to produce different indicator categories. The photo panels consisted of a series of photographs presenting 360° site documentation, both wide- and close-angle, and taken at two different times throughout the day to reduce the effects of shading.

It is likely there are multiple contributing factors to the error rate found in this study, versus an actual fire incident. The evaluators did not have an opportunity to personally gather additional information, such as interviews, early bystander or security photographs and, most significantly, view the entire fire or greater origin area. Another possible factor is the burning conditions: flat ground, mono fuel type, and a thick matt fuel bed with minimal wind speed may not have generated adequate fire intensity differential between the backing, lateral, and advancing fire front to produce adequate directional and spread type fire effect patterns. In-person inspection of the GOA may have alerted an experienced investigator that the conditions present may not provide generally reliable fire patterns and reliance on other data sets would be necessary, or the entire plot could be processed as the SOA.

An error of 90° would represent the results from a random-number generator. Thus, to accurately locate a specific point of origin, the error must be substantially less than 90°. The average error found was 103°, which is slightly worse than a random-number generator would produce; however, this is not considered to be a statistically significant difference from 90°. Assessing directional indicators using only photographs is more difficult than direct visual on-site observation. However, both the NWCG Guide and the FI-210 course are based on an educational presentation in terms of photographs, and examination of actual burned artifacts is not utilized as a necessary training aid for identification of directional indicators.

This study shows that the six indicator categories for which sufficient data were available did not provide reliable data on fire spread direction. Yet, the burns took place in what would reasonably be considered near-ideal conditions, with regard to generating artifacts with unambiguous directional indications. Many of the fire effects captured in the photo panels appeared similar to the photographs found in Chapter 1B of the NWCG Guide. The NWCG Guide and FI-210 repeatedly caution investigators that “Certain circumstances occur creating possible exceptions that apply to most fire pattern indicator categories”. Yet, viable guidance is not given on these “circumstances” and it is difficult to understand why test fires under conditions of simple, non-extreme fuel, weather, and topography conditions would somehow necessarily fall under the category of “exceptions”; however, the results imply they did.

We believe that additional validation studies should be undertaken to identify specific indicators that are reliable under a particular range of conditions, as well as identifying conditions that produce indicators which are not reliable. Furthermore, we do believe that highly experienced fire investigators are generally able to correctly identify the specific origin area of a wildland fire in a wide range of conditions. Often, the needed evidence may be simple and incontrovertible, e.g., video records or credible eyewitness testimony. Experienced investigators may be able to discover data that provides the time the fire perimeter was located at specific points; analyzing this data in a timeline and context of fire behavior conditions can then narrow the GOA and possibly assist in defining the SOA. Yet, we also believe that, in most cases, the specific origin area of a wildland fire can be successfully determined based on a complete examination of the fire scene that includes gathering contextual information, witness information, plus valid fire pattern indicators. The challenge will be to identify under what conditions scene inspections using visual indicators is not a valid or accurate method to determine a specific origin or ignition area. We believe experienced, successful investigators take great effort to collect comprehensive data, including fire pattern indicators, prior to assessing the totality of the evidence. They then use the assembled data to test all key conclusions of the working hypothesis, including validity of the visual fire pattern indicators discovered during the physical scene inspection prior to selecting the final hypothesis. Notwithstanding this, even the most experienced investigators occasionally encounter fires where all the available data only supports a finding of “undetermined” area of fire origin.

We find it noteworthy that the recent treatise on wildland fire behavior by Finney et al. [31] offers this caveat: “Studies in the laboratory, in the field and by modeling have probed physical, ecological, and meteorological dimensions of wildfire phenomena across a range of spatial scales, but have also exposed contradictions and persistent mysteries, even for what appear to be the simplest and most observable behaviors. Consequently, the body of scientific observations, correlations, predictions, and speculations cannot yet be claimed to constitute a coherent physical theory of how wildfires spread and behave”. It reasonable to conclude that, at this time, the important fire investigation concept that directional indicators can be reliably used to triangulate back to a fire’s origin does at times fall into the category of “persistent mysteries” noted by Finney et al., especially in the absence of corroborating data.

We also note that the history of using the NWCG indicators goes back at least to 1978, when the first edition of the Guide was published. Some 45 years have elapsed between 1978 and 2023, and several generations of fire investigators have relied, often completely, on the guidance associated with these indicators. Yet, the research base underlying these indicators is minimal. We have approached the subject here solely as an exercise in validation. For investigators, it is troubling that the Guide provides seven general contextual circumstances that may put indicators into the “Possible Exceptions” category. Additional warnings are listed for most of the 11 indicator categories. The lone guidance on this crucial issue is a list with two bullet points with six sub points. Investigators are not provided the guidance to adequately distinguish whether the conditions they encounter fall under the scope of the “valid guidance” or the “exception”. It must be a priority to give far more specific guidance for when conditions fall into the “exceptions” and how to make such determinations. This may need to occur after some of the lacking science research has been obtained. Research is needed in the science underlying each of these 11 indicator types, along with the specifics of how passing fire fronts generate fire effects on artifacts.

Due to limitations of the size of the burn area, the focus in the present study was primarily on micro indicators. In actual wildfires, macro indicators are likely to be present, especially large V- or U-patterns, and accompanied by vast differential damage levels. These could not be examined in the present study but often provide more reliable directional indication narrowing the location of the GOA and possibly pointing to the SOA.

Finally, we noted at the start that the list of indicators has changed with each new edition of the NWCG Guide. Presumably, this was due to learning empirically about the validity, or lack thereof, of certain indicators. There is no published record of the process which was involved in this winnowing and modifying. Had this history been documented, it might have enabled some better guidance with regard to knowing the validity for the different indicators, along with their limitations, that is, exactly under what circumstances should they be presumed to be operative, or not.

5. Conclusions

Four highly experienced evaluators were requested to determine if the post-fire visual fire effects on 404 artifacts formed valid fire pattern directional indicators. They assessed fire effects on 332 of the artifacts they found to be valid fire direction pattern indicators. The average error from the true direction of the fire’s approach, compared to the mean vector determined by the evaluators was 103°. Such a large error band will not enable the investigator to triangulate back to the correct origin. The high error rate followed no specific patterns. Each evaluator had roughly the same number of accurate to inaccurate answers, and the errors among the indicator categories, artifact type, and individual artifact vs. cluster evaluation all showed minimal differences. While some answers were very close to the correct answer, no analysis technique was found to narrow down the wide range of scatter.

Fire investigators are trained in using fire pattern indicators as described in the NWCG Guide and taught in FI-210, and skilled investigators can use these teachings along with extensive knowledge and experience with fire behavior, fire suppression, and other data sources to regularly produce reliable determinations of an often quite small specific origin area. At the same time, experienced investigators also know there are situations where it is not possible to reliably interpret fire patterns and thereby locate a fire’s origin using these patterns. Currently, how to gather and use data to determine the necessary context from which to assess the distinction between reliable and unreliable indicators is left to the individual investigator to refine with time and experience. Only a handful of investigators have the experience of a sufficient quantity and quality of investigated fires, along with needed mentorship to develop the skills and knowledge to optimally master the wildfire investigation skill set.

Efforts to improve the accuracy of wildfire origin determination must be based on science, and properly validated through research. Research has demonstrated the multiple layers of complexities that affect the characteristics of a burning wildfire front as it passes a given point. The present study demonstrates that there are many unknown complexities in tracing the path of a wildfire back to the origin. There are two key areas of additional study required to accurately understand what the fire effects impart onto physical artifacts. One is to continue to validate fire directional patterns and to determine what patterns are likely to form and provide accurate fire directional information within a specific range of fuel, weather, and topographic conditions. The second area of study needed is to understand on a more fundamental basis the process whereby fire patterns are formed.

The research presented here solely consisted of an endeavor to validate the NWCG indicators. It might be asked why the results did not show a better prediction of the actual fire travel. Unfortunately, this is not possible to answer. As indicated in the section on “The science basis of directional indicators”, technical research underlying the indicators has been extremely scant. Even more problematically, the NWCG never published scientific explanations for the basis of the establishment of the indicators. This is notable since the list of indicators was substantively revised in 2005 and 2016, yet technical details justifying those changes were not published. We encourage the NWCG to conduct the necessary studies to establish a scientific basis for the indicators, and to publish their results in the peer-reviewed scientific literature. It is likely that such a process will result in changes to the NWCG list of indicators, along with clarifications of conditions under which any given indicator is, or is not, expected to be reliable.