A Comparison of CALPUFF and LAPMOD Against the Project Sagebrush Datasets

Abstract

This paper presents the validation of CALPUFF and LAPMOD against the short-range and high time resolution tracer study dataset of Project Sagebrush (PSB). The meteorological fields for both models are prepared with the CALMET diagnostic model starting from the exhaustive meteorological data collected during PSB. The PSB releases were undertaken under different meteorological and turbulence conditions. The validation results—judged by means of several statistical parameters—indicate that the models are generally in satisfactory agreement with the observations, sometimes even when time- and space-paired data are considered. However, in four simulations carried out under low wind and very stable conditions, the model performances are poor. This may be due to the inability of CALMET to reproduce the vertical wind direction variations in a shallow layer close to the ground, but also to inappropriate turbulence dispersion algorithms in the dispersion models. This issue will be further investigated in future work.

1. Introduction

Atmospheric dispersion models (ADMs) are important tools to evaluate existing air quality together with monitoring stations. Moreover, they are the only tools capable of predicting future air quality expected after the realization of new industrial plants or modifications of existing ones. The importance of air dispersion models is also recognized both by the previous EU Directive 2008/50/CE [1] on ambient air quality and by the new one 2024/2881 [2]. Additionally, the US-EPA provides a list of preferred models, as well as guidance for their use in predicting the ambient concentrations of air pollutants [3].

Due to their importance, ADMs must be continuously validated against field experiments data to verify their ability to estimate observed concentrations. Some field experiments continue to remain important, but they are becoming dated, since they were conducted in the 1950s and 1960s. The tracer studies include, for example, Kincaid [4], Indianapolis [5] and Prairie Grass [6]. These have been extensively used to validate ADMs (e.g., [7]). A new tracer study named Project Sagebrush (PSB) was performed in 2013 (phase 1, PSB1) [8] and 2016 (phase 2, PSB2) [9] in flat terrain. PSB offers the advantage of using technologies not available during older field experiments, therefore providing a lot of information for understanding atmospheric dispersion and validating both meteorological and dispersion models. During PSB, an inert species (SF6) was released, and concentration measurements were taken with a time resolution of 10 min up to a distance of 3.2 km. These features make PSB very useful for verifying the ability of ADMs to estimate concentrations at relatively short distances and with high time resolution. Therefore, validation against PSB data may also be important to evaluate the ability of a model to simulate odor pollution, that often impacts relatively close to the source.

PSB data have already been used to validate models, mainly meteorological ones. For example, Ngan et al. (2018) [10] used the PSB1 data to evaluate the meteorological field obtained with the WRF meteorological model [11] and the concentrations estimated by the HYSPLIT particle dispersion model [12]. WRF was used with five nested domains, with the grid resolution going from 27 km to 333 m, using inline and offline approaches, as well as different planetary boundary layer schemes and a large-eddy simulation parameterization. HYSPLIT was used to reproduce the concentrations of IOP2, IOP3, IOP4 and IOP5, where IOP indicates Intensive Observation Period, or each release of the field experiment. IOP1 was excluded because the wind flow patterns caused the tracer plume to go in the opposite direction to the sampling array [8].

Similarly, Thomas and Kurzeja (2023) [13] used WRF and HYSPLIT to simulate the PSB1 releases. They implemented a Large Eddy Simulations (LES) model in WRF, and used it with four nested domains with grid resolutions of 3 km, 1 km, 200 m and 67 m. The authors simulated IOP3 and IOP5 and discussed only IOP5. The WRF/LES results were in satisfactory agreement with meteorological observations but the HYSPLIT SF6 concentrations overestimated the observations.

Bhimireddy and Bhaganagar (2018) [14] used WRF with multiple nested domains, starting with a grid size of 24 km, and downscaled it through LES up to a grid size of 150 m. With this approach, they simulated the atmospheric boundary layer (ABL) processes of IOP2-5 during PSB1. WRF was used with three boundary layer schemes, three surface layer schemes and two micro-physics schemes in order to perform a sensitivity analysis. The output of the model was compared to the meteorological observations collected during PSB1.

Finn et al. [15] used the PSB1 data to evaluate the correctness of the parameterization of the horizontal plume spread, whose formulation had been obtained empirically from the results of old tracer experiments. They found that the horizontal plume spread calculated with the modern instrumentation used in PSB1 was larger than that measured in many previous field studies by up to a factor of 2, and the discrepancies tended to increase with downwind distance.

Large and rapid wind direction changes were observed by Finn et al. (2018) [16] in low wind speed conditions in the very stable boundary layer during PSB2. This is in contrast with the common assumption that the plume is very narrow in stable atmospheric conditions due to the small values of the wind direction standard deviation (σ_θ). The authors observed that these wind direction changes are particularly frequent when the wind speed at 2 m agl is less than 1.5 m/s and are associated with sensible heat and momentum fluxes near zero. Another observation of [16] is that the magnitude of the wind direction changes is larger close to the ground and limited a few meters above it.

Another study investigating the results of PSB2 [17] demonstrated, by means of co-located samplings, that concentration variability at the same place, due to stochastic factors and independent of measurement uncertainty, increases the total observational uncertainty closer to the source from about 20% (daytime) to 40% (very stable conditions). The large increase in concentration variability is linked with the suppression of turbulent mixing, small eddy length scales, and meandering in very stable conditions. These results are really challenging when comparing observations with model predictions.

In this work, the results of PSB1 and PSB2 are used to validate two ADMs. One is the popular Lagrangian puff model CALPUFF [18], and the other one is the Lagrangian particle model LAPMOD [19]. As far as the authors know, this is the first time that both PSB1 and PSB2 have been used to validate ADMs because, as described above, many validations used only few IOPs of PSB1. Validation of ADMs against the PSB2 data, particularly for the four IOPs carried out under very stable conditions, is very challenging.

The first part of this paper describes in detail the Project Sagebrush (PSB), the models and their configurations. Then the results for each IOP of PSB1 and PSB2 are presented and discussed. The validation was performed quantitatively using several statistical parameters that are summarized by a rank introduced to show the behavior of each model at a glance.

2. Materials and Methods

In this paragraph, the methodologies adopted are described in detail to allow other researchers to replicate the results if needed.

2.1. Project Sagebrush

The Air Resources Laboratory’s Field Research Division (ARLFRD), part of the National Oceanic and Atmospheric Administration (NOAA), in collaboration with the Laboratory for Atmospheric Research at Washington State University (WSULAR), conducted a set of tracer releases at the Idaho National Laboratory (INL). The project was divided into two series of releases. PSB1 (Project Sagebrush Phase 1) was carried out in October 2013 during daytime (near-neutral and unstable conditions), while PSB2 (Project Sagebrush Phase 2) was carried out with light wind conditions in July, August (unstable conditions) and October (nighttime stable conditions) in 2016. PSB1 is described in [8], and PSB2 is described in [9].

The field releases were performed at the INL Grid 3 experimental site, which is in an almost flat plain on the western edge of the Snake River Plain in southeast Idaho (USA). The site is about 1500 m above the mean sea level.

Both PSB1 and PSB2 were characterized by the continuous release of SF6 close to ground level. All the releases lasted 2.5 h, while the samplings lasted for two hours, commencing 30 min after the release start in order to establish a quasi-steady-state SF6 concentration field across the sampling array. Many tracer studies were conducted in flat terrain from the 1950s and 1960s (e.g., [4,6]). PSB offers the advantage of using technologies not available during those older studies, therefore providing a lot of information to understand atmospheric dispersion and to validate models.

2.1.1. Source Characterization

In both PSB1 and PSB2, the point source was placed at the center of the INL Grid 3, whose coordinates are 43.59076 N, 112.93782 W, or E = 343,571.59, N = 4,828,245.40 within UTM zone 12T (where the letter T indicates a latitude between 40 and 48 degrees north). The SF6 was contained in two bottles and was released from the outlet end of a garden hose at a height of 1.5 m above ground level. The release diameter was about 5/8″ or 1.59 cm [20]. The garden hose was oriented horizontally to avoid imparting any vertical momentum to the tracer [9]; therefore, the vertical exit speed is considered null.

Five IOPs were realized during PSB1, with the release properties specified in

Table 1. PSB1 release information. Each release lasted 2.5 h.

IOP	Date (2013)	Start Time (MST)	Release Rate (g/s)	Temperature (°C)
1	2 October	14:00	10.177	16.5
2	5 October	12:30	9.986	16.8
3	7 October	12:30	9.930	24.9
4	11 October	13:30	1.043	17.1
5	18 October	12:30	1.030	15.6

. The first three IOPs have a release rate about one order of magnitude greater than the last two. The reason is that during IOP1–3, a real-time analyzer was installed within an aircraft, and the release rate was selected in order to provide tracer concentrations sufficiently high to be measured by such an instrument [8]. On the other hand, during IOP4–5, the aircraft was not available.

Four IOPs were conducted with light wind conditions during daylight in July and August 2016. A further four IOPs were conducted in light wind conditions at nighttime during October 2016.

Table 2. PSB2 release information. Each release lasted 2.5 h.

IOP	Date (2016)	Start Time (MST)	Release Rate (g/s)	Temperature (°C)
1	26 July	11:30	0.1922	37.0
2	27 July	11:00	0.1460	36.5
3	4 August	12:30	0.1218	32.6
4	5 August	12:00	0.1466	32.1
5	13 October	03:30	0.0147	13.5
6	20 October	03:30	0.0120	12.9
7	21 October	03:30	0.0120	12.3
8	26 October	18:00	0.0119	21.4

summarizes the release information for PSB2. The release rates were designed to provide measurable tracer concentrations within the dynamic sampling ranges of the bag and fast response samplers at all sampling distances [9].

Release temperatures were not available for PSB1, while they were available for PSB2. The analysis of the release temperature of IOPs 1–4 of PSB2 that were performed at similar hours to those of PSB1, though in different months, showed that—on average—they are about 5 °C higher than the average air temperature measured at 2 m agl at the GRI meteorological tower during the releases. Therefore, the temperatures reported in Table 1 were obtained by adding 5 °C to the average air temperature measured during the releases of PSB1.

The release temperatures reported in Table 2 were not indicated in the technical reports describing PSB2 [9]. They were calculated by averaging over the 2.5 h of emission the gas temperature reported in the release files with a time resolution of 1 s.

2.1.2. Meteorological Data

Meteorological data were collected with different instruments both in PSB1 and PSB2. This paragraph provides a brief summary of the measures that were collected. The detailed information is available in [8] for PSB1 and [9] for PSB2.

The following instrumentation was common to PSB1 and PSB2:

• A tower of 62.3 m of height (GRI) over which different instruments were mounted, including sonic anemometers. The GRI was located at an arc distance of 200 m from the INL Grid 3 center (release location) with an angle of 235 degrees. Wind direction and wind speed were measured at different heights above the ground: 2 m, 9.96 m, 15.31 m, 45.1 m and 60.05 m. Air temperature was measured at 1.5 m, 10.7 m, 14.9 m, 45 m and 59.6 m. Solar radiation, barometric pressure and precipitation were also measured at GRI. Three sonic anemometers were placed at heights of 4 m, 30 m and 45 m, and were named G1, G2 and R1, respectively. Four further sonic anemometers were placed at 2 m, 8 m, 16 m and 60 m above the ground.
• A lower tower (COC) of 30 m was located at an arc distance of 499 m from the INL Grid 3 center with an angle of 60 degrees. Wind speed and direction were measured over this tower at the following heights above the ground: 2 m, 10 m and 30 m. These meteorological variables were available with a time resolution of 5 min in PSB1 and of 10 min in PSB2. For PSB2, the 1 s time resolution files were also available.
• An energy flux station (FLX) was located at an arc distance of 916 m from the INL Grid 3 center with an angle of 51 degrees. It is a permanent installation designed to measure how the shrub-steppe habitat of the INL interacts with the global energy cycle. Measurements were performed on two separate towers as follows: air temperature, relative humidity, barometric pressure and solar radiation were measured on the tripod of the first tower, while a sonic anemometer (at 3.2 m above the ground) and open path infrared gas analyzer (IRGA) were mounted on the tripod of the second tower.
• A radar wind profiler (PRO) and a Radio Acoustic Sounding System (RASS) were co-located at an arc distance of 828 m from the INL Grid 3 center with an angle of 56 degrees. The PRO was configured to take measurements of wind speed and direction at 28 levels covering a vertical range from 159 m to 2895 m with a vertical resolution set at 101 m. Wind data were collected for 25 min intervals twice each hour from minute 5 to minute 30 and from minute 35 to minute 60. The RASS was configured to measure air temperature with a vertical resolution of 105 m from 165 m to 1633 m. Temperature data were collected for 5 min intervals twice each hour from minute zero to minute 5 and from minute 30 to minute 35.
• A minisodar (ASC) was located at an arc distance of 816 m from the INL Grid 3 center with an angle of 57 degrees. It is a remote sensing device that measures the vertical profiles of wind speed and direction in the lowest levels of the atmosphere. The ASC vertical range was set from 30 m to 200 m, with a height resolution of 10 m.
• Meteorological balloons (radiosonde) were launched at an arc distance of about 400 m from the INL Grid 3 center with an angle of about 55 degrees before and after each IOP. For each IOP, the first balloon was launched about 15 min before the sampling start, and the second balloon was launched about 15 min after the sampling end.
• Additionally, many meteorological stations managed by ARLFRD are present across the Eastern Snake River Plain covering the test area. These stations measure wind speed and direction, air temperature, relative humidity, precipitation, solar radiation and other variables. Data are averaged over 5 min periods. The stations are classified within two hypothetical rings, based on the distance from the test area. Ring 1 contains the 9 meteorological stations within a radius of 10 miles, while ring 2 contains the 10 meteorological stations located between 10 and 20 miles.

During PSB1 the following meteorological instrumentation was present:

• A 10 m open lattice aluminum meteorological tower (TOW) was located near the center of the 3200 m arc at about 44.5 degrees arc angle. Cup anemometers and vanes were used to measure wind speed and direction at 2 m and 10 m heights. Due to problems with power supply, TOW did not work during IOP4 and IOP5 of PSB1.
• A 3D sonic anemometer (R2) was located close to TOW. It measured wind speed, wind direction and turbulence fields at 3.2 m above the ground.
• A 3D sonic anemometer (R3) was located at an arc distance of 3200 m at 7 degrees arc angle. It measured wind speed, wind direction and turbulence fields at 3.2 m above the ground.
• A 3D sonic anemometer (R4) was located at an arc distance of 3200 m at 82 degrees arc angle. It measured wind speed, wind direction and turbulence fields at 3.2 m above the ground.
• A minisodar (ART) was located close to TOW. It suffered the same power supply problems of TOW; therefore, it did not work during IOP4 and IOP5 of PSB1.

During PSB2 the following instrumentation was present for the IOPs from 5 to 8 (October 2016), excluding the sonic anemometer G2, that was present during all the IOPs:

• A ceilometer was located at an arc distance of about 400 m from the INL Grid 3 center with an angle of about 54 degrees to measure the boundary layer height and clouds base. It did not work in IOP8.
• Six 3D sonic anemometers were located at different positions to measure wind speed, wind direction and turbulence fields. The height above the ground was about 3 m for all the instruments. G2 (present during all the IOPs) was located at an arc distance of 1000 m at 150 degrees arc angle. EC1, EC2 and ST2 were located at an arc distance of 400 m at 25, 85 and 150 degrees arc angle, respectively. ST1 and ST3 were located at an arc distance of 800 m at 25 and 277 degrees arc angle, respectively.

It is evident from this brief description that Project Sagebrush is important not only to validate air dispersion models, but also to validate meteorological models, as done, for example, by [10,13,14].

2.1.3. Samplings

SF6 concentrations were measured with programmable bag samplers that acquired time-sequenced air samples in 12 individual Tedlar^® bags for each measurement point. The sampling time for each bag was 10 min in order to cover each of the 2 h long IOPs. The samples were then analyzed at the Tracer Analysis Facility (TAF) in Idaho Falls, ID, by means of gas chromatographs (GC). A full description of the sampling procedure is available in [8,9].

During PSB1, a total of 112 primary bag samplers were deployed on four sampling arcs during each IOP. For IOPs 1–3, these samplers were deployed on the 400 m, 800 m, 1600 m and 3200 m arcs. For IOPs 4 and 5, the samplers were deployed on the 200 m, 400 m, 800 m and 1600 m arcs. These samplers were at 1 m agl. Additionally, other samplers were deployed at different heights above the ground at three positions: at 201 m from the source with an angle of 53 degrees (1 m, 5 m, 10 m and 15 m agl); at 408 m from the source with an angle of 59 degrees (1 m, 5 m, 10 m, 15 m and 20 m agl); at 499 m from the source with an angle of 60 degrees (1 m, 5 m, 10 m, 15 m, 20 m, 25 m and 30 m agl). The angles are clockwise from north. Twenty-two other samplers were deployed for quality control purposes. They included 16 field duplicates, 3 field control, and 3 field blank samplers.

During PSB2, 150 samplers were deployed for each IOP; they were mounted at 1 m agl. For the daytime IOPs (IOPs 1–4), 36 samplers were placed along each of the 100 m, 200 m and 400 m arcs. They were placed at 6-degree intervals from 276 degrees azimuth to 126 degrees azimuth. Additionally, 16 samplers were deployed on the 800 m arc at 6-degree intervals from 0 degrees azimuth to 90 degrees azimuth. For the nighttime IOPs (IOPs 5–8), 36 samplers were placed along each of the 100 m, 200 m and 400 m arcs at 6-degree intervals from 312 degrees azimuth to 162 degrees azimuth. For the daytime IOPs, vertical sampling was conducted at 1 m, 5 m, 10 m, 15 m, 20 m and 25 m agl on the 100 m arc, at an angle of 24 degrees for IOPs 1–2, and 352 degrees for IOPs 3–4. For the nighttime IOPs, vertical sampling was conducted at the same heights on the 400 m arc at an angle of 57 degrees. Additionally, bag samplers were deployed on four fixed 10 m towers on the 100 and 200 m arcs at 1, 3, 6 and 9 m heights. As for PSB1, other samplers were deployed for quality control purposes.

2.2. Simulation Models

2.2.1. CALMET

CALMET [21] is a diagnostic meteorological model which reconstructs the 3D wind and temperature fields and micrometeorological variables (e.g., mixing height, Monin–Obukhov length, friction and convection velocity) starting from meteorological measurements—at the surface and at upper layers—and/or output of a prognostic model, and geophysical data (orography and land use).

The micro-meteorological parameters are needed by the dispersion models to calculate their turbulent dispersion coefficients according to the different parameterizations implemented. The unformatted output of CALMET is directly read both by CALPUFF and LAPMOD, without the need for any post-processing.

The diagnostic module uses a two-step approach for the computation of the wind field. In the first step, an initial guess wind field is adjusted for kinematic effects of terrain, slope flows and terrain blocking effects. The second step consists in an objective analysis procedure to introduce observational data into the wind field of the first step to produce a final wind field.

CALMET version 6.5.0, level 150223 was used in this study to produce the output variables with a time step of 300 s (5 min).

2.2.2. CALPUFF

CALPUFF [18] is a Lagrangian puff dispersion model belonging to the list of alternative models of the US-EPA [22]. Alternative models are those that can be used in regulatory applications with case-by-case justification to the Reviewing Authority in situations where the preferred models are not applicable or available. The puffs are advected by the average wind along their vertical extension, while the atmospheric turbulence has the effect of increasing their size [18]. When puffs become very high, wind shear cannot be correctly described. For this reason, CALPUFF includes a “puff-splitting” algorithm that allows splitting of puffs under specific conditions.

CALPUFF is capable of simulating many types of sources, including point, area, buoyant line and volume. Starting from version 7, CALPUFF is also capable of simulating road sources.

The model calculates the concentration at a specific receptor by summing the contribution of each puff at such positions with an analytical expression [23].

CALPUFF version 7.2.1 level 150618 was used in this study.

2.2.3. LAPMOD

LAPMOD [19] is an open source Lagrangian particle model that simulates many types of sources. Computational particles are transported by the combination of the wind at their height and a random component, which is calculated according to the turbulence level [24].

Two numerical plume rise schemes are implemented in LAPMOD for point sources: Janicke and Janicke [25] and Webster and Thomson [26]. LAPMOD describes the orientation of the stack tip by means of two angles: the azimuthal angle (stack orientation on the horizontal plane) and the polar angle (tilting of the stack with respect to the vertical). Rain-capped stacks can also be simulated.

The concentration fields in LAPMOD are calculated with kernel methods (e.g., [27]), that allow using relatively fewer particles with respect to Lagrangian particle models that adopt a counting method to calculate concentrations [28].

Validations of LAPMOD against the experimental datasets of Kincaid (rural conditions), and Indianapolis (urban conditions) can be found in [29,30]. The results of the two validations describe LAPMOD as a reliable model according to the performance evaluation criteria proposed by [31] that are based on FA2, NMSE and fractional bias [32]. An intercomparison between LAPMOD and other dispersion models for odor applications was presented in [33]. Another intercomparison between LAPMOD and CALPUFF, specifically for rain-capped and horizontal stacks, is discussed in [34]. Finally, [35] described the introduction in LAPMOD of a modified turbulence scheme to better represent the dispersion of radionuclides at a Chinese nuclear power plant when buildings are present. Their results show that agreement between wind tunnel observations and model predictions increases when the new scheme is activated. This may be an important result when considering odor pollution, because buildings are often present and the impacts occur over relatively short ranges.

LAPMOD version 2025-03-25 was used in this study.

2.2.4. Configuration of the Two Dispersion Models

The emissions were determined by means of a point source placed 1.5 m agl with a diameter of 1.59 cm. Since the release direction was horizontal, the vertical mechanical momentum was suppressed by forcing the exit velocity to 0.001 m/s, as suggested by the US-EPA [36]. The release temperature and release rate for each IOP are reported in Table 1 and Table 2.

The values used for the most important input variables are summarized in

Table 3. CALPUFF input variables.

Variable	Description	Value
NSECDT	Length of modeling time-step (s)	300
MGAUSS	Vertical distribution used in the near field (0: Uniform; 1: Gaussian)	1
MCTADJ	Terrain adjustment method (3: Partial plume path adjustment)	3
MRISE	Plume rise method (0: Briggs; 1 Numerical)	1
MWET	Wet removal (0: Not modeled; 1: Modeled)	0
MDRY	Dry removal (0: Not modeled; 1: Modeled)	0
MDISP	Method to compute dispersion coefficients	2
MPARTL	Partial plume penetration of elevated inversion modeled (0: No; 1: Yes)	1
MPDF	PDF used for dispersion under convective conditions (0: No; 1: Yes)	0

MDISP = 2 means that dispersion coefficients are obtained from internally calculated sigma v, sigma w using micrometeorological variables.

and

Table 4. LAPMOD input variables.

Variable	Description	Value
LDEPD	Activation of dry deposition	False
LDEPW	Activation of wet deposition	False
ITDMQ	Time step (s) to query meteorological variables at particle’s position (0 to interpolate at any timestep)	0
LNOW	Suppress the vertical component of the CALMET wind speed	False
NPART	Particles released every 60 s	120
IPRTYPE	Plume rise type (1 for Janicke and Janicke; 2 for Webster and Thomson)	2
LSTD	Activate stack tip downwash	False
LPPP	Activate partial plume penetration	True
LPIT	Activate plume-induced turbulence	True
IDTORD	Time step to write concentrations (s)	600
NSAM	Number of samplings to get average concentrations over IDTORD	2
CCA	Algorithm for calculating concentrations (1: Gaussian kernel; 2: Uliasz uniform kernel; 3: Uliasz parabolic kernel)	1
SIGNUM	Number of sigma units defining the volume associated with each particle and for searching contributing particles	3

, respectively, for CALPUFF and LAPMOD.

The stack tip downwash (STD) was not activated because the release is horizontal and, in any case, the release diameter is very small (the effect of STD is to decrease the stack height by a maximum of 3 diameters). Dry and wet deposition were not activated.

The positions of the discrete receptors (sampling sites) used to validate the models in this study are represented in Figure 1 by means of blue squares, while the red circle represents the source location. All the receptors are placed at 1 m agl. It is noted that the three panels of Figure 1 have different scales. During PSB1 (Figure 1a), the receptors placed from 400 m to 3200 m from the source were used for IOP1-3, while the receptors placed from 200 m to 1600 m from the source were used for IOP4-5. Concerning PSB2, the receptors used in convective conditions during the summer (IOP1-4) are shown in Figure 1b, while those used in stable conditions during the nights or in the late afternoon of October are shown in Figure 1c.

2.3. Meteorological Fields of Each IOP

As described above, a lot of meteorological instrumentation was present within and around the area of study. Therefore, considering also that the area is practically flat, the meteorological fields during the IOPs were determined using only the CALMET diagnostic meteorological model [21]. An alternative procedure would be, for example, to use—in addition to CALMET or alone—a prognostic meteorological model like WRF [11], as performed for example in [10,13]. However, in this work it is assumed that, over such a small area, the meteorological fields can be reliably reproduced using the extensive meteorological measurements that were carried out in PSB.

2.3.1. CALMET Domain

The domain of the CALMET meteorological model is shown in Figure 2. The UTM coordinates of its lower left corner are E = 336,000, N = 4,821,000 within zone 12T. In each direction, 70 grids were used, with a resolution of 200 m; therefore, the size of the square domain is 14 km. A smaller grid size (i.e., better spatial resolution) was not deemed necessary because the area of interest is mostly flat.

The terrain level over the simulation domain was determined with the SRTM data (Shuttle Radar Topography Mission) [37] with a spatial resolution of about 30 m on the line of the equator. The average terrain level over the 200 m grids ranges from 1471 m (northeastern part of the domain) to 1554 m (northwestern part of the domain). The average terrain elevation over the CALMET domain is 1502 m. The Copernicus Global Land Cover Dataset 2019 [38] with a resolution of 100 m was used for describing the land use. Almost all the grid cells (99.3%) are classified with the land use “rangeland”, which is associated with a roughness length z₀ of 5 cm. This value is in good agreement with those found in [8,9,15] using wind profiles: a median z₀ of 3 cm for SW winds (typically, common during the day), and a median z₀ of 3.8 cm for NE winds (typically, common during the night).

Along the vertical direction, 12 levels were used, with the top of domain at 4000 m agl.

2.3.2. Surface Meteorological Data

The available surface meteorological measurements collected during PSB were redundant when running CALMET; therefore, some of them were not used in input to the model. For example, the Mesonet station GRI that measures wind at 15 m agl was located exactly where the 62.3 m high tower was located; therefore, its measurements are redundant. After verifying that the Mesonet GRI wind and the wind at 15 m at the tower were comparable, it was decided to keep the observations at the tower.

Cloud cover and sky altitude were obtained from the two ASOS stations closest to the test area: Pocatello Regional Airport (PIH), located approximately 80 km SSE from the test area, and Friedman Memorial Airport (SUN), located approximately 110 km west from the test area. The ASOS data were downloaded from the Iowa State University internet site [39]. These data report sky altitude in feet, while cloud cover is reported with a 3-letter category (e.g., CLR, FEW, SCT). Following the indications presented in Table 3 of [40], an equivalence was established between the category and the cloud cover in tenths needed by CALMET, as reported in

Table 5. Equivalence between cloud cover category and cloud cover in tenths.

ASOS Cloud Cover	Description	Cloud Cover (Tenths)
CLR	Clear	0
FEW	Few	1
SCT	Scattered	3
BKN	Broken	7
OVC	Overcast	10

. Data were available with a time resolution of 1 h in 2013 (PSB1) and of 5 min in 2016 (PSB2) in station PIH, while they were available with a time resolution of 20 min in both 2013 and 2016 in station SUN. Cloud cover and ceiling height for CALMET were determined starting from the data of the closest station (PIH) with the intention to consider the second station (SUN) only as a backup when one of the two variables was not available in PIH. In the event, there was no need to use the SUN data because the PIH data were complete. The cloud cover and ceiling height in PSB1 varied every 60 min or 20 min, while in PSB2, they varied every 5 min or 20 min. These two variables were considered uniform over the whole domain.

The observations of the following “surface stations” were used to feed CALMET in both PSB1 and PSB2:

• GRI Tower (62 m) at a height of 2 m agl. The ceiling height and cloud cover from the PIH ASOS station were assigned to this station. The code used in CALMET to identify this station is 10002.
• Five Mesonet stations within the CALMET domain (excluding GRI): 690, LOS, NRF, PBF and TRA. The codes used in CALMET to identify these stations are 12001 (690), 12002 (LOS), 12003 (NRF), 12004 (PBF) and 12005 (TRA).
• COC Tower (30 m) at a height of 2 m agl (only wind speed and direction). The code used in CALMET to identify this station is 11002.

Additionally, three 3D sonic anemometers R2, R3 and R4 (station codes: 13002, 13003 and 13004) were used in PSB1, while TOW was not used due to the power supply problems in IOP4 and IOP5. During PSB2, the G2 sonic anemometer was used (station code 14003), except in IOP4, in which it was not available.

The UTM12T coordinates and the anemometer height of the meteorological stations used in CALMET are summarized in

Table 6. UTM12T Coordinates and anemometer height of the surface stations used in CALMET.

ID	E (m)	N (m)	H agl (m)	Description
10002	343,398	4,828,134	2.0	GRI at 2 m agl
12001	342,618	4,821,805	15.0	Central Facilities (690)
12002	337,756	4,823,692	15.0	Lost River Rest Area (LOS)
12003	345,867	4,834,540	15.0	Naval Reactors Facility (NRF)
12004	348,963	4,823,312	15.0	Critical Infrastructure Complex (PBF)
12005	341,066	4,827,621	15.0	Reactor Technologies Complex (TRA)
11002	344,002	4,828,486	2.0	COC at 2 m agl
13002	345,847	4,830,495	3.2	Sonic anemometer R2
13003	344,005	4,831,403	3.2	Sonic anemometer R3
13004	346,736	4,828,642	3.2	Sonic anemometer R4
14003	344,077	4,827,352	3.1	Sonic anemometer G2

. The position of the surface meteorological stations with respect to the source is shown in Figure 2.

In CALMET, the wind measured by surface stations, within the routine DIAGNO, is always extrapolated from the anemometer height to first model the vertical level using a neutral log profile over land. This means that, since the first (lower) vertical cell in CALMET must be from the ground (0 m agl) to 20 m agl, and the horizontal wind components are calculated in the middle of this cell, the lower height at which wind is available is 10 m agl. Then, when many observations were available at different heights at the same position—as for the GRI and COC towers—only the lower measurements were used, to avoid wind extrapolation at 10 m agl from the measurements carried out, for example, at 2 m, 15 m, 20 m, etc. The choice to consider the observations of lower height is because they are the most representative for the release (source height is 1.5 m agl), and the lowest observation height at the towers is 2 m agl.

The meteorological variables measured by the COC tower are available with a time resolution of 5 min in PSB1 and 10 min in PSB2. For PSB2, the 1 s time resolution files were also available. In order to use the same time resolution (5 min) in both phases, a Python program (version 3.9.1) was developed to obtain the 5 min averages from the 1 s ones. To check the correctness of the Python program, it was used to calculate the 10 min averages, that were then compared with those already available in the dataset.

The SURFCSV meteorological processor of CALMET was used to prepare the surface meteorological file of each single station; then, the different files of a specific IOP were merged to obtain a single SURF.DAT file.

2.3.3. Upper Air Meteorological Data

The CALMET simulation must start before 5 LST to correctly initialize the mixing height; also, the radio soundings must be available before and after the initial and final simulation times. During PSB local radio soundings were performed 15 min before and after the tracer release; then, excluding IOP5-7 of PSB2, the initial sounding was always after 5 LST (see Table 1 and Table 2). Therefore, additional radio soundings were needed to carry out the CALMET simulations. These vertical data were collected from the Boise airport (USAF: 72681) located about 264 km west of the release site, at a height of 874 m asl. The Boise vertical profiles were downloaded from the internet site of the University of Wyoming, Department of Atmospheric Science [41]. On 4 and 5 August 2016 (IOP3 and IOP4 of PSB2) some vertical profiles were not available from the University of Wyoming. Therefore, for those two days, the vertical profiles were downloaded from the IGRA (Integrated Global Radiosonde Archive) [42].

Also, the initial local vertical profile (the one starting approximately 15 min before the sampling) for IOP6 of PSB2 (20 October 2016) was not available, probably due to technical problems. Therefore, in order to perform the CALMET simulation, the Boise profile was used.

The final UP.DAT files used in input for each CALMET simulation were obtained by merging the local and the Boise vertical profiles. Each UP.DAT file contains the values of pressure, height above sea level (asl), temperature, wind direction and wind speed for different vertical levels. The maximum height considered was 10,000 m asl.

The launch position of the local radio soundings was very close to the COC surface station shown in Figure 2.

2.4. Statistical Parameters

The following statistical indexes were used to quantitatively evaluate the models’ performances (e.g., [32,43]): percent of predictions within a factor of 2 (FA2) and 5 (FA5) from the observations, factor of exceedance (FOEX), correlation coefficient (R), fractional bias (FB), normalized mean square error (NMSE) and figure of merit in space (FMS).

The FAα, where typically α = 2 and/or α = 5, is determined as the percentage of values that satisfy Equation (1), with Pi and Mi that are the i-th prediction and measurement, respectively.

$${\displaystyle \frac{1}{\mathrm{\alpha }}}\le {\displaystyle \frac{{\mathrm{P}}_{\mathrm{i}}}{{\mathrm{M}}_{\mathrm{i}}}}\le \mathrm{\alpha }$$

(1)

The FOEX is determined as the number of overpredictions N_{(Pi > Mi)}, divided by the total number N of data, minus 0.5, and expressed as a percentage (Equation (2)). Therefore, it ranges from −50% (the model always underpredicts) to +50% (the model always overpredicts).

$$\mathrm{F}\mathrm{O}\mathrm{E}\mathrm{X}=100\times \left({\displaystyle \frac{{\mathrm{N}}_{\left({\mathrm{P}}_{\mathrm{i}}>{\mathrm{M}}_{\mathrm{i}}\right)}}{\mathrm{N}}}-0.5\right)$$

(2)

The correlation coefficient R is defined by Equation (3), where the overbars represent the average value. R ranges between −1 and +1. A value R = +1 (complete positive correlation) corresponds to all the pairs (Mi, Pi) laying on a straight line with positive slope in the scatter diagram. The complete negative correlation (R = −1) corresponds to all the pairs on a straight line with negative slope. A value of R near to zero indicates the absence of correlation between the observations and predictions.

$$\mathrm{R}={\displaystyle \frac{\sum _{\mathrm{i}=1}^{\mathrm{N}}\left({\mathrm{M}}_{\mathrm{i}}-\overline{\mathrm{M}}\right)\left({\mathrm{P}}_{\mathrm{i}}-\overline{\mathrm{P}}\right)}{\sqrt{\sum _{\mathrm{i}=1}^{\mathrm{N}}{\left({\mathrm{M}}_{\mathrm{i}}-\overline{\mathrm{M}}\right)}^2\sum _{\mathrm{i}=1}^{\mathrm{N}}{\left({\mathrm{P}}_{\mathrm{i}}-\overline{\mathrm{P}}\right)}^2}}}$$

(3)

The fractional bias (FB) is defined as shown by Equation (4). FB ranges from −2 to +2, where positive values indicate overprediction and negative values indicate underprediction.

$$\mathrm{F}\mathrm{B}=2{\displaystyle \frac{\overline{\mathrm{P}}-\overline{\mathrm{M}}}{\overline{\mathrm{P}}+\overline{\mathrm{M}}}}$$

(4)

The normalized mean square error (NMSE) is defined by Equation (5). It is always a positive number giving information about the deviation between predictions and measurements.

$$\mathrm{N}\mathrm{M}\mathrm{S}\mathrm{E}={\displaystyle \frac{1}{\mathrm{N}}}\sum _{\mathrm{i}=1}^{\mathrm{N}}{\displaystyle \frac{{\left({\mathrm{P}}_{\mathrm{i}}-{\mathrm{M}}_{\mathrm{i}}\right)}^2}{\overline{\mathrm{P}}\overline{\mathrm{M}}}}$$

(5)

The figure of merit in space (FMS) is calculated as the intersection over the union of predicted p and measured m concentrations in terms of the number N of samples with concentrations greater than a given threshold [43]. The threshold used in this analysis has been chosen arbitrarily as a small value (5 ppt).

$$\mathrm{F}\mathrm{M}\mathrm{S}=100\times {\displaystyle \frac{{\mathrm{N}}_{\mathrm{p}}\cap {\mathrm{N}}_{\mathrm{m}}}{{\mathrm{N}}_{\mathrm{p}}\cup {\mathrm{N}}_{\mathrm{m}}}}$$

(6)

In order to evaluate the model performances at a glance, without considering in detail all the statistical indexes, a rank was defined by combining those indexes, similarly to the procedure described in [43]. A rank ranging from 0 (worst model performance) to 5 (best model performance) was defined in this study as:

Rank = R + (1 − |FB/2|) + (1 − 2 |FOEX|/100) + FA5/100 + FMS/100

(7)

The conversion from the concentration values calculated by the two models (in µg/m³ or ng/m³) to concentrations (mixing ratio) expressed in pptV was carried out as in [8], considering the mean atmospheric temperature (K) and pressure (Pa) at GRI during each of the two-hour release periods.

A background of 9 ppt was subtracted from all SF6 observations of PSB1 and negative values set to zero, as performed, for example, in [13]. The value of 9 ppt was chosen because it is the ceiling of the average value reported in the paragraph titled “Atmospheric background checks of SF6 at the tracer analysis facility” in [8]. Concerning PSB2, the same paragraph in [9] reports a background value a bit higher; therefore, with similar reasoning, the background value used for PSB2 was 10 ppt.

3. Results and Discussion

The results are presented in the following subparagraphs for each IOP, both by pairing in time and space the observed and predicted concentrations, both by time-averaging the observed and predicted concentrations at each sampling point over the 2-hour-long sampling interval. Of course, the comparison of time-paired data is more demanding; therefore, lower agreement between observations and predictions is expected with respect to the time-averaged data.

For each IOP, the 5 min CALMET data were extracted at the release location (E = 343,571.59, N = 4,828,245.40, UTM zone 12T) for the first vertical level (10 m agl) and for the 2 h length of the sampling interval (i.e., without considering the first 30 min of the release). The wind roses obtained from those data are then represented to better understand the fate of the SF6 plume. It is anticipated that, for PSB1, the wind roses obtained in this work would be quite similar to those reported in [14]. No wind roses were found in the scientific literature for PSB2.

The statistical analysis was carried out by considering only the observations associated with quality flags lower than 3 (values flagged with 3 or more are of bad quality) [8,9].

3.1. PSB1–IOP1

As reported by [8], during IOP1 conducted on 2 October 2013 from 14:30 to 16:30 MST, winds were very light and variable with mostly sunny conditions but filtered through patchy cirrostratus. Overall conditions were highly non-stationary and the adverse wind directions made for generally poor tracer sampling. The wind rose obtained from CALMET at the release location (Figure 3) is consistent with such a description. Calms (i.e., winds below 0.5 m/s) are predicted for 25% of the time. The average wind speed determined by CALMET during IOP1 is 0.9 m/s, with a maximum value of 2.0 m/s. The mode of the wind speed distribution is between 1 m/s and 2 m/s, with exactly 50% of the events. The wind blows prevalently from E-SE; therefore, the cloud is going to miss the sampling sites represented in Figure 1a most of the time. Indeed, as shown, for example, in Figure 4, the computational particles of LAPMOD at the end of the IOP (16:30 MST) close to the source are directed toward the west, while higher particles, far from the source, go toward the northeast, lightly impacting on the sampling points close to north (i.e., those with smaller angles). However, since these particles are relatively high over the ground, the calculated concentration values are small.

The Pasquill–Gifford (PG) stability classes calculated by CALMET for PSB1-IOP1 are always very unstable (class B, excluding class C during the first five minutes of the release).

Since the SF6 cloud is missing the sampling points, the statistical analysis is not reported for PSB1-IOP1.

3.2. PSB1–IOP2

During IOP2 of PSB1, conducted on 5 October 2013 from 13:00 to 15:00 MST, the weather was mostly sunny, and the winds were generally relatively light (under 3 m/s) during the first half of the release, increasing somewhat over the second half. Wind directions varied but were consistently southwesterly with few exceptions. This description, reported by [8], agrees with the wind rose obtained with CALMET at the release location (Figure 5). The average wind speed determined by CALMET during IOP2 is 3.1 m/s, with a maximum value of 5.0 m/s. The mode of the wind speed distribution is between 2 m/s and 3 m/s, with 45.8% of the events. The PG stability classes calculated by CALMET for PSB1-IOP2 are very unstable for the first 75 min of the release (class B), then evolve to moderately unstable (class C) for the remaining time, excluding the last five minutes, that are class D.

The statistical parameters calculated for CALPUFF and LAPMOD are summarized in

Table 7. Statistical parameters for PSB1-IOP2.

Statistics	Paired	400 m arc	800 m arc	1600 m arc	3200 m arc
N	1489	28	28	28	28
Max Obs (ppt)	62,484	14,429	1936	438	140
Max LAPMOD (ppt)	121,920	9568	1816	335	126
Max CALPUFF (ppt)	121,722	22,263	5583	1870	554
FA2 LAPMOD	27.5	60.7	67.9	60.7	64.3
FA5 LAPMOD	42.3	100.0	92.9	89.3	82.1
FA2 CALPUFF	22.2	25.0	7.1	0.0	3.6
FA5 CALPUFF	40.8	64.3	46.4	39.3	46.4
Corr. LAPMOD	0.42	0.56	0.47	−0.03	0.19
Corr. CALPUFF	0.72	0.30	−0.20	−0.24	0.23
FB LAPMOD	0.25	0.08	0.14	0.21	−0.02
FB CALPUFF	1.12	1.00	1.32	1.50	1.35
FMS LAPMOD	68.9	100.0	100.0	100.0	92.6
FMS CALPUFF	81.2	100.0	100.0	100.0	96.4
NMSE LAPMOD	13.26	0.25	0.29	0.40	0.40
NMSE CALPUFF	8.43	1.50	3.37	5.93	4.07
FOEX LAPMOD	−10.0	14.3	14.3	17.9	0.0
FOEX CALPUFF	32.4	50.0	50.0	50.0	50.0
Rank LAPMOD	3.2	4.2	4.0	3.4	3.9
Rank CALPUFF	2.7	2.4	1.6	1.4	2.0

. Considering the time- and space-paired data (N = 1489), the two models predict almost the same maximum concentration value, which is more or less double the maximum observation. The values of FA2 and FA5 are comparable for the two models. The FOEX indicates that LAPMOD slightly underpredicts, while CALPUFF overpredicts. The ranks indicate better agreement of LAPMOD with the observations with respect to CALPUFF.

Concerning the time-averaged concentrations at each sampling point of the four arcs, the statistical parameters in Table 7 show a good agreement of LAPMOD with the observations. The ratio between the LAPMOD and the CALPUFF ranks is close to or greater than 2. Also, for the arcs at 400 m and 800 m, the LAPMOD rank is at least 4, with the best theoretical value equal to 5. Even at the arc of 3200 m, the LAPMOD rank is close to 4. The time-averaged concentrations at each sampling point of the arcs are represented in Figure 6.

3.3. PSB1–IOP3

As described by [8], IOP3 of PSB1 was conducted on 7 October 2013 from 13:00 to 15:00 MST. The weather was mostly sunny, and winds were consistently moderate to strong, in excess of 7 m/s throughout the release under generally clear skies, and wind directions were consistently southwesterly and showed minimal variation. All estimates of stability indicate near-neutral conditions. Flows exhibited a high degree of spatial and temporal homogeneity and stationarity. This description is in complete agreement with the wind rose obtained with CALMET at the release location (Figure 7), which shows only two directions: SW, the prevailing one, and SSW. The average wind speed determined by CALMET during IOP3 is 8.8 m/s, with a maximum value of 10.3 m/s. All the wind speeds are above 6 m/s; for this reason, the PG stability classes calculated by CALMET for PSB1-IOP3 are always neutral (class D).

As shown by the statistical parameters summarized in

Table 8. Statistical parameters for PSB1-IOP3.

Statistics	Paired	400 m arc	800 m arc	1600 m arc	3200 m arc
N	1488	27	27	28	28
Max Obs	188,921	67,989	16,307	3309	583
Max LAPMOD	129,856	64,605	13,550	2257	487
Max CALPUFF	71,329	30,055	8545	2377	713
FA2 LAPMOD	47.6	33.3	44.4	53.6	50.0
FA5 LAPMOD	58.7	40.7	51.9	60.7	60.7
FA2 CALPUFF	44.9	63.0	66.7	50.0	46.4
FA5 CALPUFF	57.6	77.8	70.4	67.9	64.3
Corr. LAPMOD	0.73	0.96	0.95	0.92	0.88
Corr. CALPUFF	0.83	0.97	0.96	0.95	0.92
FB LAPMOD	−0.24	−0.23	−0.38	−0.50	−0.22
FB CALPUFF	−0.29	−0.49	−0.31	0.03	0.63
FMS LAPMOD	63.7	66.7	72.2	68.8	64.3
FMS CALPUFF	58.7	78.3	72.7	71.4	65.0
NMSE LAPMOD	5.78	0.22	0.43	1.12	0.85
NMSE CALPUFF	4.94	0.98	0.61	0.30	1.07
FOEX LAPMOD	−15.5	−5.6	−24.1	−32.1	−21.4
FOEX CALPUFF	2.6	−16.7	−5.6	14.3	25.0
Rank LAPMOD	3.5	3.8	3.5	3.3	3.6
Rank CALPUFF	3.8	4.0	4.1	4.0	3.4

, the two models predict quite well the observed concentrations, with CALPUFF showing results that are slightly better than LAPMOD. Considering the time- and space-paired data (N = 1488), the statistical parameters calculated for the two models are very similar; indeed, the final ranks calculated are also similar: 3.5/5 for LAPMOD and 3.8/5 for CALPUFF.

The time-averaged concentrations at each sampling location over the arcs are shown in Figure 8. The concentration distribution over the sampling locations is more or less Gaussian, with the maximum values at the central angles. At the closest arcs, LAPMOD predicts quite well the maximum values, while it underpredicts the lower values at the borders of the arcs. However, notwithstanding this qualitative positive evaluation of LAPMOD, according to the ranks, CALPUFF behaves better in three of the four arcs, with values of at least 4 (where the maximum is 5).

3.4. PSB1–IOP4

As described by [8], IOP4 of PSB1 was performed on 11 October 2013 from 14:00 to 16:00 MST. The weather was mostly sunny, and the meteorological conditions were similar to those observed during IOP3 with some differences. The wind speeds and turbulence were generally lower and winds were generally favorable (i.e., blowing toward the sampling sites) southwesterly. Indeed, the wind rose obtained with CALMET at the release location (Figure 9) shows wind speeds a bit lower than those reconstructed for IOP3 (Figure 7), and the prevailing direction is SW. The average wind speed determined by CALMET during IOP4 is 5.1 m/s, with a maximum value of 6.4 m/s. The mode of the wind speed distribution is between 5 m/s and 6 m/s, with 41.7% of the events. The PG stability classes calculated by CALMET for PSB1-IOP4 are C and D during the first hour of the release, then almost always D during the second hour.

The statistical parameters calculated for IOP4 are summarized in

Table 9. Statistical parameters for PSB1-IOP4.

Statistics	Paired	200 m arc	400 m arc	800 m arc	1600 m arc
N	1464	28	27	26	27
Max Obs	48,669	14,993	3287	587	84
Max LAPMOD	56,052	18,917	2982	617	150
Max CALPUFF	18,152	12,015	3399	970	317
FA2 LAPMOD	31.2	64.3	74.1	42.3	33.3
FA5 LAPMOD	48.0	85.7	81.5	80.8	63.0
FA2 CALPUFF	34.4	92.9	70.4	50.0	25.9
FA5 CALPUFF	56.4	100.0	100.0	76.9	63.0
Corr. LAPMOD	0.69	0.96	0.94	0.78	0.76
Corr. CALPUFF	0.76	0.98	0.97	0.78	0.78
FB LAPMOD	0.07	0.05	−0.17	0.05	0.25
FB CALPUFF	0.03	−0.13	0.23	0.70	1.18
FMS LAPMOD	65.7	100.0	92.6	84.6	54.2
FMS CALPUFF	77.5	100.0	100.0	100.0	69.2
NMSE LAPMOD	5.02	0.14	0.13	0.37	0.90
NMSE CALPUFF	2.61	0.07	0.09	1.02	4.48
FOEX LAPMOD	−13.5	−7.1	−9.3	11.5	5.6
FOEX CALPUFF	19.5	−17.9	46.3	38.5	31.5
Rank LAPMOD	3.5	4.6	4.4	4.2	3.7
Rank CALPUFF	3.7	4.6	3.9	3.4	2.9

and show that the two models predict quite well the observed concentrations, with LAPMOD showing results that are slightly better than CALPUFF. The global analysis with time- and space-paired data (N = 1464) shows that the maximum 10 min average concentration is better predicted by LAPMOD (exceeding the observed value a little) with respect to CALPUFF (underestimating the observed value). On the other hand, the remaining statistical indexes are slightly better for CALPUFF than for LAPMOD, resulting in a final rank slightly better for CALPUFF (3.7/5) than for LAPMOD (3.5/5). Then, as for PSB1-IOP3, the final ranks of the global analysis are similar.

The time-averaged concentrations at each sampling location over the arcs are shown in Figure 10. As for PSB1-IOP3, the concentration distribution over the sampling locations is more or less Gaussian, with the maximum values at the central angles. At the closest arc, both models predict very well the time-averaged observations with a rank of 4.6/5. At the remaining three arcs, the agreement between CALPUFF and the observations decreases, while the one between LAPMOD and the observations remains high, with ranks greater than 4 at the 400 m and 800 m arcs, and a rank of 3.7/5 at the farthest arc (1600 m).

3.5. PSB1–IOP5

IOP5 of PSB1 was performed on 18 October 2013 from 13:00 to 15:00 MST. As described by [8], the weather conditions were similar to those of IOP3 and IOP4: the sky was mostly sunny, and wind came from the SW quadrant. As reported, wind speeds increased from about 3–4 m/s at the start of the tracer sampling period (13:00) to 3–6 m/s at the end (15:00).

The wind rose obtained with CALMET at the release location (Figure 11) is in agreement with the description of the weather conditions, also showing more directional variability with respect to IOP3 and IOP4. The prevailing direction is again SW. The average wind speed determined by CALMET during IOP5 is 4.6 m/s, with a maximum value of 5.2 m/s. The mode of the wind speed distribution is between 4 m/s and 5 m/s, with 66.7% of the events. The PG stability classes calculated by CALMET for PSB1-IOP5 are B and C during the first 40 min of the release, then C and D for the rest of the release.

The statistical parameters calculated for IOP5 are summarized in

Table 10. Statistical parameters for PSB1-IOP5.

Statistics	Paired	200 m arc	400 m arc	800 m arc	1600 m arc
N	1502	28	28	28	28
Max Obs	26,852	13,024	2186	342	67
Max LAPMOD	52,259	14,750	2897	688	139
Max CALPUFF	17,566	11,956	3456	1021	327
FA2 LAPMOD	32.4	60.7	67.9	42.9	35.7
FA5 LAPMOD	53.3	75.0	75.0	64.3	60.7
FA2 CALPUFF	37.7	100.0	53.6	0.0	0.0
FA5 CALPUFF	56.5	100.0	96.4	60.7	3.6
Corr. LAPMOD	0.66	0.89	0.94	0.93	0.95
Corr. CALPUFF	0.86	0.96	0.98	0.96	0.97
FB LAPMOD	0.13	−0.11	−0.02	0.48	0.62
FB CALPUFF	0.23	0.05	0.55	1.20	1.48
FMS LAPMOD	76.6	100.0	82.1	79.2	93.8
FMS CALPUFF	78.5	100.0	100.0	85.7	66.7
NMSE LAPMOD	4.52	0.23	0.15	0.97	1.53
NMSE CALPUFF	1.07	0.03	0.44	3.42	8.62
FOEX LAPMOD	−3.5	−21.4	−14.3	10.7	28.6
FOEX CALPUFF	25.2	14.3	46.4	50.0	50.0
Rank LAPMOD	3.8	4.2	4.2	3.9	3.6
Rank CALPUFF	3.6	4.7	3.7	2.8	1.9

. As for IOP4, they show that the two models predict quite well the observed concentrations, with LAPMOD showing results that are slightly better than CALPUFF, particularly for the farthest arc (1600 m). The global analysis with time- and space-paired data (N = 1502) shows that the statistical indexes calculated from the results of the two models are very similar; indeed, the final ranks are 3.8/5 and 3.6/5, respectively, for LAPMOD and CALPUFF.

The time-averaged concentrations at each sampling location over the arcs are shown in Figure 12. Again, the concentration distribution over the sampling locations is more or less Gaussian, with the maximum values at the central angles. The charts show that, excluding the closest arc (200 m), CALPUFF has a general tendence to overestimate the observations. At the closest arc, both models predict very well the time-averaged observations, with a rank of 4.2/5 for LAPMOD and 4.7/5 for CALPUFF. As observed for IOP4, at the remaining three arcs, the agreement between CALPUFF and the observations decreases, while the one between LAPMOD and the observations remains high: 4.2/5 at the 400 m arcs, 3.9/5 at the 800 m arcs, and 3.6/5 at the farthest arc (1600 m). At the farthest arc the CALPUFF rank drops to 1.9/5.

3.6. PSB2–IOP1

IOP1 of PSB2 was performed on 26 July 2016 from 12:00 to 14:00 MST. As described by [9], the weather conditions were hot and dry with light and variable winds throughout the IOP. The skies were clear and sunny during the first hour of the measurement period with some cloudiness during the second hour. Overall conditions were unstable and highly non-stationary in both time and space.

The wind rose obtained with CALMET at the release location (Figure 13) shows great variability of the wind, both for direction and intensity. Considering the positions of the sampling sites during this IOP (Figure 1b), a certain mass of SF6 was not sampled, specifically, the one emitted while wind was blowing from NE. The average wind speed determined by CALMET during IOP1 is 2.3 m/s, with a maximum value of 4.1 m/s. The mode of the wind speed distribution is between 2 m/s and 3 m/s, with 45.8% of the events. The PG stability classes calculated by CALMET for PSB2-IOP1 are always very unstable (A or B).

The statistical parameters calculated for PSB2-IOP1 are summarized in

Table 11. Statistical parameters for PSB2-IOP1.

Statistics	Paired	100 m arc	200 m arc	400 m arc	800 m arc
N	1507	37	36	36	15
Max Obs	11,540	2807	584	117	25
Max LAPMOD	13,907	2879	489	177	13
Max CALPUFF	5726	2715	823	215	64
FA2 LAPMOD	18.6	59.5	61.1	55.6	66.7
FA5 LAPMOD	33.4	91.9	86.1	94.4	86.7
FA2 CALPUFF	19.0	75.7	47.2	5.6	0.0
FA5 CALPUFF	36.0	91.9	72.2	38.9	46.7
Corr. LAPMOD	0.30	0.55	0.54	0.11	0.31
Corr. CALPUFF	0.53	0.56	0.45	0.37	0.77
FB LAPMOD	−0.10	−0.19	−0.20	0.10	−0.03
FB CALPUFF	0.38	0.28	0.81	1.24	1.33
FMS LAPMOD	55.3	100.0	100.0	91.2	57.1
FMS CALPUFF	60.3	100.0	100.0	91.7	53.3
NMSE LAPMOD	12.15	0.40	0.60	1.25	0.57
NMSE CALPUFF	4.20	0.32	1.17	3.17	3.52
FOEX LAPMOD	3.3	−23.0	−5.6	8.3	10.0
FOEX CALPUFF	33.3	12.2	47.2	50.0	50.0
Rank LAPMOD	3.1	3.9	4.2	3.8	3.5
Rank CALPUFF	2.6	4.1	2.8	2.1	2.1

. The global analysis with time- and space-paired data (N = 1507) shows that the values of FA2 and FA5 are relatively small for both the two models. The final ranks are 3.1/5 for LAPMOD and 2.6/5 for CALPUFF; therefore, it seems that LAPMOD is more in agreement with the observations with respect to CALPUFF.

The time-averaged concentrations at each sampling location over the arcs are shown in Figure 14. Excluding the closest arc (100 m), CALPUFF generally overestimates the observations. The behavior of the models at the first arc is similar, with a final rank of 3.9/5 for LAPMOD and 4.1/5 for CALPUFF. At the other three arcs, the agreement between LAPMOD and the observations remains good, with ranks ranging from 3.5/5 (800 m arc) to 4.2/5 (200 m arc), while the one between CALPUFF and the observations decreases, with ranks in the range 2.1/5 to 2.8/5.

3.7. PSB2–IOP2

IOP2 of PSB2 was performed on 27 July 2016 from 11:30 to 13:30 MST. According to the technical report describing PSB2 [9], the weather conditions were hot and dry with wind speeds near 3 m/s over most of IOP2. The skies were clear and sunny. Overall conditions were unstable and highly non-stationary in both time and space. Estimates of stability based on traditional Pasquill–Gifford (PG) schemes were mainly class B with some class A and C. The tracer plume was mostly confined to the bag sampling array although wind directions sometimes advected part of the plume away from the sampler array.

The wind rose obtained with CALMET at the release location (Figure 15) shows that the wind blows prevalently from SSW and SW. There are also few winds from ENE that move SF6 outside the sampling locations shown in Figure 1b. The average wind speed determined by CALMET during IOP2 is 3.5 m/s, with a maximum value of 4.8 m/s. The mode of the wind speed distribution is between 3 m/s and 4 m/s, with 58.3% of the events. The PG stability classes calculated by CALMET for PSB2-IOP2 are always very unstable (A or B).

The statistical parameters calculated for PSB2-IOP2 are summarized in

Table 12. Statistical parameters for PSB2-IOP2.

Statistics	Paired	100 m arc	200 m arc	400 m arc	800 m arc
N	1521	35	36	36	16
Max Obs	9333	3850	692	128	20
Max LAPMOD	20,865	6035	791	206	39
Max CALPUFF	5618	3405	1000	269	92
FA2 LAPMOD	30.6	42.9	44.4	27.8	18.8
FA5 LAPMOD	41.6	80.0	72.2	52.8	56.3
FA2 CALPUFF	25.3	82.9	11.1	11.1	18.8
FA5 CALPUFF	43.4	91.4	72.2	36.1	31.3
Corr. LAPMOD	0.46	0.57	0.43	0.21	−0.19
Corr. CALPUFF	0.72	0.87	0.79	0.60	−0.02
FB LAPMOD	0.19	0.00	−0.14	0.22	0.43
FB CALPUFF	0.35	0.15	0.83	1.22	1.47
FMS LAPMOD	50.5	94.3	77.1	79.2	50.0
FMS CALPUFF	67.2	100.0	86.1	61.1	56.3
NMSE LAPMOD	11.33	1.22	1.77	3.84	2.50
NMSE CALPUFF	2.74	0.21	1.52	4.50	7.11
FOEX LAPMOD	−15.9	−21.4	−19.4	5.6	6.3
FOEX CALPUFF	29.6	30.0	50.0	50.0	50.0
Rank LAPMOD	3.0	3.9	3.5	3.3	2.5
Rank CALPUFF	3.1	4.1	3.0	2.0	1.1

. The global analysis with time- and space-paired data (N = 1521) shows that some statistical parameters are better for one model and some others for the other one. For example, the FA2 for LAPMOD exceeds the one for CALPUFF, while the opposite is true for FA5. Again, correlation is better for CALPUFF, while FB is better for LAPMOD. Accordingly, the final ranks are similar, with 3.1/5 for CALPUFF and 3.0/5 for LAPMOD.

The time-averaged concentrations at each sampling location over the arcs are shown in Figure 16. At the closest arc (100 m), CALPUFF performs well, with an FA2 equal to 82.9% and FA5 equal to 91.4%; the correlation is also good, and the FMS is 100%. As a result, the CALPUFF rank at 100 m is 4.1/5, while the LAPMOD rank is 3.9/5. Figure 16 shows that LAPMOD overpredicts the observations at the central angles of the closest arc. At the remaining three arcs, the agreement between LAPMOD and the observations is higher than the agreement between CALPUFF and the observations, with the LAPMOD ranks ranging from 2.5/5 (800 m arc) to 3.5/5 (200 m arc), and the CALPUFF ranks ranging from 1.1/5 (800 m arc) to 3.0/5 (200 m arc). In particular, as shown by Figure 16 and by the FOEX values in Table 12, CALPUFF always overpredicts the observations at the three farthest arcs.

At the 200 m arc, the maximum values predicted by the models are at angles within the range 18–30 degrees, while the maximum observed values are close to 54–60 degrees. A similar shift is also observed at the 400 m arc.

3.8. PSB2–IOP3

IOP3 of PSB2 was performed on 4 August 2016 from 13:00 to 15:00 MST. According to the technical report describing PSB2 [9], the weather conditions were warm and dry, with clear skies and light and very variable winds throughout the IOP. Overall conditions were unstable and highly non-stationary in both time and space. Estimates of stability based on traditional PG schemes were all class A during the first hour and a mix of class A and B in the second hour. There was considerable variation in wind direction, both in time and space.

The wind rose obtained with CALMET at the release location (Figure 17) shows the high variability in wind direction. Some directions do not transport the plume over the sampling locations shown in Figure 1b, for example, those from NE and NNE. Calms (i.e., winds below 0.5 m/s) are predicted for 4.2% of the time. The average wind speed determined by CALMET during IOP3 is 2.1 m/s, with a maximum value of 5.0 m/s. The mode of the wind speed distribution is between 1 m/s and 2 m/s, with 54.2% of the events. The PG stability classes calculated by CALMET for PSB2-IOP3 are always very unstable (A or B, with also C for a short time). More specifically, class A was determined for the first 50 min of the simulation, while the rest of the time was always B, excluding 10 min with C.

The statistical parameters calculated for PSB2-IOP3 are summarized in

Table 13. Statistical parameters for PSB2-IOP3.

Statistics	Paired	100 m arc	200 m arc	400 m arc	800 m arc
N	1538	37	36	36	16
Max Obs	7753	1174	188	25	6
Max LAPMOD	6588	1518	218	31	7
Max CALPUFF	4215	1495	460	127	35
FA2 LAPMOD	14.1	45.9	30.6	33.3	18.8
FA5 LAPMOD	30.1	70.3	75.0	86.1	75.0
FA2 CALPUFF	7.8	32.4	5.6	0.0	0.0
FA5 CALPUFF	18.3	54.1	19.4	0.0	0.0
Corr. LAPMOD	0.36	0.56	0.42	0.73	0.87
Corr. CALPUFF	0.53	0.54	0.41	0.76	0.85
FB LAPMOD	0.47	0.28	0.59	0.74	0.78
FB CALPUFF	1.09	0.88	1.46	1.69	1.72
FMS LAPMOD	35.0	83.8	75.0	86.2	12.5
FMS CALPUFF	35.1	83.8	75.0	69.4	6.3
NMSE LAPMOD	18.00	0.73	1.17	0.98	0.86
NMSE CALPUFF	10.00	1.41	5.54	11.86	12.92
FOEX LAPMOD	23.8	23.0	38.9	38.9	50.0
FOEX CALPUFF	43.4	44.6	50.0	50.0	50.0
Rank LAPMOD	2.3	3.5	2.9	3.3	2.3
Rank CALPUFF	1.6	2.6	1.6	1.6	1.1

. The global analysis with time-paired data (N = 1538) shows that the statistical parameters calculated for LAPMOD—even though not very good—are almost always better than those calculated for CALPUFF. The FOEX index equal to 43.4% implies that CALPUFF overestimates almost always. Indeed, the LAPMOD rank is 2.3/5, while the CALPUFF one is 1.6/5.

The situation improves a bit when the time-averaged concentrations over the arcs are considered (Figure 18). Even the time-averaged CALPUFF predictions always overpredict the time-averaged observations, excluding a couple of points at the first arc (100 m). The FOEX index calculated for CALPUFF is 44.6% at the first arc and 50% at the remaining 3 arcs. Also, for the two farthest arcs (400 m and 800 m), both FA2 and FA5 for CALPUFF are zero. The ranks at the first arc are 3.5 for LAPMOD and 2.6 for CALPUFF. The worst agreement between models and observations is at the farthest arc (800 m) with a rank of 2.3 for LAPMOD and 1.1 for CALPUFF:

3.9. PSB2–IOP4

IOP4 of PSB2 was performed on 5 August 2016 from 12:30 to 14:30 MST. According to the technical report describing PSB2 [9], the weather conditions were warm and dry with mostly sunny skies at the start of the test, with cirrus clouds gradually building over the course of the test period. The winds were light, generally less than 3 m/s, and rather consistently northeasterly. Overall conditions were unstable but were the closest to achieving stationarity of any of the daytime IOPs. Estimates of stability based on traditional PG schemes were mostly class A in the first hour and class C in the second hour. Wind directions were such that the tracer plume missed the bag sampling array in whole or part throughout the measurement period.

The wind rose obtained with CALMET at the release location (Figure 19) shows the winds blow from the whole arc from N to NE, with few exceptions. As observed in the technical report describing the experiment [9], the wind directions were such that the tracer plume missed the sampling locations (Figure 1b) most of the time. The average wind speed determined by CALMET during IOP4 is 2.3 m/s, with a maximum value of 3.1 m/s. The mode of the wind speed distribution is between 2 m/s and 3 m/s, with 70.8% of the events. The PG stability classes calculated by CALMET for PSB2-IOP4 are always very unstable (A or B). More specifically, class A was determined for the first 70 min of the simulation, while the rest of the time it was in class B. Class C mentioned by [9] in the description of IOP4 was not predicted.

The statistical parameters calculated for PSB2-IOP4 are summarized in

Table 14. Statistical parameters for PSB2-IOP4.

Statistics	Paired	100 m arc	200 m arc	400 m arc	800 m arc
N	1533	37	36	35	16
Max Obs	13,167	2254	357	30	2
Max LAPMOD	2891	712	148	28	0.02
Max CALPUFF	5106	2119	606	152	0.24
FA2 LAPMOD	29.8	5.4	11.1	17.1	0.0
FA5 LAPMOD	31.4	27.0	33.3	31.4	0.0
FA2 CALPUFF	19.4	10.8	5.6	0.0	0.0
FA5 CALPUFF	22.7	13.5	13.9	5.7	0.0
Corr. LAPMOD	0.79	0.91	0.84	0.89	0.67
Corr. CALPUFF	0.69	0.92	0.91	0.93	0.63
FB LAPMOD	−1.17	−1.22	−1.06	−0.54	−1.80
FB CALPUFF	0.72	0.56	1.08	1.51	−0.61
FMS LAPMOD	36.7	62.5	87.5	60.0	0.0
FMS CALPUFF	18.0	27.0	22.2	26.3	0.0
NMSE LAPMOD	217.07	21.83	13.74	3.37	301.29
NMSE CALPUFF	21.20	0.99	3.86	18.33	23.89
FOEX LAPMOD	9.9	31.1	19.4	18.6	43.8
FOEX CALPUFF	30.4	44.6	50.0	50.0	43.8
Rank LAPMOD	2.7	2.6	3.1	3.2	0.9
Rank CALPUFF	2.1	2.2	1.7	1.5	1.4

. The global analysis with time-paired data (N = 1533) shows that the statistical parameters are not very good for both models. For example, the FA2 value is about 30% for LAPMOD and about 20% for CALPUFF. The ranks are 2.7/5 and 2.1/5, respectively, for LAPMOD and CALPUFF.

The time-averaged concentrations over the arcs are illustrated in Figure 20. The observations are present only in the leftmost part of each arc, looking at them from the source. The CALPUFF concentrations remain non-null even at sampling locations where the observations and the LAPMOD predictions are null. At the closest arc (100 m) CALPUFF predicts the few initial non-null observations better than LAPMOD, then it overestimates the other observations. The ranks calculated at the first arc are 2.6/5 and 2.2/5, respectively, for LAPMOD and CALPUFF. At the second (200 m) and third (400 m) arcs, CALPUFF always overestimates the observations, as shown by Figure 20 and by the FOEX values equal to their maximum (50%). The ranks calculated at these two arcs are above 3/5 for LAPMOD and below 2/5 for CALPUFF. At the farthest arc (800 m), a non-null concentration is measured only at the sampling point located north of the source (0 degrees). FA2 and FA5 for both models are zero, as well as the FMS. The ranks are 0.9/5 and 1.4/5, respectively, for LAPMOD and CALPUFF.

3.10. PSB2–IOP5

IOP5 of PSB2 was performed on 13 October 2016 from 04:00 to 06:00 MST. As described by the technical report of PSB2 [9], winds were very light, usually less than 1 m/s near the surface. Wind directions were north-northwest within a few meters of the surface but often varied considerably in both time and space. Above the lower few meters, winds were generally north-northeast. A combination of wind direction and plume spread resulted in at least some portion of one or both limbs of the plume being truncated at the edge of the sampler array in all 10 min periods. This was most pronounced during the first hour of the IOP and for the 100 m arc. Estimates of stability based on traditional PG schemes were mainly classes E and F.

The wind rose obtained with CALMET at the release location (Figure 21) shows the winds blow from the whole arc from W to NNE. Some of these directions are not compatible with the transport of SF6 over the sampling locations (Figure 1c). As observed in the technical report [9], winds were very light: calms (i.e., speed less than 0.5 m/s) 8% of the time, the average value is 1.2 m/s and the maximum speed is 2.3 m/s. The mode of the wind speed distribution is between 1 m/s and 2 m/s, with 52.0% of the events. The PG stability calculated by CALMET for PSB2-IOP5 is always class F (very stable).

The statistical parameters calculated for PSB2-IOP5 are summarized in

Table 15. Statistical parameters for PSB2-IOP5.

Statistics	Time Paired	100 m arc	200 m arc	400 m arc
N	1496	39	35	36
Max Obs	117,676	16,855	9876	4387
Max LAPMOD	9634	2553	616	119
Max CALPUFF	4820	2350	861	338
FA2 LAPMOD	28.1	2.6	2.9	22.2
FA5 LAPMOD	32.4	7.7	2.9	22.2
FA2 CALPUFF	36.8	2.6	2.9	25.0
FA5 CALPUFF	48.1	28.2	14.3	44.4
Corr. LAPMOD	0.11	0.11	0.21	0.22
Corr. CALPUFF	0.29	0.37	0.47	0.48
FB LAPMOD	−1.82	−1.80	−1.90	−1.93
FB CALPUFF	−1.58	−1.56	−1.67	−1.64
FMS LAPMOD	38.5	53.8	51.9	68.2
FMS CALPUFF	61.6	100.0	92.6	86.4
NMSE LAPMOD	179.64	21.72	67.93	140.82
NMSE CALPUFF	68.53	8.08	17.54	23.47
FOEX LAPMOD	−28.6	−50.0	−30.0	−38.9
FOEX CALPUFF	−26.7	−50.0	−27.1	−41.7
Rank LAPMOD	1.3	0.8	1.2	1.4
Rank CALPUFF	2.1	1.9	2.2	2.1

. The global analysis with time-paired data (N = 1496) shows that the observations of this IOP are not well reproduced by the models: FA2 and FA5 are small, particularly for LAPMOD, and the FOEX indicates that both models underpredict the measurements. The ranks are 1.3/5 and 2.1/5, respectively, for LAPMOD and CALPUFF.

The time-averaged concentrations over the arcs are illustrated in Figure 22. At the two closest arcs (100 m and 200 m), there are observations well above the background values at the sampling locations at north and NNW of the source; this seems impossible because, for example, there are no wind components from the south in the wind rose (Figure 21). However, the CALMET wind field is not homogeneous; therefore, there could be compatible winds in other points of the domain. Indeed, as reported by [9], the winds varied considerably in both time and space. Both LAPMOD and CALPUFF fail in reproducing the time-averaged concentrations: they do not predict concentrations at all the points where there are significative observations, and where there are predicted values, they are very small. At the first arc (100 m), the LAPMOD rank is as low as 0.8/5, and it does not increase a lot in the other two arcs. The CALPUFF ranks are around 2/5 at all the three arcs.

The bad agreement between observations and models in this IOP may be due to the inability of CALMET to reproduce the observed wind directions along the vertical. Indeed, the first CALMET vertical level is 20 m depth, and in this layer, each horizontal cell has a specific wind direction. The comments about the weather conditions during this IOP [9] mentioned a north-northwest wind direction within a few meters of the surface, and a north-northeast wind direction above the lower few meters. CALMET cannot reproduce this vertical wind direction variation within its first vertical layer. An additional CALMET simulation was performed considering the measurements at 10 m agl for the GRI and COC stations. The results (not shown here) of the two dispersion models fed by the new meteorological dataset do not improve.

3.11. PSB2–IOP6

IOP6 of PSB2 was performed on 20 October 2016 from 04:00 to 06:00 MST. As described by the technical report of PSB2 [9], winds were very light, usually about 1 m/s near the surface. Estimates of stability based on traditional PG schemes were mainly class E during the first hour and class F during the second hour. Wind directions were generally north-northwest within a few meters of the surface. Above the lower few meters, winds were generally north-northeast. Wind directions were similar to those in IOP5 with north-northwest winds in the lower few meters and northeasterly winds above. These directions resulted in the plume largely missing the sampling locations during the first hour and only the northern limb of the plume was sampled during the second hour. For this reason, IOP6 was probably one of the least effective or useful tests during PSB2.

The wind rose obtained with CALMET at the release location (Figure 23) shows the winds blow prevalently from NNE, with some events also from NNW. Remembering the sampling locations shown in Figure 1c, only the few winds blowing from NNW are capable of transporting SF6 over the rightmost receptors (looking from the source). The average wind speed is 1.8 m/s and the maximum speed is 2.7 m/s. The mode of the wind speed distribution is between 1 m/s and 2 m/s, with 62.5% of the events. The PG stability calculated by CALMET for PSB2-IOP6 is always class F (very stable).

The statistical parameters calculated for PSB2-IOP6 are summarized in

Table 16. Statistical parameters for PSB2-IOP6.

Statistics	Time Paired	100 m arc	200 m arc	400 m arc
N	1487	38	37	37
Max Obs	13,524	5178	1421	312
Max LAPMOD	5593	1327	381	79
Max CALPUFF	3672	1751	684	283
FA2 LAPMOD	44.3	5.3	0.0	0.0
FA5 LAPMOD	45.3	10.5	5.4	8.1
FA2 CALPUFF	82.5	47.4	62.2	70.3
FA5 CALPUFF	85.2	60.5	78.4	75.7
Corr. LAPMOD	0.40	0.87	0.79	0.52
Corr. CALPUFF	0.75	0.95	0.95	0.86
FB LAPMOD	−1.60	−1.58	−1.65	−1.80
FB CALPUFF	−0.86	−0.90	−0.81	−0.73
FMS LAPMOD	15.1	100.0	40.0	25.0
FMS CALPUFF	43.5	88.9	90.0	87.5
NMSE LAPMOD	321.73	46.92	46.40	86.78
NMSE CALPUFF	65.04	9.07	4.41	3.42
FOEX LAPMOD	−4.1	26.3	23.0	25.7
FOEX CALPUFF	−40.0	−31.6	−41.9	−41.9
Rank LAPMOD	2.1	2.7	2.0	1.4
Rank CALPUFF	2.8	3.4	3.4	3.3

. The global analysis with time-paired data (N = 1487) shows that the observations of this IOP are not well reproduced by the models, particularly by LAPMOD. The ranks are 2.1/5 and 2.8/5, respectively, for LAPMOD and CALPUFF.

The time-averaged concentrations over the arcs are illustrated in Figure 24. As anticipated, the SF6 cloud was intercepted only by the rightmost sampling sites when viewed from the source. The dispersion models were able to predict the cloud only at these locations, but the concentration values were underestimated, particularly by LAPMOD. The CALPUFF ranks are quite good: they are 3.3/5 and 3.4/5 at the three arcs, while the LAPMOD ranks range from 1.4/5 at 400 m and 2.7/5 at 100 m.

Again, the bad agreement between observations and models in this IOP may be due to the inability of CALMET to reproduce the observed wind direction along the vertical, as commented for the PSB2-IOP5.

3.12. PSB2–IOP7

IOP7 of PSB2 was performed on 21 October 2016 from 04:00 to 06:00 MST. As described by the technical report of PSB2 [9], estimates of stability based on traditional PG schemes were mainly class F. IOP7 had the most stable atmosphere and lightest winds of all of the IOPs. Winds were mostly less than 1 m/s near the surface and highly variable in direction in both space and time, especially during the first hour of the IOP. After 05:00 MST, the near-surface wind directions organized around west-northwest. Wind directions above the surface varied with time and height and ranged from consistently northeast aloft to northeast (early) or northwest (late) at lower levels.

The wind rose obtained with CALMET at the release location (Figure 25) shows the winds blow prevalently from NW, with some events also from N and SW. Remembering the sampling locations shown in Figure 1c, some of the wind directions (N, NE, E) are not capable of transporting SF6 over the receptors. The winds are calm 8.3% of the time; the average wind speed is 1.1 m/s and the maximum speed is 1.9 m/s. The mode of the wind speed distribution is between 1 m/s and 2 m/s, with 62.5% of the events. The PG stability calculated by CALMET for PSB2-IOP7 is always class F (very stable).

The statistical parameters calculated for PSB2-IOP7 are summarized in

Table 17. Statistical parameters for PSB2-IOP7.

Statistics	Time Paired	100 m arc	200 m arc	400 m arc
N	1472	38	34	37
Max Obs	98,977	13,971	6859	2261
Max LAPMOD	8590	2160	403	85
Max CALPUFF	4047	1524	609	217
FA2 LAPMOD	8.5	5.3	0.0	0.0
FA5 LAPMOD	14.3	13.2	11.8	8.1
FA2 CALPUFF	11.8	10.5	14.7	35.1
FA5 CALPUFF	24.9	47.4	58.8	67.6
Corr. LAPMOD	0.18	0.19	0.31	−0.06
Corr. CALPUFF	0.45	0.46	0.64	0.49
FB LAPMOD	−1.74	−1.71	−1.82	−1.89
FB CALPUFF	−1.41	−1.45	−1.46	−1.37
FMS LAPMOD	38.8	78.9	76.5	67.6
FMS CALPUFF	64.4	100.0	100.0	100.0
NMSE LAPMOD	111.68	17.41	37.19	70.11
NMSE CALPUFF	41.56	7.30	9.75	9.09
FOEX LAPMOD	−24.8	−50.0	−50.0	−50.0
FOEX CALPUFF	−5.0	−47.4	−50.0	−17.6
Rank LAPMOD	1.3	1.2	1.3	0.8
Rank CALPUFF	2.5	2.3	2.5	3.1

. The global analysis with time-paired data (N = 1472) shows—as for IOP6—that the observations of this IOP are not well reproduced by the models, particularly by LAPMOD. The ranks are 1.3/5 and 2.5/5, respectively, for LAPMOD and CALPUFF.

The time-averaged concentrations over the arcs are illustrated in Figure 26. The models highly underpredict the observed concentrations, particularly in the central part of the arcs. LAPMOD underpredicts even more than CALPUFF, as shown by the FOEX values. The LAPMOD ranks are very low, particularly for the farthest arc (0.8/5). At the same arc, the CALPUFF rank is 3.1/5.

Again, the bad agreement between observations and models in this IOP may be due to the inability of CALMET to reproduce the observed wind direction along the vertical, as commented for the PSB2-IOP6.

3.13. PSB2–IOP8

IOP8 of PSB2 was conducted on 26 October 2016 from 18:30 to 20:30 MST. According to [9], the earlier nighttime start was an attempt to conduct the tracer release during a more reliably southwest flow. However, during the first hour, wind directions on GRI were out of the southwest, as desired, but then shifted to a more generally northerly direction during the second hour. In contrast, wind directions at COC during the first hour tended to be more easterly during the first hour, shifted to mostly south and southwest near the end of the first hour, then shifted back to northeasterly at the end of the IOP. Wind directions above the surface tended to be much more consistent with near-surface winds than seen for the other nighttime IOPs. Estimates of stability based on traditional PG schemes were mainly class D but included some class E and a few class F. These results suggest that IOP8 had the least stable atmosphere of all the nighttime IOPs. Due to the timing, it is likely that the atmosphere was still in something more like a transition state than the other nighttime IOPs. Wind speeds were mostly 1–2 m/s near the surface. Wind directions exhibited significant variability, both in time and across the study area.

The wind rose obtained with CALMET at the release location (Figure 27) shows the winds blow prevalently from NNW, a direction not compatible with the transport of SF6 at the sampling locations (Figure 1c). Calm winds occur 4.2% of the time; the average wind speed is 1.9 m/s and the maximum speed is 2.9 m/s. The mode of the wind speed distribution is between 2 m/s and 3 m/s, with 50.0% of the events. The PG stability calculated by CALMET for PSB2-IOP8 is always class F (very stable).

The statistical parameters calculated for PSB2-IOP8 are summarized in

Table 18. Statistical parameters for PSB2-IOP8.

Statistics	Time Paired	100 m arc	200 m arc	400 m arc
N	1546	39	36	37
Max Obs	25,433	3807	1135	488
Max LAPMOD	5355	687	183	31
Max CALPUFF	3864	1084	376	119
FA2 LAPMOD	12.3	10.3	2.8	10.8
FA5 LAPMOD	21.7	23.1	13.9	27.0
FA2 CALPUFF	20.8	28.2	50.0	16.2
FA5 CALPUFF	37.6	71.8	83.3	54.1
Corr. LAPMOD	0.22	0.05	−0.27	−0.18
Corr. CALPUFF	0.54	0.66	0.72	0.30
FB LAPMOD	−1.43	−1.43	−1.66	−1.75
FB CALPUFF	−0.86	−0.99	−0.61	−0.73
FMS LAPMOD	28.2	92.3	52.8	29.4
FMS CALPUFF	44.7	92.3	88.9	75.7
NMSE LAPMOD	85.35	10.05	17.49	38.32
NMSE CALPUFF	28.45	3.43	1.28	3.93
FOEX LAPMOD	−10.2	−32.1	−33.3	−23.0
FOEX CALPUFF	3.7	−26.9	−22.2	4.1
Rank LAPMOD	1.8	1.8	0.9	1.1
Rank CALPUFF	2.9	3.3	3.7	3.2

. The global analysis with time-paired data (N = 1546) shows that the observations of this IOP are not well reproduced by the models, particularly by LAPMOD. The ranks are 1.8/5 and 2.9/5, respectively, for LAPMOD and CALPUFF.

The time-averaged concentrations over the arcs are illustrated in Figure 28. The LAPMOD model underpredicts the observed concentrations, while the CALPUFF predictions are closer to the observations, even though the peaks are not correctly predicted. The CALPUFF ranks are over 3/5, while the LAPMOD ranks are smaller than 2/5.

4. Conclusions

In this paper, the Project Sagebrush (PSB) data [8,9] of both phases (PSB1 and PSB2) were used, first to reconstruct the meteorological fields with CALMET, and then to validate the dispersion models CALPUFF and LAPMOD. An intercomparison was performed both by pairing in time and space predictions and observations, both by averaging them at each sampling location over the 2 h length of each IOP.

As shown in the previous paragraphs, sometimes LAPMOD performs better than CALPUFF and sometimes vice versa. For example, in PSB1-IOP3 and in PSB2-IOP2, the performances of the two models are similar. In the other IOPs of PSB1, LAPMOD performs better than CALPUFF, as well as in the IOPs carried out in July and August 2016. CALPUFF often overestimates the observations, as for example, in IOP2, IOP4 and IOP5 of PSB1, or IOP1–3 of PSB2.

The two dispersion models fail in reproducing the observations during the four IOPs of PSB2 conducted during very stable conditions in October 2016 (IOP5–8). This failure may be due to the inability of the CALMET meteorological model to create the observed wind direction variation in the few meters above the ground. Indeed, the first vertical level of CALMET has a height of 20 m, within which the wind direction is constant for each horizontal cell. Of course, other reasons for the failure may be related to wrong or non-appropriate physics within the dispersion models when simulating stable low wind situations, which might have larger horizontal wind variations, as described by [16]. As pointed out by [17] analyzing the PSB2 results, the data indicate that the large increase in concentration variability is linked with the suppression of turbulent mixing, small eddy length scales, and meandering in very stable conditions.

Considering the statistical ranks defined in this paper, CALPUFF performs better than LAPMOD in IOP5-8 of PSB2. However, the charts show that both models greatly underestimate in IOP5 and IOP7. Underestimation is also present in IOP8 of PSB2, but less pronounced with respect to IOP5 and IOP7. Concerning IOP6, according to [9], it was probably one of the least effective or useful tests during PSB2, because the SF6 plume often went far from the sampling locations.

In general, the agreement of the predictions of the two models with the observations is satisfactory, but for the four October 2016 IOPs of PSB2. The term “satisfactory” is used because, for the IOPs in which the wind blows effectively towards the sampling sites and for the time-averaged concentrations, some of the performance evaluation criteria proposed in [31] to define a “good” model are reached. For example, [31] requires that the following three rules be observed:

• the fraction of predictions within a factor of two of observations is about 50% or greater (i.e., FA2 > 50%);
• the mean bias is within ± 30% of the mean (i.e., roughly |FB| < 0.3);
• the random scatter is about a factor of two to three of the mean (i.e., roughly NMSE < 1.5).

The above rules are not firm guidelines, and it is necessary to consider all performance measures when making a decision concerning model validity; therefore, many other statistical parameters were considered in this work.

Additional investigation is needed to understand the models’ behavior during these IOPs and possibly introduce physics and algorithms capable of improving the performances in very stable conditions. At this point, the direction for future investigations is not precisely defined, but four possible activities are sketched below:

• As pointed out above, the bad performances of both models in reproducing the results of the four very stable and low-wind IOPs of PSB2 might be related to the inability of the CALMET meteorological model to create the observed wind direction variation in the few meters above the ground. Unfortunately, if this is the problem, there are not many options to solve it, other than modifying CALMET to allow a better spatial resolution in the vertical direction, that is, to allow the use of a first level with a thickness of less than 20 m.
• Concerning CALPUFF, as shown by the MDISP value in Table 3, it was decided to calculate the dispersion coefficients from internally calculated values of σ_v and σ_w using the micrometeorological variables (friction velocity, convective velocity, Monin–Obukhov length) calculated by CALMET. Since PSB is characterized by the presence of ultrasonic anemometers, a future test would be to use CALPUFF with MDISP = 1, which means using dispersion coefficients computed from measured values of σ_v and σ_w.
• Additionally, only wind measurements from cup anemometers were used to reconstruct the wind fields with CALMET close to the source. These anemometers require more wind energy to start measuring (i.e., moving) than the sonic anemometers which operate without moving parts. Therefore, the minimum wind speed registered by a sonic anemometer is lower than the one measured by a cup anemometer. It is possible that using the wind observations of sonic anemometers as input to CALMET in place of those of cup anemometers, particularly close to the release point, might result in a better description of the wind field. This is important for IOP5-8 of PSB2 that is characterized by low winds.
• Finally, it has been realized that the description of the turbulence within the stable atmospheric boundary layer (SBL) in LAPMOD must be improved. Indeed, in the SBL, the characteristics of the wind speed fluctuations are markedly different from those in neutral or unstable conditions due to the suppressive effects of stratification [44], leading to weaker turbulence overall. Moreover, the turbulence becomes highly anisotropic: the horizontal fluctuations (especially streamwise) dominate, while the vertical fluctuations are strongly suppressed. Turbulence can become intermittent, with bursts of turbulent activity separated by quiescent periods [45]. The turbulence algorithm currently used in LAPMOD describes the wind variances as linear, decreasing with height, capturing the weaker turbulence near the top of the boundary layer, which is expected due to suppression by stratification. However, the anisotropy is not represented because σ_w is assumed to be equal to σ_v, while it should be σ_w << σ_v due to the strong suppression of vertical motions. Thus, the description of turbulence in LAPMOD within the SBL will be thoroughly reviewed and modified.