Evaluating the Impacts of Parameter Uncertainty in a Practical Transportation Demand Model

Abstract

The inherent uncertainty in travel forecasting models—arising from potential and unknown errors in input data, parameter estimation, or model formulation—is receiving increasing attention from both the scholarly and practicing communities. In this research, we investigate the variance in forecasted traffic volumes resulting from varying the mode and destination choice parameters in an advanced trip-based travel demand model. Using Latin hypercube sampling to construct several hundred combinations of parameters across the plausible parameter space, we introduce substantial changes to implied travel impedances and modal utilities, on the order of a 10 percent variation. However, the aggregate effects of these changes on forecasted traffic volumes are small, with a variation of approximately 1 percent on high-volume facilities. It is likely that in this example—and perhaps in others—the network assignment places constraints on the possible volume solutions and limits the practical impacts of parameter uncertainty. Nevertheless, parameter uncertainty may not be the largest contributor to error in practical travel forecasts. Further research should examine the robustness of this finding across other less constrained networks and within activity-based travel model frameworks.

1. Introduction

The inherent accuracy and uncertainty in travel forecasting models are receiving increasing attention from the scholarly and practicing communities. As an example of this attention, the Standing Committee on Transport Forecasting of the Transportation Research Board has made uncertainty one of its primary research agenda issues [1], following a major report from the Federal Highway Administration on the topic [2]. Given that such models are used in the allocation of billions of dollars of infrastructure financing each year, the financial risks for inaccurate or imprecise forecasts are high [3,4]. Systemic under- or over-prediction could lead to substantial over- or under-investment in the highway network [5].

Transportation demand forecasting models, like other mathematical-statistical models, might be abstracted into the following basic form:

$$y=f(X,\beta )$$

where y is the variable being predicted based on input data X, moderated through a specific functional form $f\left(\right)$ and parameters $\beta$. Three general sources of error may cause a forecast value $\widehat{y}$ to differ from the “true” or “actual” value of y [6]:

• The input data X might contain errors due to inaccuracies in the base year or an inaccurate projection of land use, petroleum price, or other key input variables. This was among the primary issues identified by Hoque et al. [5] in a historical analysis of the accuracy of travel forecasts.
• The model form $f\left(\right)$ may be improperly specified. Variables that play a major role in travel behavior may not be included due to a lack of information, or the unobserved error components may have a different correlation than was assumed during model development. A detailed description of specifying mode choice model variables and the nesting of error structures is given by Koppelman and Bhat [7]. This category includes deeper uncertainty issues such as unforeseen shifts in behavior.
• The parameter estimates $\widehat{\beta }$ of the “true” parameters $\beta$ may have incorrect values. This may be because the parameters were estimated on an improperly specified model $f\left(\right)$, or because the estimation dataset was improperly weighted.

Of these potential sources of error, only the third is substantively addressed in classical statistics. The standard errors of the model parameter estimates from a theoretical perspective address the parameter uncertainty question to a great degree. Yet even this source of uncertainty has been largely ignored in transportation forecasts, and model development documentation often elides the variance in these values completely [8]. Zhao and Kockelman [9] examined the effects of this parameter uncertainty in a trip-based model of a contrived 25-zone region, but a systemic analysis of this uncertainty in a practical model is not common.

In this research, we investigate the uncertainty in traffic forecasts resulting from plausible parameter uncertainty in an advanced trip-based transportation demand model. Using a Latin hypercube sampling (LHS) methodology, we simulate one hundred potential parameter sets for a combined mode and destination choice model in Roanoke, VA, USA. We then assign the resulting trip matrices to the highway network for the region and evaluate the PM and daily assigned traffic volumes alongside the variation in implied impedance and accessibility.

This paper proceeds with a description of the model design and simulation sampling methodology in Section 3, followed by a discussion of the variations in mode, destination, and traffic performance measures in Section 4. This paper concludes in Section 5 with a summary of the key findings alongside a presentation of limitations and related indications for future research.

2. Literature Review

Uncertainty has been examined in various ways over the last two decades and is becoming increasingly important for researchers. This review looks at why uncertainty is important to evaluate in transportation demand models and discusses research that has been conducted to evaluate this uncertainty. Rasouli and Timmermans [6] presented an extensive literature review on this topic. An overview of the literature and the sources of uncertainty they evaluated can be found in

Table 1. Studies of forecasting uncertainty.

Reference	Uncertainty Source(s) Evaluated
Rodier & Johnson (2002) [10]	Input Data
Zhao & Kockelman (2002) [9]	Input Data & Parameter Estimates
Clay & Johnson (2005) [11]	Input Data & Parameter Estimates
Flyvbjerg (2005) [3]	Model Form
Armoogum et al. (2009) [12]	Model Form
Duthie et al. (2010) [13]	Input Data & Parameter Estimates
Welde & Odeck (2011) [14]	Model Form
Yang et al. (2013) [15]	Input Data & Parameter Estimates
Manzo et al. (2015) [16]	Input Data & Parameter Estimates
Petrik et al. (2016) [17]	Input Data & Parameter Estimates
Petrik et al. (2020) [18]	Model Form & Parameter Estimates
Hoque et al. (2021) [5]	Input Data

.

Model accuracy is the basis for studying the uncertainty of input data and/or parameter estimates. Travel forecasters have always been cognizant of the uncertainty in their forecasts, especially as project decisions are made using these models, often with high financial impacts.

Flyvbjerg et al. [3] collected data from various forecasting traffic models with an emphasis on rail projects. They used the forecast data for a given year and the actual values that were collected for the same year. Their study found that there is a statistical significance in the difference between the estimated and actual values. Rail projects are generally overestimating passenger forecasts by 106%, and half of road projects have a traffic forecast difference of plus or minus 20%. They did not identify where this inaccuracy came from, but they identified that it was important for future research.

Armoogum et al. [12] looked at uncertainty within a forecasting model for the Paris and Montreal metropolitan regions. The sources of uncertainty analyzed were the calibration of the model, the behavior of future generations, and demographic projections. A jackknife technique, rather than sampling methods, was used to estimate confidence intervals for each source of error using multiple years of analysis. This technique is a way to reduce the bias of an estimator and permits the estimation of confidence intervals to produce variance estimates. They found that the longer the forecasting period was, the larger the uncertainty. Generally, the model forecast is within 10–15%, reaching higher percentage ranges for variables with small values or small sample sizes.

Welde and Odeck [14] compared actual and forecast traffic values for 25 toll and 25 toll-free roads in Norway. They evaluated the accuracy of Norwegian transportation planning models over the years. Generally, traffic models overestimate traffic. This study found that toll projects, on average, overestimated traffic, but only by an average of 2.5%. Toll-free projects, however, underestimated traffic by an average of 19%. They concluded that Norwegian toll projects have been fairly accurate, with probable cause coming from the scrutiny that planners face when developing a toll project. A similar scrutiny should then also be placed on toll-free projects as they are significantly less accurate.

These articles show that models have errors that affect traffic projections by a significant amount. These articles identified that the error existed but did not quantitatively identify the source of the error. The most researched source of error has been model form, but that research has mostly been excluded from this review as it is not the main focus of this research. The second most researched form has been input data. Chronologically, Rodier and Johnston [10], Zhao and Kockelman [9], Clay and Johnston [11], Duthie et al. [13], Yang et al. [15], Manzo et al. [16], and Petrik et al. [17] have all researched input errors, with all but the first authors also looking at parameter estimate errors. Parameter estimation error has been the least researched source of uncertainty, with no studies focusing only on this source of error. Petrik et al. [18] looked at parameter estimates but with a focus on model form error. The details of each study are described below in chronological order.

Rodier and Johnston [10] looked at uncertainty in socioeconomic projections (population and employment, household income, and petroleum prices) at the county level for the Sacramento, California region. They wanted to know if the uncertainty in the range of plausible socioeconomic values was a significant source of error in the projection of future travel patterns and vehicle emissions. They identified ranges for population and employment, household income, and petroleum prices for two scenario years (2005 and 2015). The ranges varied based on the scenario year and the socioeconomic variable. They changed one variable at a time for a total of 19 iterations of the model run for 2005 and 21 iterations for 2015. Their results indicated that errors in projections for household income and petroleum prices are not significant sources of uncertainty, but error ranges for population and employment projections are significant sources of change in travel and emissions. The input data of population and employment were significant to model result uncertainty.

Zhao and Kockelman [9] looked at the propagation of uncertainty through each step of a trip-based travel model from variations among inputs and parameters. This analysis used a traditional four-step urban transportation planning process (trip generation, trip attraction, mode split, and trip assignment) on a 25-zone sub-model of the Dallas-Fort Worth metropolitan region. The Monte Carlo simulation was used to vary the input and parameter values. These values were all assigned using a coefficient of variation ($c_v$) of 0.30. The four-step model was run 100 times with 100 different sets of input and parameter values. The results of these runs showed that uncertainty increased in the first three steps of the model and the final assignment step reduced the compounded uncertainty, although not below the levels of input uncertainty. The authors determined that uncertainty propagation was significant from changes in inputs and parameters, but the final step nearly stabilized the uncertainty to the same amount as assumed (0.30 $c_v$ assumption with a 0.31 $c_v$ in the trip assignment results).

Another study that looked at input data uncertainty was Clay and Johnston [11]. These researchers varied three inputs and one parameter to analyze the uncertainty of outputs on a fully integrated land use and travel demand model of six counties in the Sacramento, California region. The variables used for analysis were productions, commercial trip generation rates, perceived out-of-pocket costs of travel for single-occupant vehicles, and concentration parameters. Exogenous production, commercial trip generation rates, and the concentration parameter varied by plus or minus 10, 25, and 50%, while the cash cost of driving varied by plus or minus 50 and 100%. This resulted in 23 model runs—one for each changed variable and one for the base scenario. Their research showed that any uncertainty in the inputs resulted in a large difference in the vehicle miles traveled output, although this difference was a lower percentage than the uncertainty in the input.

Duthie et al. [13] evaluated uncertainty at a different level. They used a small generic gravity-based land use model with the traditional four-step approach, using a coefficient of variation of 0.3, as suggested by Zhao and Kockelman [9], for both inputs and parameters, albeit using antithetic sampling. In this sampling method, pairs of negatively correlated realizations of the uncertain parameters are used to obtain an estimate of the expected value of the function. The uncertainty was evaluated on the rankings of various transportation improvement projects. They found that there are a few significant differences that arise when changing the input and parameter values that result in different project rankings, and thus, neglecting uncertainty can lead to suboptimal network improvement decisions.

Yang et al. [15] evaluated a quantitative uncertainty analysis of a combined travel demand model. They looked at input and parameter uncertainty using a coefficient of variation of 0.30. Rather than using a random sampling method for choices, they used a systematic framework with a variance–covariance matrix. Their research found that the coefficient of variation of the outputs is similar to the coefficient of variation of the inputs and that the effect of parameter uncertainty on output uncertainty is generally higher than that of input uncertainty. This finding contradicts the finding by Zhao and Kockelman [9]. The authors concluded that improving the accuracy of parameter estimation is more effective than enhancing input estimation, as they found that in most steps of the model, the impact of parameter uncertainty was more significant than that of input uncertainty.

Manzo et al. [16] looked at uncertainty in model input and parameters for a trip-based transportation demand model in a small Danish town. They used a triangular distribution with LHS to create the range in parameters; using the information from Zhao and Kockelman (2002) [9], they also used a coefficient of variation of 0.30 and 100 draws, choosing these values as they had been previously used. Their addition to the research on uncertainty involved examining uncertainty under different levels of congestion. Their research found that there is an impact on the model output from the change in input and parameter uncertainty and requires attention when planning. Also, model output uncertainty was not sensitive to the level of congestion.

Petrik et al. [17] evaluated uncertainty in mode shift predictions due to uncertainty from input parameters, socioeconomic data, and alternative specific constants. This study was based on a high-speed rail project in Portugal as a component of the Trans-European Transport Network. They collected survey data and developed discrete choice models. The authors created their own parameter values from the collected data, obtaining the mean or “best” value from the surveys and the corresponding t-statistic. With these, they generated 10,000 samples each of parameter values, socioeconomic inputs, and mode-specific constants, using bootstrap resampling, Monte Carlo sampling, and triangular distribution methods, respectively. The authors found that variance in alternative specific attributes is the major contributor to output uncertainty, in comparison to variance in parameters or socioeconomic factors. Socioeconomic data had the least contribution to overall output variance, and there was a relatively insignificant mode shift due to variability in parameters.

Petrik et al. [18] used an activity-based microsimulation travel demand model for Singapore to evaluate model form and parameter uncertainty. This model has 22 sub-models and 817 parameters. The authors determined which of the 817 parameters the sub-models were most sensitive to and applied a full sensitivity analysis of the top 100 of the parameters, preserving correlations. Using the mean parameter values and the standard deviations available for each, the authors used Latin hypercube sampling with 100 draws to look at the outcomes of the change in each parameter value. Different-sized samples of the model population were also considered in their research. They found that of the 100 most sensitive parameter values, the outcome coefficient of variation varied from 3% to 49%. The variance of the parameter variables did not exceed 19% and, thus, the results from the parameter uncertainty were higher than the variance in the parameters. They also found that the results of the parameter uncertainty were higher than simulation uncertainty.

In transportation demand models, when uncertainty is analyzed, most research to this point has focused on input uncertainty or model forms, rather than parameter estimate uncertainty [6]. Of the 12 articles in this review, two focus only on input data as the subject of their uncertainty research, three focus on model form uncertainty, one looks at both model form and parameter estimate uncertainty, and six focus on both input data and parameter estimate uncertainty. No researchers have looked at parameter estimate uncertainty as the only source of error in their models. When parameter uncertainty has been examined in existing literature, it is often in conjunction with input errors, or on small and non-practicing models. No studies that we could identify have used real models for their analyses. Uncertainty research is needed as transportation demand models provide estimates and forecasts for decision-makers and policymakers. An inaccurate model or large output variance could change what decisions are made and when [1]. Thus, there is a critical research need for a detailed exploration of parameter estimation uncertainty in a practical travel model.

3. Model Design and Methodology

3.1. Model Design

To examine the effects of parameter input sensitivity, we adapted a trip-based travel demand model from the Roanoke Valley Transportation Planning Organization (RVTPO, https://github.com/xinwangvdot/rvtpo, accessed on 1 July 2024). The RVTPO model provides an ideal testing environment for this research because it uses an integrated mode and destination choice framework common in more advanced trip-based models. At the same time, its small size (approximately 215 zones) means the entire model runs in a few minutes and, thus, allows for efficient testing of multiple model runs.

The total number of passenger trips T traveling from zone i to zone j on the highway in a period t is as follows:

$$T_{ijt}=P_i\ast {\mathcal{P}}_{\mathrm{auto}}(\beta ,C_{ijt})\ast {\mathcal{P}}_j(\gamma ,A_j,MCL{S}_{ijt})\ast {\Delta }_t$$

(1)

where P is the production at zone i; ${\mathcal{P}}_{\mathrm{car}}$ is the car mode choice probability determined by utility parameters $\beta$ and the travel costs C between i and j at time period t; ${\mathcal{P}}_j$ is the destination choice probability of choosing destination j given the utility parameters $\gamma$, attractions A, and the impedance as the mode choice model logsum $MCL{S}_{ijt}$. The time of day and direction factor $\Delta$ finalize the total assigned trips.

The trip production $P_i$ and attraction $A_j$ numbers were extracted from the RVTPO model and held constant. These values were determined from the socioeconomic (SE) data at a zonal level; the SE data include information by zone for the total population, number of households, total workers, and workers by employment type. The trip productions are organized by zone and trip purpose. The trip purposes used in this model are home-based work (HBW), home-based other (HBO), non-home-based (NHB), commercial vehicles (CVs), internal–external (IXXI), and external–external (XX). Only the first three are analyzed, but all of the purposes are assigned to the network. CV, IXXI, and XX trips were kept fixed for this analysis.

The two-parameter vectors, $\beta$ and $\gamma$, describe the mode choice model and destination choice model coefficients, respectively. The mode choice model estimates how many trips from i to j will happen on each available mode k. This model analyzes three modes of transportation: auto, non-motorized, and transit. The mode by which a trip is made is determined by the calculated utilities for the three modes. These utilities use parameter values as well as time and distance skims X for each mode. Skims are either the time or distance to travel between zone pairs in a period. Travel time for the auto used the single occupancy vehicle peak period time, non-motorized travel time used the distance skim multiplied by a factor of average walking speed (3 mph), and transit time used the walk to bus peak time. The mode choice parameters (constants and coefficients) were also obtained from the RVTPO model. These values are shown in

Table 2. Choice model parameters.

	Variable	HBW	HBO	NHB
Mode Choice Coefficients
In−vehicle travel time	${\beta }_{ivtt}$	−0.0250	−0.0150	−0.0200
Travel cost	${\beta }_{tc}$	−0.0016	−0.0024	−0.0025
Walk distance	${\beta }_{wd}$	−0.0625	−0.0375	−0.0500
Auto operating cost (cents/mile)	${\beta }_{ac}$	13.6000	13.6000	13.6000
Mode Choice Constants
Transit constant	$k_{trn}$	−0.3903	−1.9811	−2.2714
Non-motorized constant	$k_{nmot}$	−1.2258	−0.3834	−0.8655
Destination Choice Parameters
Households	${\gamma }_{hh}$	0.0000	1.1657	0.5664
Other + Office	${\gamma }_{oth+off}$	0.0000	0.8064	0.5626
Office	${\gamma }_{off}$	0.4586	0.0000	0.0000
Other	${\gamma }_{oth}$	1.6827	0.0000	0.0000
Retail	${\gamma }_{ret}$	0.6087	2.2551	5.1190

.

The utility equations for the mode choice model are as follows:

$$\begin{array}{cc}\hfill U_{auto}& ={\beta }_{ivtt}\ast X_{auto}+{\beta }_{tc}\ast {\beta }_{ac}\ast X_{dist}\hfill \\ \hfill U_{nmot}& =k_{nmot}+20\ast {\beta }_{wd}\ast X_{nmot}\hfill \\ \hfill U_{trn}& =k_{trn}+{\beta }_{ivtt}\ast X_{trn}\hfill \end{array}$$

If the distance was greater than 2 miles, non-motorized travel was excluded as an option. In general, modes with longer times received lower probabilities and, therefore, lower proportions of trips. These utilities were used to calculate the MCLS by the following:

$$MCL{S}_{ij}=ln\left(\sum _{k\in K}{e}^{U_{ijk}}\right).$$

(2)

This logsum value was then used as the primary impedance for a destination choice model [19].

The destination choice model estimates the number of trips between origin and destination pairs using the size of the destination (number of attractions), accessibility by multiple modes (in the MCLS), and destination choice parameters. These parameter values are also shown in Table 2. The destination choice utility is the primary impedance (mode choice logsum value) plus the natural log of the size term, where the size term is calculated as follows:

$$A_j={\gamma }_{hh}\ast \mathrm{HH}+{\gamma }_{off}\ast \mathrm{OFF}+{\gamma }_{ret}\ast \mathrm{RET}+{\gamma }_{oth}\ast \mathrm{OTH}+{\gamma }_{oth+off}\ast \mathrm{OFFOTH}$$

(3)

HH is the total number of households in zone j. OFF, RET, and OTH are the jobs in zone j by employment type, e.g., office, retail, and others, respectively. The destination choice utility is then transformed into a destination choice logsum value with the following:

$$DCLS=ln\left(\sum _{j\in J}{e}^{ln\left(A_j\right)+1\ast MCL{S}_{ij}}\right)$$

(4)

The probability of both mode choice and destination choice is calculated by dividing the exponentiated utility by the corresponding logsum. These probabilities, in conjunction with the trip productions, can calculate the number of production-attraction (PA) trips between each zone for each mode and purpose. The auto trips are calculated by multiplying the probability of the destination by the PA pair, the production for each origin, and the probability of an auto mode choice by the PA pair. This results in PA auto trips. The same process is followed for the other two modes. These PA trips are converted into origin–destination (OD) trips by multiplying the trips by the corresponding time of day factors (see Equation (1)). These trips are calculated using Bentley’s CUBE and the RVTPO model. The trips, by time period, are assigned to the highway network by the shortest path by time using free flow speed and with link capacity as a restriction.

3.2. Uncertainty Design

Within the mode and destination models, there exists uncertainty within the parameters in Table 2. Sampling methods can take the defined uncertainty and choose potential parameter values within the possible range. Two common methods for parameter sampling include the Monte Carlo (MC) simulation and Latin hypercube sampling (LHS). In general, MC simulation draws independently from multiple distributions, while LHS makes draws that cover the parameter space more efficiently and can capture the joint distribution between two or more parameter values [20]. As a result, LHS can reduce the number of draws needed to fully recreate the statistical variance in a model, but the reduction amount is unknown and may not be universal to all problems [15]. And though more potential methods are being developed and employed in related research [21], this research only considers these two.

With the trip-based model described above, MC and LHS methods were used to develop alternative parameter sets to evaluate uncertainty. To identify a standard deviation for each parameter, we asserted a coefficient of variation $c_v=0.10$ for the four mode choice coefficients and the destination choice parameters; the mode choice constants remained fixed across all iterations. The literature had identified a coefficient of variation of 0.30 [9], but this analysis caused an unrealistic value of time and, thus, it was changed to 0.10. The value of time is a ratio in units of money per time that should be compared to the regional wage rate and was generally on the order of USD 10 per hour in the early 2010s [22] when the RVTPO choice coefficients were developed. A $c_v$ of 0.30 implied a value of time extending from USD 2 to USD 32 per hour, whereas a $c_v$ of 0.10 implied values between USD 6 to USD 14 per hour, which we assess as more reasonable for this context. The standard deviation for sampling the parameters was, therefore, equal to 0.10 multiplied by the mean, where the mean values in this situation are the base scenario parameters (as identified in Table 2).

The MC random sampling uses the R function of rnorm. LHS uses the lhs package in R. The full code for both methods can be found in a public GitHub repository (https://github.com/natmaegray/sensitivity_thesis, accessed on 1 July 2024). In our simulations, we first aim to ensure that we explore the full parameter uncertainty space of the model. Second, we aim to ensure that we run a sufficient number of simulations so that outlying and extreme draws do not overly influence our analysis. Therefore, we designed and presented a short experiment to evaluate the average mode choice logsum in the model, determined by 100 and 600 draws of parameters via both MC and LHS.

Figure 1 shows the distributions of the HBW parameters when using 100 and 600 draws, including the distribution of the implied value of time, which is an indirect number based on two separate draws. In general, these distributions show that LHS gives normally distributed parameters with fewer draws than MC sampling, as expected by theory (Helton and Davis, 2003 [20]).

To determine if LHS is effective with a reasonable number of iterations, the cumulative mean and the cumulative standard deviation of the mean MCLS value for all zones (see Equation (2)) were calculated for each additional draw for both sampling methods. MCLS is an impedance term, which is an important value for destination choice and region routing. The mean MCLS, x, was used as a measure of outcome possibilities to simplify a complex term as a single value to compare across all iterations. The cumulative mean is calculated as follows:

$${\mu }_i={\displaystyle \frac{x_1+\dots +x_i}{n}}$$

(5)

and the cumulative standard deviation is calculated as follows:

$$S{D}_i=\sqrt{{\displaystyle \frac{\sum {(x_i-{\mu }_i)}^2}{n-1}}}.$$

(6)

Figure 2 illustrates how the average MCLS stabilizes as the number of draws increases, and the cumulative standard deviation is used to show the 95% confidence interval of that mean.

For these two trip purposes (as well as the third purpose, which we do not show), both sampling methods had a stabilized mean by 100 draws. The LHS method’s standard deviation ribbon was generally thinner than that for the MC method. Given the narrower cumulative standard deviation, and the fact that the parameter values are better normally distributed when using LHS, we decided to use 100 sets of parameters drawn via LHS to evaluate the effect of parameter uncertainty in our analysis.

4. Sensitivity Analysis Results

Each parameter draw was applied to the RVTPO model, generating mode choice utilities, destination choice utilities, and trip matrices for each draw. The resulting uncertainty can then be quantified using the outputs from the trip-based model. This section will first look at the uncertainty of trips by mode, and how the mode split changes when the parameters vary. Then uncertainty will be quantified using the highway-assigned trips, and how to link volume changes across each draw. The results will then be summarized.

4.1. Mode Choice Trips

Uncertainty can be evaluated by looking at how mode choices change. The total number of trips by purpose is fixed, but the number of trips by each mode changes as a result of mode choice, combined with the availability of modes in the travel time skims.

Table 3. Coefficient of variation of trips by mode.

	Base	Mean	SD	${\mathit{c}}_{\mathit{v}}$
HBW
Auto	103,320	103,298	537.07	0.0052
Non-Motorized	1103	1105	50.38	0.0456
Transit	13,254	13,274	566.01	0.0426
HBO
Auto	250,489	250,475	453.11	0.0018
Non-Motorized	4310	4316	235.24	0.0545
Transit	9276	9283	363.09	0.0391
NHB
Auto	60,212	60,209	78.28	0.0013
Non-Motorized	736	737	35.77	0.0485
Transit	1576	1579	74.89	0.0474

lists the base trip amount by mode and purpose. It also lists the average number of trips across all 100 iterations, with the corresponding standard deviation and coefficient of variation. For HBW trips, there are 103,320 auto trips. Across all 100 iterations, there is a mean value of 103,298 trips with a standard deviation of 527.07. This results in a coefficient of variation of 0.0052 or 0.52% in the number of auto trips. The other modes of transportation are included and similar patterns can be seen in HBO and NHB. The results listed in the table show that the variations of the output trips—by mode and purpose—are less than the input variations (as all $c_v$s are smaller than 0.10). This confirms previous research that shows that the outcome variance is less than or near the parameter variance (Clay and Johnston, 2005 [11]); Zhao and Kockelman, 2002 [9]). In all three purposes that were evaluated, the coefficient of variation in auto trips is lower than transit or non-motorized trips, meaning that there is greater confidence in the model’s accuracy in generating auto trips. The input parameter variability has a smaller effect on auto trips than on trips on the other modes.

The variation among mode choices can be visualized graphically using a density of a scaled change in trips by mode. Figure 3 shows density plots for HBW trips by mode for 12 zones. The zones are divided into three volume categories, as follows: low is less than 200 trips per zone, mid is 200 to 700 trips per zone, and top is greater than 700 trips per zone; four zones are randomly selected from each volume category. Zones that do not have any transit accessibility were excluded. These zones show a very high density of auto trips, as the ability to choose transit was removed, the choice to choose auto was more certain. The zones included in Figure 3 all have greater certainty in auto trips, as the change in trips across all 100 iterations is relatively small. This reinforces the previous claim that the model has more confidence in auto trips than the other modes. It is also important to note that the modes are correlated to each other. In zones with greater confidence in one mode, the other modes are more confident as well. Since the number of trips by origin zone is held constant, when there is an increase in trips on one mode, there must be a decrease in trips on one or both of the other modes. Also, the distribution of non-motorized trips is similar for every zone, suggesting that non-motorized trips are generally the most variable mode, which one can see in the spread of the graphic. This is also verified using Table 3, where $c_v$ is the largest for the non-motorized mode across all three purposes.

4.2. Link Volume

Highway volumes are the most commonly used outputs of a travel model. Uncertainty can additionally be evaluated by looking at how the assigned link volume varies across iterations. Figure 4 displays the variation in forecasted link volume spatially. This shows that the links with the highest standard deviation in forecast volume are high-volume roads, including freeways and principal arterials, where the majority of traffic is internal to the study region. Although these links have the largest standard deviation, when compared to the total volume of the road, the variation is, in reality, very small. A standard deviation of 400 vehicles on a road with 40,000 total vehicles corresponds to a small variation (1%).

The highway assignment results can be grouped by facility type to show how the coefficient of variation compares to link volume. Figure 5 shows the coefficient of variation for the daily volume assigned to each network link, across the 100 draws, plotted against the mean forecast link volume for each link. The values represent the volume for 100 randomly sampled links for each facility type. The plots show that for high-volume roads such as major arterials and freeways, the coefficient of variation converges to approximately 0.01, or about 1% of the road’s total forecast volume. For lower-volume links, the coefficient of variation is more widely distributed, with some local roads and small collectors having considerably higher values. Some links in the model show no variation at all; these are presumably links near the edges of the model region where the only traffic is to and from external zones (trips that were held constant in this framework).

Variation among links can also be visualized with a density plot of the total volume across all iterations, as shown in Figure 6. In this plot, the densities of the forecast volumes in three randomly selected links in each of the freeways, collectors, and arterial functional types are plotted alongside the baseline forecast and the Average Annual Weekday Daily Traffic (AAWDT) measured by the Virginia Department of Transportation, to which the model estimates were calibrated. In all cases, the error or uncertainty in the forecast is considerably narrower than the error inherent in the model construction, as evidenced by the fact that the AAWDT target value is well outside the bell curve created by the statistically varied simulation forecasts.

As expected, by using the base parameter values as the mean of the LHS parameter sampling, the base results are at or near the median of the statistical density for each link’s volume. But, notably, the estimated volumes are not perfectly, normally distributed as might be naively expected. In this case, the combined effects of the mode and destination choice parameter sampling appear to be constrained by the geographic specificity of the RVTPO model network. Even as demand for trips varies between zone pairs, the realities of the highway capacity, volume delay, and static user equilibrium procedures may limit the possibilities for forecast highway volumes.

5. Conclusions

The results of this research show that despite large variations in mode and destination choice parameters—and consequently, in accessibility—the impact of this variation on assigned highway volumes is limited. To our knowledge, this is the first systematic evaluation of parameter uncertainty in a practical travel model in the literature, with prior research being limited to toy networks (e.g., [9]). The resulting uncertainty in the output forecasts was generally smaller than the input parameter variance, confirming the results of Petrik et al. [18] in a different context. In this application, at least, the variation in mode and destination choice probabilities appears to be constrained by the capacities and procedures of the highway network assignment.

Several limitations must be mentioned in this research. First, we did not attempt to address the statistical uncertainties in trip production estimates; these may play a substantially larger role than the destination and mode choice parameters, given that lower trip rates may lead to lower traffic volumes globally, which could not be “corrected” by the traffic assignment. Second, a different methodology of sampling might have produced a different result at the extremes than the results of LHS. Additionally, the relatively sparse network of the RVTPO model region—lacking parallel high-capacity highway facilities—may have meant that the network assignment process would converge to a similar solution point regardless of modest changes to the trip matrix. If the number of paths between nodes is limited and constrained by highway capacity, there are only so many solutions to any highway assignment process. It may be that in a larger network with more path redundancies or more alternative transit services, the assignment may not have been as helpful in constraining the forecast volumes. In general, these findings are based on a specific trip-based travel demand model, which may not be applicable to all contexts or regions. The results might vary significantly in different geographic areas or under different modeling frameworks, such as activity-based models. Attempting this research again with a variety of models and geographic regions would be a valuable research priority.

In this research, we only had the estimates of the statistical coefficients and, therefore, had to assume a coefficient of variation to derive variation in the sampling procedure. It would be better if model user guides and development documentation more regularly provided estimates of the standard errors of model parameters. The ideal would be variance–covariance matrices for the estimated models, enabling researchers to ensure that covariance relationships between sampled parameters are maintained. Future research might reconsider the present experiment but allow for a correlation between parameter values.

Notwithstanding these limitations, statistical parameter variance does not appear to be the largest source of uncertainty in travel forecasting. There are likely more important factors at play that planners and government agencies should address. Research on all sources of uncertainty is somewhat limited, but in many ways has been hampered by the burdensome computational requirements of many modern travel models [4]. This research methodology benefited from a lightweight travel model that could be repeatedly re-run with dozens of sampled choice parameters. One strategy for applying this methodology to larger models may involve using the relatively recent TMIP-EMAT exploratory modeling toolkit [23]. But a better understanding of the other sources of uncertainty—model specification and input accuracy—might also benefit from lightweight models constructed for transparency and flexibility rather than heavily constrained models emphasizing precise spatial details and strict behavioral constraints. This might allow forecasts to be made with an ensemble approach [24], identifying preferred policies as the consensus of multiple plausible model specifications.