Effectiveness of Variable Message Signs on Utah Roadways

Abstract

This paper presents the findings of a dual-approach research study conducted for the Utah Department of Transportation to evaluate the effectiveness of variable message sign (VMS) treatments on Utah roadways. The study aimed to determine the impact of VMS treatments on mobility and safety for Utah’s unique road conditions and configurations. Two primary analyses were performed: a diversion rate analysis to assess the effectiveness of VMS messages on Utah freeways during incidents and a weather analysis to evaluate the effectiveness of VMS messages on driver speeds in Utah canyons during winter weather. The findings of the diversion rate analysis indicate that the activation of VMS messages during crash incidents increased diversion rates by 18 percent. The weather analysis showed that the activation of VMS messages warning drivers to slow down due to inclement weather conditions increased driver speeds by 0.23 mph (0.37 kph) during weather conditions, which was statistically significant, but holds little practical significance. The methodologies and findings of this study will assist departments of transportation and other interested parties in developing their VMS policies to more effectively influence driver behaviors.

1. Introduction

Messages posted on variable message signs (VMS) are generally accepted as an effective strategy to improve mobility and safety; however, specific research to show these benefits are inconsistent. Departments of transportation (DOTs) use VMS to convey information to the traveling public that may not be readily apparent. These messages provide critical information with the intent of influencing driver behavior, including informing drivers’ route choices and speeds. While VMS communicate valuable information, they constitute a large investment for DOTs, including upfront capital and ongoing operations and maintenance costs. A lack of understanding of effective VMS uses specific to the driving environment can result in suboptimal VMS placement and underutilization of funds.

This research evaluated the effectiveness of VMS treatments on improving mobility and safety in two specific use cases. First, the effectiveness of VMS on Utah freeways to divert drivers from the freeway during incidents was measured. Second, the effectiveness of VMS in Utah canyons to encourage reduced speeds during winter weather was evaluated. Both evaluations utilized mixed-effects models, which were not observed in the literature. Speed, volume, and weather data from Utah Department of Transportation (UDOT) sensors were collected and evaluated to measure the level of effectiveness of VMS for each use case. The diversion rate analysis utilized the ratio of crash day speeds to control day speeds as its dependent variable, and the weather analysis assigned one of multiple possible weather observations to a single speed observation using a minimization-maximization equation. The findings of this research identify effective uses for VMS to assist in guiding VMS deployment strategies and provide reliable methods for state- or region-specific studies.

The paper proceeds with a literature review of prior studies analyzing the effectiveness of VMS with road data. The methodology for the diversion rate analysis and the weather analysis are then outlined and the results and discussion for each analysis are presented. Finally, the conclusions of the study are summarized.

2. Literature Review

VMS are used to inform the traveling public of information that may not be readily apparent. Roadway-specific messages generally include warnings about dangerous weather conditions, snowfall or rockfall on the roadway ahead, work sites, and/or crash-related congestion. If communicated effectively, drivers are enabled to make better choices, which improves safety and mobility on roadways. Many drivers report seeing VMS messages [1] but determining their effect on driver behavior is much more complex.

The literature on VMS is very broad and contains studies on driver compliance psychology and how the structure of VMS messages, trust in VMS, and perceived urgency can influence driver compliance. These studies will not be discussed here, but Wu et al. [2] have provided a systematic review of VMS literature and the factors that influence driver compliance.

Of the existing research on VMS evaluations, one of the primary methods utilized has been driver surveys [3,4,5,6,7,8,9,10,11]. Stated preference surveys are often used because they are simple and can provide generalized findings. Many practical findings have been identified using stated preference surveys including that driver compliance is correlated with driver demographics [3,7,9], that trust in VMS may be dependent on how frequently a driver uses the roadway [10], and drivers may rely more on VMS for incident information than their smartphones [11]. While stated preference surveys may be simple to create, they can be quite expensive to distribute and may not fully reflect a driver’s true behavior when viewing a VMS compared to their stated response [4].

A less common, but potentially more accurate, method to evaluate the effect of VMS messaging on driver behavior is empirically based road data studies. This method is used to gather and analyze real-world data on how drivers react or behave historically on roadways, providing insight into the true behavior of drivers instead of individual statements of behavior. There are a limited number of empirically based studies, which provide valuable insight into the varying methodological approaches and findings on the effectiveness of VMS.

Table 1. Road Data Evaluation Studies.

Author(s)	Site Location	Type of Study	Effect of VMS Messaging
Lam and Chan; 2001 [12]	Hong Kong	With-and-Without	Reduction in travel time delay during crash or work zone congestion
Levinson and Huo; 2006 [13]	MN, USA	With-and-Without	No clear reduction in travel times, but VMS and ramp meters reduced total delay up to 136 vehicle-hours per peak period incident
Erke et al.; 2007 [14]	Oslo, Norway	With-and-Without	Reported 20 percent of vehicles diverted based on the provided route recommendation
Ghosh et al.; 2018 [15]	Singapore	Before-and-After	Increased diversions by 14 percent
Romero et al.; 2020 [16]	Spain	With-and-Without	Increased diversions onto tolled highway
Basso et al.; 2021 [17]	Chile	Before-and-After	No significant influence on driver behavior (speed, lane change, traffic volume)
Xuan and Kanafani; 2014 [18]	CA, USA	Before-and-After	No significant effect on diversions
Glendon and Lewis; 2022 [19]	Australia	Before-and-After	Reduction in speeds; effect varied by time of day and day of week
Rämä and Kulmala; 2000 [20]	Finland	Before-and-After	Reduction in speeds by 0.75 mph (1.2 kph)
Keshari et al.; 2025 [21]	MI, USA	Before-and-After	“Slippery Road Conditions/Reduce Speeds” message reduced speeds by 0.6–0.7 mph
Sui and Young; 2014 [22]	WY, USA	With-and-Without	VMS speed reductions range from 5 mph to 20 mph above reductions caused by weather

shows eight studies that evaluated the effect of VMS using empirical road data.

The first two studies developed computer prediction models to aid in measuring the effectiveness of VMS. Lam and Chan [12] utilized a time-dependent traffic assignment model to evaluate the effect of providing travel times on VMS. The researchers defined a ratio between the total network time with and without VMS at different locations in the roadway network and determined that VMS had more of an effect in reducing travel time during non-recurrent congestion than in recurring congestion. While Lam and Chan [12] had a statistically strong model, they used statistical distributions for their input data rather than empirical data from roadway conditions, meaning results may differ from real-world conditions.

Conversely, Levinson and Huo [13] used flow rate data collected from loop detectors to train their discrete diversion choice model. A t-test performed using their training data found that VMS can significantly influence driver diversion rates within 10 min of activation. Statistical prediction models are useful tools for simulating real-world outcomes and can yield consistent results, especially when utilizing empirical road data.

Other researchers have directly analyzed empirical road data without using models. Erke et al. [14] measured the proportion of vehicles that diverted onto a specific route recommended by a VMS. While 20 percent of vehicles diverted according to the recommended route, the study period was from 10:30 p.m. to midnight over two nights, for a total of three hours of observation, which is a noticeably limited study period. However, as Levinson and Huo [13] found that diversion rates are significantly affected by the presence of a VMS message within 10 min of activation, the findings by Erke et al. [14] are still reliable.

Ghosh et al. [15] used flow rate data from loop detectors to calculate a flow change rate, which is a moving percent difference from the median value. Data were used from days without crashes to determine the median flow rate and data from days with crashes were used to determine how the flow change rate was affected when a VMS message was activated due to the crash. While the researchers found that VMS messages have a significant effect on the outgoing flow rate of an offramp, they also acknowledged that the presence of a crash can be obtained through many channels (e.g., television, radio, internet) and that it is difficult to fully determine the individual effect of VMS on diversions.

Romero et al. [16] collected flow data from highway loop detectors in Madrid, Spain to evaluate if VMS influenced the diversion rate onto a tolled highway from a non-tolled highway. The researchers found that diversion rates rose when the VMS displayed information. Specifically, when incident messages and the travel time on each of the highways were shown, the diversion rate to the tolled highway increased significantly.

Basso et al. [17] utilized anonymous vehicle data collected by automatic vehicle identification gates stationed before and after a VMS to determine how speed and lane change behavior were affected by non-recurrent VMS message activations. Six months of data were analyzed, and the researchers found that 88 percent of the time, VMS messages did not reduce driver speeds, and 72 percent of the time, messages failed to initiate driver lane changes.

Xuan and Kanafani [18] developed a two-part methodology involving a with-and-without analysis and a before-and-after analysis conducted on a flow rate dataset from freeway loop detectors. The combination of these two analyses as well as the consideration of the effect of visible congestion created a robust analysis to measure the effect of VMS message activation on diversion rates, while finding no significant impact with the data used.

When considering the impacts of VMS messaging on vehicle speeds, various analyses can be conducted depending on the types of data collected. Glendon and Lewis [19] displayed three different anti-speeding messages on a portable VMS near a school to evaluate under what conditions VMS could reduce speeds. The reduction in speeds observed varied depending on the day of week and time of day, but speed reductions were observed during the week the messages were displayed and partial speed reductions were seen the week after.

Rämä and Kulmala [20] used vehicle speeds to determine how effectively VMS messaging could reduce driver speeds when posting weather-related messages. By comparing speeds before and after the VMS location, they found a significant reduction in speeds of 0.75 mph (1.2 kph). This analysis used visual observations of roadway conditions to determine if the roads were icy but did not collect any sensor detected weather data.

Keshari et al. [21] found very similar results nearly 25 years after Rämä and Kulmala [20] when posting winter-weather related messaging on VMS in Michigan. Two messages were tested (e.g., “Bridge Ices Before Road/Reduce Speeds,” “Slippery Road Conditions/Reduce Speeds”) during winter weather conditions before bridges at three locations and found that the second message (“Slippery Road Conditions/Reduce Speeds”) reduced speeds by 0.6–0.7 mph compared to the default travel time messages. In addition, the researchers observed that vehicles with a faster initial speed had the largest speed reductions.

Sui and Young [22] used speed data to measure the impact of VMS messaging but also integrated weather attributes as collected by road weather stations. Using a standard linear regression model, the researchers found VMS messaging had a significant impact on driver speeds above those seen by weather interactions.

As is shown in the literature, many different methodologies for evaluating the effect of VMS using empirical road data have been developed. The most commonly used methods are hypothesis testing and measuring effect sizes [13,17,19,20], but time-dependent traffic assignment [12], binary logit [16], non-parametric regression [18], and linear regression [14,18,22] models have also been used. Keshari et al. [21] utilized a fixed-effects model and generated separate models for different treatment sites and vehicle types, but a mixed-effects model has not been used in the literature. In addition, most of studies in the literature review only considered single uses of VMS, such as lane, speed, or diversion changes. This research evaluates how VMS impacts vehicle diversions when crashes occur downstream as well as how speeds change when winter weather messaging is presented. The combination of two approaches provides a wider perspective on the uses of VMS and its effectiveness in a variety of situations.

Overall, this research contributes novel research by using mixed-effects models and evaluating the effect of VMS on both diverting and slowing drivers in the same geographic region.

3. Methodology

The methodology for this research is presented in this section. First, the diversion rate analysis will be discussed, followed by the weather analysis. Each section will discuss the data preparation and the evaluation methodology used.

3.1. Diversion Rate Analysis

The diversion rate analysis measured the effect of VMS messages located on Utah highways in diverting drivers to alternate routes when a crash was present downstream. The effect of VMS messages pertaining to downstream crashes were analyzed with a combination of road data types to evaluate their effect.

For the diversion rate analysis, aggregated highway volumes in 15 min bins were collected at corresponding mainline and offramp stations on the day of the crash and on a control day. Mainline speeds were also collected to control for congestion. This section outlines the data preparation and the methodology developed for this analysis, which included a mixed-effects model.

3.1.1. Diversion Rate Analysis Data

Multiple types of data were collected for this analysis. VMS message data were retrieved from UDOT’s TransSuite server to identify message context and duration. While various message types were contained in this dataset (e.g., travel time information, amber alerts, etc.), only messages pertaining to crashes were selected for analysis. These messages contained information on the location or effect of a downstream crash (e.g., congestion, stopped traffic, or blocked lane messages). However, it was uncommon for messages to contain a specific diversion route or offramp for drivers to follow.

Crash data were collected in collaboration with the researchers of UDOT Report UT-23.05 [23], which focused on measuring the benefits of expanding Incident Management Teams on Utah freeways. The researchers evaluated a metric called Excess Travel Time (ETT) which calculated the aggregate delay time of drivers due to a crash. The data for individual crashes on Utah freeways were gathered from March to August of 2018 and 2022. The researchers selected these timeframes in coordination with UDOT officials to avoid interactions with inclement winter weather and the unique traffic conditions present during the COVID-19 pandemic. The ETT metric was calculated in hours and crashes were sorted based on total delay. This dataset provided the time, milepost location, and impact of all Utah freeway crashes during the study periods.

To select crashes that would be ideal for a diversion rate analysis, a set of four criteria were used:

• The crash needed to have a high ETT. A higher ETT was theorized to correlate with a larger number of drivers impacted, longer queues, a higher potential for drivers to divert, and more freeway offramps available for analysis. While a high ETT threshold was not quantified, crashes were sorted by ETT and those with the highest ETT were evaluated first.
• At least one alternate route for drivers to bypass the crash was required. Alternate routes around the crash were critical so vehicles had an incentive to exit the roadway.
• The posting of at least one VMS message upstream of the crash pertaining to the crash was required.
• Crash queues needed to be independent from other congestion or crashes on the freeway. This was important for reducing the effect of confounding variables on the crash queues and to increase the probability that any increase in vehicles diverting were due to the incident being studied.

These four selection criteria were chosen specifically to control, to the best of the research team’s ability, the situations under which these crashes occurred and to provide the best circumstances under which to study the effect of VMS messages. It is important to note that the selection of these criteria implies that crashes occurring on rural freeways, low-volume corridors, or stretches of roadway without ITS devices are not represented. In addition, because crashes with a higher ETT were prioritized, short-duration and moderate-impact crashes were less likely to be included.

To select crashes for analysis, the crash selection criteria were implemented in a vetting workflow shown in Figure 1. First, crashes with the highest ETT were chosen because they had the highest vehicle hours of delay and the potential for higher observable diversion rates. Each crash date and time were then entered into ClearGuide, an online Global Positioning System (GPS)-based traffic visualization tool. This tool enabled the research team to verify that a crash occurred at the time and place indicated, observe the impact of the queues on vehicle speeds caused by the crash, and ensure that there were no confounding crashes or congestion that would interfere with the queues caused by the crash. If no interfering congestion was found, all potential VMS messages posted around the time and location of the crash were identified and each message was checked for relevance and proximity to the crash. Then, the UDOT Traffic website [24] was utilized to identify each offramp between the crash and the last applicable VMS. Last, UDOT Performance Measurement System (PeMS) loop detector data was used to identify the mainline and offramp volumes and mainline speeds for analysis.

Based on results found in the literature, it was predicted that once a VMS message is activated, a difference in behavior will be observed within 10 min [13]. To accommodate for this, volumes for 20 min before and 40 min after the activation of the sign were collected. This allowed for a control period before the VMS activation time as well as accounting for the time the vehicles would take to travel to the offramp from the VMS during the after time period. Data were also collected for a control day with the same time period and offramps as the crash to provide data for the same segment under normal conditions. The control day was selected as either the week before or week after the crash on the same day of the week without a crash or incident. In addition to the volumes, speeds for the mainline segments were also collected on the crash and control days to provide a metric to control for the formation of visible queues.

Eight crashes were selected for the diversion rate analysis, three from 2018 and five from 2022. A total of 650 flow and speed observations were compiled from 32 offramp locations across Salt Lake and Utah counties. The messages of the selected crashes varied but typically were presented by alternating two messages. The first presented information on lane closures or crash details (e.g., “Left Lanes Clsd 5 Miles Ahead,” “Crash 19 Miles Ahead,” “Stopped Traffic at 2500 S”) and the second gave appropriate direction to the drivers (e.g., “Use Caution,” “Expect Delays,” “Reduce Speed,” “Prepare to Stop,” “Use Alt”). Although a specific threshold for high ETT was not quantified, the ETT of the eight selected crashes ranged from 7192.00 to 1404.61 h of total delay.

3.1.2. Diversion Rate Analysis Methodology

The research team was interested in measuring the effect of VMS messages that warned drivers about a downstream crash and how the posting of these messages affected diversion rates at ramps immediately upstream from the crash. To estimate this effect, a mixed-effects model was developed, as shown in Equation (1). The dependent variable of the model needed to compare the diversion rate of the crash day to a typical diversion rate on the control day, so the ratio of the crash day diversion rate over the control day diversion rate was selected. A concern that spurious correlations could appear as a result of using a ratio as the dependent variable in a regression arose [25], so the reciprocal of the control day diversion rate was included as a predictor variable to measure the stability of the model. A variable indicating if the VMS had a message active or not at each observation time was included.

The research team was also concerned that slower speeds and visible congestion on the roadway may incentivize drivers to divert off the highway rather than stay and incur higher travel times, so the speed on the highway was controlled for in the model. Because the dependent variable compared the number of crashes on the crash day to the associated control day, the same structure was used for vehicle speed by using the ratio of the mean vehicle speed on the mainline of the highway at each observation on the crash day to that of the control day. The research team also had concerns that the effect of the VMS might be due to a change in vehicle speeds rather than a VMS activation, so the interaction between VMS and the ratio of crash day to control day speeds was included. The interaction term estimates how the effect of VMS messages on diversion rates changes depending on vehicle speeds. For example, drivers might ignore VMS messages when visible congestion is low, but when traffic congestion is present, drivers may turn to VMS messages for critical information, which in turn increases diversion rates. Lastly, the random effects for each crash and offramp were included to control for variation in the observed values for each individual crash and at each offramp. Data on crash lane location was not available to the authors and could not be included in the model. All crashes were on large freeways of three or more lanes.

DR = RCD + VMS + SR + VMS × SR + (1|Crash:Mramp)

(1)

where DR = ratio of crash day diversion rate to control day diversion rate,

RCD = reciprocal of control day diversion rate,

VMS = binary variable indicating if the VMS was active or inactive,

SR = ratio of crash day mainline speed to control day mainline speed,

VMS × SR = interaction of VMS and SR variables,

Crash = crash identification number, and

Mramp = offramp identification number.

3.2. Weather Analysis

While the diversion rate analysis was concerned with congestion-based VMS messaging, the weather analysis was concerned with driver safety warnings, specifically those pertaining to adverse winter weather conditions in Utah canyons. This analysis aimed to develop an understanding of driver behavior in response to VMS messages about weather conditions that would reduce driver response time or vehicle maneuverability.

For the weather analysis, vehicle speeds before and after the position of a VMS were collected. In addition, a variety of roadway traffic and weather factors were collected to control for confounding factors. This section outlines the data preparation and the methodology for the mixed-effects model built for the analysis.

3.2.1. Weather Analysis Data

Messages activated between 8:00 a.m. and 5:30 p.m. were chosen for analysis. To ensure a higher number and more constant sample of vehicle speeds, speed data were only collected for the hours between 7:00 a.m. and 6:00 p.m. Speed data pertaining to each message were collected in intervals of up to 30 min before message activation and up to 90 min after the message was turned off. The before activation periods were used as the control data. This method provided comparisons directly adjacent to and with similar road and weather conditions as the VMS activation period. The research team decided that by comparing observations when the sign was active to when it was inactive, reliable comparisons could be made. For each message selected for the analysis, the date and time were checked with a comprehensive crash database to ensure no crashes occurred near the VMS during the data collection period.

Canyons in Northern and Central Utah with the following attributes were selected for analysis: functioning PeMS units, road weather information system (RWIS) stations, VMS in locations that were likely to affect driver speeds, winter weather, and winter weather-related VMS messaging. Initially, five canyons had the required attributes, but one canyon had only one winter weather-related message that was outside of the selected 8:00 a.m. and 5:30 p.m. analysis window. One additional canyon had variable speed limit signs, which were outside the scope of this research and could potentially influence driver behavior. These two canyons were not chosen, and the remaining three canyons were selected as the canyons of analysis: Provo Canyon, Sardine Canyon, and Weber Canyon. Figure 2 shows the geographic locations for each of these canyons with their associated VMS, PeMS, and RWIS stations. Care was taken to select the best available data collection stations. Only stations near the roadways were selected as they provided the best estimate of real-time weather conditions at the measurement stations. VMS selected were placed on straight sections of roadways at a minimum height of 18 feet and prior to curves or other dangerous sections of roadways. Random effects are provided for each sign, but legibility was assumed to be adequate for each sign and verified using Google Streetview images.

The research team decided that only messages applicable to all vehicle types and roadway segments should be included in the analysis. Common message types that were disqualified for inclusion included messages primarily for vehicles with large surface area, (e.g., “HIGH WINDS AHEAD”) or messages that were applicable to only one portion of the canyon (e.g., “BLACK ICE 4 MILES AHEAD”). In addition, because the analysis was focused on winter weather messages, only those messages specifically pertaining to winter weather conditions were selected. Messages also needed to pertain to weather conditions that could be verified using RWIS data. Messages were selected between October 2020 to April 2022 and needed to have been activated for at least 30 min to be considered. Overall, 47 messages were selected for the analysis. The content of the messages for the weather analysis is not included in this paper but can be found in the corresponding UDOT Report UT-23.20 [26]. Most of the messages cited “Winter” or “Icy” conditions as the impetus of the message. Approximately half of the messages included the instructions to reduce speed, while the remainder instructed to use caution, while three stated roadway conditions only.

RWIS data for winter weather conditions were collected through the University of Utah MesoWest website [27]. This website conglomerates information from several weather databases from across the United States, including several dozen RWIS stations in Utah. For each message selected for analysis, data on road grip and visible precipitation were used. These were the only RWIS data categories considered perceptible by the driver and were consistently provided at each VMS location.

Speed observation data were collected using UDOT’s Freeway Performance Metrics Occurrence Data website [28]. While the diversion rate analysis used aggregated speed data, collecting occurrence data allowed for a more granular analysis of traffic patterns and conditions than aggregated data would allow. In addition, in the diversion rate analysis, the speed was used as a single factor in the mixed-effects model to control for visible queues, while in the weather analysis, speed is the main evaluation variable. Speed data for each vehicle included vehicle direction, vehicle class, speed, duration of detection, timestamp of detection, and other variables not used in the study.

To combine the VMS messages, RWIS data, and speed occurrence observation datasets, two main calculations were made. First, an adjustment to the timestamp of when the vehicle passed the VMS and second, the assignment of RWIS weather observations to each speed observation.

When considering how to determine whether a driver had observed the VMS as active or inactive, the research team decided that an adjustment to the timestamp of each vehicle’s speed observation was necessary. A simple standardized time adjustment was considered, but as each vehicle’s speed differed, a dynamic methodology to calculate the adjusted timestamp for each observation was determined to be a more accurate option, as shown in Equation (2):

$$A{T}_i=R{T}_i-\left(D_i/\left({\displaystyle \frac{1}{N_j}}\sum S_j\right)\right)$$

(2)

where $A{T}_i$ = adjusted timestamp of observation $i$,

$R{T}_i$ = recorded timestamp of observation $i$,

$D_i$ = distance from observation $i$ to VMS, ft,

$N_j$ = number of speed observations, and

$S_j$ = vehicle speed observations within time envelope, $j$.

To compute a dynamic travel time, an adjusted timestamp was calculated for each speed observation individually. First, Google Maps was used to estimate a typical travel time between the vehicle detector station and the VMS for the chosen VMS message. The travel time, t, was then added to and subtracted from the recorded timestamp, $R{T}_i$, of each individual speed observation to create a time envelope of [$R{T}_i$ − t, $R{T}_i$ + t]. For observation, i, all speed observations within the time envelope were separated into vector S_j and the arithmetic mean was taken to create an average speed observation. An adjusted timestamp ($A{T}_i$) was then calculated for speed observation, i, as a function of the recorded timestamp ($R{T}_i$), the distance from the vehicle detector station to the VMS ($D_i$), and the average speed described. Once an adjusted timestamp was determined for each speed observation, the adjusted timestamp was used to determine if the driver would have observed the sign as active or inactive.

For example, if the recorded time stamp (RT) was 8:20:00 a.m., the distance between the VMS and the speed observation was 5 miles, and the average speed of vehicles within the time envelope was 55 mph, then the adjusted timestamp would be 8:20:00–00:05:27, or 8:14:33 a.m.

After collecting the corresponding weather observation data for the analysis, the next step was to assign weather data to each speed observation. The complication with this assignment was that the RWIS and UDOT speed observation systems are not correlated, rather they come from two separate systems with different priorities and purposes. RWIS stations are not placed at regular intervals, and some are placed in locations other than adjacent to the roadway. The latter concern was resolved by selecting RWIS only near the roadway. For each speed observation, one weather observation per category would need to be assigned, but the location and time that each weather observation was taken would vary throughout the analysis period. For this purpose, an optimization method based on the calculation of the cumulative product of minimum-maximum normalizations for time and distance was developed [29] as shown in Equation (3):

$$F_{ijR}=T_{iR}\times \prod _{k=1}^k\left\{\left({\displaystyle \frac{{\mathrm{obs}}_{ijk}-\mathrm{m}\mathrm{i}{{\mathrm{n}}_{\mathrm{o}\mathrm{b}\mathrm{s}}}_{ijk}}{\mathrm{m}\mathrm{a}{{\mathrm{x}}_{\mathrm{o}\mathrm{b}\mathrm{s}}}_{ijk}-\mathrm{m}\mathrm{i}{{\mathrm{n}}_{\mathrm{o}\mathrm{b}\mathrm{s}}}_{ijk}}}\left(\mathrm{M}\mathrm{a}{\mathrm{x}}_{\mathrm{n}\mathrm{e}\mathrm{w}}-\mathrm{M}\mathrm{i}{\mathrm{n}}_{\mathrm{n}\mathrm{e}\mathrm{w}}\right)\right)+\mathrm{M}\mathrm{i}{\mathrm{n}}_{\mathrm{n}\mathrm{e}\mathrm{w}}\right\}$$

(3)

where $F_{ijR}$ = best fit factor for RWIS observation type, R, correlating to, RWIS observation, i, for speed observation, j;

R = RWIS value of interest (road grip and visible precipitation);

$T_{iR}$ = data present factor for data type, R, and RWIS observation, i; if the specific RWIS weather attribute for the given, i, is “NA” then ${\mathrm{T}}_{iR} =0$, else ${\mathrm{T}}_{iR} = 1$;

${\mathrm{obs}}_{ijk}$ = RWIS difference, i, from, speed observation, j, for, data type, k;

k = [Time difference, Distance difference];

$\mathrm{m}\mathrm{i}{{\mathrm{n}}_{\mathrm{o}\mathrm{b}\mathrm{s}}}_{ijk}$ = minimum RWIS difference, i, from speed observation, j, for data type, k;

${{\mathrm{m}\mathrm{a}\mathrm{x}}_{\mathrm{o}\mathrm{b}\mathrm{s}}}_{ijk}$ = maximum RWIS difference, i, from speed observation, j, for data type, k;

$\mathrm{M}\mathrm{a}{\mathrm{x}}_{\mathrm{n}\mathrm{e}\mathrm{w}}$ = new maximum value, or 1.0;

$\mathrm{M}\mathrm{i}{\mathrm{n}}_{\mathrm{n}\mathrm{e}\mathrm{w}}$ = new minimum value, or 0.0;

i = RWIS weather attribute data; and

j = vehicle speed observation.

This method took two or more weather data points and, based on the distance from the VMS and when the weather observation occurred, the observation that was more reasonable and relevant for the speed observation at a specific time was selected. The subset of data, ij, were identified by finding rows with the smallest difference between the identified vehicle speed observation, and the RWIS observation timestamp. Up to four times the number of RWIS stations of data rows were collected and an $F_{ijR}$ factor was calculated for each, with 1.0 being most optimal, and 0.0 being least optimal. Corresponding weather values, i, to the maximum $F_k$ were then assigned for the speed observation, j. In this way, one weather observation out of multiple possibilities were assigned to each vehicle speed observation.

3.2.2. Weather Analysis Methodology

To model the effect of VMS messaging warning drivers to slow down on individual driver speeds, a mixed-effects model was developed. Several fixed and random effects were included to control for and measure the effect of potential confounding variables. The dependent variable in the model was the speed of each individual vehicle observation. A binary fixed-effects variable indicating if the VMS phase was active or inactive was included as a main fixed effect of interest.

Historically, the AM peak (7:00 a.m. to 10:00 a.m.), Midday (10:00 a.m. to 4:00 p.m.), and PM peak (4:00 p.m. to 7:00 p.m.) have unique traffic patterns. To account for this, a data category was created to put each observation into one of three bins based on when the speed was observed, referred to as the time interval. The exact bins were chosen loosely, providing for fluctuations in the traffic flow. To understand the traffic flow, the Peak Hour Factor (PHF) and hourly flow were calculated for each observation and used as fixed effects in the model.

At first, the different PeMS stations were considered as a random effect, but in comparing models, it was determined that adding an additional constraint did little to change the model results, so it was removed. Instead, a fixed effect variable using the distance between the VMS and the speed observation location was used to estimate the effect of distance traveled from the VMS on the model.

From the large amount of available RWIS data, ultimately two variables were selected as fixed effects: road grip and visible precipitation. These variables were determined as the most likely to have an effect of a driver’s speed. For categorical variables such as the sign phase, time interval, and visible precipitation conditions, the analysis was conducted using the speeds in one category as the basis for comparing against all other variables.

In the data, observations were sourced from seven different VMS and 47 different messages with unique phrasing and intent across two different winter seasons. Because the messages came from varying areas and were presented on different VMS, it was decided that the mixed-effects model should include random effects for each VMS and each VMS message.

With the necessary variables defined, the mixed-effects model equation was defined as outlined in Equation (4).

$$SD=SP+TS+MPD+PHF+HrF+Gr+Vis+\left(1|VMS:MN\right)$$

(4)

where SD = observed vehicle speed (mph),

SP = sign phase (active or inactive),

TS = time state of the observation,

MPD = milepost difference between the VMS and observed speed,

PHF = Peak Hour Factor,

HrF = hourly flow rate (vph),

Gr = road grip,

Vis = visibility factor,

VMS = VMS ID number, and

MN = message number.

4. Results

The results for this research are presented in this section. First, the diversion rate analysis results will be discussed. Then, the weather analysis results will be outlined.

4.1. Diversion Rate Analysis Results

Before discussing the findings of the diversion rates model, it is important to note that the estimate of the interaction term between VMS and the ratio of the crash day mainline freeway speeds to the control day mainline freeway speeds, SR, was taken at the median SR value of 0.987 rather than at 0. This is because a ratio of speeds of 0 is not logical for speed data. This adjusted value is referred to as $S{R}_{adj}$ in the following tables. This shift in where the estimate is taken does not affect any other estimates in the model.

Table 2. Diversion Rates Mixed-Effects Model Results.

	Diversion Rates Model
Predictors	Estimates	CI	p
(Intercept)	0.78	0.58–0.97	<0.001
RCD	0.03	0.02–0.04	<0.001
VMS	0.18	0.12–0.24	<0.001
SRadj	−0.78	−0.99–−0.58	<0.001
VMS × SRadj	−0.50	−0.72–−0.29	<0.001
Random Effects
σ2	0.13
τ00 Crash:Mramp	0.04
τ00 Crash	0.05
ICC	0.42
NCrash	8
NMramp	31
Observations	624
Marginal R2/Conditional R2	0.455/0.681

shows the results of the mixed-effects model. The estimate of RCD, or the reciprocal of the control day speeds, is low at 0.03, which verifies that the model is stable and thus is appropriate to model the ratio of the diversion rates directly as the dependent variable. The estimate for the binary VMS variable that indicates if the VMS was activated is 0.18, indicating that the activation of a VMS was associated with an increase in diversion rates by approximately 18 percent after adjusting for the regressors included in the mixed-effects model. The 95 percent confidence interval bounds for this effect ranging from 12 percent to 24 percent is significant. The adjusted SR was negatively correlated, meaning that as speeds decrease, diversion rates increase. The estimate of −0.78 means that for a decrease in speeds of 10 percent, the diversion rates increase by approximately 7.8 percent. The interaction between VMS and $S{R}_{adj}$ is negatively correlated, demonstrating that on days when the crash day speeds are slower than the compared control day, the effect of a VMS message on the diversion rates is higher. This interaction is important to note because not including it could potentially inflate the supposed influence that the activation of VMS have on diversion rates.

The relationship between the position of the offramp in relation to the crash and diversion rates was of interest, so the random effects of each offramp for each crash were plotted against the location of each offramp in relation to the crash. Figure 3 shows the random effects of each offramp within each crash using a single connected line in the figure for the offramps associated with a particular crash. Crash IDs are correlated to crash details included in the corresponding report [26]. Note that the x-axis is reversed to represent the offramp closest to the crash on the right of the figure. As the lines move to the left, the number of offramps between the crash and the collected data point increases. If a trend of increased diversion rates at offramps closer to the crash was present in the data, there would be a clear upward trend of the random effects as the offramp positions moved closer to the crash. While that may be the case to some degree, the correlation is not strong enough to warrant this finding. A more crucial finding from this figure is that the random effects for each crash are unique. To the extent that any two crashes share a pattern, it seems that as a whole, they have a relatively constant random effect across all offramps.

4.2. Weather Analysis Results

The results of the mixed-effects model for the weather analysis are presented in

Table 3. Weather Analysis Mixed-Effects Regression Results.

	Weather Speed Model
Predictors	Estimates	CI	p
(Intercept)	28.942	26.07–31.82	<0.001
SignPhase [On]	0.228	0.17–0.29	<0.001
TimeState [AM]	−0.311	−0.37–−0.26	<0.001
TimeState [PM]	−2.393	−2.42–−2.36	<0.001
MilePost Diff	0.015	0.01–0.02	<0.001
PHF (per 0.2)	0.502	0.47–0.54	<0.001
Hourly Flow (per 500 vph)	2.894	2.85–2.94	<0.001
Road grip (per 0.1)	3.344	3.32–3.37	<0.001
Visibility [Light Precip]	−1.613	−1.65–−1.57	<0.001
Visibility [Mod. Precip]	−2.669	−2.74–−2.6	<0.001
Visibility [Heavy Precip]	−5.104	−5.2–−5.01	<0.001
Random Effects
σ2	81.4
τ00 MessageNum:VMS.ID	8.97
τ00 VMS.ID	12.84
ICC	0.21
NMessageNum	47
NVMS.ID	7
Observations	1,426,105
Marginal R2/Conditional R2	0.190/0.361

. It should be noted that some estimates have been scaled to make their effect easier to comprehend. For example, because the PHF cannot exceed 1.0, the estimate has been scaled so it refers to the effect of vehicle speeds in mph for every increase in PHF of 0.2. Similarly, hourly flow has been scaled to a rate per 500 vph and road grip to a 0.1 increase. In addition, 1.4 million observations were included in the dataset, so all interactions were statistically significant. In this case, practical significance and statistical significance may not be the same due to the large number of observations.

The effect for SignPhase [On], or when a VMS was activated, was 0.228, meaning that the activation of a VMS message increases driver speeds by approximately 0.23 mph (0.37 kph) after adjusting for the regressors included in the mixed-effects model. It is interesting to note that the effect is positive, so the presence of a VMS message increased speeds marginally.

The milepost difference between where the speed observation was collected compared to the location of the VMS had a minimal effect on driver behavior, at a speed increase of 0.015 mph per increase in 1 mile traveled from the VMS. This finding suggests that the influence of the VMS message does not change with the distance between the driver and the VMS. The most significant effects were the road grip and hourly flow, even with the transformations.

The time state fixed effects results indicate that drivers reduced speeds during AM and PM peak periods. This suggests that traffic conditions during these peak periods differed significantly from midday traffic conditions. However, in these canyons, PHF and hourly flows also differ widely from day-to-day depending on weather conditions. In major canyons, including Provo Canyon and Sardine Canyon, these results likely hold true, but it is difficult to confirm this effect in Weber Canyon due to the limited data available and the lower traffic volumes.

Figure 4 plots the random effects for each individual VMS. Corresponding VMS IDs are noted in Figure 2. Based on these random effects, the presence of VMS in Provo Canyon and Weber Canyon tend to reduce speeds while the activation of VMS in Sardine Canyon tend to increase speeds. This effect also appears to be correlated with the length of the canyon, as Provo Canyon and Weber Canyon are both shorter than Sardine Canyon. However, causation cannot be assumed; this correlation is observational only.

It is also interesting to note that the confidence intervals for Provo Canyon and Weber Canyon both include an effect of zero mph. This indicated that it is possible for these VMS to have no effect on driver speeds. However, it is also important to note that out of the 1.4 million observations, Provo Canyon and Weber Canyon consisted of approximately 30 percent of the observations while Sardine Canyon consisted of the remaining 70 percent. The size of the confidence intervals are likely in direct response to the differences in population sizes for the varying VMS.

5. Discussion

In the diversion rate analysis, the effect of activating a VMS on the diversion rate during crash situations was an increase in diversions off the highway by 18 percent. The level of this effect is supported by Ghosh et al. [15] who also used empirical flow data for days with crashes and found increases in diversion rates by 14 percent. These results are also similar to studies showing that diversions increase due to route change messaging [14] and that drivers will even divert from non-tolled highways onto tolled highways when presented with VMS messages with incident information [17]. In contrast, prior research has also shown that VMS do not significantly influence lane changing behavior [17] for diversions, even when considering visible congestion [18].

To capture the influence of visible congestion on driver diversions during crash situations, a proxy measure was included in the regression. It was defined as the ratio of highway speeds on the crash day to speeds on the same weekday one week before or after (i.e., control day). The adjusted speed ratio indicated that when speeds decrease by 10 percent, the diversion rate increased by 7.8 percent. This suggests that traffic speed may become a more important factor for diversion rates as speeds decrease, which is supported by prior literature that visible congestion is an important factor in diversions [18].

In the weather analysis, the effect of activating a VMS with a winter weather message was an increase in speeds by 0.23 mph. This effect is very small and shows that the VMS had very little effect on driver speeds. Although the increase in speeds was statistically significant, it was unlikely that such a minor increase in speeds holds practical significance. Because the VMS may have been more likely to display a sign warning drivers to slow their speeds during inclement weather, the actual speeds when the VMS were activated were likely lower than during baseline conditions but may have been slightly faster than would have been predicted by other variables in the model. An additional reason for this may be the difference in sample sizes, as the number of speed observations when the VMS was active was more than eight times the number of observations when the VMS was inactive. It should also be considered that the weather within these canyons is often inconsistent. Thus, drivers may be led to believe that the message is no longer applicable for their location.

Additional findings from the model suggest that visible precipitation and road grip had strong impacts on vehicle speeds. The effects of visible precipitation were consistent with conventional understanding that as visible precipitation worsened, speeds decreased. A significant increase in speed reduction was present from heavy to moderate visible precipitation states. Additionally, as road grip levels increased, so did speeds, which indicated that drivers responded to either how the vehicle responded to road conditions or that drivers were able to predict road grip conditions by external weather conditions, including perceived air temperature and previous precipitation. However, it was challenging to confirm one hypothesis or another due to the limited research on this behavior.

The results from this analysis indicating a slight change in speed due to VMS activation are supported by Rämä and Kulmala [20] as well as Keshari et al. [21], who also found small changes in speed from 0.6 to 0.8 mph, respectively. While this study measured a slight increase in speeds, the two prior studies found slight decreases in speeds. The data used in these studies varied; Rämä and Kulmala (2000) used similar data to the weather analysis (e.g., disaggregate speeds from loop detectors), while Keshari et al. [21] used LiDAR to track individual vehicle speeds. This study does stand apart from prior research by using dynamic weather data assignment to each VMS and using a large sample dataset.

In practice, these findings suggest that DOTs may see mobility benefits from focusing resources on incident-management applications involving VMS messages. Further research should be conducted on weather-related safety VMS messaging to determine if speed compliance is higher with certain vehicle classes or with certain types of messages.

There were several limitations to this research, which may yield future research opportunities. First, it was difficult to know with certainty whether drivers were aware of VMS despite the sign being active or inactive. Second, crash lane location data were not available, which could impact the severity and length of the resulting congestion. Future research should evaluate the impact of the location of a crash on crash-related congestion and driver diversions, particularly when VMS messages are also present. Third, in the diversion rate analysis, many factors could not be controlled for using the mixed-effects model that could affect diversion rates, such as driver familiarity of roadways and alternate routes, driver usage of GPS wayfinding apps such as Google Maps, behavior changes based on driver urgency to arrive at their destination, and the presence of a visible queue. Such GPS data was not available to the research team, so the use of GPS by drivers was assumed as a constant on treatment and control days. It is unclear the extent to which the effect found in the diversion rate analysis is impacted by these factors and how much of the effect may be due to them, particularly the presence of visible queues and slowing vehicles. Prior research using stated preference surveys has shown that visible queues can impact a driver’s decision to divert [3] and the speed variable was included as a proxy for visible queues, but future research should evaluate how much of an effect is due to VMS information compared to visible queues. Fourth, crash severity was not included in the diversion rate analysis due to the low number of crashes, but future research should consider its effect with larger crash sample sizes.

For the weather analysis, because the analysis was limited to weather-related VMS messaging, the findings of the study can only be applied to these types of messages and cannot be applied to any other type of VMS message. Also, vehicle class information was not available in the weather analysis but should be considered in the future. Basso et al. [17] previously found that larger vehicles complied more with lane change messages, particularly compared to motorcyclists, so this effect could be seen in speed evaluations as well.

Both the diversion rates and weather analyses would have been strengthened with additional data to train the regression model to predict behavior rather than solely measuring the effect of VMS messages. Future research on VMS could utilize not only highway flow data but also speed data to better understand the effect of VMS on vehicle speeds during crash situations. Advanced technologies, such as the use of eye-tracking, could also be used in future research to identify how VMS are perceived by drivers and how they behave towards different VMS scenarios.

6. Conclusions

VMS are generally accepted as an effective strategy to improve mobility and safety, but research has not consistently illustrated these benefits. VMS constitutes a significant investment from DOTs, so this research was conducted to better understand effective uses of VMS. The research presents a with-and-without and a before-and-after analysis of the effect of VMS on driver behavior. The with-and-without, or diversion rates, analysis evaluated the effect of VMS on diversion rates during crash situations on Utah highways. VMS message and crash data from March to August of 2018 and 2022 were collected and four crash selection criteria were chosen to guide selection of appropriate crashes for analysis. Eight crashes were chosen and 650 flow and speed observations were obtained from UDOT for the day of the crash and a control day a week before or a week after. A mixed-effects model was built to evaluate the effect of VMS during each incident. The dependent variable was selected as the ratio of diversion rates on the crash day compared to the control day. It was found that when VMS messages were active during incidents along I-15 warning drivers of future delays, diversion rates from the freeway increased by 18 percent.

The before-and-after, or weather, analysis evaluated the change in driver speed due to VMS messages in Utah canyons during winter weather conditions. Sardine Canyon, Weber Canyon, and Provo Canyon in Utah were selected as study areas and VMS message, speed occurrence, and weather data were collected. Overall, 47 messages were selected from October 2020 to April 2022 and over 1.4 million speed and weather observations were obtained and combined into a single dataset. A methodology based on the cumulative product of minimum-maximum normalizations for time and distance [29] was built to assign a weather observation to each vehicle occurrence observation. A mixed-effects model was built with eight independent variables from available traffic, time, and weather data, and random effect variables for the VMS ID and VMS message number were included. The model showed that when VMS messages presented weather-related messaging during the winter season, driver speeds increased by 0.23 mph (0.37 kph), a practically insignificant value. These results show that while VMS messages may inform drivers of weather conditions, they had little effect on reducing driver speeds.

Overall, the results of this study show that VMS are effective in diverting drivers off highways during crash situations but may have less effect in slowing drivers when presenting weather-related messaging. The results do show that the effects of precipitation and road grip provided more of an impact on reducing vehicle speeds than the presence of a VMS message. The results of this study can be useful for DOTs to consider when designing and implementing VMS deployment strategies or when developing studies to measure VMS impact in their respective geographies.