Predicting Traffic Sign Retro-Reflectivity Degradation Using Deep Neural Networks

Abstract

Traffic signs are essential for the safe and efficient movement of vehicles through the transportation network. Poor sign visibility can lead to accidents. One of the key properties used to measure the visibility of a traffic sign is retro-reflection, which indicates how much light a traffic sign reflects back to the driver. The retro-reflection of the traffic sign degrades over time until it reaches a point where the traffic sign has to be changed or repaired. Several studies have explored the idea of modeling the sign degradation level to help the authorities in effective scheduling of sign maintenance. However, previous studies utilized simpler models and proposed multiple models for different combinations of the sheeting type and color used for the traffic sign. In this study, we present a neural network based deep learning model for traffic sign retro-reflectivity prediction. Data utilized in this study was collected using a handheld retro-reflectometer GR3 from field surveys of traffic signs. Sign retro-reflective measurements (i.e., the $R_A$ values) were taken for different sign sheeting brands, grades, colors, orientation angles, observation angles, and aging periods. Feature-based sensitivity analysis was conducted to identify variables’ relative importance in determining retro-reflectivity. Results show that the sheeting color and observation angle were the most significant variables, whereas sign orientation was the least important. Considering all the features, $R_A$ prediction results obtained from one-hot encoding outperformed other models reported in the literature. The findings of this study demonstrate the feasibility and robustness of the proposed neural network based deep learning model in predicting the sign retro-reflectivity.

1. Introduction

Highway traffic signs are fundamental means of conveying essential information to road users. They inform, regulate, and warn the drivers to ensure their safe and smooth movement through the transportation networks [1]. Signs must be detectable, readable, and understandable to the road users at a distance corresponding to their purpose. They must be designed to fulfill these requirements by proper selection of sign size and color, letters and numerals size and style, and the retro-reflective materials used for the background and legend. During the daytime, signs are usually well illuminated by ambient sunlight and, assuming accurate placement, are generally easy to read. However, at nighttime the scenario is rather different. Drivers of all ages often experience decreased visibility at night compared to daytime driving [2]. Inadequate or faded signs are difficult to read at night, especially for older drivers, and this is supposed to contribute significantly to vehicle crashes, injuries, and fatalities. Previous studies suggest that driver-related factors account for over 90% of total crashes [3]. To ensure the safety of drivers and vehicles, the local department of transportation (DOT) must have periodic traffic sign evaluation and replacement programs [4]. Evaluation of sign retro-reflection is one of the key strategies in this regard. It is measured by an index that is commonly known as “retro-reflectivity” or “coefficient of retro-reflectivity”, which is expressed in units of candela per lux per square meter (cd/lx/m²). Retro-reflectivity of traffic signs’ sheeting gradually deteriorates over time, thus making signs progressively less visible at night [5]. Highway agencies are using a variety of sign sheeting materials that provide greater luminance with significantly improved results.

Degradation of sign retro-reflectivity has been the focus of several of the previous studies (e.g., [5,6,7]). In previous works, simple regression models were explored and no advanced machine learning techniques were sought for predicting sign retro-reflectivity degradation. The predictive performance of the traditional regression model in this regard is also typically low, with one of the highest accuracies obtained by a study conducted by [1]. The weaker prediction performance of the statistical regression-based methods may be attributed to assumptions regarding predefined associations between variables and the linear form of the utility function, which may not necessarily hold for most real-world problems. Frequently, regression-based statistical models are unable to capture latent correlations among the predictor variables. Additionally, in previous studies, researchers have proposed models that are capable of predicting sign retro-reflectivity for a specific combination of sign sheeting grade and color. To the best of our knowledge, no study has incorporated sign retro-reflectivity deterioration for different sign attributes and aging conditions into one model. In this paper, we present a deep learning based regression model that is capable of predicting sign retro-reflection as a function of sheeting type/grade, sheeting brand (the manufacturer of the sheeting), sheeting color, observation angle, orientation (sign facing) and age. Developing a unified model allows our system to learn and cover the degradation patterns of all the different combinations of sheeting brands, sheeting material, color, observation, and orientation angle. The model outputs the expected retro-reflection value for a given year utilizing the traffic sign features. The main contribution of this study can be summarized as:

• We present a neural network based deep learning approach for traffic sign retro-reflection prediction. Most of the existing works investigated linear polynomial regression models that are unable to effectively model the complex relationship between the different predictors including sign age and retro-reflection.
• We present models that learn the retro-reflectivity degradation patterns for different combinations of traffic sign features. This is unlike previous works where separate models were used to learn the retro-reflectivity degradation pattern for different combinations of traffic sign features such as sheeting, color, or observation angle.
• Feature-oriented sensitivity analysis is conducted to examine the significance of each feature for determining the sign retro-reflectivity at the end of useful/service periods.

The remainder of this paper is organized as follows: Section 2 explores prior works in estimating the retro-reflection of traffic signs and identifying factors that affect the degradation of retro-reflection. Section 3 provides a description of the study area and data collection and their description. Section 4 details the methodology including the experimental configuration for the models. Section 5 highlights the study results. Finally, in Section 6 we summarize the key findings from the current work and provide an outlook for future work.

2. Related Work

Prediction of traffic sign functional service life and factors mainly responsible for retro-reflectivity deterioration are crucial for planned maintenance schemes to ensure a satisfactory level of service. In the past, several studies were initiated to explore factors contributing to sign retro-reflectivity. For example, Black et al. were pioneers in investigating the effect of factors such as signs elevation, orientation, solar radiation levels, temperature, precipitation, and age on sign retro-reflection [6]. A total of over 5000 signs were surveyed. It was reported that precipitation, sign age, elevation, and the temperature had a strong positive association with sign degradation patterns while there was no statistical evidence for the effect of sign direction on the same. However, this study does have few limitations. The majority of sign samples surveyed were of ASTM Type-I sheeting. The Sample size of type-III sheeting with an age greater than 15 years was low. Moreover, the developed deterioration models had $R^2$ values below 0.5, indicating that age could not explain the variability in sign degradation. Kirk et al. conducted a study to analyze factors affecting sign retro-reflection [5]. The authors noted that a sign’s age did not cause significant reduction in retro-reflectivity, whereas orientation (sign direction) was identified as a significant predictor variable. Wolshon et al. conducted a study to analyze the degradation of retro-reflectivity of different sign sheeting [7]. The authors developed 12 models to predict the functional service life of traffic sign according to their retro-reflectivity. The 12 models were proposed for 2 sheeting types, 3 colors, and the conditions for whether signs were cleaned or not. It was found that from the three variables (age, orientation, and the distance between the sign and the road), only age has a positive correlation with the degradation of retro-reflectivity. In another study, Bischoff and Bullock collected data for 1341 in-service traffic signs to examine retro-reflection degradation trends of white, yellow, and red-yellow colored signs [8]. Study findings showed that white and yellow signs had an average service life of 12 years, while red sign sheeting on average had 10 years of service life. The study further reported that sign orientation has a weak correlation with retro-reflectivity. This study collected three measurements on each sign, whereas the ASTM standard procedure requires four measurements. Immaneni et al. investigated the role of age in the degradation of sign retro-reflectivity for 1000 service signs surveyed in the state of North Carolina, USA [9]. Different types of regression models were developed. $R^2$ for the majority of models were less than 0.5, indicating that there may be other factors such as exposure, temperature, weather conditions, and air pollution that can significantly influence the rate of deterioration. In their study, Huang et al. proposed a series of mathematical models (linear mathematical model, second-order polynomial model, and cubic regression models) to predict the retro-reflectivity deterioration pattern of ASTM Type-I, Type-II, and Type-III sheeting deployed along the Jiangxi Yongwu expressway in China [10]. Results showed that both quadratic and cubic regression models were better predictors of sign retro-reflectivity. This study considered only two sheeting types and three colors. A lower sample size (N = 230) and low model predictions are the main issues of this study.

Over the years, different methods were proposed to forecast sign retro-reflectivity as a function of explanatory variables such as sign aging, sheeting type, sign sheeting orientation, temperature, precipitation, dew, etc. [1,7,11,12,13]. In his thesis, Swargam compared the performance of the existing regression models and a newly developed neural network model in predicting the retro-reflectivity for Type-I and Type-II sign sheeting [12]. Data for over 1100 signs from Ascension Parish were used in the prediction experiments. The study demonstrated the superior performance of neural networks against regression models in terms of the $R^2$ metric. The study recommended considering several explanatory variables in the analysis in order to improve the prediction accuracy. Accurate prediction of sign retro-reflection is vital for the optimization of maintenance programs and sign a replacement. Babic’ et al. used logistic regression to predict sign retro-reflectivity using data collected for 21,467 traffic signs along 30 state roads in the Republic of Croatia [14]. Three types of retro-reflective sheeting were considered. The authors developed a binary logistic regression model to evaluate whether or not signs meet the prescribed retro-reflection values. The proposed model could accurately predict the retro-reflectivity values for signs meeting the minimum retro-reflection standard. However, prediction results for signs not meeting the minimal prescribed retro-reflectivity values were not satisfactory. In an attempt to model the degradation of retro0reflectivity over time, Rasdorf et al. experimented with linear, logarithmic, polynomial, power, and exponential models [15]. They found that the linear model was the best model to predict the expected retro-reflectivity and they had unique model for each sheeting and color combination. However, the strength of the relationship between age (the only feature used) and retro-reflectivity ($R^2$) was weak (less than 0.5) which indicate that their models was achieving poor accuracy. Brimley and Carlson conducted a detailed review of long-term sign deterioration studies carried out by state agencies in the US [16]. It was reported that, out of all the predictors, age had the most significant contribution to deterioration of sign sheeting. Similarly, it was also found that signs facing South deteriorate at a much faster rate than any other direction.

Carlson and Hawkins developed a photometric model to establish retro-reflectivity for overhead guide signs using minimum luminance required for sign legibility [17]. The elderly population was specifically emphasized in developing the retro-reflectivity limits. After establishing minimum retro-reflectivity limits, the model was used to investigate the effect of factors such as distance to sign, headlight intensity, and vehicle speed on sign retro-reflection. Minimum retro-reflectivity limits were recommended for overhead guide signs, overhead mast-arm mounted, and post mounted street name signs for different sign sheeting and color. The minimum retro-reflectivity values recommended by this study were developed at an entrance angle of −4.0 degrees and observation angle of 0.2 degrees and are thus not applicable to other measurement conditions. Furthermore, the study considered only two sheeting colors (white and green), which reduces its applicability to other regions having different sign sheeting types and colors. Another study investigated the influence of frost and dew on traffic sign retro-reflectivity and noted an average decrease of 79% and 60%, respectively, in the retro-reflectivity values [18]. Engineering grade (Type-I) sheeting was observed to be severely affected as compared to high intensity (Type-III) sheeting. This study collected sign retro-reflectivity data for only 130 in-service signs during a period of one year (2001–2002). Consideration of a greater sample size over extended service lives may lead to more useful conclusions for practitioners. Furthermore, sign readings were obtained at only one observation angle (0.2°) and entrance angle (0.4°). Khrapova examined the effect of factors such as dirt deposit, drizzle, precipitation, and dew on sign surfaces, which may cause a reduction in sign retro-reflectivity [19]. Study results revealed that precipitation on sign surfaces significantly impair sign retro-reflective properties, decreasing it by over 76% in most cases. The presence of drizzle and dew were also noted to degrade sign retro-reflection. Similarly, it was noted that the presence of dirt deposits on sign surfaces also had statistical evidence of deteriorating retro-reflectivity. Appropriate sign cleaning resulted in over 60% improvement in retro-reflection levels. Saleh and Fleyeh conducted a comprehensive review of factors influencing the night-time visibility of retro-reflective traffic signs [20]. The study reported that predominant factors include vehicle headlight characteristics such as height and color, weather and ambient conditions, angle of illumination, and type of retro-reflective material used.

Khalilikhah and Heaslip utilized Random Forest (RF) to predict traffic sign vandalism as the function of several predictor variables such as mount height, size, background color, exposure, land cover, and road type [21]. Chi-square test results showed that there was a significant association between all the sign attributes and vandalism. The authors also provided a ranking of predictor variables on the rate of traffic sign vandalism. A study conducted by Khrapova et al. (2020) presented a detailed survey on the utilization of sophisticated vehicle-camera systems for the recognition of retro-reflective traffic signs using the Czech Republic as a case study [22]. Field signs’ retro measurements were taken by a retro-reflectometer followed by doing the same using modern camera systems. The study identified a number of essential parameters/traffic sign characteristics that significantly impact the performance of traffic sign detection and recognition (TSDR). The fundamental concept of sign retro-reflectivity studies is analogous to pavement marking retro studies. In a recent study, Ho et al. (2021) proposed a scheme to identify significant controllable factors and covariates affecting the retro-reflectivity performance of pavement markings from the perspective of highway safety [23]. The fitted naive-mixed linear model results revealed that any of the following covariates might be used: equivalent single axle load (ESAL), equivalent axle load, or traffic volume.

Pike et al. conducted a study for determining the expected service life of a traffic sign with respect to retro-reflection [24]. The authors proposed different linear regression models for each combination of sheeting class and color. Their model achieved poor results, with $R^2$ being 0.4436. Babic’ et al. predicted traffic signs’ functional service life using different regression-based modeling approaches such as linear, logarithmic, and exponential [1]. The proposed methods showed better prediction accuracy compared to previous studies with a coefficient of determination ($R^2$) value of over 0.5 for all the models. The authors argued that predicting traffic sign degradation based on their reference models can be very useful to practitioners for planning and optimization of the maintenance system. Although $R_A$ prediction performance expressed in terms of $R^2$ were good, low sample size, particularly, for sheeting type-III was the main issue. Furthermore, the authors compared the minimum $R_A$ values for new signs and not with in-field service signs, which is more appropriate. Nathan et al. used linear and non-linear regression models to predict traffic signs’ expected life for 1600 signs surveyed throughout the state of North Carolina in the US [25]. The results of the developed regression models suggested that the average sign replacement interval should be extended to 12 years. Variables such as sign age, amount of shade, and the type of retro-reflective material had significant correlation to the retro-reflectivity degradation of traffic signs.

Rasdorf and Machado developed a microscopic simulation model for examining sign replacement strategies using North Carolina (NC) sign data [26]. The authors argued that establishing a reasonable trade-off between sign-cost and conditions is vital for sign replacement practices. The proposed model could successfully simulate sign damage, grace period, blanket replacement, spot replacement (initiated beyond regular inspection), daytime inspections, and retro-reflectivity degradation. Results showed that the grace period was effective in reducing sign maintenance and replacement costs. Karimzadeh and Shoghli (2020) presented a detailed literature review of different predictive analytics used for highway maintenance activities as life expectancy and deterioration models, pavement marking prediction models, and sign retro and degradation models [27]. The survey of the included studies indicated that the deterministic models have mainly dominated the field of traffic sign deterioration, their life expectancy, and retro-reflectivity prediction. In another study, Machado and Rasdorf evaluated the performance of different sign management practices considering data for three case studies in the US: South Carolina, North Carolina, and Virginia [28]. It was concluded that both blanket replacement and expected sign life significantly reduce the need for the day and night time inspections, with North Carolina having the most mature sign management program. More recently, a study conducted by Hawkins (2021) emphasized the proper selection of application of retro sign sheeting for the specific application [29]. This report thoroughly summarized the key issues involved in the process of decision-making from different aspects, such as retro-reflectivity science, retro-reflectivity materials, application of fluorescent materials, minimum retro-reflectivity standards and specifications, sign performance and degradation, field measurement of sign retro-reflectivity, safety analysis, and economic benefits. The study reported that the roadway environment should be one of the key factors in selecting the sign sheeting. The recommendation concluded that sheeting with the highest grades should be deployed in urban areas since there is a significant demand for drivers’ attention, particularly during the night.

It is evident from previous works that mostly simple regression models were explored to investigate traffic sign degradation. Advanced Machine techniques have not been explored enough in the domain of sign retro-reflectivity. In addition, previous studies have separate models for each of the different combinations of sheeting and colors. To the best of our knowledge, no study has introduced a unified and robust model incorporating different sheeting characteristics, aging conditions, and other predictor variables. This study was undertaken to address this research gap.

3. Study Area and Data Collection

Field survey for sign retro-reflectivity data collection was conducted along selected segments of national motorways M-1 and M-2 near Islamabad–the capital city of Pakistan. The main purpose of selecting this section was to maintain similar ambient weather conditions. However, traffic volume for both the motorways were different as was evident from the amount of dust deposited on the surface of the signs. The mean summer temperature in the study area is approximately 41 C, with an average rainfall of around 90 mm while the average winter temperature is around 12 C. Figure 1 shows the study area in its entirety.

Sign retro-reflectivity measurements were obtained using handheld calibrated Retrosign retro-reflectivemeter, also known as GR3. The instrument enables single-handed operation and has single-touch controls, which allows the operator to work easily with the device. The instrument works on the principle of light retro-reflection. When the equipment is placed in contact with the sign surface, a beam of light from an internal light source falls on the sign surface and reflects back and is recorded. The retro-reflectivity value ($R_A$) is then automatically displayed on a digital screen by computing the ratio of light reflecting back to the original intensity of light hitting the surface. Typically, $R_A$ value is expressed in units of candelas per lux per meter squared (cd/lx/m²). Before the start of the experiment, the instrument was calibrated using the provided calibration kit. The detailed procedure for survey field traffic signs using GR3 can be found in internationally recognized standards such as ASTM E 1709, ASTM E 2540, AASHTO M 268, and EN 12899. At the start of the field measurements, sign surfaces were cleaned with detergents and cloth with the aim of assessing the impact of dust. However, multiple observations showed that sign cleaning improves the retro-reflectivity values only marginally (<10%). So, the study continued with uncleaned surfaces, which were later-on used for the experiments.

$R_A$ values were taken at observation angles of 0.2°, 0.5°, and 1.0° and an entrance angle of −4°, simultaneously. The observation is the vertical angle formed between the light ray originating from the vehicle headlight and the reflected ray towards the driver. It is basically a function of vehicle headlight height with respect to the target sign. An observation angle of 0.2° simulated the driving condition of small vehicles (such as passenger cars), and an angle of 1.0° replicates the night-time travel conditions for large vehicles (such as trucks). The entrance angle designates the angle between the light beam falling on the sign surface at any arbitrary point and hypothetical axis perpendicular to the sign surface at the same point. The entrance angle increases with the decreasing distance of the vehicle with reference to the target sign. To observe the effect of sign sheeting placement on sign visibility performance, R_A reading were taken at various instrument rotations/orientations, i.e., 0, 45, 90, 135, and 180 degrees. A minimum of four replicate readings were obtained for each sheeting type (with the same brand, grade, age, and color), when available. $R_A$ readings were also obtained in 4 age groups, i.e., 0, 2, 5, and 10 years for signs installed in the field. An aging period of zero refers to the readings taken from fresh samples sheeting temporarily attached (for seven days) to field signs with due permission from the national highway authority (NHA). Field retro-reflectivity data for signs surveyed was then transferred to a spreadsheet. Data with incomplete and erroneous duplicate records was removed to make it ready for subsequent analysis. Tentative dates for the exact installation year of different signs along the selected road sections in the study were also obtained from NHA.

Table 1. Summary of sign sheeting surveyed during the field study.

Feature	Range	Values
Brand	3	3M, Avery Dennison, TongMing
Sheeting Type	3	Diamond, Engineering Grade, HIP
Color	4	Blue, Green, White, Yellow
Orientation	5	0°, 45°, 90°, 135°, 180°
Observation Angle	3	0.2°, 0.5°, 1.0°
Age (years)	4	0, 2, 5, 10

shows the detailed attributes of sign sheeting surveyed during the field study. A total of 426 signs were used in the current study.

Table 2. Distribution of surveyed sign samples by age, type, and color *.

Age Category (Years)	Engineering Grade					HIP					Diamond Grade					Total
	W	G	Y	B	R	W	G	Y	B	R	W	G	Y	B	R
0	6	6	6	6	6	6	6	6	6	6	6	6	6	6	6	90
2	6	10	8	7	-	12	14	9	8	2	8	10	8	10	3	115
5	7	9	8	12	1	15	11	9	9	4	10	10	9	7	2	123
10	8	7	7	9	-	10	13	8	6	1	8	6	8	7	-	98
Total	27	32	29	34	7	43	44	32	29	13	32	32	31	30	11	426

* Letters W, G, Y, B, and R denote the colors: white, green, yellow, blue, and red, respectively.

shows the distribution of sign sheeting samples among various age categories.

Table 3. Minimum $R_A$ values for new sheeting type classification as recommended by ASTM D4956.

ASTM Sheeting Type	R${}_{\mathit{A}}$ for Various Sheeting Color (cd/lx/m2)
	White	Yellow	Green	Blue
Engineering Grade	70	50	9	4
High Intensity Prismatic	250	170	45	20
Diamond grade	380	285	58	26

provides the minimum R${}_A$ values for new sheeting type classification as recommended by ASTM–D4956.

4. Methodology

In this section, we first present the methodology and the detailed configuration of the experimental design for the proposed methods. Then, we present the evaluation metrics utilized to assess the adequacy of the adopted methods.

4.1. Retro-Reflection Degradation Prediction Models

We implemented a deep neural network model to perform the prediction of retro-reflection of traffic signs given its age and other features related to the sign, such as sheeting type, color, and observation angle. Neural networks were chosen because of their superior ability to capture complex relationships between predictors and the prediction value. In a simple feed-forward deep learning model, a series of non-linear transformation (layer) is applied to the input X. Each intermediate layer will take the output of the previous layer as the input. The last layer will output the $R_A$ prediction. A simple feed-forward layer contains a number of neurons and each neuron is nothing but a linear transformation $Z=WX+b$ and an activation function $f\left(Z\right)$, where W and b are the weights and the bias, respectively. The activation function is used to perform non-linear transformation on the data. The goal of training a deep learning model is to find the optimal W and B for each neuron that minimizes the error on the validation dataset. To find the optimal network layouts (number of layers, neurons per layers), we utilized a grid search technique with 5-fold cross-validation on the combined set of training and validation data. We first experimented with a different number of layers and the number of neurons in each layer (with ReLU activation in the hidden layers). Afterwards, we found the optimal number of layers and neurons, and experimented with the different activation functions, dropout, and batch normalization [30]. The final architecture of the model is illustrated in Figure 2 including the details on the number of parameters in each layer of the model and the total number of parameters in the model.

We trained our model to be able to predict the retro-reflection value for a traffic sign, where the model will be able to cover all the possible sheeting types, brands of sheeting, color, orientation, observation angle, and the sign ages available in the dataset.

The dataset utilized in this study contains both categorical variables, namely the brand, sheeting type and color, and continuous variables such as observation angle, orientation angle, and sign age. While formulating the experimental design, we divided the available data into three sets: training, validation, and testing. The splitting was done based on the combination of brand and sheeting type. This was done to ensure that the model could be evaluated on truly unseen traffic sign. The data was approximately divided into 55% training, 22% validation and the remaining 22% for testing. Since it is usually challenging to incorporate categorical data to network models, we utilized two different feature engineering techniques (i.e., one-hot encoding and entity embedding) to improve the predictive performance of the proposed methods.

In one-hot encoding, for each categorical feature we create a vector with length equal to the number of possible categories in that feature. The vector consists of all zeros except for one index, which is set to 1 corresponding to a unique category in that feature. Since we have five categorical features: 3 brands, 3 sheeting types, 4 colors, 5 orientation and 3 observation angles, applying one-hot encoding expanded our feature space from 6 to 19 dimensions.

While one-hot encoding can convert categorical features to numbers, it fails to capture the relationship between the different categories since each category will be either 0 or 1. The possible relationship that can be captured between category, for example, is how sheeting type A can degrade to a rate closer to B while degrading at double the rate of C. In addition, one-hot encoding will expand our feature space to the number of different categories in each of our features, which can slow the training process and consumes more memory. Entity embedding is an approach that maps each category in a feature to a vector in the Euclidean space [31]. This mapping can be fixed or be learned during the model training process. Using this approach, the embedding will be able to learn some relationship between each category by their distances in the embedding space. Embedding is represented as $n\times m$ weight matrix, where n is the number of unique categories and m is the embedding dimension.

For this method, we experimented with two implementations. For the first approach, we gave each categorical feature its own embedding space. For the second approach, we created one embedding space for all the categorical features. By sharing the embedding space, we hoped that we would be able to capture the relationship between the different features such as how each sheeting type has different retro-reflection performances for each observation angle. For the feature age, we applied min-max normalization, where we divided each age by 10. While for the retro-reflection values we applied standard scaling according to the following equation:

$$x=\frac{x-mean}{standarddeviation}$$

(1)

Linear and Polynomial Regression Models

Additionally, we also implemented linear and polynomial regression models to compare with the neural network system since most of the previous studies used these models to predict traffic sign retro-reflection degradation. In linear regression, the target value—retro-reflectivity—is assumed to be a linear combination of the predictors. Mathematically:

$$\widehat{y}=w_0+w_1{x}_1+\dots +w_d{x}_d$$

(2)

where, $\widehat{y}$ is the predicted retro-reflectivity, $x=(x_1,x_2,\dots ,x_d)$ are the predictors such as age and sheet color, $w=(w_1,w_2,\dots ,w_d)$ are the coefficients, and $w_0$ is the intercept, also known as bias.

The goal is to find the coefficients $w_i$’s and the bias b that minimizes the residual sum of squares between the actual and the predicted retro-reflectivity values for the samples in the training set. We trained our linear regression model using stochastic gradient descent in order to find the optimal weights and bias. As for the polynomial regression model, we generate a expanded feature set $\widehat{x}$ from x that contains all the polynomial combinations of the features in x with degree 2, in addition to the original features, which is then fed to the regression model.

4.2. Experiment Execution

The implementation of the linear regression is provided in Python’s scikit-learn library. We used SGDRegressor class to implement both the linear and the polynomial models. To convert the data to a set of polynomial combinations, we used the PolynomialFeatures class. For the neural networks we used TensorFlow and Keras. Additionally for the neural networks, we experimented with different optimizers and loss functions with different hyper-parameters using grid search with 5-fold cross-validation. We found that using the mean squared error as a loss function and ADAM optimization with the learning rate = 0.005, and the rest of the parameters being the default values, achieved the best performance [32]. We also incorporated the learning rate reduction strategy, which reduces the learning rate by a factor of 0.2 if the validation loss is not improving. This technique was implemented using the ReduceLROnPlateau class in Keras, and it stops decreasing the learning rate once it hits a threshold of 0.00001. Similarly, for the linear and polynomial model, we applied a grid search with 5-fold cross-validation and found that using the adaptive learning rate (reduce the learning rate when the validation loss is not improving) with the initial learning rate set to 0.01 performed the best. Once we got all the hyper-parameters, we trained the models on the training set until it converged on the validation set. The training was done on a local computer with an Intel 4970k CPU and Nvidia GTX 970 GPU (the code for our experiments can be found at: https://github.com/aalkhulaifi605/Predicting-Traffic-Sign-Retroreflectivity-Degradation-Using-Deep-Neural-Networks; access on: 22 November 2021).

4.3. Model Evaluation

We compared the results using two metrics: mean absolute error and the coefficient of determination ($R^2$). The mean absolute error is the mean of the sum of all the differences between the predicted and the actual value, and it is the most widely used metric for regression problems. MAE is defined by mathematical expression shown below:

$$MAE(y,\widehat{y})=\frac{{\sum }_{i=0}^n|y_i-{\widehat{y}}_i|}{n}$$

(3)

where, $y_i$ is the actual value for sample i, ${\widehat{y}}_i$ is the predicted value for sample i, n is the total number of samples.

The coefficient of determination ($R^2$) is the square of the correlation between the predicted and the actual value. The reason for selecting this metric was due to it being used by most of the previous studies in the field of retro-reflectivity degradation prediction. The coefficient of determination ($R^2$) is defined as:

$$\begin{array}{r}\begin{array}{c}\hfill R^2(y,\widehat{y})=1-\frac{{\sum }_{i=0}^n{(y_i-{\widehat{y}}_i)}^2}{{\sum }_{i=0}^n{(y_i-{\overline{y}}_i)}^2}\end{array}\end{array}$$

(4)

$$\begin{array}{r}\begin{array}{c}\hfill where\overline{y}=\frac{{\sum }_{i=0}^n{y}_i}{n}\end{array}\end{array}$$

(5)

5. Results and Discussions

As different works use different datasets, a fair and objective comparison is not possible. Looking at the experiment settings in the literature, we noticed that the previous studies do not use the brands or the manufacturer information as predictors when fitting their regression models. Moreover, we wanted to evaluate the deep learning model’s performance against unseen pairs of sheeting types and brands to increase the difficulty of the prediction task and further assess its generalization capabilities on unseen data. Accordingly, we conducted our experiments on the data after it had been divided into training, validation, and testing sets based on the combination of brand and sheeting type such that we see samples of brand and sheeting type combinations in the test set which are not seen during the training. In addition, we excluded the brand name from the list of predictors. In the current setup, the validation and test sets only include combinations of brand and sheeting type that are missing in the training set. All our results are reported as the mean and standard deviation of total of 10 trails.

The prediction results of our models using all the features can be seen in

Table 4. Model performance on the dataset using all features. The best performing models are in bold.

Model	Test Set		Train Set		Validation Set
	R2	MAE	R2	MAE	R2	MAE
Linear	0.426 ± 0.009	218.873 ± 3.983	0.475 ± 0.001	179.068 ± 1.233	0.551 ± 0.003	185.022 ± 0.471
Polynomial	0.759 ± 0.007	133.855 ± 1.73	0.783 ± 0.009	109.492 ± 1.087	0.776 ± 0.004	117.316 ± 1.913
NN one-hot encode	0.976 ± 0.008	34.809 ± 5.090	0.937 ± 0.007	45.900 ± 2.093	0.842 ± 0.011	106.836 ± 2.473
NN entity embedding	0.156 ± 0.533	142.562 ± 67.674	0.136 ± 0.524	146.049 ± 64.579	0.112 ± 0.472	189.063 ± 51.190
NN shared-entity embedding	0.967 ± 0.015	36.425 ± 6.324	0.941 ± 0.008	44.165 ± 2.726	0.823 ± 0.012	109.403 ± 1.746

. The best results are shown in bold. We noticed that by including orientation, our results improve in our neural network models, while for the polynomial model the improvement is very minimum. All the variations of the neural network models outperform the linear and polynomial models by a significant margin. The linear model achieved $R^2$ value lower to the results reported by [1], which can be attributed to the fact that we trained one model for all the combinations of features instead of training multiple models. In comparison, the polynomial model seems to outperform the linear model by a wide margin. The neural network with one-hot encoding achieved the best results with a mean MAE of 34.809. The shared embedding space performed well, achieving an MAE of 36.425. The separate embedding space achieved the second lowest score at mean MAE of 142.562. We believe that due to the limited number of categories in each feature, the embedding space was too small for the model to capture the relationship between the categories. In

Table 5. Samples from the test sets and predicted $R_A$ by our proposed NN with one-hot encoding.

Sheeting Type	Color	Orientation Degrees	Observation Angle	Age (Years)	True ${\mathit{R}}_{\mathit{A}}$	Predicted ${\mathit{R}}_{\mathit{A}}$
Engineering Grade	Blue	90	0.2	0	44	42.81
Engineering Grade	Blue	90	0.2	2	37	39.25
Engineering Grade	Blue	0	0.5	10	5	2.08
Engineering Grade	Green	0	0.2	5	65	55.02
Diamond	Blue	135	0.2	0	176	201.84
Diamond	Yellow	45	1	2	86	96.82
Diamond	Blue	180	1	10	5	8.96
Diamond	White	45	1	5	92	91.76
Diamond	Green	0	1	0	22	20.89
Diamond	Green	180	0.5	10	30	54.29

we present samples of traffic signs extracted from our test set with the predicted $R_A$ by the neural network with one-hot encoding. Moreover, we plotted the training and validation loss history of the NN with one-hot encoding in Figure 3.

5.1. Sensitivity Analysis for Predictor Variables

To further explore the effect of each feature, we ran the the experiment again on the same dataset but opted out one feature from the set.

Table 6. Model performance without using the orientation feature. The best performing models are in bold.

Model	Test Set		Train Set		Validation Set
	R2	MAE	R2	MAE	R2	MAE
Linear	0.415 ± 0.006	221.568 ± 2.409	0.471 ± 0.001	179.954 ± 0.770	0.541 ± 0.004	185.272 ± 0.566
Polynomial	0.750 ± 0.012	132.970 ± 3.780	0.771 ± 0.008	109.279 ± 1.575	0.762 ± 0.006	119.627 ± 1.675
NN one-hot encode	0.943 ± 0.010	45.842 ± 1.736	0.922 ± 0.006	49.533 ± 1.500	0.806 ± 0.010	113.542 ± 1.827
NN entity embedding	0.830 ± 0.328	58.192 ± 37.122	0.814 ± 0.326	62.361 ± 35.680	0.704 ± 0.287	125.268 ± 31.858
NN shared-entity embedding	0.947 ± 0.009	43.499 ± 3.342	0.920 ± 0.005	49.902 ± 2.041	0.802 ± 0.010	114.515 ± 3.831

displays the performance of our models without using the orientation angle as input feature. We noticed that the performance of our neural network with separated entity embedding has improved significantly, which indicates that the model separated entity embedding model is sensitive to features. Furthermore, we noticed that the performances of the linear and polynomial models are slightly lower to the models using the orientation angle. Similarly, our neural network with one-hot encoding and shared entity embedding models saw a small drop in $R^2$ in the test set.

For the other features, we showcase the results of all our models without using the sheeting type to predict the $R_A$ In

Table 7. Model performance without using the sheeting-grade as feature.The best performing models are in bold.

Model	Test Set		Train Set		Validation Set
	R2	MAE	R2	MAE	R2	MAE
Linear	0.458 ± 0.002	158.070 ± 0.483	0.347 ± 0.001	187.760 ± 0.526	0.473 ± 0.003	206.605 ± 0.453
Polynomial	0.662 ± 0.00	104.527 ± 2.81	0.484 ± 0.005	148.619 ± 1.816	0.641 ± 0.008	164.317 ± 2.408
NN one-hot encode	0.696 ± 0.189	89.238 ± 20.076	0.493 ± 0.121	131.022 ± 6.356	0.664 ± 0.155	145.569 ± 17.192
NN entity embedding	0.674 ± 0.240	91.204 ± 23.455	0.475 ± 0.173	132.115 ± 10.051	0.640 ± 0.222	148.903 ± 24.300
NN shared-entity embedding	0.754 ± 0.013	83.534 ± 1.776	0.533 ± 0.001	128.681 ± 0.602	0.716 ± 0.002	139.721 ± 1.319

; results are shown without considering sheeting grades. In

Table 8. Model performance without using sheeting-color as the feature.

Model	Test Set		Train Set		Validation Set
	R2	MAE	R2	MAE	R2	MAE
Linear	0.130 ± 0.004	276.517 ± 1.175	0.337 ± 0.000	193.298 ± 0.415	0.452 ± 0.002	192.773 ± 0.363
Polynomial	0.153 ± 0.008	234.420 ± 1.120	0.477 ± 0.003	153.787 ± 0.451	0.573 ± 0.002	168.896 ± 0.568
NN one-hot encode	0.116 ± 0.048	207.708 ± 2.139	0.507 ± 0.003	137.832 ± 0.564	0.597 ± 0.008	164.044 ± 2.011
NN entity embedding	0.064 ± 0.050	193.421 ± 20.986	0.304 ± 0.251	150.266 ± 15.014	0.362 ± 0.291	189.160 ± 31.557
NN shared-entity embedding	0.100 ± 0.036	197.911 ± 20.700	0.416 ± 0.184	141.451 ± 8.116	0.493 ± 0.215	174.087 ± 21.275

, the model’s prediction results are shown without using the sheeting color, while in

Table 9. Model performance without using observation-angle as the feature. The best performing models are in bold.

Model	Test Set		Train Set		Validation Set
	R2	MAE	R2	MAE	R2	MAE
Linear	0.273 ± 0.004	232.371 ± 1.673	0.334 ± 0.001	186.637 ± 0.578	0.274 ± 0.005	230.905 ± 1.712
Polynomial	0.391 ± 0.023	189.451 ± 7.526	0.411 ± 0.017	165.034 ± 4.728	0.299 ± 0.013	221.326 ± 3.460
NN one-hot encode	0.470 ± 0.113	143.518 ± 20.682	0.437 ± 0.123	139.916 ± 24.536	0.302 ± 0.073	213.667 ± 17.343
NN entity embedding	0.365 ± 0.240	147.358 ± 17.318	0.348 ± 0.237	142.102 ± 20.553	0.229 ± 0.164	215.156 ± 13.779
NN shared-entity embedding	0.524 ± 0.005	135.174 ± 2.860	0.503 ± 0.002	129.420 ± 2.366	0.331 ± 0.007	208.217 ± 2.631

the same are shown excluding the observation angle from the analysis. All our neural network models and the polynomial model show a significant increase in error while the linear model showed an increase by a relatively small amount. This highlights that there exists an important relationship between the sheeting type, color, and observation angle with the degradation rate of the $R_A$ value that the neural network models and the polynomial model can capture while the linear model fails to effectively capture and utilize those relationships.

The 3M brand has the highest $R_A$ values, keeping all other parameters constant such as grade, color, orientation, observation angle, and age. It can be argued that this sheeting type is made up of a high-quality beaded glass structure that reflects the light back more efficiently. Comparing the grade, the diamond grade has the highest $R_A$ values while the engineering grade possesses the lowest values. Diamond sheeting has high quality prismatic and cube structures capable of reflecting a significant proportion of incident light. Furthermore, $R_A$ has a high value at an observation angle of 0.2-degrees, primarily due to the narrow cone for light traversing and reflecting without much loss in incident beam intensity. The higher $R_A$ value of yellow and white color is intuitive and may be attributed to their effective retro-reflection properties compared to dark blue and green, which absorb more light instead of reflecting it back. Similarly, it can be argued that a sheeting orientation of 0 and 180 degrees is more effective because more embedded cones in the sheeting structure responsible for light reflection are aligned during such orientations.

5.2. Comparison of Predicted Versus Target $R_A$

In Figure 4, we visualize the scatter line plot for actual versus predicted $R_A$ values with the x-axis denoting the number of observations (for randomly selected 100 instances) and the y-axis representing the $R_A$. Figure 5 shows the regression plot between predicted and target (actual) retro-reflectivity values using the prediction results from the best model. A regression trend line indicating the prediction values is shown by a solid line in the figure. The values of the $R^2$ for this best fit regression line is given in Table 4. It can be seen that our prediction is correlated with the base which explains the high $R^2$ score of the adopted models. It may be noted from the figure that the predictive performance of a plotted one-hot encoding model is good at low actual $R_A$ input values. Although, very few points representing the relatively high values for $R_A$ are more scattered/dispersed around the predicted line values, the overall prediction of the performance is adequate.

5.3. Permutance-Based Feature Importance Analysis for $R_A$

In literature, studies have used only a few variables to assess retro-reflectivity degradation of traffic signs deployed along highways. An accurate prediction of traffic sign retro-reflectivity for different sign sheeting under various circumstances and field conditions is very important. It can give clear indication of effective sign management practices for the respective signs. Additionally, it is established that retro-reflectivity performance for different sheeting is entirely different. It further depends on the height of the vehicle headlight (which is simulated by the observation angle for GR3), sheeting directions, and temperature, precipitation, and snow condition in an area. It is, therefore, imperative to establish individual variable importance for retro-reflectivity output. For our study, we had a dataset containing field retro-reflectivity measurement for different sheeting brands, three sheeting grades, different sheeting orientation, instrument observation angles, and sign age. We designed a permutation-based feature importance test on the data to evaluate the effect of each feature on the retro-reflection value. In this test, we measured the reduction in the performance of the model when a single feature is shuffled randomly. To perform this test, we divided our data into training and testing and fitted the training data on a random forest regression model and evaluated the permutation feature importance with the test data. The test was performed in several steps, i.e., permute a single feature, perform prediction, and calculate the importance score based on the difference in error between the original data and the permuted data. The process was applied 10 times, and each time the shuffling was performed with different seeds. The results for the feature importance tests are shown in Figure 6.

It may be noted from the figure that the parameter orientation has a score close to zero, which confirms the previous studies, suggesting that it has a very weak correlation with the degradation of retro-reflection. It can also be observed from this figure that sheeting color and observation angle have a higher score, indicating that they have a significant bearing on the ultimate retro-reflection values. The brand of sheeting also has a low score, which suggests that the difference in the performance of the same sheeting type and color between the different brands is small. As shown in the analysis above, we first run the experiment using all the features to identify the best model. This was followed by experimenting with the effect of each feature by removing it from the model. This will help to further evaluate the effect of different features on prediction performance since some features can be determined to be not important in one model but can be important in another model that is able to capture complex relationships.

5.4. Comparison with the State-of-the-Art on Retro-Reflectivity Prediction

In this section, we will present a comparison of our work with the state-of-the-art on retro-reflectivity prediction. In

Table 10. Model performance with reference to previous studies.

Study	Dataset Size	Model	${\mathit{R}}^2$
Black et al. [6]	5722	Linear model	0.500
Swargam [12]	1107	Neural networks	0.81
Rasdorf et al. [15]	1047	Linear model	0.481
Immaneni et al. [9]	6000	Linear model	0.520
Pike et al. [24]	525	Linear model	0.444
Babić et al. [1]	487	Linear model	0.570
Huynh et al. [25]	1600	Linear and non-linear models	0.67
Present Study	426	Deep neural networks	0.976

, we present our best performing model and compare it with the results from other published works covered in this paper. Our neural network model with entity embedding outperforms other approaches significantly in terms of the $R^2$ score. This illustrates that the neural network based system is capable of learning the degradation of retro-reflection effectively for traffic signs with different feature combinations. It should be noted that an objective comparison is not possible due to the use of different datasets by different published works including the use of different predictor variables. Accordingly, the comparison should be treated as a qualitative one in order to understand the effectiveness of different approaches.

5.5. Predicting Useful Life of Traffic Signs Based on the Degradation Model

It is understood that using the system for predicting the useful life of traffic signs is a more practical use-case for the end users and practitioners. Accordingly, in this section we present an approach to predicting the useful life of traffic sign using the trained machine learning system. In order to achieve this objective, we predict the retro-reflectivity values of sign boards at different ages and look at the age that predicts the minimum acceptable retro-reflectivity value. This age is accepted to be the useful life of that traffic sign. In Figure 7a–c, we illustrate the idea with the help of plots from three different traffic sign instances from the test set, each with a different color. We plot the predicted retro-reflectivity values at various ages denoted by the green color. In addition, the actual values are also plotted using the violet color. We can observe from the plots that the predicted values from the system are closely aligned to the actual values, which demonstrates the effectiveness of the system in learning the retro-reflectivity degradation function. Furthermore, looking at the age where the traffic signs reach the minimum retro-reflectivity values, we can see that the system predicts useful life of 17.36, 14.58, and 15.06 years for the traffic sign represented by the plots (a), (b), and (c), respectively. These values are quite reasonable if we compare these values to the recommended useful life of traffic signs (cf. [33]). It should be noted that we use the minimum retro-reflectivity value for white = 30 (cd/lx/m²), green = 7 (cd/lx/m²), and yellow = 15 (cd/lx/m²) as suggested in literature (cf. [34,35]). We also notice from the plots that the learned function is not entirely linear across all the age spectrum. For higher retro-reflectivity values, the degradation seems to be close to a linear function but later the rate of degradation slows down.

6. Conclusions

Traffic signs play a vital role in the safe movement of vehicles. Signs must be clearly visible to drivers to serve their intended purposes. Sign retro-reflectivity gradually deteriorates over time, making them less visible, particularly at night. They need appropriate treatment or warrant timely replacement before the retro-reflectivity index ($R_A$) reaches the minimum acceptable limits. Therefore, it is vital to accurately predict sign retro-reflectivity for different sign sheeting under various circumstances, weather, and aging conditions. Previous studies in this regard have mostly focused on simple regression-based statistical modeling. However, these models have several unrealistic assumptions that limit their applications for large scale real-world applications. Further, they also suffer from weak prediction accuracy. We discussed some notable papers in this area and highlighted the shortcomings, including the requirement to have a separate model for each combination of sheeting type and color. In order to fill this research gap through this study, we aimed to explore the problem of modeling the degradation of retro-reflection for field traffic signs using neural network based deep learning systems. A handheld Retrosign GR3 retro-reflectometer was used to collect the field signs’ readings. Field data was collected on different sign attributes such as sheeting brand, grade/type, color, sign different orientation, observation angles, and sign aging. We implemented three systems with different feature engineering methods. Study analysis showed that a neural network with one-hot encoding achieved the best results with an MAE of 34.809 and an $R^2$ score of 0.976 on unseen test data, outperforming all previously proposed models for traffic signs retro-reflection prediction. Finally, we conducted a number of sensitivity analysis experiments to explore the role and importance of each feature in predicting the $R_A$ value for a given traffic sign. Variable sheeting color and observation angles were identified as the most significant predictor variables for $R_A$.

The findings of the current study highlighted how different neural-network architectures could capture complex relationships to effectively learn sign degradation patterns. Knowledge gained from this study could serve as essential guidance to transport agencies for effective sign management practices. In future studies, detailed datasets containing information on other explanatory attributes such as weather conditions, sun exposure, air pollution could be considered, which can enhance the prediction performance of models and make them applicable to different situations including usage in different regions and countries.