This section describes the way in which the final crash prediction models are developed. As noted above, six patch lengths, i.e., 1500 ft, 1000 ft, 400 ft, 200 ft, 100 ft, and 50 ft, were considered. The patch length denotes the way in which the 3D roadway surface is spatially divided. However, no matter the patch length, in each patch there are four explanatory variables linked to it, namely the number of crashes that occurred, AADT value, Gaussian curvature (GC), and mean curvature (MC). In addition to the initial values of these variables, transformations were also considered, e.g., AADT
2, GC
2 MC
3, in order to identify the optimal scale and combination of these variables. All of these transformations and the justification of the optimal scale are presented in Amiridis, K. 2019, Appendix F [
18]. Moreover, statistical interactions of the explanatory variables, e.g., Gaussian*mean, are also considered. Finally, it should be noted here that the statistical regression model that was utilized is the negative binomial regression because overdispersion is present in the data and because it was intended to keep the statistics relatively simple in order to retain the focus of the research on the use of 3D geometric explanatory variables in highway safety rather than the statistical methods utilized per se. After all, the typical regression model that is utilized for crash prediction modeling is indeed negative binomial regression.
The analysis conducted will serve a dual purpose: (1) demonstrate the proof-of-concept of the proposed 3D approach; (2) evaluate the predictive power of the model. These two objectives can be viewed as independent, i.e., failure in demonstrating the predictive power of the model does not mean that the proof-of-concept is violated. For example, 3D metrics may be proven to have a statistically significant effect in crash modeling, but the reason for potential failure in adequately predicting actual crashes may simply rest on the fact that more explanatory variables are required in the model. The proof-of-concept relies on the verification that the 3D differential geometry metrics of Gaussian and mean curvature are statistically significant crash predictors. This can be successfully demonstrated if it is proven that the coefficients of the metrics are indeed statistically significant. It should also be noted that depending on whether historical crashes are available, two strategies come into play in order to predict crashes in the most effective way.
3.2.1. Proof-of-Concept
In order to provide the proof-of-concept in the most concrete way, all years, i.e., 2004–2017, and all seven roadways entered the same model which will be called the “Integrated Model” (IM) to be distinguished from the models that will be developed for the second objective, i.e., prediction evaluation. The objective of this effort was to establish that it is meaningful to incorporate 3D highway geometry in crash prediction models. Although the predictive power of the model was not evaluated at this point, this step was crucial because failure to address the statistical significance of the 3D metrics in crash prediction would render any further discussion of prediction power evaluation meaningless. Moreover, the type of the final explanatory variables that will enter this model will function as the basis of the predictive power evaluation of the model. For example, if the variables, AADT, Gaussian2, and mean3 are proven to be the finalists, then these exact variables would be considered in order to evaluate the predictive power of the model; a logic that holds true in most predictive models. For example, even for the variables that come into play in the SPFs in the HSM with a specific transformation, e.g., exp(AADT), it does not mean that this particular transformation is optimal in all cases; it simply means that this transformation is on average adequate.
Although not explicitly stated, a part of the statistical analysis essentially touches the field of spatial statistics since the selection of an acceptable patch length is of the utmost importance because it functions as the basis of all further (traditional) statistical analysis. To proceed with this effort, two-stage simultaneous testing was undertaken that would define the optimal patch length and model to be used. First, for each patch length considered, models with the variables of interest were developed and the most appropriate was selected in terms of statistical significance and the Akaike Information Criterion (AIC) evaluation criterion. Second, these models were then compared to identify the most appropriate patch for analysis and the power of prediction evaluation. Since there are six patch lengths tested, six “final models” will eventually be compared to each other.
All of the combinations of the explanatory variables that were utilized until the final model was decided, for all patch length combinations, are detailed in Amiridis, K. 2019, Appendix G [
18]. The criterion according to which the models were compared was the AIC; the lower the AIC, the more informationally rich the model is. The AIC also functions as an adjusted R-square in the sense that it penalizes the number of variables that enter the model. Furthermore, in order for a model to be further considered as a finalist for additional evaluation, all of the coefficients of the explanatory variables that enter the model must be statistically significant, i.e.,
p-value < 0.05. The demand for a
p-value < 0.05 is associated with the fact that a significance level of 5% is considered; in fact, each
p-value, depending on the number of explanatory variables that enter the model, must be less than the predefined “familywise”
p-value, which in this case is set to 0.05, according to Bonferroni or any other type of correction (Myers et al. 2010) [
25]. Roughly speaking, this means that if two explanatory variables are considered then the
p-value of the coefficient of each variable must be less than approximately 0.025
) assuming that the two explanatory variables are independent in order for the “overall
p-value” to be less than 0.05.
More generally, the Bonferroni correction, or any other type of correction, should be applied when explanatory variables are simultaneously inserted into a statistical model. More specifically, the significance level has been assumed to be 5 percent, i.e., there is a 95 percent confidence that the true parameters belong in the constructed confidence interval. However, the significance level of 5% should not be applied to each coefficient, but to the model as a whole; therefore, in order for the significance level of the whole model to be kept at the 5 percent significance level, the p-value of each coefficient should be less than 5 percent. The value of each p-value in order to achieve a “familywise” error of 5 percent is imposed by the pertinent correction method used, e.g., Bonferroni or Tukey, and by the number of variables; the more variables, the stricter, i.e., lower, the p-value must be.
It should be noted here that AADT, GC and MC are to be used as explanatory variables, i.e., main effects, in the statistical analysis through a multivariate regression analysis. However, even when multiple explanatory variables are intended to enter the model, the analysis should always begin by visualizing the explanatory variables vs. the dependent variable. Negative binomial regression, which is the regression type that will be applied here, is a member of the family of Generalized Linear Models (GLMs). Each regression member of the GLMs is associated with a function that is called the “canonical link function”, which actually represents the optimal transformation that should be applied to the dependent variable in order to satisfy desirable statistical properties such as unbiased parameters (Hardin and Hilbe 2012) [
26]. In the case of the Poisson and negative binomial regression, the aforementioned function is indeed the “log-link function”. If logarithmic transformation is not applied to the dependent variable, then the so-called “identity link function” is applied, meaning that the dependent variable is simply the variable “crashes”. Results will be produced even if the log-link function is not applied but the reliability of the results is weakened because the log-link function is the “canonical” link function for negative binomial regression. This is why the Poisson and negative binomial regression models are also often called log-linear models, meaning that the explanatory variables have a linear relationship with the logarithm of the dependent variable. Therefore, in this case the dependent variable will be LN(crashes).
The typical visualization process in order to identify the optimal transformation of each explanatory variables is via scatterplots. The scatterplots for the explanatory variables AADT, GC, and MC are shown in
Figure 5,
Figure 6 and
Figure 7, respectively, for the 100 ft patch.
According to
Figure 5, AADT seems to have a rather linear relationship with LN(crashes), whereas Gaussian curvature (
Figure 6) seems to have a cubic relationship with LN(crashes), and mean curvature (
Figure 7) has an essentially quadratic relationship with LN(crashes). However, these observations hold true only when the explanatory variables are plotted one by one against the dependent variable; in other words, there is no guarantee that the nature of these relationships will remain the same when all of the explanatory variables enter the model. However, this procedure has revealed that, especially for the mean and Gaussian curvature, there is some indication that their relationship may not be linear in nature with LN(crashes) and therefore quadratic and cubic transformations may be appropriate for testing.
For each patch, 38 variable combinations were tested until the analysis was finalized. The models considered each variable alone and in a variety of combinations in order to determine the most appropriate and meaningful combination. All of these combinations for each patch length are presented in Amiridis, K. 2019, Appendix G [
18]. The process for determining whether a model was appropriate was based on an initial determination of whether all of the coefficients of the model were statistically significant and accounting for the Bonferroni correction. Then the statistically significant models were compared with the AIC criterion. It is noted that, as a rule of thumb, when two models are compared and their AIC difference is greater than 10, then this difference is “significant”, meaning that the model with the lowest AIC should be kept instead (Hardin and Hilbe 2012) [
26]. Finally, the assumptions according to which the model is based, e.g., normality of deviance residual distribution, must also be satisfied.
A summary of the variables used in the best models for each patch length are summarized in
Table 4.
Table 4 shows which explanatory variables were statistically significant without listing their corresponding coefficient values, as these may contain more than ten 10 significant digits, thereby compromising the readability of
Table 4; however, the full numerical values are available in Amiridis, K. 2019, Appendix G [
18]. The final suggested models as shown in
Table 4 indicate that the Gaussian curvature (GC) and mean curvature (MC) of 3D surfaces play a crucial role in crash prediction since they are statistically significant in all models in which the Bonferroni correction has also been accounted for. In fact, not only are the Gaussian and mean curvature statistically significant in all models, but their
p-values are also less than 0.001 in all models. The insertion of these two differential geometry metrics is actually the new aspect that this research introduces to the literature. The use of these metrics can be considered promising because the Gaussian and mean curvatures are the cornerstones of the study of 3D mathematical surfaces as a whole in differential geometry. Moreover, the fact that transformed geometric metrics, e.g., GC
3 and MC
2, and the two-way interaction term GC*MC are inserted into the model, in the 100 ft patch length model, emphasizes the complexity by which roadway geometry affects crash occurrence, a fact that cannot be revealed in such an explicit manner through conventional 2D geometric metrics. Finally, in terms of computational statistics stability, when a variable is entered into a model with a power, e.g., quadratic, it is beneficial if the “lower power terms” are also included in the model, e.g., linear, for computational reasons. Fortunately, as demonstrated in
Table 4, this is the case for both the GC
3 and MC
2 variables since the variables GC
2 and GC, as well as MC, are also included in the model with
p-values < 0.001, meaning that even the Bonferroni correction is amply satisfied.
The criterion used in order to select the most appropriate patch length was based on the overall error prediction which is estimated as the difference between the observed and model-predicted number of crashes. A summary of the predictive ability of each patch length, i.e., the associated error percentage to each, is shown in
Table 5; it is noted that 1534 crashes occurred during the 2004–2017 period. Although it may be considered adequate on a practical basis to conclude that the 100 ft patch is the most pertinent patch length for the analysis, an additional statistical metric will also be considered to further validate this assertion, for the comparison among the different patch lengths. The additional statistical measurement used is the Predicted Error Sum of Squares or the so-called PRESS (Caroni and Oikonomou 2017) [
27]. PRESS is used in order to compare regression models in terms of their ability to predict new values; the model preferred is the one with the smallest value of PRESS (
Table 5).
The selected patch length for the final model corresponds to a length of 100 ft because it was observed that this patch length provides the best modeling ability. Even though a smaller patch length leads to an increase in the predictive power of the model, this was true up to a “cut-off” patch length, which in this case was estimated to be 50 ft. In this case, “cut-off” indicates that after a certain point the overall error is not practically improved with reduction of the patch length.
The results of the model corresponding to the 50 ft patch were identical to the ones derived from the 100 ft patch (
Table 5). Moreover, a 100 ft patch may be considered more appropriate for transportation-related applications because vehicles that have a length over 50 ft such as combination trucks, recreational cars, and buses can be analyzed in a more reliable manner by incorporating a larger surrounding roadway geometry. Therefore, for transportation-related consistency and the practical effect of overall error reduction, as well as computational speed purposes, it was decided to utilize the 100 ft patch for the crash modeling process.
The final model corresponding to the 100 ft patch length is summarized in
Table 6, whereas the regression model is presented in Equation (1). The AIC for the models considered ranged from 11,183 to 11,803. The final model that was kept was indeed the one with the lowest AIC of 11,183 while the second-best model had an AIC of 11,232. It is noted that all of the explanatory variables of the final model have a
p-value < 0.001, a fact that essentially demonstrates the proof-of-concept of this research: 3D geometric roadway metrics can successfully function as explanatory variables in crash predictive models.
As noted above, the presence of the Gaussian curvature and mean curvature of 3D surfaces supports the significance of these variables as crash predictors and their potential interaction with other variables—interactions that can by no means be captured in the 2D analysis.
At this point, the “Integrated Model” in which all years and roadways are included has been finalized and presented in Equation (1) above. The IM essentially functions as a proof-of-concept for the inclusion of the 3D metrics in crash prediction models and can, at least theoretically, be used for crash prediction purposes in other roadways. This model may be particularly useful when the purpose of an analysis is not the prediction of crashes in absolute numbers, but the comparison of alternatives, e.g., different alignments, in terms of estimating which alternative reduces crash frequency. In addition, it is suggested that the specific coefficient values (
Table 6) be used for crash prediction purposes only when no historical crash data are available; if crash data are available for a specific roadway segment they should be certainly used in order to incorporate the “special crash pattern” in the adjusted model to be discussed in the next section. Finally, when several years of crash data are available, it is advised that, for crash prediction purposes, the years enter the model as dummy variables. The latter is suggested in order to account for seasonal and time effects. This is further discussed in the next section in which the predictive power of the model is evaluated.
The magnitude of regression coefficients reported in
Table 6 should be interpreted in the context of variable scaling and transformation. Several explanatory variables, including curvature-based indicators, are derived from squared or higher-order geometric terms and are characterized by small numerical ranges. When such variables are included in a log-link negative binomial regression framework, relatively large coefficient values may result without implying unreasonable effects or numerical instability. Accordingly, coefficient interpretation in this study focuses on statistical significance, direction of effect, and contribution to model fit, rather than absolute magnitude. All coefficients were verified and found to be consistent with the underlying data and model specification.
3.2.2. Model Structure and Predictive Ability Evaluation
The ultimate objective of this research is the determination of the predictive ability of the proposed model based on 3D metrics on safety predictions. The comparison is based on the crash predictions as estimated from the model and the IHSDM. The IHSDM predicts crashes per year for a given roadway through the Empirical Bayes model. To account for the differences that arise throughout the years such as the number of crashes and AADT, IHSDM needs to develop a separate prediction for each year and this approach was considered and applied in the suggested model to obtain an accurate and fair comparison. It is therefore important to consider this in the model developed here and determine how to best approach it. There are two options for incorporating the “year effect” in this analysis: (1) use a separate model for each year developing predictions one year at a time; (2) insert dummy variables for years to account for the different AADT of each year. The following presents this analysis and the determination of which approach is more appropriate. It should also be noted that there is no concern whether the dummy variables are statistically significant or not at this point; their purpose is to simply increase the predictive ability of the model by accounting for the yearly variation of AADT and random effects in general.
The evaluation will be accomplished by creating training data, i.e., assuming that a certain year is not included in the dataset, running the analysis, and then predicting the crashes of that year and reporting the residuals. For example, the way in which the predictive power of the model will be evaluated for the year 2017 is as follows. Suppose that crash data are available for the years 2004–2016 and that the intention is to predict the crashes for the year 2017. The predictive model will include the explanatory variables of the IM, i.e., AADT, GC, GC
2, GC
3, MC, MC
2, and GC*MC. Higher-order curvature terms are included as statistical representations of nonlinear geometric effects and are not intended to imply direct physical design variables. For the use of the dummy variable approach, in addition to the explanatory variables, a number of dummy variables equal to the number of years of crashes minus 1 is used. In this case, for the 13 years of available data (2004–2016 period), 12 (=13-1) dummy variables will be used. The crash predictions for the year 2017 will be calculated in the following form (Equation (2))
or finally:
The term “-LN(13)” is present in Equation (3) in order to convert the prediction model on a per-year basis since the model is based on 13 years of data. In statistical terminology, especially for GLM, this “-LN(13)” term is the so-called offset in the negative binomial regression (Hardin and Hilbe 2012) [
26]. The crash predictions for any other year will be calculated with the same exact procedure and rationale. The model structure is evaluated using both approaches, with and without dummy variables, and then the predictions compared to the actual number of crashes. The approach that results in a prediction closer to the actual number of crashes would be the one used.
In
Table 7, the crash prediction breakdown per year and roadway segment is presented in which there are three columns for each roadway segment: (1) actual crashes (AC); (2) predicted crashes without utilization of the dummy variables approach (W/O); (3) predicted crashes with utilization of the dummy variables approach (W/). In addition,
Table 8 presents the errors/residuals corresponding to the models with and without the dummy variable approach, as well as the corresponding crash improvement (CI) that has been achieved with the dummy variable approach.
The summary row in
Table 8 denotes that the inclusion of dummy variables results in predictions that are closer to the actual number of crashes than those excluding them. Moreover, the insertion of dummy variables is preferred, in general, over the creation of separate models for each year because it is statistically more appropriate: the Bonferroni correction can be applied in a much more robust manner, since the familywise error is explicitly defined, and small sample size issues, which are in general present in crash datasets, are alleviated with the dummy variable approach.
Therefore, at this point it is decided to utilize the dummy variable approach in order to compare the crash predictions of the suggested model with those derived from the IHSDM. The comparison follows in the next section.
The next step involves evaluation of the assumptions of the model developed, since every regression model is based on some statistical, mostly distribution-related, assumptions. This applies in this case as well, and therefore these assumptions must be checked in order to validate the reliability of the model. In practical/applied terms, failure in assessing these assumptions would mean that the coefficients of the model are not reliable, i.e., the coefficients are inflated or deflated compared to the true parameters. Moreover, the defined confidence levels of the coefficients may not hold true, a fact that means that the exported p-values from the models may be highly distorted, which, in turn, means that although the model may be considered statistically significant based on the explanatory variables’ p-values, it may in fact not be statistically significant since the results may be only artificially in favor of rejecting the null hypotheses.
Many techniques have been suggested for the assumption assessment of regression models, but especially in the case of GLMs this matter remains an open research problem. Therefore, for the scope of this research, the basic assumption assessment techniques for which there is a general agreement in terms of their effectiveness and pertinence from the scientific community will be checked. More specifically, the assumption assessment was based on two elements: (1) residual analysis, and (2) influential points identification. Both residential analysis and influential points identification, via the Cook’s distance concept, are explicitly described in Amiridis, K. (2019) [
18]. It is noted that the assumption assessment of the final regression model which corresponds to the 100 ft patch was conducted with success.