Evaluation of Beginner Driver Education in Oregon

Abstract

Although driver education (DE) is widely accepted as an effective teen driver safety measure and widely available in the United States, Canada and elsewhere, evaluations have generally failed to show that such formal programs actually produce safer drivers. To address the issue of safety effects as part of a larger investigation, two studies were conducted to examine whether the Oregon Department of Transportation (ODOT)-approved DE program was associated with reductions in collisions and convictions. In the first study, DE status among a relatively small sample of teens who completed an online survey was not found to have a significant effect on collisions and convictions. In the second study, of a much larger population of teen drivers, DE status was associated with a lower incidence of collisions and convictions. On balance, this suggests that the safety effects of DE are either neutral, based on the results of the first Oregon study, or cautiously optimistic based on the results of the second study. The implications of these findings are discussed in terms of making improvements in DE that are evidence-based, and the need for further evaluation to establish that improved and new programs meet their safety objectives.

1. Introduction

Driver education (DE) is designed to teach new drivers the rules of the road and the driving skills to prepare for the road test and obtain a driver’s license. It is recognized and promoted as a safety measure that dates back to the early 1930s in the United States and Canada. Today and for many years, most jurisdictions in the United States, Canada, and elsewhere have delivered DE in high schools and/or commercial driving schools. Most recently, a U.S. National Young Driver survey administered in public schools revealed that almost 80% of students with a driver’s license reported participating in formal DE [1]. These programs typically consist of both “theoretical” instruction in the classroom and “practical” training in the vehicle [2]. Although there are similarities across traditional DE programs, there is considerable variation in their content and delivery across jurisdictions as well as within some jurisdictions. In some cases, DE is a mandatory requirement of the licensing process and in others it is not compulsory, although provisions may be in place that encourage teens to complete DE (e.g., obtain a learner or provisional/intermediate license at an earlier age; fewer hours of supervised driving practice required).

Although DE provides an efficient means to learn how to drive, until recently, past evaluations have generally failed to show that such formal programs produce safer drivers [3,4,5,6,7,8,9,10,11,12,13]. This is the case in evaluations of the safety effects of DE that have been conducted internationally, and not just in the United States. Poor evaluation methods may be part of the reason for this, but even relatively well-designed evaluations have produced findings that raise questions about the safety benefits of DE. To illustrate, the DeKalb County, Georgia randomized control trial (RCT) involved the development and evaluation of a specialized curriculum with over 70 h of classroom, simulation, closed-course, and on-road training [14]. Despite having a very large sample of teen drivers randomly assigned to the new program, a condensed version of the new program containing only the minimum training required to obtain a license or to a control group, this investigation along with several subsequent re-analyses of the datasets failed to show a consistent positive effect of DE on collisions. In this regard, one re-analyses found a small significant effect of the condensed version compared to the control group [15] and, although one of the analyses in the original study reported that the new program was associated with fewer crashes in the first six months of driving, this positive result and other findings have been hotly contested in the literature on methodological grounds [7,16,17].

As a result of this investigation, support and funding for DE dramatically decreased at state and federal levels in the United States. For several decades, DE was no longer viewed as a priority safety measure [9,18]. However, over the past decade, in recognition of the research on DE, and in an effort to promote more uniformity in programs, there has emerged a concerted effort at the federal level and within the leadership of the DE field in the United States to improve traditional programs [19]. There have also been a few recent evaluations of DE programs that have further informed our understanding of their safety effects. A recent evaluation in Nebraska, for example, reported that DE was associated with a small but statistically significant reduction in collisions. A descriptive epidemiological study [20] that examined a census of all teen drivers in Nebraska found that teens taking DE were less likely than teens not taking DE to be involved in crashes during their first two years of driving. A hierarchical logistic regression showed, for example, that in year one of driving the DE cohort had significantly fewer crashes than the non-DE cohort (11.1% versus 12.9%). The analysis suggested that the non-education group was 1.22 times more likely than the DE group of crashing after taking into account key demographic factors such as age, sex, race/ethnicity, urbanicity and household income. This evaluation had several strengths, the principle of which was that it was a census of all Nebraska teen drivers receiving their provisional license during the study period meaning the large study population eliminated problems of reduced power due to small sample sizes. This study was not a true randomized, controlled experiment, as teens self-selected whether they took DE or not. Although statistical controls were applied to account for key demographic differences between teens that took or did not take DE, these statistical procedures did not fully compensate for the lack of random assignment. However, random assignment is difficult and perhaps not feasible in DE evaluation, especially when evaluating a well-established program delivered state-wide.

2. Purpose

As part of a larger evaluation of beginner DE programs in Oregon and Manitoba [21], the two studies described in this paper examined the collision involvement of DE and non-DE teen drivers in the state of Oregon. The purpose of the larger investigation was to demonstrate a more comprehensive evaluation of DE than has been previously done and to highlight how evaluations can better support program development in DE. More specifically, its objectives were to:

>

Generate new knowledge about the safety and operational effectiveness of DE;

>

Provide new information about how to improve the delivery and content of DE to enhance its safety impact;

>

Demonstrate the implementation of the Comprehensive Guidelines for evaluating DE developed by the AAA Foundation for Traffic Safety (AAAFTS) [22]; and,

>

Showcase more effective and constructive methods to evaluate DE.

This paper focuses on the safety effectiveness of DE in Oregon using two study designs. The first is a longitudinal comparison of DE and non-DE teens’ safety performance over the first few months of independent driving. The second study is a cross-sectional comparison of driving records of a large population of DE and non-DE teens. The primary measures of safety performance examined in both studies are collisions and convictions.

3. Driver Education in Oregon

Oregon was identified as a desirable evaluation site because of the well-established, state-approved DE program, which is not found in many U.S. states [2]. This program is also highly regarded in the DE community nationally, although it has not been included in a formal comparative assessment with other state DE programs. Oregon, however, was the second U.S. DE program reviewed by an expert panel under the National Highway Traffic Safety Administration (NHTSA) program assessment process [23]. This involves assessing a state’s current DE program structure and operations against the U.S. Novice Teen Driver Education and Training Administration Standards [19].

The Oregon state-approved DE program is administered by the Transportation Safety Division (TSD) of the Oregon Department of Transportation (ODOT). The certified program is encouraged but not required of teens. It is offered primarily in high schools as well as in community colleges (public providers) and in commercial driving schools (private providers). Both public and private providers of approved programs are subject to Oregon Administrative Rules and audits to ensure compliance with laws and rules pertaining to the operation of the approved provider’s program and instructor certification. Providers receive a reimbursement of $210 per student. The teen must have a learner permit before the start of the first DE class and complete the program in order for the school to be eligible for this student reimbursement. The program also includes a “Parent Involvement Resource Guide” and a “Driver Education Risk Prevention Curriculum”, a resource that includes classroom and in-car lesson plans, homework assignments, and entrance and exit exams.

The state-approved DE program is directly related to the graduated licensing system. To qualify for a provisional license, learner permit holders under the age of 18, must complete an ODOT-approved traffic safety education course and complete 50 h of supervised driving practice, or complete an additional 50 h of supervised driving practice (total of 100 h) certified by a parent or legal guardian. Accordingly, the approved-DE program is not mandated, but there is an incentive for taking it tied to a lower requirement in the number of hours of supervised driving practice. Despite this incentive, only a minority (about 30%) of licensed teens ages 16–17 complete an ODOT-approved DE program annually.

The approved DE program includes classroom instruction, behind-the wheel instruction, practice driving observation (activity done in back seat while another student drives), and a requirement for parental involvement. The program must include a minimum of 30 h of classroom instruction not exceeding six hours per week and three hours per day. The DE program also requires a minimum of six hours of behind-the wheel instruction not exceeding 90 min of driving per day per student.

Driving simulation may also be used, with four hours of simulation equal to one hour of behind-the-wheel instruction. However, simulation instruction cannot exceed more than one-half the required hours specified for behind-the-wheel instruction. As well, driving simulation may not precede classroom instruction. The program also allows a minimum of six hours of practice driving observation not exceeding three hours of observation per day per student. As well, there is a requirement for the involvement of a parent, legal guardian, or supervising adult that includes participating in a parent meeting held the first day of class. The meeting is attended by the parent, legal guardian, or supervising adult and the student, and it typically covers policies, procedures, curriculum, and questions regarding the program. Parents are also required to submit documentation, in the form of a log or other means, demonstrating to the provider that a minimum of five hours of supervised home practice was conducted prior to the completion of the course. This supervised home practice is not counted as a part of the classroom, behind-the-wheel, or practice driving observation of the provider course.

Classroom and in-vehicle instruction are delivered concurrently, which means that no less than four and no more than 10 h of classroom instruction will be completed before starting behind-the-wheel instruction. The intention is to ensure that the classroom and behind-the-wheel instruction are integrated and coordinated.

4. Method

4.1. Study One: Survey Participants

The first study used a prospective, longitudinal design to examine the safety performance of samples of teens who had or had not completed the Oregon ODOT-approved DE program. These teens were recruited into the study shortly after they obtained a provisional instruction permit and participated in an online survey prior to any teen completing the approved DE program. Their safety performance was measured in terms of whether they had collisions and convictions once they obtained a provisional license. The safety performance data for teens participating in this study were provided by ODOT Driver and Motor Vehicle Services Division (DMV). In regard to collisions, Oregon law requires drivers to file an accident report with DMV if there is damage to property of over $1500, if any vehicle is towed from the scene due to damage from the crash, or there is injury or death resulting from the crash. Convictions for traffic violations arise from not obeying a rule of the road, driver licensing or registration law, or vehicle equipment law.

4.1.1. The Sample and Survey Procedures

ODOT supplied weekly databases of Oregon teen drivers who had obtained their provisional instruction permit (PIP). This weekly sample included names and addresses, which were used to mail a recruitment letter that included a $5 bill, and offered a second $5 for completion of the survey online (along with parental consent). Several pretests revealed that these procedures yielded the highest response rate in the most cost effective method [21].

Letters were mailed beginning 20 April 2011, and continued weekly for 19 weeks until the targeted number of completes—5000—had been reached. A sample size of 5000 completions had been determined as providing adequate statistical power to detect significant differences in collisions between the Non-DE and DE groups.

Letters were mailed to a total of 12,181 teen drivers, yielding 5496 completed questionnaires (for a response rate of 45%). An additional 303 teens had completed the questionnaire during pretesting. In total, 5007 teens completed the questionnaire and had parental consents. The final sample used for analyses, however, comprised 4272 teens. Since the intention was to survey teens recently issued a provisional instruction permit, those who did not have a provisional instruction permit issue date (n = 21) were dropped from the sample, because the length of time on the permit was unknown. Teens issued an instruction permit rather than a provisional instruction permit because they were 18 years of age and over, or who had previously been issued a provisional instruction permit that had, for example, expired and then been issued again, were also excluded from the sample (n = 297).

Teens who said they had already completed DE at the time of the survey were also excluded from the analyses—as they would have been already influenced by DE (n = 192)—along with those students who had reported they were currently enrolled in a DE program, but for whom there was no official DE program completion date according to the DE data provided by ODOT (n = 88). Finally, those teens who had been enrolled in a DE program for more than seven days when they had completed the survey were excluded, as they would have also been exposed to DE instruction/training (n = 137). Those teens who had just enrolled within the last seven days were unlikely to have been influenced by DE at the time of the survey, so they were included in the final sample.

Teens included in the sample had been issued their first provisional instruction permit within six months from their survey date, and on average they had completed the survey about two months after having obtained their permit. Three teens had been identified as deceased. Records for these drivers were examined up until their licensing stop date rather than the data extraction date.

4.1.2. Survey Data

An online survey questionnaire developed and administered to Oregon teens shortly after they obtained their provisional instruction permit generated data to identify and control for pre-existing differences between DE and Non-DE teen drivers. The survey data contain information regarding driving attitudes, expectations, beliefs, skills, and behaviors/behavioral intentions. A copy of the questionnaire and a description of questionnaire development, including cognitive testing of question items with teens and two pilot studies to test for ceiling effects and to identify items and scales of low reliability are provided in the final report of the full investigation [21]. This final report also contains additional description of the different measures analyzed in this paper, including details on the variable coding/scoring systems used.

4.1.3. Licensing Data

The licensing data contain the licensing information for all Oregon teens in the study sample. Information includes the type of license issued (e.g., provisional instruction permit, provisional license, full driver’s license) and the date that it was issued. Demographic information, including age (date of birth), gender, and address were included in this file.

4.1.4. Driver Education Data

DE information was provided by the ODOT Transportation Safety Division for all teens completing DE over the study period (1 January 2010 to 30 September 2012). These data were used to determine which teens completed an ODOT-approved DE program—i.e., all teens in the DE data file had taken the ODOT-approved DE program. Thus, all cases from this file that were matched to the survey data were identified as the DE group, and all cases from the survey data that were not matched to the DE data were identified as the non-DE group. Of the final sample, a total of 1000 survey participants were matched to the DE data (DE group), and 3272 were not matched to these data (Non-DE group).

4.1.5. Urban/Rural Data

Weekly databases provided by ODOT of Oregon teen drivers who had obtained their provisional instruction permit, used to contact teens for participation in the survey, contained the names and addresses of teens. This information was used to create urban and rural designations for teens at the time they were issued their first provisional instruction permit. Monthly listings of all teens issued a provisional license from 1 January 2010 to 30 October 2012 were also provided from ODOT, containing similar information to that of the weekly listings. Addresses of teens from this file were used to determine the urban/rural split for teens at the time of their first provisional license issuance date. Zip codes were used to distinguish between urban and rural locations as indicated by the Oregon Office of Rural Health according to the U.S. Census Bureau definitions:

>

Urbanized Area (UA):

≫

Consists of contiguous, densely settled census block groups (BG) and census blocks (at least 500 people per square mile (ppsm) that together encompass a population of more than 50,000.

>

Urban Cluster (UC):

≫

Consists of contiguous, densely settled BGs and census blocks (500 ppsm) that together encompasses a population of at least 2500 people, but less than 50,000 people.

>

Rural:

≫

All population and territory that is not a UA or UC, or all geographic areas 10 or more miles from the centroid of a population center of 40,000 or more [24].

UA and UC were combined for the urban versus rural comparison.

4.1.6. Linking Survey Data with Driver Information

A list containing the driver license number of teens in the sample was sent to contacts at ODOT who provided DMV driver record and collision information for each survey participant. A 100% match was achieved. These files included licensing data and driver history data. These data were extracted on 22 October 2012, nearly 13 months after the last survey completion.

4.1.7. Statistical Analyses

The safety performance of DE and Non-DE teens were compared in terms of crashes and convictions. A Poisson regression could not be used to investigate the effect of various factors on these unsafe driving events because the data were not Poisson distributed. Negative binomial regression was also considered, but the high number of zeros present in the collision and conviction counts made this type of analysis inappropriate. For this reason, logistic regression was used, as this particular analysis does not assume such a distribution of the data. This logistic regression was used to investigate various factors examined in the survey that may be associated with teens having had a collision and a conviction. These analyses were performed for these unsafe events occurring from the date the teens were issued their provisional license until the ODOT data extraction date.

4.2. Study Two: Historical Data

The previous section described an in-depth, prospective, longitudinal study of a relatively small sample of teen drivers that compared the safety performance of teens who have taken the ODOT-approved DE program with that of those who have not taken this DE program over their first few months of independent driving. This section describes a retrospective study of a much larger population that has been driving for a greater length of time, including those that have or have not taken the ODOT-approved DE program. Historical records are used to examine the effects of the ODOT-approved DE program on collisions and convictions.

4.2.1. Study Sample

The study sample included Oregon teens issued their provisional instruction permit between 1 January 2005 and 31 December 2008; this was defined as the data intake period. The driver and crash records of these teens were tracked from their intake or selection date—i.e., when they were issued an instruction permit—to 17 September 2009, the date ODOT extracted the data for this investigation. The total number of teen drivers in this sample was 94,342.

4.2.2. Licensing Data

The licensing data contain the licensing information for all Oregon teens in the study sample who may or may not have taken the ODOT-approved DE program. Information includes the type of license issued (e.g., provisional instruction permit, provisional license, full driver’s license) and the date that it was issued. Demographic information, including age, gender, and address were included in this file. Zip codes were used to distinguish between urban and rural locations as indicated by the Oregon Office of Rural Health according to the U.S. Census Bureau definitions: UA, UC; and Rural (see definitions for Study One provided previously). If licenses were issued to teens whose address was listed as being outside the state of Oregon, these cases were dropped from the analyses. Note there were only 204 such cases after the data had been merged.

4.2.3. Driver Education Data

The DE data contained information on teens that have been recorded in ODOT files as having completed ODOT-approved DE, indicating the date the program was completed. This data set contained official records of all teens having completed DE between 30 June 2004 and 30 June 2009.

All teens in the DE data file had taken the ODOT-approved DE program. Thus all cases from this file that were matched to the licensing data (containing both DE and non-DE teens) were identified as the DE group. There was 100% match with those issued a provisional instruction permit in the licensing data input period. All cases coming from the licensing data that were not matched to the DE data were identified as the non-DE group.

4.2.4. The Driver History Data

This data set contained the driver records of teens including, collisions, and convictions, as well as the date on which each incident occurred.

4.2.5. Statistical Analyses

The driver performance data were Poisson distributed so Poisson regression was used to determine whether DE was associated with collisions and convictions after taking into account other factors such as demographic characteristics (age, gender), place of residence (urban versus rural), and length of time on a provisional instruction permit as well as on a provisional license. For these analyses, the number of crashes or convictions since obtaining a provisional license was used as the dependent variable and length of time since obtaining a provisional license was used as the exposure variable.

5. Results

5.1. Study One: Survey Participants

There are a total of 4272 teen drivers in this sample and 23% had completed the ODOT-approved DE program. Since issued a provisional license, DE and non-DE teens had 62 and 127 collisions, respectively. The DE and non-DE teens also had 28 and 106 convictions, respectively.

5.1.1. Collisions

Logistic regression was used to investigate various factors examined in the online survey that may be associated with teens having had a collision. In this analysis the dependent variable is the presence (versus absence) of collisions. Independent variables included DE status, age at provisional license issuance date (17 versus 16 years of age), gender (male versus female), whether the teen lived in a rural or urban area, self-reported race (white versus non-white), father’s level of education, length of time holding a provisional instruction permit, and length of time holding a provisional license. All other independent variables were composite scales derived from responses in the online survey. Scales and indices included in this analysis are as follows: Graduated Driver License (GDL) knowledge, GDL overall support, GDL-support specific requirements, GDL influence, Safe driving knowledge, Self-rated skills, Perceived likelihood of crash, Risk taking behavior, Risky driving attitudes, Risk taking attitudes, Lifestyle, Tolerance of Deviance, Parental Monitoring, Exposure, Time perspective, and Responsibility when driving. For an effect to be considered significant, its’ p-value has to be smaller than 0.05.

Table 1. Logistic regression with collision versus no collision as dependent variable. DE: driver education; GDL: Graduated Driver License; PIP: provisional instruction permit; OR: odds ratio; and SE: standard error.

Factor	OR	SE	%	p-Value
DE status	1.74	0.60	74%	0.10
Age at provisional license issuance date	0.73	0.24	−27%	0.34
Male	1.34	0.43	34%	0.37
Race White	0.27	0.22	−73%	0.10
Urban	0.92	0.30	−8%	0.81
Father’s level of education	0.60	0.20	−40%	0.13
GDL knowledge	0.97	0.07	−3%	0.65
GDL overall support	0.74	0.15	−26%	0.13
GDL-support specific requirements	0.72	0.21	−28%	0.26
GDL influence	1.23	0.33	23%	0.44
Safe driving knowledge	0.88	0.07	−12%	0.12
Self-rated skills	1.03	0.30	3%	0.9
Perceived likelihood of crash	0.93	0.16	−7%	0.68
Risk taking behavior	0.66	0.26	−35%	0.28
Risky driving attitudes	1.11	0.30	11%	0.71
Risk taking attitudes	1.84	0.55	84%	0.04
Lifestyle	0.90	0.24	−10%	0.68
Tolerance of deviance	0.43	0.18	−57%	0.05
Parental Monitoring	0.64	0.20	−36%	0.16
Exposure	0.93	0.96	−8%	0.94
Time Perspective	0.88	0.24	−12%	0.63
Responsibility when Driving	1.25	0.40	24%	0.49
Length of time held PIP(in months)	1.00	0.07	−1%	0.9
Length of time on provisional license (in months)	1.17	0.05	17%	p < 0.01

%-change indicates the percent change in the odds for collisions for each unit increase in the independent variable.

shows the results of the logistic regression analysis.

Logistic regression revealed that having a higher score on the Risk taking attitudes scale (indicating a higher agreement with various statements that tap normlessness, peer pressure, and risk-taking propensity) significantly increases the likelihood of having had a collision, by 84%. Having held a provisional license for a longer period of time significantly increases the likelihood of having had a collision, by 17%. DE status was not found to have a significant effect on collision involvement (OR (odds ratio) = 1.74; p = 0.10).

5.1.2. Convictions

Logistic regression was used to investigate various factors examined in the online survey that may have been associated with teens having had a conviction for a traffic violation. In this analysis the dependent variable was the presence (versus absence) of convictions. Independent variables were the same as used in the collision analysis (see previous section).

Table 2. Logistic regression with conviction versus no conviction as dependent variable.

Factor	OR	SE	%	p-Value
DE	1.33	0.60	33%	0.53
Age at provisional license issuance	2.01	0.62	101%	0.02
Male	2.63	1.03	163%	0.01
Race White	0.53	0.72	−47%	0.64
Urban	0.56	0.22	−44%	0.13
Father’s level of education	0.54	0.24	−46%	0.17
GDL knowledge	1.00	0.08	0%	0.98
GDL overall support	0.59	0.14	−41%	0.03
GDL-support specific requirements	1.30	0.46	30%	0.45
GDL influence	1.79	0.64	79%	0.10
Safe driving knowledge	1.01	0.10	1%	0.92
Self-rated skills	1.14	0.39	14%	0.70
Perceived likelihood of crash	1.03	0.21	3%	0.88
Risk taking behavior	0.92	0.42	−8%	0.85
Risky driving attitudes	1.56	0.47	56%	0.14
Risk taking attitudes	0.91	0.35	−9%	0.80
Lifestyle	1.05	0.38	5%	0.90
Tolerance of deviance	0.76	0.40	−24%	0.60
Parental Monitoring	0.85	0.34	−15%	0.69
Exposure	0.80	0.70	−20%	0.80
Time Perspective	1.52	0.45	52%	0.16
Responsibility when Driving	1.91	0.91	91%	0.18
Length of time held PIP (in months)	0.96	0.07	−4%	0.54
Length of time on provisional license (in months)	1.28	0.06	28%	p < 0.01

%-change indicates the percent change in the odds for convictions for each unit increase in the independent variable.

below shows the results of the logistic regression analysis. As can be seen, logistic regression revealed that as age at provisional license issuance increases, the likelihood of having had at least one conviction significantly increases (p = 0.02). Results also showed that being male significantly increases the likelihood of having had a conviction (p = 0.01). Having a higher score (indicating a higher level of support) regarding overall support for GDL significantly decreased the likelihood of having had a conviction by 41% (p = 0.03). Finally, a teen having held a provisional license for a longer period of time increased the likelihood of having experienced a conviction by 28%. DE status did not have a significant effect on conviction involvement (OR = 1.33; p = 0.53).

5.2. Study 2: Historical Records

There are a total of 94,342 drivers in this study population and 20,755 (22%) had completed the ODOT-approved DE program. Since issued a provisional license, DE and non-DE teens had 2715 and 11,055 collisions, respectively. The DE and non-DE teens also had 5799 and 41,384 convictions, respectively.

5.2.1. Collisions

Poisson regression was employed to examine collision counts from the date the teens were issued their provisional license until the extraction date (17 September 2009) using several independent variables, including DE status (DE versus Non-DE), gender, age when issued provisional license, residence in an urban or rural setting, and the length of time a provisional instruction permit was held (in months). The length of time from the provisional license date of issuance to the extraction date was used as an exposure variable. Results are summarized in

Table 3. Results from Poisson analysis of collision since issued provisional license.

Dependent Variable: Crash Count Since Issued Provisional License
Independent Variable	Coefficient	p-Value	%-Change	%SD
DE status	−0.04	p < 0.05	−4.3%	−1.8%
Gender	−0.01	p = 0.45	−1.3%	−0.6%
Age	0.16	p < 0.01	17.0%	7.5%
Urban	0.05	p < 0.01	5.4%	2.6%
PIP length (in months)	−0.01	p < 0.01	−1.1%	−4.9%

%-change indicates the percent change in expected collision count for a unit increase in the independent variable; %SD indicates the percent change in expected collision count for a standard deviation increase in independent variables.

.

As shown in Table 3, having completed a DE course significantly decreases the expected number of collisions by 4.3% (p < 0.05). Gender did not have a significant effect on collision rates, but all other variables in the model did show significant results. With regard to age, being one year older (17 versus 16 years of age) significantly increases the expected number of collisions by 17.0% (p < 0.01), and residing in an urban area significantly increases the expected number of collisions by 5.4% (p < 0.01). It was also found that an increase of one month in the length of time under a provisional instruction permit results in a decrease of 1.1% in the expected number of crashes (p < 0.01). If time under provisional instruction permit is re-scaled to an increase of six months, the percentage decrease in expected crashes is 6.6%.

5.2.2. Convictions

Poisson regression was also employed to examine the convictions from the date the teens were issued their provisional license until the extraction date (17 September 2009) using various independent variables including DE status (DE versus Non-DE), gender, age when issued provisional license, residence in an urban or rural setting, and the length of time a provisional instruction permit was held (in months). The length of time from the provisional license date of issuance to the extraction date was used as an exposure variable. As shown in

Table 4. Results from Poisson analysis of convictions since issued provisional license.

Dependent Variable: Conviction Count Since Issued Provisional License
Variable	Coefficient	p-Value	%-Change	%SD
DE status	−0.499	p < 0.01	−39.3%	−18.7%
Gender	0.632	p < 0.01	88.1%	37.1%
Age	0.498	p < 0.01	64.6%	25.7%
Urban	0.017	p = 0.07	1.7%	0.8%
PIP length (in months)	−0.041	p < 0.01	−4.0%	−17.4%

%-change indicates the percent change in expected conviction count for a unit increase in the independent variable; %SD indicates the percent change in expected conviction count for a standard deviation increase in independent variables.

, having completed a DE course significantly decreases the expected number of convictions by 39.3% (p < 0.01). Results also revealed that being male increases the expected number of convictions by 88.1% (p < 0.01). With regard to age, being one year older (17 versus 16 years of age) also significantly increases the expected number of convictions by 64.6% (p < 0.01). Residing in an urban area does not significantly increase the expected number of convictions at the 5% level. It was also found that an increase of one month in the length of time under a provisional instruction permit results in a decrease of 4.0% in the expected number of convictions. If this is re-scaled to an increase of six months the percentage decrease in expected convictions is 21.7%.

5.3. Safety Performance: The Influence of Confounding Factors

The two studies on DE safety effects discussed in the previous section produced mixed results: one study focusing on a small sample of survey participants suggested that DE was not associated with fewer collisions and convictions; the second one focusing on a much larger state-wide sample suggests that DE was associated with fewer collisions and convictions. Due to the small sample size in the first study, the teen survey participants may not have been representative of all teen drivers in the state with respect to crash involvement. Although the larger sample of teen drivers produced results suggesting a positive effect of DE on collisions and convictions, only a few factors could be controlled in the analyses.

To address whether the relationship between collisions/convictions and DE may have been biased because important teen attributes derived from the survey in the first study were omitted from the regression models for the larger sample in the second study, sensitivity analyses were conducted. For the small sample of teen drivers, this involved examining three models:

>

Model 1: crash involvement, adjusted for time since issuance of provisional license only, with DE status.

>

Model 2: crash involvement with DE, age, sex, urban/rural, length of time on provisional license, and length of time on provisional instruction permit (all variables controlled in the regression model used with the larger sample).

>

Model 3: crash involvement with Model 2 variables and all the other questionnaire-based variables, which were not available for the larger sample.

For more direct comparison, the models for the larger sample were restricted to 24 months since issued a provisional license, which was more comparable to the tracking period for the smaller sample of teen drivers. The analyses with the larger sample involved producing Models 1 and 2, as described above.

Table 5. Influence of Confounding Factors on Collisions. RR: risk rate; and IRR: incidence rate ratio.

Samples	Potential Confounders in Model for Collisions	OR/RR (for DE)	SE (OR)	p-Value	Coefficient	%
Study Subjects	None	1.5236/1.5066	0.3982744	0.107	0.42108	52.4
	Age, Sex, Urban, PIP length	1.5860/1.5662	0.4264788	0.086	0.46122	58.6
	Age, Sex, Urban, PIP length, NDS variables	1.74, 1.8256/1.7963	0.5956708	0.105	0.55473	74.1
		IRR (for DE)	SE (IRR)	p-Value	Coefficient	%
State-wide Retrospective	None	0.9346088	0.0200187	0.002	−0.0676272	−6.5
	Age, Sex, Urban, PIP length	0.9574815	0.0208123	0.046	−0.04345	−4.3
State-wide Retrospective Restricted (24 months)	None	0.8971727	0.020805	<0.001	−0.10851	−10.3
	Age, Sex, Urban, PIP length	0.9170248	0.0215365	<0.001	−0.08662	−8.3

and

Table 6. Influence of Confounding Factors on Convictions.

Samples	Potential Confounders in Model for Convictions	OR/RR (for DE)	SE (OR)	p-Value	Coefficient	%
Study Subjects	None	0.7288/0.7330	0.2453852	0.347	−0.31641	−27.1
	Age, Sex, Urban, PIP length	1.0326/1.0319	0.3604287	0.927	0.03206	3.3
	Age, Sex, Urban, PIP length, NDS variables	1.33, 1.2834/1.2770	0.5962632	0.531	0.28175	32.5
		IRR (for DE)	SE (IRR)	p-Value	Coefficient	%
State-wide Retrospective	None	0.5334959	0.5334959	<0.001	−0.6283038	−46.7
	Age, Sex, Urban, PIP length	0.607047	0.0086182	<0.001	−0.4991491	−39.3
State-wide Retrospective Restricted (24 months)	None	0.5462046	0.0088163	<0.001	−0.60476	−45.4
	Age, Sex, Urban, PIP length	0.618501	0.0100977	<0.001	−0.48046	−38.1

show the results of the models for collisions and for convictions. The results show that controlling versus not controlling for the variables in the questionnaire does not cause the coefficient of DE to change much. As can be seen in the tables, any results regarding DE that were significant, remain significant after controlling for additional sets of variables and any results regarding DE that were not significant remain non-significant after controlling for additional sets of variables, both for models using collisions as well as convictions.

In addition to considering potential changes to levels of significance when comparing models with and without control variables, it can be argued that for those models with significant effects, coefficients for DE should be compared to see if they differ from one another. For example, in Table 5 the state-wide retrospective restricted results for collisions show that the coefficient for DE is significant in the models with and without control variables. The question, then, is whether coefficients for DE are different between both models, i.e., is 0.935 (SE = 0.020) different from 0.957 (SE = 0.021)? Statistical significance as well as the actual magnitude of the effect are important in this regard. It is perhaps not surprising that the coefficients turned out to be significantly different (Chi-square = 42.01; degrees of freedom (df) = 1; p < 0.0001) given the large sample size of these state-wide retrospective analyses (n = 94,117). More importantly, the actual magnitude of the difference between the coefficients is nominal, one suggesting there was a 6.5% reduction in collisions whereas the other one suggests a 4.3% reduction. Essentially, this means there was no meaningful difference between the two models.

Similar conclusions can be drawn about all models in that levels of significance of coefficients for DE do not change, and when results were significant, no meaningful differences regarding magnitude or direction of effects were found between DE coefficients from models with or without control variables.

This suggests that controlling for all those questionnaire-based variables might not be too important to draw inferences about the safety performance of the Oregon DE program, thus there is some limited evidence suggesting that the results from the simple retrospective analysis using the larger sample probably are not biased by not controlling for these other potentially confounding variables. Of course, other factors associated with self-selection not controlled in the analysis may also influence the relationship between DE and safety performance.

The additional analysis also revealed two other important findings. First, the logistic regression showed in Table 5 above estimates that the covariate DE was not statistically significant (p-value = 0.107, 0.086 and 0.105, respectively) to predict the occurrence of crashes. These non-significant differences between the DE and Non-DE groups from this more sophisticated logistic regression model suggest that DE was not associated with higher collision involvement.

The second important finding is that the additional regression models for the larger sample of state-wide teen drivers show that when the analysis was restricted to 24 months since obtaining a provisional license, rather than the entire 54-month tracking period, the percent reduction was almost doubled—from 4.3 to 8.3—which means that having completed a DE course significantly decreases the expected number of collisions by 8.3% (p < 0.001). Thus, the positive effects of DE on collisions, after controlling for key variables, are greater several months to 24 months following obtaining a provisional license than over a longer period of time, although benefits are still apparent after 54 months. If DE has a positive effect on collisions, it would be expected to be apparent shortly after graduation, a period in which teens are at the greatest crash risk, and then dissipate over time as other factors, such as increasing driving experience, influence driving behaviors and potentially overwhelm DE effects, and these results are consistent with this expectation.

6. Summary and Discussion

The DE program evaluated in Oregon in this investigation is administered by the Transportation Safety Division of ODOT and this program is available through both public and private providers. Approximately 30% of teen drivers complete the ODOT-approved DE program.

As part of a larger investigation, two studies were conducted to examine the safety performance of Oregon DE and Non-DE teens in terms of collisions and convictions. In the first study, a sample of teen drivers were surveyed prior to any teen completing the ODOT-approved DE program and their collisions and convictions were compared once they were on a provisional license. A related study used historical records to examine the collisions and convictions of a much larger population of Oregon teens who had and had not completed the ODOT-approved DE program.

6.1. Safety Performance: Survey Participants

This part of the overall investigation examined the safety performance of the ODOT-approved DE program defined in terms of collisions and convictions. The results of logistic regression analysis controlling for other independent variables are consistent with most previous evaluations which consistently report that DE fails to reduce collisions and convictions.

Given these findings, it may be the case, as others have concluded that it is “unrealistic” to expect that DE as now constituted, by itself will improve safety performance measured in terms of fewer collisions and convictions. [10,12]. Such a conclusion, however, may not be warranted based on these finding because of study limitations. The sample of teen survey participants was small, less than 5000 teen drivers, and relatively few of them had collisions as determined by official driver records. In this regard Peck [17], in a review of the literature on the effectiveness of DE, had shown that as many as 35,000 drivers would be required in a two group design to reliably detect a 10% reduction in crashes. The fact that the collision data of the teen survey participants did not fit a Poisson distribution, which is typically the case with count data of uncommon or rare events such as collisions, also suggests that this sample of teens may not have been representative with respect to crash involvement. This may be the case even though the sample was drawn from the population of all teens obtaining a provisional instruction permit over the study period and there was a reasonable response rate with over 40% of these teens participating in the survey. It is also possible that although the regression analysis controlled for key teen attributes, there were unknown biases because other pre-existing factors that differentiate DE from Non-DE teens and influence collision involvement were not included in the model. The study also used “proxy measures” of exposure (e.g., length of time on a provisional license) which may not have accurately captured driving exposure shown in other studies using different “proxy” measures (e.g., vehicle ownership) to be associated with collision involvement [25,26].

It is possible that if this study had used a larger sample of teen drivers, tracked their driver record over a longer period of time, and collected self-reported collisions rather than relying on less-frequently reported collisions from official records, the results might have been different. Although the original research design had proposed a longer tracking period as well as capturing self-reported data on collisions, both practical and cost constraints precluded doing so and these are now important limitations of this study, which should be the focus of follow-up research.

6.2. Safety Performance: Historical Records

To address some of the limitations mentioned above, the second study did use a much larger population of Oregon teen drivers and a retrospective design to examine the safety performance of the ODOT-approved DE program. This study found that this program has safety-related benefits, in terms of being associated with lower incidences of collisions and convictions. This investigation also took into account important self-selection factors by controlling for age, gender, driving exposure, and place of residence (urban/rural). These are all factors shown in the larger investigation to differentiate between teens who voluntarily take the ODOT-approved DE program and those that do not. However, the study used only official records so it was not possible to control for other pre-existing factors that could potentially account for differences in the collisions of DE and Non-DE teens.

6.3. Safety Performance: The Influence of Confounding Factors

Sensitivity analyses were conducted using the smaller sample of teen survey participants and the larger sample of state-wide teen drivers to determine whether the relationship between collisions/convictions and DE observed for the larger sample may have been biased because potentially important teen attributes derived from the survey were omitted from the regression models or the larger sample. The results suggest that controlling for all those questionnaire-based variables might not be too important to draw inferences about the safety performance of DE, thus there is some limited evidence suggesting that the results from the simple retrospective analysis using the larger sample probably are not biased by not controlling for these other potentially confounding variables. Of course, other factors associated with self-selection not controlled in the analysis may also influence the relationship between DE and safety performance.

Finally, the additional analyses also suggest that the positive effects of DE on collisions, after controlling for key variables in the larger sample, are greater several months to 24 months following obtaining a provisional license than over a longer period of time, although benefits are still apparent over a 54-month period.

6.4. Implications

Previous reviews of the evaluation literature consistently report that DE fails to reduce collisions and convictions. The results of the current evaluation of the ODOT-approved DE program have been mixed. The first study, using a relatively small sample of Oregon teen drivers that controlled for the influences of various teen attributes and crash-related factors, suggests there are no beneficial effects of DE on collisions and convictions. These results, however, should be treated cautiously as they are based on a small sample size which may make them unreliable.

The second study, using a larger sample of Oregon teen drivers and controlling for fewer factors, suggests fewer collisions and convictions. This latter finding is encouraging for DE in Oregon, especially since it is generally consistent with earlier positive findings by Raymond et al. [27] in a NHTSA-funded evaluation of the Oregon GDL program. It is also consistent with a recent population-based evaluation of DE in Nebraska which also reported positive safety benefits [20]. However, the findings from this study and the earlier ones should be interpreted cautiously, and not overvalued, in light of study limitations, principal of which is self-selection bias.

On balance, the two studies suggest that at worst, the ODOT-approved DE program is not associated with increased collision involvement, and at best it is associated with a modestly but statistically significant lower incidence of collisions. This suggests that the overall findings on the safety effects of DE are either neutral, based on the results of the first Oregon study, or cautiously optimistic based on the results of the second study.

Traditional DE programs, like the one offered in Oregon, may have had less of an effect than expected for the reasons suggested by Williams et al. [12]:

>

The courses generally are of short duration, and most time has to be spent teaching basic vehicle handling skills. This leaves less time to try to teach safe driving skills;

>

The audience for DE may also be relatively unmotivated regarding safety, the primary motivation being to learn enough to get a driver’s license;

>

Probably the biggest impediment to DE effectiveness involves the inherent difficulties in affecting lifestyle and developmental factors: the attitudes, motivations, peer influences, and cognitive and decision-making skills that are so influential in shaping driving styles and crash involvement (p. 11).

More recently, Thomas et al. [10], in a study taking a fresh look at DE in America, have concluded that “the expectation that DE by itself will lead to a decreased teen crash rate is unrealistic.” Although this may hold some merit, the fact is that DE programs have been developed and are marketed not just to teach teens how to drive and pass the road test but also to produce safer drivers, which typically translates to drivers that have lower collision rates. It is possible, however, that the potential safety effects of DE can be more realistically expected if DE is better coordinated with other teen driving safety measures. At least two related-investigations in Denmark point to positive crash effects of coordinating DE and driver license testing criteria [28,29]. As well, better integrating DE with GDL, which have proven safety effects may provide a means to achieve greater safety benefits. In this regard, Mayhew et al. [30,31] have recommended a comprehensive GDL Framework that involves identification of basic GDL features that have the greatest safety benefits, and reinforcing policies for basic GDL features, including DE, licensing and testing requirements, and in-vehicle monitoring technology.

Other developments that may enhance DE include the Novice Teen Driver Education and Training Administrative Standards which were developed by representatives from the DE professional community with assistance from the NHTSA and published during the period that the present investigation was underway. The Standards document states that: “The goal of DE and training is to transfer knowledge, develop skills, and enhance the disposition of the teen, so he/she can perform as a safe and competent driver, thereby contributing to the reduction of crashes, fatalities, and injuries.” These National Administrative standards also have been updated and expanded to provide guidance as to how traditional DE programs should be substantially restructured to potentially better achieve their safety goals [32].

Also, the ODOT-approved DE program has recently been assessed by a panel of experts against the initial standards. The brief review of the Oregon program against the national standards provides some basis for understanding how the program could be modified to improve its impact beyond that seen in the current studies. Priority recommendations identified for the Oregon program were, for example: increasing classroom hours from 30 to 45 h; increasing behind-the-wheel instruction from six hours to ten hours; increasing in-car observation from six hours to 10 h; and, requiring second stage education of at least ten hours. As well, the expert panel recommended that the ODOT-Transportation Safety Division (ODOT-TSD) should establish a procedure for providing an end-of-course evaluation or progress report to parents. This end-of-course “debriefing” could be a written student progress report which includes areas of successful completion of safe driving practices and any necessary recommendations for continued practice prior to licensing.

Only a few other jurisdictions have undergone this NHTSA review, including Maryland, Vermont, Delaware, Idaho, and Michigan. Other jurisdictions may benefit from participating in this review process. Jurisdictions in Canada and elsewhere can also use these administrative standards to provide guidance for an internal review of their own DE and training programs to identify workable improvements. While these standards have not yet been proven to result in measurable changes in safety performance, the present investigation makes it clear that major improvements in DE are needed, and the standards provide a direction to start a systematic program of development and evaluation that may lead eventually to substantial safety benefits. Perhaps it is unrealistic to expect that traditional DE will lead to fewer teen crashes, but there is no reason this should not be a goal of improved programs and the focus of future evaluation to determine whether enhancements lead to better outcomes.

Existing DE programs in the United States and Canada may also benefit from developments in other countries. This is especially the case because traditional programs in North America may be too narrowly focused and fail to adequately address wider lifestyle issues which determine how drivers actually behave on the road and not how skilled they are at driving (i.e., what drivers actually do versus what drivers are capable of doing). In this context, the Goals for Driver Education (GDE) principles were originally developed by the European Union in the Guarding Automobile Drivers through Guidance, Education and Technology (GADGET) project [33,34]. The GDE Matrix provides a hierarchical schema of the driver’s task, outlining the personal situation within which all drivers undertake driving, including preconditions, attitudes, abilities, demands, decisions and behavior, categorized into four levels: (1) Vehicle maneuvering; (2) Mastery of traffic situations; (3) Goals and context of driving; (4) Goals for life and skills for living. Traditional DE programs in the U.S. and Canada typically focus on levels one and two, and to some extent level three, without any or much focus on the fourth level, despite the fact that these higher levels are likely the major influences on teen driving behavior giving rise to their elevated crash risk. Although applying GDE principles to improve DE makes sense on logical grounds, the safety effects have not yet been evaluated.

Existing DE programs can certainly be improved, while new ones need to adopt practices that are evidence-based. Non-traditional approaches that have been researched and may hold promise to enhance DE include: resilience training to address lifestyle factors; insight training to address optimism bias; TeenSMART, a crash reduction program; RAPT, a risk awareness and perception training program; and the Probationary or “P” Drivers program, which aims to effect behavior change among young newly licensed drivers, currently being evaluated in Australia [35,36,37,38,39,40]. It is equally important that program enhancements are evaluated and new programs piloted to determine the extent to which they meet their stated objectives.