Validation of the “Wisconsin Criteria” for Obtaining Dedicated Facial Imaging and Its Financial Impact at a Level 1 Trauma Center

Abstract

Introduction: According to national trauma databases, 25% of all people injured have some type of facial injury. The gold standard for diagnosing bony trauma of the facial skeleton has been computed tomography (CT) scan. In 2011, the “Wisconsin criteria” were established and subsequently validated as a method to predict which patients truly have a facial fracture and warrant further imaging. The purpose of this study is to externally validate these criteria and determine the economic impact they might have on avoiding unnecessary CT scans. Methods: This was a retrospective chart review. We collected 1000 patients who had undergone facial CT related to trauma and retrospectively applied the “Wisconsin criteria” based on physical exam documentation. Results: Of the 1000 facial CT scans obtained, we identified 408 fractures, 12% of which required operative intervention. The “Wisconsin criteria” applied to our patient population had a sensitivity of 90% and a negative predictive value of 93%. Using these criteria resulted in a missed fracture rate of 2.8%. Finally, had these criteria been used to determine when a facial CT scan was indicated, our institution could be saving over US$300 000 annually. Conclusion: The “Wisconsin criteria” are a reliable method to screen for facial fracture in trauma patients. Using a validated instrument tool to guide decision-making, we can avoid obtaining low-value imaging studies. This can have a large economic impact while maintaining safety and reliability in accurate diagnosis.

Introduction

Trauma continues to be a leading cause of morbidity and mortality in the United States. In the most recent National Trauma Data Bank report, the highest rate of injury was seen in those aged 25 to 34 years; however, fatality rate is highest in those over the age of 84. Of all patients injured, approximately 25% presented with facial injury.[1] The diagnosis of facial injury, especially in the setting of polytrauma, can be a challenge. Head computed tomography (CT) is routinely performed in the acute setting; however, only 12% of these patients may have concomitant facial fractures.[2] Furthermore, many missed fractures are of the lower third of the face.[3] Most facial bone CT protocols specifically scan from the mandible to the frontal sinus and use a 3-mm slice thickness compared to 5 mm of most standard head CT protocols.[4] Due to these difficulties, there is value in establishing clinical criteria for when obtaining dedicated facial imaging is indicated.

In 2011, the “Wisconsin criteria” were established as a guide in determining which patients would benefit from dedicated facial imaging.[5] This study showed a sensitivity of 98.2% in diagnosing facial fracture if one or more of the following 5 clinical findings were present: bony step-off or instability, periorbital swelling or contusion, Glasgow coma scale (GCS) score <14, malocclusion, or tooth absence. The same authors had a follow-up study with internal validation showing 97.4% sensitivity and a missed fracture rate of 2.6%.[6] Unfortunately, an attempt of external validation by Harrington et al showed 81% sensitivity and 60% negative predictive value (NPV), raising concerns for the broad applicability of this method.[7]

Dedicated imaging of the facial structures is a powerful tool in the diagnosis and ultimate management of injuries. However, this should be used in the most prudent manner to avoid excess radiation exposure for the patient and increased cost of care. There may be more appropriate screening methods for certain injuries, such as plain X-ray or orthopantomogram, and appropriate decision methods should be employed when deciding to obtain specialized facial CT scans.[8] The goal of this study is to determine whether the “Wisconsin criteria” are a valid tool in determining the presence or absence of facial fracture, and if so, what might the economic impact be of using these criteria to determine the need for facial CT.

Methods

This study is a retrospective chart review. After obtaining institutional review board approval, we reviewed data from a level 1 trauma center located in the midwestern United States. Data were obtained by searching EPIC (EPIC Systems Corporation, Verona, Wisconsin) electronic medical record for all special midface CT scans (mandible through frontal sinus obtaining 3 mm slices) using billing codes 763839 or 763847. Patient charts were reviewed and the “Wisconsin criteria” were applied post hoc to determine the need for facial CT. Inclusion criteria were patients who received a CT scan at initial evaluation in the emergency department for trauma and had adequate documentation. Patients were excluded if there was no specific mention of the presence or absence of the criteria in the chart. Additional information collected included the type of fracture, mechanism of injury (blunt or penetrating), need for surgical intervention, and if a maxillofacial specialist was consulted. Demographic information was also collected, including age, ethnicity, and gender. Based on previous studies’ sample sizes and to ensure statistical power, we stopped data collection once we included 1000 patients. This coincided with the time period from May 5, 2012, to September 14, 2014. Sensitivity, specificity, NPV, and positive predictive value (PPV) were calculated (JMP Pro 14). A P value of <.05 was considered significant.

Results

Most patients were male (62.4%), with a mean age of 30 years (range: 0.1-98). The primary mechanism of injury was blunt trauma (99.7%). A total of 408 fractures were identified by dedicated maxillofacial CT imaging (40.8%;

Table 1. Demographics of 1000 Included Patients.

).

Maxillofacial trauma consultation was obtained acutely in 48.2% of patients with fractures. Of the patients with fractures, 49 (12%) required surgical intervention. Nasal bone fractures were the most common (53.1%) followed by periorbital fractures (30%). Mandibular and maxillary fractures were less common (6.1% and 15.1%); however, 20% of patients with these fractures had a complex pattern identified (

Table 2. Fracture Characteristics.^a.

).

Like the study by Sitzman et al, we excluded those patients with isolated nasal bone fractures for our analysis. We calculated the sensitivity, specificity, NPV, and PPV using each of the “Wisconsin criteria” (

Table 3. Wisconsin Criteria Applied to Our Data Set.

). The presence of bony step-off or a GCS <14 both had 99% specificity; however, sensitivity was quite low at 7% and 2%, respectively. Periorbital ecchymosis was the most sensitive of the individual criteria (70%).

Overall, the presence of any of the criteria had a sensitivity and specificity of 90% and 52%, respectively, and a PPV and NPV of 44% and 93%, respectively. When identifying operative fractures, these criteria showed 94% sensitivity and 40% specificity while having a PPV and NPV of 6% and 99%, respectively. The sum of the present criteria was also analyzed. Having more than one criterion present per patient, in comparison to any one criterion, did not necessarily correlate with improved prediction of fracture. Lastly, using receiver operating characteristics to evaluate these criteria as a predictive model, we found the area under the curve to be 0.73 (Figure 1).

Applying the “Wisconsin criteria” to the patients in our study, there would have been a missed fracture rate of 2.8%. Half of these missed fractures were maxillary fractures and one-third were mandibular fractures, a majority of which were nondisplaced. These criteria would have failed to identify a single operative fracture for an overall missed operative fracture rate of 0.12% (

Table 4. Characteristics of Fractures Missed by Criteria.

).

Discussion

Maxillofacial trauma remains common, yet there are currently no reliable clinical criteria for guiding decisionmaking in obtaining dedicated special midface CT. The goal of this study was to determine whether specific clinical decision tools are useful in identifying fractures and which patients should undergo further imaging. We also wanted to evaluate the economic impact such tools could have if found to be accurate. Previously, there have been contradicting reports regarding the utility of these criteria in decision-making. An internal validation by Sitzman et al was consistent with the original study; however, an external validation by Harrington et al failed to do so on account of low sensitivity and NPV.[6,7]

Our study has the largest sample size to date evaluating the use of specific clinical exam findings to reliably predict whether a facial fracture is present. Our demographic is similar to Sitzman’s group regarding age, male predominance, and mechanism of injury. We did, however, have a lower overall fracture rate (40% vs 60%). When applied retrospectively, the “Wisconsin criteria” were 90% sensitive in ruling out facial fractures. More importantly, the absence of all five of the criteria had an NPV of 93%, which is similar to the original article (

Table 5. Comparison of Published Data.

). We do not find these criteria, however, to be very specific in identifying fractures.

The use of clinical findings to determine the need for further imaging in other specialties is shown to have high sensitivity.[9,10,11] From an economic standpoint, using a validated decision tool is key to ensuring CT scans which are likely to provide clinical benefit are obtained. In 2009, it was estimated that US$210 billion was wasted on unnecessary services.[12] Furthermore, low-value care (providing little to no benefit) tends to be overused, especially in the under- or noninsured.[13] Using our results of an NPV of 93% and applying this to our data set, we calculated 306 low-value CT scans over a 29-month period. In 2009, the cost of a special midface CT scan at our institution was US$1040, which translates to US$318,240 annually or US$10,973/month of wasted health-care spending. When adjusted for inflation, this translates to US$12,839/month in 2018 dollars.[14] Nonindicated CT scans are also leading to increased exposure to ionizing radiation. The risk of cancer from radiation is due to cumulative exposure, so the younger someone is exposed, the greater the lifetime risk. Smith-Bindman et al estimated that it would take 7350 routine head CTs of 20-year-old males (median of 2.1 mSv/scan) to cause one radiation-induced cancer.[15] Trauma patients tend to be young males, putting this patient cohort especially at risk.

Our results validate this tool for screening out fractures (high sensitivity and NPV); however, across the board, we have low diagnostic capability (low specificity and PPV). The role of physical exam in making diagnosis of facial fractures can often be limited due to the degree of soft tissue injury/edema and is especially difficult for periorbital fractures.[16] In our sample, periorbital ecchymosis was the most common finding, if present, but had the lowest specificity. Most trauma patients have undergone head CT, which may act as a valid screening tool for identifying facial fractures; however, it too has difficulty with periorbital fractures.[17] Ricci et al also compared patients receiving both head CT and facial CT. They identified twice as many operative fractures on facial CT in those patients who had fracture identified on head CT initially.[18] Furthermore, obtaining a facial CT scan in the correct patient may allow enhanced decision-making and improve timely care, especially for complex periorbital fractures.[19]

In our data set, the most frequently missed fracture was isolated maxillary fractures. The original study excluded these fractures; however, we included them for completeness. Imaging is not typically indicated in these fractures and management is often nonsurgical.[20] Nondisplaced mandible fractures were the next most frequently missed. These patients did not report malocclusion and were also treated nonoperatively. The one missed operative fracture was also a mandible fracture. This patient was missing most teeth prior to presentation and had no occlusion to report at baseline.

Unfortunately, there are still limitations and challenges to this study. This was a single-center study, with a specific patient population, which may not be applicable in other areas. For example, we had essentially no penetrating trauma. Secondly, we are dependent on documentation and physical exam information from a variety of providers, many of whom have different training backgrounds and may not reliably diagnose specific conditions. Of all the fractures identified with imaging, approximately half were seen acutely by the maxillofacial trauma team and half were not. Our cost analysis is simplified to our institution for representative purposes. A detailed cost analysis, such as microcosting, would require knowledge of different reimbursements from multiple payers and is beyond the scope of this study, Lastly, this is a retrospective study that precludes strong conclusions from definitively being made.

There have now been 4 studies, including this one, to determine whether the “Wisconsin criteria” are a valid decision tool in determining when to obtain facial CT scan[5,6,7] (Table 5). We were able to validate this tool as a sensitive predictor of the absence of fracture and support its use as a decision-making method for obtaining special midface CT scan. Periorbital fractures represent the most difficult to diagnose, and given the low sensitivity of clinical findings, these criteria may result in unnecessary CT scans but will likely not miss a fracture. Lastly, specific situations may warrant simple X-ray imaging. Mandibular trauma without malocclusion or missing teeth may be best studied with orthopantomogram prior to considering CT.

Conclusion

Our results support the use of the “Wisconsin criteria” to assist in the screening of patients with isolated facial trauma and triaging when to order dedicated maxillofacial CT imaging. This may be of special importance in highvolume centers. In the absence of these findings, a midface CT is unlikely to provide additional clinical benefit. Over time, this may lead to a reduction in waste, improved cost savings, and overall enhanced delivery of high-value health care. All patients with facial injuries should undergo the appropriate trauma workup.