Efficiency of Orthopaedic Audits in a Level-1 Trauma Centre Using a Modified Clavien–Dindo Complications Classification

Abstract

Background: Complications are an inevitable part of orthopaedic surgery, and how one defines complications can have an impact on the ability to learn from them. There have been issues with a commonly used classification system first outlined by Clavien and Dindo. Our aim was to evaluate a modification of this classification system developed for use in our department, with our hypothesis being that this could make our audit presentations more efficient. Methods: A modified Clavien–Dindo Classification was prospectively applied to all complications recorded in the orthopaedic departmental audits at our institution for a 12-month period. The audit discussion was recorded and analysed and compared with the complication audits for the preceding 12-month period. Results: We analysed eight audit presentations before and eight audit presentations after the introduction of a modified Clavien–Dindo classification to our orthopaedic complications over a 2-year period. The number of PowerPoint™ slides (103 vs. 84, p = 0.03) and the time spent on each presentation (88 vs. 71 min, p = 0.02) decreased significantly with the introduction of the modified classification. The novel system was found to have high inter-observer reliability. Conclusions: The modified Clavien–Dindo classification system is a reproducible classification system for orthopaedic complications. It made our audit presentations more efficient.

1. Introduction

Complications are an inevitable part of orthopaedic surgery, but how one defines a given complication can impact on what learnings may be taken away from a given case. Clavien and Dindo et al. previously published their classification system for surgical complications, which is based on the type of therapy required to correct the complication [1].

Table 1. Original classification of surgical complications as per Clavien–Dindo [1].

Grade	Definition	Examples of Therapy
I	Any deviation from normal post-operative course without the need for pharmacological, surgical, endoscopic, or radiologic treatment	IV fluids, antipyretics, anti-emetics, electrolytes, physiotherapy
II	Requiring pharmacological treatments other than those allowed in grade I	IV antibiotics, blood transfusions, parenteral nutrition
III	Requiring surgical or endoscopic treatment III-A: intervention not under GA III-B: intervention under GA	Return to theatre for washout of septic wound, gastrointestinal endoscopy, radiological angioembolisation
IV	Life-threatening complication requiring specialist intensive care management IV-A: single-organ dysfunction IV-B: multiple-organ dysfunction	Admission to ICU for haemofiltration or ventilation
V	Patient mortality	n/a

GA = general anaesthetic, IV = intravenous, and ICU = intensive care unit. n/a = Not applicable. In grade V complications there is no therapy possible as the outcome is patient mortality.

outlines the classification system in its published format. The system was developed for a general surgical population in response to growing concerns that surgical classifications were becoming increasingly diverse and hard to interpret. This system is useful in that it avoids subjective and meaningless terms such as “minor”, “moderate”, and “severe”. The system was originally validated on a population of 6336 general surgical patients at multiple hospitals [1], then re-validated five years later on a similar but larger patient population and across multiple hospitals and countries [2].

Since its first publication in 2004, the Clavien–Dindo classification system (CDS) has been widely adopted across multiple surgical specialties and has been the basis of results in randomised controlled trials (RCTs) [3,4,5]. It has also been validated in both paediatric orthopaedic surgery [6], vascular surgery [7], oral and maxillofacial surgery [8], and spinal deformity correction surgery [9,10]. Specifically, in an orthopaedic context, the complications of surgical dislocation of the hip have been evaluated by Sink et al. [11]. They evaluated 302 patients undergoing 334 hip procedures over eight North American treatment centres and found an overall complication rate of nine percent; many of these cases were clinically unimportant heterotopic ossification cases [12]. They applied the original, validated Clavien–Dindo classification system to their patient cohort. This was the first use of this classification system in orthopaedic surgery [11]. But their paper used the original classification system in an elective surgical population. It was not adapted for use in an orthopaedic trauma population. Due to this study, many orthopaedic departments now refer to the classification system as the “Clavien-Dino-Sink” classification system [13,14,15,16].

Guisse et al. have also recently published their results in the setting of adolescent idiopathic scoliosis (AIS) [9]. Their paper described a modified Clavien–Dindo–Sink classification system applied to an AIS patient group undergoing posterior instrumented spinal fusion. The authors modified the existing classification system after reviewing an AIS multi-centre database of over 8000 patients. They grouped the complications into the following categories: Death, Gastrointestinal, Instrumentation, Medical, Neurologic, Spinal Cord, Pain, Pulmonary, Pseudoarthrosis or Nonunion, Reoperation/Revision, Surgical Sites/Incisions, and Transfusions. They modified the original Clavien–Dindo classification (Table 1) to be more inclusive of all complications, with a particular focus on those which caused an increase in hospital length of stay as a descriptor for grade II classifications. The authors found that their modified system was had high inter-observer reliability, with a kappa value of 0.9 [9]. This research shows that the modification of an existing system can still be a reliable method of classification in terms of inter-observer agreement.

The use of the CDS in surgical practice has been increasing. The objective nature of the classification system (i.e., the therapy used to correct the complication informs how complications are graded) is appealing [17]. As such, in our department, we have been using the original Clavien–Dindo classification system for several years in our morbidity and mortality audit meetings. Two meetings are conducted every three months, meaning there are eight such meetings each year [18]. Each case is graded I–V according to the original classification (Table 1). Our department operates in a level-1 trauma centre, as a large percentage of our patients are polytrauma patients. Anecdotally, we have found that for our orthopaedic trauma patient population, this classification is lacking specificity, with most complications falling into grades II or III, from a standard post-op blood transfusion through to multiple revision surgeries for an infected arthroplasty. The danger with a classification system such as this is that because the categories are broad and inclusive, complications tend to be recorded and presented, and they can appear heterogeneous in an audit presentation.

This meant that our audit presentations would often be inefficient, with lots of time spent presenting and discussing complications which were either expected or not preventable but needed to be discussed as they would fall into the aforementioned grade II or III in the Clavien–Dindo classification, and by virtue of this fact, they had to be presented and given the same emphasis as something which merits thorough discussion or a case where a complication is preventable—these are the cases where clinical improvements can be made. This often meant that other parts of the audit cycle, such as discussions about pharmacy errors, presentations from nursing and allied health staff, and preventable orthopaedic complications, suffered due to a lack of available time during meetings.

Hence, a new classification system was sought to try to improve the efficiency at which complications could be presented yet still cover the important points for each case. The senior author (SL) proposed a modification of the CDS for use in our hospital’s (level-1 trauma centre) department audit. This was rolled out after discussion and agreement with the surgical consultants within the unit.

This research sought to evaluate the use of this modified CDS via a prospective evaluation. The primary aim of our study was to evaluate whether the application of a modified Clavien–Dindo classification for orthopaedic auditing would make our audit presentations more efficient in terms of the time spent presenting/discussing complications. The secondary aim of the research was to test the inter-observer reliability of this modified CDS, as has been carried out elsewhere [19].

2. Materials and Methods

2.1. Modified Classification System

The senior author (SL) devised a modification of the Clavien–Dindo complications system and sought consensus agreement on subclassifications from other consultant surgeons in our department prior to its implementation in departmental audit presentations. The revised classification system can be found in

Table 2. Modified Clavien–Dindo classification system employed in our institution.

Modified Grade	Description
Grade I	Any deviation from post-op course without the need for non-standard intervention.
	Allowed regimens: Antiemetics, analgesics, antipyretics, electrolytes, and physiotherapy
Grade II	Complication requiring pharmacological treatment (other than allowed drugs in grade I)
	Examples: Blood transfusions, TPN, IV antibiotics
Grade III	Requiring surgical, radiological or endoscopic intervention
	III-A: NOT under general anaesthesia
	III-B: Under general anaesthesia
Grade IV	Life threatening complication requiring immediate care
	IV-A: Requiring pharmacological treatment on ward alone (+/− ICU outreach)
	IV-B: Requiring ICU admission (e.g., for inotropic support)
	IV-C: Requiring specialist ICU intervention (e.g., ECMO, haemofiltration, etc.)
Grade V	Patient mortality
	V-A: Expected or non-preventable
	V-B: Unexpected or preventable
VTE	Additional classification for any complication, including a thromboembolic event (VTE/PE)
	VTE-A: Prophylaxis was prescribed and in line with local guidelines
	VTE-B: Prophylaxis WAS NOT prescribed or WAS prescribed but not in line with local guidelines

Each case is graded according to the above based on the therapy required to correct it. Grades III, IV, and V all have sub-grades depending on the specifics of the case. TPN = total parenteral nutrition, IV = intravenous, ICU = intensive care unit, ECMO = extra-corporeal membrane oxygenation, and VTE = venous thromboembolism.

. We sought to modify the existing CDS to reflect the spectrum of complications in our trauma population. Our modification expands upon grade IV complications, with the addition of specialist ICU intervention such as extra-corporeal membrane oxygenation (ECMO) or renal replacement therapy, it also subclassifies grade V into preventable and non-preventable deaths. These specific subclassifications were added as we identified that the cases falling into these categories were those taking up much of the audit time, and thus, we attempted to streamline our audit process by adding these subclassifications.

We defined a non-preventable complication as one where despite best medical treatment, the outcome would have been the same. This was initially determined by the lead author, validated by the senior author, and confirmed by department consensus during the audit presentation. We have also added separate categories for thromboembolic complications and for whether there was a prescription of prophylaxis (either mechanical or chemical) to allow us to evaluate thromboembolic complications with regard to our institutional protocols and, hence, assess compliance. In each complication that involved a thromboembolic complication, the VTE-A/B classification system was assigned to a case in addition to the standard I–V grading.

2.2. Development and Implementation

Permission from our hospital ethics institutional review board (IRB) was granted for the conduction of this study without the need for formal ethical approval, as there was no individual patient information or recruitment of human subjects. The modified Clavien–Dindo complications scale was implemented during our departmental quarterly audit meetings, of which there were two per month, from Q2 2021 onwards for a period of 12 months. We compared this 12-month period with the preceding year, which served as a “control” group. This means we analysed eight audit presentations before (Q2 2020–Q1 2021) and eight after (Q2 2021–Q1 2022) the introduction of the modified CDS classification system. During the second study period, the presenters of the audit meetings were aware of the new classification system but were blinded to the fact that it was being studied prospectively, reducing the potential effect of Hawthorne bias [20].

2.3. Research Question and Data Recording

We were interested in how efficient the audit presentations were, and as such, the primary outcome recorded was the number of PowerPoint™ slides in each audit presentation and the time (in minutes) spent on each quarterly presentation. We decided to use these proxy measures for audit efficiency for ease of data collection. During the study period, at the height of the COVID-19 pandemic, all audit presentations were conducted remotely and able to be recorded. Hence, this information was easily retrievable from recordings. All audit presentations (PowerPoint™ files) were archived in our departmental records and retrieved for the purposes of this research. This information was recorded using audit presentation files from our departmental archives and online meetings recorded using Teams software Version 1.7.00.9457 (Microsoft, Redmond, WA, USA). The secondary outcome was the proportion of time spent on each audit presentation on complications. We tested the inter-observer agreement of the classification system after a review of the audit presentation slides; two independent observers reviewed each case in the study period and classified them according to the definitions of the new system, shown in Table 2. In the case of a dispute, the senior author had the power to make the final decision.

2.4. Data Analysis

Continuous variables were compared between cohorts using a paired Student’s t-test. All data were tabulated and analysed using Microsoft Excel™ software Version 16. Inter-observer reliability was assessed using a percentage of cases where there was total agreement.

3. Results

We analysed the PowerPoint™ slides and online recordings of a total of 16 audit presentations: 8 presentations from before and 8 presentations from after the introduction of the new classification system. The characteristics of these audit presentations is presented in

Table 3. Comparison of audit presentations during the two study periods.

	Original CD Scale (Q2 2020–Q1 2021)	Modified CD Scale (Q2 2021–Q1 2022)
Number of audit presentations	8	8
Number of total complications	176	165
Number of complication cases presented at each meeting	21.63 ± 3.2	20.63 ± 3.1
Number of slides in each presentation	103.13 ± 10.50	84.36 ± 3.35 *
Time spent for audit (mins)	87.66 ± 13.80	71.72 ± 3.05 ^
Proportion of audit time spent discussing complications (%)	53 ± 4	37 ± 3

Values presented represent mean ± standard deviation. * p = 0.03; ^ p = 0.02. CD = Clavien–Dindo.

. During the study, we recorded a total of 341 complications, with 176 being recorded in the first study period and 165 being recorded in the second study period. The distribution of complications by classification during our research period from Q2 2020 to Q1 2022 can be seen in

Table 4. Distribution of classification cases audited across the two different study periods.

Original CD Scale (Q2 2020–Q1 2021)		Modified CD Scale (Q2 2021–Q1 2022)
I	20	I	23
II	45	II	40
III	70	III A	8
IV	16	III B	58
V	25	IV A	7
		IV B	6
		IV C	0
		V A	6
		V B	17
Total	176	Total	165
		VTE A	32
		VTE B	12

For complications where a thromboembolic event was identified, the subclassification of VTE-A/B was applied in addition to the I–V grading. This was to assist in compliance with local protocols. Hence the total number of complications is displayed above these data. CD = Clavien–Dindo; VTE = venous thromboembolism.

. During the second audit period, we recorded 44 complications relating to either venous thromboembolism (VTE) or pulmonary embolism (PE). These were additionally classified according to VTE-A/B as per Table 2; thus, 12 of the 44 (27%) cases were not prescribed venous thromboprophylaxis in accordance with our hospital guidelines.

There was a statistically significant decrease in the number of slides presented per audit after the introduction of the new classification system (103 vs. 84, p = 0.03), and the total time spent on each audit presentation (88 vs. 71 min, p = 0.02) (Figure 1 and Figure 2).

The proportion of each meeting spent on complications was recorded and tabulated. In the first study period, over half (53%) of the audit presentation, on average, was dedicated to the discussion of complications. After the implementation of the new classification system, 37% of our meeting time was dedicated to complication discussion (p = 0.04).

In terms of assessing the inter-observer reliability of the new classification system, in 94% of cases, both observers agreed with the classification of the case according to the new system when reviewing the PowerPoint™ slides presented at the audit meeting. In the other 6%, there was disagreement, and the final decision on classification was made by the senior author.

4. Discussion

Our research suggests that a modified Clavien–Dindo complication classification system could be applied at our hospital for audit presentation. We found that it reduced the time spent on discussing orthopaedic complications by 19 PowerPoint™ slides and 17 min per presentation, respectively. The number of complications between the two study periods was not different, and the spread of complications across classification grades was not different. The proportion of each audit meeting spent discussing surgical complications decreased significantly from 53% to 37%, indicating that the reduction in total meeting time was attributable to this change. We believe that our modified classification system improved the efficiency of our audit meetings, in line with previous research [21].

Audits of surgical complications prior to the CDS lacked standardisation and relied on subjective clinician-initiated interpretations of the cases. Subjective terminology such as “mild”, “moderate”, and “severe” was often used [22]. Despite the introduction of the classification system in its original form [1], it did not receive widespread adoption. The authors continued their study on complications and then proposed a modification of their original system, removing subjective terms such as “minor” and “major”. They increased the number of categories from five to seven to allow for more precisely defined groups, and they removed length of stay from the classification system, purportedly to allow for better inter-specialty comparisons [2]. In the first five years following the publication of this second article, the CDS was used in 214 published papers, with the highest uptake being in the field of transplantation [23]. Certainly, in the general surgical fields, it is the standard way of reporting complications for the purpose of scientific research.

Ongoing quality assessment is a core tenet of our practice as surgeons, and it forms part of the “core competencies” of the Royal Australasian College of Surgeons and is essential for trainees of all nine surgical specialties in Australia and New Zealand [24]. Any evaluation of surgical competence relies on a uniform way to report negative outcomes. This is crucial to gather credible data and benchmark between institutions. In light of the decreased mortality with many common surgical procedures, it is critical to record complications accurately because small variability may lead to large statistic variations [25]. A contemporaneous, locally derived classification system such as ours allows for the objective recording of orthopaedic complications by the treatment needed to correct the complication. We have used it as a standardised method within our department across different audit periods and found it to be reliable when applied to our population of orthopaedic patients in a level-1 trauma centre. As recording is standardised, there is also potential for the standardised recording of complications across different departments or health networks. With most health networks moving toward an electronic medical record system in recent years, this opens potential cross-network collaborative opportunities.

A good classification system narrows room for interpretation, which the CDS does. We have tailored this classification scale to the specific needs of the audit in our department. By modifying the existing, proven classification system slightly [1,2], we feel we have achieved both relevance for our population and maintained the reliability of the original classification system. We added a separate category for venous thromboembolism (VTE) in our classification because it is very common in our trauma population, and often different specialties (neurosurgery, cardiothoracics) generate competing interests regarding the most appropriate prophylactic regime. The most common scenario is that a cranial injury precludes VTE chemoprophylaxis. Hence, we added this category as a way to monitor local compliance with our guidelines. Venous thromboembolism is a major problem in the polytrauma population, and we felt deserved special attention when revising this classification system [26]. The CDS has been shown to be efficient in its use, reproducible, and logical [27]. It has also been shown to be reliable when translated into other languages, with a kappa index of 0.85—indicating a strong correlation between English and Spanish versions [23]. It is a useful classification system in that it has been shown to predict the occurrence of negative outcomes in papers examining surgical resection for gastric cancer, with a higher CD grade predicting disease-free survival (p < 0.001) [28]. It has also been shown to be related to hospital length of stay [29] and health-related quality-of-life measures in the peri-operative period (<6 weeks) [30].

In our CDS modification, we included subcategory modifications in grade IV and V to be more specific with regard to each case. With regard to “expected” or “non-preventable” mortality, unfortunately, with our population of high-energy trauma patients, despite maximal treatment, the outcome of death is very likely on presentation to hospital, and these were the cases given a classification of grades V-A. Thankfully, these cases are the minority. These cases are, of course, still discussed in our complications audits, but we feel that where the outcome was expected, there is less opportunity to draw a learning point from the cases, and so we try to be more efficient with the time spent on these cases [31].

Our revised classification system has been shown to make our audit presentations more efficient, with this improvement being defined by the time taken for the presentations and the number of slides presented. This is despite static case numbers over our study period. As we are a level-1 trauma centre, the bulk of our operative load was trauma rather than elective surgery, and hence, the effect of COVID lockdowns on our case numbers was not significant. The saving of 17 min for each audit presentation allows our team to focus on all areas of the audit presentation equally rather than complications taking up more than half of every meeting. Particularly, the subclassification of mortality in grade V allows us to be more direct with the discussion of cases of severely injured polytrauma patients for whom the outcome of mortality is very likely at the time of presentation to hospital. By spending less time discussing surgical complications via only focussing on the cases where there is potential for change or learning points, there is more time in the audit meetings for multi-disciplinary discussions with our colleagues from other disciplines or specialties, which we feel are important for fostering teamwork and promoting important multi-disciplinary links within our unit and with our colleagues [32].

Our research is limited in that it is based on a small sample size, includes some retrospective elements, and is representative of a single-institution study. Further prospective data collection would strengthen the results of the research. A strength of this study is that the presenters during audit presentations were aware of the new classification system being introduced to the unit but not made aware of the ongoing research to limit Hawthorne bias. The applicable lessons from this research are that the existing frameworks for surgical complications can be altered to suit local patient populations and practices.

5. Conclusions

A modified Clavien–Dindo classification for surgical complications is applicable to orthopaedic complications in a level-1 trauma centre population. It allows for a more efficient audit, according to our study using the proxy measure of PowerPoint™ audit slides and time spent discussing complications.