A Review of Quality Measures for Assessing the Impact of Antimicrobial Stewardship Programs in Hospitals

The growing problem of antimicrobial resistance (AMR) has led to calls for antimicrobial stewardship programs (ASP) to control antibiotic use in healthcare settings. Key strategies include prospective audit with feedback and intervention, and formulary restriction and preauthorization. Education, guidelines, clinical pathways, de-escalation, and intravenous to oral conversion are also part of some programs. Impact and quality of ASP can be assessed using process or outcome measures. Outcome measures are categorized as microbiological, patient or financial outcomes. The objective of this review was to provide an overview of quality measures for assessing ASP and the reported impact of ASP in peer-reviewed studies, focusing particularly on patient outcomes. A literature search of papers published in English between 1990 and June 2015 was conducted in five databases using a combination of search terms. Primary studies of any design were included. A total of 63 studies were included in this review. Four studies defined quality metrics for evaluating ASP. Twenty-one studies assessed the impact of ASP on antimicrobial utilization and cost, 25 studies evaluated impact on resistance patterns and/or rate of Clostridium difficile infection (CDI). Thirteen studies assessed impact on patient outcomes including mortality, length of stay (LOS) and readmission rates. Six of these 13 studies reported non-significant difference in mortality between pre- and post-ASP intervention, and five reported reductions in mortality rate. On LOS, six studies reported shorter LOS post intervention; a significant reduction was reported in one of these studies. Of note, this latter study reported significantly (p < 0.001) higher unplanned readmissions related to infections post-ASP. Patient outcomes need to be a key component of ASP evaluation. The choice of metrics is influenced by data and resource availability. Controlling for confounders must be considered in the design of evaluation studies to adequately capture the impact of ASP and it is important for unintended consequences to be considered. This review provides a starting point toward compiling standard outcome metrics for assessing ASP.


Introduction
Antimicrobial resistance (AMR) is a growing public health threat which has attracted the attention of national and international bodies. A recent World Health Organization (WHO) surveillance of resistance to antibacterial drugs in bacteria commonly associated with hospital and community infections revealed increasing resistance and/or decreased susceptibilities in the studied bacteria [1]. Resistance of Escherichia coli to third-generation cephalosporins and fluoroquinolones and Staphylococcocus aureus to methicillin (Methicillin Resistant Staphylococcous aureus, MRSA) are reported to be 50% or more in five out of the six WHO regions [1]. Klebsiella pneumoniae resistance to third-generation cephalosporins is reported to be greater than 50% in all six WHO regions. Carbapenem-resistant K. pneumoniae is reportedin all WHO regions, with reports in two regions exceeding 50%. Also, non-susceptibility of Streptococcus pneumoniae to penicillin is reported to be more than 50% in all six WHO regions. A related English AMR surveillance report revealed increased resistance of E. coli and K. pneumoniae to ciprofloxacin, third-generation cephalosporins, gentamicin, and imipenem/meropenem [2]. The report however indicated decreased resistance of Pseudomonas aeruginosa to ceftazidime, gentamicin, and imipenem/meropenem [2]. Recent reports highlightthat patients with infection caused by drug resistant bacteria have a two-fold increase in mortality compared to those with infection with sensitive bacteria [1,3].
Available estimates indicate that between 25%-50% of hospitalized patients receive antibiotics, with between 30% and 50% of antibiotic use being inappropriate [4,5]. Published literature demonstrates a strong link between antibiotic use and the development of resistance [3,[6][7][8]. Antimicrobial stewardship programs are therefore quality motivated interventions aimed at improving the use of antibiotics in healthcare facilities. The primary goal is to optimize clinical outcomes and minimize unintended consequences such as Clostridium difficile infection (CDI) and resistance [4,9]. Strategies to achieve these goals have included prospective audit with intervention and feedback, and formulary restriction and preauthorization. Supplemental strategies include education, guidelines and clinical pathways, antimicrobial cycling and scheduled antimicrobial switch, antimicrobial order forms, automatic stop orders, combination therapy, streamlining or de-escalation of therapy, dose optimization, conversion from parenteral to oral therapy, and computer surveillance and decision support [4,10].
A Cochrane review of interventions to improve antibiotic prescribing for hospital inpatients classified these strategies into three main groups namely: i persuasive interventions: these include education, audit and feedback, guidelines and clinical pathways. ii restrictive interventions: formulary restriction, prior approval or preauthorization from infectious diseases (ID) physician, microbiologist or pharmacists, automatic stop orders, antimicrobial cycling or scheduled switch, antibiotic order forms. iii structural interventions: computerized records, computerized decision support, example computer physician order entry (CPOE) [11].
Effective antimicrobial stewardship programs require a multidisciplinary team with responsibility for promoting prudent antimicrobial use. The Infectious Diseases Society of America and the Society for Healthcare Epidemiology of America (IDSA/SHEA) ASP guidelines [4] recommend a multidisciplinary team which includes an ID physician and a clinical pharmacist with infectious diseases training as core members. Inclusion of a clinical microbiologist, information system specialist, infection control specialist, hospital epidemiologist, and hospital administrator is considered optimal. The English antimicrobial stewardship (AMS) toolkit: "Start Smart, then Focus" recommends other core members should be present, an acute care physician, a surgeon, a senior member of the pharmacy management team, an anesthetist, a pediatrician, and a senior nurse [12].
There is an increased call on healthcare organizations to develop quality measures or indicators to monitor and evaluate the impact of ASP [4,[12][13][14]. Previous reviews have reported on the impact of ASP in reducing antimicrobial cost, AMR, superinfection, and patient outcomes (such as length of stay (LOS), readmission rate, and mortality) [11,15]. The objective of this review was to provide an overview of reported quality measures for assessing ASP and report on the impact of published antimicrobial stewardship studies on these measures, with particular focus on patient outcome measures.

Methods
A literature search of papers published in English between the 1990 and June 2015 was identified through a search of five databases: Scopus, Medline, CINAHL, Pubmed, and Embase using the following search terms: ("antimicrobial stewardship" OR "antimicrobial stewardship program" OR "antibiotic control program" OR "antibiotic policy" OR "antibiotic management program"); (outcomes OR impact OR "quality measure" OR "performance measures" OR "length of stay" OR "clinical improvement" OR "C. difficile infection" OR mortality OR resistance OR readmission OR MRSA).

Study Inclusion and Exclusion Criteria
Only primary research studies, published in English and which met the following criteria were included:

‚
Defined and/or developed quality measures for assessing ASP in hospital settings.

‚
Used quality performance measures (such as change in antimicrobial use) and outcome measures (including resistance patterns, rates of CDI, LOS, readmission, mortality, and cost savings) in evaluating impact of ASP.
‚ Involved adult inpatients in acute and community hospital settings.
Studies excluded were: ‚ Those that reported prevalence of ASP without evaluation of impact.
‚ ASP studies in pediatrics and long term care facilities.

Results
The initial search returned 4319 articles. Of these, 152 met the inclusion criteria, and a full-text evaluation was carried out on 63 studies. In summary, four studies defined quality metrics for evaluating ASP. Twenty-one studies assessed impact on antimicrobial utilization and cost, 25 studies evaluated impact on resistance patterns and/or rate of CDI. Thirteen studies assessed impact on patient outcomes including mortality, LOS, and readmission rates. Table 1 includes studies [16][17][18][19] that developed or defined quality measures for evaluating ASP. Methods used were modified Delphi technique, survey, and interviews. Bumpass et al. [19] surveyed ID physicians' and pharmacists' opinions of AMS metrics considered important in evaluating ASP. The authors reported that although appropriateness of antimicrobial use, infection-related mortality, and antibiotic associated length of stay were considered more important outcomes by those surveyed, antimicrobial use and cost were the most commonly collected metrics.

Impact of ASP on Quality Measures
Impact of ASP on different quality measures is summarized in Tables 2-4. This is grouped into: ‚ impact on antimicrobial use and cost savings ‚ impact on C. difficile infection and resistance patterns ‚ impact on patient outcomes (LOS, readmission rate, mortality) Some of the studies used more than one measure in assessing impact and majority (29) employed before-after or pre-post-intervention (quasi-experimental) design without control. The pre-phase consisted of retrospective collection of baseline data before ASP implementation.

Impact of ASP on Antimicrobial Use and Cost of Antimicrobials
Change in the use of specific antibiotic or antibiotic class is considered a process measure [4,20]. The majority of the programs that assessed impact of ASP on antibiotic use also assessed cost savings. Twenty-one studies assessed impact on antimicrobial use and/or cost. The majority of the studies that reported significant cost savings did not provide the cost of implementing the program. Table 2 summarizes studies  that assessed the impact of ASP on antimicrobial use and cost of antimicrobials.

Impact of ASP on Resistance Patterns and Clostridium Difficile Infection (CDI)
A total of 25 studies  assessed the impact of ASP on microbiological outcomes (bacterial resistance patterns) and/or CDI rates. Thirteen of the 25 studies assessed impact on CDI rates and other outcomes. Nine out of 13 studies reported a statistically significant reduction in the rate of CDI following ASP implementation [41,42,51,52,54,56,58,61,64]. Khan and Cheesbrough [44] and Malani et al. [62] reported a progressive fall in the rate of C. difficile-associated diarrhea (CDAD) and 50% decrease in the likelihood of developing CDI respectively. A restriction policy on ciprofloxacin and ceftriaxone resulted in a 70.20% reduction in the CDI with non-significant effect on extended spectrum beta lactamases (ESBL)-producing colliforms (p = 0.075) [63]-a proxy for antimicrobial resistance development.
Twelve studies assessed the impact on the resistance patterns of organisms commonly associated with hospital infections (ESKAPE: Enterococcus faecium, Staphylococcocus aureus, Klebsiella pneumoniae, Acinetobacter baumannii, Pseudomonas aeruginosa, and Enterobacter species, e.g., Escherichia coli) [67] and reported a reduction in resistance or unchanged susceptibilities in these organisms following AMS interventions. Saizy-Callaert et al. [45] reported a significant fall in the rate of ESBL-producing Enterobacteriaceae (p < 0.001) following the development of a multidisciplinary consultative approach which included developing local prescribing consensus with all prescribers; restricted prescriptions policy; regular audits of use of restricted antibiotics and institutional wide training and information for prescribers. A restriction policy on ceftazidime resulted in a significant decrease in A. baumannii (p = 0.01) [48]. Table 3 summarizes studies that assessed impact on CDI rate and resistance patterns.
A summary of studies that assessed impact on patient outcomes is provided in Table 4.  NS-Non-significant, LOS-Length of stay, ABSSSIs-acute bacterial skin and skin structure infections.

Discussion
Our review identified a number of quality measures used in assessing ASP in primary studies. These include change in antimicrobial use, cost savings, resistance patterns of some difficult to treat organisms, rates of CDI, length of stay (LOS), readmission rate and mortality. These measures were classified into two main categories namely process and outcome measures [20,81]. Change in antimicrobial use (such as total quantity of antimicrobial or targeted antimicrobial class) measured usually in the WHO recommended defined daily dose (DDD)/100 or 1000 patient-days [81] is a process measure [4,20]. Other process measures recommended for use in assessing ASP include documentation of indication for antibiotic prescribed, documentation of stop/review date, 48-72 hours review after initiation of antibiotic therapy, level of adherence to hospital-specific guidelines, level of acceptance of AMS recommendations, time to appropriate therapy in patients with sepsis, and rate of de-escalation of initial therapy [12,14,82]. Outcome measures are categorized into microbiological, clinical and financial outcomes [82]. Microbiological outcomes include measures such as percentage of difficult to treat organisms e.g., MRSA, ESBL-producing Enterobacteriaceae, rate of isolation of resistant organisms, and rate of CDI [83]. Clinical outcome measures used in assessing impact of ASP include all-cause mortality, LOS and readmission rates; clinical improvement and rate of adverse antimicrobial reactions have also been recommended [20,82,84].
Currently, there are no standard, universally accepted metrics for assessing ASP. For example, DDD whilst widely used in quantifying and reporting antimicrobial use continues to be debated because of its limitations [85]. The limitations of using DDD include its inability to provide information on the number of patients actually exposed to antibiotics; it cannot be used for children, and it underestimates the use for drugs that require reduced dosage due to renal impairment [85,86]. Morris et al. [18], in a structured panel to determine quality metrics for ASP, suggested days of therapy/1000 patient-days as a more appropriate measure for public reporting of ASP impact. Similarly, Aldeyab et al. [85] in a study that adjusted DDD to include age-adjusted comorbidity score (DDD/100 bed-days/age-adjusted comorbidity score) concluded that the modified unit provides "an innovative approach to measuring antibiotic use while taking into account the effect of patient case mix". Prescribed daily dose has also been suggested as an alternative or a complement to DDD [20]. Whether these metrics provide the appropriate standards for assessing ASP has not been determined. However, the majority of the studies that reported significant reductions in antimicrobial use employed DDD/1000 or 100 patient-days as the metric [24,33,34].
Assessing the impact of ASP on resistance using the identified metrics has inherent limitations. This is because several factors affect the development of resistance, which makes it difficult to establish a clear causal association between AMS interventions and decrease in resistance [9,20,49]. However, ASP especially those employing restriction on use of 'high-risk' antibiotic classes (second-and third-generation cephalosprins, fluoroquinolones) have been shown to reduce resistance and/or improve bacterial susceptibilities [47,63,65]. Although stewardship interventions have been shown to reduce resistance, their use as a primary measure for evaluating ASP has been cautioned [87]. Rate of CDI has been used as a measure for assessing ASP. Programs incorporating restriction or avoidance of the 'high-risk' antibiotic classes and clindamycin are notably associated with significant reduction in CDI rate [42,44,51,52,54,58,63]. Studies with marked reduction in CDI often also have strict infection control programs in place; which makes the association between ASP and the reduction in CDI rate difficult. However, infection control alone has been shown not to effectively control the outbreak of CDI. A significant reduction in rates followed stewardship interventions that involved restriction or avoidance of the 'high-risk' antibiotics [42,52,63].
Evaluation of ASP requires the use of patient-specific measures that demonstrate attainment of the primary goal. However, some limitations affect effective evaluation. These include difficulty in establishing a clear causal association between ASP interventions and measures such as mortality and LOS due to confounders that affect these measures [9,20]. Mortality related to antimicrobial-resistant organisms and infection-related hospital stay has been suggested as better patient measures for use in assessing impact [18,70]. Lack of personnel, funds, and health information technology personnel, and the inability to generate and analyze ASP-specific data have also been identified as limitations to effective ASP [9,88]. Inadequate study design also limits a clear association between ASP interventions and reported impact. Studies of interventions to improve hospital antimicrobial use are reported to be largely of poor design [11].
This review did not apply the strict quality criteria required for a systematic review of included studies, and risk of bias was not assessed. The purpose of this review was to provide an overview of the quality measures used in assessing ASP in primary studies, therefore all study designs were included. Studies assessing patient specific outcomes were of particular interest. Future work is planned to include evaluation and impact of ASP in pediatric patients.

Conclusions
Patient outcomes need to be a key component of ASP evaluation. The choice of metrics is influenced by data and resource availability. Controlling for confounders and unintended adverse consequences must be considered in the design of evaluation studies to adequately capture the impact of ASP. This review provides a starting pointfor compiling standard outcome metrics for assessing ASP.

Conflicts of Interest:
The authors declare no conflict of interest.