Measuring Antibiotic Stewardship Programmes and Initiatives: An Umbrella Review in Primary Care Medicine and a Systematic Review of Dentistry

Antibiotic stewardship aims to tackle the global problem of drug-resistant infections by promoting the responsible use of antibiotics. Most antibiotics are prescribed in primary care and widespread overprescribing has been reported, including 80% in dentistry. This review aimed to identify outcomes measured in studies evaluating antibiotic stewardship across primary healthcare. An umbrella review was undertaken across medicine and a systematic review in dentistry. Systematic searches of Ovid Medline, Ovid Embase and Web of Science were undertaken. Two authors independently selected and quality assessed the included studies (using Critical Appraisal Skills Programme for the umbrella review and Quality Assessment Tool for Studies with Diverse Designs for the systematic review). Metrics used to evaluate antibiotic stewardship programmes and interventions were extracted and categorized. Comparisons between medical and dental settings were made. Searches identified 2355 medical and 2704 dental studies. After screening and quality assessment, ten and five studies, respectively, were included. Three outcomes were identified across both medical and dental studies: All focused on antibiotic usage. Four more outcomes were found only in medical studies: these measured patient outcomes, such as adverse effects. To evaluate antibiotic stewardship programmes and interventions across primary healthcare settings, measures of antibiotic use and patient outcomes are recommended.


Introduction
Antimicrobial resistance is a serious global public health problem responsible for increased healthcare costs and poorer clinical outcomes [1]. The World Health Organisation (WHO) global action plan on antimicrobial resistance aims "to ensure, for as long as possible, continuity of successful treatment and prevention of infectious diseases with effective and safe medicines that are quality-assured, used in a responsible way, and accessible to all who need them" [2]. All Member States have been urged to develop nation action plans aligned with the objectives of the global action plan [3] and a governance framework of these plans is being produced, including arrangements for monitoring and evaluation [4].
While antimicrobials include antibiotic, antiviral and antifungal medicines, most antibiotic prescribing occurs in primary healthcare [5,6], with dentistry accounting for an estimated 10% of international usage [7,8]. Unnecessary prescribing of antibiotics is widespread with studies in both United States (US) and United Kingdom (UK) showing 80% use not in accordance with guidance [9][10][11][12].
Antibiotic stewardship programmes (ASPs) in primary care are essential to try to curb antibacterial resistance [13]. Antibiotic prescribing behaviour is known to be context specific and ASPs appropriate for use in secondary care services (hospital inpatient and outpatient settings) may be less relevant for primary healthcare services delivered by generalists in community settings [14]. Similarly, evaluation of ASPs and interventions must be careful to ensure they measure both the intended benefits as well as potential unintended negative consequences [15]. Whilst outcome measures of antibiotic usage have been published by international multidisciplinary groups, the authors were aware of a more extensive set of metrics used in some studies of primary healthcare [6,16].
This review aimed to identify a suite of outcome measures used in published studies to evaluate ASPs and interventions across primary care medicine and dentistry. An umbrella review (systematic review of systematic reviews) was used to identify outcome measures from the numerous studies across primary care medicine and a systematic review was used to determine the metrics from dentistry, where fewer studies have been published [14]. Comparison of the outcomes measured between medical and dental settings enabled opportunities to inform the evaluation of new antibiotic stewardship interventions by researchers, and support evaluation of ASPs by healthcare providers in primary medical and dental care.

Protocol and Research Questions
The protocol for this two-part systematic review was registered in PROSPERO [17]. Both reviews conformed to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement. The first research question "What is known from published systematic reviews about outcomes employed to measure outcomes in antibiotic stewardship programmes/interventions across primary medical care?" was addressed by an umbrella review. The second question: "What is known from the published literature about metrics employed to measure outcomes in antibiotic stewardship programmes/interventions for dental care?" was addressed by undertaking a systematic review. Primary care was defined as "the first point of contact in the healthcare system, including general practice medicine and dental services" [14,18].

Search Strategies and Study Selection
In May 2020, three databases were searched from their earliest dates: Ovid Embase, Ovid Medline and Web of Science. The search strategies and terms were developed in consultation with an information specialist at the University of Melbourne. The search terms and strategy for both reviews are shown in Supplementary Material Tables S1 and S2. The search strategies used a "human" search limit as animal studies were not eligible for the reviews and an "English language" limit due to a lack of resources for translation.
Research studies published in peer-reviewed journals were included if they reported measurement of ASP or interventions to optimise antibiotic use in non-specialist primary medical or dental care. Studies related to specialist care delivered in primary care settings (e.g., hospital outpatients and ambulatory care) and/or primary care services delivered in hospital settings were excluded. Studies which did not include antibiotic prescribing were also excluded.
After performing the search using each database, the titles and abstracts were extracted into Endnote X9 and duplicates were removed. Separate Endnote libraries were created for the umbrella review and the systematic review. Two authors (LT and WT) screened all titles and abstracts independently for potential inclusion. Full text of all the shortlisted studies were assessed independently for eligibility by LT and WT. Discrepancies were resolved through discussion. Summaries of the selection processes used for the umbrella and systematic reviews are shown in Figures 1 and 2, respectively. Antibiotics 2020, 9, x FOR PEER REVIEW 3 of 22

Quality Assessment
The Critical Appraisal Skills Programme (CASP) Checklist for Systematic Reviews was used assessing the quality of studies included in the umbrella review (Supplementary Material Table S3) [19]. The validated 16-item Quality Assessment Tool for studies with Diverse Design (QATSDD) for the systematic review for quality assessment of included studies. (Supplementary Material Table S4) [20]. These tools were conducted separately by both LT and WT and discrepancies were resolved through discussion. Studies that scored less than 50% in the QATSSD assessment failed the quality assessment and were not included in the systematic review.

Extraction of Outcome Measures
Measures used to evaluate ASPs and interventions in both the umbrella review in primary medical care and the systematic review of primary dental care were extracted by two authors (LT and WT) independently. Disagreements were resolved through discussion. Categories were developed by one author (LT) to group the outcome measures and checked independently with another author (WT). Outcomes that did not relate to the aims of this review were excluded from the extraction process.

Comparing the Outcome Measures
After completion of extracting and categorising the measures used to quantify the ASP outcomes, these metrics were compared and contrasted across both settings (primary medical care vs dental care).

Study Selection
For the umbrella review across primary medical care, 3052 studies were identified for possible inclusion in the review. After duplicates were removed, 2355 were screened, 21 eligible for full text review, and eleven of these excluded with reasons. This resulted in ten articles that met the inclusion criteria and were of adequate quality ( Figure 1).
For the systematic review across primary dental care, 4234 studies were identified for possible inclusion in the review. After duplicates were removed, 2704 were screened, 11 eligible for full text review, and six of these excluded with reasons. This resulted in five studies that met the inclusion criteria and were of adequate quality ( Figure 2).

Umbrella Review across Primary Medical Care
The characteristics of the included studies in the umbrella review are shown in Table 1 [21][22][23][24][25][26][27][28][29][30]. The included systematic reviews encompassed 109 individual primary research studies with a range of publication dates (1992-2016). Participants were general practitioners, and adults, children and parents of child patients with common infections such as respiratory tract and urinary tract infections. The ASPs and interventions being evaluated included point-of-care testing, the use of real-time epidemiology and pharmacy-led interventions. Analysis of the overlap between these 109 studies found 13 appeared in two or more of the systematic reviews (Supplementary Material Table S5). The characteristics of the five studies included in the systematic review across primary dental care are shown in Table 2 [31][32][33][34][35]. The study participants were all dentists: four of the studies specified general dental practitioners and one did not specify the type of dentists included. The author of the latter study was contacted to confirm that only general dentists participated in the study [35]. The studies were published from 2006-2020. The ASPs and interventions being evaluated included: audit and feedback, education, advice from the health board, dissemination of guidelines and implementation of a dental-specific prescribing tool.

Outcomes Measured in Primary Medical Care
Eight ASPs and intervention outcomes were identified (as shown in Table 3) with a total of 27 individual outcomes measured in primary medical care. Details of the individual factors from of the systematic reviews included in the umbrella review across primary medical care are presented in Table 1. Hu et al., 2016 [25] included 13 studies in their systematic review and 12 in their meta-analysis. Only the results for the latter 12 were presented, so these were included in this review.

Outcomes Measured in Primary Dental Care
Four outcomes measured to evaluate ASPs and interventions were identified, as shown in Table 3. Details of the individual factors from each of the primary research studies included in the systematic review of primary dental care are presented in Table 2. Whilst Seager et al., 2006 [34] attempted to use patient satisfaction as an outcome measure, data collection was discontinued due to a low rate of questionnaire return.

Comparing the Outcomes Measured between Primary Care Medicine and Dentistry
Three outcome measures were found across both primary medical and dental care studies. These all measured antibiotic usage: by quantity (number), rate and quality (including appropriateness).
Four outcome measures were unique to primary medical care and related to patient outcomes and experiences: re-consultation rates, adverse effects, severity of symptoms, and patient reported outcomes such as satisfaction. One outcome measure was unique to dentistry: The confidence and attitude of dentists towards the intervention (Table 3).  The aim of this pilot study was to evaluate a multimodal intervention to improve dental prescribing. The intervention comprised two parts: Targeted education about drug use in dentistry and an online prescribing tool.
Number of antibiotic prescriptions before and after the intervention Number of inappropriate indications for antibiotic prescription before and after the intervention Accuracy of the prescriptions according to the Australian therapeutic guidelines before and after the intervention Confidence and attitude of practitioners towards the online dental prescribing tool.

Discussion
An extensive list of outcomes measured in studies of primary medical and dental care to evaluate ASPs and interventions has been collated. It presents options for researchers testing new interventions in primary medical and dental care as well those wishing to evaluate the implementation of ASPs across primary healthcare. It should also enable the translation of existing metrics to other primary care settings, including from medicine to dentistry. While most ASPs have focused on various healthcare settings (such as hospital, ambulatory and outpatient settings) [6,16,36,37], this is the first comprehensive study to focus on measuring the outcome of ASPs and interventions in non-specialist primary medical and dental care.

Antibiotic Stewardship and Resistance
Whilst antibiotic stewardship was introduced as a way to tackle antibiotic resistance [38], this review has shown that most studies, especially in dentistry, focus primarily on reducing antibiotic usage. The reason is that it is impractical to measure directly a change in the resistance profile of bacteria associated with an antibiotic stewardship intervention and to relate cause (reduce antibiotic prescribing) and effect (reduced antibiotic resistance) [39]. Correlating the timing of the antibiotic exposure to the development of resistance further compounds the difficulty of measuring resistance. Furthermore, the development and selection of antibiotic resistance is influenced by a range of factors, including antibiotic use in agriculture, aquaculture as well as in healthcare. In addition, socio-economic factors may affect rates of resistance, such as overcrowding of urban areas, poor infection control in healthcare facilities and the availability of substandard quality of antibiotics [1,40].
By contrast antibiotic usage is relatively easy to measure and various international studies have demonstrated that a reduction in antibiotic use can result in changed patterns of resistance to antibiotics. A Cochrane systematic review has demonstrated that the prescribing of an antibiotic in primary care for a respiratory or urinary infection results in patients developing bacterial resistance to that antibiotic that may persist for up to 12 months [41]. At the population level, decreased consumption of macrolide antibiotics in Finland resulted in decreased levels of resistance of group a streptococci to erythromycin [42]. It is understandable, therefore, why all of the studies included in these reviews employed measures of antibiotic usage as a proxy measure for tackling antibiotic resistance rather than measuring it directly.

Antibiotic Usage versus Quality of Antibiotic Use
Antibiotic usage as expressed as quantity, rates, percentage and relative ratios was commonly used across both medical and dental primary care. As these outcome measures are easy to interpret and monitor, they are used widely across healthcare settings [39]. A systematic review followed by a multidisciplinary consensus procedure determined a range of metrics to measure quantity of antibiotic use in outpatient settings [16]. These included the defined daily dose (DDD)/1000 patients/day metric, which is often used for international comparison of antibiotic consumption [43] as defined by the World Health Organisation [44], and was also used in the randomised trial employed across all NHS dental practices in Scotland to measure antibiotic use [32]. However, quantity measurements based on the DDD unit is based on standard dosing and regimen to a 70kg adult [16]. It is therefore not suitable for measuring antibiotic use in children and only provides a rough estimate of use and comparisons between countries [16,45]. The difficulty arises as guidelines and practice differ between places and over time. Furthermore, whilst DDD might be useful for researchers and for international comparisons, for practical use in primary care it is less relevant. The number of antibiotic courses started is potentially more relevant than DDD as it indicates the number of patients exposed to antibiotics rather than the total amount of antibiotics used [46].

Clinical and Patient Outcome Measures
A key finding of this study was that none of the dental studies employed clinical or patient outcome measures, such as adverse effects to evaluate ASPs. A study of antimicrobial stewardship in outpatient settings drew similar conclusions, that most of the metrics related to antibiotic usage rather than patient or clinical outcomes [6]. Incorporating patient-related or clinical outcomes alongside measures of antibiotic use brings the metrics identified closest to the definition of antimicrobial stewardship employed in Australia-to reduce harm whilst also curtailing the incidence of antibiotic resistance [47].
Infections of the head and neck may rapidly become life-threatening if they spread into and along the oropharyngeal fascia, such as the pharyngeal space [48]. With the increasing incidence of antibiotic resistant infections, infections which are treated by antibiotics alone (rather than with a surgical intervention to remove the source of the infection) will continue to spread unabated. It is anticipated that there will be an increased in the incidence of systemic conditions such as sepsis and the resurgence of Lemierre's syndrome (a rare but potentially severe condition involving suppurative thrombophlebitis of the internal jugular vein) is anticipated [49,50].
With rates of antibiotic resistance differing between places and over time, and as there is wide variety in the way clinical services are provided in different parts of the world, including between high and low-middle income countries, selecting the right outcome measures for the context is vital. One set of metrics will not suit all primary healthcare settings and so it is recommended that this suite of clinical and patient outcome metrics be used or adapted when evaluating future ASPs and interventions. Whilst many of the clinical and non-clinical factors that influence antibiotic prescribing by clinicians in primary medical and dental care (including patient expectations and workload), there are also differences between the contexts [14]. Further research is indicated to test which of the clinical and patient outcome metrics identified from medical studies will best measure outcomes in the dental context whilst guarding against negative unintended consequences.
Measures of quantity are crucial as they provide baseline measures for monitoring and tracking antibiotic use, but they do not provide insight into the quality of patient care. Reduced antibiotic usage does not always correlate with improved clinical outcomes and the significance of changes in antibiotic prescription rates is not well understood [26]. For this reason, the Infectious Diseases Society of America (IDSM) has recommended using outcome measures that focus concurrently on the quality (appropriate indication and regimen, such as antibiotic type, dose, duration, route) as well as quantity of antibiotic prescribing to ensure that individual patients have been treated appropriately [51]. Similar to other areas of clinical dentistry where patient-centred outcomes to measure various aspects of oral health are recommended [52], metrics that incorporate both antibiotic use as well as clinical outcomes are preferable.
Interestingly whilst all of the studies in this review measured quantity of antibiotic use, only four of the systematic reviews across medical care [24,26,29,30] and three studies in dental care [31,33,35] reported using the metric "in accordance with guidelines". The appropriateness of the guidelines will therefore affect the quality of prescribing. Guidelines also change over time: The recent Australian dental antibiotic guidelines have recommended an extended spectrum of antibiotics coverage for acute odontogenic infections that is in contrast with the remainder of the international dental community [53]. Guidelines also vary between countries; the antibiotic prophylaxis guidelines for infective endocarditis in the UK [54] is different compared to Australia [55] and the US [56]. It is therefore recommended that indicators of both quality and quantity should be used to evaluate ASPs and interventions. More than one quantity should be considered to gain a better understanding of antibiotic use [16].
Incorporating clinical outcomes in addition to antibiotic use and quality antibiotic use is important to provide further insight to the effectiveness of the ASPs and interventions on individual patient care. For example, a recently published dental intervention pre-post study showed a decrease in antibiotic use and an overall decrease in inappropriate prescribing, but a significant number of prescriptions were still for inappropriate indications [35]. Clinicians have poor adherence to guidelines as socio-behavioural theory has been shown that they tend to form internalised, tacit guidelines, continuously modified by personal experience, leaders in the field and continuing education [57]. Several factors influence antibiotic prescription and are different for each context [14]. Due to high rates of unnecessary antibiotic prescribing, it is expected that ASPs and interventions that target inappropriate prescribing would naturally see a concurrent reduction in antibiotic use. Furthermore, since it is accepted that the use of broad-spectrum antibiotics contributes more towards antibiotic resistance due to the increased selection of bacteria, targeting the use of broad-spectrum antibiotics as illustrated in one of the systematic reviews of medical care seems a sensible strategy [29].

Limitations
Publication bias and the possible exclusion of studies in languages other than English are limitations for both the umbrella and systematic review. However, it is unlikely that this would have provided other outcome measures as most studies already used overlapping metrics. Since most studies in the dental systematic review comprised of multifaceted interventions, it is not possible to separate the individual components of the intervention and relate them to a specific metric. The broad range of methodologies in both reviews and heterogeneity of trial designs made it difficult to directly compare the utilisation and choice of specific outcome metrics. Some reported outcomes were subjective such as severity of symptoms, belief in the effectiveness of antibiotics, and thus at risk of reporting bias [22]. Outcome measures that were not considered to be of direct relevance to the aims of the study were also excluded, such as patient and parental knowledge measured by interviews in the patient's home after the initial consultation [22]. In addition, some reviews included in the umbrella review reported that the included studies were set in western, European countries and US primary care so may not be generalisable to other low-or middle-income countries or settings [22,24,25].
To maximise benefits from implementing an intervention to change prescribing behaviour, interventions need to be tailored to the specific context to target the specific delivery method that is best for the specific group of clinicians [27]. Nonetheless, it may be possible to translate the comprehensive list of outcome measures across primary medical and dental care and to adapt them further across other primary healthcare settings. Assessing effects using randomized intervention studies adhering to an implementation framework would be ideal [58].

Conclusions
A suite of clinical and patient outcome measures used to evaluate ASPs and interventions in primary medical and dental care has been collated. This will be a useful resource for those selecting appropriate measures to monitor and evaluate the delivery of ASPs in primary healthcare contexts across medicine and dentistry. Employing a mix of clinical and patient outcomes, and quantitative and qualitative measures is recommended. The outcomes selected for measurement should address both delivery of the intended benefits as well as avoidance of negative unintended consequences.
Supplementary Materials: The following are available online at http://www.mdpi.com/2079-6382/9/9/607/s1. S1: Example of the search strategy used to identify relevant papers for the umbrella review across primary medical care, S2: Example of the search strategy used to identify relevant papers for the systematic review across primary dental care, Table S1: Methodological quality assessment of systematic reviews included in the umbrella review across primary medical care using the Critical Skills Appraisal Programme (CASP) checklist for systematic reviews, Table S2: Methodological quality assessment of studies included in the systematic review across primary dental care using the Quality Assessment Tool for Studies with Diverse Design (QATSSD), Table S3: Mapping the ten systematic reviews of the umbrella review to their 111 constituent primary research studies.