Clinical Outcomes Used in Clinical Pharmacy Intervention Studies in Secondary Care

The objective was to investigate type, frequency and result of clinical outcomes used in studies to assess the effect of clinical pharmacy interventions in inpatient care. The literature search using Pubmed.gov was performed for the period up to 2013 using the search phrases: “Intervention(s)” and “pharmacist(s)” and “controlled” and “outcome(s)” or “effect(s)”. Primary research studies in English of controlled, clinical pharmacy intervention studies, including outcome evaluation, were selected. Titles, abstracts and full-text papers were assessed individually by two reviewers, and inclusion was determined by consensus. In total, 37 publications were included in the review. The publications presented similar intervention elements but differed in study design. A large variety of outcome measures (135) had been used to evaluate the effect of the interventions; most frequently clinical measures/assessments by physician and health care service use. No apparent pattern was established among primary outcome measures with significant effect in favour of the intervention, but positive effect was most frequently related to studies that included power calculations and sufficient inclusion of patients (73% vs. 25%). This review emphasizes the importance of considering the relevance of outcomes selected to assess clinical pharmacy interventions and the importance of conducting a proper power calculation.


Introduction
Suboptimal choice of outcomes to assess health care interventions may result in lack of implementation of potentially effective interventions, which could have benefitted the care of patients.
Traditionally, new interventions and services in health care have been implemented if they seemed reasonable, but in recent times with scarce resources, documentation of (cost) effect is essential before implementing a new service. Clinical pharmacy services, including medication reviews, are among many other interventions exposed to documentation of the suggested effect, and indeed, systematic reviews have found some effect of clinical pharmacist interventions in inpatient care [1][2][3][4][5]. However, evaluation of clinical pharmacy services is challenging due to the interventions often being complex and non-specific, and the purpose is often to optimise the use of medications, reduce medication-related risks and improve symptom control [6,7]. Consequently, choice of outcome measures is difficult.
However, choice of outcomes is not the only challenge when conducting outcome research; other essential components include quality of the study, study design, type of intervention, the patient population, etc. [8]. The Donabedian framework is frequently used to evaluate clinical pharmacy services. The model consists of three elements; structure, process and outcome. Structure is the context • described primary research • were published in English • described interventions delivered by clinical pharmacists Publications were excluded if they: • were not published as a research paper (e.g., reviews, books, congress abstracts, posters, reports, protocols) • did not include outcome data • presented data for a secondary study, where the original study had been published previously • had been conducted in primary care • included 100 patients or less The search was performed for the period up to 2013 using PubMed (TRHN).

Assessment
All titles and publication types from the original search were reviewed independently by TRHN and LJK. Subsequently, abstracts were reviewed by the two authors. Thereafter, full-text articles were reviewed independently by CO and LJK. Finally, CO and LJK extracted data form the studies independently. At every step, disagreements were resolved by consensus. The data extracted were details regarding the study, the intervention, outcomes and power calculation.
For each included study, the variable used for power calculation was categorized as "primary outcome" irrespective of whether it was stated to be the "primary outcome" by the authors. Also, when more than one variable was stated to be "primary outcome" by the authors, only variables supported by power calculations were categorized as "primary outcome". In contrast, if no power calculation was presented and no primary endpoint was stated, all outcomes were categorized as "secondary outcomes" irrespective of the authors stating otherwise. Some measures were excluded due to assessing qualitative aspects or being descriptive: Number of drugs, drug-related problems (DRPs), acceptance rates, medication knowledge if not assessed using a validated tool, drug burden index, inhalation technique, medication errors unless linked to an event/clinical assessment, drug attitude, quality of well-being, appropriateness of prescribing of individual drugs, self-reported asthma symptoms.

Study Selection
A total of 672 studies were identified in the PubMed search ( Figure 1). After removing 11 papers due to duplicate publication and non-English language, in-and exclusion criteria were applied to 661 unique publication titles and subsequently to 432 unique abstracts ( Figure 1). Of these, 241 full-text publications were reviewed, and 204 were excluded due to: Study conducted in primary care (n = 90), outcomes not clearly presented (n = 7), ≤100 pts (n = 98), and secondary article (n = 9). Finally, 37 unique publications were included in the review . Two publications were based on one study, but since different outcome measures were presented in the respective papers, both were included [33,34].
Some measures were excluded due to assessing qualitative aspects or being descriptive: Number of drugs, drug-related problems (DRPs), acceptance rates, medication knowledge if not assessed using a validated tool, drug burden index, inhalation technique, medication errors unless linked to an event/clinical assessment, drug attitude, quality of well-being, appropriateness of prescribing of individual drugs, self-reported asthma symptoms.

Study Selection
A total of 672 studies were identified in the PubMed search ( Figure 1). After removing 11 papers due to duplicate publication and non-English language, in-and exclusion criteria were applied to 661 unique publication titles and subsequently to 432 unique abstracts ( Figure 1). Of these, 241 full-text publications were reviewed, and 204 were excluded due to: Study conducted in primary care (n = 90), outcomes not clearly presented (n = 7), ≤100 pts (n = 98), and secondary article (n = 9). Finally, 37 unique publications were included in the review . Two publications were based on one study, but since different outcome measures were presented in the respective papers, both were included [33,34].

Description of Studies
The included studies had been conducted in 16 countries in Europe, Asia, Australasia, Middle East and North America, and most frequently in the US with ten studies ( Table 1). The majority of the studies had been conducted at one hospital (n = 30), but four studies included patients from three hospitals and one from 10 hospitals ( Table 1). Number of patients included in the study ranged from 105 to 4290 (Table 1). The type of wards and study populations varied considerably, but the majority included patients were suffering from a chronic disease (Table 1).
A traditional randomized, controlled design was applied for the majority (n = 26) of the studies ( Table 2). The interventions provided appeared similar but differed in types of elements. However, more than half of the studies (n = 20) included a combination of patient counselling, medication review and interdisciplinary collaboration ( Table 2). Only two studies were finalised with no further follow up at discharge [38,48] (Table 2). All other studies presented interventions which included post-discharge contact with health care professionals or follow-up for effect evaluation-or both-and two studies described interventions with a duration of two years [20,49].

Description of Outcome
The included studies used a plethora (135) of outcome measures to evaluate their interventions ranging from two [15,46] to 13 [14] ( Table 3). The most prevalent measures included laboratory measures, clinical measures/assessments by physician and health care service use, however, a large variety of measures within the categories were used. A mixture of generic and disease specific measures was reported ( Some of the studies had selected a primary outcome measure directly related to medication use and knowledge [21,32,34,36,41,44,45,47,50], while others chose measures which may be consequences of the interventions (e.g., laboratory tests, hospital readmission and mortality [14,[16][17][18]20,22,23,[25][26][27][29][30][31]35,38,[40][41][42][43]49]). Adherence, HbA1c values, LDL values, emergency department visits, and hospital readmission were used as primary as well as secondary outcomes.            No apparent pattern was established among primary outcome measures with significant effect in favour of the intervention.
More than half (n = 21) of the studies did not present any power calculation (n = 13) or did not include sufficient patients according to their power calculation (n = 8) (Table 3). Of the 26 primary outcome measures showing a statistically significant effect, 73% reported a power calculation and included sufficient patients according to the power calculation. Only 25% of the 16 primary outcome measures with no statistically significant effect reported a power calculation and included a sufficient number of patients (Table 3).

Discussion
The literature review included 37 publications worldwide describing quite similar intervention elements but differing in study design. A large variety of outcome measures had been used to evaluate the effect of the interventions; most frequently clinical measures/assessments by physicians and health care service use. No apparent pattern was established among primary outcome measures with significant effect in favour of the intervention, but positive effect was most frequently related to studies that included power calculations and sufficient inclusion of patients.

Outcome Measures
The large variety of outcomes used in the included studies may be explained by the lack of consensus of optimal outcome measures for this type of intervention [11,12].

Generic Versus Disease Specific Tools
Since the interventions are usually complex and the patient populations are often heterogeneous, optimal outcome measures to ensure comparison between studies should be generic. Indeed, numerous generic measures were included in the studies (e.g., adherence measures, ADEs, service use and HRQoL). However, diverging methods were used (e.g., for assessment of adherence (self-reported and objective)), a variety of elements were used (e.g., to assess ADEs (potential and preventable)), different time periods were used (e.g., for assessment of emergency department visits (3 days, 30 days 12 months)) and various tools were used (e.g., for assessment of HRQoL (SF 12, SF 36, self-rated global health)). Even if similar interventions are selected, comparison between the studies would be complicated by differences in type of outcome measure-and design, inclusion criteria, etc.
The large number of disease-specific tools reported as outcome measures may derive from an expectation of these being more relevant for the particular cohort (diversity of patients across studies)-and perhaps an expectation of these measures being more sensitive to change, than generic measures.
Mortality/survival was reported as outcome measures in six studies. The only study providing a power calculation and including sufficient patients showed a positive effect on "Time from randomization to death from any cause" [49]. The continuous variable may be an easier way to evaluate a rare event such as mortality, which usually requires large sample sizes or long follow-up periods to ensure sufficient power [7,8]. However, the aspect of time of follow up is important, since there is a risk of a short follow up resulting in insufficient data (few patients have died) as well as excessive (most patients have died), and this time period is likely to vary according to the characteristics of the included patients. This further complicates the comparison between studies. Hence, survival analysis may be the optimal measure for this outcome. When no effect on an outcome is found in studies with insufficient power, it may be interpreted as "evidence of absence" as in a Cochrane review, while the interpretation should be "absence of evidence" due to lack of power in the included studies [2,51].

Primary Versus Secondary Outcomes
Primary outcomes are used to determine the effect of the intervention, while secondary outcomes evaluate additional effects of the intervention. However, power calculation is only done on primary outcome measures [13]. The number of outcome measures used in the included studies varied considerably (2)(3)(4)(5)(6)(7)(8)(9)(10)(11)(12)(13), which may be explained by different needs to determine additional effects of the individual interventions. Laboratory measures, clinical measures/assessments by physician and health care service use were prevalent measures, which may be explained by these measures often being documented as a part of routine patient assessment, and hence easy to collect. Still, they seem to be relevant outcome measures to assess the effect of the studies.

Target Groups for Results
Another reason for selecting several outcome measures may be the importance of evaluating the intervention with respect to different stakeholders. The importance of an effect may vary according to the perspective, (e.g., patient, care-givers, health care professionals, decision makers and researchers) may not agree on, which outcome measure is the most important [8].

Relevant Outcomes
Further discussions about which outcomes may be relevant to quantify the desired effects of clinical pharmacy interventions are needed. It is important to consider whether an effect can indeed be expected on the selected outcomes [8,11,12]. New approaches to standardize outcome measures in clinical trials are emerging, and the results of this review confirm the need for a standard set of core outcome measures [11,12]. If the aim of clinical pharmacist interventions is to improve symptom control, reduce medication-related risks, improve benefits of medication use and prevent development of conditions, it is possible that outcomes such as preventable adverse drug events, measures directly related to medication use and knowledge, and other soft endpoints are likely to be more appropriate than hard endpoints such as mortality and hospital readmission, since they measure aspects which may be affected by the interventions [8]. A variety of these measures have been used as primary outcome measures in the included studies with varying results.
Finally, it should be kept in mind that even more outcomes may have been used to assess clinical pharmacy interventions, however, a publication bias may exist, which may have led to exclusion of some non-significant or negative outcomes.

Implementation Rate of the Clinical Pharmacy Intervention
Clinical pharmacy interventions usually include provision of professional knowledge to a team of health care professionals or directly to the patient [1,7]. The processes involved when providing knowledge are quite complex, and consequently it is often difficult to measure the pharmacist's contribution to a multidisciplinary team [8]. Hence, applying process measures as suggested by the Donabedian model is useful to document the tasks actually provided by the clinical pharmacist. Frequently used process measures include type and number of drug-related problems (DRPs) identified, the acceptance rate of suggested recommendations made by the clinical pharmacist to address these DRPs, and implementation rates [1]. However, the acceptance rates and implementation rates of suggested recommendations vary considerably between studies, with usually around 65-70% acceptance rates-but some as low as 40% [1,2]. Whether low acceptance and implementation rates are due to suboptimal recommendations, barriers among physicians to accept and implement recommendations, or poor collaboration in the health care team remains unclear, and no suggestions of a minimum requirement for acceptance or implementation rates exist. This pose another challenge of interpreting outcomes, since studies with a sufficient number of included patients may not have had a proper exposure of the intervention to intervention patients. Consequently, the success of the clinical pharmacy intervention may be highly dependent on individual participants in the health care team, including the clinical pharmacist herself.

Limitation
Various methods exist to assess the quality of intervention studies (e.g., criteria developed by the Cochrane Effective Practice and Organisation of Care Review Group [52]). No formal quality assessment of the included studies was performed in the present review due to the exploratory nature of the review, however, ensuring sufficient power in a study is essential to avoid Type II errors, and more than half of the studies either did not include sufficient patients according to their power calculation or the power calculation was missing. This risk of Type II errors complicates the assessment of the potential effect and relevance of the selected outcome variables [13].
Types of statistical analyses used were not systematically collected. Comparison between studies may be further compromised, when different analyses are used i.e., continued variables (linear regression and ANOVA), binary outcomes (logistic regression), time to event (survival analysis), etc., since type of analysis is important for interpretation of the results.
Other aspect regarding the analyses, which was not systematically collected, were handling of dropouts and incomplete data (e.g., "last observation carried forward", exclusion, imputation, etc.) These may also affect the results and hence the interpretation of results differently.
Further, studies including 100 patients or less were excluded. It is likely that if they had been included, the proportion of studies with no reported power calculation and insufficient power may have been higher.

Conclusions
Type, frequency and result of clinical outcomes used to assess the effect of clinical pharmacy interventions in inpatient care varied considerably among the included studies. The most frequently reported outcome measures included clinical measures/assessments by physician and health care service use. No obvious pattern was established among primary outcome measures with significant effect in favour of the intervention, but positive effect was most frequently related to studies with presentation of power calculations and sufficient inclusion of patients. This review emphasizes the importance of considering the relevance of outcomes selected to assess clinical pharmacy interventions. Further discussion and consensus is needed with regard to selection of types of outcomes to ensure comparison of the effects among clinical pharmacy studies. Furthermore, conducting a proper power calculation and including the sufficient number of patients in the study according to the power calculation should be a prerequisite when publishing an outcome evaluation of clinical pharmacy intervention studies.