Comparing effectiveness with efficacy : outcomes of palliative chemotherapy for non-small-cell lung cancer in routine practice

Introduction Randomized controlled trials (rcts) are the “gold standard” for establishing treatment efficacy; however, efficacy does not automatically translate to a comparable level of effectiveness in routine practice. Our objectives were to ■ describe outcomes of palliative platinum-doublet chemotherapy (ppdc) in non-small-cell lung cancer (nsclc) in routine practice, in terms of survival and well-being; and ■ compare the effectiveness of ppdc in routine practice with its efficacy in rcts. Methods Electronic treatment records were linked to the Ontario Cancer Registry to identify patients who underwent ppdc for nsclc at Ontario’s regional cancer centres between April 2008 and December 2011. At each visit to the cancer centre, a patient’s symptoms are recorded using the Edmonton Symptom Assessment System (esas). Score on the esas “well-being” item was used here as a proxy for quality of life (qol). Survival in the cohort was compared with survival in rcts, adjusting for differences in case mix. Changes in the esas score were measured 2 months after treatment start. The proportion of patients having improved or stable well-being was compared with the proportion having improved or stable qol in relevant rcts. Results We identified 906 patients with pre-ppdc esas records. Median survival was 31 weeks compared with 28–48 weeks in rcts. After accounting for deaths and cases lost to follow-up, we estimated that, at 2 months, 62% of the cohort had improved or stable well-being compared with 55%–63% who had improved or stable qol in rcts. Conclusions The effectiveness of ppdc for nsclc in routine practice in Ontario is consistent with its efficacy in rcts, both in terms of survival and improvement in well-being.


INTRODUCTION
Randomized controlled trials (rcts) and meta-analyses have-based on improvements in survival, quality of life (qol), and symptom control-established palliative platinum-doublet chemotherapy (ppdc) as the standard of first-line care for advanced non-small-cell lung cancer (nsclc) 1,2 .Based on that research, multiple clinical guidelines [1][2][3] recommend ppdc as a standard first-line treatment.
However, the efficacy demonstrated in trials does not automatically translate into a comparable level of effectiveness in the general population, and a recent Cochrane review concluded that observed differences cannot be explained by differences in study design alone (that is, rct vs. observational) 4 .Population-based outcome studieswhich we have previously referred to as "phase iv" studies 5 provide a mechanism for assessing treatment effectiveness and are therefore complementary follow-ups to practicechanging rcts 6 .The same approach was previously used to describe the effect of the adoption of concurrent chemoradiotherapy for cervical cancer 7 and of adjuvant chemotherapy for nsclc 8 , and to describe survival outcomes with ppdc in advanced nsclc 9 .
Treatment effectiveness-in terms of qol, well-being, and symptom control-is not commonly studied at a population level because qol and symptoms are rarely captured systematically at a population level.The recent introduction of the Edmonton Symptom Assessment System (esas) for the routine assessment of symptoms in patients attending Ontario's regional cancer centres (rccs) makes it possible to evaluate, in routine practice, the effectiveness of palliative treatment with respect to patient well-being.The esas, a 9-item clinical tool, was developed for the rapid assessment of symptoms and overall well-being 10 and has been found to be valid and reliable in palliative care settings 11,12 .
To our knowledge, no studies have addressed the effectiveness of palliative chemotherapy for nsclc in terms of well-being or qol.The present study describes well-being and survival after first-line ppdc in nsclc patients in the general population of Ontario and compares the observed effectiveness of ppdc in routine practice with its reported efficacy in rcts.

Study Design and Population
This retrospective cohort study of patients receiving routine care at Ontario's rccs used the Ontario Cancer Registry (ocr) 13 to identify cases of nsclc.All patients treated with standard, first-line ppdc for stage iii or iv nsclc with palliative intent at an rcc between April 2008 and December 2011 were included.First-line ppdc was defined as cisplatin or carboplatin combined with any one of gemcitabine, vinorelbine, docetaxel, or paclitaxel.Patients were excluded if they had previously received chemotherapy, curative surgery, or curative radiotherapy.

Context
Ontario is a Canadian province of approximately 13 million people with a single-payer universal health insurance program.The rccs deliver approximately 50% of all chemotherapy in the province.

Data Sources
All data used to conduct this study were provided by Cancer Care Ontario (cco).Detailed information about these data can be found on the cco Web site at https://www.cancercare.on.ca/ext/databook/db1011/whnjs.htm.The ocr provided information about patient age, sex, vital status, and date of death.Information about TNM stage was linked to the ocr from the cco stage database, in which stage capture for lung cancer is 86% for cases diagnosed in 2008 14 and more than 90% for cases diagnosed from 2009 onward 15 .The Canadian Institute for Health Information's Discharge Abstract Database (details available at http://www.cihi.ca/cihi-ext-portal/internet/en/document/types+of+care/ hospital+care/acute+care/dad_metadata) was linked to the ocr to provide information about surgery.Records of radiotherapy were linked to the ocr from cco's Radiation Planning/Treatment Activity database.Records of chemotherapy were linked to the ocr from the cco Systemic Drug Delivery Event database to provide detailed information about all chemotherapy administered at Ontario's rccs, including treatment intent, type of drug, and date of drug administration.The database is populated by cco's automated drug prescribing system, which is used exclusively at the rccs and is therefore of high quality.Scores from the esas were linked to the ocr from cco's Symptom Management Reporting Database.

Survival
Survival was calculated from the date of first ppdc.Dates of death were complete to November 2011.Patients alive at that time were censored.Factors associated with survival were evaluated using Cox regression.Median survival in the routine-care cohort was compared with that reported in relevant phase iii rcts.Five rcts were selected for survival comparisons [16][17][18][19][20] based on their exclusive use of ppdc regimens (including platinum plus a new agent) and their inclusion in a 2010 systematic review of first-line systemic chemotherapy for advanced nsclc 21 or one of two metaanalyses cited within that review 22,23 that focused only on ppdc regimens consisting of platinum plus a new agent.Two additional rcts were selected based on their reporting of detailed qol results 24,25 .To account for differences in case mix (age, sex, stage, and histology), the mean of covariates method was used to standardize median survival in the cohort with that in the comparator rcts 26 .

Well-Being and Symptomatic Status
Scores on the esas measure well-being and 8 common cancer symptoms.Patients rate their well-being and symptoms on an 11-point scale from 0 (best) to 10 (worst).The esas data can be entered electronically by the patient or completed on paper and then uploaded by clinic staff 12 .Since 2008, cco has used the esas in an attempt to assess all patients at every rcc visit.Collection of esas data is still incomplete, but by 2010, 59% of lung cancer patients seen at rccs were being assessed by esas at least once each month 27 .
Baseline esas scores were defined as those measured within the 30 days before (including the date of) the first chemotherapy treatment.Baseline scores were categorized as mild or absent (0-3), moderate (4-6), or severe (7-10)  based on cut-offs previously suggested 28 .The single esas well-being item was selected as a proxy for general qol in our study.
The definition of 2-month esas scores was those measured at 2 months (8 weeks ± 2 weeks after the first chemotherapy treatment and before the 3rd cycle) to coincide with the end of the 2nd cycle of treatment.Patient well-being at 2 months was classified as improved, stable, or deteriorated based on change in the esas score from baseline to 2 months (Figure 1).An increase of 2 or more points was classified as deteriorated, a change of 0 or ±1 as stable, and a decrease of 2 or more as improved.A 2-point change was selected as the cut-off point for defining a clinically meaningful difference because it was consistent with the accepted statistical method of choosing a cut-point (corresponds to 0.5 of the standard deviation of the measurement tool 29 , which was 1.4 points for the well-being item).
We identified 76 patients who were seen on 2 consecutive days with esas records for each visit.In an opportunistic test-retest comparison of the well-being scores for those patients, we found that 67% had scores within 1 point of each other.That observation suggests that a change of 2 or more points also represents a change greater than normal day-to-day variation.
The association between patient-related and diseaserelated characteristics and change in well-being at 2 months (dichotomized as improved or stable and deteriorated) were assessed using multivariate log binomial regression.Secondary outcomes were changes in specific symptom scores at 2 months.The change in well-being for the cohort at 2 months was also compared with the 2-month change in global qol (measured using the 30-question Quality of Life Questionnaire from the European Organisation for Research and Treatment of Cancer) reported by Gridelli et al. 24 and von Plessen et al. 25 .Sensitivity analyses were performed to explore the nature of missing esas data at 2 months and to assess the likely impact on the analyses.
All analyses were performed using the SAS software application (version 9.3: SAS Institute, Cary, NC, U.S.A.).Our study was approved by the Health Science Research Ethics Board at Queen's University.

Baseline Characteristics of the Study Population
Figure 2 describes the selection of the routine care cohort.After excluding patients who had received curative treatments, 1625 patients who had undergone ppdc were eligible for inclusion.Of those 1625 patients, 906 (56%) had a baseline esas record available for assessment.Table i describes and compares the characteristics of patients with and without a baseline esas.No statistically significant differences in age, sex, histology, or stage between the groups were observed.Moreover, the median survival in the two groups was not significantly different (32.6 weeks vs. 31.3weeks, p = 0.51).
Table ii describes the well-being and symptom status of patients before initiation of ppdc.Of baseline well-being scores, 43% were in the best or mild category, 37% in the moderate category, and 20% in the worst or severe category.Notably, 45%-56% of scores for shortness of breath, tiredness, and loss of appetite (common symptoms of advanced nsclc) fell into the moderate and severe categories; only 17% of nausea scores were moderate or severe at baseline.

Treatment
Table i describes the platinum doublets and numbers of cycles of chemotherapy that patients received.Cisplatinbased doublets were given to 43%, and carboplatin-based doublets, to 57%.Fewer than half the patients (42%) completed 4-6 cycles; 23% completed only 1 cycle.The specific platinum doublets used and the number of cycles completed were similar in the two groups.

Survival
The median overall survival in this study was 31 weeks [95% confidence interval (ci): 29 weeks to 34 weeks].Table iii shows the results of the Cox proportional hazards regression of patient factors versus survival.Poorer baseline well-being was associated with significantly shorter survival [hazard ratio (hr) for moderate scores: 1.27; 95% ci: 1.06 to 1.51; and hr for severe scores: 1.65; 95% ci: 1.33 to 2.04].Female sex (hr: 0.86; 95% ci: 0.73 to 1.01) and stage iii disease (hr: 0.83; 95%ci: 0.66 to 1.03) were associated with an approximate 15% reduction in the hazard, but those differences did not reach statistical significance.A slight trend toward increased risk with increasing age was observed, but was not statistically significant.Responses at 2 months were categorized as improved/stable/deteriorated as shown in Figure 1. in the comparison rcts.Median age in the groups was similar, but compared with the routine care cohort, the rct groups contained more men, less stage iv disease, less adenocarcinoma, and more squamous cell carcinoma.Median survival in our cohort was 31 weeks compared with 28-48 weeks in the rct groups.Given that sex, age, stage, and histology are established prognostic factors, we adjusted our survival estimates to the case mix of each comparison rct (Table iv).The adjustment resulted in the survival estimate for the routine care cohort moving closer to the survival estimates for the rct groups in three trials 16,18,20 , farther away in two trials 17,25 , and in neither direction in one trial 19 .The 95% cis for the adjusted survival estimates contained the rct survival estimates in three trials 17,20,25 .

Well-Being and Symptoms
Table ii shows the changes in esas score at 2 months.An esas record corresponding to 2 months after initiation of ppdc was available for 453 patients.Only patients who were alive and who had a completed esas at 2 months were considered in the assessment of change in esas score.Well-being had improved in 51% of patients; it was stable in 20%, and it had deteriorated in 29%.Overall, the symptom burden improved or remained stable for most patients.Some of the highest proportions of improved or stable scores were seen for the disease-related symptoms of pain (75%), shortness of breath (67%), and appetite (67%).The highest proportions of deteriorated scores were seen for treatment-related symptoms of drowsiness and tiredness (also a disease-related symptom), both at 37%.
Table v shows the results of the log binomial regression assessing the association of various patient factors with change in well-being at 2 months.In the univariate and multivariate analyses, baseline well-being was the only independent variable associated with change in wellbeing.Compared with patients who had at mild score at baseline, patients with moderate [relative risk (rr): 1.24; 95% ci: 1.05 to 1.47] or severe scores (rr: 1.24; 95% ci: 1.03 to 1.50) at baseline had a higher rr for improved or stable well-being at 2 months.Sensitivity Analyses: Although the survival analyses were performed on all 906 members of the routine-care cohort, the analyses of change in well-being were restricted to patients with a completed 2-month esas (n = 453).Of the missing 453 patients, 120 had died; 333 were alive, but missing a 2-month esas.Sensitivity analyses comparing the measured 453 with the missing 333 (data not shown) found no statistically significant differences between the groups in age, sex, stage, histology, baseline well-being, number of ppdc cycles completed, or median survival.Those findings suggest that the 333 were effectively missing an esas at random rather than in any way related to patient, treatment, or outcome factors.Assuming the same distribution of improved or stable and deteriorated scores for the missing 333 patients as for the measured 453, and classifying the 120 patients who died as deteriorated, we therefore estimate that, in the full routine care cohort, 44% experienced improved well-being; 18%, stable well-being; and 38%, deteriorated well-being.

Comparison to RCTs:
The estimate of 62% of patients with improved or stable well-being from the sensitivity analyses is directly in line with estimates of improved or stable qol reported by the rcts: 63% in Gridelli et al. 24 and 55% in von Plessen et al. 25 .

DISCUSSION AND CONCLUSIONS
Our main finding is that patients undergoing standard ppdc in routine practice achieve survival that is comparable to-and symptomatic responses that are comparable to, if not better than-those reported in rcts of ppdc.
Median survival in our cohort was consistent with the lower end of the survivals reported in rcts (range: 28-48 weeks), even after standardization to the case mix of each comparison rct.Patients in the routine care cohort were similar to those included in rcts, except for the proportions of women and of adenocarcinoma, which were both considerably higher in the current study, and of squamous cell carcinoma, which was lower.That change in case mix is consistent with recent observations 30 .
In the survival analysis, we found that only baseline well-being was associated with overall survival.Survival duration was significantly shorter for patients with the poorest well-being before treatment than for those with the best well-being (poorest: 22 weeks; 95% ci: 18 weeks to 27 weeks; best: 37 weeks; 95% ci: 31 weeks to 44 weeks).That observation is consistent with the results of previous studies that have shown the prognostic value of pretreatment qol 31 .
At 2 months into treatment, 71% of patients reported improved or stable well-being.More than 40% of patients in the routine care cohort reported reduced scores for shortness of breath, tiredness and loss of appetite, and pain, with an additional 22%-30% of patients remaining stable.Importantly, we observed no effect of patient age on change in well-being (those 70-89 years of age compared with those 50-69 year years: rr: 1.01; 95% ci: 0.89 to 1.15), lending further support to the growing evidence that fit elderly patients are no less likely to benefit from treatment in terms of qol 8,32 .The only patient factor associated with improved or stable well-being 2 months after treatment initiation was baseline well-being (moderate baseline score rr: 1.23; 95% ci: 1.09 to 1.36; severe baseline score rr: 1.45; 95% ci: 1.27 to 1.66).It is perhaps not surprising that patients whose well-being was rated worst at the outset more often felt better with treatment.Analyses of change in well-being were restricted to those for whom a 2-month esas was available (n = 453).Of the missing 453, 120 had died, and 333 were alive but lacked a 2-month esas record.Comparisons between patients measured and unmeasured at 2 months found no statistically significantly differences in disease or treatment characteristics or in median survival, suggesting randomness of the missing esas data for the 333 patients.In sensitivity analyses, we therefore applied the improved or stable and deteriorated distributions of the measured patients to the missing patients, and classified those who had died as deteriorated.The result was an estimate that 62% of the full routine care cohort experienced improved or stable well-being, which is consistent with the 55% 24 and 65% 25 improved or stable qol estimates reported by the rcts.
A few methodology limitations of our study (in addition to the missing data already noted) warrant acknowledgment.The mean covariates method used for calculating adjusted survival estimates has certain limitations.This method calculates the average hazard for the average individual, which is not the same as the average survival for a group of heterogeneous individuals.The method does not perform as well as effect sizes increase and covariates become more common.A preferred alternative is the corrected group prognosis method 26 , but the latter method requires knowledge of the joint distribution of variables within the trial data, to which we had no access.
We used patient well-being measured by the esas as a proxy for general qol in our comparisons with the rct qol data.Although well-being and qol are certainly closely related concepts, subtle differences might make direct comparisons between the two less precise.Additionally, detailed information about performance status, treatment toxicity, and comorbidities in the present study and about baseline qol in the comparison rcts was not available and thus limited our ability to compare our results with those from the rcts.We also cannot rule out the possible influence on our results of other palliative care interventions, placebo effects, or shifts in patient response.Indeed, recent work has demonstrated that early palliative care has an independent positive effect on both qol and survival 33 .However, the same limitation is true of the rcts and therefore does not invalidate the efficacy-effectiveness comparison.
The chemotherapy treatment data were restricted to regimens administered to patients at Ontario's rccs who had available baseline esas scores.Those results might therefore not be generalizable to all patients with nsclc.However, our patient group is probably more representative of the general nsclc population than are clinical trial participants.Furthermore, patient characteristics and survival in patients with and without esas scores were not substantially different, suggesting that our results might be generalizable to all patients undergoing first-line ppdc at Ontario's rccs.
In an era of ever-increasing fiscal restraint in health care, it is imperative to ensure that treatment programs are producing their intended results.Our results suggest that the effectiveness of ppdc delivered in Ontario's rccs is consistent with results from relevant rcts in terms of both survival and patient well-being.

FIGURE 1
FIGURE 1 Frequency distribution of change in Edmonton Symptom Assessment System well-being score at 2 months (453 patients).Change in score was calculated by subtracting the 2-month score from the baseline score, such that a decline in the score represents a positive change (that is, improvement in well-being).

FIGURE 2
FIGURE 2 Identification of the routine care cohort.a Used in survival analysis.b Used in well-being and symptom analysis.ESAS = Edmonton Symptom Assessment System.
Table iv compares the baseline characteristics of patients in the routine care cohort and

TABLE II
Baseline scores and changes at 2 months, Edmonton Symptom Assessment System

TABLE III
Factors associated with survival in 906 patients treated with palliative platinum-doublet chemotherapy and assessable for baseline well-being CI = confidence interval; HR = hazard ratio; NSCLC = non-small-cell lung cancer.Current Oncology, Vol.22, No. 3, June 2015 © 2015 Multimed Inc.

TABLE IV
Survival in randomized controlled trials (RCTS) compared with survival in a routine-care cohort of patients treated with palliative platinum-doublet chemotherapy a Adjusted for age, sex, stage, and histology to the distribution in the comparison RCT.b Results from the platinum-based arm only.c Results from the 3-and 6-cycle arms combined.NR = not reported.Current Oncology, Vol.22, No. 3, June 2015 © 2015 Multimed Inc.

TABLE V
Factors associated with changes in well-being at 2 months in 453 patients treated with palliative platinum-doublet chemotherapy and assessable after 2 months CI = confidence interval; RR = relative risk; NSCLC = non-small-cell lung cancer.