Prospective validation of risk prediction indexes for acute and delayed chemotherapy-induced nausea and vomiting

younger patient age, platinumor anthracyclinebased chemotherapy, low alcohol consumption, earlier cycles of chemotherapy, previous history of morning sickness, and prior emetic episodes after chemotherapy. The acute and delayed scoring systems both had good predictive accuracy when applied to the external validation sample (acute— auroc: 0.69; 95% confidence interval: 0.59 to 0.79; delayed—auroc: 0.70; 95% confidence interval: 0.60 to 0.80). Patients identified by the scoring systems to be at high risk were 2.8 (p = 0.025) and 3.1 (p = 0.001) times more likely to develop grade 2 or greater acute and delayed cinv.

One of the major breakthroughs in oncology since the early 1990s has been the development of the serotonin receptor antagonist anti-emetics-for example, ondansetron, granisetron, tropisetron 1 .Although chemotherapy-induced nausea and vomiting (cinv) remains one of the most feared side effects of cancer therapy, it is better controlled than ever before.That better control is partly a result of the availability of the newer neurokinin-1 receptor antagonists (for example, aprepitant), the better use of older agents (dexamethasone, for instance), and the publication of evidence-based anti-emetic guidelines [2][3][4] .

Background
Despite the use of standardized anti-emetic guidelines, up to 20% of cancer patients suffer from moderate-to-severe chemotherapy-induced nausea and vomiting (cinv)-that is, grade 2 or greater according to the U.S. National Cancer Institute Common Terminology Criteria for Adverse Events, version 4.0.We previously developed cycle-based prediction models and associated scoring systems for acute and delayed cinv.As part of the validation process, we prospectively evaluated the ability of the scoring systems to accurately identify patients deemed to be high risk for grade 2 or greater cinv.

Methods
Patients who were receiving any chemotherapy for solid tumours and who consented to participate were provided with symptom diaries.Compliance to the diaries was enhanced by 24-hour and 5-day telephone callbacks after chemotherapy in every cycle.All patients received anti-emetic prophylaxis as prescribed by the treating physician.Before each cycle of chemotherapy, the acute and delayed cinv scoring systems were used to stratify patients into low-and high-risk groups.Logistic regression modelling was then applied to compare the risk for grade 2 or greater cinv between patients considered to be at high and at low risk.The external validity of each system was also assessed using an area under the receiver operating characteristic curve (auroc) analysis.
Traditionally, cinv is separated into an acute and a delayed phase.Acute events occur within the first 24 hours after chemotherapy; delayed cinv develops between days 2 and 5 after 5 .However, despite the use of modern anti-emetics and evidence-based guidelines, up to 20% of patients will still experience moderate-to-severe cinv events [that is, grade 2 or greater according to the U.S. National Cancer Institute (nci) Common Terminology Criteria for Adverse Events (ctcae), version 4.0], with nausea being particularly problematic 6,7 .Given that nci ctcae grade 2 nausea is defined as "oral intake decreased without significant weight loss, dehydration or malnutrition" and grade 2 vomiting is defined as "3-5 episodes (separated by 5 minutes) in 24 hours," the impact of these events on patient quality of life cannot be understated 8 .
To help identify patients that are at higher-thanaverage risk for grade 2 or greater acute and delayed cinv, we previously developed and prospectively validated cycle-based risk prediction models 9,10 .Major predictors for the development of acute and delayed cinv were consistent with the literature and included age less than 40 years, female sex, platinum-or anthracycline-based chemotherapy, low alcohol consumption, emesis in earlier cycles of chemotherapy, previous history of morning sickness or pregnancyinduced emesis, and prior emetic episodes within the same or previous regimens of chemotherapy 9,10 .The models were subsequently used to develop numerical scoring systems (indexes) that, before each cycle of chemotherapy, are able to identify patients at high risk for acute and delayed cinv (Table i).
The indexes are easy to apply using patient information that is readily available to the patient care team.It was demonstrated that the risk of acute and delayed cinv rises as the cumulative risk score increases 9,10 .From the initial model development study, patients with risk scores of 7 or more (acute cinv) and more than 16 (delayed cinv) were classified as being at "high risk" for grade 2 or greater events 9,10 .
The advantage of using a cycle-based model is that cinv outcomes data from the previous cycle is used to "fine tune" the prediction of cinv risk for subsequent chemotherapy cycles.As a result, the ability of each index to accurately identify patients at high risk is improved as new information is entered into the model with each treatment cycle.As part of the ongoing validation process, we prospectively evaluated the ability of the scoring systems to accurately identify patients deemed to be high risk for grade 2 or greater cinv in a geographically distinct cancer centre that was not part of the initial model development study.

Patients
The intent of the present cohort study was to prospectively validate the prediction scoring systems for acute and delayed cinv in an independent sample of patients receiving outpatient chemotherapy.To challenge the generalizability of each cinv index, chemotherapy was not limited to a single regimen or disease site.Patients with a range of malignancies who were scheduled to receive outpatient chemotherapy at The Ottawa Hospital Cancer Centre were approached about the study.If written informed consent was received, the initial data collection consisted of patient demographics, disease-related information, and potential predictive factors for cinv such as a history of motion sickness, a history of morning sickness during a previous pregnancy (if applicable), and daily alcohol consumption.
Just before each cycle of chemotherapy, additional information (such as the scheduled anti-emetic prophylaxis, the anticancer agent or agents prescribed, the patient's expectation for nausea after chemotherapy, food intake the morning of chemotherapy, and number of hours slept the night before chemotherapy) was collected.Anxiety levels were also measured using a 4-point Likert scale (none, mild, moderate, high).
For the patients in the present study, no predefined anti-emetic prescriptions were built into each chemotherapy regimen; rather, anti-emetics were prescribed by the medical oncologist, and no adjustment was made based on acute or delayed emesis score.Permission to conduct the study was received from the local institutional ethics review board.

Classification of Patients into High-and Low-Risk Groups
Before each cycle of chemotherapy, the acute and delayed scoring systems were applied to estimate risk scores for each patient for that cycle of chemotherapy (Table i).Because of the association between the calculated risk score and the probability of moderate-tosevere cinv, patients with an acute score of 7 or more and a delayed score of more than 16 were categorized as being at high risk for a cinv event 9,10 .

Collection of Outcomes Data
At enrolment, patients were provided with a diary for daily self-reporting of events.The patients were to record items such as the number of vomiting episodes; the occurrence, intensity, and duration of nausea in the first 24 hours and during days 2-5 after chemotherapy; and the use of non-prescribed drugs at home for emesis control.The nci ctcae, version 4.0 (Table ii), was used to capture the grade of both acute and delayed cinv (grades 0-4).To obtain additional information on the patient's perception of the severity of emetic events, each episode of nausea and vomiting was rated using a 4-point Likert scale [none, mild, moderate, severe ( had been completed accurately.After completion of each chemotherapy cycle, patients were asked to use a 4-point Likert scale (1 = terrible to 4 = excellent) to rate overall control of vomiting and nausea.

Statistical Analysis and Validation of Scoring Systems
Demographics and disease characteristics are presented descriptively as means, medians, or proportions.The primary endpoints in the current study were the incidence of moderate-to-severe (that is, nci ctcae grade 2 or greater) acute and delayed cinv.Those endpoints were defined as a composite measure consisting of nci ctcae grade 2-4 nausea and vomiting or of moderate-to-severe vomiting and nausea as described in the 4-point Likert scale.
To measure the association between the calculated risk score and the probability of acute and delayed cinv, four univariate logistic regression analyses were undertaken, with adjustment for clustering on each cycle number.Patient score and risk category (high vs. low) were the lone independent variables in the logistic regression models.The intent of the analysis was to determine the probability of acute and delayed cinv by patient score and the odds ratio (or) for an acute and delayed event by risk category (high vs. low).The probability of acute and delayed cinv for each patient was determined using the formula where the constant and the model coefficient for the variable "risk score" was obtained from the univariate logistic regression analyses, with patient score as the sole independent variable.The goodness of fit of the univariate models was then assessed using the Hosmer-Lemeshow test.
As part of the validation process, the predictive accuracy of each risk scoring system was determined by measuring the specificity, sensitivity, and area under the receiver operating characteristic (roc) curve."Discrimination" refers to the ability of a diagnostic test or predictive index to accurately identify patients at low or high risk for the event under investigation and is often presented as the area under the roc curve.A predictive instrument with a roc of 0.70 or greater is considered to have good discrimination, and an a roc of 0.5 is equivalent to a coin toss.All of the statistical analyses were performed using the Stata software application (version 11.0: StataCorp, College Station, TX, U.S.A.).

RESULTS
The study enrolled 95 patients between June 2010 and October 2010, and those patients received 181 cycles of chemotherapy.Most of the patients had breast (64.2%), gastrointestinal (16.8%), genitourinary (5.6%), and lung malignancies (13.7%,Before each cycle of chemotherapy, 89.5% of patients received a 5-HT 3 anti-emetic (for example, ondansetron or granisetron) as part of their primary prophylaxis.The neurokinin-1 receptor antagonist aprepitant was used in 28 of 181 cycles (15.5%) preand post-chemotherapy (Table iv).After each cycle of chemotherapy, 5-HT 3 anti-emetics were used in 82.9% of patients.Dexamethasone was used after chemotherapy in 52 of 181 cycles (28.7%).Before the subsequent cycle of chemotherapy, 21 patients (10.5% of cycles) stated they had used a non-prescribed treatment at home for nausea and vomiting control.Those drugs included dimenhydrinate, Pepto-Bismol (Procter and Gamble, Toronto, ON), and antacids.Table v presents potential prior risk factors and cinv outcomes data.Before each cycle of chemotherapy, patients expected to have nausea and vomiting in 37.6% of cycles, and their anxiety was moderate to high in 54% of cycles (Table v).
After completing a chemotherapy cycle, patients were asked to use a 4-point Likert scale to rate overall control of vomiting and nausea.Over the 181 cycles of systemic therapy, 84.5% considered the control of vomiting to have been "excellent"; "excellent" control of nausea was lower at 65.7% (Table v).When the composite endpoint of moderate-to-severe nausea and vomiting (grade 2 or greater) was determined, an acute cinv event occurred in 17.7% of cycles and a delayed event in 18.2%.Nausea was the predominant symptom, with each moderate-to-severe event lasting between 2 and 3 hours.
Before each cycle of chemotherapy, the acute and delayed risk scores were calculated for each patient (Table i). Figure 1 illustrates the association between  the probability of acute cinv and the calculated score in our cohort of patients.Consistent with our original model development studies, patients with higher scores had an increased likelihood of experiencing an acute cinv event.The association between the calculated risk score and the probability of acute cinv was also statistically significant.For each additional unit, the risk of acute cinv showed a 30% relative increase (or: 1.30; p = 0.015).
Before each cycle of chemotherapy, patients were also classified as being at high or low risk for acute cinv.In the original publication describing the acute cinv model development, the roc analysis suggested that a risk score of 7 was the cut-point between high and low risk for acute cinv.Using that cut-point, the associated sensitivity was 71.9%, and the specificity was 52.3% (Table vi).The univariate logistic regression analysis using the cut-point of 7 revealed that, compared with patients considered by the index to be at low risk, patients considered to be at high risk were 2.8 times more likely to experience a moderate-to-severe acute cinv event (or: 2.8; p = 0.025).However, only 55.8% of patients were correctly classified as high-and low-risk using a risk score of 7 as the cut-point.By contrast, raising the cut-point to 9 reduced the sensitivity (that is, 6 true positives-patients who experienced a grade 2 or greater event-were missed), but it improved the specificity (that is, 40 additional true negatives were picked up) and the proportion of patients correctly classified to 74.6% (Table vi).The or for developing a moderate-to-severe acute cinv using a cut-point of 9 increased to 4.3.
The interpretation is that raising the cut-point to 9 would reduce the number of true negatives (that is, people deemed to be at low risk by the index who do not have a cinv event) at the expense of missing some true positives (6 patients in the present study).
Figure 2 illustrates the association between the probability of delayed cinv and the calculated risk score.The findings of the univariate logistic regression analysis, with patient risk score as the lone predictor variable, generated a relative odds of 6% (or: 1.06; p = 0.001) for each additional unit on the delayed cinv index.Stated differently, the risk of a delayed cinv increased by 6% for every additional unit (Figure 2).
Before each cycle of chemotherapy, patients were also classified as being at high or low risk for delayed cinv.In the original publication describing the delayed cinv model development, the roc analysis suggested a risk score more than 16 as the cutpoint between high and low risk for delayed cinv.Using that cut-point, the associated sensitivity was 75.8%, and the specificity was 50%, with only 54.7% of patients being correctly classified (Table vi).The logistic regression analysis revealed that, compared with patients having scores of 16 or less, patients with risk scores greater than 16 (that is, at high figure 1 Association between the probability of acute chemotherapy-induced nausea and vomiting (cinv) and the calculated score.ci = confidence interval.), patients with a risk score of 7 or more were considered to be at high risk for an acute cinv event.Patients with a risk score of more than 16 were considered to be at high risk for a delayed cinv event.b The proportion of patients having a cinv event who had been classified as high risk.c The proportion of patients not having a cinv event who had been classified as low risk.d Risk of a moderate-to-severe cinv event in patients determined to be at high compared with low risk by the respective scoring system.or = odds ratio; ci = confidence interval.risk according to the original classification) were 3.1 times more likely to have a delayed cinv event (or: 3.1; p = 0.018).If the cut-point score were to be increased to more than 20, the sensitivity would be reduced to 57.6% (that is, 6 true positives would be missed), but the specificity would increase to 77.0% (that is, 40 additional true negatives would be picked up) and the proportion of patients correctly classified to would rise to 73.5% (Table vi).However, as with the acute index, raising the delayed risk score cut-point to more than 20 would mean missing 6 true positives.
In the final phase of our validation study, the calculated risk scores and the probabilities for acute and delayed cinv events from each patient were used in a roc analysis.The findings suggest that the area under the roc curve for the acute and delayed risk indexes was acceptable at 0.69 (95% confidence interval: 0.59 to 0.79) and 0.70 (95% confidence interval: 0.60 to 0.80) respectively, supporting the external validity of each prediction index.

DISCUSSION AND CONCLUSIONS
Understandably, cinv is one of the most feared and expected side effects cancer therapy.Our findings, consistent with those of other studies, revealed that between 15% and 20% of cancer patients still experience moderate to severe nausea and vomiting 7,9,10 .The consequences of the emetogenic events include any or all of treatment delays, dose reductions, the need for additional prophylaxis, health care resource consumption (that is, home hydration), and premature discontinuation of chemotherapy.
The occurrence of chemotherapy-induced toxicity such as emesis has traditionally been believed to be unpredictable.As a result, many oncologists prescribe anti-emetic prophylaxis based on the intrinsic emetogenicity of the chemotherapy drugs used, paying less attention to individual patient factors.
They generally wait to see how patients cope with the first treatment cycle before adjusting anti-emetics as needed for subsequent cycles.Patient-centred care could be substantially improved if episodes of significant cinv could be accurately predicted through the use of validated risk-scoring systems.If accurate identification of high-risk patients were available, escalated prophylaxis and heightened follow-up might be able to be offered.Identification might also offer an opportunity to forewarn the patient and initiate a more intensive early monitoring scheme and an action plan for early intervention.In addition, patients at low risk might be able to receive less anti-emetic therapy, sparing them some of the toxicities of those medications (for example, insomnia from steroids, or constipation and headaches from 5-HT 3 antagonists).
We previously developed and prospectively validated scoring systems to estimate the risk of moderate-to-severe acute and delayed cinv 9,10 .As part of the ongoing validation process, we evaluated the overall performance of those systems in a new sample of patients from a cancer clinic not involved in the original model development study.Our findings demonstrate that the indexes have acceptable discrimination in estimating individual risk and in classifying patients into high-or low-risk categories.Compared with patients considered to be low-risk, patients classified as high-risk were approximately 3 times more likely to experience an acute or delayed grade 2 or greater cinv event.As a result, the indexes are reasonable tools to use in the clinic for tailoring anti-emetic therapy and for patient counselling.As indicated by our findings, prophylaxis with dexamethasone and post-chemotherapy aprepitant was used in only 28.7% and 15.5% of cycles respectively.Those agents as well as others (for example, synthetic cannabinoid anti-emetics for younger patients) could be offered to patients identified as high risk by the cinv indexes, illustrating how those tools could be used to optimize patient care.
In our original studies, we proposed scores of 7 or greater and more than 16 for classifying patients as high risk for acute and delayed cinv respectively 9,10 .However as suggested by the present results, one of the drawbacks in using those risk cut-points is an increased number of false positives (that is, patients classified as high risk who do not experience a cinv event).In our patient cohort, that approach would translate into an additional 40 patients unnecessarily receiving additional anti-emetic therapy.Raising the acute and delayed cinv cut-points to 9 or greater and more than 20 would reduce the number of false positives by 40 patients, but would also miss 6 true positives.The application of our risk indexes therefore requires a balance between sensitivity (that is, the true-positive rate) and specificity (that is, the true-negative rate) for a given clinical situation.
In our patient cohort, we would need to ask this question before using the cinv indexes in practice: "Does identifying an additional 6 patients who will actually develop cinv outweigh the overtreatment of 40 patients if the higher risk score cut-point is used?"Such decisions would have to consider factors such as the intent of chemotherapy (palliative vs. curative), patient concerns, and the side effects and costs of overtreating selected patients with additional anti-emetics.
A number of limitations in the current study have to be acknowledged.The sample size was small, and patient data were obtained from only a single institution.As indicated by the frequency of dexamethasone and aprepitant prescription, not all patients received the available anti-emetic agents according to treatment guidelines 3,4 .Our incidence of moderate-to-severe cinv could therefore, in theory, have been reduced through the rigorous application of the guidelines.Approximately 64% of our cohort consisted of breast cancer patients, only 24.2% were men, and no patients had gynecologic malignancies.Future validation studies should enrol more men and more patients from those underrepresented disease sites to expand the validation of the risk indexes.
Despite their limitations, the risk indexes performed relatively well.They are easy to apply, and they are able to discriminate between high-and lowrisk patients.Also, the risk threshold can be varied depending on the patient's or clinician's risk tolerance.The application of these prediction tools in the clinic can be an important source of patient-specific risk information for the practicing oncologist.A Canadian breast cancer-funded randomized control trial (epic), in which patients are being randomized to an anti-emetic regimen based on their emetic score or physician choice, is underway.We hope that the epic study will evaluate the ability of the indexes to improve overall nausea and vomiting control by customizing an individual patient's anti-emetic regimen based on their overall risk profile.

e420
Current OnCOlOgy-VOlume 19, number 6, DeCember 2012Copyright © 2012 Multimed Inc.Following publication in Current Oncology, the full text of each article is available immediately and archived in PubMed Central (PMC).

figure 2
figure2 Association between the probability of delayed chemotherapy-induced nausea and vomiting (cinv) and the calculated score.ci = confidence interval.

e416 Current OnCOlOgy-VOlume 19, number 6, DeCember 2012 Copyright
© 2012 Multimed Inc.Following publication in Current Oncology, the full text of each article is available immediately and archived in PubMed Central (PMC).

table i
Risk scoring system for acute and delayed chemotherapy-induced nausea and vomiting (cinv) Current OnCOlOgy-VOlume 19, number 6, DeCember 2012Copyright © 2012 Multimed Inc.Following publication in Current Oncology, the full text of each article is available immediately and archived in PubMed Central (PMC).

Table iv
). Slightly more than half the patients (56.8%) had stage iii or iv disease.Approximately 75% of the patients were chemotherapy-naïve, and 35.8% had other concomitant medical conditions (for example, diabetes, cardiovascular disease; Tableiv).The type of chemotherapy received was platinum-based in 35.9% of the patients and anthracycline-based in 29.3%.

table ii U
.S. National Cancer Institute Common Terminology Criteria for Adverse Events, version 4.0, definition of grades 2-4 nausea and vomiting e418 Current OnCOlOgy-VOlume 19, number 6, DeCember 2012 Copyright © 2012 Multimed Inc.Following publication in Current Oncology, the full text of each article is available immediately and archived in PubMed Central (PMC).a Diabetes or cardiovascular, gastrointestinal, musculoskeletal, thyroid, other disease.

table v
Risk factors and acute and delayed nausea and vomiting outcomes data a Based on the original publications(Dranitsaris et al., 2009 9;Petrella et al., 2009 10), an acute score of 7 or more was considered to be high risk for acute cinv, and a delayed score greater than 16 was considered to be high risk for delayed cinv.cinv= chemotherapy-induced nausea and vomiting.Current OnCOlOgy-

VOlume 19, number 6, DeCember 2012
Copyright © 2012 Multimed Inc.Following publication in Current Oncology, the full text of each article is available immediately and archived in PubMed Central (PMC).