Publication patterns of cancer cost-effectiveness studies presented at major conferences

s of ceas presented at the annual meetings of the American Society of Clinical Oncology (asco), the American Society of Hematology (ash), and the International Society for Pharmacoeconomics and Outcomes Research (ispor) between 1997 and 2007 were reviewed. Time-to-event analysis was performed to assess the timeliness of publication and to examine factors associated with time to publication. Summary statistics were used to assess discrepancies in incremental cost-effectiveness ratios (icers) between abstract and publication.


INTRODUCTION
New oncology drugs can offer significant improvements in efficacy while providing hope for many patients with cancer.Unfortunately, these new drugs are associated with significantly high costs 1 .In the United States, recent developments in cancer treatment are consuming an increasingly larger proportion of national health care expenditures [2][3][4] .Spending on cancer drugs rose to $11 billion in 2004 from $3 billion in 1997 (267% increase), while overall Medicare spending rose only to $309 billion from $210 billion (47%) during the same period 5 .
To address these rising costs, cost-effectiveness analyses (ceas) are performed to examine the "value for money" attained by novel therapies.A cea compares the cost of the intervention with the effect, yielding a cost per effect that can be compared across multiple interventions 2 .To be useful to policymakers and stakeholders, cea studies need to be published with rigorous methodology and in a timely manner.Ideally, publication is accomplished in parallel with the clinical evidence that supports a new intervention.Once a drug has been shown to be effective, policymakers are reliant on ceas to allocate funding for drugs.
However, despite the importance of ceas, previous studies have raised concerns about delays in the publication of such analyses that were pre-planned

Objective
To be useful to policymakers and stakeholders, cost-effectiveness analyses (ceas) should be published in a timely manner and without bias.The aims of the present study were to examine the time between conference abstract presentation and subsequent publication, to determine the factors associated with time to publication, to evaluate potential publication bias, and to examine discrepancies in the results between abstract and publication.

Methods
Abstracts of ceas presented at the annual meetings of the American Society of Clinical Oncology (asco), the American Society of Hematology (ash), and the International Society for Pharmacoeconomics and Outcomes Research (ispor) between 1997 and 2007 were reviewed.Time-to-event analysis was performed to assess the timeliness of publication and to examine factors associated with time to publication.Summary statistics were used to assess discrepancies in incremental cost-effectiveness ratios (icers) between abstract and publication.

Results
Of 164 abstracts identified, 65 (39.6%) were subsequently published.The 1-, 2-, 3-, and 5-year publication rates were 12.8%, 25%, 34.2%, and 40.5% respectively.Abstracts were more likely to be published if presented at asco than at ispor (hazard ratio: 1.94; p = 0.038).There was no direct evidence of publication bias for abstracts with favourable icers.Comparing icers between abstracts and publications, the mean absolute difference was 23.8%; 50% of studies had a change in icer exceeding 10%.

Current OnCOlOgy-VOlume 20, number 6, DeCember 2013
Copyright © 2013 Multimed Inc.Following publication in Current Oncology, the full text of each article is available immediately and archived in PubMed Central (PMC).and conducted alongside clinical trials 6 .The reasons for such delays in publication are not clearly understood.One reason may be that economic evaluations can be time-consuming to construct, because they typically involve trial data modelled over time and across populations, and data obtained from external sources 6 .Additionally, given that most readers of clinical journals are physicians, not economists or policymakers, manuscripts presenting important clinical findings are often reviewed and published in an expedited fashion 6 .Much less is known about the publication timeliness of cancer-related ceas, which have not generally been pre-planned and conducted alongside clinical trials.Timeliness of publication is important so that reliable and peer-reviewed economic data can be made available when decisions on adoption and reimbursement are made.
With respect to new treatments, concerns have been raised over discrepancies between early clinical data presented at scientific meetings and subsequent manuscript publication [7][8][9][10] .The discrepancies range from differences in reported effect estimates 7 to goals and conclusions 8,9 and differences in sample sizes 10 .To date, we could identify no studies that have looked at discrepancies in cea abstracts and subsequent manuscripts.
The potential for discrepancies would preclude the adoption of early cea data presented in meeting abstracts for drug funding decisions.Equally concerning are potential biases in publication, whereby only statistically significant results are ultimately published.Those biases could potentially be mitigated by comprehensive and timely publication of all early cea data.Krzyzanowska et al. 11 found publication bias in a substantial number of large phase iii randomized control trials presented at an international oncology meeting: The transition from abstract presentation to manuscript publication was associated with whether the final result was statistically significant.Whether there is a similar publication bias based on favourable icers in ceas that have been conducted and presented in abstract form is unknown at this time.
It remains unclear whether there is a significant degree of non-publication for ceas in oncology, or whether significant delays occur between the conduct of such studies and the time of publication.The goals of the present study were to examine the time between abstract presentation at conferences and full publication, to determine the factors associated with time to publication, to evaluate potential evidence of publication bias, and to examine discrepancies in results between abstract and publication.

Data Collection
We manually reviewed abstracts from the American Society of Clinical Oncology (asco), the International Society for Pharmacoeconomics and Outcomes Research (ispor), and the American Society of Hematology (ash) annual meetings from 1997 to 2007.Cost-effectiveness analyses related to malignancies, with primary outcomes such as incremental cost per life-year or quality-adjusted life-year (qaly) gained were included.Studies that were pure costidentification studies, cost-consequences analyses, or cost-minimization analyses were excluded.For each abstract identified, a search for final manuscript publication was conducted using medline, HealthStar, CancerLit, and EconLit, using the names of all authors on the abstract submission or keywords contained within the title of the abstract 11 .All retrieved publications were compared with the original abstracts to ensure that they represented the same study.

Data Extraction
For each abstract identified, these data were extracted: incremental cost-effectiveness ratio (icer), cancer type, country where the study was carried out, and quality indicators, including discussion of time horizon, appropriate description of perspective (societal, third-party payer, etc.), use of discounting, sensitivity analyses, and indication of conflicts of interest.All cost-effectiveness ratios were converted to U.S. dollars at the exchange rate prevalent in the year of publication 12 .

Outcome Measures
The primary outcome of interest was time between abstract presentation and manuscript publication.As a secondary outcome, any discrepancies between the icers in the abstracts and the final manuscripts were evaluated.Final manuscript publication of the abstract was defined as the event of interest.Abstracts that were not published by July 1, 2010, were censored.

Statistical Analysis
The time-to-publication analysis was performed using the Kaplan-Meier method.Potential predictors of time to publication were assessed by univariate Cox propor tional hazards models.In addition, to examine factors that might lead to a higher icer, exploratory univariate regressions were performed using the logarithm of the icer (because icers are skewed to the right) as the outcome variable in a multiplicative model 13 .Discrepancies in the icers reported in abstracts and final publications were examined by calculating the difference between the two icers for the fully published studies.All analyses were performed using the SAS software application (version 9.2: SAS Institute, Cary, NC, U.S.A.).

Current OnCOlOgy-VOlume 20, number 6, DeCember 2013
Copyright © 2013 Multimed Inc.Following publication in Current Oncology, the full text of each article is available immediately and archived in PubMed Central (PMC).
Important information about the design of the ceas was often not mentioned in the abstracts.Fewer than 60% of the abstracts mentioned the time horizons and perspectives of the studies (Table i).
Approximately 70% of the abstracts reported the use of sensitivity analyses, but only 16.5% reported using probabilistic sensitivity analysis.

Potential Predictors of Publication
Abstracts presented through asco were more likely to be published than those presented through ispor   (hazard ratio: 1.94; p = 0.038).Similarly, abstracts seemed more likely to be published sooner when presented at ash than when presented at ispor (hazard ratio: 2.03; p = 0.072; Table ii).
Compared with abstracts that did not report the use of sensitivity analyses, those that described such analyses trended toward timely publication (hazard ratio: 1.42; p = 0.063).Characteristics that did not increase abstract publication included descriptions of the base year of analysis, the time horizon, discounting, cost-utility analysis, and financial conflicts of interest.Assumptions of proportional hazards were examined and found not to be violated.

Relationship Between Publication of the Pivotal Clinical Trial and Abstract Presentation of the Associated CEA
We explored the possibility that cea publication delays were the result of delays in publication of the clinical data on which the economic analysis was based.We found that most clinical studies were published before presentation of the cea abstracts.Of 164 cea meeting abstracts, 100 were associated with pivotal clinical trials, 76 of which were published before presentation of the associated cea abstracts.
Median time for cea abstract presentation was 324 days after clinical trial publication; the 1st and 3rd quartiles were 42.5 and 785.5 days respectively.The mean time was 484 days.

Abstracts with Cost-Utility Analyses and Potential Predictors of Favourable ICERs
Of the 164 abstracts identified, 96 reported cost-utility analysis.The median icer for these 96 cost-utility analyses was US$20,500 per qaly, with 89.6% of the studies reporting a favourable icer, based on a cut-off of US$100,000 per qaly as the threshold for "costeffective."If the definition of a favourable icer were to be changed to a cut-off of US$50,000 per qaly, then 80.2% of the abstracts reported icers at that threshold or lower (Table i).Compared with abstracts that did not explicitly take a societal perspective, those that did take such a perspective reported icers that were higher by a factor of 1.95 (95% confidence interval: 1.02 to 3.74; p = 0.045).Factors such as country of origin of the cea and cancer type were not associated with the reported icer (Table iii).

Abstract and Final Publication ICERs
Of the 65 abstracts that went on to full-length publication, only 48 were included in our comparison for discrepancies between abstracts and publications.In publication, the other 17 studies reported icers as ranges or as a dominant value, neither of which could be compared with the abstract icers.Of the 48 studies compared for discrepancies, 23 (48%) reported a decrease in icer, 14 (29%) reported an increase in icer, and 11 (23%) reported no change.In 24 of the studies (50%), the change in icer exceeded 10%, with a mean absolute difference between abstract and manuscript of 23.8% (Figure 3).The

INTERPRETATION AND CONCLUSIONS
We found that, after presentation of early cea findings at major conferences, the rate of full publication was low, and the full publications typically appeared too late to facilitate open discussion among stakeholders about drug funding.The 5-year publication rate of 40.5% compares unfavorably with the 74% 5-year publication rate for clinical results from large phase iii cancer studies 11 .It is possible that complete analysis and preparation of a final manuscript may take longer for economic evaluations than for clinical trials.Alternatively, clinical results may be prioritized for publication over reporting of economic and other secondary outcomes.To explore the possibility that delays in the publication of cea abstracts were the result of delays in the publication of clinical study data, we examined cea abstracts associated with clinical studies and found that 76% of clinical studies (76 of 100) were published before presentation of the cea abstracts (at a median of 324 days).With the clinical data published almost a year in advance, it would be difficult to argue that cea publication delays are mainly a result of delays in the publication of clinical trial data.
We identified several predictors of early publication.Abstracts describing sensitivity analyses were published in a more timely fashion, perhaps because those studies represented complete analyses at the time of meeting presentation and were imminently ready for final publication.We also found that abstracts presented at cancer-specific conferences (asco and ash) appeared to be published sooner than those presented for an economic audience (ispor).That finding might be explained by greater interest on the part of journals, editors, and audiences in economic studies focusing on more clinically relevant issues.To facilitate knowledge translation and rapid dissemination of economic evaluations, future efforts to understand how those factors influence timeliness in publication are warranted.
A delay in the final publication of cea results would be palatable if preliminary cea data were reliable.Unfortunately, we found meaningful changes in reported icers from abstract to full publication.If policymakers and stakeholders are to make informed economic decisions, they may have to wait for published peer-reviewed results or accept that the icers presented in abstract form may change in the full publication by an average of ±24%.
We observed no direct evidence of publication bias with respect to abstracts having favourable icers.That result contrasts with the findings of Bell et al. 12 , which suggested a trend of publication for  favourable icers.The lack of evidence for publication bias has several potential explanations.First, favourable icers were reported by most abstracts (more than 90%), raising the possibility that studies with unfavourable icers are not submitted (or accepted) to conferences in the first place.Alternatively, given that favourable and unfavourable icers are both informative for decision-making and that the international cut-offs used for drug funding are subjective and variable, it is possible that researchers are submitting favourable and unfavourable results alike for peer review.Moreover, journal reviewers and editors may be publishing submissions with unfavourable icers at a rate similar to that for submissions with favourable results-a situation that contrasts with that for clinical trials, where there is typically more interest in trials with positive or significant clinical findings 14 .
Our study suggests that cost-utility analyses taking a societal perspective were associated with higher icers.That value difference might have from incorporation of costs from a wider perspective (for example, taking into account indirect costs incurred as a result of the new intervention).It is also possible that the use of a societal perspective serves as a surrogate for higher-quality cost-utility analyses and therefore provided a more accurate estimation of the true icers.That explanation is also supported by the findings of Bell et al. 12 suggesting that studies of higher methodologic quality were less likely to report icers below US$20,000 per qaly.
Our study is potentially limited by its dependence on abstracts as an early indication of an economic evaluation being performed.It is possible that many studies were never submitted for abstract consideration, thus leading to an underestimate of the extent of publication bias.The paucity of information presented within abstracts also meant that we were unable to reliably and consistently extract information about studies looking at individual data compared with published results or about whether authors of the original clinical trials also conducted the cea.Additionally, our sample of cea abstracts was small in size, precluding the ability to explore predictors of publication in a multivariable fashion.Finally, we were unable to reliably extract data on the role of pharmaceutical sponsorship, because those data were not required for abstract submission in the earlier years of our study period.Prior research has suggested a link between sponsorship and non-publication of clinical results.Whether sponsorship influences early dissemination of cea results requires further exploration.
Our findings have several implications.Full publications of ceas are unlikely to be available when needed most: at the time when stakeholders are evaluating the balance between clinical effectiveness and costs.Preliminary abstracts are poor surrogates for final manuscripts.And, finally, the quality of abstract reporting may be an early indicator of ultimate publication.
Investigators and trial sponsors have a responsibility to disseminate the results of cancer trials, including clinical and secondary outcomes 11 .Previous proposals to mitigate non-publication have advocated for the creation of trial registries 15 .A priori trial designs that include prospective economic evaluations and inclusion of such outcomes in trial registries might encourage future publication of economic results.Mandating a higher quality of cea abstract presentation at conferences might improve the reliability of data reported in preliminary form and facilitate timely publication.Although efforts have been made in the past to promote the dissemination of clinical and economic results together 14 , more emphasis is clearly required to ensure that important economic information is reported in a timely fashion.

ACKNOWLEDGMENTS
This work was presented in part at the 2009 asco annual meeting and at the 2010 ash annual meeting.

figure 1
figure 1 Time from abstract presentation to publication.The solid line represents the Kaplan-Meier plot for time from abstract presentation to manuscript publication.The dashed lines represent the limits of the 95% confidence band.

figure 2
figure 2 Publication rates by conference.Each line represents the Kaplan-Meier plot for time from abstract presentation to manuscript publication for a specific conference.asco = American Society of Clinical Oncology; ash = American Society of Hematology; ispor = International Society for Pharmacoeconomics and Outcomes Research.

figure 3
figure3 Percentage change in the incremental cost-effectiveness ratio (icer) from abstract to publication for forty-eight cost-utility analyses.Each bar represents a cost-utility study, and the height of the bar represents the percentage change in icer from abstract to publication.Twenty-three studies reported a decrease, fourteen reported an increase, and eleven reported no change in icer.

table iii
Potential predictors of incremental cost-effectiveness ratio (icer) in cost-utility analyses a (univariate analyses) For the assessment of whether the factors listed in the table predict icer, the icer was log-transformed to satisfy the assumption of normality for linear regression.b Significant at p < 0.05.asco = American Society of Clinical Oncology; ash = American Society of Hematology; ispor = International Society for Pharmacoeconomics and Outcomes Research; psa = probabilistic sensitivity analysis. a