Factors associated with publication of randomized phase iii cancer trials in journals with a high impact factor

A compilation of ifs is published annually by Thomson–Reuters as Journal Citation Reports. Although if was not designed to measure journal quality, and the ifs of journals publishing in various medical disciplines will vary, articles published in medical journals with a high if are often perceived to be of high quality and clinical relevance2. In addition, if can be affected by editorial practices. For example, the if can be increased by including a higher percentage of review articles, which are cited more often than other types of articles3. Regardless of the many caveats associated with if, busy clinicians might, because of time constraints, read only journals with high ifs. Their opinions and clinical practices are therefore more likely to be influenced by articles published in such journals. In a review of published emergency medicine studies, the strongest predictor of citations per year was the if of the publishing journal4. Furthermore, implicitly or explicitly, ifs can influence committees in assessing academic promotions, merit, funding, grants, and appointments3,5,6. ABSTRACT


INTRODUCTION
The impact factor (if) of a journal for a given year is calculated by dividing the number of citations that year to articles published in the journal in the preceding 2 years by the number of "citable" articles published in those 2 years 1 .For example: A compilation of ifs is published annually by Thomson-Reuters as Journal Citation Reports.
Although if was not designed to measure journal quality, and the ifs of journals publishing in various medical disciplines will vary, articles published in medical journals with a high if are often perceived to be of high quality and clinical relevance 2 .In addition, if can be affected by editorial practices.For example, the if can be increased by including a higher percentage of review articles, which are cited more often than other types of articles 3 .
Regardless of the many caveats associated with if, busy clinicians might, because of time constraints, read only journals with high ifs.Their opinions and clinical practices are therefore more likely to be influenced by articles published in such journals.In a review of published emergency medicine studies, the strongest predictor of citations per year was the if of the publishing journal 4 .Furthermore, implicitly or explicitly, ifs can influence committees in assessing academic promotions, merit, funding, grants, and appointments 3,5,6 .

Background
Impact factor (if) is often used as a measure of journal quality.The purpose of the present study was to determine whether trials with positive outcomes are more likely to be published in journals with higher ifs.

Methods
We reviewed 476 randomized phase iii cancer trials published in 13 journals between 1995 and 2005.Multivariate logistic regression models were used to investigate predictors of publication in journals with high ifs (compared with low and medium ifs).
In a review of neonatal literature, Littner et al. 7 recently showed that randomized phase iii clinical trials (rcts) with negative results are more likely than those with positive results to be published in journals with lower ifs.Whether that observation is true in oncology as well is not clear.The primary objective of the present study was to determine whether oncology rcts with statistically significant positive results are more likely than those with negative results to be published in journals with higher ifs.

Identification of Studies
Using the oncology section of Journal Citation Reports for 2005, journals with an if exceeding 3 were identified (n = 51).Another 3 journals in the general or internal medicine category that frequently publish oncology rcts were also identified.We then limited our scope to journals that had published at least 3 original rcts during the period of interest (n = 13).The identified journals publish nearly all the oncology rcts in the English language [8][9][10][11]  Phase iii trials were defined as rcts of cancer therapy published in one of the identified journals between 1995 and 2005.Trials with fewer than 100 patients per arm were excluded because their statistical power is sufficient to detect only large differences in treatments, and thus would be more likely to have a negative primary outcome.The rcts were identified in a manual search of each journal.Exclusion criteria were a pediatric population (<18 years of age); investigation of supportive care or palliative care, education, diagnostic imaging, screening or prevention; evaluation of different dosing schedules of the same regimen; and evaluation of bone marrow transplantation.Articles were also excluded if they involved only meta-analyses, subset analyses, overviews, or pooled data from two or more trials.For multiple reports (the same trial published more than one time), the first publication was arbitrarily chosen for inclusion in the study.

Data Collection
For each identified trial, the information collected included year of publication; journal of publication; cancer type; intent of therapy, type of therapy, and duration of accrual; number of patients randomized; number of treatment groups; use of any blinding; sample size calculation; nationality of authors; method of analysis; time to publication; and whether an independent committee reviewed response data.The source of trial funding was categorized as "nonindustry" (for sponsorship by a government, foundation, or other nonprofit agency), "industry" (for full or partial sponsorship by a pharmaceutical company), or "not stated."If funding was not explicitly stated, the primary endpoint of the rct was recorded as the outcome used in the sample size calculation.Data describing the sample size calculation (present or absent) and the type of analysis [intention-to-treat (itt) or other] were used to assess quality of reporting.The nationality of authors was defined as North American, European, international, or other using the affiliations of the authors or the cooperative group.The term "international" was used if authors came from more than one region.
A trial was classified as positive if an experimental arm was statistically superior to the control arm (p ≤ 0.05) for the primary endpoint.Studies for which the primary endpoint was not explicitly defined were classified as positive or negative based on overall survival.Noninferiority studies were classified as positive if the criteria for noninferiority were met; otherwise, they were classified as negative.
Data collection was carried out by one investigator (PAT).As a measure of data integrity, a second investigator (SW) evaluated a random sample of 61 publications, with an overall disagreement rate of 1.21%.Differences were resolved by consensus.

Statistical Analysis
Study characteristics are summarized using descriptive statistics such as median, proportion, and frequency.Time to publication was calculated by subtracting the year that enrollment was completed from the year of publication.Categories of predictors were grouped for statistical power after data collection, but before analysis.Breast, colorectal, lung, and prostate cancers were grouped as "common" tumours.Intent of therapy was grouped as adjuvant or other, trial results as positive or negative, primary endpoints as overall survival or other, source of funding as industry or other, and method of statistical analysis as itt or other.
Potential predictors of publication in high-if journals were defined before data analysis and included cancer type, type of therapy, intent of therapy, sample size, accrual rate, primary endpoint, nationality of authors, publication year, duration of accrual, time to publication, presence of an independent review committee, inclusion of the sample size calculation, itt analysis, industry funding, and positive outcome.Because of the potential influence of journal-specific editorial practices over inclusion of a sample size calculation, supportive analyses were performed excluding that variable.Predictors of publication in high-if journals (compared with low-and medium-if journals) were investigated using univariate and multivariate logistic regression models.Linear regression was also performed using actual if as the outcome.Results for both methods were similar, and so only the logistic regression results are presented.Multivariate models were constructed using a forward stepwise selection process.
Because if can vary by discipline, analyses were performed in two ways: cluster analyses assumed that trials conducted within the same tumour site were correlated, and a comparison of only mediumif journals with low-if journals excluded general medical journals (New England Journal of Medicine, JAMA, The Lancet).
Tests for trends over time were performed using the Cochran-Armitage test 12 .Because the consort statement was initially published in 1996 and then updated in 2000 [13][14][15] , rcts were grouped by publication year (1995-1997, 1998-2000, and 2001-2005)  for the analysis of reporting quality over time.All tests were two-sided, and a p value of 0.05 or less was considered statistically significant.

Trends and Characteristics of the RCTs
In total, 828 rcts were identified, and 476 met the inclusion criteria.Figure 1 lists the reasons for exclusion.Table i summarizes the characteristics of eligible rcts.More than half the studies were published in journals with a medium if (n = 261, 54.8%), and the most common cancer site was the breast (n = 114, 24.0%).Median time to publication was 4 years (range: 1-18 years).Across all years, publications from European authors accounted for 52.9% of the total.
With respect to quality of reporting, 70 articles (14.7%) contained no explicit description of the sample size calculation.The method of statistical analysis was treatment received in 6 articles (1.3%) and not stated in 110 articles (23.1%), and the primary endpoint was not stated in 36 articles (7.6%).However, the reporting of those measures improved significantly (p < 0.001) over time (Figure 2).
The study outcome was classified as positive (statistically significant results favouring the experimental arm) in 162 trials (34.0%).Approximately 75% of the 106 trials published in low-if journals had a negative outcome, as did 65% of the 261 trials published in medium-if journals.However, more than 60% of the 109 trials published in high-if journals had positive outcomes [Figure 3(A)].By tumour site, the highest proportion of trials with positive outcomes appeared in high-if journals compared with mediumand low-if journals [Figure 3(B)].

3.2.1
Supportive Analyses After excluding the general medical journals (New England Journal of Medicine, The Lancet, JAMA), only journals with low and medium ifs remained.Tables ii and iii present results from the logistic regression analyses (univariate and multivariate respectively).In the multivariate analysis, itt analysis was most strongly associated with publication in the medium-if journals Journal of Clinical Oncology and Journal of the National Cancer Institute (p = 0.008), as was positive outcome (p = 0.025), adjuvant therapy trial ( p = 0.044), and nationality of authors ( p < 0.001).Interestingly, industry-funded studies also entered the multivariable model (p = 0.027), although it must be noted that automated selection processes are known to occasionally produce false-positive results.A cluster analysis gave results similar to those for the overall analysis, and thus for brevity, those results are not presented.

DISCUSSION
We reviewed 476 phase iii randomized cancer trials published between 1995 and 2005, finding that trials with statistically significant positive outcomes were more likely to be published in high-if journals.More than 60% of the trials published in high-if journals had positive outcomes; only 25% trials published in low-if journals had positive outcomes.Those results are consistent with results in earlier reviews of trials in hematology and neonatology 7,16 .Taken together, those findings suggest an "impact factor bias" in publication, an observation that is concerning, given that trials published in journals with high ifs are, by definition, more widely read and cited.Thus, the more frequent publication of trials with positive outcomes in those journals will not only provide a biased view of the literature to readers, but will also likely exert more influence on the practice patterns of clinicians.Highly cited studies do not always constitute the new "gold standard," because they could potentially be followed by trials demonstrating contradictory or stronger effects 17 .Gluud et al. 18 examined the relationships between journal if, trial quality, and study outcome in 530 hepatobiliary rcts.However, their review did not include data about the primary endpoint, and the median sample size was only 45-meaning that most of those rcts would not be powered for survival.The percentage of positive trials was 71% in their series, much higher than the 34% observed in our study or the 33% observed in a review by Yanada et al. 16 .Gluud et al. 18 did not observe any association between if and study outcome, but it is likely that differences with respect to primary endpoints in hepatobiliary trials account for the observed high prevalence of positive trials and the lack of an association between study outcome and if.
"Publication bias" refers to the publication or non-publication of research findings depending on the nature and direction of those findings 19 .It has been well documented that, compared with rcts reporting negative results, those showing significantly positive results are more likely to be published quickly [20][21][22][23] .In a review of outcome reporting bias studies, three publications found that, compared with trials having nonsignificant outcomes, those with statistically significant outcomes had better odds of being fully reported 24 .The same studies found that, in 4%-50% of published trials, at least one primary outcome had been changed, introduced, or omitted since registration of the original protocol 25 .Another possible manifestation of publication bias is the increased likelihood that trials with positive outcomes will be published in journals with higher ifs.
Previous studies have indicated that failure to publish results predominately from lack of submission or loss of interest by investigators because of negative results rather than f rom rejection of manuscripts 23,26 -28 .Two groups have systematically evaluated the characteristics of submitted manuscripts associated with publication in JAMA 29 and in the British Medical Journal, The Lancet, and the Annals of Internal Medicine 30 .Positive results were reported in 51.4% of studies submitted to JAMA and in 87% of studies submitted to the other three major journals.However, submitted manuscripts were not more likely to be published if they reported positive outcomes, demonstrating a lack of bias at the editorial level for those journals.Investigators should therefore be encouraged to submit trials with negative outcomes, but conducted with high methodologic standards, to high-if journals.
The consort statement was created to improve the quality of reporting of rcts and was first published in 1996 [13][14][15] .We found that data about the essential elements of rcts such as primary endpoint or sample size calculation were missing in 7.6% and 14.7% of publications respectively.Although 4 of the 13 journals selected for the present study have not formally endorsed the consort statement (International Journal of Radiation Oncology • Biology • Physics, Leukemia, Breast Cancer Research and Treatment, and British Journal of Cancer), only 42 of the publications in our study (8.8%) were published in those journals.In our analysis, reporting of the primary endpoint, the sample size calculation, and the method of statistical analysis improved with time.The quality of reporting is a reflection both of the submission and of editorial practices.
Pharmaceutical company involvement in medical research is an area of growing concern.A source of funding can influence research design and outcome 8,9,31,32 .In a multivariate analysis of rcts in breast, colorectal, and non-small-cell lung cancer, a significant p value for the primary endpoint and industry sponsorship were each independently associated with endorsement of the experimental arm 8 .We found that industry funding was not associated with publication in high-if journals, which is consistent with an earlier study focusing on economic analyses in oncology 33 .When the analysis was limited to oncology journals, industry funding was associated in logistic regression, but not in linear regression, with publication in the Journal of Clinical Oncology or the Journal of the National Cancer Institute.In our study, sponsorship for rcts with partial or full funding from a pharmaceutical company were categorized as "industry."The absence of a "mixed" category might have influenced our results.
Trials assessing adjuvant therapy and involving uncommon tumour types were more likely to be published in high-if journals.Whether that likelihood reflects a preference of the submitting authors or of the editorial boards is unknown.Such trials might possibly be perceived to be of higher clinical importance and more likely to be published in high-if journals.North American authors were more likely to publish in higher-if journals.One potential explanation for that observation is that two of the three high-if journals in the present study are published in North America, and several analyses have suggested the presence of an editorial bias toward accepting manuscripts whose corresponding author lives in the country where the publishing journal is located 30,34,35 .Alternatively, authors might be more likely to submit manuscripts to journals that are published in their country of residence.
Our review encompasses only the 1995-2005 period.Temporal changes in "impact factor bias" since the conduct of the study-specifically in the era of targeted therapy-are unknown and are a major limitation of this project.
One potential limitation of the present study is that we did not use a formal scale, such as the Jadad scale 36 , to assess the quality of rcts included in the present analysis.Currently, there is no commonly accepted measure of quality for rcts in oncology, and it is difficult to apply scales developed for other medical specialties (such as the Jadad scale) because many rcts in oncology are not double-blinded.Furthermore, assessment of methodologic quality depends on the particular scale used 37 .We applied a component method instead.Two key components that have been used by other groups were selected: analysis according to the itt principle, and description of a sample size calculation 38,39 .
Another limitation of our study is the inclusion of only 13 English-language journals.It is likely that some oncology rcts are published in journals not included in the present study, although it is also likely that the number of those publications would be small.It is important to note that we arbitrarily classified journals with an if of 3-10 as the low-if group.In other scientific disciplines, those ifs would be considered high.The inclusion of disease-specific oncology journals-namely, Leukemia and Breast Cancer Research and Treatment-in the low-if group could potentially lead to overrepresentation of those diseases.However, only 8 trials that met our inclusion criteria were published in those two journals, and thus the effect was minimal.In addition, we did not assess the clinical implications of the study intervention.Trials that have the potential to be practice-changing and that address important questions are more likely to be published in high-impact journals regardless of the outcome of the trial.

CONCLUSIONS
Our study demonstrates that an "impact factor bias" affects the publication of oncology rcts.Trials with positive outcomes are more likely to be published in journals with higher ifs, independent of other measures of methodologic quality.Investigators should be encouraged to submit trials of high methodologic quality to high-if journals regardless of the study outcomes.

CONFLICT OF INTEREST DISCLOSURES
This research received no specific grant from any funding agency in the public, commercial, or not-forprofit sectors.The authors have no financial conflicts of interest to disclose.
2009 if = citations to articles published in 2007 and 2008 / articles published in 2007 and 2008.

figure 1 consort
figure 1 consort flow diagram for included and excluded randomized controlled trials.

figure 2
figure 2 Trends in report quality indicators for randomized controlled trials in oncology.
. Journals ifs were arbitrarily categorized as low, medium, or high according to Journal Citation Reports for 2005.The low group (if: <10) included

table ii
Univariate analysis for predictors of publication in journals with a higher impact factor

e570 Current OnCOlOgy-VOlume 21, number 4, August 2014
Copyright © 2014 Multimed Inc.Following publication in Current Oncology, the full text of each article is available immediately and archived in PubMed Central (PMC).

table iii
Multivariate analysis for predictors of publication in journals with a higher impact factor Current OnCOlOgy-VOlume 21, number 4, August 2014 Copyright © 2014 Multimed Inc.Following publication in Current Oncology, the full text of each article is available immediately and archived in PubMed Central (PMC).