Publication Trends of Pediatric and Adult Randomized Controlled Trials in General Medical Journals, 2005–2018: A Citation Analysis

Policy has been developed to promote the conduct of high-quality pediatric randomized controlled trials (RCTs). Whether these strategies have influenced publication trends in high-impact journals is unknown. We aim to evaluate characteristics, citation patterns, and publication trends of pediatric RCTs published in general medical journals (GMJs) compared with adult RCTs over a 13-year period. Studies were identified using Medline, and impact metrics were collected from Web of Science and Scopus. All RCTs published from 2005–2018 in 7 GMJs with the highest impact factors were identified for analysis. A random sample of matched pediatric and adult RCTs were assessed for publication characteristics, academic and non-academic citation. Citations were counted from publication until June 2019. Among 4146 RCTs, 2794 (67.3%) enrolled adults, 591 (14.2%) enrolled children, and 761 RCTs (18.3%) enrolled adult and pediatric patients. Adult RCTs published in GMJs grew by 5.1 publications per year (95% CI: 3.3–6.9), while the number of pediatric RCTs did not show significant change (−0.4 RCTs/year, 95% CI: −1.4–0.6). Adult RCTs were cited more than pediatric RCTs (median(IQR): 29.9 (68.5–462.8) citations/year vs. 13.2 (6.8–24.9) citations/year; p < 0.001); however, social media attention was similar (median(IQR) Altmetric Attention Score: 37 (13.75–133.8) vs. 26 (6.2–107.5); p = 0.25). Despite policies which may facilitate conduct of pediatric RCTs, the publishing gap in high-impact GMJs is widening.


Materials and Methods
We searched GMJs indexed under the "Medicine-General and Internal" category in Journal Citation Reports [9]. Excluded were those GMJs focused solely on adult internal medicine (e.g., Annals of Internal Medicine) and reviews (e.g., Cochrane Database of Systematic Reviews). We selected the seven highest ranked GMJs ranked by two- We defined pediatric and adult RCTs as follows: pediatric RCTs were those RCTs in which the population of interest, and corresponding intervention and comparison groups, were all patients less than 18 years of age. Adult RCTs were defined as RCTs including exclusively adult patients (≥18 years old) in their population of interest as well as their intervention and comparison groups. Mixed RCTs were defined as RCTs with intervention and/or comparison groups including pediatric (<18 years old) and adult (≥18 years old) patients.
RCTs in the included journals were identified using an electronic search strategy and refined using a machine learning algorithm, RobotSearch [18]. An Ovid Medline search strategy (1 January 2005-1 June 2018) was developed by the study team's information science specialist (QM). The Cochrane Highly Sensitive Search, optimized for sensitivity and specificity [19], was used to ascertain RCTs at the search strategy level. This search strategy also stratified RCTs by journal, index year of publication, and age cohort (pediatric, adult, mixed). After comparing Medline age search queries [20,21], we deemed built-in age filters were sensitive enough for our purposes with a sensitivity of 94% compared with a hand search [20]. The electronic search strategy is presented in full in Table 1. Table 1. Ovid Medline search strategy for locating indexed pediatric, adult, and mixed randomized controlled trials (RCTs).

Search Strategy Description
1. randomized controlled trial.pt.

10 and 14
All studies meeting above criteria (base RCT number). 16. limit 15 to "all child (0 to 18 years)" Limit to studies indexed as including children. 17 Non-randomized study designs can be erroneously classified as RCTs in Ovid Medline [22,23]. To improve the specificity of our search, we used a validated machine learning algorithm (RobotSearch) to further refine our ability to accurately identify RCTs. RobotSearch uses a machine learning algorithm to yield a highly sensitive (98.5%) and specific (87.6%) method to screen the results of our search strategy and effectively identify RCTs in our search results [18]. Accuracy of RobotSearch was verified by a hand-search of all 97 publications detected by our search strategy from JAMA in the journal year 2018. RobotSearch had a sensitivity of 97.0% and a specificity of 83.8% when compared to a hand search.
We measured pediatric and adult RCT quantity as the number of indexed RCTs in each calendar year across all seven selected journals. In addition, we assessed the number of RCTs over the study period for each individual journal.
We also assessed publication characteristics and impact for a subset of pediatric and adult RCTs. A sample of 10% of included pediatric RCTs ascertained in each electronic search was selected using a random number generator and matched (1:1) to an adult RCT published in the same journal and year.
The following RCT characteristics were collected: publishing journal, RCT design (parallel, cluster, crossover, factorial), hypothesis type (superiority, noninferiority, equivalence), "positive" or "negative" results as reported by the authors, sample size, number of centres in multicentre trials, number of countries involved in multinational trials, continents involved in the RCT (Africa, Asia, Europe, North America, Oceania or South America), degree of blinding of intervention (no blinding, single-blind, double-blind or more), and industry funding (yes/no). RCTs were classified as being conducted on a continent if at least one study centre was located on that continent. In cases where centre location and other characteristics were ambiguous in the trial manuscript and included supplements, clinical trial registries provided information that obviated the need for further investigation. Clinical trial registries used for additional clarification included ClinicalTrials.gov, International Standard Randomised Controlled Trials Number (ISRCTN) registry, Karolinska Clinical Trial Registration, Australian New Zealand Clinical Trials Registry (ANZCTR), and the Netherlands Trial Register (NTR). While many trials were registered in other national trial databases, these trials were all either registered in one of the above registries as well, or provided information was unambiguous enough to not warrant further investigation.
Study sample size was collected because it may serve as a proxy for a study's susceptibility to random error. Small sample sizes may result in the overestimation or underestimation of treatment effects [24,25]. We collected data on how many centres participated in each study and where they were located; multicentre and particularly multinational trials are indicators, at least in part, of the generalizability of a trial. We defined RCTs with double-blinding or greater as ones blinding the patient and any other parties, including the investigators, outcomes assessor, data collectors, and data analysts, among other possible parties.
We defined "industry-funded" RCTs as those with financial compensation reported from a for-profit organization, as opposed to institutional grants and governmental organizations. We explored the relationship between funding source and the authors conclusions in our sample, as industry funding has been previously shown to be associated with publication and interpretation biases of results [26][27][28]. If a relationship exists in either adult or pediatric RCTs this may imply a form of publication bias or biased interpretation of trial results [29].
RCT impact was assessed using a variety of measures of academic and non-academic citation. These measures were collected through Web of Science and Scopus. The Altmetric Attention Score and PlumX Index account for references to the study from various forms of media. The PlumX measure is a count of all types of references, stratified across five different domains. The Altmetric Attention Score provides a measure of the amount of attention an article has received since publication [30]. This measure is presented as a weighted count of all types of citation/media categories, each receiving a different weighting. Web of Science was used to provide a conservative citation count. Web of Science, generally speaking, has a narrower coverage of biomedical journals than Scopus and will therefore give a more conservative citation count [31,32]. As there will be variation in citation estimates based on the database used, we have opted to analyze both Web of Science and Scopus data. Altmetric data from Web of Science and PlumX data from Scopus were used to provide estimates of non-academic citation. Scopus indexes a broader range of journals than Web of Science, although Web of Science and Scopus have considerable overlap [31,32]. We collected data on citation numbers and frequencies from Web of Science and Scopus. Analyses were conducted using Web of Science and Scopus data; however, results did not differ between the two. We elected to use Web of Science data for the primary analysis.
The numbers of pediatric and adult RCTs per year published across the seven selected GMJs, 2005 to 2018, were assessed using linear regression. An interaction term was used to determine whether the (change in) publication rate of pediatric RCTs versus adult RCTs was different. Given that some trials report on combined pediatric and adult populations, a sensitivity analysis was conducted with the assumption that all trials indexed as including both pediatric and adult patients were either pediatric or adult trials, respectively.
Pediatric and adult RCT characteristics were compared using chi-squared tests for nominal and ordinal outcomes. Mann-Whitney tests were conducted to assess for differences in sample size and to compare impact scores of pediatric and adult RCTs. A sensitivity analysis using Scopus data in place of Web of Science data for citation number, average citation frequency, and citation data was conducted and no notable differences were found. All p-values were 2-tailed, and values <0.05 were considered significant. Analyses were conducted using STATA 15.0 (StataCorp, College Station, TX, USA).
Children 2020, 7, 293 5 of 12 was conducted and no notable differences were found. All p-values were 2-tailed, and values <0.05 were considered significant. Analyses were conducted using STATA 15.0 (StataCorp, College Station, TX, USA).     Table 3 compares characteristics of adult (n = 64) and pediatric (n = 64) RCT samples. Adult and pediatric RCTs had similar proportions of parallel-group RCTs, superiority hypotheses, and positive results.      Table 3 compares characteristics of adult (n = 64) and pediatric (n = 64) RCT samples. Adult and pediatric RCTs had similar proportions of parallel-group RCTs, superiority hypotheses, and positive results. Pediatric RCTs were more likely to be conducted in Africa (28.1% vs. 9.4%, p = 0.007) and Oceania (23.4% vs. 7.8%, p = 0.02), while adult RCTs were more likely to be conducted in Europe (70.3% vs. 34.4%, p < 0.001), and North America (59.4% vs. 25.0%, p < 0.001). All pediatric RCTs from Africa were conducted exclusively in Africa. All but four of these RCTs address an infectious disease such as malaria (9/18 Table 4 summarizes measures of academic and non-academic citation. Using Web of Science, adult RCTs were cited more often compared to pediatric RCTs (median (IQR): 29.9 citations/year (IQR: 12.5 to 53.3) vs. 13.2 citations/year (IQR: 6.9 to 24.9), p < 0.001). Changing databases did not change these results; adult RCTs were also cited more often compared to pediatric RCTs using Scopus (median (IQR): 33.7 citations/year (13.9 to 57.1) vs. 14.2 citations/year (13.9 to 57.1), p < 0.001). Adult trials also had higher PlumX "Citations" scores. Altmetric scores for adult RCTs were similar to pediatric RCTs

Discussion
Our study found that adult RCTs published in high-impact GMJs are continuing to increase in number with no change in the number of published pediatric RCTs. RCT characteristics were similar between adult and pediatric trials, although adult RCTs were more likely to be multinational. Adult RCTs were more frequently cited in academic publications, but not in other media. Despite important policies to promote the conduct of RCTs in children implemented in the past two decades, our results suggest there may be ongoing barriers to the publication of high-quality pediatric RCTs in GMJs.
Our findings are similar to a report of the five highest impact GMJs conducted by our study team from 1985-2004 [8]. We found similar growth in the number of RCTs involving adults (5.1 RCTs per year (95% CI: 3.3 to 6.9) in the current study vs. 4.7 RCTs per year (95% CI 3.6 to 5. RCTs per year (95% CI −0.0 to 0.9)). Taken together, the results of these two studies suggest that the gap between the annual number of published adult and pediatric RCTs in GMJs has continued to widen for over 30 years. There are a number of possible explanations for our findings. First, legislative and other policy initiatives may have been ineffective at promoting the initiation, completion, or publication of pediatric trials. Initiatives have aimed to promote the conduct of pediatric RCTs by giving industry sponsors increased patent durations and vouchers for expedited product review. For some interventions, regulations set out by these policies mandate conduct of a pediatric RCT alongside an adult RCT unless a waiver is warranted. Examples of these policies include the Best Pharmaceuticals for Children Act and Pediatric Research Equity Act-both now permanently under the Food and Drug Administration Safety and Innovation Act of 2012 in the United States [14,15,33] and the 2007 Pediatric Regulation in Europe [13], respectively. Although there have been a greater number of interventions evaluated for safety and efficacy in children [16,33], industry and governmental stakeholders may still lack adequate incentives to sponsor trials to completion. For instance, the Pediatric Rare Diseases Voucher Program, which provides direct incentives for the conduct of pediatric RCTs aiming to treat rare diseases, has not been associated with any increase in the rate of pediatric drug testing [17] and has since undergone revisions [34].
When a pediatric RCT is initiated, unique aspects of pediatric RCTs such as nuances of informed consent may lead to poor recruitment. An inability to meet recruitment targets in pediatric trials has been associated with the non-submission of RCTs for publication [35]. However, evidence suggests poor recruitment is not independently attributable to the pediatric setting [36]. There may also be ethical and logistical barriers to conducting high-quality pediatric RCTs such as unique aspects of consent/assent in pediatric trials [37] in addition to the common struggle of poor recruitment [35,36]. Consent rates in pediatric RCTs have improved over time, but are still considered suboptimal [38,39]. However, it is noteworthy that our findings are qualitatively similar to findings before the institution of policy changes to minimize these barriers [8], suggesting other factors may be at play. Publication may be more challenging because pediatric RCTs may have suboptimal quality [7,40,41], although comparative studies consistently found no detectable difference in quality between pediatric and adult RCTs [6][7][8]. While we did not use a standardized quality assessment tool, we did find differences in some indicators of study quality or rigor (e.g., blinding) but not others (e.g., sample size, multicentre trials).
There could also be barriers at the level of GMJs. Authors of pediatric RCTs may not submit trials to GMJs and/or the editors of GMJs may be less likely to accept them. Authors of pediatric RCTs may not submit their RCTs for publication due to factors such as perceptions about statistical or clinical significance, agreement with their initial hypotheses, and authors' perceptions of the trials' likelihood of acceptance by high-ranking GMJs [42]. Because of these perceptions, authors of pediatric studies may prefer submitting their works to smaller, more "pediatric-friendly" journals. However, our findings are also similar to results from a report of a large difference between the growth of adult and pediatric RCTs per year (90.5 RCTs/year vs. 16.9 RCTs/year) from 1985-2005 in general pediatric and specialty journals, suggesting that our findings may not be unique to GMJs [6].
The editors of GMJs may prefer publishing research of general medical interest, which may be more focused on adult-onset disease, particularly those prevalent in high-income countries with an aging population. Conversely, pediatric issues such as infant mortality, infectious diseases, and diseases of poverty may have larger impacts in lower income regions of the world [12]. Pediatric RCTs conducted in lowand middle-income countries also face several unique challenges due to the importance of community engagement, power differentials, medical mistrust, and other ethical issues. These challenges may act as barriers to the production of high-quality pediatric RCTs from low-and middle-income countries [43]. We also observed that pediatric RCTs conducted in Africa frequently study infectious diseases and diseases of poverty such as malnutrition. This may explain our finding of more pediatric RCTs conducted in global regions with a larger concentration of low-income countries (e.g., Africa), and published in GMJs with a more deliberate focus on global health (e.g., the Lancet). The promotion of RCTs in a global health setting may be one strategy to increase publication of pediatric RCTs, although such studies may not address important health issues affecting children in high-income and middle-income countries. Another explanation is that journals may preferentially publish adult RCTs due to higher citation counts and a perceived higher impact. Adult RCTs were more frequently cited in GMJs, which may reflect study quality or impact. However, a relationship between citations and study quality has not been consistently reported [44], and academic citation metrics have been criticized for their poor ability to measure true research impact [45]. Previous studies have not compared adult and pediatric trials in terms of broader impact, including social media. Our finding of similar social media scores in Altmetric suggests that the non-academic attention received by pediatric RCTs published in GMJs may not differ substantially from adult RCTs.
One of the unique strengths of our study is our use of a study selection strategy which combined a highly sensitive search strategy for detecting RCTs [19] with a machine learning algorithm, RobotSearch [18]. Although this RCT filtering technology has not been used much previously, comparison with manually abstracted data from one journal year found that this method had high, albeit imperfect, precision. While age indexing was suboptimal, sensitivity analyses showed that even if all mixed RCTs are misclassified this would not have affected our findings.
This study has a few important limitations. First, we limited our analysis to GMJs. The majority of pediatric RCTs-including many high-quality trials-are published in general pediatric or pediatric subspecialty journals. Second, our study was limited to seven GMJs with the highest impact factor, meaning RCTs published in these journals are likely not representative of all GMJs, and certainly not all pediatric and adult RCTs. Third, we used a well-accepted cut-off of 18 years to distinguish between pediatric and adult RCTs [41]. Although this age cut-off is widely used to define pediatric populations, it is to some extent dependent on the geographical region of study. If we used a different age cut-off, we may have found somewhat different findings. Fourth, while the addition of the machine learning algorithm to our search strategy did increase efficiency so that we could identify a large number of studies, it did not have perfect sensitivity. Although we may have missed some trials, our full-text review process ensured optimal specificity. Fifth, our secondary analyses of study characteristics and impact were underpowered to detect small differences between groups due to the relatively small numbers of RCTs compared. Lastly, we assessed RCT impact using traditional academic as well as more contemporary non-academic metrics, but such metrics are approximate estimates of true impact. The influence of the RCTs on clinical practice and policy could not be assessed.

Conclusions
Despite policies which may promote the conduct of high-quality pediatric RCTs, we found no substantial increase in pediatric RCTs published in high-impact GMJs, while the number of published RCTs in adults has continued to grow, a gap that has now persisted for over 30 years.