Knowledge Production in Two Types of Medical PhD Routes—What’s to Gain?

Abstract

Purpose: To assess the assumption that differences exist between the traditional and publication-based PhD routes in terms of the thesis’ length and the scientific publications originating from it. Method: A retrospective comparative study on medical PhD theses offered by an online repository was performed. All free full-text medical PhD theses defended at United Kingdom institutions between 2003 and 2015 were analyzed and assigned to the traditional (TT) or publication based thesis (PBT) group. Several characteristics of theses and thesis-related articles were collected and analyzed. The thesis-related articles were investigated regarding quantity and visibility (citations, impact factor, and journal rank). Results: The theses length proved similar in PBT and TT group. PBT group included significantly more studies than TT group (mean 4.44 vs. 2.67) also reflected in significantly more thesis-related articles. The percentage of articles listed in Web of Science and published in a journal with impact factor proved significantly lower in TT compared with PBT group. On the contrary, article citations were significantly higher for TT. Both groups published similarly in high-ranked journals (Q1 or Q2). Conclusion: The research productivity originating from the PBT group was, as expected, significantly larger but not significantly more visible than those from TT group.

1. Introduction

Production of new scientific knowledge has been a well-analyzed subject, which ranges from individual researcher’s productivity to that of teams, of departments, and of institutions. Many factors were assessed with regard to research productivity [1,2,3]. In current circumstances, when academics are expected to publish in high impact scientific journals [4] and manage large amounts of work [5], doctoral productivity was not left unexamined. Previous studies showed that most new knowledge has been produced by young scientists and recruits [6] and that most major scientific discoveries were the contribution of young people [2]. Doctoral research is considered an important source of scientific publications for universities [7]. This study is placed in the context of general academic pressure to publish and of emerging new graduate or doctoral programs, which have expanded beyond the traditional PhD in many countries. Our research questions whether the knowledge production, publication, and visibility differ significantly in the traditional PhDs than in the PhDs by publication. Furthermore, the two PhD routes are contrasted regarding theses’ length and several aspects related to knowledge publication.

In the biomedical sciences, in the case of the traditional doctoral thesis (monograph), the PhD student does research under the supervision of his/her coordinator, writes and defends the thesis and, according with individual institution requirements, publishes the results before or after the defense of the thesis. The traditional PhDs test the individual’s integrity as a researcher, but in most cases, do not lead to the development of collaborative skills. The non-traditional publication-based doctorate programs allow the PhD student to study part-time and the thesis consists of a collection of multi-authored manuscripts and publications [8]. The publication-based doctorate is in many cases financially supported by grants and the pressure to generate more publications is high. This type of PhD encourages, besides the development of research and writing skills of the PhD student, also the development of collaborative skills. The shift to publication-based PhDs was due to several factors, such as a better focus on the development of the doctoral student’s skills and training, improvements of submission and completion rates, improvements of supervision quality, changes in the examination process of a thesis, and the introduction of benchmarking [9].

Literature Review

The doctorate by publication originated from Northern Europe and can be trailed back to the early twentieth century, when it was awarded to staff members for a collection of previously published papers [10]. In time, other countries adopted this publication-based route, like Bulgaria, Czech Republic, Estonia, United Kingdom (UK), and Australia [11,12].

In 1996, the United Kingdom Council for Graduate Education defined the PhD by publication as the award given to a candidate whose thesis consists entirely or predominantly of refereed and published articles in journals or books which are already in the public domain. Submissions for this award are required to consist of a coherent body of work of the same quality, rigor, and volume as a standard (or traditional) PhD in the specific field and to constitute an original contribution to knowledge [13]. There is also a prospective model where young researchers accumulate publications during candidature with the purpose of constructing a PhD [14].

The two above-mentioned PhD routes coexist and are subject to the same academic requirements for the creation of an independent and cohesive body of new scientific knowledge [8]. This has led to a controversy over which PhD route is more suitable. Studies contrasted the two PhD routes from different perspectives like: the educational process and its use to the graduate’s development, the graduate’s employment perspectives after the PhD award, the quality and the ownership of the research output, relevance and timely dissemination of research results, and the thesis structure and quality.

The traditional PhD’s main outcome is the research content [9]. Writing a monograph is considered an isolating process, it does develop writing abilities but not those needed for scientific article production, supervisors evaluate large bodies of work mostly at the end of candidature and have little time for dissemination of results [10,11,15]. Furthermore, the PhD candidates have difficulties in publishing their results and, by the time a traditional thesis is submitted, it might have lost its appeal [10,11,15].

The PhD by publication focused on the researcher’s training [9]. Collaborating with other researchers (provided the doctoral student has a significant contribution to research), writing articles and dealing with critical appraisal from blinded journal referees were considered to help develop better communication and writing skills, abilities needed for scientific article production [10,11,16]. In addition, the peer-review publishing process could be proof of research relevance [10].

There are conflicting views on the PhD by publication being adaptable to changes in interests as it progresses [10,17]. Compared with the traditional thesis, the collection of publications can make the thesis appear disjointed and repetitive [18], in some cases possibly contributing to a lack of coherence and depth [19]. Alternatively, the collection of publications divides the thesis into more manageable components, leading to a closer relationship with practice than was often possible with the traditional PhD, to timely dissemination of new knowledge (a great advantage in fields where knowledge devalues quickly), and to the distribution of supervisory workload throughout the candidature [10,11].

Golde and Dore stated there was a mismatch among the purpose of doctoral education, the aspirations of the students, and the realities of their careers within and outside academia [20]. Helping develop transferable skills during the training period for PhD by publication was intended to increase the graduate’s chances at employment outside academia [9].

So far, the research on doctoral theses has been scarce because of theses’ extended text, limited access to full-text theses from university libraries, and the perceived low quality or poor design of many dissertations [7]. Some of these factors may have diminished due to online repositories and emerging doctoral routes.

Several studies have assessed the advantages and disadvantages of the traditional PhD versus the PhD by publication regarding the doctoral educational process, ownership, and the thesis structure and quality [9,10,11,12,13,14,15,16,17,18,19,20]. There were few studies which aimed at assessing doctoral knowledge production in the medical field, and to the authors’ knowledge, none to contrast knowledge production of the traditional and publication-based PhDs. This study focused on assessing the assumption that the main difference between the traditional PhD and publication-based PhD routes, besides composition, was the size of the thesis and the publications originating from it, as it was only stated in a previous study by Davies and Rolfe [10]. Nonetheless, aspects like publication time span, publication visibility, and some publication factors were evaluated as well. The traditional PhD route may test the integrity of a researcher, but may not necessarily encourage the collaborative skills, while publication-based PhD route sustains publications and encourage the collaborative skills.

A comparison of the two PhD routes based on the quantity of new knowledge produced by doctoral students during their candidature and the assessment of the thesis-related articles’ visibility would complement the subject. Such a comparison would be valuable in the current context of growing pressure on doctoral students to publish during candidature [2,21], and to complete their PhD in a shorter time frame [22].

The aim of our study was to assess the assumption that there are differences between the traditional and publication-based PhD routes in terms of the thesis’ length and the scientific publications originating from it.

More specifically, this study intends to:

• Assess theses’ length differences between the traditional and publication-based PhD routes based on total number of words, on pages with and without the References and Appendix Sections;
• Assess knowledge productivity differences between the two PhD routes based on the number of studies presented in theses, on the number of thesis-related articles published or accepted for publication until PhD completion;
• Assess publication time differences between the two PhD routes based on the time span between two consecutive articles from the same thesis;
• Identify aspects of the two PhD routes which can contribute to doctoral knowledge publication: number of contributors, research funding, PhD student as first thesis-related article author, number of authors for thesis-related articles, journal location;
• Assess thesis-related articles’ visibility differences between the two PhD routes based on the number of citations, journals’ impact factor, and rank.

2. Materials and Methods

This is a longitudinal study providing a descriptive and analytical research, which evaluated bibliographic and bibliometric factors, as well as aspects regarding scientific collaboration and publication time span.

So far, most studies evaluating PhD research outcomes focused on a single type of higher education award, gathered data from one faculty or institution, usually using a single group of theses or dissertations. One of these studies used an effective data collection method: searches through institutional official repositories were made when analyzing scientific publications; collections of bibliometric and visibility data were created based on information provided by the PubMed, Web of Science, and Journal Citation Reports web-sites when identifying factors or aspects influencing publication of scientific articles [23]. For our study, we applied the methods mentioned above, but also complemented them by gathering information or double-checking information from the PhDs’ full-text theses and from the scientific journals’ official web-sites.

2.1. Data Collection

The theses included in the analysis came solely from Manchester eScholar Services [24] supported by The University of Manchester Library). Post-graduate research students at the University of Manchester must submit since 1 December 2009 her/his thesis as a measure to increase visibility and impact. When the PhD student owns the copyright, the PhD student has the right to restrict access or make visible the thesis and the thesis made visible are freely available to everybody by World Wide Web. The university recommends PhD students make their thesis visible if they do not intend to publish, in part or full, with a commercial publisher or do not intend to commercialize any aspect of the thesis. However, the University has the right to restrict access to the thesis that was initially made visible [22]. This repository was chosen as a representative repository of biomedical theses for several reasons: (1) contained traditional (TT) and publication-based (PBT) theses [23]; (2) it presented clear labels on PhD type or thesis format; (3) it offered a sufficiently homogenous collection of free full-text theses; (4) it provided complete metadata records; (5) theses followed the requirements issued from a single institution, the requirements being very similar for the PhD with traditional format, PhD with alternative format, and PhD by Published Work. In our opinion, to establish a PhD thesis is based on publications just by looking at the thesis’ title (if it explicitly says “studies” or “essays”) or just by examining the thesis for a set of manuscript-looking chapters [25] is not such a precise method. We consider that using the labels offered by the institution awarding the PhD is a much more objective method.

The search strategy used was:

• Content Type = ‘Thesis’ AND
• Academic Department = ‘Faculty of Medical and Human Sciences’ AND
• Publish year = all

The search and data collection started in January 2014 and lasted until January 2016. In the study were included only the free full-text medical PhD theses completed at UK universities defended from January 2003 until December 2015 inclusive (data was available between 2003 and 2015).

For each thesis included in the analysis, several variables were collected (

Table 1. Data collected: variable name, description, and source.

Variable Name (Type)
Thesis data from repository web page
1. author name (for data management)
2. thesis title
3. thesis format (PhD traditional/PhD by publication/alternative)
4. completion date (mm/yyyy), mm= month, yyyy=year (four digits)
5. number of thesis contributors (sum of supervisors, co-supervisors and advisors)
Thesis data from thesis text
6. acknowledged research funding (yes/no)
7. number of all pages until the References section
8. total thesis pages (References and Appendix included)
9. word count (if mentioned)
10. number of studies elaborated and included in the thesis
11. number of articles published or accepted for publication until PhD completion (original full text articles, systematic reviews, meta-analysis)
Article data from other (see superscripts letters)
12. number of authors (No.) a
13. graduate is the first author (yes/no) b (he/she has the highest participation [26,27])
14. journal full name (for data management) a
15. UK journal (yes/no) c
16. if article was indexed in Web of Science (yes/no) d
17. number of citations (from all Web of Science data bases collected on 10 January 2016) d
18. impact factor (IF) for in the year when the article was published e
19. journal rank for the article publication year (as Q1, Q2, Q3, or Q4) f
20. time span between two consecutive articles from the same thesis (months; if computation was possible) g

^a: thesis text/ journal’s web-site; ^b: thesis title page & article; ^c: journal’s web-site/NLM Catalog; ^d: Web of Science; ^e: Journal Citation Reports (JCR); ^f: InCites™ Journal Citation Reports; ^g: journal’s web-site/PubMed Catalog.

).

The word count stated in the theses was used to assess theses’ length differences between the traditional and publication-based PhD routes based. This method of evaluation was used since university regulations usually stipulate the maximum length of a thesis in number of words [28,29]. Since not all theses in our study presented word counts, the numbers of pages with and without the References and Appendix Sections were collected and contrasted. Despite the fact that the number of pages is not a reliable indicator since the requirements for the theses included in the analysis are not identical and the PBT in most of the cases include the ‘version of record’ of published articles, this information was analyzed as an alternative in case of missing the information related with word counts of the theses.

In order to assess knowledge productivity differences from the two PhD routes, the number of research studies (systematic investigation of theories and hypothesis) presented in the theses were contrasted, as well as the number of thesis-related articles. In the medical domain, the number of studies included in a thesis did not always correspond to that of thesis-related articles and this fact is related to the university awarding the PhD (there is no consensus among UK universities guidelines on the quantity and the quality of the publications) [12,14,23,30]. Sometimes, a study can be divided into several articles, or several studies may be combined to create one article [31]. Even more, arranging a manuscript to fit a journal’s requirements can lead to rephrasing of the study’s initial text. There are some that think textual analysis is a good method to detect thesis-related articles [7], but in our opinion it cannot always correctly detect all medical thesis-related articles. Thus, two independent researchers, with the same expertise and experience in medical research methods, browsed each thesis from our study and established, with full consensus, the number of studies elaborated and included in the thesis representing original research pieces the PhD student conducted. They also established, for each thesis, the thesis-related articles published or accepted for publication until PhD completion (presented in a list or in the thesis’ text or appended at the end of the thesis or in the Reference section of the thesis). Those articles with topics not related to the subject of the PhD thesis were excluded from the analysis and there were no article searches beyond the date of thesis completion. Since there is a growing pressure on doctoral students to publish during candidature [2,21], we focused our analysis just on the thesis-related articles published or accepted for publication before the thesis completion date, in an attempt to quantify the graduates’ ability to respond to this pressure.

An efficient method for assessing a medical PhD student’s time to completion and publication of an article is to compute the difference in months between the date when the study concluded and the manuscript’s date when it was first published online [32]. We could not apply this method, as there were no objective means to obtain the dates when the studies from our sample were concluded. From the journals’ web-sites we collected the date when a thesis-related article was first published online, and if this was not available, we collected the given publication date. We considered all theses presenting at least two thesis-related articles and chose, as a reference point in time, the oldest published thesis-related article from each thesis. Time span (TS) indicator or the difference in months between two consecutive articles published or accepted for publication was computed and assigned to the chronologically newer article. Time span (TS) indicator was used to approximate the time to publication of a PhD thesis-related article.

Regarding some of the factors influencing doctoral knowledge publication, we considered not only PhD supervisors [7], but also on the co-supervisors and advisors provided by the Manchester eScholar Services, and grouping them resulted in the number of thesis’ contributors, which we used to evaluate the scientific help that a PhD candidate received during candidature.

Furthermore, the affiliation bias in the peer-review process [33] lead to the question of whether or not there are any differences between the two PhD routes regarding the journals where the articles were published (journals from or outside of the UK).

So far, journals’ impact factors and ranks (or quartile) were used to assess articles’ visibility [34]. Since these two indicators are computed based on the citations of all articles, including articles other than those from this study, their results cannot reflect the visibility and impact of only the articles from our sample. Thus, the two PhD routes were compared based on the number of citations for each thesis-related article.

Seventy-five theses defended between 2003 and 2015 accomplished the inclusion criteria and were split into two groups. The 43 PhDs with Traditional format theses formed the Traditional Theses (TT) group. The publication-based theses (PBT) group was comprised of 3 PhD by Published Work (PW, retrospective publication-based thesis) and 29 PhDs with Alternative format theses (AT, prospective publication-based thesis). We associated PW and AT PhD theses in the same group since PW theses had similar structure and purpose as the AT theses according to the awarding institutions’ regulations [29,30].

The TT group included 115 studies and had 41 thesis-related articles while the PBT format included 142 studies and had 56 thesis-related articles.

2.2. Statistical Analysis

The two PhD theses groups were compared in regard to thesis’ length, studies and article counts, awarded financial support, and number of contributors. Furthermore, the articles included in the analysis were compared regarding: number of authors, if the graduate was first author, if the article was published in a UK journal, number of citation in Web of Science, Impact Factor (JCR), journal rank (InCites™ Journal Citation Reports), and TS indicator. Data on nominal scale was summarized as numbers and percentages. Metric data were summarized as mean ± standard deviation and respectively median and inter-quartile range as (Q1–Q3) (where Q1 is the 25th percentile and Q3 is the 75th percentile) according with the variable distribution. Z-test for proportions was used to compare groups on data from nominal scale. Mann-Whitney-U test was used to compare quantitative data whenever these proved not normally distributed.

All statistical tests were performed with Statistica software 8.0 (StatSoft Inc., OK, USA). Two-tailed p-values of less than 0.05 were considered to indicate statistical significance.

3. Results

3.1. Assessment of Theses’ Length

Not all the 75 medical PhD theses explicitly mentioned the word counts in the theses. More precisely, 19 theses from TT group and 23 from PBT group offered this information and there was no statistical difference between the means for thesis words between the two groups (see

Table 2. Aspects regarding traditional and publication-based theses express as mean (standard deviation).

Item	TT (n = 43)	PBT (n = 32)	P-Value *
No. of pages without References and Appendix	219 (63.54)	196 (62.34)	0.299
Total No. of pages	274 (86.57)	250 (81.22)	0.563
No. of declared words	67,897 (16,704) a	58,969 (15,498) b	0.071
No. of studies	2.67 (1.55)	4.44 (1.62)	<0.001
No. of thesis-related articles	0.95 (1.06)	1.75 (1.48)	0.017
No. of contributors	2.16 (0.92)	2.66 (0.82)	0.023

TT = Traditional theses; PBT = Publication-based theses; ^a: n = 19; ^b: n = 23; * Mann-Whitney test.

). Since all 75 theses were created and defended based on the same regulations we considered the number of pages as an alternative to word counts. In this manner, the number of pages without the References and Appendix sections was evaluated as to focus solely on the body of research of the theses. Next, the total number of pages was evaluated to take into account References and Appendix sections (which offer information that varies from thesis to thesis). Regarding the number of pages, there were no statistical differences between the two groups (see Table 2). We concluded that the theses originating from the TT group are similar in length compared to those from the PBT group.

3.2. Assessment of Knowledge Productivity

Our quantitative analysis considered the studies and thesis-related articles to be essential for approximating thesis’ knowledge production. There were significantly lower values for TT compared with PBT theses (see Table 2) in regard of number of studies included in the theses (p < 0.001). This result is to be expected since theses based on publications focus just on manuscript elaboration and publication. No PhD-related article included in the analysis presented any issues or was withdrawn at the time of inclusion.

On publication quality assessment, the articles from both groups of this study were all published in peer-reviewed journals. Of the 75 theses 61.3% (46/75) presented articles published or accepted for publication until PhD completion, while 20 TT and 9 PBT presented only submitted articles. Figure 1 illustrates article distribution for the two theses formats. In the PBP group, theses exhibiting ≥3 theses-related articles had the highest percentage (34%). Opposite, the highest percentage of TT had no theses-related articles published or accepted for publication until PhD completion (45%). The publication rate indicates how many of the thesis studies are also published. Since there is no certainty in traditional theses that their studies are presented in publishable form, it is a challenge to establish accurately how many of the studies were part of published articles. Thus, the publication rate was computed just for the PBT group and the result was 39.4% (56/142).

Similarly to the number of studies (see Table 2), the TT presented significantly less thesis-related articles compared with PBT theses (p < 0.02). These results could suggest a higher emphasis on study and article creation in PBTs than in traditional theses, in the medical domain.

3.3. Identification of Aspects Which Can Contribute to Doctoral New Knowledge Publication

On the subject of acknowledging research funding in the theses, the 69.8% (30/43) of TT did not prove significantly different than the 62.5% (20/32) of PBT (p = 0.509). In spite of this, the 0.87 mean of theses-related articles from the TT group with research funding proved significantly lower than the 1.8 mean of theses-related articles from the PBT group with research funding (p = 0.019). There was no significant difference when comparing articles (1.24) from all theses with research funding to all those (1.40) from theses without research funding (p = 0.721). In other words, funds were awarded regardless of the chosen PhD route, but the PhD students from publication-based group with research funding published more articles.

On the subject of evaluating scientific aid, the mean of contributors for all the theses included in the analysis was 2.3 (0.912 SD), where the 2.16 mean of contributors to TT proved significantly lower than the 2.66 mean of contributors to PBT (see Table 2, p < 0.03).

Analyzing the distribution of contributors among all theses, 18.7% (14/75) of theses had one contributor, 34.7% (26/75) of theses had two contributors, 38.7% (29/75) of theses had three contributors, and 8% (6/75) of theses had four or more contributors. The theses with two contributors presented a mean of 0.88 theses-related articles significantly lower than the 1.46 mean of theses-related articles from theses with ≥3 contributors (p = 0.021).

In addition to having the highest percentage of no theses-related articles, the TT group also proved to have a significantly higher percentage of just one contributor compared with PBT (p < 0.01, Figure 2). To conclude, the PhDs based on publications attracted more contributors than the traditional PhDs.

Another aspect of scientific aid was the thesis-related articles’ number of authors. Only one thesis-related article was authored just by the PhD student, the rest of the articles being multi-authored. The 5.24 mean of article authors from TT group was not significantly different than the 4.82 mean of article authors from PBT group (p = 0.445). This result could support the idea that medical PhD students collaborated in research groups and published with their thesis’ supervisors, co-supervisors, advisors and other team members, regardless of the PhD route.

Regarding graduates orientating their articles towards publication in journals outside the UK, the 48.8% (20/41) of articles from the TT was not significantly different than the 66.1% (37/56) of articles from PBT (p = 0.085).

The PhD student was the first author in 61% (25/41) of articles from TT significantly less than the 85.7% (48/56) of articles from PBT (p = 0.005). This attests that the PBT encourages candidates to publish as authors in key positions (such as first author) compared with TT.

To sum up, PhDs based on publication were similar to the traditional PhDs based on the proportions of awarded funding and article publication journals from and outside the UK. Nevertheless, PhDs based on publication attracted more supervisors, co-supervisors and advisors, and the PhD student was promoted more often as first article author.

3.4. Assessment of Publication Time

When comparing all eligible theses (prospective and retrospective) based on the time span between two consecutive articles from the same thesis, there were no significant differences between the publication time span of articles from TT group and those from PBT group (

Table 3. Aspects regarding thesis-related articles from traditional and publication-based theses.

	TT		PBT		P-Value *
	Mean (SD)	n	Mean (SD)	n
TS indicator with retrospective theses	12.33 (6.91)	18	14.24 (19.89)	33	0.252
TS indicator without retrospective theses			8.44 (6.37)	27	0.040
Cites number with retrospective theses	19.26 (23.19)	39	18.19 (32.21)	52	0.051
Cites number without retrospective theses			13.47 (18.44)	43	0.017
Impact Factor	3.16 (1.56)	23	2.77(1.52)	45	0.291

TT = Traditional theses; PBT = Publication-based theses; TS = Time span; * Mann-Whitney test.

). However, if the thesis-related articles from retrospective PBT were excluded from the analysis, significant differences were obtained for number of article citations and TS indicator (p < 0.05, Table 3). The PhD students could benefit from a shorter publication time in order to confront the growing pressure to publish during candidature.

3.5. Assessment of Thesis-Related Articles’ Visibility

On evaluating the visibility of research from high quality journals, most of the thesis-related articles were listed in Web of Science (WOS). More exactly, 95.1% (39/41) of articles from TT and 92.8% (52/56) of articles from PBTs were listed in WOS. Focusing on these articles’ citations to evaluate their visibility, the mean of citations for the TT articles was statistically similar to the one for PBT articles, yet was significantly larger when the articles from the retrospective PBTs were excluded (see Table 3).

On evaluating the visibility of journals, just the thesis-related articles published in journals listed in Journal Citation Reports (JCR) were evaluated next (meaning journals listed in WOS which had an Impact Factor). The percentage of articles published in journals listed in WOS which had an Impact factor proved significantly lower in TT group (59%, 23/39) compared with PBT group (86.5%, 45/52) (p = 0.005). When evaluating the Impact Factor values, there was no significant difference between the means of the TT and PBT groups (see Table 3).

As the Impact Factor was debated considerably and scientists established that this indicator differed according to the research domain, we went one step further and evaluated journal visibility based on journal ranking according to the research domain, provided by JCR. Basically, for each thesis-related article published in a JCR listed journal we marked the Journal Rank in the form of quartile (Q1 to Q4). No significant differences were observed between the percentages of thesis-related articles published in highly ranked journals (Q1 or Q2) when TT group was compared with PBT group (Figure 3).

In conclusion, there were no visibility differences between the thesis-related articles from PhDs based on publication and those from traditional PhDs.

4. Discussion

The present observational study was focused on free full-text medical PhD theses available on Manchester eScholar Services, defended between 2003 and 2015. The results indicated that the theses originating from the traditional PhDs were similar in length compared to those from the publication-based PhDs (see Table 2). This could imply that a PhD student could choose any of the two PhD routes and the thesis would have a similar length. Following the analysis to find distinctions between the two PhD routes, we compared the number of studies included in the thesis and thesis-related articles. The results showed as expected a higher emphasis on study and article creation in publication-based PhD theses than in traditional PhD theses in the medical domain (Table 2). A larger amount of publications could suit the PhD student as a proof of good research and writing skills, as well as a basis to build on the future career. In the attempt to verify if these differences were related to other factors besides the PhD route, the analysis was extended to evaluate scientific aid and financial support that the PhD student received. The publication-based PhDs were similar to the traditional PhDs based on the proportions of awarded funding. Even so, publication-based PhDs attracted more thesis contributors (supervisors, co-supervisors, and advisors), encouraging development of collaborative skills. An evaluation of publication practices exhibited by PhD students followed next. The PhD student was promoted more often as first article author in publication-based PhDs’ articles, and both PhD groups the thesis-related articles were published in similar proportions in journals from and outside the UK. One aspect related to publication practices was the approximation of time elapsed between two consecutive articles related to the same thesis. There were no time span differences, suggesting that PhD students published in comparable amounts of months. After excluding the thesis-related articles from retrospective publication-based PhDs, the thesis-related articles from prospective publication-based PhDs revealed a shorter time span than those from the traditional PhDs. Following next was an inquiry on article visibility and perceived quality since the two PhD routes were similar regarding the aspects already presented, except for the amount of articles. Both journals’ Impact Factor and rank proved to be similar for the articles related to the two PhD routes. These results were not satisfactory since both the journal’s impact factor and the rank rely on a series of articles unrelated to this study. Moreover, there are meaningful differences between research domains that could impose on this study’s comparison. As a result, individual article citations were contrasted and the two PhD routes proved once more to have similar articles visibility. However, after excluding from the analysis the thesis-related articles from retrospective publication-based PhDs, thesis-related articles from traditional PhDs exhibited a higher number of citations than the one for thesis-related articles from publication-based PhDs. In conclusion, these results suggest that PhD students choosing the publication-based PhD route could attract more scientific contributors, could elaborate and publish more thesis-related articles (of comparable visibility to the traditional PhD research), and could publish their doctoral results more quickly.

In what follows, we discuss the results for each of the assessments made in relation to other studies in the medical domain.

4.1. Assessment of Theses’ Length

Davies and Rolfe stated that one main difference between the traditional and publication-based PhD theses was the theses length [10]. Contrary to this, our analysis proved that such a difference does not exist (Table 2) in terms of number of pages or number of words declared in the theses. These results could be explained by the requirements, since all the theses were collected from the same repository. The word count analysis was improved by using the page counts. Future research could evaluate thesis length by recounting for each thesis the words, instead of using the word count declared by PhD graduates.

A study on UK medical PhDs completed in 2007–2011 found a 72,433 mean for words [7], a value larger than the ones we found: 58,969 for publication-based theses and 67,897 for traditional theses (see Table 2).

4.2. Assessment of Knowledge Productivity

Our quantitative analysis considered the studies and thesis-related articles to be essential for approximating theses’ knowledge production, as conference papers, oral presentations and posters were considered of low value, plus very few students contributed to books or book chapters [6,35,36]. To prevent the inclusion of false-positive results we did not use textual analysis due of its limitations. Also, narrowing the article search just to those published or accepted for publication until PhD completion offered a different perspective on doctoral publication with regard to publication pressure, a very important aspect which articles beyond PhD completion cannot pinpoint.

With regard to the number of studies included in the theses (see Table 2), there were significantly lower values for traditional theses (TT) compared with publication-based theses (PBT). This result was consistent with the one from another UK study on medical theses defended during 2000 and 2010 from 39 universities, showing that 52.4% of the 82 theses produced publications [37]. The obtained result could suggest a higher emphasis on study creation in PBTs than traditional theses, as it was expected given the purpose of the publication-based thesis. This could also imply that students in the medical domain with a PBT practiced article-writing more than those with a traditional thesis.

The TT presented a significantly less number of thesis-related articles compared with PBT theses. This result is an expected outcome due to the PBT format’s purpose (high emphasis on dissemination of results) and it is in accordance with the difference identified by Davies and Rolfe [10]. However, the gap between the means of the two groups is not large, and this could be the effect of the pressure to publish during candidature for both thesis formats. The fact that PBT have increased numbers of thesis-related articles could prove valuable since successful publishing of results from PhD studies enhance future scholarly activity [11] and increase the success of post-doctoral and grant applications [38].

To this point, in the medical field there were no studies comparing actual PhD output from traditional and publication-based theses. A UK study analyzed the knowledge production of 51 UK medical PhDs completed in 2007–2011 using textual analysis to identify thesis-related articles, and detected that 87.5% of the articles were published before or in the same year of thesis completion [7]. This indicates that most of the article publication was done before PhD completion, making our method very relevant to the subject on PhD knowledge publication. In Scandinavia, publishing during candidature is considered very important [23]. A study on a random sample of 72 biomedical dissertations from Swedish universities confirmed an average of 5.7 papers per dissertation [39].

Publication rates have increased over time and across different fields, including graduate students’ publication rates prior to PhD completion [40]. For example, in medical university from Germany the number of doctoral theses with published articles rose from 33% in 1998 to 52% in 2008 [41]. A paper on Croatian medical Masters and PhD theses suggested that the difference between universities’ publication rates was due to institutional emphasis directed towards publication [42].

4.3. Identification of Aspects Which Can Contribute to Doctoral Knowledge Publication

Next, we present our findings on the aspects which can influence doctoral knowledge publication.

Research funding is an essential part of medical research. In this study, the two PhD routes were awarded research funding in similar proportions, suggesting that the research its self was not a determining factor that the PhD route chosen. The fact that just about 60% of all the PhD students were awarded research funding could be explained by the results of some studies, which identified that medical professionals without a PhD degree are less likely to receive funding than those who already have a PhD degree [43,44]. The impact of research funding was evaluated with regard to scholarly publication and article publication was similar in both groups. This result was inconsistent to that of a study which found that recipients who acquired grants had greater scholarly output than those who did not [43]. When we included the PhD route in the evaluation, the students from publication-based PhD group with research funding published significantly more articles than those from the traditional PhD group with research funding, also an expected result since—in most of the cases—the funder requests publication in scientific literature as an output.

On the subject of scientific aid, the increase of publications at doctoral level improved in the case of collaborations with more experienced scientists than without their collaboration [45]. The results from this study support this idea as the theses with three or more contributors presented significantly more theses-related articles than those theses with two contributors (p < 0.05). The limited sample of PhD theses allowed just the proportional comparison of thesis groups and number of contributors. The traditional theses group proved to have a higher percentage of just one contributor compared with publication-based theses (p < 0.01, Figure 2). In addition to this, the PhDs based on publications attracted more contributors than the traditional PhDs, a result that can explain Robin and Kanowski warning of the potential for supervisors to capitalize on co-authorship [17].The analysis of thesis-related articles revealed no significant differences in regard to number of articles authors (p > 0.05, Table 3). The collaboration of PhD students in research groups offers them possibility to publish with advisors, supervisors and other team members [7]. In Pinheiro et al. study more than 50% of all respondents, who graduated after 2000, had at least one publication co-authored with their supervisor. They argue that supervisor participation as a co-author provides students with informal knowledge about how to structure a publication and communicate with editors and reviewers. They could not conclude if publishing with one's supervisor leads to greater productivity down the road because of learning and socialization effects, or if it is simply the result of a supervisor picking the more productive students to work with from the start [40].

On evaluating the PhD students’ article authorship in this study, the PhD student was the first author in a significantly larger number of articles from PBT compared with TT (p < 0.01). This could suggest that the PBT encourages PhD students to publish as authors in key positions (such as first author) compared with TT. Similar to our PBT percentage, the UK study analyzing the knowledge production of 51 UK medical PhDs completed in 2007–2011 found that the doctoral student was first author for 85% of the articles, while the mean number of article authors was 5 (similar to our results: 5.24 for TT, 4.82 for PBT) [7]. A study on medical PhD students in Denmark enrolled during 2003–2012 showed that the research student was the first author on 43% of the overall 224 papers, and 90% were original articles [46]. Publication as first author can be a decisive factor in a future scientific career as a US study has confirmed that, in the biomedical research, junior investigators with more than six first-author papers within 10 years were more likely to have a successful career than those with six or less first-author papers, regardless of the journal's impact factor [47].

4.4. Assessment of Publication Time

The publishing time (from completion to publication) of a scientific research in biomedical scientific journals was estimated to vary from 6 to12 months in general [48] or between 31 and 533 days [49], but may be longer for some fields. As the prospective publication-based theses group revealed significantly lower time span than the traditional theses group (p < 0.05, Table 3), PhD students could benefit from a shorter publication time in order to confront the growing pressure to publish during candidature.

Due to the fact that time spent between manuscript submission and its final publication may fluctuate among journals, journal publishers, and disciplines [50], some studies have used the median to estimate the publication time of an article. For example, a study on primary medical research found that half of the publications evaluated were published in less than 23 months [32].

4.5. Assessment of Thesis-Related Articles’ Visibility

In our study, all articles were published in peer-reviewed journals and almost all were listed in Web of Science (regardless of their thesis’ format), denoting a good quality of thesis-related articles. No significant differences between groups were observed in terms of number of citations of thesis-related articles indexed in WOS (Table 3). Given that the number of article citations is related to the passing of time, the analysis of citations was conducted also after withdrawing of retrospective PBT thesis-related articles, and in this case the number of citations proved significantly lower in PBT compared with TT group (Table 3).

Starting from the presumption that publications in upper tier journals produced a significantly greater contribution, it was also assumed that a journal’s quality is reflected on its publication’s quality. The Impact Factor has been used to reflect research performance and influence [51,52]. Our two groups had similar IF (Table 3), and were similar in terms of high quality journals (Q1 or Q2, Figure 3).

Publication output has a significant impact on academic selection and promotion, access to future research funds, and opportunities for professional development [53]. In consequence, university policies may change regarding publication rates and visibility. Universities could also benefit from thesis-related articles’ visibility in terms of institutional rankings and research funding. The PhD graduates could benefit from it as successful publishing of results from PhD studies in shorter time can enhance future scholarly activity [11] and increase the success of post-doctoral and grant applications [37]. Adopting publication-based doctoral routes (prospective and retrospective) may be considered a response to the growing pressure on doctoral students to publish during candidature.

4.6. Study Limitations and Further Research

The present study was based just on one repository to collect PhD theses in just one scientific field leading to a small-scale study. This is the main limitation of our study. The obtained results are directly linked with the regulations of the repository, in this case the Manchester eScholar Services. The Manchester eScholar Services used in this analysis clearly classify the type of thesis and has no requirements in terms of number of publications, the sole difference being in just the structure of the thesis. Certainly, the reported results are applicable to the investigated repository and the generalization of these results must be tested on a large-scale study by investigation of more than one repository and more than one institution, since the requirements vary considerably across institutions. The present study is a preliminary study focused on designing a research methodology for future work, which should include other repositories and/or different institutions around the world.

Future studies will collect data from several repositories to render the results more accurate. Furthermore, it is well-known that research production, publication rates, and working practices vary according to scientific field and discipline [1]. Disciplinary differences were not taken into account because of the small number of theses per medical discipline, but could be a future research investigation.

We did not consider the quality of the thesis-related articles in our study and this could be another limitation. The quality of the articles is highly important and despite the increased number of publications [54], a decline related with the reproducibility of experiments could be seen in coverage provided by the Science Citation Index [55].

5. Conclusions

Our analysis of a medical theses sample showed that publication-based theses indeed included more studies and more published thesis-related articles than the traditional ones, as expected. Our results indicated that the publication-based PhD could benefit contributors, as a larger number of good quality articles can increase researcher visibility, provide easier or quicker career advancement, and increase the opportunity and appropriateness for participation to new grant submission calls.

In conclusion, the traditional and publication-based PhD routes had theses of similar length, but the research productivity originating from the publication-based theses was, as expected, significantly larger but not significantly more visible than those from traditional theses.