Development of doctoral studies began in 19th century Germany and were based on the so-called Humboldt model [1
]. This model has so far influenced almost every form of doctoral education in the world. The model is based on an apprenticeship, meaning that doctoral candidate works for serval years under the supervision of a mentor on a specific research problem. After gathering a coherent body of knowledge on a problem, a candidate would publish a doctoral thesis, which would be usually evaluated by a committee of experts. During the 20th century, diversification of the Humboldt model by its adaptation to many specific national frameworks resulted in a variety of somewhat similar doctoral degree granting models [1
Likewise, doctoral studies granting the Doctor of Philosophy (PhD) degree in the biomedical field are numerous and diverse through the world and EU. Criteria for completion of such studies commonly include research work, which is usually supervised by a PhD commission or council, and oftentimes they require a publication of an original scientific article [2
]. However, within this general framework a great deal of heterogeneity exists. The Organisation for PhD Education in Biomedicine and Health Sciences in the European System (ORPHEUS) was founded in 2004 with the mission of synchronising criteria and ensuring the integrity and quality of PhD degrees in the biomedical field [3
]. One year after, as a part of third cycle of the Bologna process, the Salzburg principles were adopted as a foundation for EU-wide doctoral degree synchronisation [4
]. Since 2012, ORPHEUS publishes, in a spirit of Salzburg principles, a Best Practices e-textbook which addresses key issues in biomedical PhD education [5
]. Among these, the issue of graduate school structure encompasses a need for a regular review and the updating of a graduate school structure.
In this vein, the international European Research Council (ERC) delegation visited the University of Split, School of Medicine and examined teaching conditions, study management, students and mentors in our Biology of Neoplasms (BN) PhD Programme. The delegation raised some serious concerns regarding our BN programme. Consequently, in an attempt to reform the structure of the PhD programme, we tried to identify factors that determine academic success and failure in biomedical PhD programmes by searching the MEDLINE database, the largest scientific literature database in the field of biomedicine. However, as others have also noted [6
], quantitative information which would help guide our decision making regarding the PhD programme reform turned out to be rare and scarce, especially when it comes to European PhD programmes. Based on this, we decided to do a quantitative analysis of our PhD programme and identify factors that would predict academic success or failure, so that in the future structural reform of our PhD programme positive factors can be amplified.
2. Materials and Methods
The BN PhD Programme is a typical PhD programme developed to comply with the rules of the Bologna process. It has six semesters—semesters one to four are dedicated to teaching and research, while the last two semesters are exclusively dedicated to research. Since 2014, PhD programme have introduced stricter graduation criteria with the first authorship of two (previously one) articles on the thesis topic, published in a journal indexed in Web of Science with an impact factor (IF) >1 or one article with an IF ≥4. In our settings, only PhD students in full-time employment in academia have an obligation to graduate within 5 years of their enrolment in a PhD programme. Others do not have a specific time limit regarding graduation. Students employed in academia dedicate at least 50% of their working time to scientific duties, and the other part of their working hours might be dedicated to teaching and administrative tasks. Students not employed in academia must deal with scientific obligations outside of their standard working hours (40 h a week in Croatia). Furthermore, students that are employed in academia have their tuition fully subsidised by an academic institution, whereas others might or might not have their tuition subsidised depending on their employer. Admission to BN PhD programmes were liberal—i.e., a candidate having a Doctor of Medicine (MD), Doctor of Dental Medicine (MDM) or master’s degree in some field of biomedicine and with a grade point average of at least 3.5 in the Croatian grading system (5 being the maximum and 1 being the lowest grade) could enrol. Candidates were not required to have PhD mentors at the time of admission process. In this analysis, we included all of the students enrolled in the programme since its beginning in 2006. Students who enrolled after 2017 were excluded because they were unable to defend their PhD thesis before April 2019 when our study ended.
2.2. Outcome Measures
We tried to quantity academic success or failure trough fallowing outcome measures: the proportion of students who graduated, time to degree (from enrolment to defence of the PhD thesis), the number of published manuscripts on thesis topic, with the PhD student as first author, the IF of the journal where the PhD student was published as a first author, the ratio of the IF and time to graduation. Additionally, attrition rates and the reasons for the attrition were analysed.
2.3. Statistical Analysis
Data are presented as median and interquartile ranges for non-normally distributed data, mean and standard deviation for normally distributed data or as a proportion. Normality of data was tested by a Shapiro–Wilk test and normality probability plot correlation coefficient. To formalise differences between groups, we used the chi-squared, chi-squared for trend or Fisher’s exact test for nominal variables and t-test or ANOVA for continuous variables. Furthermore, multiple logistic regressions and model selections based on AIC (Akaiake Information Criterion) were used to find the optimal model for predicting outcome measures. For modelling purposes, i.e., for the purpose of identifying predicting factors, only analyses of 2007–2013 group were carried out. Inference was aided by point and interval (95% CI) estimates, goodness of fit measures, AIC and p values; the latter was interpreted in accordance with the ASA (American Statistical Association) Statement on p value.
A comparison of the characteristics of the students in the two surveyed periods is shown in Table 1
. In total, there was a higher proportion of females—77:47 (62.1%). Students who enrolled in 2006–2013 were older than students who enrolled in 2014–2017 (33.32 ± 8.13 versus 28.61 ± 4.8 years). Students were mostly doctors of medicine or dental medicine (92%). These were mostly part-time students, fully staffed at the hospital or private practice (69%). Only 13% were “full time” students in various scientific projects at the School of Medicine or various research institutes.
3.1. Factors That Influence Graduation Rate
The cumulative graduation rate of students enrolled in the Biology of Neoplasms PhD programme was 17.7% (22/124) (Table 1
). Most of the thesis topics were clinical. In order to discover predictors of graduation, we analysed the 2006–2013 dataset because it was richer in outcomes. Students who self-paid their tuition comprised 50% of the 2006–2013 group; their graduation percentage was 12.5%. Among students who were partially subsidised, graduation percentage was 20%, while among students who were fully subsidised by an institution, 73.3% graduated. These results show a clear positive trend between financial support and the proportion of graduating students (χ2
= 16.79, p
< 0.0001) (Figure 1
Furthermore, students employed by academic institutions had a graduation rate of 69%, followed by students in industry and private practice with 28.5% graduating. Only 13% of PhD candidates working in hospitals had graduated by the time data were collected (χ2
= 9.64, p
= 0.0081), (Figure 1
Of the 21 PhDs defended, 14 (67%) were mentored by experienced mentors (more than three PhDs theses defended prior to this mentorship), 6 (28%) were mentored by moderately experienced mentors (1 to 3 PhDs previously in mentorship), and 1 (5%) was led by an inexperienced mentor (so far he/she did not mentor doctoral dissertations). Thus, a trend can be found between the mentor ratio and the graduation rate (χ2
= 9.71, p
= 0.0018, Figure 1
Since in our setting all of the PhD students employed in an academic institution have their tuition fees fully subsidised, we decided to use model selection in order to find an optimal model which would predict the success of graduation. We considered age, sex, employment institution, mentor experience and tuition subsidy as potential predictors. After the model selection process, the most parsimonious model had only two variables—i.e., academia as institution (Odds ratio (OR) 11.66, 95% CI—1.78 to 126.9) and experienced mentor (OR 6.96, 95% CI—3.15 to 22.15). Prediction accuracy was excellent (Table 2
and Table 3
) and there were no issues with predictors correlating with each other.
3.2. Factors That Influence Time to Graduation
In total, the average time to doctoral degree was 79.1 months. Time to graduation did not depend on the type of institution in which the student was employed (ANOVA, R2 = 1.5%, p = 0.8654), nor on the level of tuition subsidy (ANOVA, R2 = 1.9%, p = 0.8346). Furthermore, the time to graduation of students with highly experienced mentors (85.54 ± 32.65 months) was not different compared to those with moderately experienced mentors (83.17 ± 30.35 months), (t-test, p = 0.8824).
3.3. Factors That Influence Academic Productivity
The average impact factor (IF) of journals in which the studies were published was 2.66 ± 1.51 and it was the highest for the first publication. The average number of publications included in the thesis was two, one student had three papers and another one had one paper (with IF > 4); more details are presented in Table 4
Students working in academic institutions had an average IF of 3.61 ± 1.79, those working in hospitals had an average IF of 1.85 ± 0.37 and finally students working in private practice or industry published in journals with an average IF of 1.74 ± 0.42. Comparing the average IF among of students working in academia with those who work in hospitals or in private practice and industry, the differences are characterised by fallowing p values (posthoc Welch t-test, p = 0.0193 and p = 0.0143, respectively).
Stratification of students by tuition subsidy does not seem to have any impact on the average IF (ANOVA, R2 = 24.2%, p = 0.0822). Students with highly experienced mentors seem to have a higher average IF then those with moderately experienced mentors. (Welch t-test, mean difference = 1.227 ± 0.621, p = 0.069).
During the study period, 21 (16.94%) students dropped out for a variety of reasons, the most common being personal factors (28.57%) (Table 5
). Upon more detained analysis, not having a mentor turned out to be the most important factor in predicting student dropout probability, because none of the students who had a mentor dropped out (Fisher’s exact test p
< 0.0001) (Figure 1
3.5. Effect of Changing Criteria for Graduation
In order to analyse the effect of stricter graduation criteria, we quantified academic success as a ratio of a candidate’s IF average and time to graduation. One candidate graduated from the group that enrolled after 2014, and her average IF to time to graduation ratio was at the 95th percentile of distribution derived from pre-2014 graduation data. Furthermore, the proportion of students who did not find a mentor decreased from 48% to 35% (χ2 =1.836, p = 0.1755). The proportion of students with unexperienced mentors remained the same with 11%, the proportion of students with moderately experienced mentor increased from 16.5% to 35.5%, and finally the proportion of students with highly experienced mentors decreased from 48.1% to 35.5%; the p value for these changes was 0.1102. Fraction of students employed by academia dropped from 16.4% to 7% (χ2 = 2.641, p = 0.4503).
Our data imply that only one-fifth of the total enrolled students of Biology of Neoplasms have graduated. This low completion rate is consistent with similar biomedical PhD programs (Faculty of Medicine in Belgrade 16.6%, Faculty of Medicine in Izmir 26%, Faculty of Medicine, Public Health and Nursing University Gadjah Mada 27.94%; personal communication) as well as with the worldwide data [7
]. On the other hand, an average completion rate of at least 66% has been reported in a recent European survey [8
]. In the most successful countries such as the Netherlands, which has a graduation rate of 75%, candidates pursue PhDs in research grant funding [9
]. They work as full-time students, less as candidates who pay their tuition independently. Dutch universities receive monetary rewards for each successfully completed PhD from their government. On the contrary, if the research fellowship is completed without PhD, the candidate is left without a salary and the only option is that institution pay for him/her to finish the programme [9
A possible reason for the low graduation rate in our PhD programme is that most candidates are part-time students—i.e., they do not work in academia. In fact, more than 75% of them are medical practitioners at the beginning of their professional career. Second, the motivation of PhD candidates to enrol in the PhD programme is sometimes based on obtaining additional points for the residency admission in the hospitals, so it turns out that they are not committed to PhD study. These students were not intrinsically motivated to find a research topic and mentor. The similar objection was noted by a PhD programme in another Croatian University [10
Furthermore, employment in academia can be considered as a proxy of how much working time is dedicated to scientific duties and it is not surprising that when combined with the mentor’s experience it dramatically increases odds (approximately 18-fold when expressed as natural log. of odds) for successful completion of the graduate programme. It is somewhat surprising that none of the considered predictors correlated with time to graduation.
When it comes to academic productivity expressed as average IF of papers that were published as a part of PhD thesis, with the student as a first author, PhD candidates with highly experienced mentors and those employed in academia outperform the others.
Attrition or dropout rates in our programme was 16.94%, which, interestingly, approximates the graduation rate. The main predictor of dropping out was an inability to find a PhD mentor.
If the results of our retrospective analysis are compared to ORPHEUS best practice recommendations, it is obvious that the strongest predictors such as employment in academia (i.e., time dedicated to PhD research), experienced mentors and inability to find a mentor correlate well with the content of the former dealing with the research environment and supervisors. Thus, it can be said that our study lends quantitative support to the recommendations.
Since this case study represents a retrospective study without validation cohort, its conclusions should be taken with some degree of reservation.
In conclusion, we were able to show that employment in academic institutions and experienced mentors are strong predictive factors of successful graduation in PhD programme; on the other hand, not finding a mentor was a strong predictive factor for dropping out of the graduate programme. Our results can be easily extended to other biomedical PhD programmes in Europe since a large majority of them are regulated by and adopted rules of the Bologna process; furthermore, our results represent a data-based confirmation of ORPHEUS best practices recommendations. Finally, one of the remaining problems that our study points to is the PhD education of practicing physicians, who by definition cannot dedicate themselves to a great extent to PhD research due to their workload. The solution to this problem may be in the establishment of MD/PhD combined programmes modelled after physician-scientist training programmes, which represent time-tested standards for combined MD and PhD education in the United States.