Making the Transition from Student to Resident: A Method to Individualize a PGY1 Program

A Postgraduate Year One (PGY1) resident’s concerns, limitations, and strengths may be self-identified early in the residency year but are reliant on self-awareness and insight. Program directors commonly find difficulty in identifying a resident’s specific knowledge deficits at the beginning of the program. A standardized resident examination can identify limitations early in training and these results can be incorporated into a tailored resident development plan. A total of sixty-two PGY1 residents completed the examination pre- and post-training over a five-year timespan. Scores increased in most core disciplines in each of the five years, indicating an overall improvement in resident knowledge throughout their PGY1 year. The approach of utilizing the scores for the resident’s individualized plan allows for customization to ensure that the resident addresses knowledge gaps where necessary.


Introduction
The transition from student to resident can be a challenge for both the new resident and the residency program director (RPD). Many residents struggle with balancing clinical obligations and other residency requirements. The majority of concerns, limitations, and strengths may be revealed during a discussion with the resident, however sometimes it is difficult to pinpoint specific knowledge deficits at the beginning of the program. The ASHP Accreditation Standard for Postgraduate Year One (PGY1) programs, states that the residents' development plan should be designed to address each resident's unique learning needs, and include both incoming strengths and weaknesses [1]. To our knowledge, there is currently no literature assessing resident readiness prior to a PGY1 residency or describing how to appropriately individualize the resident's development plan. Currently there is no standard for clinical knowledge assessment upon exit from pharmacy school. In an effort to try to identify specific limitations early in the year, we began administering an examination covering a broad range of topics given during orientation in July of the residency year. This examination was used to help the resident and RPD identify strengths and weaknesses during the orientation month and create an individualized development plan. At the end of the residency year, the residents took the same exam and their results compared (pre-and post-residency experience). The aim of this study it to describe the process of incorporating examination results into development plans for 62 residents given the examination over the last five years.

Methods
A bank of questions was developed by preceptors from each of the core rotation disciplines: critical care (including solid organ transplantation and nutrition), drug information, internal medicine (including ambulatory care and oncology), operations, pediatrics, practice management, and psychiatry (Table 1). Full details on the exam were previously published [2]. Questions were assigned a difficulty rating: graduating pharmacy student (level 1), up to clinical specialist (level 3). Examples of the three levels of difficulty can be found in Appendix A. The majority of questions did not change from year to year; however, if a particular question was routinely missed on pre-and post-examination or if practice guidelines changed, the question was exchanged for a question in the same practice area with a similar degree in difficulty. The examination, consisting of 50 questions, was structured to cover diverse patient populations and disease states with varying complexities. Five different classes of residents were assessed over a five-year time frame, July 2011 to June 2016. This examination was given to each resident at the beginning of their orientation month and during the last month of their PGY1 residency year. The results were provided to the residency advisors and residents to help develop their customized plans for the year. At the end of each year the exam was assessed; questions that involved diseases with new standards of care or medications no longer available or which performed poorly were replaced with questions of the same level of difficulty and discipline. A Student's t-Test was utilized to compare differences in overall score, as well as changes in knowledge in the three levels and seven disciplines in the pre-and post-exams (Microsoft ® Excel ® 2010). A p-value less than 0.05 was considered statistically significant. This project was approved as exempt from IRB approval.

Results
A total of sixty-two PGY1 residents completed the examination pre-and post-training over a five-year timespan: nine residents in year one, fourteen in year two, and thirteen in years three, four, and five. The results of each "pre-residency training" examination were incorporated into the resident's development plan (Appendix B). The approach of utilizing the scores within the customized plan aids in identifying knowledge gaps with the individual, and allowing for customization to ensure that the resident sees growth where necessary. Scores increased in most core disciplines in each of the five years, indicating an overall improvement in resident knowledge throughout their PGY1 year.

Discussion
The use of a standardized resident examination was first used within our institution and results are described in detail elsewhere [2]. We previously described the utility of this examination to quantify changes in resident knowledge throughout the year as well as provide insight into potential targeted areas for improvement in the residency program. Likewise, the results of this examination are useful for detecting potential knowledge deficits of the resident and developing a customized PGY1 learning plan. Identified early in the year, this may help reduce unnecessary stress for both the resident and program director. It has been reported that 6%-15% of pharmacy students have some form of academic deficiency during their training [3]. Currently there is not published data on the percentage of pharmacy residents reporting difficulty or needing intervention during their PGY1 year. However, our medical colleagues have noted that approximately 8%-15% of residents struggle during their residency [3]. Additionally, unlike pharmacy school, most pharmacy residency programs do not have structured assistance programs designed to help the resident [4].
Given that our exam was addressing core disciplines, we were able to hone in on particular areas that may need further development for a resident. For example, a resident who did well in internal medicine but struggled with critical care may benefit from having a critical care experience earlier in the year or an additional rotation in that area to help increase their knowledge base. Alternatively, utilizing the areas of strength may help a resident with the selection of a PGY2 specialty residency. This too would be beneficial early in the year to allow for more experience with that specialty prior to the annual recruitment process for PGY2 programs.
There are several limitations to our study. First, a larger proportion of questions focused on the medicine and critical care core disciplines which parallel the resident experiences in our PGY1 residency program. Accordingly, exam performance in other core disciplines may not accurately reflect each resident's knowledge in that area. We did not account for testing bias, including the possibility of score inflation from repeated exposure to exam questions. Studies of higher-stakes examinations in medical training, however, have shown that prior exposure does not significantly improve performance [5,6]. Finally, we did not assess if this type of exam correlated with self-or preceptor-assessed performance.
The medical profession utilizes standardized examinations throughout school and during residency training to ensure that certain minimum knowledge requirements are met. Although licensure exams are required in the United States, pharmacy has lagged behind in ensuring that there is a minimum level of competency obtained from clinical training by allowing board exams to be optional. A significant amount of literature exists surrounding the use of examinations to facilitate medical resident selection however they do not predict performance in the residency [7]. Rather exam performance did predict first time pass rates of board exams following residency training [7]. As such we suggest that further investigation of and implementation of an exam requirement at the end or the beginning of pharmacy residency training will facilitate standardization of minimum knowledge requirements and ensure that the resident has met a standard not currently available. Additional studies are necessary to determine the benefits of standardized exams in pharmacy residency training.
Although we expected to see improvement in exam scores once residents completed their PGY1 year, we feel that giving a comprehensive written examination at the beginning of the residency year helps both the resident and program director construct an individualized, comprehensive development plan which helps with program planning. Being able to identify specific strengths and weaknesses at the beginning of the year can help make the year more successful for the resident by tailoring it to better fit their specific needs. Future research is needed to determine the role of post-graduate residency training and competency testing. This leads to a final question: with advanced pharmacy degrees and a high level of clinical practice should the pharmacy community embrace the medical competency model with testing following each step of training to ensure that the product produced meets minimum standards? Author Contributions: Each author helped write the manuscript and contributed questions to the examination. In addition, Jason Haney and Brian McKinzie analyzed the data.

Conflicts of Interest:
The authors declare no conflict of interest.

Appendix A Example Questions
Appendix A1. Medicine, Level 1

1.
A 55-year-old Caucasian female with a past medical history significant for hypertension, type 2 diabetes, stage 3 chronic kidney disease, and gastroesophageal reflux disease (GERD) presents to your clinic today complaining of unilateral left lower extremity swelling and pain.