Next Article in Journal
Recent Advances in Sickle-Cell Disease Therapies: A Review of Voxelotor, Crizanlizumab, and L-glutamine
Next Article in Special Issue
Impact of Different Attitudes toward Face-to-Face and Online Classes on Learning Outcomes in Japan
Previous Article in Journal
A Snapshot of Potentially Inappropriate Prescriptions upon Pediatric Discharge in Oman
Previous Article in Special Issue
Exploring the Effect of Virtual Education on Student Pharmacists’ Confidence toward APPE Readiness
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Brief Report

Important Factors in Remote Experiential Education

1
Massachusetts College of Pharmacy and Health Sciences, School of Pharmacy, Boston, MA 02115, USA
2
Massachusetts College of Pharmacy and Health Sciences, School of Pharmacy, Manchester, NH 03101, USA
*
Author to whom correspondence should be addressed.
Pharmacy 2022, 10(5), 122; https://doi.org/10.3390/pharmacy10050122
Submission received: 26 August 2022 / Revised: 14 September 2022 / Accepted: 16 September 2022 / Published: 24 September 2022
(This article belongs to the Collection New Insights into Pharmacy Teaching and Learning during COVID-19)

Abstract

:
Onsite and in-person experiential education has been well established to prepare practice-ready healthcare professionals, such as pharmacists. From COVID-19, the integration of remote educational delivery has occurred. As healthcare disciplines adjust to new experiential styles and innovate traditional methods, this paper highlights key areas for remote experiential education that can influence student experiences. Factors that are of importance to continuous quality improvement are described. A survey, utilizing the cloud-based software platform Qualtrics® headquartered in the United States, was developed to evaluate whether remote rotation delivery was comparable to traditional onsite experiential education, to assist with quality improvement for virtual experiential education, and to ensure the redesigned educational model meets accreditation standards for two schools of pharmacy. Numerous factors including work, time zone, Office of Experiential Education and preceptor responsiveness, and technology, were examined. Chi-Square test, t-test for proportions and odds ratios were utilized to evaluate results. Students with technology concerns throughout a remote rotation had a more than two-fold increase in identifying the virtual experience as worse than most/all other in-person rotations (p = 0.01). Preceptor responsiveness to questions and concerns significantly impact student perceptions of educational quality (p < 0.05). The majority of students perceived remote experiential education is equal to onsite experiences. Since continuous quality improvement is required by pharmacy accreditors and many other healthcare programs offering clinical opportunities, identifying factors is of importance to make future interventions in the remote experiential education delivery. This type of experiential learning became essential with COVID-19 impacting onsite clinical placements, and information can be used across health science disciplines at large.

1. Background

The majority of advanced clinical experiences for healthcare students has been in-person, onsite contact time. Due to the pandemic stemmed from coronavirus (COVID-19), a modification to incorporate high levels of remote/virtual experiential education is warranted. Schools must rise to these unprecedented challenges.
Advanced Pharmacy Practice Experience (APPE) onsite and in-person education has been well established to prepare practice-ready pharmacists within a Doctor of Pharmacy (PharmD) program [1]. During this phase of the program, students build competency in the application of the Pharmacists’ Patient Care Process (PPCP) [1,2]. The Accreditation Council for Pharmacy Education (ACPE) in the United States requires students have 1440 advanced experiential hours providing direct patient care with various patient populations and interprofessional education [3,4,5,6].
All pharmacy students at Massachusetts College of Pharmacy and Health Sciences across three campuses are required to complete six advanced rotations in the last year of their program, including inpatient general medicine, ambulatory care, community pharmacy, and hospital/health system pharmacy rotations. Each rotation is 6 weeks in length and 40 hours per week. A combination of full-time pharmacist faculty and adjunct faculty are used to conduct experiential rotations. In March 2020, the majority of hospital, health-system, and ambulatory experiential sites throughout New England were no longer able to accommodate students in person, due to becoming a hot spot at this time for COVID-19 and drastic increases in hospitalizations. As 489 students in the PharmD Class of 2020 across three campuses at two separately accredited pharmacy programs needed at least one last advanced rotation to graduate, massive efforts by the centralized Office of Experiential Education (OEE), in partnership with full-time faculty, worked to devise and accommodate remote experiential education. With the remote delivery, only full-time faculty or adjunct faculty with high awareness of ACPE accreditation standards were utilized as a preceptor in APPE core rotations. Students were required to complete the same core rotation type that they were initially assigned to for this last APPE block, unless a prior advanced elective rotation could account for the remaining APPE core rotation. For example, if a student completed an inpatient specialty elective that could account for a core inpatient general medicine rotation, the prior elective was adjusted to document the core rotation completion, and the last APPE block became a remote elective. Students had their preceptor changed to full-time faculty or adjunct faculty with high levels of ACPE awareness for this last core APPE block. Electives were reassigned when needed if a preceptor could not pivot to a remote format. Adjusted electives were based on what could be offered and adapted remotely such as digital health, informatics and drug information opportunities.
By previously completing five 6-week APPE rotations each onsite throughout the 2019–2020 academic year, these students had significant onsite experiential opportunities in which to compare to a remote experience. A survey, utilizing the cloud-based software platform Qualtrics® headquartered in the United States, was developed to evaluate whether the new rotation delivery was comparable to traditional onsite experiential education, to assist with quality improvement for virtual delivery in the next academic year, and to ensure experiential education meets accreditation standards. Accreditation requires pharmacy programs to administer Curriculum Quality Surveys to evaluate whether experiential rotations are of high quality; whether preceptors provided individualized instruction, guidance and evaluation; and whether information technology resources and educational resources provided were conducive to learning [7].
OEE hypothesized that the majority of students would be of the opinion that the remote experience would be equal to most/all other rotations they previously had in-person. As healthcare disciplines nationwide adjust to new experiential delivery styles [8,9,10,11,12] and integrate flexibility allotted by accreditors, [1] the aim of this paper is to describe the survey results and to highlight key areas for remote delivery of experiential education that can influence student experiences across all health care programs.

2. Approach

The majority of the 39-question survey was linked to accreditation standards and continuous quality improvement (CQI). Upon completion of the remote rotation, students answered demographic and basic information questions including rotation type, living arrangements, and paid work experience while attending the remote rotation. Questions from a section related to the virtual rotation experience to guide CQI, student support, and student satisfaction, all of which are areas highlighted within ACPE accreditation standards or guidance documents, were analyzed [3,13]. Eleven questions were asked in which students rated their level of agreement or disagreement on a five-point Likert scale. Three of the eleven questions were linked to accreditation standards, six questions were linked to factors that could impact the virtual delivery format and were considered CQI-related questions, and two questions asked students to evaluate responsiveness of their preceptor and OEE. The number of preceptors the students interacted with was also evaluated, as rotations in the remote format were adjusted to have team-based precepting for about half the core rotations. In this adjusted model, there was still the expectation that students should receive individualized instruction.
A student perception question was asked regarding the virtual rotation quality compared to previous onsite rotations. This question was cross tabulated with other survey questions using Chi-Square test, t-test for proportions and odds ratios. Students could provide free text information. This methodology allowed for analysis of factors that impact student perceptions and required accreditation components.

3. Evaluation

Table 1 shows the number of students, rotation type, and survey response rates for the remote APPE across the three campuses. The majority of students attending this remote rotation (76.7%, n = 375) responded to the survey, and 355 students completed the survey in its entirety. The percent of students who responded to the survey (n = 375), when compared to the percent of students in each rotation type during the remote rotation (n = 489), is within 0.2%–1.8% of each other for all rotation types.
Of the 355 fully responding students, 48.5% completed paid work experience while also completing the remote rotation. The majority worked similar hours to pre-pandemic. Approximately half (49.3%) of the remote rotations were conducted in a group format with 2+ preceptors collaborating together.
Highlights from the eleven Likert-scale questions in Table 2 include 39.1% of respondents (n = 139) somewhat to strongly agreed they had technology concerns throughout the rotation and 15% (n = 53) somewhat to strongly agreed that time zone differences were a concern for this rotation. The majority of respondents somewhat to strongly agreed that their experiential and academic expectations were met for this rotation (91.5%) and they received individualized attention from their preceptor during the rotation that met their expectations (92.4%). For this block, the majority of respondents somewhat to strongly agreed their preceptor (94%, n = 303), and OEE (66%, n = 156) was responsive to their questions and concerns.
Students evaluated the remote rotation quality compared to their on-site rotations. This question resulted in 16.6% (n = 59) of students being of the opinion that the virtual rotation was better than most/all other rotations, 67.6% (n = 240) reported the virtual rotation as equal to most/all other rotations, and 15.8% (n = 56) viewed this rotation as worse than most/all other rotations.
Summarizing Table 3, students working and the amount of hours worked, time zone differences, various group formats that preceptors utilized, and responsiveness from OEE, did not have a statistically significant impact on the student’s perception for this rotation being better than, equal to, or worse than most/all other prior rotations (p > 0.05 for all). The preceptor’s response to questions and concerns was a significant finding that impacted student perceptions of the quality when using Chi-Square testing (p < 0.05). Of the students who did not strongly agree that their preceptor was responsive to questions and concerns, 36.7% had the opinion that the remote rotation was of worse than most/all other rotations (n = 49). In contrast, if a student strongly agreed their preceptor was responsive, 12.4% had the opinion that the remote rotation was of worse than most/all other rotations (n = 275).
Students who had technology concerns throughout this rotation had a more than two-fold increase in identifying the virtual experience as worse than most/all other rotations (odds ratio 2.24, 95% CI 1.20, 4.16, p = 0.01) compared to students with no technology concerns; 23% of students with technology concerns evaluated the virtual session as worse than most/all other rotations compared to 11.8% of students with no technology concern.
Based on these results, the majority of students perceived remote experiential education is equal to or better than their onsite experiences.

4. Implications

In March 2020, there were no vaccines for COVID-19, there were shortages on personal protective equipment (PPE), and there were many unknowns. There are now vaccinations and PPE readily available, and yet there is still a need to be prepared to offer remote clinical education with ongoing COVID-19 surges, future natural disasters, or other.
Key findings that impact remote experiential education include the following: (1) technology issues play a principle role in a student’s satisfaction level, and needs to be addressed effectively, and (2) preceptor responsiveness to students’ questions and concerns impact student perceptions. Interventions made with this information include focusing on technology referral support during remote experiential delivery and providing preceptor development with an emphasis on responding to student questions and concerns. For technology, students had to adapt with the means they had during this APPE block, and no school-issued laptop or communication device was distributed at large. Within the 6 weeks of this rotation, the school pivoted to ensure laptop or communication devices could be available for students in need, and also looked into internet accessibility options such as a hotspot device to provide to students who were without one. These discoveries are applicable to any clinical placement that integrates remote experiential education.
Historically with onsite rotations, the program used a model where one specific preceptor is the point person for each student. In the remote setting as OEE and faculty worked together to shift rotations, many faculty elected to collaborate together in teams of 2, 3, or 4+ preceptors for their assigned students. This teamwork expanded student opportunities to collaborate beyond one person, expanded direct patient care opportunities, and allowed preceptors to diversify topics with the unanticipated remote offering. Knowing team-based precepting had benefits and did not negatively impact student experiences, it is a model to consider for future use.
For students that somewhat to strongly disagreed OEE was responsive to questions and concerns, students did not provide any comments on how to improve the experience. Anecdotally through emails received from students, there was dissatisfaction from some students on not receiving a reassignment to a top rotation pick, on being assigned to a preceptor not of their choosing, or on being in a remote experiential placement altogether. Given the environment and loss of onsite clinical placements, some of the dissatisfaction could not be avoided. Communication with students during these times is crucial.
By evaluating factors potentially impacting rotations (work, time zone differences, preceptor responsiveness, etc.) and student perceptions of the remote rotation being better than, equal to, or worse than onsite experiential education, this allowed the program to pinpoint factors that can significantly influence student perceptions of the remote quality. Since CQI and evaluating student satisfaction is required by accreditors such as ACPE, identifying factors is important to make future interventions in the remote experiential delivery. This project included a large sample size for identification and evaluation, allowing for development and improvement of future remote rotations if needed. Limitations include awareness that by not completing a virtual rotation, students would be ineligible to graduate on time due to experiential rotation displacements brought forth by the pandemic. To address this limitation, the cross-tabulation method was utilized to evaluate particular experiential factors that most significantly impacted student perceptions. Further, although the authors can confirm the majority of students got the same APPE type in the remote format and the majority of students got a preceptor they had ranked as a top 10 preceptor, the exact breakdown for the number of changes made is not readily available. A limitation of this is not having the information to cross-tabulate as a factor for satisfaction level. While the study was limited to pharmacy students only, what was learned can be translated to all types of academic clinical rotations.

Author Contributions

Conceptualization, J.L.P.; methodology, J.L.P.; validation, T.E.; formal analysis, T.E.; investigation, J.L.P.; resources, J.L.P., T.E. and J.D.G.; data curation, T.E. and J.L.P.; writing—original draft preparation, J.L.P.; writing—review and editing, J.D.G., T.E. and N.C.; project administration, J.L.P. and N.C. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

This research was approved by the MCPHS institutional review board with exempt status. Protocol Number: IRB061220P.

Informed Consent Statement

Written informed consent was waived by the institutional review board.

Data Availability Statement

The data presented in this study are available on request from the corresponding author. The data are not publicly available due to academic privacy.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Accreditation Council for Pharmacy Education. PharmD Program Accreditation—Accreditation Council for Pharmacy Education. Available online: https://www.acpe-accredit.org/pharmd-program-accreditation/ (accessed on 25 August 2022).
  2. Joint Commission of Pharmacy Practitioners. Pharmacists’ Patient Care Process. Available online: https://jcpp.net/wp-content/uploads/2016/03/PatientCareProcess-with-supporting-organizations.pdf (accessed on 25 August 2022).
  3. Accreditation Council for Pharmacy Education. Accreditation Standards and Key Elements for the Professional Program in Pharmacy Leading to the Doctor of Pharmacy Degree (Standards 2016). Available online: https://www.acpe-accredit.org/pdf/Standards2016FINA.L.pdf (accessed on 25 August 2022).
  4. Gilliam, E.; Nuffer, W.; Thompson, M.; Griend, J.V. Design and activity evaluation of an Advanced-Introductory Pharmacy Practice Experience (aIPPE) course for assessment of student APPE-readiness. Curr. Pharm. Teach. Learn. 2017, 9, 595–604. [Google Scholar] [CrossRef] [PubMed]
  5. Rathbun, R.C.; Hester, E.K.; Arnold, L.; Chung, A.; Dunn, S.P.; Harinstein, L.M.; Leber, M.; Murphy, J.A.; Schonder, K.S.; Wilhelm, S.; et al. Importance of direct patient care in advanced pharmacy practice experiences. Pharmacotherapy 2012, 32, e88–e97. [Google Scholar] [CrossRef] [PubMed]
  6. Gubbins, P.O.; Micek, S.T.; Badowski, M.; Cheng, J.; Gallagher, J.; Johnson, S.G.; Karnes, J.H.; Lyons, K.; Moore, K.G.; Strnad, K.; et al. Innovation in clinical pharmacy practice and opportunities for academic--practice partnership. Pharmacotherapy 2014, 34, e45–e54. [Google Scholar] [PubMed]
  7. American Association of Colleges of Pharmacy. Curriculum Quality Surveys Information. Available online: https://www.aacp.org/research/curriculum-quality-surveys-information (accessed on 25 August 2022).
  8. Law, E.; Scott, M.C.; Moon, Y.S.K.; Lee, A.J.; Bandy, V.T.; Haydon, A.; Shek, A.; Kang-Birken, S.L. Adapting pharmacy experiential education during COVID-19: Innovating remote preceptor resources, tools, and patient care delivery beyond virtual meetings. Am. J. Health Syst. Pharm. 2021, 78, 1732–1738. [Google Scholar] [CrossRef]
  9. Liao, F.; Murphy, D.; Wu, J.C.; Chen, C.Y.; Chang, C.C.; Tsai, P.F. How technology-enhanced experiential e-learning can facilitate the development of person-centred communication skills online for health-care students: A qualitative study. BMC Med. Educ. 2022, 22, 60. [Google Scholar] [CrossRef]
  10. Mohammad, I.; Berlie, H.D.; Lipari, M.; Martirosov, A.L.; Duong, A.A.; Faraj, M.; Bacon, O.; Garwood, C.L. Ambulatory Care Practice in the COVID-19 Era: Redesigning Clinical Services and Experiential Learning. J. Am. Coll. Clin. Pharm. 2020, 3, 1129–1137. [Google Scholar] [CrossRef] [PubMed]
  11. Weber, A.M.; Dua, A.; Chang, K.; Jupalli, H.; Rizwan, F.; Chouthai, A.; Chen, C. An outpatient telehealth elective for displaced clinical learners during the COVID-19 pandemic. BMC Med. Educ. 2021, 21, 174. [Google Scholar] [CrossRef] [PubMed]
  12. Yoon, C.; Lee, J.; Fong, E.; Lee, J.I. The Virtual Team Member: Remote Engagement of Medical Students in COVID-19 Care. Med. Sci. Educ. 2021, 1–8. [Google Scholar] [CrossRef]
  13. Accreditation Council for Pharmacy Education. Guidance for the Accreditation Standards and Key Elements for the Professional Program in Pharmacy Leading to the Doctor of Pharmacy Degree (Standards 2016). Available online: https://www.acpe-accredit.org/pdf/GuidanceforStandards2016FINA.L.pdf (accessed on 25 August 2022).
Table 1. Students completing each rotation type remotely.
Table 1. Students completing each rotation type remotely.
Students in Each
Rotation
Students Who Responded to the SurveyResponse Rate
Virtual rotation type completed Count (n)Percent in block 8Count (n)Percent in block 8Percent in block 8
APPE Ambulatory Patient Care8216.8%6417.1%78%
APPE Community Pharmacy6312.9%5013.3%79.4%
APPE Elective19139.1%14037.3%73.3%
APPE Institutional (Hospital/Health System Pharmacy)7114.5%5514.7%77.5%
APPE Internal Medicine (Inpatient General Medicine)8216.8%6617.6%80.5%
Total489100.0%375100%76.7%
Table 2. Survey questions related to accreditation standards, continuous quality improvement, and responsiveness from preceptors and the Office of Experiential Education.
Table 2. Survey questions related to accreditation standards, continuous quality improvement, and responsiveness from preceptors and the Office of Experiential Education.
Strongly Agree % Somewhat Agree % Neither Agree Nor Disagree % Somewhat Disagree % Strongly Disagree %
My experiential and academic expectations were met for this specific rotation type A56.9%34.6%4.5%3.1%0.8%
During this rotation, I completed direct patient care activities (important: please use direct patient care activities as defined by the ACPE best practice list that you had to document at the beginning of this rotation) B28.7%23.1%26.8%9.6%11.8%
During this rotation, I completed at least one activity, assignment, or topic discussion related to COVID-19/coronavirus A63.7%16.3%6.2%5.6%8.2%
I received individualized attention from my preceptor during this rotation that met my expectations B73.5%18.9%3.9%2.3%1.4%
For the delivery of this rotation, I enjoyed the virtual format A46.2%29.9%13.0%6.8%4.2%
I prefer to have a letter/numeric grade for this virtual rotation rather than a pass/fail grade A63.1%14.1%15.2%5.1%2.5%
I had technology concerns throughout this rotation B11.8%27.3%15.5%15.2%30.1%
Time zone differences were a concern for this rotation A6.5%8.5%16.1%9.3%59.7%
The Office of Experiential Education was helpful in the rotation reassignment process for block 8 A45.4%20.8%27.6%3.4%2.8%
The Office of Experiential Education was responsive to my questions and concerns for this rotation block C46.0%19.8%24.9%5.1%4.2%
My preceptor was responsive to my questions and concerns for this rotation block C84.9%8.6%4.3%0.3%1.9%
A Questions aligned to continuous quality improvement for the remote rotation. B Questions aligned to ACPE accreditation standards for the remote rotation. C Questions aligned to responsiveness for questions/concerns in the remote rotation.
Table 3. Chi-squared test: cross tabulation questions to whether the virtual rotation was better than, equal to, or worse than most/all other APPE rotations that were onsite.
Table 3. Chi-squared test: cross tabulation questions to whether the virtual rotation was better than, equal to, or worse than most/all other APPE rotations that were onsite.
Cross Tabulation Responses (n)
QuestionResponse Choices (n, %)Remote APPE Was Better than Most/All Other RotationsRemote APPE Was Equal to Most/All Other RotationsRemote APPE Was Worse than Most/All Other Rotationsp-Value
Did you also complete paid work experience while completing APPE virtually?Yes (172, 48.5%)2911726p > 0.05
No (183, 51.5%)3012330
Compared to the rest of the APPE year, the number of hours per week that I worked at my job during this remote APPEIncreased (54, 31.4%)9369p > 0.05
Stayed the same (95, 55.2%)176414
Decreased (23, 13.4%)3173
Time zone differences were a concern for this rotationStrongly agree (23, 6.5%)4145p > 0.05
Somewhat agree (30, 8.5%)5196
Neither agree nor disagree (57, 16.1%)8409
Somewhat disagree (33, 9.3%)6243
Strongly disagree (212, 59.7%)3614333
I had technology concerns throughout this rotationStrongly agree (42, 11.8%)42612p > 0.05
Somewhat agree (97, 27.3%)126520
Neither agree nor disagree (55, 15.5%)12385
Somewhat disagree (54, 15.2%)9405
Strongly disagree (107, 30.1%)227114
My remote rotation was primarily conducted in the following formatI was primarily with one preceptor for the entire rotation (180, 50.7%)3012030p > 0.05
I collaborated with 2–3 preceptors and their students throughout the entire rotation on various activities (67, 18.9%)15439
I collaborated with 4 or more preceptors and their students throughout the entire rotation on various activities (108, 30.4%)147717
The Office of Experiential Education was responsive to my questions and concerns for this rotation blockStrongly agree (109, 46.0%)207712p > 0.05
Somewhat agree (47, 19.8%)6356
Neither agree nor disagree (59, 24.9%)9419
Somewhat disagree (12, 5.1%)264
Strongly disagree (10, 4.2%)091
Not applicable (n = 118)
My preceptor was responsive to my questions and concerns for this rotation blockStrongly agree (275, 84.9%)5019134p < 0.05
Somewhat agree (28, 8.6%)11710
Neither agree nor disagree (14, 4.3%)176
Somewhat disagree (1, 0.3%)010
Strongly disagree (6, 1.9%)132
Not applicable (n = 31)
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Prisco, J.L.; Goldman, J.D.; Eguale, T.; Carace, N. Important Factors in Remote Experiential Education. Pharmacy 2022, 10, 122. https://doi.org/10.3390/pharmacy10050122

AMA Style

Prisco JL, Goldman JD, Eguale T, Carace N. Important Factors in Remote Experiential Education. Pharmacy. 2022; 10(5):122. https://doi.org/10.3390/pharmacy10050122

Chicago/Turabian Style

Prisco, Jennifer L., Jennifer D. Goldman, Tewodros Eguale, and Nicole Carace. 2022. "Important Factors in Remote Experiential Education" Pharmacy 10, no. 5: 122. https://doi.org/10.3390/pharmacy10050122

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop