A Peer Mentoring Program for Faculty Development in Presentation Skills

Abstract

Junior faculty are often called upon to deliver high-stakes large-group presentations. Training in the skills needed to do this effectively is often lacking. We devised a 1.25 h coaching program. The coach analyzed a practice run of the presentation using a locally developed assessment tool. Areas covered included public speaking skills, promoting learner understanding and retention, creating a dynamic learning climate, and optimal use of slides. COVID-19 necessitated a switch to virtual coaching, and we studied the impact of virtual vs. in-person coaching. We added two additional coaches and studied the transferability of the coaching component. There was high uptake of the offered coaching. Participant surveys showed improved comfort levels with large-group presentations; had a sense that their presentation skills had improved; showed an increased likelihood of volunteering for future speaking opportunities; and were likely to recommend the program. Comparisons between virtual and in-person coaching showed no statistical difference, and there was little difference between the original coach and the subsequent two coaches. Qualitative assessments revealed broad areas in which faculty felt the program had been most impactful. The coaching program was well-received and resulted in concrete positive changes in presenter behavior. Conducting the coaching in a virtual manner may increase the feasibility of the intervention, as would expanding the coach pool.

1. Introduction

Peer mentoring is being increasingly studied as a tool to improve teaching skills of medical faculty [1,2,3]. Its effectiveness is based on the principles of direct observation and deliberate practice. The direct observation of clinical skills has been shown to improve feedback [4,5,6] and increase learner confidence [2,7] and skill [7,8]; it is likely that direct observation of teaching skills confers similar benefit, but more data are needed. Deliberate practice, which involves repetitively executing specific skills with an ongoing receipt of feedback to gain mastery of those skills, is a critical component of achieving competency [9].

Many junior faculty members in academic institutions are required to give didactic presentations to showcase their academic niche and introduce their work to the medical community. These large-group presentations, often involving esoteric scientific content, require special skill to capture and maintain the attention of the audience. While there are well-established pedagogical techniques which can be applied to this format, most junior faculty are not provided with any kind of training in presentation skills. Peer mentoring, which incorporates both direct observation and deliberate practice, has not been well-studied specifically to improve the presentation skills of medical faculty, and questions of sustainability [10] and the ideal peer mentoring model [11] have not yet been answered in the literature.

The goal of this project was to enhance the presentation skills of junior faculty through the mentoring process of direct observation, targeted feedback, and deliberate practice. Our primary research objective was to evaluate the effect of this intervention on mentored faculty. We also sought to assess the impact of moving this mentoring from an in-person to virtual format with the onset of COVID-19, as well as the outcome of employing additional mentors.

2. Materials and Methods

Roughly 10–12 junior (i.e., assistant professor-level) faculty members deliver a Department of Medicine Grand Rounds presentation each year at the University of Wisconsin. These 60 min presentations are typically given in a large auditorium, to an audience comprising >50 members of the Department of Medicine, as well as an online audience of 50–100 members. The content usually includes the clinical and/or research focus of the faculty member. Each assistant professor scheduled to give Grand Rounds in 2015–2022 was prospectively invited to participate in the mentoring program.

Junior faculty who are to present Grand Rounds were contacted via email by the chair of the Department of Medicine Education Committee and offered the opportunity to receive feedback and guidance on their upcoming presentation. From 2015–2020, those who chose to participate were referred to a faculty member who has previously undergone training in public speaking skills and has significant experience with the direct observation and mentoring of teaching and presentation skills. Starting in 2020, with the transition to virtual presentations because of the COVID-19 pandemic, the mentoring sessions were also conducted remotely, and two other faculty members were included as mentors to reduce the burden on the primary mentor. These additional faculty mentors were also experienced medical educators. They received training from the original mentor, including instructions on how to use and apply the assessment tool, and took part in a discussion about mentoring presentation skills.

The presenter and one of the three mentors met in the auditorium, and the presenter delivered a practice run of the presentation. During the practice run, the mentor conducted a comprehensive analysis of the teaching performance using a locally developed assessment tool adapted from the literature [12] and previously validated instruments [13,14,15]. We added recommendations for virtual presentations during the pandemic adapted from published recommendations [16] (Table 1).

Table 1. Locally developed assessment tool.

Dynamic Presentation
Topic	Explanation
Open with a “POW statement”	Catchy/grabby opening sentence (could be a story, a question, a case, or an unusual statistic) that captures the attention of the audience immediately. Also try and share your enthusiasm for the topic which engages the audience (i.e., why did this aspect of this topic spark your interest?). Attach relevance to your material early on, so your audience is motivated to pay attention (e.g., is there some way you can explain how this topic is relevant to your audience?).
Eye Contact	Maintain as much eye contact with the audience as possible, because this is more interesting than if you are looking at the screen or the laptop. Use arrows and circles to direct your audience’s attention to something on the slide, which will allow you to maintain more eye contact with the audience (instead of using a laser pointer).
Gestures	Gestures make your talk more dynamic (and the bigger the gestures the better, i.e., out of the “strike zone”).
Movement	Consider coming out from behind the podium (either standing next to it or moving away from it, make sure and use a remote slide advancer if you do this). Make purposeful steps as you make each point, as it frees you up to convey more energy, and generally makes your talk more expressive and engaging (which is why TED talks do not have podiums!).
Pauses/Non-Words	Avoid non-words like “um”, “uh”, and “you know”; eliminating these helps you sound more articulate; being mindful of this can help, as can practicing your talk out loud a few times. Strive to be comfortable with pauses as your thoughts gather, rather than feeling a need to fill the space. Tip: Adding deliberate pauses at key moments in your talk will increase the emphasis of those points or allow your audience to process a more complicated explanation.
Vocal Variety, Facial Expression	The more inflection in your voice and the more facial expression, the more dynamic your speaking is. Smiling occasionally can make a speaker come across as more engaging.
Energy	The more energy you can present while speaking, the better everything else is (e.g., the vocal variety, the facial expression, the movement, etc.), and this translates into people being more engaged. Use the normal nervousness of the talk to translate into more energy.
Promoting Understanding and Retention
Communication of Teaching Goals	Think about what exactly ARE your goals for your audience? Come up with and state what you want the audience to be able to do by the end of your talk that they could not do when they entered the room. Explaining the specific goals you have for your learners will help frame the rest of the talk for them, and they will know what they are supposed to learn. Emphasize the relevance of these goals to the audience, which will make them more likely to pay careful attention. Distinguish between the educational objectives/goals you have for the audience members (which often involves verbs like list, identify, distinguish, name) versus the list of topics you are going to cover, which we would call the outline or agenda. Use take-home points at the end (and/or middle!), which helps improve learner retention of your key points.
Organization	Add an overview/outline at the beginning, which can help your listeners stay organized as you move through the talk. Be as clear as possible in announcing when you are moving to a new topic (transition statement): this will help keep your audience from getting lost as you move from subject to subject.
Clarity (avoiding miscommunication and misunderstanding)	Define any new terms. Use analogies and examples to explain new concepts. Using cases is a great way to contextualize the material.
Emphasis of critical teaching points	Cue important points (e.g., “OK, this is a really key point, so listen carefully…”). Vary voice quality/speed to emphasize a key point. Use visual cues (e.g., arrows or circles in PowerPoint) to highlight key points. Repetition is good (e.g., re-explaining new words/concepts or reiterating key take-home points).
Active Learning	A fundamental part of adult learning theory; your audience retains more when actively engaged. Try using polling software to ask audience opinion, learn audience experience, or conduct mini quizzes.
Leveraging the Slides
YOU are the speaker!	Remember that the slides are there to interpret you; you are not there to interpret the slides (which is why you want to minimize the amount of text on the slides and avoid looking at the computer or the screen).
The 7 × 7 rule: 7 bullets per slide, 7 words per bullet	The text does not need to include all the information. Minimizing words will prevent the audience from having choose between reading your slides or listening to you; if you find you are wrapping to the next line on your slide, it is time to cut some words! Aim for larger font sizes (>28 point).
Using Images	The more you can replace text with images, the more engaged your audience will be (e.g., a few photos of patients or providers and/or a few more simple tables/graphs).
Build Sequencing	Introduce bullets and images ONLY when you are ready to discuss them; this will focus your audience’s attention only on what you are currently talking about and also simplifies busy slides for them. Consider this also when showing complicated slides, e.g., with tables/graphs.
Virtual presentations
Lighting	The best is natural lighting in front of you, the next best is a ring light, and the worst is lighting behind you
Camera	Look at your camera frequently, as this is analogous to eye contact with your audience. Your camera should be in front of you and at eye level.
Audio	Test ahead of time. Use headphones and microphone if possible (instead of the laptop mic).
Energy	Amp this up for virtual presentations. Use more voice inflection and facial expression and minimize other body movement (rotating chair, rocking on feet). Minimize hand gestures on video if you are located close to the camera (head and upper body) since this is distracting.
Background	Use a minimalist approach with little behind you or blur your background.
Chat function	Assign someone to monitor the chat.

The mentor recorded the practice run on an iPad to enhance and augment feedback. Feedback focused on specific, low-inference behaviors, to maximize the likelihood of actual behavior change. Broad areas covered included techniques of public speaking; creating a dynamic learning climate; techniques to promote understanding and retention by the learner; and optimal use of slides. The presenter repeated certain portions of the lecture after feedback was delivered to allow deliberate practice to improve skill. The entire mentoring session took approximately 1.25 h. Table 2 illustrates specific examples of changes made after mentoring. Each mentored faculty member was asked to complete an evaluation of the program after their presentation.

Table 2. Two examples of before-mentoring and after-mentoring introductory statements and slides.

Pre-Mentoring	Post-Mentoring
Ex 1: Opening statement: “Thank you for that great introduction. I’m going to tell you today about some of the work my lab is doing in trying to understand how different nutrients regulate both weight, obesity, as well as glycemic control in mice and humans”.	Ex 1: Opening statement: “What should we eat?? If you are a laboratory mouse, it’s very easy, you eat the food that’s provided to you by your caretakers and research team. But as humans, we have a lot more options”.
Ex 2: First slide: Ex 2: Narrative: “So idiopathic pulmonary fibrosis is one of a number of interstitial lung diseases. Certainly, to a general audience like this, it may not be apparent where this disorder fits in among the alphabet soup of interstitial lung diseases. This is a schematic showing how one might organize their thought process about interstitial lung diseases”.	Ex 2: First Slide: Ex 2: Narrative: “Would you be surprised I told you that idiopathic pulmonary fibrosis kills as many people in the United States per year as automobile accidents? I think for the non-pulmonologists out there, that might be a bit of a shocking statement. But it’s true” [17].

All descriptive numerical data are summarized as mean and standard deviation. We used Student’s t-test for numerical comparisons. Categorical data are expressed as counts and frequencies and compared using Fisher’s exact test. The regression odds analysis is conducted using a logistic regression model. All figures are constructed using slide plots. We analyzed all data using SAS 9.4 and STATA version 17. Open-ended responses were analyzed for themes, grouped into categories, and reported by total number and percents.

3. Results

A total of 51 of 53 (96%) assistant professors scheduled to give Grand Rounds between September 2015 and April 2022 accepted the invitation to be mentored. All faculty completed the evaluation form which included four questions for speakers to answer using a 1–10 scale (where 10 was the top of the scale). The questions were: do you feel more comfortable delivering a large-group presentation; to what extent do you feel your presentation skills have improved; are you more likely to volunteer for speaking opportunities in the future; and would you recommend this mentoring program to others? High ratings were seen in all domains, with relatively lower ratings for likelihood to volunteer for future speaking opportunities.

To assess the possible variable impact of virtual mentoring, we assessed the ratings by those mentored virtually as compared to those mentored in-person and found no difference (Table 3). To assess the use of different mentors on the effectiveness of the program, we compared the original mentor (Mentor A) to the additional mentors (Mentor B and C). The evaluation of the program was similar among speakers who were mentored by the additional mentors as compared to the original mentor with one statistically significant, but minimally important, difference in recommending the mentoring program to others (Table 4).

Table 3. Evaluation of Grand Rounds mentoring program by teaching mode.

	All Faculty Mentored	Mentored in-Person	Mentored Virtually	P	OR	95% CI	P
	N = 51 Mean (SD)	N = 31 Mean (SD)	N = 20 Mean (SD)
Presentation comfort	9.0 (1.0)	8.9 (1.1)	9.2 (0.93)	0.46
Extent that presentation skills improved	8.9 (0.99)	8.9 (0.96)	8.9 (1.1)	0.92
Would volunteer for future speaking opportunities	7.6 (1.4)	7.6 (1.4)	7.7 (1.6)	0.69
Would recommend program	9.9 (0.31)	10 (0)	9.9 (0.49)	0.09	0.92	0.62–1.36	0.68

Table 4. Evaluation of Grand Rounds mentoring program by faculty mentored.

	Total N = 51	Mentored by Primary Mentor N = 38 Mean (SD)	Mentored by Additional Mentors N = 13 Mean (SD)	P	OR	95% CI	P
Presentation comfort	9.0 (1.0)	8.9 (1.0)	9.3 (0.95)	0.24	0.65	0.32–1.32	0.24
Extent that presentation skills improved	8.9 (0.99)	8.9 (0.98)	8.8 (1.1)	0.88	1.05	0.56–1.99	0.88
Would volunteer for future speaking opportunities	7.6 (1.4)	7.6 (1.3)	7.9 (1.8)	0.43	0.83	0.53–1.30	0.43
Would recommend program	9.9 (0.31)	10 (0)	9.8 (0.60)	0.02 *

* statistically significant at p ≤ 0.05.

Speakers were asked to qualitatively describe the three factors where mentoring most impacted or changed their presentation. We analyzed the themes and found that improvements regarding slide design, audience interaction, and body language were the most common (Table 5).

Table 5. Speaker self-evaluation comments.

Open-Ended Comments	In-Person Presentation: Total Comments = 79 Frequency of Comment (% Total Comments)	Virtual Presentation: Total Comments = 48 Frequency of Comment (% Total Comments)
Improvement in slide data and design	26 (32.9)	19 (39.6)
Improvement in body language, transitions, and gestures	16 (20.2)	5 (10.4)
Better audience interaction	15 (18.9)	12 (25.0)
The POW statement was very helpful	12 (15.2)	2 (4.2)
Stage presence	5 (6.3)	No comments
Reduced nervousness & built confidence	2 (2.5)	4 (8.3)
7 × 7 rule very useful	2 (2.5)	No comments
Improved on delivery	1 (1.3)	4 (8.3)
Time management	No comments	1 (2.1)

4. Discussion

Our effort to improve presentation skills of faculty using a mentoring model has been successful, with clear lessons learned; some of which are identified in the literature [18]. Our primary objective was to evaluate the impact of this mentoring on trainees, and it seems clear that trainees had self-assessed improvements in their presentation skills, along with increased comfort and confidence in large-group teaching formats. We have been gratified that almost all invited participants have engaged in this non-mandatory opportunity, and we believe that speaks to the desire faculty have to collect feedback on their teaching skills, particularly junior faculty who are often thrust into prominent teaching situations without having had training in the necessary skills. We note that the high evaluation scores we see for self-efficacy and confidence do not translate into an equally high desire to volunteer for future speaking opportunities; this may be due to the inherent human reluctance to speak in front of large audiences, which will require more effort to overcome.

Our participants’ most frequently stated areas of improvement (slide design, audience interaction, body language) were different when compared to a study of lecturers in emergency medicine [18]. Participants in that study rated the most significant improvements in “provides a brief outline”, “provides a conclusion for the talk”, and “clearly states goal of the talk”. This may be due to differences in mentoring strategies and emphasis.

Introducing the option of virtual mentoring seemed to have no impact on the speakers’ perception of the effectiveness of the mentoring. Virtual mentoring adds flexibility and convenience to the mentoring process and allows for mentoring in environments where in-person meetings cannot occur.

Additional mentors’ evaluation outcomes were very similar to the original mentor, suggesting that the standardization of mentoring is feasible. This would reduce the burden on a single mentor and implies that the mentoring program described above is transferable and potentially generalizable.

Limitations of the Study

This study has several limitations. The sample size is relatively small and is limited to assistant professors in one department of a single academic medical center who were mentored by more senior faculty members. We may not have seen the markedly positive results in effectiveness of the program if we had also mentored more experienced faculty. This program is relatively time-intensive, and departmental support is needed for time spent mentoring. Other departments may not be able to offer similar levels of support or have the resources and expertise to build a formal program in presentation mentoring. An important limitation is that learners may view presentation quality differently than the mentors reviewing it. Furthermore, the effectiveness and success of the program was measured by self-assessment, which may confer some bias and heterogeneity. Finally, we recognize that mentoring, by its nature, can depend on the style and technique of the mentor, and this could potentially limit the generalizability of this intervention to other mentors. However, our use of standardizing techniques did not show a significant difference based on the specific mentor.

5. Conclusions

Overall, the results of this study suggest that direct observation and deliberate practice, incorporated into a program of peer mentoring, are effective at increasing self-efficacy and confidence in didactic presentation skills. The maintenance of effectiveness when moving the mentoring to a virtual format and when broadening the mentor pool may enhance the feasibility and sustainability of this program.