Next Article in Journal
“Discovery Towards Strategy”—Co-Creating an Institutional Innovation Strategy for Digital and Medical Education
Previous Article in Journal
Student-Centered Active Learning Improves Performance in Solving Higher-Level Cognitive Questions in Health Sciences Education
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
IME
Volume 3
Issue 3
10.3390/ime3030027
ime-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Assessing Medical Student Lifestyle Medicine Skills Using an Objective Structured Clinical Examination

by
Denise Kay
1,*,
Magdalena Pasarica
1,
Caridad A. Hernandez
1,
Analia Castiglioni
1,
Christine A. Kauffman
1,
Feroza Daroowalla
1 and
Saleh M. M. Rahman
2
1
College of Medicine, University of Central Florida, Orlando, FL 32827, USA
2
Frank H. Netter MD School of Medicine, Quinnipiac University, North Haven, CT 06473, USA
*
Author to whom correspondence should be addressed.
Int. Med. Educ. 2024, 3(3), 363-373; https://doi.org/10.3390/ime3030027
Submission received: 13 August 2024 / Revised: 17 September 2024 / Accepted: 18 September 2024 / Published: 22 September 2024
Browse Figures
Versions Notes

Abstract

:
(1) The purpose of this project was to create and collect validity evidence for a lifestyle medicine objective structured clinical examination (OSCE) station to assess medical students’ performance related to lifestyle medicine competencies. (2) We developed a lifestyle medicine case/station with an associated observation checklist and rubric. We piloted the checklist and rubric in one lifestyle medicine OSCE station, securing triplicate scores of each student’s performance. For analysis, generalizability (G) theory was utilized for observation checklist data and interclass correlation coefficients (ICC) for patient encounter notes (PENs). (3) One hundred and fifteen third-year medical students completed the lifestyle medicine OSCE station in the Internal and Family Medicine Clerkship. The generalizability coefficient and Phi-coefficient based on the number of encounters (P = 115), facet 1 (nfacet1 = 10 assessment tool checklist items), and facet 2 (nfacet2 = two performance ratings in the live examination) were 0.71 and 0.69, respectively. The average interclass correlation coefficient (ICC) measure for PEN was 0.79 (CI = 0.69–0.85). (4) For this OSCE station, the G-coefficient provides positive indicators for the validity of the observation checklist items. Similarly, the ICC result provides validity evidence for the usefulness of the PEN rubric for capturing lifestyle medicine knowledge reflected in students’ PEN notes.

1. Introduction

The health of the adult population in the United States is increasingly compromised by the prevalence of chronic disease, with 60% of adults managing at least one and 42% managing two or more chronic diseases [1]. Beyond the impact on an individual’s quality of life [2] and productivity [3], chronic diseases account for USD 4.5 trillion in annual healthcare costs [4]. While the average rate of increase is expected to level off in the coming years, the projected increase rate is still at 5% annually, resulting in a projected annual US healthcare expenditure of USD 6.8 trillion by 2030 [5].
The chronic conditions with the greatest economic impact on annual healthcare spending include heart disease, cancer, and diabetes. These diseases and other chronic diseases are associated with modifiable lifestyle behaviors [6], and increasingly convincing evidence suggests that lifestyle changes may prevent and/or treat multiple diseases including cardiovascular disease [7,8,9,10], type 2 diabetes [11], and cancers [12]. For example, evidence-based recommendations for the treatment of type 2 diabetes mellitus [13] start with lifestyle changes, and, if indicated, drugs are added.
“Lifestyle” is a broad concept that incorporates the behaviors and style in which a person lives. Many aspects of a person’s lifestyle can directly affect their overall health, including nutrition, activity, sleep, smoking, and other behaviors. The American College of Lifestyle Medicine (ACLM) defines lifestyle medicine (LM) as the use of therapeutic lifestyle interventions as a primary modality to treat chronic conditions including, but not limited to, cardiovascular diseases, type 2 diabetes, and obesity [14].
In the last decade, international and national publications and organizations have emphasized the need for lifestyle medicine education [15,16], suggesting that interventions that expand medical professional’s abilities to help patients implement health-promoting behaviors are long overdue. Efforts in the medical education community are evident, including the development of the Lifestyle Medicine Education Collaborative, which comprises more than 350 members across the US and abroad [17], and the identification of formalized lifestyle medicine competencies [18].
As the momentum for including lifestyle medicine in the undergraduate medical curriculum continues to build, it is essential to develop assessment tools to evaluate lifestyle medicine competencies and generate validity evidence for their use. While lifestyle medicine Objective Structured Clinical Examination (OSCE) examples are available [19], they lack reports of validity evidence for use in undergraduate medical education.
The purpose of this research was to create and collect validity evidence for a lifestyle medicine OSCE station for use in the assessment of undergraduate medical student’s performance in six lifestyle medicine competencies [18] demonstrated when applying a lifestyle medicine approach with patients. The six competencies are as follows:
  • Demonstrate knowledge that specific lifestyle changes can have a positive effect on a patient’s health outcomes.
  • Use nationally recognized practice guidelines to assist patients in self-managing their health behaviors and lifestyle.
  • Collaborate with patients to develop evidence-based, achievable, specific, written action plans.
  • Assess patient and family readiness, willingness, and ability to make health behavior changes.
  • Perform a history of lifestyle-related health status.
  • Establish effective relationships with patients and families to effect and sustain behavioral change using evidence-based counseling methods and tools and follow-up.
We intend to lay the foundation for assessing lifestyle medicine clinical skills in the undergraduate medical education community. Our assumptions for the assessment of learning are based on propositions from cognitive learning theory, specifically conceptual change and knowledge integration processes, wherein learners “merge two or more originally unrelated knowledge structures into a single structure” [20]. In our case, the two previously unrelated knowledge structures include the history-taking skills students acquired in their preclinical skills training (prior knowledge structures) and the components of a lifestyle medicine history introduced during their clinical training in the Internal/Family Medicine Clerkship (new knowledge structures) (see Figure 1).
We defined “validity evidence” using Kane’s Interpretation/Use Argument (IUA) approach, wherein validity of an assessment is based on the projected interpretation and use of the assessment tool [21]. Our underlying assumption was that the undergraduate medical student’s history taking and patient encounter notes would demonstrate lifestyle medicine knowledge and principles in the patient encounter (as captured in the LM OSCE checklist) and in the utilization of LM principles and assumptions in the treatment and/or management plan presented in the PEN (as demonstrated with the LM PEN OSCE rubric). We would further interpret passing scores on the checklist and rubric as evidence of students’ conceptual shift towards an understanding and use of lifestyle medicine in both how the patient complaint was perceived and the options available for treatment and management. With these propositions met, we would accept that the LM OSCE checklist and rubric could be presented as validity evidence for their use in future iterations of this station and for collecting additional validity evidence with new stations.

2. Materials and Methods

We deployed the following research plan:
2.1
Develop a lifestyle medicine case/station, as well as an observation checklist for scoring the student’s encounter with the standardized patient (SP) and a rubric for scoring the associated Patient Encounter Note (PEN).
2.2
Pilot the checklist and PEN rubric in one lifestyle medicine OSCE station. Secure triplicate scores of performances for each student.
2.3
Conduct appropriate data analysis to establish baseline validity evidence for the checklist.

2.1. The LM OSCE Station Case, Checklist, and PEN Note

We developed an LM OSCE case to be included in the Internal Medicine/Family Medicine (IM/FM) clerkship OSCE. Since the case is an assessment artifact, we have not made it available here but can willingly share it upon request to the first or second authors. To score the case, we developed a checklist for SPs to use to capture student performance after the encounter and a rubric for faculty scoring the Patient Encounter Note. The checklist incorporated previously validated SP checklist items [22] that assessed professionalism, ability to foster a relationship, information gathering, and sharing skills, with an additional ten items designed to assess student’s performance related to targeted lifestyle medicine competencies typically employed during a patient encounter. The Patient Encounter Note rubric for four PEN items was created to assess students’ skills related to lifestyle medicine competencies, specifically lifestyle-medicine-related history taking, knowledge and clinical reasoning, determining the readiness for the change stage, and setting clear personalized management goals. Each PEN item was anchored to a scale descriptor related to the quality of the post-encounter note (see Appendix A: SP Checklist and PEN rubric for items and associated point values). The members of our institution’s Clinical Skills Assessment Committee (CASC) comprise clinical faculty with five or more years of experience developing and grading OSCE stations. The CSAC reviewed and refined the LM OSCE station case and, through iterative review, reviewed and refined the newly created items until they were satisfied with the content, or face validity, as previously described [22].

2.2. Piloting the LM Case, OSCE Checklist, and PEN Rubric-Triplicate Scores

In 2017–2018, a lifestyle medicine case was included as one station of a four station OSCE assessment for the Internal and Family Medicine (IM/FM) Clerkship. All IM/FM clerkship students (115) were required to complete all four stations of the assessment. Three SPs used the LM OSCE station checklist to score each student’s performance. With input from the second author, one SP educator trained the SPs to use the checklist for scoring. The LM items were scored as done or not done. Scoring for the SP encounter took place at two times.
Time 1: Live scoring at the time of the exam—Scoring was completed for the OSCE by the original rater (SP), who used the lifestyle medicine OSCE station checklist to score the encounter live at the OSCE station. One faculty member used the rubric to assign a score to the student’s PEN note.
Time 2: Video scoring to collect validity evidence—Two additional SPs per student used the lifestyle medicine OSCE station checklist to score a video recording of the OSCE station, and two additional faculty per student used the PEN rubric to score the PEN.
Scoring for the PEN notes was completed by three faculty using the PEN scoring rubric. This project was determined not human research by the Institution’s Review Board (May 2018).

2.3. Data Analysis

There are differing opinions regarding the appropriate analysis for validity considerations. Many research designs require the assessment of inter-rater reliability (IRR) to demonstrate consistency among observational ratings provided by multiple coders. Others suggest that, in cases with two or more coders, intra-class-correlation (ICC) is most appropriate when assessing IRR for ordinal, interval, and ratio variables [23]. However, when considering behavioral measurements, generalizability (G) theory is particularly relevant for evaluating dependability (or reliability) [23,24] as it allows researchers to address questions such as the following: Are the judges or the sampling of tasks the major source of measurement error? Can the reliability of the measurement be improved by increasing the number of tasks or the number of judges, or is some combination of the two more effective? Are the test scores adequately reliable for making decisions about the level of a person’s performance for a certification decision? The G-coefficient is used as a reliability index for making a normative decision, while the Phi-coefficient is used for criterion-based decisions.
Analysis of the SP encounter ratings captured in the OSCE station checklist: Given the differing opinions regarding the best statistical method for determining validity, we applied G theory to measure the reliability of the checklist under different facets with facet 1 = assessment tool checklist items and facet 2 = performance ratings captured by the SP in the live examination and later via video observation of the same student by different raters. We further conducted a D study, which used information from the G study to design a measurement procedure that minimizes errors for a particular purpose. All student performance ratings captured with the checklist at the LM OSCE station were included in the analysis.
Analysis of PEN ratings captured using the PEN scoring rubric: PENs were scored by three faculty members (the original rater and two additional faculty members) using the PEN scoring rubric. Since the PEN rubric assessed narrative text rather than performance data, PEN score reliability was calculated using the interclass correlation coefficient (ICC).

3. Results

A total of 115 students participated in the OSCE station.

3.1. Results from the SP Encounter Based on the OSCE Station Checklist (See Table 1 and Table 2)

  • Time 1: The generalizability coefficient (G-coefficient) and Phi-coefficient based on the number of encounters (P = 115), facet 1 (nfacet1 = 10 assessment tool checklist items), and facet 2 (nfacet2 = 2 performance ratings in the live examination) were 0.71 and 0.69, respectively. Three percent variance accounted for the students, less than one percent of the total variance was accounted for by the rater, and 9.4% of the total variance accounted for the rater and item interaction.
  • Time 2: In the facet 2 analysis with video assessment, similar variances were found. The generalizability coefficient (G-coefficient) and Phi-coefficient based on the number of encounters (P = 115), facet 1 (nfacet1 = 10 assessment tool checklist items), and facet 2 (nfacet2 = 2 performance ratings in the video examination) were 0.68 and 0.65, respectively. Three percent variance accounted for the students, less than one percent of the total variance was accounted for by the rater, and 8.9% of the total variance accounted for the rater and item interaction. The D-study indicates the projection of reliability where, in this case, double scoring increased reliability by 14 points (Figure 2).

3.2. Results of PEN Ratings Using the Scoring Rubric

The average interclass correlation coefficient (ICC) measure for PEN was 0.79 (CI = 0.69–0.85), with a Cronbach’s alpha = 0.79, considered excellent.

4. Discussion

The intent for this project was threefold: (1) to provide a tool for potential use in the assessment of trainee’s knowledge of lifestyle medicine principles and practices in simulated patient settings, (2) to collect initial validity evidence for use of the tool in the assessment of lifestyle medicine knowledge and skills as demonstrated in a standardized OSCE encounter, and (3) to provide a tool that can serve as a starting place for additional research in the assessment of LM clinical skills. The assessment of learning was based on principles of conceptual change, specifically knowledge integration. Our assumption was that the skills captured in the observation checklist and the clinical reasoning captured in the PEN scoring rubric would demonstrate the knowledge integration between the newly introduced LM knowledge and skills and students’ previous clinical skills training and that this represents conceptual change.
We adopted previously established criteria for standardized patient encounters as the foundation for the observation rubric and added additional criteria identified by medical education faculty who utilize lifestyle medicine in clinical practice and have experience teaching lifestyle medicine to both medical students and in faculty development settings. Similarly, we used previous formats for PEN rubrics to develop a scoring rubric customized to assess the level of lifestyle medicine principles reflected in the learner PENs. We piloted the checklist and rubric on one OSCE station case and secured triplicate scores for both the observation checklist and the PEN scoring rubric.
Validity is not a fixed property of the assessment tool but varies with subjects, settings, purpose, and other factors. In performance-based assessments, we need to consider potential influences on assessment scores, such as rater bias, the relative difficulty of items or stations, the rater’s or examinee’s attention or mood, the abilities of standardized patients, and the overall environment [25,26]. G theory offers a way to quantify the variance contributed by these factors, which G theory refers to as facets [27,28]. Each form of a given facet is called a condition. This study examined the items for assessment and methods of assessment (live examination versus video assessment) for possible variabilities. From the resulting G-coefficient value, we can confidently say that the items were valid, and there was limited variability from the assessment method. We thus interpret a passing score on the checklist as evidence of successful knowledge integration of the previously acquired history taking, conceptualization of the patient complaints, and associated treatment plans with the newly introduced lifestyle medicine knowledge, skills, conceptualization of the patient complaint, and associated treatment plans that now incorporate lifestyle medicine principles. Further, the D coefficient suggests that this assessment technique and items are reliable and able to predict similar data under different conditions and that the reliability in future iterations can be increased through the use of two raters. With these findings, we propose that the LM OSCE checklist and PEN rubric are appropriate for assessing LM knowledge and skills for this station and for future validity testing with new stations.
We recognize that one limitation of this project was that random assignment to different conditions would have neutralized other systemic variabilities. Still, these findings provide promise for future assessment of lifestyle medicine knowledge and skills in the same and/or new OSCE stations in undergraduate medical education settings. Future research can include using the checklist and rubric under randomized conditions, with new OSCE stations, and in varied medical education training settings.

Author Contributions

Conceptualization, D.K. and M.P.; data curation, S.M.M.R.; formal analysis, S.M.M.R.; funding acquisition, M.P.; investigation, M.P., C.A.H., A.C. and C.A.K.; methodology, D.K. and M.P.; project administration, D.K. and M.P.; supervision, D.K. and M.P.; validation, C.A.H., A.C., C.A.K. and F.D.; writing—original draft, D.K. and S.M.M.R.; writing—review and editing, M.P., C.A.H., A.C., C.A.K., F.D. and S.M.M.R. All authors have read and agreed to the published version of the manuscript.

Funding

This research was partially funded by an Innovations in Medical Education grant from the University of Central Florida College of Medicine, Medical Education Department.

Institutional Review Board Statement

The study was determined as not human research by the University of Central Florida Institutional Review Board (SBE-17-13109, 4/18/2017).

Informed Consent Statement

Not applicable.

Data Availability Statement

The data presented in this article are not readily available because they include results from medical school summative assessments.

Conflicts of Interest

The authors declare no conflicts of interest.

Appendix A. SP Checklist and PEN Rubric

Lifestyle Medicine OSCE Checklist—Obesity Case
SP Name: ___________________ Student Name: _____________________________ Date: ________ Time: _______
Instructions: ∏ the appropriate box to indicate rating
Rating Scale: 1—Yes = Completed 0—No = Not completed
Professionalism and Ability to Foster a RelationshipDONENOT
DONE
1. You washed your hands before patient contact and maintained clean technique throughout encounter.
2. You verified my identity by confirming my name and date of birth
3. You introduced yourself using first, last name, and role (e.g., medical student or working with Dr. X).
4. You explained the purpose of the encounter within the first 1–2 min
5. Friendly communication
( ) You did not greet me, or greeted mechanically, or communicated with me rudely during the encounter.
( ) Your greeting and/or behavior during the encounter was generally polite but impersonal or distant. ( ) You greeted me warmly and communicated with me in a friendly, personal manner throughout the encounter.
( ) Your greeting and overall communication were friendly and compassionate. Overall, you created an exceptionally warm and friendly environment that made me feel comfortable to tell you all of my problems.
6. Respectful treatment
( ) You showed an obvious sign of disrespect during the encounter. e.g.: You treated me as an inferior. ( ) You did not show disrespect to me. However, I observed some signs of condescending behavior.
Although I believe it was unintentional, it made me feel that I was not at the same level with you.
( ) You gave several indications of respecting me. If there was a physical exam, this includes draping me appropriately.
( ) You were exceptionally respectful throughout the encounter. Your verbal and nonverbal communication showed respect for my privacy, my opinions, my rights, and/or my socioeconomic status.
7. Interest in me as a person
( ) You never showed interest in me as a person. You only focused on the disease or medical issue.
( ) In addition to talking about my medical issue, you spent some time getting to know me as a person. ( ) You spent some time exploring how my medical issue affects my personal or social life.
( ) You were exceptionally interested in me as a person. You not only explored how my problem affects my personal and social life, but also showed your willingness to help me address those challenges.
Information Gathering Skills
8. Listening to my story
( ) You rarely gave me any opportunity to tell my story and/or frequently interrupted me while I was talking, not allowing me to finish what I said. Sometimes I felt you were not paying attention (for example, you asked for information that I already provided).
( ) You let me tell my story without interruption, or only interrupted appropriately and respectfully. You seemed to pay attention to my story and responded to what I said appropriately.
( ) You allowed me to tell my story without inappropriate interruption, responded appropriately to what I said, and asked thoughtful questions to encourage me to tell more of my story.
( ) You were an exceptional listener. You encouraged me to tell my story and checked your understanding by restating important points.
Information Sharing Skills
9. Appropriate vocabulary
( ) You used vocabulary that was too simple or too complex for me, or frequently used medical terms without explaining them to me. Sometimes I could not understand what you said to me without asking for explanations of terms you used.
( ) Your vocabulary was generally appropriate but you sometimes inadvertently used medical terms without explaining them to me.
( ) Your vocabulary was appropriate and if needed you provided brief explanations of any medical terms you used without my prompting.
( ) Your vocabulary was appropriate and you always provided clear and full explanation of relevant medical terms you used. In addition, you helped me better my understanding of my condition with the medical terms you explained to me.
10. Providing clear explanation
( ) You rarely explained things to me; you did not help me better understand my situation.
( ) You gave me only brief explanations of my situation; you did not help me understand what would happen next.
( ) You gave me a full and understandable explanation of my situation, pertinent findings, and important next steps.
( ) You gave me a full explanation of my situation, your thinking about it and your recommendation, and probed my understanding by letting me summarize pertinent information.
11. Discussion of options/plans
( ) You did not explain any options or plans, you just told me what you would do without asking for my opinion.
( ) You explained options to me, but did not involve me in decision making. If you solicited my opinion, you just ignored it. You made all the decisions for me based on your medical opinion.
( ) You discussed options with me, made recommendations, solicited my opinion regarding the options/plans, and incorporated my opinion into your medical planning.
( ) You not only solicited my input, but also explored the reasons for my choice and showed your understanding and respect for my decisions by negotiating a mutually agreeable plan.
12. Encouraging my questions
( ) You did not solicit questions, or frequently avoided my questions, or did not provide helpful answers.
( ) You sometimes asked if I had questions, but seldom waited at least 5 s to allow me to formulate questions. You addressed my questions briefly without avoiding them.
( ) You actively encouraged me to ask questions, paused to allow me to formulate them, and provided clear and sufficient answers to all of my questions.
( ) You actively encouraged me to ask questions several times during the encounter, with sufficient wait time. You spent significant time and effort to answer my questions clearly and confirmed that I understood the answer and that my concerns were addressed.
13. Closing the encounter
( ) You ended the session abruptly without discussion of next steps or follow up.
( ) You briefly explained what to expect next, but left out essential elements such as a summary of the session and your assessment, the timeline for next steps, and/or asking if I had any questions.
( ) You summarized the session and your assessment and fully clarified next steps. You asked if I had any questions about the plan.
( ) In addition to summarizing the session and clarifying plans, you provided a safety net by explaining possible unexpected outcomes and when and how to seek help, and/or asked about any possible barriers to the plan, and/or affirmed my agreement and commitment to the plan.
History taking skillsDONENOT
DONE
14. You were asked what foods you most often (or typically) eat for breakfast AND lunch AND dinner
15. You were asked what beverages you most often (typically) consume (type AND quantity)
16. You were asked if you typically snack between meals (type AND frequency)
17. You were asked about siting habits during the day (duration AND purpose) for example watching TV, playing video games, reading, etc…
18. You were asked about physical activity habits (duration AND type AND intensity)
19. You were asked about how many hours of sleep you get per night (average or usually)
20. You were asked about Tabaco use (type AND duration AND quantity)
21. You were asked about alcohol use (type AND duration AND quantity)
22. You were asked about illegal drugs use (type AND duration AND quantity)
23. You were asked if you tried to lose weight before. If yes, how?
Adapted from Iramaneerat, C.; Myford, C.M.; Yudkowsky, R.; Lowenstein, T. Evaluating the effectiveness of rating instruments for a communication skills assessment of medical residents. Adv. Health Sci. Educ. Theory Pract. 2009, 14, 575–594. https://doi.org/10.1007/s10459-008-9142-2 [22].
Lifestyle Medicine PEN rubric—Obesity Case
  • What are the relevant lifestyle profile patterns of this patient?
    • No entry (0 points)
    • Key lifestyle profile items are missing or lifestyle history data described, but not summarized in lifestyle profile. (10 pt)
    • Most key lifestyle profile items are identified, but poorly documented or disorganized. (20 pt)
    • Most key lifestyle profile items are identified in a concise organized manner. (30 pt)
    • All key lifestyle profile items are identified in a concise organized, manner, with little irrelevant information. (40 pt)
    Correct answer: Diet is high in simple carbs and fat. Patient has sedentary activity with minimal exercise of light intensity. Insufficient/low quality sleep. No Tobacco, no drugs. Within normal limits alcohol intake, however high calorie content.
  • Which profile items could be targeted for evidence based lifestyle intervention? Describe 2 items.
    • No profile items are correctly identified or no entry (0 pt)
    • One profile item is correctly identified (5 pt)
    • Two profile items are correctly identified (10 pt)
    Correct answer: Diet is high in simple carbs and fat. Patient has sedentary activity with minimal exercise of light intensity. Insufficient/low quality sleep. High calorie alcohol intake.
  • What stage is the patient’s readiness for change at the beginning of this encounter? Justify your answer in 3 bullet points.
    • The stage of change is incorrectly identified or no entry (0 pt)
    • The stage of change is correctly identified but not justified (5 pt)
    • The stage of change is correctly identified and justified (10 pt)
    Correct answer: Preparation. Reasons: takes the charge to make an appointment with physician to treat obesity, started walking for exercise (as a small change), did not take actions that will be sufficient enough to cause a meaningful weight loss, is aware of the problem, somewhat ownership of the problem, is motivated intrinsically by the physical appearance and health concerns
  • What were the 2 most important lifestyle change recommendations you can make for this patient using the SMARTER goal approach (specific, measurable, attainable, relevant, time bound, evaluate, re-evaluate/adjust)
    • No entry (0 points)
    • Goal is not accurate, missing most elements of SMARTER goal. (10 pt)
    • Goal is accurate, presented in the correct format, with most elements of SMARTER goal described in a concise, organized manner. (30 pt)
    • Goal is accurate, presented in the correct format, with all elements of SMARTER goal described in a concise, organized manner. (40 pt)
    Correct answer example:
    • S: Replace sugar containing soda with diet soda
    • M: Will keep track of this in the calendar on the phone
    • A: I can do this because I do not really mind the diet soda taste
    • R: Because this will decrease, empty calories and increase chances of weight loss T: Starting tomorrow
    • E: Review checkmarks on the calendar every 2 weeks R: Review body weight every 3 months

References

  1. Buttorff, C.; Teague, R.; Bauman, M. Multiple Chronic Conditions in the United States; RAND Corporation: Santa Monica, CA, USA, 2017; Available online: https://www.rand.org/pubs/tools/TL221.html (accessed on 16 June 2021).
  2. Sagner, M.; Katz, D.; Egger, G.; Lianov, L.; Schulz, K.-H.; Braman, M.; Behbod, B.; Phillips, E.; Dysinger, W.; Ornish, D. Lifestyle medicine potential for reversing a world of chronic disease epidemics: From cell to community. Int. J. Clin. Pract. 2014, 68, 1289–1292. [Google Scholar] [CrossRef] [PubMed]
  3. Partnership to Fight Chronic Disease. The Impact of Chronic Disease on Business and U.S. Competitiveness. Almanac of Chronic Disease. 2009. Available online: https://www.fightchronicdisease.org/sites/default/files/docs/2009AlmanacofChronicDisease_updated81009.pdf (accessed on 15 June 2021).
  4. Center for Medicare & Medicaid Services. National Health Expenditures 2022 Highlights; 2023. Available online: https://www.cms.gov/newsroom/fact-sheets/national-health-expenditures-2022-highlights (accessed on 30 January 2024).
  5. Keehan, S.P.; Fiore, J.A.; Poisal, J.A.; Cuckler, G.A.; Sisko, A.M.; Smith, S.D.; Madison, A.J.; Rennie, K.E. National health expenditure projections, 2022–2031: Growth to stabilize once the COVID-19 public health emergency ends. Health Aff. 2023, 42, 886–898. [Google Scholar] [CrossRef] [PubMed]
  6. National Center for Chronic Disease Prevention and Health Promotion. About Chronic Diseases; 2022; About Us|National Center for Chronic Disease Prevention and Health Promotion (NCCDPHP) CDC. Available online: https://www.cdc.gov/chronic-disease/about/?CDC_AAref_Val=https://www.cdc.gov/chronicdisease/about/index.htm (accessed on 17 September 2024).
  7. Arena, R.; Guazzi, M.; Lianov, L.; Whitsel, L.; Berra, K.; Lavie, C.J.; Kaminsky, L.; Williams, M.; Hivert, M.-F.; Franklin, N.C.; et al. Healthy lifestyle interventions to combat noncommunicable disease—A novel nonhierarchical connectivity model for key stakeholders: A Policy Statement From the American Heart Association, European Society of Cardiology, European Association for Cardiovascular Prevention and Rehabilitation, and American College of Preventive Medicine. Mayo Clin. Proc. 2015, 90, 1082–1103. [Google Scholar] [CrossRef] [PubMed]
  8. Schröder, H.; Salas-Salvadó, J.; Martínez-González, M.A.; Fíto, M.; Corella, D.; Estruch, R.; Ros, E. Baseline adherence to the Mediterranean diet and major cardiovascular events: Prevención con dieta mediterránea trial. JAMA Intern. Med. 2014, 174, 1690–1692. [Google Scholar] [CrossRef] [PubMed]
  9. Hu, F.B.; Willett, W.C. Optimal diets for prevention of coronary heart disease. JAMA 2002, 288, 2569–2578. [Google Scholar] [CrossRef] [PubMed]
  10. Franklin, B.A.; Cushman, M. Recent advances in preventive cardiology and lifestyle medicine: A themed series. Circulation 2011, 123, 2274–2283. [Google Scholar] [CrossRef] [PubMed]
  11. Dutton, G.R.; Lewis, C.E. The look ahead trial: Implications for lifestyle intervention in type 2 diabetes mellitus. Prog. Cardiovasc. Dis. 2015, 58, 69–75. [Google Scholar] [CrossRef] [PubMed]
  12. Song, M.; Giovannucci, E. Preventable incidence and mortality of carcinoma associated with lifestyle factors among white adults in the united states. JAMA Oncol. 2016, 2, 1154–1161. [Google Scholar] [CrossRef] [PubMed]
  13. Standards of medical care in diabetes-2016: Summary of revisions. Diabetes Care 2016, 39 (Suppl. 1), S4–S5. [CrossRef] [PubMed]
  14. American College of Lifestyle Medicine. Available online: https://lifestylemedicine.org/about-us/ (accessed on 15 June 2021).
  15. Safeer, R. Clinicians and leaders want more lifestyle medicine and preventive care. NEJM Catal. 2020, 1–4. [Google Scholar] [CrossRef]
  16. Eisenberg, D.M.; Burgess, J.D. Nutrition Education in an Era of Global Obesity and Diabetes: Thinking Outside the box. Acad. Med. 2015, 90, 854–860. Available online: https://search.ebscohost.com/login.aspx?direct=true&db=edo&AN=103596119&authtype=shib&site=eds-live&scope=site&custid=current&groupid=main&authtype=shib (accessed on 23 June 2021).
  17. Trilk, J.L.; Muscato, D.; Polak, R. Advancing lifestyle medicine education in undergraduate medical school curricula through the lifestyle medicine education collaborative. Am. J. Lifestyle Med. 2018, 12, 412–418. [Google Scholar] [CrossRef] [PubMed]
  18. Lianov, L.; Johnson, M. Physician competencies for prescribing lifestyle medicine. JAMA 2010, 304, 202–203. [Google Scholar] [CrossRef] [PubMed]
  19. Nawaz, H.; Petraro, P.V.; Via, C.; Ullah, S.; Lim, L.; Wild, D.; Kennedy, M.; Phillips, E.M. Lifestyle medicine curriculum for a preventive medicine residency program: Implementation and outcomes. Med. Educ. Online 2016, 21, 29339. [Google Scholar] [CrossRef] [PubMed]
  20. Schneider, M. Knowledge integration. In Encyclopedia of the Sciences of Learning; Seel, N.M., Ed.; Springer: Boston, MA, USA, 2012; pp. 1684–1686. [Google Scholar]
  21. Kane, M. Validating the interpretations and uses of test scores. J. Educ. Meas. 2013, 50, 1–73. [Google Scholar] [CrossRef]
  22. Iramaneerat, C.; Myford, C.M.; Yudkowsky, R.; Lowenstein, T. Evaluating the effectiveness of rating instruments for a communication skills assessment of medical residents. Adv. Health Sci. Educ. Theory Pract. 2009, 14, 575–594. [Google Scholar] [CrossRef] [PubMed]
  23. Cicchetti, D. Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instrument in psychology. Psychol. Assess. 1994, 6, 284–290. [Google Scholar] [CrossRef]
  24. Hallgren, K. Computing inter-rater reliability for observational data: An overview and tutorial. Tutor. Quant. Methods Psychol. 2012, 8, 23–34. [Google Scholar] [CrossRef] [PubMed]
  25. Keszei, A.P.; Novak, M.; Streiner, D.L. Introduction to health measurement scales. J. Psychosom. Res. 2010, 68, 319–323. [Google Scholar] [CrossRef] [PubMed]
  26. Streiner, D.L.; Geoffrey, N.R.; Cairney, J. Health Measurement Scales: A Practical Guide to Their Development and Use, 5th ed.; online edn; Oxford Academic: Oxford, UK, 2014. [Google Scholar] [CrossRef]
  27. Brennan, R. Performance assessments from the perspective of generalizability theory. Appl. Psychol. Meas. 2000, 24, 339–353. [Google Scholar] [CrossRef]
  28. Streiner, D.L. Starting at the beginning: An introduction to coefficient alpha and internal consistency. J. Personal. Assess. 2003, 80, 99–103. [Google Scholar] [CrossRef] [PubMed]
Ime 03 00027 g001
Figure 1. Diagram comparing the prior knowledge structures in traditional clinical skills OSCEs to the integration of new knowledge captured by using the lifestyle medicine OSCE checklist and PEN rubric with the lifestyle medicine case.
Figure 1. Diagram comparing the prior knowledge structures in traditional clinical skills OSCEs to the integration of new knowledge captured by using the lifestyle medicine OSCE checklist and PEN rubric with the lifestyle medicine case.
Ime 03 00027 g001
Ime 03 00027 g002
Figure 2. Decision study (D) of potential source of measurement errors from number of items (facet 1) and performance ratings (facet 2) indicating an increase in performance ratings increases reliability by 14 points.
Figure 2. Decision study (D) of potential source of measurement errors from number of items (facet 1) and performance ratings (facet 2) indicating an increase in performance ratings increases reliability by 14 points.
Ime 03 00027 g002
Table 1. G study analysis of 115 UCF College of Medicine third-year, 2017–2018, internal/family medicine clerkship student lifestyle medicine OSCE station results as rated by the standardized patient in the live examination using the lifestyle medicine OSCE assessment checklist.
Table 1. G study analysis of 115 UCF College of Medicine third-year, 2017–2018, internal/family medicine clerkship student lifestyle medicine OSCE station results as rated by the standardized patient in the live examination using the lifestyle medicine OSCE assessment checklist.
SourcedfSSMSVarianceG
p114.00114.5581.0050.0360.712
F19.0031.6823.5200.013
F21.001.4131.4130.001Phi
P*F11026.00241.6680.2360.0940.687
P*F2114.0011.5370.1010.005
F1*F29.002.5310.2810.002
P*F1*F21026.0049.0190.0480.048
p = student, F1 = checklist, F2 = SP scores in the live examination.
Table 2. G study analysis of 115 UCF College of Medicine third-year, 2017–2018, internal/family medicine clerkship student lifestyle medicine OSCE station assessment results as rated by faculty via the video observation using the lifestyle medicine OSCE assessment checklist.
Table 2. G study analysis of 115 UCF College of Medicine third-year, 2017–2018, internal/family medicine clerkship student lifestyle medicine OSCE station assessment results as rated by faculty via the video observation using the lifestyle medicine OSCE assessment checklist.
SourcedfSSMSVarianceG
p114.00102.6970.910.0310.678
F19.0038.7504.3060.017
F21.000.2500.2500.000Phi
P*F11026.00231.250.2250.0890.653
P*F2114.0012.7500.1120.006
F1*F29.002.3230.2580.002
P*F1*F21026.0048.6770.0470.047
p = student, F1 = checklist, F2 = faculty scores via the video examination.
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Kay, D.; Pasarica, M.; Hernandez, C.A.; Castiglioni, A.; Kauffman, C.A.; Daroowalla, F.; Rahman, S.M.M. Assessing Medical Student Lifestyle Medicine Skills Using an Objective Structured Clinical Examination. Int. Med. Educ. 2024, 3, 363-373. https://doi.org/10.3390/ime3030027

AMA Style

Kay D, Pasarica M, Hernandez CA, Castiglioni A, Kauffman CA, Daroowalla F, Rahman SMM. Assessing Medical Student Lifestyle Medicine Skills Using an Objective Structured Clinical Examination. International Medical Education. 2024; 3(3):363-373. https://doi.org/10.3390/ime3030027

Chicago/Turabian Style

Kay, Denise, Magdalena Pasarica, Caridad A. Hernandez, Analia Castiglioni, Christine A. Kauffman, Feroza Daroowalla, and Saleh M. M. Rahman. 2024. "Assessing Medical Student Lifestyle Medicine Skills Using an Objective Structured Clinical Examination" International Medical Education 3, no. 3: 363-373. https://doi.org/10.3390/ime3030027

APA Style

Kay, D., Pasarica, M., Hernandez, C. A., Castiglioni, A., Kauffman, C. A., Daroowalla, F., & Rahman, S. M. M. (2024). Assessing Medical Student Lifestyle Medicine Skills Using an Objective Structured Clinical Examination. International Medical Education, 3(3), 363-373. https://doi.org/10.3390/ime3030027

Article Metrics

Back to TopTop