Next Article in Journal
Disinfecting Action of Gaseous Ozone on OXA-48-Producing Klebsiella pneumoniae Biofilm In Vitro
Next Article in Special Issue
The Impact of the COVID-19 Pandemic on Research and Volunteering Activities among Medical Students: A Cross-Sectional Study among Romanian and International Students from One Medical Faculty from Romania
Previous Article in Journal
The Association of COVID-19 Stressors and Family Health on Overeating before and during the COVID-19 Pandemic
Previous Article in Special Issue
Development and Evaluation of Interactive Flipped e-Learning (iFEEL) for Pharmacy Students during the COVID-19 Pandemic
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:

Planning a Collection of Virtual Patients to Train Clinical Reasoning: A Blueprint Representative of the European Population

Anja Mayer
Vital Da Silva Domingues
Inga Hege
Andrzej A. Kononowicz
Marcos Larrosa
Begoña Martínez-Jarreta
Daloha Rodriguez-Molina
Bernardo Sousa-Pinto
Małgorzata Sudacka
7 and
Luc Morin
Medical Education Sciences, University of Augsburg, 86159 Augsburg, Germany
School of Medicine and Biomedical Sciences, University of Porto, 4050-513 Porto, Portugal
Department of Bioinformatics and Telemedicine, Jagiellonian University Medical College, 30-688 Krakow, Poland
Aragón Health Research Institute (IIS-Aragón), University of Zaragoza, 50009 Zaragoza, Spain
Institute and Clinic for Occupational, Social and Environmental Medicine, University Hospital, LMU Munich, 80336 Munich, Germany
MEDCIDS—Department of Community Medicine, Information and Health Decision Sciences, Faculty of Medicine, University of Porto, 4200-319 Porto, Portugal
Department of Medical Education, Jagiellonian University Medical College, 30-688 Krakow, Poland
Pediatric and Neonatal Intensive Care Unit, DMU 3 Santé de L’enfant et de L’adolescent, APHP Paris Saclay, Bicêtre Hospital, 94270 Le Kremlin-Bicêtre, France
Institute of Integrative Biology of the Cell, CNRS, CEA, Paris Saclay University, 91190 Gif-sur-Yvette, France
Author to whom correspondence should be addressed.
Int. J. Environ. Res. Public Health 2022, 19(10), 6175;
Submission received: 29 March 2022 / Revised: 11 May 2022 / Accepted: 17 May 2022 / Published: 19 May 2022


Background: Virtual patients (VPs) are a suitable method for students to train their clinical reasoning abilities. We describe a process of developing a blueprint for a diverse and realistic VP collection (prior to VP creation) that facilitates deliberate practice of clinical reasoning and meets educational requirements of medical schools. Methods: An international and interdisciplinary partnership of five European countries developed a blueprint for a collection of 200 VPs in four steps: (1) Defining the criteria (e.g., key symptoms, age, sex) and categorizing them into disease-, patient-, encounter- and learner-related, (2) Identifying data sources for assessing the representativeness of the collection, (3) Populating the blueprint, and (4) Refining and reaching consensus. Results: The blueprint is publicly available and covers 29 key symptoms and 176 final diagnoses including the most prevalent medical conditions in Europe. Moreover, our analyses showed that the blueprint appears to be representative of the European population. Conclusions: The development of the blueprint required a stepwise approach, which can be replicated for the creation of other VP or case collections. We consider the blueprint an appropriate starting point for the actual creation of the VPs, but constant updating and refining is needed.

1. Introduction

Virtual patients (VPs) are “interactive computer simulation[s] of real-life clinical scenarios for the purpose of medical training, education, or assessment” [1]. Their importance has been increasing over the years [2], especially since the beginning of the COVID-19 pandemic [3]. VPs provide a safe environment in which learners can work at their own pace and learn from errors without harming a patient [4]. They are typically designed to unfold in a step-by-step manner, revealing the information about a patient over time leading the learner to the final diagnosis [5,6]. During the process, multimedia elements add authenticity to the VP scenario. Overall, several studies indicated that VPs have the potential to train students in clinical reasoning [7,8,9].
Clinical reasoning is a complex process in which healthcare professionals (e.g., physicians, physiotherapists, nurses) gather and interpret information, generate hypotheses, derive a final diagnosis, and develop treatment plans [10]. VPs demonstrated their effectiveness in improving components of clinical reasoning such as data gathering, generating differential and final diagnoses, and developing a treatment plan [11,12]. However, this process is not only influenced by medical knowledge, but, according to situativity theory, also by contextual factors related to the patient and to the encounter [13,14]. Such factors can be the setting (e.g., emergency room, general practice), or the patient’s age, sexual orientation, comorbidities [15], or behavior [5].
Therefore, providing a collection of VPs to medical students offers the potential for deliberate training of clinical reasoning, but the careful selection of these contextual factors, key symptoms, and diagnoses is crucial [16,17]. Such a balanced selection of key symptoms and (differential) diagnoses in a VP collection prepares students for situations they are likely to experience in practice [18]. Moreover, it enables them to train clinical reasoning by comparing and contrasting, i.e., when they face cases with similar clinical findings, they have to weigh different options based on the relative probability of each diagnosis and the typicality of findings [10,19]. Deliberately varying contextual factors and including atypical presentations influences the complexity of a VP [10,19]. Furthermore, contextual factors are important to create realistic and authentic scenarios [20] that represent the diversity of a patient population adequately. An under- or non-representation of marginalized groups holds the danger of an unintended hidden curriculum [21]. i.e., unintended messages that can bias students, who for example might be subconsciously trained to perceive patients that are male, heterosexual, white, and cis (i.e., the opposite of transgender) as the standard patients in Western countries [22,23,24].
However, previous studies showed that existing VP or case collections tend to represent the real world only to a limited extent in terms of key symptoms, diagnoses, and contextual factors [24,25,26]. For example, one case collection [25] scarcely included patients with a disability, migration background, or chronic conditions despite their worldwide relevance in healthcare [27]. Although resources and didactical advice for the creation of individual VPs are available [18,28], there is hardly any guidance for designing a collection of VPs. Previous VP collections [29] and projects such as [30,31] did either not follow or publish a sophisticated planning approach.
To address this shortcoming on an international level, we formed an interdisciplinary partnership of six institutions from Poland, Germany, Spain, Portugal, and France. In our project, iCoViP (international Collection of Virtual Patients) [32], we aimed at planning and delivering a diverse and realistic VP collection that facilitates deliberate practice of clinical reasoning and meets educational requirements of medical schools in Europe. Therefore, our objectives were to develop and assess a reusable process that helps medical educators plan or evaluate VP collections ensuring that they are
realistic in terms of patient characteristics in Europe,
aligned to educational objectives of medical curricula in Europe to ease curricular integration, and
suitable to train clinical reasoning by comparing and contrasting similar cases in varying contexts.

2. Materials and Methods

Due to the lack of guidance on how to implement such a process, we based our planning on our experience from previous projects [25,30,31] and developed a four-step approach to create the blueprint for our VP collection (see Figure 1). Consequently, this blueprint will serve as the basis for the subsequent VP creation. As agreed upon in the grant proposal of the project, we aimed at creating 125 new VPs to extend an existing collection of 75.
  • Step 1: Definition of criteria for the blueprint
As a first step, we defined the criteria, i.e., variables which needed to be specified for all VPs. We started by extracting criteria from the literature and the experience of our consortium to describe the VPs in the blueprint. Afterwards, we discussed these criteria with all partners and grouped them into (i) disease-related, (ii) patient-related, (iii) learner-related, and (iv) encounter-related criteria, which can be briefly described as followed:
(i) Disease-related: These criteria include the final diagnoses and key symptoms, i.e., the complaints that are the primary reasons for the VPs’ visit. This ensures that our collection will cover the most common diseases and symptoms as the main aspects of the clinical reasoning process. We mapped the final diagnoses to the Medical Subject Heading (MeSH) [33] and the national competency frameworks of partner countries which describe the learning outcomes to be covered by medical schools [34,35,36,37,38]. The mapping with the intended learning outcomes and a standard biomedical thesaurus like MeSH facilitates a curricular integration of VPs covering specific learning objectives. Additionally, we clustered the final diagnoses in disease groups based on their pathogenetic pathway, similarly to the VINDICATE approach used in medical education [39]. We also agreed on covering the onset of the disease, e.g., acute, or chronic, to adequately represent the importance of chronic diseases [27] and avoid their underrepresentation. Finally, we included the closure of the scenario, i.e., whether a patient dies or is successfully discharged, to provide a realistic outcome and avoid tabooing the dying of patients.
(ii) Patient-related: These criteria are the VPs’ characteristics, including their age, sex, sexual orientation, profession, ethnicity, cultural or migration background, disability, and addiction/substance abuse. These aspects are crucial to ensure a diverse and authentic patient population [24,25] and to raise awareness for common biases in the clinical reasoning process [40,41,42].
(iii) Learner-related: We included the role of the learner in the VP scenario as a criterion, as this is an important factor in simulation-based environments [43]. The learner is cast for instance in the role of a resident or consultant, being responsible for a patient.
(iv) Encounter-related: We included the setting in which the consultation with the VPs takes place, such as a university hospital or a doctor’s office, as a criterion. In doing so, we aimed at avoiding an overrepresentation of well-equipped university hospitals and emergency departments and at providing a realistic variety of facilities. Table 1 provides an overview of all included criteria. Once we agreed on all the criteria and the permitted values for list selection under each criterion, we developed a template that allowed partners to populate the blueprint with VP data.
  • Step 2: Identification of data sources for assessing the representativeness of the blueprint.
All partners agreed on a list of suitable key symptoms that the VPs should present with; these were extracted from the literature [44,45] and available national competency frameworks [34,35,36,37,38].
To identify frequent diagnoses, we based our literature search on a systematic review by Finley et al. that reports on the most common conditions in primary care [46]. Additionally, we included articles that recommend diagnoses to be covered in medical education from a daily-practice perspective [47,48,49].
To assess whether our populated blueprint is, as intended, a realistic representation of the European patient population, we identified the most recent, pertinent data from the World Health Organization (WHO) and statistics available for Europe to compare to our VPs.
All selected references and data sources are included in Table 1.
  • Step 3: Populating the blueprint
In this step, we mapped the existing 75 VPs into our blueprint and asked each partner institution to add 25 VPs they intended to develop based on their curricular needs.
  • Step 4: Refinement and reaching consensus
First, we compared the results of step 3 (i.e., the distributions of suggested patient-, learner-, and encounter-related criteria of the blueprint) to the identified literature and statistics (see step 2). Then, if necessary and feasible, we asked partner institutions to modify their VP scenarios to represent a more realistic patient population.
To foster students’ comparing and contrasting of similar cases, which is vital to build competency in clinical reasoning, we excluded key symptoms that were only present in one VP from the final blueprint. Therefore, adjustments had to be made in some cases, leading to a reduced number of key symptoms in the final blueprint.
As the diagnoses of the initially outlined VPs differed substantially from the results of the literature, we applied a Delphi-like approach to reach consensus on the final diagnoses (for details see Appendix A). This approach is widespread in medical education research [62] and curriculum planning [63,64].
Once the final blueprint was approved by all partners, we mapped all VPs to MeSH and the national competency frameworks and performed descriptive analysis for all criteria.

3. Results

Our three main results are the populated blueprint describing the VPs, which has been published as part of the project [65], the descriptive statistics of the criteria, and the actual decision-making process. In the following sections, we show the results of our descriptive analyses.

3.1. Analysis of Disease-Related Criteria

Our final list of VPs includes 29 common key symptoms [65]. The Delphi-like approach resulted in a total of 176 diagnoses across several healthcare disciplines displayed in our blueprint. Of these, 22 are covered by more than one VP [65].
The final diagnoses of our VPs include all 11 main disease categories of the German national catalog of learning objectives in medicine (NKLM) [34], and 15 of 17 disease categories listed in the national framework published by the Polish Ministry of Science [35]. The blueprint also covers 17 of 18 categories in the whitepaper published by the Spanish National Agency for Quality and Accreditation [37], and 11 of 12 categories listed in the French catalog of learning objectives [38]. The Portuguese national framework consists of 5 main groups of diseases that are all covered by our VPs [36]. Although these frameworks vary in their structure and content, they mostly focus on diseases of the cardiovascular, digestive, and respiratory systems. The missing categories were particular topics, such as “safe use of medication” in the French learning objective catalog, or specific disciplines, such as ophthalmology in the Spanish and the Polish frameworks.
Regarding the disease groups of the VINDICATE mnemonic, the most frequent groups were “infectious” (31%), “vascular” (14%) and “immunologic” (14%) (for further values see Appendix B, Table A1).
The VP’s onset of symptoms was acute, subacute, or chronic for 62%, 19%, and 19%, respectively. At the end of the scenario, 6% of the VPs die, 36% require long-term treatment, and 59% are successfully discharged.

3.2. Analysis of Patient-Related Criteria

Our blueprint of 200 VPs includes all age groups, from newborns up to 93 years, with an age distribution of 12% between 0–14, 61% between 15 and 64, and 28% above 65. Compared to the general population in Europe in 2021 (15% between 0–14, 64% between 15 and 64, and 21% above 65) [50], our VPs are older, considering that elderly people tend to seek medical care more frequently than the average population.
The final blueprint includes 1% intersexual or transgender patients compared to an estimated rate of up to 2% in European adults [52]. Eight percent of our VPs are described to be homo-or bisexual, which is slightly higher than the population in Europe, in which the lesbian, gay, bisexual, and transgender (LGBT) population varies among countries but is reported highest in Germany with over 7% in 2016 [53].
Regarding ethnicity, our blueprint comprises White (89%), Black (8%), Asian (2%), and Hispanic (2%) VPs. However, we did not find suitable European data sources for comparison.
The professions of the VPs include 27 of the 38 occupational sectors covered in the Eurostat employment statistics [55] with the five most common occupations being teaching professionals, health professionals, researchers & engineers, personal service workers, and sales workers. The most common occupations in the EU are sales workers, office associate professionals, personal service workers, teaching professionals, and drivers & vehicle operators [55]. The proportion of unemployed VPs among the working-age population is 6%, a bit lower than the average seasonally adjusted unemployment rate in the EU, which ranged between 7 and 8% in 2020 and 2021 [56].
In our blueprint, we have 8% of disabled VPs, compared to an estimated prevalence of 6 to 10% for disabilities in Europe [57].
We included 13% of VPs with a relevant cultural or migration background, for example, refugees or VPs with language barriers. In the EU, the proportion of non-EU citizens and people that were born outside of the EU was 5% and 8%, respectively, in 2020 [58].
In the age group over 15 years, 17% of the VPs are smokers, compared to an average of 18% daily smokers in 2019 in the EU [59]. Moreover, 8% of the VPs in this age group have a relevant alcohol consumption which corresponds to the proportion of adults in the EU with daily alcohol consumption [60].
Table 2 provides a summary of patient-related criteria; all criteria are presented in detail in Table A1 (Appendix B).

3.3. Analysis of Encounter- and Learner-Related Criteria

The learner role is in 46% of the scenarios a resident, in 30% a consultant, and in 18% an intern (for more details see Appendix B, Table A1). About one-third (34%) of the encounters occur in a medical practice, 23% in a hospital, 21% in a university hospital, 12% in an outpatient clinic, and 11% in a rural hospital. Of all hospital settings, 31% take place in an emergency department.

4. Discussion

We developed a blueprint for a VP collection that covers a wide range of medical education needs while ensuring a realistic degree of diversity. The approach we followed to develop this blueprint was challenging and time-consuming. Nevertheless, we believe that it was worth the effort as it enabled us to meet our stated objectives.
To reach our first objective of being realistic in terms of patient characteristics at an international level, we compared the VPs described in our blueprint to sources of population statistics, such as WHO or EUROSTAT. We did not rely on randomizing clinical data for the development of the blueprint [66] but rather reached consensus through a cycle of group discussions, a modified Delphi-approach, and verification steps. Overall, our analyses show that the VPs described in the blueprint represent a patient population more diversely and realistically when compared to previous initiatives [24,25,67]. However, despite our best efforts, we still have some deviations we will refine further during the VP development. For example, the VPs cover a wide range of occupations, including unemployed and retired people, but we have an overrepresentation of professions our consortium is familiar with, such as health and teaching professionals. Also, the proportion of disabled VPs and chronic onsets is lower than we intended it to be. On the other hand, about one-third of our VPs require long-term treatment, emphasizing the importance of chronic conditions in healthcare. We will keep these aspects in mind during the actual development of VPs and refine the collection accordingly.
Our second objective was to ensure the applicability and integrability of the VP collection for medical schools in Europe. To reach this objective, we considered different perspectives from health professions, educators, and researchers across Europe, similar to previous international and interdisciplinary approaches [30,31]. Thus, we were able to consider cultural differences of involved countries [68] as well as country-specific diagnostic and therapeutic approaches [69]. Additionally, we mapped the VPs in our blueprint to national competency frameworks. Through this approach, we hope to ease the integration of the VP collection into the curricula of other schools and increase its acceptance by healthcare educators in Europe [70,71,72].
Our third objective was to plan a VP collection that is suitable to deliberately practice clinical reasoning by comparing and contrasting similar case presentations. There is no gold-standard available for the optimal composition of a VP collection in terms of key-symptoms, diagnoses, and contextual variation. However, Kassirer et al. pointed out that learning clinical reasoning is fostered by a larger collection of VPs [73]. As we based our approach on available guidance as far as possible [28,74], we believe that covering a range of 29 key-symptoms and 176 final diagnoses is a good starting point, but we also see the need for further expanding the collection in the future. Looking at teaching of clinical reasoning through the lenses of situativity theory [13] emphasizes the importance of the context in making diagnostic and therapeutic decisions. For example, the diversity of the VP characteristics exposed in our blueprint provides a valuable opportunity to provide thoughtful feedback to students on how such factors influence clinical reasoning and can be the source of cognitive errors. A discussion or review of errors is a suitable approach to improve clinical reasoning as long as it includes elaborated feedback [75,76].
In addition to reaching our objectives, our discussions helped us in developing a better understanding of patient- and healthcare-related aspects in other countries, which will enrich our teaching and research. For instance, we had a vivid discussion on defining migration background and differences in countries of origin between our partner countries.
Finally, we believe that our approach emphasizes the importance of creating VP or case collections that are diverse, realistic, as well as applicable and shareable on an international level. We hope that implementing such a VP collection in study programs will contribute to their internationalization, which has the potential to facilitate the mobility of health professionals in Europe.
We are aware that our approach has several limitations.
First, we could not identify data sources based on the patient population in Europe to compare our patient-related criteria. Therefore, we had to use statistical data based on the general population in Europe, which certainly differs in terms of age distribution. In addition, we had to obtain information from various data sources, which may have used different methodologies and underlying definitions. However, our aim was not to simulate a perfect real-world patient population but to consider aspects of diversity and an approximation to a real-world population.
Second, we did not yet perform sub-analyses on the distributions involving the combination of different criteria themselves (for example, how disability status or migration background are distributed in relation to the VPs’ sex). During the VP development, further analyses will be undertaken to overcome gender- and ethnicity-related stereotypes.
Furthermore, despite having a diverse group of partners in the project, we must acknowledge that for instance no Scandinavian or Balkan country is represented in our group and therefore, we might be missing the perspective of Northern or Southeastern European countries.
The experience of the recent COVID-19 pandemic taught us that the global health situation can change dramatically in a very short time frame. Global warming, political conflicts, economic crises, but also accelerating technical innovation are likely to contribute to increased fluctuations in patient populations characteristics. Our paper adds to the existing body of knowledge by proposing a methodology to help educators to efficiently reach a realistic VP or case mix suitable for clinical reasoning training. We foresee that there will be a growing demand for such methods to make the educational system fit the actual needs of the changing society. At the same time, we stress that the authenticity reached by such a methodology is not given for ever and that revisions and updates of VP collections are required to keep them up to date.

5. Conclusions

In this study, we developed a process for designing a blueprint for a VP collection that provides a realistic picture of the European patient population, meets educational requirements, and facilitates deliberate practice of clinical reasoning. We believe that our work can help healthcare educators to assess their own VP or case collections, expand their collections within their schools (or in collaboration with others), or implement a similar process. Despite some limitations, we think that the blueprint, which is the product of this process, is an appropriate starting point for creating the VPs. The dynamic nature of the pandemic situation will probably make it inevitable that we will have to do further adaptations and refinements during the creation phase. Also, we consider the blueprint a dynamic document we will regularly consult and update during and after creating the VPs to preserve the high quality of our VP collection.

Author Contributions

All authors contributed substantially to the conceptualization and data curation. A.M. and L.M. analyzed the data. A.M., L.M. and I.H. drafted the first version of the manuscript. All authors reviewed and edited the draft version of the paper and agreed on the final version. All authors have read and agreed to the published version of the manuscript.


The creation of these resources has been (partially) funded by the ERASMUS+ grant program of the European Union under grant no. 2020-1-DE01-KA226-005754. Neither the European Commission nor the project’s national funding agency DAAD are responsible for the content or liable for any losses or damage resulting from the use of these resources.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Data analyzed and generated during the present work are available in the manuscript and online: [65].


We would like to thank all members of the iCoViP consortium and associate partners who participated in the development and population of the blueprint for their valuable contribution to this work. In addition, we would like to thank Sabina Berg and Joanna Fąferek for providing a language and grammar review.

Conflicts of Interest

The authors declare no conflict of interest related to this work. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results.

Appendix A

Figure A1. Flowchart of a Delphi-like approach to reach consensus on the final diagnoses to be included in the blueprint.
Figure A1. Flowchart of a Delphi-like approach to reach consensus on the final diagnoses to be included in the blueprint.
Ijerph 19 06175 g0a1

Appendix B

Table A1. Analysis results of all disease-, patient-, learner- and encounter-related criteria.
Table A1. Analysis results of all disease-, patient-, learner- and encounter-related criteria.
CriterionValuesN% TotalEU/European Data for ComparisonReference
Key symptomsN/Asee blueprint [65][34,35,36,37,38,44,45]
Final diagnosisN/Asee blueprint [65][46,47,48,49]
Disease groupVascular2814.0%N/A
Infectious 6231.0%N/A
Neoplastic 2211.0%N/A
Congenital 136.5%N/A
Autoimmune, immunologic2713.5%N/A
Traumatic 157.5%N/A
Endocrine, metabolic189.0%N/A
Subacute 3819.0%N/A
Scenario closureLong-term treatment7236.0%N/A
Successfully discharged11758.5%N/A
Age0–14 years1911.5%15.1%[50]
15–64 years12261.0%64.1%
≥65 years5527.5%20.8%
Male 9748.5%48.3%
Transgender 10.5%Estimated proportion:
Intersexual 10.5%
Sexual orientationHeterosexual11557.5%N/A
Homosexual 126.0%Proportion of LGBT:
Bisexual 42.0%
Not sexually active (child)2110.5%N/A
Not stated4823.5%N/A
EthnicityWhite 17889.0%N/A
Black 157.5%N/A
Asian 31.5%N/A
Other 10.5%N/A
ProfessionNaming of profession5 most frequent occupations: teaching professionals, health professionals, researchers & engineers, personal service workers, sales workers 5 most frequent occupations: sales workers, office associate professionals, personal service workers, teaching professionals, and drivers & vehicle operators[55]
Unemployed 76.3% 16.5–7.6% 1[56]
Retired 5627.5% N/A
Student 94.5%N/A
Not stated73.5%N/A
No18693.0% N/A
Relevant cultural, language, or migration backgroundYes2713.5%5.1% non-EU citizens
8.3% born outside EU
No17386.5% N/A
Addiction/Substance abuse *Smoker3016.6% 218.4% 2 daily smoking[59]
Ex-Smoker116.1% 2N/A
Alcohol147.7% 28.4% 2 daily alcohol[60]
Illegal drugs32.4% 3“Last month prevalence”:
1.2–8.7% 3
Other10.6% 2N/A
Not Stated3517.5%N/A
Learner’s roleStudent31.5%N/A
Intern 3618.0%N/A
Consultant 5929.5%N/A
Not Stated94.5%N/A
Setting *Hospital4623.0%N/A
University Hospital4120.5%N/A
Rural Hospital2110.5%N/A
Outpatient Clinic 2411.9%N/A
Emergency Room6130.5%N/A
* Numbers do not sum up to 200 (100%) since combinations of values are possible. 1: Referring to people aged 15–74 years who are actively seeking work. 2: Referring age group ≥15 years. 3: Referring age group 15–64 years. N/A = Not applicable.


  1. Ellaway, R.; Candler, C.; Greene, P.; Smothers, V. An Architectural Model for MedBiquitous Virtual Patients. 2006. Available online: (accessed on 21 March 2022).
  2. Lang, V.J.; Kogan, J.; Berman, N.; Torre, D. The Evolving Role of Online Virtual Patients in Internal Medicine Clerkship Education Nationally. Acad. Med. 2013, 88, 1713–1718. [Google Scholar] [CrossRef] [PubMed]
  3. Hege, I.; Sudacka, M.; Kononowicz, A.A.; Nonnenmann, J.; Banholzer, J.; Schelling, J.; Adler, M.; Espinoza, B.; Garrido, M.A.; Radon, K. Adaptation of an international virtual patient collection to the COVID-19 pandemic. GMS J. Med. Educ. 2020, 37, Doc92. [Google Scholar] [CrossRef] [PubMed]
  4. Edelbring, S.; Dastmalchi, M.; Hult, H.; Lundberg, I.; Dahlgren, L.O. Experiencing virtual patients in clinical learning: A phenomenological study. Adv. Health Sci. Educ. 2011, 16, 331–345. [Google Scholar] [CrossRef] [PubMed]
  5. Hege, I.; Dietl, A.; Kiesewetter, J.; Schelling, J.; Kiesewetter, I. How to tell a patient’s story? Influence of the case narrative design on the clinical reasoning process in virtual patients. Med. Teach. 2018, 40, 736–742. [Google Scholar] [CrossRef]
  6. Kononowicz, A.A.; Narracott, A.J.; Manini, S.; Bayley, M.J.; Lawford, P.V.; McCormack, K.; Zary, N. Framework for Different Levels of Integration of Computational Models Into Web-Based Virtual Patients. J. Med. Internet Res. 2014, 16, e23. [Google Scholar] [CrossRef]
  7. Plackett, R.; Kassianos, A.; Kambouri, M.; Kay, N.; Mylan, S.; Hopwood, J.; Schartau, P.; Gray, S.; Timmis, J.; Bennett, S.; et al. Online patient simulation training to improve clinical reasoning: A feasibility randomised controlled trial. BMC Med. Educ. 2020, 20, 245. [Google Scholar] [CrossRef]
  8. Dekhtyar, M.; Park, Y.S.; Kalinyak, J.; Chudgar, S.M.; Fedoriw, K.B.; Johnson, K.J.; Knoche, C.F.; Martinez, L.; Mingioni, N.; Pincavage, A.T.; et al. Use of a structured approach and virtual simulation practice to improve diagnostic reasoning. Diagnosis 2021, 9, 69–76. [Google Scholar] [CrossRef]
  9. Watari, T.; Tokuda, Y.; Owada, M.; Onigata, K. The Utility of Virtual Patient Simulations for Clinical Reasoning Education. Int. J. Environ. Res. Public Health 2020, 17, 5325. [Google Scholar] [CrossRef]
  10. Trowbridge, R.L.; Rencic, J.J.; Durning, S.J. Teaching Clinical Reasoning; American College of Physicians: Philadelphia, PA, USA, 2015. [Google Scholar]
  11. Botezatu, M.; Hult, H.; Tessma, M.K.; Fors, U. Virtual patient simulation: Knowledge gain or knowledge loss? Med. Teach. 2010, 32, 562–568. [Google Scholar] [CrossRef] [Green Version]
  12. Plackett, R.; Kassianos, A.P.; Timmis, J.; Sheringham, J.; Schartau, P.; Kambouri, M. Using Virtual Patients to Explore the Clinical Reasoning Skills of Medical Students: A Mixed Methods Study. J. Med. Internet Res. 2020, 23, e24723. [Google Scholar] [CrossRef]
  13. Durning, S.J.; Artino, A.R.; Boulet, J.R.; Dorrance, K.; van der Vleuten, C.; Schuwirth, L. The impact of selected contextual factors on experts’ clinical reasoning performance (does context impact clinical reasoning performance in experts?). Adv. Health Sci. Educ. 2011, 17, 65–79. [Google Scholar] [CrossRef] [PubMed]
  14. McBee, E.; Ratcliffe, T.; Picho, K.; Artino, A.R.; Schuwirth, L.; Kelly, W.; Masel, J.; Van Der Vleuten, C.; Durning, S.J. Consequences of contextual factors on clinical reasoning in resident physicians. Adv. Health Sci. Educ. 2015, 20, 1225–1236. [Google Scholar] [CrossRef] [PubMed]
  15. Persky, S.; Eccleston, C.P. Medical student bias and care recommendations for an obese versus non-obese virtual patient. Int. J. Obes. 2010, 35, 728–735. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  16. Ericsson, K.A. Deliberate Practice and the Acquisition and Maintenance of Expert Performance in Medicine and Related Domains. Acad. Med. 2004, 79, S70–S81. [Google Scholar] [CrossRef] [PubMed]
  17. Norman, G. Research in clinical reasoning: Past history and current trends. Med. Educ. 2005, 39, 418–427. [Google Scholar] [CrossRef] [PubMed]
  18. Huwendiek, S. Design and implementation of virtual patients for learning of clinical reasoning. GMS J. Med. Educ. 2019, 36, Doc33. [Google Scholar] [CrossRef]
  19. Bowen, J.L. Educational Strategies to Promote Clinical Diagnostic Reasoning. New Engl. J. Med. 2006, 355, 2217–2225. [Google Scholar] [CrossRef] [Green Version]
  20. Kim, S.; Phillips, W.R.; Pinsky, L.; Brock, D.; Phillips, K.; Keary, J. A conceptual framework for developing teaching cases: A review and synthesis of the literature across disciplines. Med. Educ. 2006, 40, 867–876. [Google Scholar] [CrossRef]
  21. Lawrence, C.; Mhlaba, T.; Stewart, K.; Moletsane, R.; Gaede, B.; Moshabela, M. The Hidden Curricula of Medical Education: A Scoping Review. Acad. Med. 2018, 93, 648–656. [Google Scholar] [CrossRef]
  22. Massie, J.P.; Cho, D.Y.; Kneib, C.J.; Burns, J.R.; Crowe, C.S.; Lane, M.; Shakir, A.; Sobol, D.L.; Sabin, J.; Sousa, J.D.; et al. Patient Representation in Medical Literature: Are We Appropriately Depicting Diversity? Plast. Reconstr. Surg. Glob. Open 2019, 7, e2563. [Google Scholar] [CrossRef]
  23. Murphy, M. Hiding in Plain Sight: The Production of Heteronormativity in Medical Education. J. Contemp. Ethnogr. 2014, 45, 256–289. [Google Scholar] [CrossRef]
  24. Turbes, S.; Krebs, E.; Axtell, S. The Hidden Curriculum in Multicultural Medical Education: The role of case examples. Acad. Med. 2002, 77, 209–216. [Google Scholar] [CrossRef] [PubMed]
  25. Urresti-Gundlach, M.; Tolks, D.; Kiessling, C.; Wagner-Menghin, M.; Härtl, A.; Hege, I. Do virtual patients prepare medical students for the real world? Development and application of a framework to compare a virtual patient collection with population data. BMC Med. Educ. 2017, 17, 1–7. [Google Scholar] [CrossRef] [PubMed]
  26. Finucane, P.; Nair, B. Is there a problem with the problems in problem-based learning? Med. Educ. 2002, 36, 279–281. [Google Scholar] [CrossRef] [PubMed]
  27. Busse, R.; Blümel, M.; Scheller-Kreinsen, D.; Zentner, A. Tackling chronic disease in Europe; WHO: London, UK, 2010. Available online: (accessed on 21 March 2022).
  28. Posel, N.; McGee, J.B.; Fleiszer, D.M. Twelve tips to support the development of clinical reasoning skills using virtual patient cases. Med. Teach. 2014, 37, 813–818. [Google Scholar] [CrossRef] [PubMed]
  29. Küfner, J.; Kononowicz, A.A.; Hege, I. Virtual Patient Repositories—A Comparative Analysis. Stud. Health Technol. Inform. 2014, 205, 788–792. [Google Scholar] [CrossRef] [PubMed]
  30. Zary, N.; Hege, I.; Heid, J.; Woodham, L.; Donkers, J.; Kononowicz, A.A. Enabling interoperability, accessibility and reusability of virtual patients across Europe—design and implementation. Stud. Health Technol. informatics 2009, 150, 826–830. [Google Scholar]
  31. Kolb, S.; Reichert, J.; Hege, I.; Praml, G.; Bellido, M.C.; Martinez-Jaretta, B.; Fischer, M.; Nowak, D.; Radon, K.; The NetWoRM Group. European dissemination of a web- and case-based learning system for occupational medicine: NetWoRM Europe. Int. Arch. Occup. Environ. Health 2007, 80, 553–557. [Google Scholar] [CrossRef]
  32. International Collection of Virtual Patients. Available online: (accessed on 21 March 2022).
  33. National Library of Medicine. Medical Subject Headings (MeSH). Available online: (accessed on 21 March 2022).
  34. German Medical Faculty Convention. Nationaler Kompetenzbasierter Lernzielkatalog Medizin 2.0. Available online: (accessed on 21 March 2022).
  35. Polish Ministry of Science and Higher Education. 2019. Available online: (accessed on 21 March 2022).
  36. Portuguese National Health Service. 2019. Available online: (accessed on 21 March 2022).
  37. Spanish National Agency for Quality and Accreditation (ANECA). Libro Blanco—Título en Grado en Medicina. 2005. Available online: (accessed on 21 March 2022).
  38. French Ministry of Higher Education, Research and Innovation. Competency Framework for the Second Cycle of Medical Education; French Ministry of Higher Education, Research and Innovation: Paris, France, 2020. [Google Scholar]
  39. VINDICATE. The Free Dictionary. Available online: (accessed on 21 March 2022).
  40. Sabin, J.A.; Marini, M.; Nosek, B.A. Implicit and Explicit Anti-Fat Bias among a Large Sample of Medical Doctors by BMI, Race/Ethnicity and Gender. PLoS ONE 2012, 7, e48448. [Google Scholar] [CrossRef] [Green Version]
  41. Hall, W.J.; Chapman, M.V.; Lee, K.M.; Merino, Y.M.; Thomas, T.W.; Payne, B.K.; Eng, E.; Day, S.H.; Coyne-Beasley, T. Implicit Racial/Ethnic Bias Among Health Care Professionals and Its Influence on Health Care Outcomes: A Systematic Review. Am. J. Public Health 2015, 105, e60–e76. [Google Scholar] [CrossRef]
  42. Croskerry, P. The Importance of Cognitive Errors in Diagnosis and Strategies to Minimize Them. Acad. Med. 2003, 78, 775–780. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  43. Chernikova, O.; Heitzmann, N.; Stadler, M.; Holzberger, D.; Seidel, T.; Fischer, F. Simulation-Based Learning in Higher Education: A Meta-Analysis. Rev. Educ. Res. 2020, 90, 499–541. [Google Scholar] [CrossRef]
  44. Stern, S.D.C.; Cifu, A.S.; Altkorn, D. Symptom to Diagnosis: An Evidence-Based Guide, 4th ed.; McGraw-Hill Education LLC: New York, NY, USA, 2020. [Google Scholar]
  45. Porter, R.S. The Merck Manual of Patient Symptoms: A Concise, Practical Guide to Etiology, Evaluation, and Treatment, 1st ed.; Merck: Kenilworth, NJ, USA, 2008. [Google Scholar]
  46. Finley, C.R.; Chan, D.S.; Garrison, S.; Korownyk, C.; Kolber, M.R.; Campbell, S.; Eurich, D.T.; Lindblad, A.J.; Vandermeer, B.; Allan, G.M. What are the most common conditions in primary care? Systematic review. Can. Fam. Physician 2018, 64, 832–840. [Google Scholar] [PubMed]
  47. Rolfe, I.; Pearson, S.-A.; Sanson-Fisher, R.; Ringland, C.; Bayley, S.; Hart, A.; Kelly, S. Which common clinical conditions should medical students be able to manage by graduation? A perspective from Australian interns. Med. Teach. 2002, 24, 16–22. [Google Scholar] [CrossRef]
  48. Schiff, G.D.; Hasan, O.; Kim, S.; Abrams, R.; Cosby, K.; Lambert, B.L.; Elstein, A.S.; Hasler, S.; Kabongo, M.L.; Krosnjar, N.; et al. Diagnostic Error in Medicine: Analysis of 583 physician-reported errors. Arch. Intern. Med. 2009, 169, 1881–1887. [Google Scholar] [CrossRef]
  49. Singh, H.; Giardina, T.D.; Meyer, A.N.; Forjuoh, S.N.; Reis, M.D.; Thomas, E.J. Types and Origins of Diagnostic Errors in Primary Care Settings. JAMA Intern. Med. 2013, 173, 418–425. [Google Scholar] [CrossRef] [Green Version]
  50. Statista. European Union: Age Distribution of Inhabitants from 2010 to 2020. Available online: (accessed on 23 March 2022).
  51. Statista. Estimated Population of Europe from 1950 to 2021, by Gender. Available online: (accessed on 21 March 2022).
  52. Goodman, M.; Adams, N.; Corneil, T.; Kreukels, B.; Motmans, J.; Coleman, E. Size and Distribution of Transgender and Gender Nonconforming Populations: A Narrative Review. Endocrinol. Metab. Clin. North Am. 2019, 48, 303–321. [Google Scholar] [CrossRef]
  53. Statista. Europe’s LGBT Population Mapped. 2016. Available online: (accessed on 21 March 2022).
  54. Williams, R.L.; Romney, C.; Kano, M.; Wright, R.; Skipper, B.; Getrich, C.M.; Sussman, A.L.; Zyzanski, S.J. Racial, Gender, and Socioeconomic Status Bias in Senior Medical Student Clinical Decision-Making: A National Survey. J. Gen. Intern. Med. 2015, 30, 758–767. [Google Scholar] [CrossRef] [Green Version]
  55. CEDEFOP. Employed Population by Sector and Occupation. 2019. Available online:§or=#1 (accessed on 21 March 2022).
  56. EUROSTAT. Unemployment Statistics. Available online: (accessed on 21 March 2022).
  57. WHO. Facts on Disability. Available online: (accessed on 21 March 2022).
  58. European Commission (EU). Statistics on Migration to Europe. 2020. Available online: (accessed on 21 March 2022).
  59. EUROSTAT. Tobacco Consumption Statistics. 2019. Available online: (accessed on 21 March 2022).
  60. EUROSTAT. Alcohol Consumption Statistics. 2019. Available online: (accessed on 21 March 2022).
  61. EMCDDA. Statistical Bulletin. 2021. Available online: (accessed on 21 March 2022).
  62. Humphrey-Murto, S.; Varpio, L.; Gonsalves, C.; Wood, T.J. Using consensus group methods such as Delphi and Nominal Group in medical education research. Med. Teach. 2017, 39, 14–19. [Google Scholar] [CrossRef]
  63. Keegan, D.A.; Scott, I.; Sylvester, M.; Tan, A.; Horrey, K.; Weston, W.W. Shared Canadian Curriculum in Family Medicine (SHARC-FM): Creating a national consensus on relevant and practical training for medical students. Can. Fam. Physician 2017, 63, e223–e231. [Google Scholar]
  64. Urushibara-Miyachi, Y.; Kikukawa, M.; Ikusaka, M.; Otaki, J.; Nishigori, H. Lists of potential diagnoses that final-year medical students need to consider: A modified Delphi study. BMC Med. Educ. 2021, 21, 234. [Google Scholar] [CrossRef] [PubMed]
  65. ICoViP Project: Blueprint for 200 Virtual Patients. Available online: Link (accessed on 21 March 2022).
  66. Tworek, J.; Paget, M.; McLaughlin, K.; Wright, B. How Dungeons & Dragons Made Us Better VPs: Randomizing Physiological Data to Rapidly Produce 97 Clinically Realistic VPs. Bio. Algorithms. Med. Syst. 2010, 6, 41–45. [Google Scholar]
  67. Huang, G.; Reynolds, R.; Candler, C. Virtual Patient Simulation at U.S. and Canadian Medical Schools. Acad. Med. 2007, 82, 446–451. [Google Scholar] [CrossRef] [PubMed]
  68. Fors, U.G.H.; Muntean, V.; Botezatu, M.; Zary, N. Cross-cultural use and development of virtual patients. Med. Teach. 2009, 31, 732–738. [Google Scholar] [CrossRef]
  69. Walldorf, J.; Jähnert, T.; Berman, N.B.; Fischer, M.R.; Edelbring, S.; Davies, D. Using Foreign Virtual Patients With Medical Students in Germany: Are Cultural Differences Evident and Do They Impede Learning? J. Med. Internet Res. 2016, 18, e260. [Google Scholar] [CrossRef]
  70. Balzer, F.; Hautz, W.; Spies, C.; Bietenbeck, A.; Dittmar, M.; Sugiharto, F.; Lehmann, L.; Eisenmann, D.; Bubser, F.; Stieg, M.; et al. Development and alignment of undergraduate medical curricula in a web-based, dynamic Learning Opportunities, Objectives and Outcome Platform (LOOOP). Med. Teach. 2015, 38, 369–377. [Google Scholar] [CrossRef]
  71. Fritze, O.; Lammerding-Koeppel, M.; Boeker, M.; Narciss, E.; Wosnik, A.; Zipfel, S.; Griewatz, J.; Fritze, O.; Lammerding-Koeppel, M.; Boeker, M.; et al. Boosting competence-orientation in undergraduate medical education—A web-based tool linking curricular mapping and visual analytics. Med. Teach. 2018, 41, 422–432. [Google Scholar] [CrossRef]
  72. Lammerding-Koeppel, M.; Giesler, M.; Gornostayeva, M.; Narciss, E.; Wosnik, A.; Zipfel, S.; Griewatz, J.; Fritze, O. Monitoring and analysis of the change process in curriculum mapping compared to the National Competency-based Learning Objective Catalogue for Undergraduate Medical Education (NKLM) at four medical faculties. Part I: Conducive resources and structures. GMS J. Med. Educ. 2017, 34, Doc7. [Google Scholar] [CrossRef]
  73. Kassirer, J.P. Teaching Clinical Reasoning: Case-Based and Coached. Acad. Med. 2010, 85, 1118–1124. [Google Scholar] [CrossRef]
  74. Khin-Htun, S.; Kushairi, A. Twelve Tips for Developing Clinical Reasoning Skills in the Pre-Clinical and Clinical Stages of Medical School. Med. Teach. 2018, 41, 1007–1011. [Google Scholar] [CrossRef]
  75. Eva, K.W. Diagnostic error in medical education: Where wrongs can make rights. Adv. Health Sci. Educ. 2009, 14 (Suppl. S1), 71–81. [Google Scholar] [CrossRef] [PubMed]
  76. Kopp, V.; Stark, R.; Fischer, M.R. Fostering diagnostic knowledge through computer-supported, case-based worked examples: Effects of erroneous examples and feedback. Med. Educ. 2008, 42, 823–829. [Google Scholar] [CrossRef] [PubMed]
Figure 1. Flowchart of our four-step approach to create the blueprint. (VP = Virtual patient).
Figure 1. Flowchart of our four-step approach to create the blueprint. (VP = Virtual patient).
Ijerph 19 06175 g001
Table 1. Overview of criteria, permitted values, and sources for comparison.
Table 1. Overview of criteria, permitted values, and sources for comparison.
CriterionValuesSources for Comparison
Key symptomsN/A[34,35,36,37,38,44,45]
Final diagnosesN/A[46,47,48,49]
Disease groupVascular/Infectious/Neoplastic/Drugs, Toxic/Idiopathic/Congenital/Autoimmune, Immunologic/Traumatic/Endocrine, Metabolic [39]N/A
Scenario closureLong-term treatment/Successfully discharged/Died [26]N/A
Sex/GenderFemale/Male/Transgender/Intersexual [24][51,52]
Sexual orientationHeterosexual/Homosexual/Bisexual/Not sexually active (child)/Not stated [24][53]
EthnicityHispanic/Black/White/Asian/Other [54]N/A
ProfessionNaming of profession/Unemployed/Retired/Student/Child/Not stated[55,56]
Relevant cultural, language, or migration backgroundYes/No [58]
Addiction/substance abuseSmoker/Ex-Smoker/Alcohol/Illegal drugs/Other/No/Not Stated [25][59,60,61]
Learner’s roleStudent/Intern/Resident/Consultant/Other/Not StatedN/A
SettingRural Hospital/University Hospital/Hospital/Practice/Outpatient Clinic/Emergency Room [25]N/A
N/A = Not applicable.
Table 2. Summary of patient-related criteria and their respective data for comparison.
Table 2. Summary of patient-related criteria and their respective data for comparison.
CriterionValuesN% TotalEU/European Data for ComparisonReference
Age0–14 years1911.5%15.1%[50]
15–64 years12261.0%64.1%
≥65 years5527.5%20.8%
Male 9748.5%48.3%
Transgender10.5%Estimated proportion:
Intersexual 10.5%
Sexual orientationHeterosexual11557.5%N/A
Homosexual 126.0%Proportion of LGBT:
Bisexual 42.0%
Not sexually active (child)2110.5%N/A
Not stated4823.5%N/A
Relevant cultural, language, or migration backgroundYes2713.5%5.1% non–EU citizens 8.3% born outside EU[58]
Addiction/Substance abuse *Smoker3016.6% 218.4% 1 daily smoking[59]
Ex–Smoker116.1% 2N/A
Alcohol147.7% 28.4% 1 daily alcohol[60]
Illegal drugs32.4% 2“Last month prevalence”:
1.2–8.7% 2
Other10.6% 2N/A
Not Stated3517.5%N/A
* Numbers do not sum up to 200 (100%) since combinations of values are possible. 1: Referring to age group ≥15 years. 2: Referring to age group 15–64 years. N/A = Not applicable.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Mayer, A.; Da Silva Domingues, V.; Hege, I.; Kononowicz, A.A.; Larrosa, M.; Martínez-Jarreta, B.; Rodriguez-Molina, D.; Sousa-Pinto, B.; Sudacka, M.; Morin, L. Planning a Collection of Virtual Patients to Train Clinical Reasoning: A Blueprint Representative of the European Population. Int. J. Environ. Res. Public Health 2022, 19, 6175.

AMA Style

Mayer A, Da Silva Domingues V, Hege I, Kononowicz AA, Larrosa M, Martínez-Jarreta B, Rodriguez-Molina D, Sousa-Pinto B, Sudacka M, Morin L. Planning a Collection of Virtual Patients to Train Clinical Reasoning: A Blueprint Representative of the European Population. International Journal of Environmental Research and Public Health. 2022; 19(10):6175.

Chicago/Turabian Style

Mayer, Anja, Vital Da Silva Domingues, Inga Hege, Andrzej A. Kononowicz, Marcos Larrosa, Begoña Martínez-Jarreta, Daloha Rodriguez-Molina, Bernardo Sousa-Pinto, Małgorzata Sudacka, and Luc Morin. 2022. "Planning a Collection of Virtual Patients to Train Clinical Reasoning: A Blueprint Representative of the European Population" International Journal of Environmental Research and Public Health 19, no. 10: 6175.

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop