What Do Clinicians Mean by “Good Clinical Judgment”: A Qualitative Study

: Good Clinical Judgment (GCJ) is associated with clinical excellence and accolades whereas poor clinical judgment is often associated with suboptimal care and the need for remediation. Although commonly referenced in practice, a shared deﬁnition for GCJ based on primary data is lacking. We interviewed 16 clinicians and surgeons across different specialties at one Canadian academic center to understand their conceptualization of GCJ. The data analysis led to the formulation of three pillars that were viewed by participants as core ingredients of GCJ. These included (1) a strong baseline knowledge and breadth of clinical experience, (2) the demonstration of curiosity, reﬂection, and wisdom, and (3) an ability to attend to contextual factors and understand the “bigger picture” when providing care to patients. Although there were inconsistent opinions regarding whether GCJ is innate or learned, participants reﬂected on strategies to support the development or improvement in clinical judgement for trainees.


Introduction
The process of making clinical decisions (i.e., decisions which relate to the observation or treatment of patients) is central to the practice of medicine [1][2][3]. Trust bestowed by colleagues, peers, and supervisors is often based on whether they make decisions that demonstrate "Good Clinical Judgment" (GCJ). Redelmeir states: "Good Clinical Judgment has been prized since the time of Hippocrates and will continue to be important for many years to come" [3]. However, it remains unclear what clinicians mean in day-to-day practice when they say that someone has GCJ, and the elements that constitute this remain largely ill-defined [4].
While there is overlap between clinical judgment and clinical reasoning, the terms are distinct from each other. Clinical reasoning is a vast and broad term used predominantly by educational scholars in research contexts in reference to the cognitive psychological processes used to make clinical decisions in empirical experiments with well-defined gold standards and stringently controlled variables [5][6][7]. In contrast, "Clinical Judgment" is the every-day term used by clinicians to describe decision making in the "real world" [1,[8][9][10], particularly when the problems are ambiguous ("missing data, conflicting information, limited time and long term trade-offs") [3] or complex (multiple outcomes of interest, multiple solution pathways, and unpredictable interaction between input variables) [11]. GCJ is frequently expressed as a physician ideal, suggesting shared meaning within this community [3]. Furthermore, learners and physicians who struggle clinically or succumb to malpractice are identified as lacking "Judgment" rather than having "Poor Clinical Reasoning" [12][13][14]. Although often referenced, "Clinical Judgement" remains poorly understood [4]. Consequently, understanding the meaning of GCJ is of paramount importance to clinicians, trainees, and educators.
Given that GCJ is often seen as a mark of excellence in the medical profession, we need an operational definition of GCJ. While we acknowledge there will be overlap between the conceptualization of clinical reasoning and clinical judgment, particularly with regards to decision making amidst uncertainty [15,16], this study did not specifically seek to differentiate those concepts. However, reaching a shared understanding of "Good Clinical Judgment" amongst practicing clinicians could serve to facilitate the coaching of trainees, provide potential anchors for remediation, and delineate markers of clinical excellence.
Thus, in this study, the authors sought to understand how GCJ is conceptualized by clinicians in day-to-day practice and use this to develop a composite definition.

Qualitative Approach and Research Paradigm
In this qualitative study framed in a post-positivism paradigm, we used an approach informed by constructivist grounded theory [17] to analyze transcripts from semistructured interviews to explore what clinicians regarded as conceptual ingredients of GCJ. A grounded theory approach was needed to build a definition representative of what the clinical community shares, where a definition synthesized from pre-existing primary data was lacking.

Researcher Characteristics
Prior to embarking on analysis, the researchers (MT, MS, LM, SB, DM, ZA) conducted informal discussion and declaration of pre-existing assumptions about clinical judgment. All researchers are practicing clinicians: LM and SG practice as general internists, and the other study team members practice as cardiologists. DM and ZA are postgraduate trainees. MT, MS, LM, and SB all have medical education training and practice as medical educators.

Setting
The study was conducted with medical and surgical specialists working on high clinical volume units at three academic teaching hospitals and one community hospital in the Canadian context. There are both undergraduate and postgraduate clinical trainees at these sites.

Sampling Strategy
We used purposive sampling to identify clinicians from a broad range of medical practices within a single Canadian Academic University who were regarded as leading clinician educators. Additional interviewees were identified by snowball sampling, asking those interviewed for additional recommendations of clinician educators they felt could contribute to the conceptualization of GCJ. We continued to iteratively sample and analyze until no new theoretical constructs could be found.

Data Collection, Processing, and Analysis Methods
Interviews were semi-structured (see Appendix A for guide), with MT serving as the primary interviewer. MT, as junior faculty, was not expected to have an influence on the content of the interviews. Interview data were recorded, transcribed verbatim, and de-identified prior to analysis. Each member of the analysis team independently read each transcript engaging in journaling and thematic coding, meeting each series of 2-3 interviews to discuss and reach consensus. Analysis informed subsequent cycles of interviews, which focused on clarifying, elaborating, and exploring previously identified themes. Themes were constantly compared and triangulated with the data as they were iteratively collected, with purposeful enrichment of the sample for diverse and divergent viewpoints. The analytic team sought to develop a coherent interpretation of each theme, and draw meaningful relationships between identified themes, in order to construct a model for learning, developing, and exercising GCJ.

Qualitative Rigor
Rigor was embedded in the careful attention to verbatim transcription, iterative sampling and analysis, the identification of each analytic team members' lens, reflective journaling by each team member, and member checking. Member checking consisted of emailing a summary of themes and the conceptual model to all interviewees asking for their opinion about its veracity and representativeness.

Ethics
This research was conducted at McMaster University and approved by the Hamilton Integrated Research Ethics Board (#3665). All participants provided written informed consent.

Interviewee Characteristics
A total of 16 interviews were conducted representing clinician educators in different stages of their career, across a diversity of different specialties as outlined in Table 1.

Identified Themes
Universally, clinical judgment was viewed as central to medical decision making. In a broad sense, it is viewed as a decision itself or the process of arriving at the decision. However, it was also viewed as a highly coveted quality that a clinician or trainee could arrive at or possess having "Good Clinical Judgment": "it's a currency if you're a doctor you have good clinical judgement. You make good decisions and have good outcomes for your patients" (P1).
When seeking to describe GCJ, three pillars emerged. It was felt that GCJ required (1) baseline knowledge and experience, (2) the demonstration of curiosity, reflection, and wisdom, and (3) an ability to attend to contextual factors to comprehend the "bigger picture" of what is important for the patient.

Baseline Knowledge and Adequate Clinical Experience
Participants clearly identified that in order to exercise GCJ, physicians must first have adequate clinical experience to recognize deviations from the expected clinical course and nuanced complexity.
"I think it actually begins with knowledge. A person has to be knowledgeable, and you have to actually understand as well as you can, physiology and pharmacology. So, it really begins with knowledge. Ah, they have to have the clinical exposure, because . . . Things don't present in classical sense all the time. They have to be on their own, so they can make mistakes, recognize them, and correct them because, that contributes to the overall experience." (P4) That said, it was clear that participants felt content knowledge alone was inadequate to achieve GCJ: "I've seen people who, they know lots of stuff and they know all the guidelines and they know all the algorithms, and they know all the rules, and if they don't, they can look it up in two in a half, seconds. But they don't have the skills, they can't bring to bear some of the other factors." (P14)

Curiosity, Reflection, and Wisdom
An ability to apply "curiosity and wisdom" in attending to clinical cases, no matter how long you have been in practice, was considered key to GCJ. This was felt to help avoid premature closure, or the risk of missing subtle cues to suggest the clinician should revisit their approach: "the mind-set that you have done it before thousands of times, but you almost feel like you are doing it for the first time again. And that is a difficult thing to do" (P2). The importance of curiosity as a trait was also thought to drive how a clinician would arrive at diagnoses or astute case evaluation: "For the cases they were initially stumped, what I found was a key feature was curiosity. And a sense of the current conceptualization of the diagnosis was insufficient or incorrect [ . . . ] curiosity drove them to find more and more information in support of the diagnosis." (P10) It was felt that individuals who approached mistakes as learning opportunities were more likely to engage in reflection and proceed to refine GCJ. Some felt that this growth mindset may need to be triggered by a challenging experience: "All of us [ . . . ] absolutely need an experience where your hundred percent crash and burn and feel horrible about something you have done . . . if you don't, I don't think you can learn or grow as a physician" (P6). Those experiences are felt to be powerful particularly when coupled with reflection: "good clinicians, I think are reflective of their own work. I wonder if I should have done this. I wondered if this could of, gone if that, if that would of, been better handled in this way?" (P1). It is important this reflection be balanced, such that individuals learn reactions that trigger them to "distance themselves from the problem, from the situation" (P2).
In addition to being clinically curious, individuals with GCJ were identified as having an ability to deal with, and acknowledge, uncertainty. "This particular clinician was able to make very big decisions . . . while understanding that there is perhaps uncertainty in the case . . . she was able to give the impression that she was acknowledging all that while at the same time proceeding forward and making decisions." (P1)

Attending to Contextual Factors and the "Bigger Picture"
Context was felt to be essential to facilitate focus on the salient nuanced elements in the case (to separate the signals from the noise), to recognize complexity and to provide care that is tailored to the individualized patient, appropriate for the time and setting: "Well trying to get them to think that this is a story, that needs to fit. It can't be out of context" (P9). Context could relate to the urgency of a decision, to the patient's goals of care, or to the social environment.
"Like here was a particular professionalism issue . . . or a communication issue that was more nuanced because, you actually had . . . an angry family. Or there was splitting in the team dynamic." (P8) Ultimately, pulling the contextual factors to formulate a "bigger picture" was felt to result in GCJ. The major subcomponents that facilitated the ability to consider the "big picture" include prioritization, flexibility, and knowing when to break the rules.

(a) Prioritization
Prioritization of clinical data and patient values is a task illustrated by many respondents as a key mark of GCJ: "Maybe part of the challenge and what constitutes expertise is knowing when to apply principle. It's almost a degree of emphasis that comes with experience but other skills as well. So, we can always say in an abstract way you want to do no harm. Or you want to respect the patient's wishes. Or you want to apply best evidence. Like there's all of these principles that one could say are applicable, in almost every case in the assessment of good clinical judgement. However, it's the notion of, can you successfully prioritize these things?" (P8) (b) Flexibility "Good Clinical Judgment" requires the clinician to adapt to changing contextual cues, such that they can redirect themselves and demonstrate flexibility in the thought process: "So . . . people [with] really good [clinical] judgement is able to . . . redirect themselves based on information that they get on real time . . . . Thoughtful, retrospective, flexible" (P3).
As part of this, clinicians identified the importance of recognizing when your diagnosis or plan no longer fits with the data: "there is something about that non-analytical process of a clinician looking at the picture and saying, 'this does not make sense'" (P10). That requires a readiness and acceptance that you will not always be right. It allows clinicians to "move forward with hypothesis that they're quite ready to have disproved" (P10).
(c) Knowing when to "break the rules" Participants frequently identified that in many clinical contexts, guidelines, algorithms, and evidence may not apply, and GCJ requires the clinician to think and act outside of these boundaries.
"They are able to tell me, 'Yes I know what the guidelines say, but in this patient, for this reason, I'm doing this.' And if they can give me a good reason, why they did something differently, I kind of get comfortable in their judgement because they're telling me, 'I know the evidence.'" (P3) In contrast, purely algorithmic decisions without consideration of context could be viewed as lacking rationale and related to poor judgment.
One physician hypothesizes that comfort acting outside expected standards is related to experience with making decisions within ambiguous situations (P15): "When you're younger, you, I find that you really think these are the rules and we have to pigeonhole people and then you get older, you get, I guess, I don't know, more comfortable with fuzzy borders".

Good Clinical Judgment: Innate, Learned, or Both?
There was some disagreement amongst study participants regarding whether GCJ was a character trait versus something that could be learned. The above characteristics that were felt important to develop GCJ were viewed as an intrinsic or innate "quality," for example, one's natural curiosity, observational skills, or desire to engage in self-reflection: "some people can observe and attend to more things and some people get focused and fixed and don't attend to the intended features instead of the big picture" (P9). It was shared by another participant that although much can be learned, there is something different overall about those clinicians with very strong clinical judgment: "I do think medicine in general is a type of job that does have somewhat of a calling to it, you do have, there is something inborn or innate to it about this job that you can't learn or can't be taught" (P6).
Others had personal experiences that they felt illustrated to them that GCJ could be improved upon: "I used to think that decision making was difficult to learn and difficult to teach . . . I thought people innately had it or they didn't . . . it's like common sense . . . but we have this trainee right now who's clinical decision making was relatively poor six months ago and he changed my mind . . . he's worked hard to build those building blocks of knowledge and experience; he has learned to listen to patients . . . to actually being a reasonable decision maker." (P4) While there was some controversy about whether this could be taught amongst those interviewed, practical recommendations came forward from the discussion that can be applied by clinician educators.
(a) Break down where the judgment is "going wrong" What may be viewed as an overall lack of GCJ could be caused by the lack of a particular ingredient" described above, such as baseline knowledge, clinical experience, understanding of contextual factors. In these cases, it was viewed as relatively simple to assist trainees in addressing these gaps.
Alternatively, participants brought forward examples where an inability to prioritize was actually related to an underlying diagnosis of anxiety: "Sometimes, what's perceived as a judgement is, for example, anxiety. People with anxiety disorders will often not be able to close their history taking . . . they just keep gathering, and gathering because, they are so afraid that they're going to miss something . . . if we are going to teach learners the first thing to be able to do, is figure out what their learning needs are and if this person has, doesn't have good clinical judgement that's a very sweeping term. We better figure out what it may possibly be." (P14) (b) Opportunities for modelling Participants shared that once they understood where trainees were struggling regarding GCJ, they would use this information to develop educational plans. This could include role modeling your own thought process in the setting of uncertainty: "I think the idea that sometimes cases are unclear and being upfront about your rationale with trainees about why you are doing what you are doing and acknowledging the fact that you might be wrong and that its okay to be wrong. But you have to have a justification for what you're, why you are doing it and then re-evaluation of that as new data becomes available." (P1) In other circumstances, participants spoke about the opportunity to use simulation to promote flexibility: "I think we have the opportunity now to bring in simulation-based teaching tools that we can throw twists, and you know, force them to think outside the box more and more often" (P5).
Finally, some participants spoke of using opportunities where there is discrepancy between the learner's decision and their own as key learning moments: "that's a very powerful learning moment [ . . . ] let's unpack the assumptions that led to that divergence in thinking" (P10).

Discussion
This qualitative study explored the conceptualization of "Good Clinical Judgment" from the perspective of clinician educators at one Canadian Academic University. Analysis revealed three key pillars, which included (1) a strong knowledge base and clinical experience, (2) personal characteristics including curiosity, self-reflection, and wisdom, and (3) an ability to attend to contextual factors in order to understand the bigger picture. This last element requires clinicians to be able to prioritize, be flexible, and understand "when to break the rules". In addition, participants provided their perspective on whether GCJ is innate or learned. The heterogeneity in responses from a single institution suggests there is ongoing differences in opinion amongst the medical education community. Some participants felt that certain individuals possess the attributes that trigger reflection required for GCJ (i.e., growth mindset and curiosity), while others do not; these differences may drive some of the challenges seen in developing GCJ. Yet, some elements of GCJ (i.e., how the trainee interacts with the situational context through better data collection: "listening to patients") were certainly aspects that some clinicians felt could be improved upon. Participants identified the importance of first understanding where judgment has "gone wrong" in order to target interventions, and the role of modeling in order to address concerns with GCJ.
The findings of this study have several theoretical implications. While clinical judgment is commonly referred to in clinical practice, to our knowledge a composite empirical definition is not well established in the literature, and there remains confusion between terms such as clinical reasoning and clinical judgement [4]. This has important assessment implications, as GCJ is frequently referenced in assessments and feedback, without an operational definition, which would facilitate a more in depth conversation between trainee and supervisor [4]. This study's empirically derived definition provides additional structure for these feedback and assessment conversations.
The concept of GCJ has links to multiple pre-existing theoretical frameworks. As has been previously established, the evaluation, diagnosis, and management decisions made by clinicians are often made utilizing knowledge structures that are encapsulated within illness scripts [18,19]. Accessing and substantiating a script requires context, which aligns with the central importance of contextual information for GCJ. From a GCJ perspective, context is also used as a key trigger to define the clinical problem and lead to subsequent sensible prioritization. Problem definition helps clinicians navigate situations where uncertainty limits the use of available scripts [11,[20][21][22], and when additional practical considerations about the patient, their social environment, and the geographical setting call for a nuanced approach to the illness state [23]. The use of contextual information to decide when the rules of guidelines, algorithms, and scientific evidence do not apply to the individual patient has been recognized experimentally amongst expert emergency medicine physicians [24].
GCJ also relates to the importance of slowing down in complex clinical situations [25][26][27][28][29][30]. This type of dynamic learning and adaptation has been referred to by Schon [31] as "Reflection in Action," and requires the individual to recognize contextual cues to identify that a particular situation does not "fit" an existing pattern known to that individual, reframing the problem, and creating a new understanding of the problem.
It comes without surprise that there are strong parallels that can be drawn between the pillars of GCJ in this study and the "Ingredients of Rich Entrustment Decisions" as defined by ten Cate and Chen [32]. Rich entrustment in their model was defined as capability, agency, reliability, integrity, and humility. Capability speaks to the importance of experience and a strong knowledge base, while curiosity, self-reflection, and wisdom mirror the ingredients agency and humility. The flexibility to prioritize and attend to contextual factors when caring for patients similarly mirror the trust ingredients of integrity and capability. While reliability is key in trust, it was less of a priority to participants in this study when conceptualizing GCJ; this may represent that clinicians focus their conceptualization of GCJ more on the thought process and decisions reached by medical professionals rather than their ability to carry out tasks entrusted to them.
While inferring the mechanisms by which GCJ develops is limited to the perceptions of participants, "Situativity theory", may be relevant to the contextual nature of GCJ. Situativity theory argues that much of practical learning occurs with the application of knowledge within a setting and interactions with other participants of that setting to give practical meaning to that knowledge [33][34][35]. Within this theoretical paradigm, the development of GCJ would depend on a strong foundational knowledge framed within a social environment, its participants, and contextual experiences. Trainees, supervisors, and educators should consider the role of contextual experiences in addition to knowledge base and personal characteristics when citing GCJ as part of feedback or assessment processes.

Limitations of Study
This study sought the views of those at a single academic institution within the Canadian context. The snowball sampling method allowed researchers to recruit established clinician educators in the field; however, it led to an unequal representation of participants across career stage, sex, and medical or surgical specialities. The extent to which the described conceptualization is shared more broadly, both nationally and internationally, requires investigation in future studies.

Summary and Future Directions
In summary, GCJ requires the interaction of three key pillars, which include baseline knowledge, clinical curiosity and wisdom, and an ability to attend to contextual factors in understanding the big picture. The acquisition and practice of GCJ is an iterative process, demanding the clinician to shape and reshape their interpretation of a clinical problem based on reflection and interaction with others in order to develop the ability to prioritize, be flexible, and know when to "break the rules." This study highlights strategies to support trainees in the development of GCJ. By identifying key ingredients of GCJ, clinician educators can seek to understand where judgment has "gone wrong" and use this information to develop educational plans. As noted above, future studies are required to understand the extent to which this conceptualization is shared by clinician educators in other settings, as well as the impact of this model on supporting the development of GCJ for learners. Informed Consent Statement: Informed consent was obtained from all subjects involved in the study.

Data Availability Statement:
The data presented in this study are available on reasonable request from the corresponding author.

Conflicts of Interest:
The authors declare no conflict of interest. Can you give an example of an instance when a trainee has had challenges with their judgement or decision making?

Appendix A
What does good clinical judgment mean to you?

Question Notes
(Show interviewee the following) The Association of Faculties of Medicine of Canada (AFMC) defines the Entrustable Professional Activities (EPA) or management decisions as "purposes evidence informed holistic management plans that include pharmacologic and non-pharmacologic components developed with an understanding of the patients' context, values and illness experience, considers priorities and other health care professions advice" What do you think of this EPA? Are there elements that you could identify good judgment with Are there environmental factors that you think may affect trainee clinical judgement?
What elements do you think are essential to clinical judgement? Do you think good judgment is about knowledge or is it an innate quality? If it is the latter, can it be learned and how can we teach it?
If a management decision is assessed to have deficits in judgement, can you usually identify one or two contributing issues for feedback?
How does your assessment of a trainee's clinical judgement affect your supervision of that trainee?
If you were to provide descriptors to an instance of poor, satisfactory and exemplary clinical judgment, what would they be