Harness a Simple Design to Make Authentic Learning Moments Visible: A Design-Based Research Study in Clinical Reasoning

Reviewer comments

Author responses

Reviewer 1

Title change suggestion:

Harnessing Simple Design to Make Authentic Learning Moments Visible: A Design-Based Research Study in Clinical Reasoning

The authors have agreed this is an improved title and have made this change.

Your central idea that a deliberately simple scaffold can create a “critical pause” where students externalize their reasoning before entering social/AI dialogue

Thank you. The authors agree as the fundamental central idea, this has been clearly articulated now throughout the manuscript.

Your study predates genAI uptake (as you note). Please frame genAI as context/implication throughout, including in the abstract, and resist any implication that genAI outcomes were measured.

We appreciate this feedback. We have decided to keep mention of the current use of genAI in HE in the Introduction and have returned to link to the key concept of creating a deliberate and simple scaffold to create a critical pause for students to externalise reasoning. We have clarified throughout the article that the contribution of the DP3: Harness a simple design, generated from the DBR study on enhancing clinical reasoning, has potential to be further explored in the context of social dialogue with genAI. We have clarified this is a conceptual implication only at this stage and not yet measured.

Add a DBR cycle timeline (what changed each cycle), enrich the RTA description (code development, reflexivity, coder roles), and provide an appendix with prompts, exemplar wheels, and analytic excerpts.

An appendix has been added demonstrating the cycle timeline and stages of DBR/RTA coding and analysis for the original study.

A timeline of how DPs evolved during the DBR cycles is in the results section (Table 4, pp.11).

The role of the researcher has been clarified using the DBR and RTA combination in the methods section (pp. 8).

If readers wish to see more exemplar wheels and analytical excerpts this can be provided on request. The purpose of this paper is not to discuss or illustrate the depth that was taken for the analysis of the project.

Operationalize “psychological authenticity” with observable indicators and show how your data support each headline claim (e.g., equity of voice, evaluative judgement, academic-integrity risk). A brief limitations paragraph will also help calibrate generalizations.

One or two lines per DP will show how “Harness a simple design” fits the broader design knowledge you generated.

Operational indicators for psychological authenticity aligned with the data set of this study are now included explicitly in the results section in Table 3 (pp. 11).

A limitations paragraph has been added after the conclusion section of this paper (pp.14).

Sections

Reviewer suggestions

Author comments

Abstract

·       Add an operational sentence to define ‘authentic communication’.

·       That genAI is a motivator and not a studied variable.

·       Add how many participants contributed to each data source, were there repeated measures.

·       Either remove mention of academic integrity risk

Thank you for these recommendations. These have all be actioned and included in the draft manuscript.

Reference to claims around academic integrity risk has been removed from the abstract, as this is not the focus of the article.

Key words

Remove numbering

Removed

Introduction

Add a sentence how authentic communication relates to clinical reasoning.

Information on authentic communication in relation to clinical reasoning has been included.  

‘the imperative to understand if decision making by students is ‘authentic’ when collaborating, is now broadly interrogated at a time when learning with generative artificial intelligence (genAI) has accelerated in education’

Tighten the logic chain:

(a) authenticity in collaborative decision making;

(b) genAI challenges evaluation of individual contribution;

(c) your solution (decision wheel) makes contributions visible

Logic has been made explicit.

Disclaimer is now shortened and added into the introduction section instead of end of methods.

‘Authentic learning is generally associated with students having ‘real-world’ experiences preparing for industry (Chambers & Broadbent, 2024; Fawns, et al., 2024)’

Nicely sets up “authenticity in assessment.” Cite Ajjawi et al. directly here and distinguish task authenticity vs evidence of authentic thinking.

‘For instance, evidencing if ‘psychological authenticity’ has occurred requires a holistic learning design to inspire students to make conscious and deliberate decisions (Chambers & Broadbent, 2024).

Provide a definition map (authentic learning; authenticity in assessment; psychological authenticity) with 1–2 anchor citations each

Additional information has been included in the introduction to frame how learning design is important to an authentic learning ‘perceived experience’ for clinical education. The authors have decided not to add a definition map into this section of the article as suggested, to avoid moving into discussion on authentic assessment, which article does not focus on explicitly.

‘This DBR project now contributes six final design principles (DPs) to enhance learning clinical reasoning development that are transferable to broader contexts where learning to make and share authentic independent and group rational decision-making is a focus. One final DP: Harness a Simple Design, will be the focus of this paper’

Since the article focuses on a single DP, add 1–2 lines listing the other five to position DP3 among peers and avoid a “single-result” impression.

A table has been added listing all six DPs from the original study with brief meanings for each has been added to the results section.

Further explanation is provided in the introduction why DP3 is the focus for this paper.

Research Objectives

How can combining independent online clinical reasoning analysis with group work support undergraduate health science students learning to perform rational decision making? (Galvin, 2023, p.26).

Clear, but the RQ blends design and outcome. Consider splitting into (i) a design question (what works, for whom, under what conditions?) and (ii) an outcome question (what changes in perceived authenticity/evaluative judgement?).

A table has been included in Appendix A (A1) listing research questions and sub questions, with disruption of how they map to advisory and descriptive elements and outcomes.

‘a digital decision wheel tool (Galvin, 2021) was embedded into subjects across undergraduate programs to be used both independently, when engaged in group work, and when being guided by an expert teacher’.

Specify how often the wheel was used, where in the weekly cycle, and with what prompts.

Additional explanation has been included in the materials and methods section on how the decision wheel was used to for non-graded learning activities for clinical reasoning development.

Materials and Methods

Add a table mapping your five 12-week cycles to DBR phases, artefact changes, data sources, and participants.

A Table 4 with his information is now included in results section.

Please add an Ethics statement (IRB, protocol number, consent procedures). Also clarify data-management procedures for storing uploaded wheel attempts.

The ethics process is now explained the Methods section. There is additional information about ethics approval in the Patents section.

Add SUPPLEMENTS with:

 (1) exemplar wheel outputs,

(2) interview/focus prompts, (I can add this as an appendix but would prefer to be asked for this if a reader is interested)

(3) codebook excerpts,

(4) DBR cycle timeline

Examples of the decision wheel were in the original manuscript for review in the results section. The authors think this is adequate for this paper to show how the decision wheel can be used for interested readers.

Codebook analysis was not used in this study. Reflexive thematic analysis was used. Examples of coding and a DBR timeline have been added in the results section and into a new Appendix B.

Please detail:

(a) coding framework development (inductive/deductive)

(b) how the three elements guided theme generation vs. post-hoc mapping, (c) how many transcripts per cycle were coded and by whom, and (d) how you handled reflexivity (positionality memoing, audit trail

An explanation of how semantic and latent coding was used when combining DBR and RTA is now explained with examples in the methods section, and an Appendix B has been added for more detail.

Change “Reflexive Thematic Analysis (rTA) method” to “Reflexive thematic analysis (RTA)” for consistency with Braun & Clarke.

This revision has been actioned.

It's actually Simon, 1972

This has been updated in the body of the text and in reference list.

‘catalogue and prioritise thoughts, rate choices to a question, and provide a summary of thinking that could easily be visible to the self and others’.

Include a worked example in an appendix with the exact prompt, categories, and an anonymized output to increase  reproducibility.

The examples provided in the manuscript are deemed adequate by the authors for this paper. Readers can contact the authors for specific information on how the decision wheel was used across twenty subjects/units were in the original manuscript for review in the results section. The authors think this is adequate for this paper to show how the decision wheel can be used for interested readers.  

Clarify whether the assessment rubric ever referenced the wheel; otherwise “reduced academic integrity risk” is hard to attribute.

Reference to claims around academic integrity risk has been removed from the abstract, as this is not the focus of the article.

It is now clarified that the use of the decision wheel was not in an assessment rubric in any subjects included in this study.

Provide a flow diagram: recruitment, inclusion, per-cycle counts, attrition. Who were in interviews vs. focus groups?

Key information is now included in Table 4 in the results section of the paper on number of focus groups, interviews across three DBR action cycles. Readers can contact the authors for more detailed information on recruitment, attrition, and who was in interviews Vs focus groups.

When using reflexive TA, it was critical for the researcher to make a subjective interpretive judgement to stop analysis when enough has been generated to share a compelling story rather than a traditional notion of data being saturated (Braun & Clarke, 2021).

complement with a paragraph on information power (case diversity, aim specificity, dialogue quality

Mention of reducing academic integrity risk has been removed from the article

Results

Consider presenting all six DPs (bullet + one line each) before zooming into DP3 for coherence.

All DPs are now included in Table 1 in the results section.

The simple design of the decision wheel was described as:

•        being a useful & versatile learning tool for decision making to use as individuals and in groups

•        enhancing inclusivity & collaboration for learning

•        encouraging learning attempts to show authentic progression of thinking

•        assisting transparency of learning and reducing academic integrity risk

•        providing better opportunities for efficient formative feedback dialogue between peers and key teachers.

Strong claims; each needs traceable evidence. For instance, tie “reducing academic integrity risk” to a data excerpt (teacher quote) or documented pattern (e.g., fewer plagiarism flags) which are currently absent.

This is a final theme definition from the central researcher PhD thesis. The readers can ask the author for more detailed data to gain traceable evidence if needed. Alternatively, the PhD thesis has detailed analysis information in Chapter 10.

Figure 3  - Add counts: how common were “flat” wheels? Did guidance reliably produce more discriminated ratings?

Reflective thematic analysis is a Big Q fully qualitative approach that did not rely on a positivist quantitative approach for meaning making.

Figure 3 is demonstrating an example of how having a collaborative experience working through a decision wheel attempt with a teacher and peers assisted in discriminated ratings.

Table 2 with quotes - Good illustrative quotes. Please report how many participants expressed similar views and clarify whether quotes are typical or exceptional.

It has been stipulated the quotes are common learning experiences in paragraph before Table 2.  

Psychological authenticity indicator table:

Turn these indicators into an operational definition (e.g., codes: voice equity, confidence to dissent, value-action coherence) and connect to Ajjawi/Chambers more explicitly.

This recommended has been actioned now for Table 3.

The students and educators did not want the tool’s simple design to change. Main improvement requests included enhancing editing functions during wheel attempts, improving questions the tool was used for and having more platforms to upload wheel attempts when working online with groups (Galvin, 2023).

Consider a short usability subtheme (affordances/constraints) and specify the edit functions users wanted

The importance of usability has been added at the end of the results section of the paper.

Discussion

The impact of time constraints (Herbert, 1972) during problem-solving activities proved to be an important design consideration for helping students become aware of using heuristics (a mental short cut) when making quick clinical decisions, a necessary skill for practitioners upon graduation to choose real-world solutions confidently (Bolton, 2018; Higgs et al., 2019; Marewski & Gigerenzer, 2012; Moore, 2020; Young, Dory, et al., 2018

Nicely ties bounded rationality to clinical practice. Add a note distinguishing fast and frugal heuristics from premature closure in clinical reasoning.

The authors have taken on board this recommendation for future publications and discussion on how bounded rationality has been used for clinical practice.

…developing confidence in articulating authentic thinking required teachers to deliberately facilitate psychological authenticity by encouraging students to feel comfortable to potentially make mistakes and to reveal ‘cognitive limitations. Tautz & Steen (2021) reiterate that we can only be mistaken in choice when we are able to choose between at least two alternatives. Hence, learning tools that enable authentic critical thinking choices to be ‘visible’ has been attributed to helping students talk, write, and demonstrate knowledge and skills to encourage rational discernment and confidence to share authentic ideas (Medina, 2017).

This is a central theoretical contribution. Consider presenting it as a staged model (Pause → Personal stance → Share → Co-regulate), with the wheel enabling the “Pause” and “Personal stance.”

The authors appreciate this feedback, and we have now made this focus more explicit in the discussion section of this paper. We will consider this feedback for presenting as a staged model for future research.

Moreover, co-regulation of learning between humans and machines is now argued to be a ‘new norm’

Forward-looking and appropriate but ensure the limit is explicit: your data do not include genAI use; thus, claims should be framed as implications or future work.

The authors have made the forward thinking focus of this sentiment clear in the discussion and limitations paragraph of the paper.

…..the use of a simple learning tool without instant and reactive external commentary on opinions enabled a ‘critical pause’ to encourage authentic psychological self-reflection to develop. Making time for a deliberate pause, helped progressive thinking to be visible to the self and others before, during, and after engaging external input from humans or machines.

Consider contrasting with chat-based instant feedback (human or AI) and argue for sequencing (solo wheel → group/AI dialog).

Once again, the authors appreciate this feedback, and we have now made this focus more explicit in the discussion section of this paper. We will consider this feedback for presenting as a staged model for future research.

Conclusion

Keep concise; avoid introducing new claims. Consider adding a concrete, testable next step (e.g., randomized rotation of “wheel-first vs. chat-first” sequences).

Minor revisions made.

Limitations paragraph

Add a paragraph on research limitations: single-institution; convenience sampling; limited number of wheel artefacts (n=40); lack of quantitative outcome measures; potential facilitator effects; absence of genAI exposure

Added to paper after conclusion in new Section 7.

Reviewer comments

Author responses

Reviewer 2

We recommend further integrating the conclusions and emphasizing a few critical issues: the role of generative artificial intelligence, especially since the research was conducted before the emergence of ChatGPT, and the question of how generalizable the results are.

Thank you for this feedback. The authors have clarified that the implication of adopting DP3: Harness a simple design with the use of genAI in HE is to facilitate a ‘critical pause’ when students are learning with humans and machines. This is clarified as being speculation only at this stage. We take on board this feedback to discuss the workings of genAI in more depth, and the role of genAI in learning, for future publications after further testing of this notion.

The outputs of the original DBR project have been explained as having inferential transferability in the Research Objectives section of the paper.

Reviewer comments

Author responses

Reviewer 3

The Abstract and Introduction heavily frame the paper as a response to the challenges posed by generative AI. The abstract explicitly states a "need to evidence if student decision making is authentic before and after engaging with generative artificial intelligence (genAI)". This positions the reader to expect findings related to GenAI. However, the authors later state in a disclaimer that "this research project was not investigating students actively using generative artificial intelligence for clinical reasoning development" and that the "thesis was submitted October 2022 one week before the new use of Chat GPT was launched". This is a fatal flaw in the paper's current structure

This sentence on explicit link with genAI has been removed from the Abstract of the paper.  

The connection to GenAI is entirely speculative and was added after the study's completion. The authors must remove all GenAI framing from the Abstract and Introduction. The paper's problem statement should be reframed around its original, valid context: enhancing authentic clinical reasoning in online and blended learning environments. The GenAI connection should be moved exclusively to the final sections of the Discussion or Conclusion as a clear area for future research or implication, explicitly stating that the current study provides a potential (but untested) tool for this new challenge.

Thank you for this feedback. The authors have considered this carefully and have decided to keep the GenAI framing in the Introduction (has been removed from the Abstract). The central ideas that the DP3: Harness a simple design facilitated a deliberate and simple scaffold to create a ‘critical pause’ for students to externalise reasoning, is more clearly emphasised throughout the paper. This central idea is explicitly linked to the speculation that the DP3 may have inferential transferability to other contexts. We have introduced the potential for this central idea to be further explored in the context of social dialogue with GenAI. We have clarified this is a conceptual implication only at this stage and not yet measured.

The authors think that if information is introduced in the Discussion section without any indication in the Introduction section, that this will not reflect a front and back end of the article. We do agree that the focus of this paper should be on the original research results. The authors do think a strength of the paper is the conceptual introduction of the idea of creating a critical pause and agree it is important to keep this as a point of discussion only at this stage, particularly how this relates to authentic learning in the face of using genAI in HE (even if not tested and validated yet).

The paper states it focuses on one of six final design principles (DPs) from a larger doctorate: "DP3: Harness a Simple Design". While this is acceptable, the rationale for focusing only on this single principle could be strengthened in the introduction. Why is "simplicity" the most critical principle to isolate and discuss, especially in light of the (now speculative) GenAI context? A stronger justification would improve the paper's focus.

Table 1 has been added to the Results section to list and explain the six final DPs that generated from the original DBR research. The authors have chosen to focus on one DP of the original project.

Although the central idea discussed in this paper, that a deliberately simple scaffold facilitated a critical pause for students to externalise reasoning, could easily be mapped to DP4: Integrate time for reflexivity and DP5: Vocalise and collaborate, the authors have chosen to focus on DP3: Harness as simple design because this principle was an enabler of DP4 and DP5 (rather than because it is more important than the subsequent principles generated). This has been now explained in the Results section of the paper.

The study's strengths are significant. The application of Bounded Rationality theory is excellent and well-illustrated in Table 1.

Thank you for this feedback.

The finding of "psychological authenticity"—that the simple tool provided a safe, non-judgmental space for students to form and articulate opinions before group discussion —is a key insight. Once the GenAI framing is corrected, these strengths should be brought to the forefront as the paper's primary contribution.

Thankyou for this feedback. We have now adjusted the GenAI framing and have ensured the finding of psychological authenticity explicitly links to the PhD research outputs, rather than primarily on the speculation of using the DP3: Harness a simple design for the use of GenAI dialogue (which the authors agree is not substantiated at this stage).

Add these references: https://edutech-journals.org/index.php/j-hytel/article/view/191

Thank you for this recommendation. The authors explored these references. Although interesting to read, several are more suited to studies exploring K-12 or VET education contexts. We reviewed our current references are agreed we have adequate representation key ideas presented in this paper.