Explainability and Human Oversight for AI-Generated Exercise Guidance in Digital Healthcare: A Governance-Oriented Narrative Review
Abstract
1. Introduction
2. Materials and Methods
3. Results and Synthesis
3.1. Thematic Grouping of Included Sources
3.2. Safety Relevance of AI-Generated Exercise Guidance
3.3. Explanation Needs Identified Across Sources
3.4. Human Oversight and Escalation Needs
3.5. Selected Regulatory and Policy Signals Relevant to Implementation Governance
3.6. Translation into Proposed Implementation Tools
4. Discussion
4.1. Proposed Framework
4.2. Applying the Framework in Digital Healthcare Services
5. Practical Implications
5.1. Implications for Developers
5.2. Implications for Clinicians and Exercise Professionals
5.3. Implications for Healthcare Organizations and Regulators
5.4. Patient Disclosure, Informed Use, and Equity
5.5. Real-World Implementation and Evaluation
6. Strengths, Limitations, and Future Research
7. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Acknowledgments
Conflicts of Interest
Abbreviations
References
- Lai, X.; Chen, J.; Lai, Y.; Huang, S.; Cai, Y.; Sun, Z.; Wang, X.; Pan, K.; Gao, Q.; Huang, C. Using Large Language Models to Enhance Exercise Recommendations and Physical Activity in Clinical and Healthy Populations: Scoping Review. JMIR Med. Inform. 2025, 13, e59309. [Google Scholar] [CrossRef]
- Pool, J.; Indulska, M.; Sadiq, S. Large Language Models and Generative AI in Telehealth: A Responsible Use Lens. J. Am. Med. Inform. Assoc. 2024, 31, 2125–2136. [Google Scholar] [CrossRef] [PubMed]
- Doherty, C.; Lambe, R.; O’Grady, B.; O’Reilly-Morgan, D.; Smyth, B.; Lawlor, A.; Hurley, N.; Tragos, E. An Evaluation of the Effect of App-Based Exercise Prescription Using Reinforcement Learning on Satisfaction and Exercise Intensity: Randomized Crossover Trial. JMIR Mhealth Uhealth 2024, 12, e49443. [Google Scholar] [CrossRef]
- Cox, E.R.; Beacroft, S.; Jansson, A.K.; Wade, L.; Duncan, M.J.; Lubans, D.R.; Robards, S.L.; Leitner, M.; Gutberlet, N.; Plotnikoff, R.C. Effects of mHealth Interventions to Prescribe Resistance Training: A Systematic Review and Meta-Analysis of Randomized Controlled Trials. Int. J. Behav. Nutr. Phys. Act. 2026, 23, 7. [Google Scholar] [CrossRef]
- Costa, D.V.S.D.; Dourado, E.P.; Fernandes, M.B.; Fernandes, E.V.; de Oliveira, D.M. Effects of the Prescription of Physical Exercises Mediated by Mobile Applications on the Health of Older Adults: A Systematic Review. Geriatrics 2025, 10, 122. [Google Scholar] [CrossRef]
- Taylor, J.L.; Myers, J.; Bonikowske, A.R. Practical Guidelines for Exercise Prescription in Patients with Chronic Heart Failure. Heart Fail. Rev. 2023, 28, 1285–1296. [Google Scholar] [CrossRef] [PubMed]
- Baptista, F.M.; Andias, R.; Rocha, N.P.; Silva, A.G. A Practice Guide for Physical Therapists Prescribing Physical Exercise for Older Adults. J. Aging Phys. Act. 2024, 32, 771–783. [Google Scholar] [CrossRef] [PubMed]
- Maher, A.; Bennett, M.; Huang, H.-C.C.; Gaughwin, P.; Johnson, M.; Brady, M.; Patterson, K.; Buettikofer, T.; Morris, J.; Rainbird, V.M.; et al. Personalized Exercise Prescription in Long COVID: A Practical Toolbox for a Multidisciplinary Approach. J. Multidiscip. Healthc. 2024, 17, 3981–3991. [Google Scholar] [CrossRef]
- Hansen, D.; Beckers, P.; Neunhäuserer, D.; Bjarnason-Wehrens, B.; Piepoli, M.F.; Rauch, B.; Völler, H.; Corrà, U.; Garcia-Porrero, E.; Schmid, J.-P.; et al. Standardised Exercise Prescription for Patients with Chronic Coronary Syndrome and/or Heart Failure: A Consensus Statement from the EXPERT Working Group. Sports Med. 2023, 53, 2013–2037. [Google Scholar] [CrossRef]
- Amann, J.; Blasimme, A.; Vayena, E.; Frey, D.; Madai, V.I.; Precise4Q Consortium. Explainability for Artificial Intelligence in Healthcare: A Multidisciplinary Perspective. BMC Med. Inform. Decis. Mak. 2020, 20, 310. [Google Scholar] [CrossRef]
- Liu, Y.; Liu, C.; Zheng, J.; Xu, C.; Wang, D. Improving Explainability and Integrability of Medical AI to Promote Health Care Professional Acceptance and Use: Mixed Systematic Review. J. Med. Internet Res. 2025, 27, e73374. [Google Scholar] [CrossRef]
- Shevtsova, D.; Ahmed, A.; Boot, I.W.A.; Sanges, C.; Hudecek, M.; Jacobs, J.J.L.; Hort, S.; Vrijhoef, H.J.M. Trust in and Acceptance of Artificial Intelligence Applications in Medicine: Mixed Methods Study. JMIR Hum. Factors 2024, 11, e47031. [Google Scholar] [CrossRef] [PubMed]
- Kostick-Quenet, K.; Lang, B.H.; Smith, J.; Hurley, M.; Blumenthal-Barby, J. Trust Criteria for Artificial Intelligence in Health: Normative and Epistemic Considerations. J. Med. Ethics 2024, 50, 544–551. [Google Scholar] [CrossRef]
- Khera, R.; Simon, M.A.; Ross, J.S. Automation Bias and Assistive Artificial Intelligence: Risk of Harm from AI-Driven Clinical Decision Support. JAMA 2023, 330, 2255–2257. [Google Scholar] [CrossRef]
- Kücking, F.; Hübner, U.; Przysucha, M.; Hannemann, N.; Kutza, J.-O.; Moelleken, M.; Erfurt-Berge, C.; Dissemond, J.; Babitsch, B.; Busch, D. Automation Bias in AI-Decision Support: Results from an Empirical Study. Stud. Health Technol. Inform. 2024, 317, 298–304. [Google Scholar] [CrossRef] [PubMed]
- Rose, S.L.; Shapiro, D. An Ethically Supported Framework for Determining Patient Notification and Informed Consent Practices When Using Artificial Intelligence in Health Care. Chest 2024, 166, 572–578. [Google Scholar] [CrossRef]
- Ursin, F.; Lindner, F.; Ropinski, T.; Salloch, S.; Timmermann, C. Levels of Explicability for Medical Artificial Intelligence: What Do We Normatively Need and What Can We Technically Reach? Ethik Med. 2023, 35, 173–199. [Google Scholar] [CrossRef]
- Ratti, E.; Morrison, M.; Jakab, I. Ethical and Social Considerations of Applying Artificial Intelligence in Healthcare: A Two-Pronged Scoping Review. BMC Med. Ethics 2025, 26, 68. [Google Scholar] [CrossRef]
- Khan, M.M.; Shah, N.; Shaikh, N.; Thabet, A.; Alrabayah, T.; Belkhair, S. Towards Secure and Trusted AI in Healthcare: A Systematic Review of Emerging Innovations and Ethical Challenges. Int. J. Med. Inform. 2025, 195, 105780. [Google Scholar] [CrossRef]
- World Health Organization. Ethics and Governance of Artificial Intelligence for Health: WHO Guidance; World Health Organization: Geneva, Switzerland, 2021; Available online: https://www.who.int/publications/i/item/9789240029200 (accessed on 29 May 2026).
- European Parliament and the Council of the European Union. Regulation (EU) 2024/1689 of the European Parliament and of the Council of 13 June 2024 Laying Down Harmonised Rules on Artificial Intelligence (Artificial Intelligence Act). Available online: https://eur-lex.europa.eu/eli/reg/2024/1689/oj/eng (accessed on 29 May 2026).
- European Parliament and the Council of the European Union. Regulation (EU) 2017/745 of the European Parliament and of the Council of 5 April 2017 on Medical Devices. Available online: https://eur-lex.europa.eu/eli/reg/2017/745/oj/eng (accessed on 29 May 2026).
- U.S. Food and Drug Administration. Clinical Decision Support Software: Guidance for Industry and Food and Drug Administration Staff; U.S. Food and Drug Administration: Silver Spring, MD, USA, 2026. Available online: https://www.fda.gov/regulatory-information/search-fda-guidance-documents/clinical-decision-support-software (accessed on 29 May 2026).
- U.S. Food and Drug Administration. Good Machine Learning Practice for Medical Device Development: Guiding Principles; U.S. Food and Drug Administration: Silver Spring, MD, USA, 2025. Available online: https://www.fda.gov/medical-devices/software-medical-device-samd/good-machine-learning-practice-medical-device-development-guiding-principles (accessed on 29 May 2026).
- U.S. Food and Drug Administration. Marketing Submission Recommendations for a Predetermined Change Control Plan for Artificial Intelligence-Enabled Device Software Functions: Guidance for Industry and Food and Drug Administration Staff; U.S. Food and Drug Administration: Silver Spring, MD, USA, 2025. Available online: https://www.fda.gov/regulatory-information/search-fda-guidance-documents/marketing-submission-recommendations-predetermined-change-control-plan-artificial-intelligence (accessed on 29 May 2026).
- Cyberspace Administration of China. Interim Measures for the Management of Generative Artificial Intelligence Services. 2023. Available online: https://www.cac.gov.cn/2023-07/13/c_1690898327029107.htm (accessed on 29 May 2026).
- Beauchamp, T.L.; Childress, J.F. Principles of Biomedical Ethics, 8th ed.; Oxford University Press: New York, NY, USA, 2019. [Google Scholar]
- Yuen, E.; Winter, N.; Savira, F.; Huggins, C.E.; Nguyen, L.; Cooper, P.; Peeters, A.; Anderson, K.; Bhoyroo, R.; Crowe, S.; et al. Digital Health Literacy and Its Association with Sociodemographic Characteristics, Health Resource Use, and Health Outcomes: Rapid Review. Interact. J. Med. Res. 2024, 13, e46888. [Google Scholar] [CrossRef] [PubMed]
- Henzler, D.; Schmidt, S.; Koçar, A.; Herdegen, S.; Lindinger, G.L.; Maris, M.T.; Bak, M.A.R.; Willems, D.L.; Tan, H.L.; Lauerer, M.; et al. Healthcare Professionals’ Perspectives on Artificial Intelligence in Patient Care: A Systematic Review of Hindering and Facilitating Factors on Different Levels. BMC Health Serv. Res. 2025, 25, 633. [Google Scholar] [CrossRef] [PubMed]
- Busch, F.; Hoffmann, L.; Xu, L.; Zhang, L.J.; Hu, B.; García-Juárez, I.; Toapanta-Yanchapaxi, L.N.; Gorelik, N.; Gorelik, V.; Rodriguez-Granillo, G.A.; et al. Multinational Attitudes toward AI in Health Care and Diagnostics among Hospital Patients. JAMA Netw. Open 2025, 8, e2514452. [Google Scholar] [CrossRef]
- Meier, E.; Rigter, T.; Schijven, M.P.; van den Hoven, M.; Bak, M.A.R. The Impact of Digital Health Technologies on Moral Responsibility: A Scoping Review. Med. Health Care Philos. 2025, 28, 17–31. [Google Scholar] [CrossRef]
- Starke, G.; Ienca, M. Misplaced Trust and Distrust: How Not to Engage with Medical Artificial Intelligence. Camb. Q. Healthc. Ethics 2024, 33, 360–369. [Google Scholar] [CrossRef] [PubMed]
- De Proost, M.; Pozzi, G. Why We Should Talk about Institutional (Dis)Trustworthiness and Medical Machine Learning. Med. Health Care Philos. 2025, 28, 83–92. [Google Scholar] [CrossRef]
- Dankwa-Mullan, I. Health Equity and Ethical Considerations in Using Artificial Intelligence in Public Health and Medicine. Prev. Chronic Dis. 2024, 21, E64. [Google Scholar] [CrossRef]
- Oliveira, D.; Rego, F.; Nunes, R. Artificial Intelligence in Exercise Prescription in Palliative Care: Perceptions and Ethical Issues. Healthcare 2025, 13, 2987. [Google Scholar] [CrossRef] [PubMed]
- Fernandes Prabhu, D.; Gurupur, V.; Stone, A.; Trader, E. Integrating Artificial Intelligence, Electronic Health Records, and Wearables for Predictive, Patient-Centered Decision Support in Healthcare. Healthcare 2025, 13, 2753. [Google Scholar] [CrossRef]
- Alanazi, A.S.; Alanazi, A.S.; Benlaria, H. Enhancing Healthcare for People with Disabilities Through Artificial Intelligence: Evidence from Saudi Arabia. Healthcare 2025, 13, 1616. [Google Scholar] [CrossRef]

| Review Component | Revised Reporting |
|---|---|
| Search dates and sources | Initial searches were conducted on 10 January 2026 and updated on 15 May 2026 in PubMed/MEDLINE, Google Scholar, and official regulatory or guidance websites from China, the European Union, the World Health Organization, and the United States. During the present revision, a targeted journal-specific check of Healthcare was also conducted on 29 May 2026 to identify recent articles directly relevant to AI exercise prescription, patient-centered digital health decision support, rehabilitation, explainability, and equity. |
| Search strings | The PubMed/MEDLINE string combined AI terms (“large language model” OR “generative AI” OR “artificial intelligence” OR “explainable AI” OR “clinical decision support”), exercise and digital-health terms (“exercise prescription” OR “exercise guidance” OR “physical activity” OR “rehabilitation” OR “telerehabilitation” OR “digital health” OR “mobile health”), and governance/safety terms (“patient safety” OR “human oversight” OR “automation bias” OR “trust” OR “governance”). Google Scholar used equivalent phrase combinations, and official websites were searched with combinations of “artificial intelligence”, “generative AI”, “clinical decision support”, “medical device”, “transparency”, “human oversight”, “software”, “change control”, and “governance”. Google Scholar was used as a supplementary interdisciplinary source, with screening limited to the first 100 relevance-ranked results for each phrase combination to improve transparency. |
| Selection flow | Records or source documents identified: n = 207. Duplicate or overlapping records removed: n = 39. Records screened by title, abstract, or summary: n = 168. Records excluded at screening: n = 103. Full texts or full documents assessed: n = 65. Full-text or full-document exclusions: n = 28, including technical-only model papers (n = 7), diagnostic AI without transferable decision-support implications (n = 6), generic wellness or marketing sources (n = 5), insufficient exercise or patient-facing relevance (n = 6), and non-authoritative commentary (n = 4). Final included source set: n = 37, including three targeted Healthcare articles relevant to AI exercise prescription, patient-centered digital health decision support, disability-related rehabilitation, and implementation equity. |
| Eligibility and exclusion | Sources were included when they informed at least one operational domain: exercise safety, digital delivery, AI explainability, human oversight, regulatory governance, digital literacy, patient attitudes, or implementation accountability. Sources were excluded when they lacked healthcare implementation relevance, transferable decision-support implications, exercise or patient-facing relevance, or a clear evidentiary or policy basis. |
| Screening and disagreements | Screening and charting were conducted by two authors (K.P. and X.L.). Disagreements were discussed with C.H. until consensus was reached. |
| Data extraction | Extracted fields included source type, population or user group, clinical or service setting, AI system or digital health technology, exercise or rehabilitation context, safety concern, explainability mechanism, human oversight mechanism, governance implication, and relevance to patient-facing digital healthcare. |
| Synthesis and status of outputs | The synthesis used framework synthesis and governance mapping in four traceable steps: (1) charting source statements related to safety, explanation, oversight, and governance; (2) grouping recurrent concerns by evidentiary role and implementation context; (3) mapping those concerns to digital healthcare workflow points; and (4) translating the mapped considerations into the proposed implementation outputs after the literature-supported themes and selected policy signals had been synthesized. These outputs are proposed conceptual and implementation-oriented tools rather than validated checklists, scoring instruments, or regulatory classifications. |
| Evidence role and quality appraisal | No single risk-of-bias tool or systematic-review quality appraisal was applied because the review combined empirical studies, reviews, ethics papers, and policy or regulatory documents rather than a single systematic-review evidence corpus. Instead, sources were categorized by evidentiary role before synthesis: empirical studies and systematic reviews informed implementation and safety considerations; exercise-prescription and rehabilitation sources informed clinical plausibility; official documents informed governance signals; and conceptual papers informed interpretive concepts. |
| Reference verification | For the present revision, the full reference list was rebuilt from DOI-, PubMed-, publisher-, or official-source records; mismatched author, title, journal, year, volume, page, and DOI fields were corrected, and in-text citation placement was rechecked. |
| Thematic Source Group | Primary Role in the Narrative Synthesis | Representative Sources |
|---|---|---|
| Digital exercise delivery and exercise-prescription evidence | Informed why exercise guidance becomes safety-relevant when it specifies dose, progression, contraindication-sensitive action, or rehabilitation strategy, and supported attention to adherence, monitoring, and remote delivery. | [1,2,3,4,5,6,7,8,9,35] |
| Explainability, trust, and automation bias literature | Informed the distinction between generic transparency and reviewable explanation, including evidence sources, reasoning paths, automation bias, trust, and human-AI reliance. | [10,11,12,13,14,15,17,19,36] |
| Professional responsibility, ethics, and patient disclosure literature | Informed patient notification, informed use, professional responsibility, institutional trustworthiness, and the ability of clinicians and exercise professionals to question, revise, or reject AI output. | [13,16,18,27,31,32,33,35] |
| Regulatory and policy documents | Informed the selected governance signals related to intended use, medical-device or clinical-decision-support status, human oversight, lifecycle governance, change control, and public-facing generative-AI service obligations. | [20,21,22,23,24,25,26] |
| Digital literacy, patient/clinician attitudes, and equity literature | Informed implementation concerns related to user understanding, clinician acceptance, digital literacy, equity, and applicability in lower-resource or differently regulated settings. | [28,29,30,34,37] |
| Jurisdiction | Selected Source(s) | Key Regulatory Trigger | Service Implication |
|---|---|---|---|
| China | Interim Measures for the Management of Generative Artificial Intelligence Services (2023) [26] | Public-facing generative AI services are subject to baseline obligations concerning lawful, safe, and regulated deployment. The measure itself does not create a dedicated exercise-prescription test. | Supports upstream governance and service-provider accountability for app-based exercise guidance, but does not by itself define what point-of-use explanation is sufficient for individualized recommendations. |
| European Union | European Union Artificial Intelligence Act (Regulation (EU) 2024/1689) [21] and Medical Devices Regulation (EU) 2017/745 [22] | If an AI function is used as a medical device or safety component supporting therapeutic or rehabilitation decisions, the interaction between AI governance and medical-device regulation can move the system toward a higher-risk compliance pathway. | Strengthens risk classification, documentation, transparency, and human oversight, while service-level explanation formats remain important when exercise guidance is delivered through apps, wearables, or remote rehabilitation pathways. |
| United States | FDA Clinical Decision Support Software guidance (current 2026 version) [23], Good Machine Learning Practice guiding principles [24], and Predetermined Change Control Plan guidance [25] | Regulatory status turns on intended use, target user, device status, and whether a healthcare professional can independently review the basis for a recommendation; patient- or caregiver-facing functions may warrant closer scrutiny. | Highlights reviewability, labeling, lifecycle governance, and change control for digital healthcare services, while leaving implementation teams to define practical patient-facing explanation and escalation mechanisms. |
| Explainability Element | Implementation Purpose | Practical Question | Example Service Feature |
|---|---|---|---|
| Evidence source disclosure | Supports justified reliance and source transparency | What is this recommendation based on? | Show a tappable evidence card with source, date, and evidence-grade badge, such as guideline, consensus statement, validated protocol, or curated rule set. |
| Risk-warning communication | Supports safer use and harm prevention | What could make this unsafe? | Use contraindication acknowledgement and display stop rules before high-intensity progression, symptom-sensitive training, or rehabilitation-sensitive tasks. |
| Reasoning-path clarification | Supports reviewability and clinical checking | Why was this recommendation selected? | Provide a concise rationale panel showing structured inputs, retrieved guidance, contraindication logic, monitoring feedback, and FITT-VP-aligned rules that shaped dose, progression, or modality. |
| Presentation of reasonable alternatives | Supports user choice and shared decision-making | What reasonable options are available? | Offer a lower-intensity, lower-impact, or professionally reviewed pathway with a brief explanation of why it was or was not prioritized. |
| Governance Area | Core Question | Minimum Safeguard | Example Action |
|---|---|---|---|
| Use-case triage | Is the function wellness support, personalized guidance, or safety-relevant decision support? | Classify risk before deployment using dose control, contraindication sensitivity, rehabilitation relevance, and likelihood of direct enactment. | Link each tier to review and documentation expectations. |
| Source transparency | Can users and reviewers see what the recommendation is based on? | Show evidence provenance, protocol source, structured-data basis, or rule-set basis. | Add source labels or concise evidence cards in patient and reviewer views. |
| Risk communication | What could make the guidance unsafe? | Display contraindications, stop rules, red flags, and escalation cues at the point of use. | Use warning acknowledgement before higher-risk progression. |
| Human review | Which outputs may warrant professional confirmation? | Define review triggers for high-risk, ambiguous, or vulnerability-sensitive cases. | Route flagged outputs to clinician or exercise-professional review. |
| Escalation pathways | What happens when symptoms worsen or risk increases? | Build referral or follow-up routes into the workflow. | Trigger contact, teleconsultation, referral, or temporary suspension. |
| Audit and monitoring | Can the service support review and improvement? | Retain logs of outputs, sources, versions, overrides, symptom reports, and escalation events where appropriate. | Use audit trails for quality review, incident analysis, and post-deployment monitoring. |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2026 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license.
Share and Cite
Pan, K.; Huang, C.; Lin, X.; Huang, S. Explainability and Human Oversight for AI-Generated Exercise Guidance in Digital Healthcare: A Governance-Oriented Narrative Review. Healthcare 2026, 14, 1716. https://doi.org/10.3390/healthcare14121716
Pan K, Huang C, Lin X, Huang S. Explainability and Human Oversight for AI-Generated Exercise Guidance in Digital Healthcare: A Governance-Oriented Narrative Review. Healthcare. 2026; 14(12):1716. https://doi.org/10.3390/healthcare14121716
Chicago/Turabian StylePan, Kaijiang, Caihua Huang, Xinyu Lin, and Shengqi Huang. 2026. "Explainability and Human Oversight for AI-Generated Exercise Guidance in Digital Healthcare: A Governance-Oriented Narrative Review" Healthcare 14, no. 12: 1716. https://doi.org/10.3390/healthcare14121716
APA StylePan, K., Huang, C., Lin, X., & Huang, S. (2026). Explainability and Human Oversight for AI-Generated Exercise Guidance in Digital Healthcare: A Governance-Oriented Narrative Review. Healthcare, 14(12), 1716. https://doi.org/10.3390/healthcare14121716

