e-Health Strategy for Surgical Prioritization: A Methodology Based on Digital Twins and Reinforcement Learning
Abstract
1. Introduction
2. Literature Review
3. Methodology
3.1. Step 1: System Overview and Dataset Specification
- : Age of patient at time of registration (years);
- : Gender, encoded as a binary or categorical variable;
- : Primary diagnosis or procedure code;
- : Initial clinical risk score assigned by medical staff;
- : Time already spent on the waiting list (in days or weeks);
- : Expected economic value or reimbursement associated with the intervention (e.g., DRG-based revenue);
- : Initial satisfaction or engagement score based on digital platform interactions;
- Data layer: Responsible for collecting, cleaning, integrating and storing heterogeneous sources of patient information, including clinical diagnostics, monitoring outputs, digital engagement records, and financial metadata.
- Digital twin layer: Creates and continuously updates a real-time digital representation of each patient, reflecting their evolving health status, economic profile, satisfaction signals, and social vulnerability. These Digital Twins form the core analytic object used in decision-making.
- Decision layer: Implements intelligent scheduling decisions through a combination of prioritization logic and reinforcement learning algorithms. Select patients dynamically according to multiple and potentially conflicting criteria.
3.2. Step 2: Digital Twin Modeling of Surgical Patients
- : Clinical risk. Captures the time-varying probability of deterioration of health or adverse outcome if surgery is delayed [41,42]. It evolves asWe calibrated based on synthetic risk gradients extracted from surgical specialties commonly associated with time-sensitive outcomes (e.g., ENT and oncology cases). The stochastic term was introduced to reflect interpatient variability and diagnostic uncertainty, allowing risk trajectories to remain dynamic and partially unpredictable while constrained by clinically plausible parameters. This formulation reflects systematic components of clinical deterioration over time.
- : Economic value. Represents the expected reimbursement or cost recovery associated with the surgical procedure of the patient [43]. This may depreciate over time due to administrative or funding restrictions:
- : Satisfaction and digital engagement. Reflects how actively and positively the patient engages with digital health tools (e.g., use of apps, satisfaction surveys) [44,45,46]:
- : Delay cost. Represents the penalty for waiting, growing over time due to the accumulation of unaddressed health needs or logistical inefficiencies [47,48,49]. We modeled as follows:In our simulation, we defined as a linear function with patient-specific slope, i.e., , where reflects the rate at which delay generates cost for patient . These values were sampled based on clinical risk categories, simulating heterogeneous sensitivity to delays. The cumulative cost thus follows a quadratic growth pattern over time, representing escalating clinical and logistical burdens.
- : Vulnerability index. Aggregates psychosocial, geographic, and economic disadvantages, updated discretely when new data become available (e.g., social work reports or survey responses) [50,51].We compute as a weighted sum of standardized vulnerability sub-indices:
3.3. Step 3: Dynamic Prioritization Based on Utility Function
- : Normalized clinical risk at time t;
- : Normalized economic value or cost recovery;
- : Normalized satisfaction or digital engagement score;
- : Normalized vulnerability index;
- : Normalized delay cost (i.e., the cumulative penalty for prolonged waiting);
- : Weight assigned to each dimension , such that .
- : regularization parameter that controls the trade-off between efficiency and equity,
- : small constant to avoid division by zero.
- : estimated surgical duration of patient ;
- : total surgical capacity (e.g., in minutes or slots) available at time t.
3.4. Step 4: Learning-Based Scheduling via Reinforcement Learning
- : State space. Each state encodes the real-time Digital Twin vectors for all patients, current surgical capacity , and contextual information (e.g., calendar day, service disruptions).
- : Action space. Each action corresponds to selecting a feasible subset of patients to be scheduled for surgery.
- : Transition function. Defines the probability of reaching the next state given the current state and action.
- : Reward function. Quantifies the utility of an action using prioritization scores and system performance metrics.
- : Discount factor. Determines the present value of future rewards.
- : fairness-adjusted utility score of patient , as defined in Step 3 and used consistently throughout the prioritization process.
- : fairness deviation penalty, quantifying discrepancies such as underrepresentation of vulnerable patients in the current schedule.
- : operational inefficiency penalty, such as unused OR capacity or scheduling gaps.
- : tunable penalty weights for fairness and efficiency, respectively.
3.5. Step 5: Integration in a Smart eHealth Platform
- Patient interface: we design this interface to allow patients to view their prioritization status, engage with preoperative content, and provide feedback through digital tools.
- Clinical dashboard: we envision this dashboard to support clinical teams by displaying prioritization scores, alerts for deteriorating patients, and visualizations of capacity usage and scheduling scenarios.
- Decision engine: we integrate our RL-based scheduling methodology into this engine, which processes real-time Digital Twin updates and returns optimized subsets of patients to be scheduled for surgery.
4. Results
- Baseline 1 (FCFS): First-Come-First-Served scheduling.
- Baseline 2 (Risk-Based): Prioritization based on the static clinical risk threshold.
4.1. Wait Time Reduction
4.2. Reduction in Clinical Risk at Surgery Time
4.3. Improvement in Operating Room Efficiency
4.4. Equity in Prioritization: Inclusion of Vulnerable Patients
4.5. Synthesis of Results
5. Discussion
6. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- Wiebe, K.; Kelley, S.; Fecteau, A.; Levine, M.; Blajchman, I.; Shaul, R.Z.; Kirsch, R. Surgical waitlist management: Perspectives from surgeons on surgical prioritization at a paediatric hospital. Paediatr. Child Health 2024, 29, 74–80. [Google Scholar] [CrossRef] [PubMed]
- Aitken, S.J.; James, S.; Lawrence, A.; Glover, A.; Pleass, H.; Thillianadesan, J.; Monaro, S.; Hitos, K.; Naganathan, V. Codesign of health technology interventions to support best-practice perioperative care and surgical waitlist management. BMJ Health Care Inform. 2024, 31, e100928. [Google Scholar] [CrossRef] [PubMed]
- Koushan, M.; Wood, L.C.; Greatbanks, R. Evaluating factors associated with the cancellation and delay of elective surgical procedures: A systematic review. Int. J. Qual. Health Care 2021, 33, mzab092. [Google Scholar] [CrossRef] [PubMed]
- Koushan, M. Improving Surgical Service Delivery: Managing Surgery Cancellations with Comprehensive Scheduling and Simulation of Operating Room Capacity Management Policies. Ph.D. Thesis, University of Otago, Dunedin, New Zealand, 2022. [Google Scholar]
- Sigurpálsson, Á.Ö. Pattern Scheduling: A Practical Approach to Preventing Surgery Cancellations Due to Uncertainty in Surgery Times, Bed Availability, and Arrivals of Semi-Acute Elective Patients. Ph.D. Thesis, University of Iceland, Reykjavik, Iceland, 2025. [Google Scholar]
- Byrnes, M.E.; Brown, C.S.; De Roo, A.C.; Corriere, M.A.; Romano, M.A.; Fukuhara, S.; Kim, K.M.; Osborne, N.H. Elective surgical delays due to COVID-19: The patient lived experience. Med. Care 2021, 59, 288–294. [Google Scholar] [CrossRef]
- Pape, T.; Savva, N.; Scholtes, S.; Kastner, C.; Horder, K. Reducing Health Inequality Through Socially Informed Scheduling of Elective Surgery. 2025. Available online: https://ssrn.com/abstract=5099611 (accessed on 10 April 2025).
- Xue, J.; Li, Z.; Zhang, S. Multi-resource constrained elective surgical scheduling with Nash equilibrium toward smart hospitals. Sci. Rep. 2025, 15, 3946. [Google Scholar] [CrossRef]
- Zamani, H.; Parvaresh, F.; Izady, N.; Farahani, R.Z. Admission, discharge, and transfer control in patient flow logistics: Overview and future research. Transp. Res. Part E Logist. Transp. Rev. 2024, 191, 103722. [Google Scholar] [CrossRef]
- Leiva-Araos, A.; Contreras, C.; Kaushal, H.; Prodanoff, Z. Predictive Optimization of Patient No-Show Management in Primary Healthcare Using Machine Learning. J. Med. Syst. 2025, 49, 7. [Google Scholar] [CrossRef]
- Akabane, M.; Kawashima, J.; Altaf, A.; Woldesenbet, S.; Cauchy, F.; Aucejo, F.; Popescu, I.; Kitago, M.; Martel, G.; Ratti, F.; et al. Enhancing recurrence-free survival prediction in hepatocellular carcinoma: A time-updated model incorporating tumor burden and AFP dynamics. Ann. Surg. Oncol. 2025, 1–9. [Google Scholar] [CrossRef]
- Ahmad, A.; Tariq, A.; Hussain, H.K.; Gill, A.Y. Equity and artificial intelligence in surgical care: A comprehensive review of current challenges and promising solutions. BULLET J. Multidisiplin Ilmu 2023, 2, 443–455. [Google Scholar]
- Varghese, C.; Harrison, E.M.; O’Grady, G.; Topol, E.J. Artificial intelligence in surgery. Nat. Med. 2024, 30, 1257–1268. [Google Scholar] [CrossRef]
- Vallée, A. Envisioning the future of personalized medicine: Role and realities of digital twins. J. Med. Internet Res. 2024, 26, e50204. [Google Scholar] [CrossRef] [PubMed]
- Abd Elaziz, M.; Al-qaness, M.A.; Dahou, A.; Al-Betar, M.A.; Mohamed, M.M.; El-Shinawi, M.; Ali, A.; Ewees, A.A. Digital twins in healthcare: Applications, technologies, simulations, and future trends. Wiley Interdiscip. Rev. Data Min. Knowl. Discov. 2024, 14, e1559. [Google Scholar] [CrossRef]
- Palvadi, S.K.; Pradeep, K.; Kadiravan, G. Exploring the Advantages and Security Aspects of Digital Twin Technology in Healthcare. In Artificial Intelligence and Cybersecurity in Healthcare; Scrivener Publishing: Beverly, MA, USA, 2025; pp. 173–205. [Google Scholar]
- Loftus, T.J.; Filiberto, A.C.; Li, Y.; Balch, J.; Cook, A.C.; Tighe, P.J.; Efron, P.A.; Upchurch, G.R., Jr.; Rashidi, P.; Li, X.; et al. Decision analysis and reinforcement learning in surgical decision-making. Surgery 2020, 168, 253–266. [Google Scholar] [CrossRef] [PubMed]
- Datta, S.; Li, Y.; Ruppert, M.M.; Ren, Y.; Shickel, B.; Ozrazgat-Baslanti, T.; Rashidi, P.; Bihorac, A. Reinforcement learning in surgery. Surgery 2021, 170, 329–332. [Google Scholar] [CrossRef]
- Ali, H. Reinforcement learning in healthcare: Optimizing treatment strategies, dynamic resource allocation, and adaptive clinical decision-making. Int. J. Comput. Appl. Technol. Res. 2022, 11, 88–104. [Google Scholar]
- Rahimi, I.; Gandomi, A.H. A comprehensive review and analysis of operating room and surgery scheduling. Arch. Comput. Methods Eng. 2021, 28, 1667–1688. [Google Scholar] [CrossRef]
- Abdalkareem, Z.A.; Amir, A.; Al-Betar, M.A.; Ekhan, P.; Hammouri, A.I. Healthcare scheduling in optimization context: A review. Health Technol. 2021, 11, 445–469. [Google Scholar] [CrossRef]
- Al Amin, M.; Baldacci, R.; Kayvanfar, V. A comprehensive review on operating room scheduling and optimization. Oper. Res. 2025, 25, 1–30. [Google Scholar] [CrossRef]
- Cardoen, B.; Demeulemeester, E.; Beliën, J. Operating room planning and scheduling: A classification scheme. Int. J. Health Manag. Inf. 2010, 1, 71–83. [Google Scholar]
- Guerriero, F.; Guido, R. Operational research in the management of the operating theatre: A survey. Health Care Manag. Sci. 2011, 14, 89–114. [Google Scholar] [CrossRef]
- Yu, C.; Liu, J.; Nemati, S.; Yin, G. Reinforcement learning in healthcare: A survey. ACM Comput. Surv. (CSUR) 2021, 55, 5. [Google Scholar] [CrossRef]
- Akkur, M.; Singh, B.; Singh, T.; Panda, B.; Marwaha, P. Sequential Optimization of Healthcare Processes using Reinforcement Learning and Sequential Function Models. In Proceedings of the 2025 International Conference on Automation and Computation (AUTOCOM), Dehradun, India, 4–6 March 2025; IEEE: Piscataway, NJ, USA, 2025; pp. 631–636. [Google Scholar]
- Kalusivalingam, A.K.; Sharma, A.; Patel, N.; Singh, V. Optimizing Decision-Making with AI-Enhanced Support Systems: Leveraging Reinforcement Learning and Bayesian Networks. Int. J. AI ML 2020, 1. [Google Scholar]
- Wu, Q.; Han, J.; Yan, Y.; Kuo, Y.H.; Shen, Z.J.M. Reinforcement learning for healthcare operations management: Methodological framework, recent developments, and future research directions. Health Care Manag. Sci. 2025, 1–36. [Google Scholar] [CrossRef] [PubMed]
- Kamel Boulos, M.N.; Zhang, P. Digital twins: From personalised medicine to precision public health. J. Pers. Med. 2021, 11, 745. [Google Scholar] [CrossRef]
- Vallée, A. Digital twin for healthcare systems. Front. Digit. Health 2023, 5, 1253050. [Google Scholar] [CrossRef]
- Jameil, A.K.; Al-Raweshidy, H. A digital twin framework for real-time healthcare monitoring: Leveraging AI and secure systems for enhanced patient outcomes. Discov. Internet Things 2025, 5, 37. [Google Scholar] [CrossRef]
- Corral-Acero, J.; Margara, F.; Marciniak, M.; Rodero, C.; Loncaric, F.; Feng, Y.; Gilbert, A.; Fernandes, J.F.; Bukhari, H.A.; Wajdan, A.; et al. The ‘Digital Twin’to enable the vision of precision cardiology. Eur. Heart J. 2020, 41, 4556–4564. [Google Scholar] [CrossRef]
- Bruynseels, K.; Santoni de Sio, F.; Van den Hoven, J. Digital twins in health care: Ethical implications of an emerging engineering paradigm. Front. Genet. 2018, 9, 31. [Google Scholar] [CrossRef]
- Johnson, M.; Albizri, A.; Simsek, S. Artificial intelligence in healthcare operations to enhance treatment outcomes: A framework to predict lung cancer prognosis. Ann. Oper. Res. 2022, 308, 275–305. [Google Scholar] [CrossRef]
- Liu, L.; Ni, Y.; Zhang, N.; Nick Pratap, J. Mining patient-specific and contextual data with machine learning technologies to predict cancellation of children’s surgery. Int. J. Med. Inform. 2019, 129, 234–241. [Google Scholar] [CrossRef]
- Luo, L.; Zhang, F.; Yao, Y.; Gong, R.; Fu, M.; Xiao, J. Machine learning for identification of surgeries with high risks of cancellation. Health Inform. J. 2020, 26, 141–155. [Google Scholar] [CrossRef] [PubMed]
- Silva-Aravena, F.; Álvarez-Miranda, E.; Astudillo, C.A.; González-Martínez, L.; Ledezma, J.G. On the data to know the prioritization and vulnerability of patients on surgical waiting lists. Data Brief 2020, 29, 105310. [Google Scholar] [CrossRef] [PubMed]
- Silva-Aravena, F.; Álvarez-Miranda, E.; Astudillo, C.A.; González-Martínez, L.; Ledezma, J.G. Patients’ prioritization on surgical waiting lists: A decision support system. Mathematics 2021, 9, 1097. [Google Scholar] [CrossRef]
- Silva-Aravena, F.; Delafuente, H.N.; Astudillo, C.A. A novel strategy to classify chronic patients at risk: A hybrid machine learning approach. Mathematics 2022, 10, 3053. [Google Scholar] [CrossRef]
- Silva-Aravena, F.; Morales, J.; Jayabalan, M.; Rana, M.E.; Gutiérrez-Bahamondes, J.H. Dynamic Surgical Prioritization: A Machine Learning and XAI-Based Strategy. Technologies 2025, 13, 72. [Google Scholar] [CrossRef]
- Wiens, J.; Guttag, J.; Horvitz, E. Patient risk stratification with time-varying parameters: A multitask learning approach. J. Mach. Learn. Res. 2016, 17, 1–23. [Google Scholar]
- Sashegyi, A.; Ferry, D. On the interpretation of the hazard ratio and communication of survival benefit. Oncologist 2017, 22, 484–486. [Google Scholar] [CrossRef]
- Chen, A.; Freedman, S.M.; Munnich, E.L.; Richards, M.R. Pricing Innovation in Surgical Care Markets; Technical Report; National Bureau of Economic Research: Cambridge, MA, USA, 2024. [Google Scholar]
- Aktas, M.; Cambon, L.; Aromatario, O. User-Engagement in Digital Health Applications: A Scoping Review. 2023. Available online: https://www.researchgate.net/publication/374451427_User-engagement_in_digital_health_applications_a_scoping_review (accessed on 10 April 2025).
- Milne-Ives, M.; Homer, S.; Andrade, J.; Meinert, E. The conceptualisation and measurement of engagement in digital health. Internet Interv. 2024, 36, 100735. [Google Scholar] [CrossRef]
- Sundaramurthi, T.; Mathews, S.; Bermudez, E.; Mahajan, S.M. Patient Engagement and Co-creation in Healthcare Services: A Scoping Review. Patient Exp. J. 2024, 11, 215–245. [Google Scholar] [CrossRef]
- Kim, J.G.; Jun, H.B.; Bang, J.Y.; Shin, J.H.; Choi, S.H. Minimizing tardiness penalty costs in job shop scheduling under maximum allowable tardiness. Processes 2020, 8, 1398. [Google Scholar] [CrossRef]
- Ala, A.; Chen, F. Appointment scheduling problem in complexity systems of the healthcare services: A comprehensive review. J. Healthc. Eng. 2022, 2022, 5819813. [Google Scholar] [CrossRef] [PubMed]
- Bauerhenne, C.; Kolisch, R.; Schulz, A.S. Robust Appointment Scheduling with Waiting Time Guarantees. arXiv 2024, arXiv:2402.12561. [Google Scholar]
- Silva-Aravena, F.; Gutiérrez-Bahamondes, J.H.; Núñez Delafuente, H.; Toledo-Molina, R.M. An intelligent system for patients’ well-being: A multi-criteria decision-making approach. Mathematics 2022, 10, 3956. [Google Scholar] [CrossRef]
- Silva-Aravena, F.; Morales, J. Dynamic surgical waiting list methodology: A networking approach. Mathematics 2022, 10, 2307. [Google Scholar] [CrossRef]
- Santomauro, C.M.; Hill, A.; McCurdie, T.; McGlashan, H.L. Improving the quality of evaluation data in simulation-based healthcare improvement projects: A practitioner’s guide to choosing and using published measurement tools. Simul. Healthc. 2020, 15, 341–355. [Google Scholar] [CrossRef]
- Chen, V.X.; Hooker, J. Balancing fairness and efficiency in an optimization model. arXiv 2020, arXiv:2006.05963. [Google Scholar]
- Li, Y.; Mao, C.; Huang, K.; Wang, H.; Yu, Z.; Wang, M.; Luo, Y. Deep reinforcement learning for efficient and fair allocation of health care resources. arXiv 2023, arXiv:2309.08560. [Google Scholar]
- Isakov, A.; Peregorodiev, D.; Tomilov, I.; Ye, C.; Gusarova, N.; Vatian, A.; Boukhanovsky, A. Real-Time Scheduling with Independent Evaluators: Explainable Multi-Agent Approach. Technologies 2024, 12, 259. [Google Scholar] [CrossRef]
- Liu, X.; Zheng, C.; Chen, Z.; Liao, Y.; Chen, R.; Yang, S. Reinforcement Learning for Patient Scheduling with Combinatorial Optimisation. In Proceedings of the International Conference on Innovative Techniques and Applications of Artificial Intelligence, Cambridge, UK, 17–19 December 2024; Springer: Berlin/Heidelberg, Germany, 2024; pp. 238–243. [Google Scholar]
Model | Mean Wait Time | 95% CI | Relative Reduction |
---|---|---|---|
FCFS (Baseline 1) | 27.2 | [26.3, 28.0] | — |
Risk-Based (Baseline 2) | 21.3 | [20.7, 21.9] | −21.7% |
RL + Digital Twin (Proposed) | 12.2 | [11.7, 12.6] | −55.1% |
Model | Mean Risk Score | 95% CI | Relative Reduction |
---|---|---|---|
FCFS (Baseline 1) | 0.712 | [0.697, 0.726] | — |
Risk-Based (Baseline 2) | 0.632 | [0.618, 0.645] | −11.2% |
RL + Digital Twin (Proposed) | 0.414 | [0.401, 0.427] | −41.9% |
Model | Mean Utilization | 95% CI | Relative Improvement |
---|---|---|---|
FCFS (Baseline 1) | 0.781 | [0.770, 0.793] | — |
Risk-Based (Baseline 2) | 0.829 | [0.821, 0.837] | +6.1% |
RL + Digital Twin (Proposed) | 0.907 | [0.901, 0.913] | +16.1% |
Model | Mean Vulnerability Coverage | 95% CI | Relative Improvement |
---|---|---|---|
FCFS (Baseline 1) | 0.226 | [0.218, 0.233] | — |
Risk-Based (Baseline 2) | 0.315 | [0.304, 0.325] | +39.4% |
RL + Digital Twin (Proposed) | 0.478 | [0.466, 0.490] | +111.5% |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2025 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Silva-Aravena, F.; Morales, J.; Jayabalan, M. e-Health Strategy for Surgical Prioritization: A Methodology Based on Digital Twins and Reinforcement Learning. Bioengineering 2025, 12, 605. https://doi.org/10.3390/bioengineering12060605
Silva-Aravena F, Morales J, Jayabalan M. e-Health Strategy for Surgical Prioritization: A Methodology Based on Digital Twins and Reinforcement Learning. Bioengineering. 2025; 12(6):605. https://doi.org/10.3390/bioengineering12060605
Chicago/Turabian StyleSilva-Aravena, Fabián, Jenny Morales, and Manoj Jayabalan. 2025. "e-Health Strategy for Surgical Prioritization: A Methodology Based on Digital Twins and Reinforcement Learning" Bioengineering 12, no. 6: 605. https://doi.org/10.3390/bioengineering12060605
APA StyleSilva-Aravena, F., Morales, J., & Jayabalan, M. (2025). e-Health Strategy for Surgical Prioritization: A Methodology Based on Digital Twins and Reinforcement Learning. Bioengineering, 12(6), 605. https://doi.org/10.3390/bioengineering12060605