Healthcare Quality Improvement Analytics: An Example Using Computerized Provider Order Entry
Abstract
:1. Introduction
2. Materials and Methods
2.1. Overview of the Quality Improvement Project
2.2. Study Duration and Data Analysis
2.3. Ethics Approval and Consent to Participate
3. Results
3.1. Demographics
3.2. Demonstrating QI Sustainability
3.2.1. Step 1: Control Chart
3.2.2. Step 2: Analysis of Variance (ANOVA)
3.2.3. Step 3: Segmented Regression Analysis
+ B2∗interventionp1 + A3∗time after 1st phase + et
- Y0: outcome variable (the mean dosing frequencies per patient in a week);
- A1: pre-intervention trend (the change in the mean dosing frequencies per patient with each week before the intervention);
- A2: trend change after intervention (the change in the trend in the mean dosing frequencies per patient after the intervention);
- A3: trend change after 1st phase post-intervention (the change in the trend in the mean dosing frequencies after the 1st phase post-intervention);
- A1+A2: the 1st phase post-intervention slope;
- A2+A3: the 2nd phase post-intervention slope;
- B0: intercept of pre-intervention (intercept at zero);
- B1: level change after intervention (the level change in the mean dosing frequencies per patient immediately after the intervention);
- B2: level change after 1st phase post-intervention;
- time1: from the start of the study period;
- time after intervention1: from the start of the 1st intervention;
- interventiont: an indicator for the 1st intervention;
- (pre-intervention = 0; post-intervention = 1);
- interventionp1: an indicator for the 2nd phase post-intervention
- (1st phase post-intervention = 0; 2nd phase post-intervention = 1);
- et: random error.
3.3. Summary
4. Discussion
5. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Acknowledgments
Conflicts of Interest
Appendix A
Characteristics | Pre-Intervention 1 | Post-Intervention 2 | ||
---|---|---|---|---|
2015 n = 38,794 (%) | 2016 n = 40,716 (%) | 2018 n = 44,328 (%) | 2019 n = 46,009 (%) | |
Sex | ||||
Female | 19,205 (49.5) | 20,480 (50.3) | 22,541 (50.9) | 23,485 (51.0) |
Male | 19,589 (50.5) | 20,236 (49.7) | 21,787 (49.1) | 22,524 (49.0) |
Age, mean ± SD, years | 57.3 ± 16.9 | 57.9 ± 16.9 | 58.6 ± 16.8 | 58.7 ± 16.7 |
18–65 | 24,622 (63.5) | 25,302 (62.1) | 27,403 (61.8) | 28,338 (61.6) |
Over 65 | 14,172 (36.5) | 15,414 (37.9) | 16,925 (38.2) | 17,671 (38.4) |
Length of stay, mean ± SD, days | 10.0 ± 21.4 | 10.8 ± 30.6 | 9.9 ± 25.8 | 9.7 ± 22.6 |
Department at discharge | ||||
Internal medicine | 13,985 (36.0) | 13,565 (33.3) | 14,767 (33.3) | 14,957 (32.5) |
Surgery | 6490 (16.7) | 9503 (23.3) | 10,644 (24.0) | 11,438 (24.9) |
Obstetrics and gynecology | 3636 (9.4) | 3865 (9.5) | 4725 (10.7) | 5295 (11.5) |
Orthopedics | 3888 (10.0) | 3466 (8.5) | 3529 (8.0) | 3473 (7.5) |
Urology | 2906 (7.5) | 3061 (7.5) | 2928 (6.6) | 3158 (6.9) |
Neurology | 2421 (6.2) | 1576 (3.9) | 1607 (3.6) | 1620 (3.5) |
Neuropsychiatry | 991 (2.6) | 935 (2.3) | 865 (2.0) | 837 (1.8) |
Pediatric | 37 (0.1) | 72 (0.2) | 120 (0.3) | 156 (0.3) |
Others | 4440 (11.4) | 4673 (11.5) | 5143 (11.6) | 5075 (11.0) |
References
- Institute of Medicine. Crossing the Quality Chasm: A New Health System for the 21st Century; The National Academies Press: Washington, DC, USA, 2001. [Google Scholar]
- Institute of Medicine. To err is human: Building a safer health system. In Committee on Quality of Health Care in America; National Academy of Sciences: Washington, DC, USA, 2000. [Google Scholar] [CrossRef]
- Warholak, T.L.; Nau, D.P. Quality and Safety in Pharmacy Practice; McGraw-Hill Medical: New York, NY, USA, 2010. [Google Scholar]
- Olmos-Ochoa, T.T.; Ganz, D.A.; Barnard, J.M.; Penney, L.S.; Chawla, N. Sustaining effective quality improvement: Building capacity for resilience in the practice facilitator workforce. BMJ Qual. Saf. 2019, 28, 1016–1020. [Google Scholar] [CrossRef] [PubMed]
- AHRQ (Agency for Healthcare Research and Quality). Module 4. Approaches to Quality Improvement. Available online: https://www.ahrq.gov/ncepcr/tools/pf-handbook/mod4.html (accessed on 5 July 2021).
- Lambl, B.B.; Altamimi, S.; Kaufman, N.E.; Rein, M.S.; Freeley, M.; Duram, M.; Krauss, W.; Kurowski, J.; O’Neill, W.E.; Seeley, P.; et al. Leveraging quality improvement science to reduce C. difficile infections in a community hospital. Jt. Comm. J. Qual. Patient Saf. 2018, 45, 285–294. [Google Scholar] [CrossRef]
- AHRQ (Agency for Healthcare Research and Quality). Ways to Approach the Quality Improvement Process. Available online: https://www.ahrq.gov/cahps/quality-improvement/improvement-guide/4-approach-qi-process/index.html (accessed on 7 July 2021).
- Holmgren, A.J.; Co, Z.; Newmark, L.; Danforth, M.; Classen, D.; Bates, D. Realising the potential of health information technology to enhance medication safety. BMJ Qual. Saf. 2020, 29, 7–9. [Google Scholar] [CrossRef]
- Robert, G.; Sarre, S.; Maben, J.; Griffiths, P.; Chable, R. Exploring the sustainability of quality improvement interventions in healthcare organisations: A multiple methods study of the 10-year impact of the ‘Productive ward: Releasing time to care’ programme in English acute hospitals. BMJ Qual. Saf. 2020, 29, 31–40. [Google Scholar] [CrossRef] [Green Version]
- Glasgow, J.M.; Davies, M.L.; Kaboli, P. Findings from a national improvement collaborative: Are improvements sustained? BMJ Qual. Saf. 2012, 21, 663–669. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Harvey, G. Methods for evaluation of small scale quality improvement projects. Qual. Saf. Health Care 2003, 12, 210–214. [Google Scholar] [CrossRef]
- Hysong, S.J.; Khan, M.M.; Petersen, L.A. Passive monitoring versus active assessment of clinical performance: Impact on measured quality of care. Med. Care 2011, 49, 883–890. [Google Scholar] [CrossRef]
- Portela, M.C.; Pronovost, P.J.; Woodcock, T.; Carter, P.; Dixon-Woods, M. How to study improvement interventions: A brief overview of possible study types. BMJ Qual. Saf. 2015, 24, 325–336. [Google Scholar] [CrossRef] [Green Version]
- Wagner, A.K.; Soumerai, S.B.; Zhang, F.; Ross-Degnan, D. Segmented regression analysis of interrupted time series studies in medication use research. J. Clin. Pharm. Ther. 2002, 27, 299–309. [Google Scholar] [CrossRef]
- Smartsheet. Quality Improvement Processes: The Basics and Beyond. Available online: https://www.smartsheet.com/quality-improvement-process (accessed on 29 August 2021).
- U. S. Department of Health and Human Services (Health Resources and Services Administration). Developing and Implementing a QI Plan. Available online: https://www.hrsa.gov/sites/default/files/quality/toolbox/508pdfs/qualityimprovement.pdf (accessed on 29 August 2021).
- Benzer, J.K.; Young, G.J.; Burgess, J.F.; Baker, E.; Mohr, D.C.; Charns, M.P.; Kaboli, P.J. Sustainability of quality improvement following removal of pay-for-performance incentives. J. Gen. Intern. Med. 2013, 29, 127–132. [Google Scholar] [CrossRef] [Green Version]
- Cho, J.; Shin, S.; Jeong, Y.; Lee, E.; Lee, E. The effect of regimen frequency simplification on provider order generation: A quasi-experimental study in a Korean hospital. Int. J. Environ. Res. Public Health 2021, 18, 4086. [Google Scholar] [CrossRef] [PubMed]
- Williams, J.; Bates, D.W.; Sheikh, A. Optimising electronic prescribing in hospitals: A scoping review protocol. BMJ Health Care Inform. 2020, 27, e100117. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Schiff, G.D.; Hickman, T.-T.T.; Volk, L.A.; Bates, D.W.; Wright, A. Computerised prescribing for safer medication ordering: Still a work in progress. BMJ Qual. Saf. 2015, 25, 315–319. [Google Scholar] [CrossRef] [PubMed]
- R Core Team. R: A Language and Environment for Statistical Computing; R Foundation for Statistical Computing: Vienna, Austria, 2013; Available online: https://www.R-project.org/ (accessed on 11 August 2021).
- Lin, M.; Lucas, H.C.; Shmueli, G. Research commentary—too big to fail: Large samples and thep-value problem. Inf. Syst. Res. 2013, 24, 906–917. [Google Scholar] [CrossRef] [Green Version]
- Faraone, S.V. Interpreting estimates of treatment effects: Implications for managed care. Pharm. Ther. 2008, 33, 700–711. [Google Scholar]
- Wikipedia. Control Chart. Available online: https://en.wikipedia.org/wiki/Control_chart (accessed on 1 May 2021).
- Fretheim, A.; Tomic, O. Statistical process control and interrupted time series: A golden opportunity for impact evaluation in quality improvement. BMJ Qual. Saf. 2015, 24, 748–752. [Google Scholar] [CrossRef] [Green Version]
- BPI Consulting. Interpreting Control Charts. Available online: https://www.spcforexcel.com/knowledge/control-charts-basics/interpreting-control-charts (accessed on 10 May 2021).
- Analytic Steps. An Overview of Descriptive Analysis. Available online: https://www.analyticssteps.com/blogs/overview-descriptive-analysis (accessed on 4 June 2021).
- Statistics Solutions. Conduct and Interpret a One-Way ANOVA. Available online: https://www.statisticssolutions.com/free-resources/directory-of-statistical-analyses/one-way-anova/ (accessed on 10 June 2021).
- Lantz, B. The impact of sample non-normality on ANOVA and alternative methods. Br. J. Math. Stat. Psychol. 2013, 66, 224–244. [Google Scholar] [CrossRef]
- Lee, S.; Lee, D.K. What is the proper way to apply the multiple comparison test? Korean J. Anesthesiol. 2018, 71, 353–360. [Google Scholar] [CrossRef] [Green Version]
- Zuckerman, I.H.; Lee, E.; Wutoh, A.K.; Xue, Z.; Stuart, B. Application of regression-discontinuity analysis in pharmaceutical health services research. Health Serv. Res. 2006, 41, 550–563. [Google Scholar] [CrossRef] [Green Version]
- Grol, R.; Grimshaw, J. From best evidence to best practice: Effective implementation of change in patients’ care. Lancet 2003, 362, 1225–1230. [Google Scholar] [CrossRef]
- Alexander, J.A.; Hearld, L.R. What can we learn from quality improvement research? A critical review of research methods. Med. Care Res. Rev. 2009, 66, 235–271. [Google Scholar] [CrossRef] [PubMed]
- Hovlid, E.; Bukve, O.; Haug, K.; Aslaksen, A.B.; von Plessen, C. Sustainability of healthcare improvement: What can we learn from learning theory? BMC Health Serv. Res. 2012, 12, 235. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Teixeira, P.G.; Inaba, K.; DuBose, J.; Melo, N.; Bass, M.; Belzberg, H.; Demetriades, D. Measurable outcomes of quality improvement using a daily quality rounds checklist: Two-year prospective analysis of sustainability in a surgical intensive care unit. J. Trauma Acute Care Surg. 2013, 75, 717–721. [Google Scholar] [CrossRef] [PubMed]
- Henry, E.G.; Ngoma, T.; Kaiser, J.L.; Fong, R.M.; Vian, T.; Hamer, D.H.; Rockers, P.C.; Biemba, G.; Scott, N.A. Evaluating implementation effectiveness and sustainability of a maternity waiting homes intervention to improve access to safe delivery in rural Zambia: A mixed-methods protocol. BMC Health Serv. Res. 2020, 20, 191. [Google Scholar] [CrossRef] [PubMed]
Characteristics | Pre-Intervention 1 n = 79,510 (%) | Post-Intervention 2 n = 90,337 (%) | SMD | p-Value |
---|---|---|---|---|
Sex | ||||
Female | 39,685 (49.9) | 46,026 (50.9) | 0.021 | <0.001 a |
Male | 39,825 (50.1) | 44,311 (49.1) | ||
Age, mean ± SD, years | 57.6 ± 16.9 | 58.7 ± 16.7 | 0.064 | <0.001 b |
18–65 | 49,924 (62.8) | 55,741 (61.7) | 0.022 | <0.001 a |
Over 65 | 29,586 (37.2) | 34,596 (38.3) | ||
Length of stay, mean ± SD, days | 10.4 ± 26.5 | 9.8 ± 24.2 | 0.024 | <0.001 b |
Department at discharge | ||||
Internal medicine | 27,550 (34.6) | 29,724 (32.9) | 0.152 | <0.001 a |
Surgery | 15,993 (20.1) | 22,082 (24.4) | ||
Obstetrics and gynecology | 7501 (9.4) | 10,020 (11.1) | ||
Orthopedics | 7354 (9.2) | 7002 (7.8) | ||
Urology | 5967 (7.5) | 6086 (6.7) | ||
Neurology | 3997 (5.0) | 3227 (3.6) | ||
Neuropsychiatry | 1926 (2.4) | 1702 (1.9) | ||
Pediatric | 109 (0.1) | 276 (0.3) | ||
Others | 9113 (11.5) | 10,218 (11.3) |
Outcome | Pre-Intervention 1 n = 79,510 | Post-Intervention | p-Value | |
---|---|---|---|---|
Phase 1 2 n = 44,328 | Phase 2 3 n = 46,009 | |||
Mean dosing frequency per day, Count ± SD | 4.4 ± 2.3 | 3.5 ± 1.8 | 3.6 ± 1.8 | <0.001 |
Variables | Coefficient | p-Value |
---|---|---|
Intercept (B0) | 4.421 | <0.001 |
Pre-intervention trend (A1) | −0.001 | 0.003 |
Level change after intervention (B1) | −0.737 | <0.001 |
Trend change after intervention (A2) | 0.001 | 0.328 |
Level change after 1st phase post-intervention (B2) | 0.041 | 0.265 |
Trend change after 1st phase post-intervention (A3) | 0.000 | 0.828 |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Cho, J.; Shin, S.; Jeong, Y.; Lee, E.; Ahn, S.; Won, S.; Lee, E. Healthcare Quality Improvement Analytics: An Example Using Computerized Provider Order Entry. Healthcare 2021, 9, 1187. https://doi.org/10.3390/healthcare9091187
Cho J, Shin S, Jeong Y, Lee E, Ahn S, Won S, Lee E. Healthcare Quality Improvement Analytics: An Example Using Computerized Provider Order Entry. Healthcare. 2021; 9(9):1187. https://doi.org/10.3390/healthcare9091187
Chicago/Turabian StyleCho, Jungwon, Sangmi Shin, Youngmi Jeong, Eunsook Lee, Soyeon Ahn, Seunghyun Won, and Euni Lee. 2021. "Healthcare Quality Improvement Analytics: An Example Using Computerized Provider Order Entry" Healthcare 9, no. 9: 1187. https://doi.org/10.3390/healthcare9091187