The National Outcomes Evaluation Programme in Italy: The Impact of Publication of Health Indicators
- compare the outcomes of health care provided by different hospitals or in different geographical areas;
- monitor trends in health care quality over time;
- identify critical areas in which to implement programmes that improve health care quality;
- promote internal and external auditing.
2. Materials and Methods
2.1. Study Design and Data Sources
2.2. Study Population
2.3. Definition and Attribution of Outcome
2.4. Coexisting Medical Conditions
2.5. Statistical Analysis
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
- Smith, P.C.; Mossialos, E.; Papanicolas, I. Performance measurement for health system improvement: Experiences, challenges and prospects. In Proceedings of the WHO European Ministerial Conference on Health Systems, Tallinn, Estonia, 25–27 June 2008; Volume 1, pp. 1–22. [Google Scholar]
- Lansky, D. Improving quality through public disclosure of performance information. Health Aff. 2002, 21, 52–62. [Google Scholar] [CrossRef] [PubMed]
- Fung, C.H.; Lim, Y.W.; Mattke, S.; Damberg, C.; Shekelle, P.G. Systematic review: The evidence that publishing patient care performance data improves quality of care. Ann. Intern. Med. 2008, 148, 111–123. [Google Scholar] [CrossRef]
- Campanella, P.; Vukovic, V.; Parente, P.; Sulejmani, A.; Ricciardi, W.; Specchia, M.L. The impact of Public Reporting on clinical outcomes: A systematic review and meta-analysis. BMC Health Serv. Res. 2016, 16, 296. [Google Scholar] [CrossRef] [PubMed]
- Fusco, D.; Barone, A.P.; Sorge, C.; D’Ovidio, M.; Stafoggia, M.; Lallo, A.; Davoli, M.; Perucci, C.A. P.Re.Val.E.: Outcome research program for the evaluation of health care quality in Lazio, Italy. BMC Health Serv. Res. 2012, 12, 25. [Google Scholar] [CrossRef]
- Pinnarelli, L.; Nuti, S.; Sorge, C.; Davoli, M.; Fusco, D.; Agabiti, N.; Vainieri, M.; Perucci, C.A. What drives hospital performance? The impact of comparative outcome evaluation of patients admitted for hip fracture in two Italian regions. BMJ Qual. Saf. 2012, 21, 127–134. [Google Scholar] [CrossRef] [PubMed]
- Colais, P.; Di Martino, M.; Fusco, D.; Perucci, C.A.; Davoli, M. The effect of early surgery after hip fracture on 1-year mortality. BMC Geriatr. 2015, 15, 141. [Google Scholar] [CrossRef] [PubMed]
- Hibbard, J.H.; Stockard, J.; Tusler, M. Hospital performance reports: Impact on quality, market share, and reputation. Health Aff. 2005, 24, 1150–1160. [Google Scholar] [CrossRef] [PubMed]
- Agency for Healthcare Research and Quality (AHRQ). AHRQ Quality Indicators. Guide to Inpatient Quality Indicators: Quality of Care in Hospitals—Volume, Mortality, and Utilization; Version 3.1.; Agency for Healthcare Research and Quality: Rockville, MD, USA, 2007. Available online: https://qualityindicators.ahrq.gov/measures/iqi_resources (accessed on 8 June 2022).
- NHS—National Centre for Health Outcomes Development. Clinical and Health Outcomes Knowledge Base Care Quality Commission. Available online: http://www.nchod.nhs.uk/ (accessed on 8 June 2022).
- Health Grades. The Seventh Annual Health Grades Hospital Quality and Clinical Excellence Study; Health Grades: Lakewood, CO, USA, 2009; Available online: http://www.healthgrades.com/media/DMS/pdf/HospitalQualityClinicalExcellenceStudy2009.pdf (accessed on 8 June 2022).
- Ministry of Health, Italy. Mattoni-Outcome Project. Available online: http://www.mattoni.salute.gov.it/ (accessed on 8 June 2022).
- Lazio Regional Health Service (RHS) Outcome Evaluation Program. Available online: https://www.dep.lazio.it/prevale2021/index.php (accessed on 8 June 2022).
- AGENAS. The Italian National Outcomes Evaluation Programme 2021 Edition. Available online: https://pne.agenas.it/ (accessed on 8 June 2022).
- Agency for Healthcare Research and Quality (AHRQ). AHRQ Quality Indicators: Guide to Prevention Quality Indicators; Version 3.1; Agency for Healthcare Research and Quality: Rockville, MD, USA, 2007. Available online: https://qualityindicators.ahrq.gov/measures/pqi_resources (accessed on 8 June 2022).
- Agabiti, N.; Ancona, C.; Forastiere, F.; Arcà, M.; Perucci, C.A. Evaluating outcomes of hospital care following coronary artery bypass surgery in Rome, Italy. Eur. J. Cardiothorac. Surg. 2003, 23, 599–606; discussion 607–608. [Google Scholar] [CrossRef]
- Fantini, M.P.; Stivanello, E.; Frammartino, B.; Barone, A.P.; Fusco, D.; Dallolio, L.; Cacciari, P.; Perucci, C.A. Risk adjustment for inter-hospital comparison of primary cesarean section rates: Need, validity and parsimony. BMC Health Serv. Res. 2006, 6, 100. [Google Scholar] [CrossRef]
- Capon, A.; Di Lallo, D.; Perucci, C.A.; Panepuccia, L. Case-mix adjusted odds ratios as an alternative way to compare hospital performances. Eur. J. Epidemiol. 2005, 20, 497–500. [Google Scholar] [CrossRef]
- California Office of Statewide Health Planning and Development. California Hospital Outcomes Project: Heart Attack Outcomes 1996–1998; Technical Guide; OSHPD: Sacramento, CA, USA, 2002; Volume 2. [Google Scholar]
- CCORT/CCS Acute Myocardial Infarction Canadian Quality Indicator Panel. Available online: http://www.ccort.ca/ (accessed on 8 June 2022).
- Commission for Health Improvement. Performance Indicators & Ratings for 2002/2003 Indicator Listings for Acute Trusts. Available online: http://www.chi.nhs.uk/Ratings/Trust/Indicator/indicators.asp?trustType=1 (accessed on 8 June 2022).
- Iezzoni, L.I. Assessing quality using administrative data. Ann. Intern. Med. 1997, 127, 666–674. [Google Scholar] [CrossRef] [PubMed]
- Champney, K.P.; Frederick, P.D.; Bueno, H.; Parashar, S.; Foody, J.; Merz, C.N.; Canto, J.G.; Lichtman, J.H.; Vaccarino, V.; NRMI Investigators. The joint contribution of sex, age and type of myocardial infarction on hospital mortality following acute myocardial infarction. Heart 2009, 95, 895–899. [Google Scholar] [CrossRef] [PubMed]
- Ugolini, C.; Nobilio, L. Risk adjustment for coronary artery bypass graft surgery: An administrative approach versus EuroSCORE. Int. J. Qual. Health Care 2004, 16, 157–164. [Google Scholar] [CrossRef]
- Gudmundsson, G.; Gislason, T.; Lindberg, E.; Hallin, R.; Ulrik, C.S.; Brøndum, E.; Nieminen, M.M.; Aine, T.; Bakke, P.; Janson, C. Mortality in COPD patients discharged from hospital: The role of treatment and co-morbidity. Respir. Res. 2006, 7, 109. [Google Scholar] [CrossRef] [PubMed]
- Löfvendahl, S.; Eckerlund, I.; Hansagi, H.; Malmqvist, B.; Resch, S.; Hanning, M. Waiting for orthopaedic surgery: Factors associated with waiting times and patients’ opinion. Int. J. Qual Health Care 2005, 17, 133–140. [Google Scholar] [CrossRef]
- Iezzoni, L.I. Risk Adjustment for Measuring Healthcare Outcomes; Health Administration Press: Chicago, IL, USA, 2003; Volume 3. [Google Scholar]
- Arcà, M.; Fusco, D.; Barone, A.P.; Perucci, C.A. Risk adjustment and outcome research. Part I. J. Cardiovasc. Med. 2006, 7, 682–690. [Google Scholar] [CrossRef] [PubMed]
- Arcà, M.; Fusco, D.; Barone, A.P.; Perucci, C.A. Introduction to risk adjustment methods in comparative evaluation of outcomes. Epidemiol. Prev. 2006, 30 (Suppl. 4–5), 5–47. (In Italian) [Google Scholar]
- Jenks, G.F. The data model concept in statistical mapping. Int. Yearb. Cartogr. 1967, 7, 186–190. [Google Scholar]
- Hox, J. Multilevel Analysis: Techniques and Applications; Lawrence Erlbaum Associates: Mahwah, NJ, USA, 2002; p. 123. [Google Scholar]
- Larsen, K.; Merlo, J. Appropriate assessment of neighborhood effects on individual health: Integrating random and fixed effects in multilevel logistic regression. Am. J. Epidemiol. 2005, 161, 81–88. [Google Scholar] [CrossRef]
- Gibberd, R.; Hancock, S.; Howley, P.; Richards, K. Using indicators to quantify the potential to improve the quality of health care. Int. J. Qual. Health Care 2004, 16 (Suppl. 1), i37–i43. [Google Scholar] [CrossRef]
- Bradley, E.H.; Holmboe, E.S.; Mattera, J.A.; Roumanis, S.A.; Radford, M.J.; Krumholz, H.M. Data feedback efforts in quality improvement: Lessons learned from US hospitals. Qual. Saf. Health Care 2004, 13, 26–31. [Google Scholar] [CrossRef] [PubMed]
- Marshall, M.N.; Shekelle, P.G.; Davies, H.T.; Smith, P.C. Public reporting on quality in the United States and the United Kingdom. Health Aff. 2003, 22, 134–148. [Google Scholar] [CrossRef] [PubMed]
- Mohammed, M.A.; Deeks, J.J.; Girling, A.; Rudge, G.; Carmalt, M.; Stevens, A.J.; Lilford, R.J. Evidence of methodological bias in hospital standardised mortality ratios: Retrospective database study of English hospitals. BMJ 2009, 338, b780, Erratum in BMJ 2009, 338, b1348. [Google Scholar] [CrossRef]
- Werner, R.M.; Konetzka, R.T.; Stuart, E.A.; Norton, E.C.; Polsky, D.; Park, J. Impact of public reporting on quality of postacute care. Health Serv. Res. 2009, 44, 1169–1187. [Google Scholar] [CrossRef] [PubMed]
- Fortuna, D.; Vizioli, M.; Contini, A.; Ghidoni, I.; Biagi, B.; Pacini, D.; Carnelos, G.; Grilli, R. Assessing clinical performance in cardiac surgery. Does a specialised clinical database make a difference? Interact. Cardiovasc. Thorac. Surg. 2006, 5, 123–127. [Google Scholar] [CrossRef]
- Rosato, S.; D’Errigo, P.; Badoni, G.; Fusco, D.; Perucci, C.A.; Seccareccia, F. Confronto tra l’uso di sistemi informativi correnti e database clinici nella valutazione delle prestazioni cardiochirurgiche. G. Ital. Cardiol. 2008, 9, 569–578. [Google Scholar] [CrossRef]
- Serumaga, B.; Ross-Degnan, D.; Avery, A.J.; Elliott, R.A.; Majumdar, S.R.; Zhang, F.; Soumerai, S.B. Effect of pay for performance on the management and outcomes of hypertension in the United Kingdom: Interrupted time series study. BMJ 2011, 342, d108. [Google Scholar] [CrossRef]
- Aylin, P.; Bottle, A.; Majeed, A. Use of administrative data or clinical databases as predictors of risk of death in hospital: Comparison of models. BMJ 2007, 334, 1044. [Google Scholar] [CrossRef]
|2010||The first edition of P.N.E. was published.|
|2012||The Italian Ministry of Health entrusted AGENAS with the national coordination of P.N.E.|
|2013||The assessment of quality of data recorded in the Health Information System was started in order to evaluate the validity of P.N.E. indicators.|
|2015||The Italian Ministry of Health defined the quality standards of hospital care according to P.N.E. results *.|
|2016||The Italian Ministry of Health established improvement programmes for the hospitals with poor performances according to P.N.E. results **.|
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.
© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Colais, P.; Pinnarelli, L.; Mataloni, F.; Giordani, B.; Duranti, G.; D’Errigo, P.; Rosato, S.; Seccareccia, F.; Baglio, G.; Davoli, M., on behalf of P.N.E. Group. The National Outcomes Evaluation Programme in Italy: The Impact of Publication of Health Indicators. Int. J. Environ. Res. Public Health 2022, 19, 11685. https://doi.org/10.3390/ijerph191811685
Colais P, Pinnarelli L, Mataloni F, Giordani B, Duranti G, D’Errigo P, Rosato S, Seccareccia F, Baglio G, Davoli M on behalf of P.N.E. Group. The National Outcomes Evaluation Programme in Italy: The Impact of Publication of Health Indicators. International Journal of Environmental Research and Public Health. 2022; 19(18):11685. https://doi.org/10.3390/ijerph191811685Chicago/Turabian Style
Colais, Paola, Luigi Pinnarelli, Francesca Mataloni, Barbara Giordani, Giorgia Duranti, Paola D’Errigo, Stefano Rosato, Fulvia Seccareccia, Giovanni Baglio, and Marina Davoli on behalf of P.N.E. Group. 2022. "The National Outcomes Evaluation Programme in Italy: The Impact of Publication of Health Indicators" International Journal of Environmental Research and Public Health 19, no. 18: 11685. https://doi.org/10.3390/ijerph191811685