Evaluating the Associations Between Leapfrog Scores and Patient Safety Culture

Abstract

Background: Patient safety is evaluated using both internal assessments of safety culture and external hospital rating systems; however, the extent to which these measures capture related dimensions of patient safety remains uncertain. Methods: This study examined the association between hospital patient safety culture ratings and hospital safety grades. Using 2024 hospital-level data, patient safety culture was measured using the overall safety rating from the AHRQ Hospital Survey on patient safety culture and hospital safety grades derived from Leapfrog Safety Grades. Results: No significant association was observed between patient safety culture ratings and the likelihood of receiving higher Leapfrog Grades. Hospital bed size, however, was significantly associated with Leapfrog Grades, with larger hospitals demonstrating lower odds of receiving higher grades. Conclusions: These findings indicate that further research is needed to assess congruence between internal perceptions of patient safety culture and external hospital safety ratings. The results underscore potential measurement differences between federal survey-based assessments and private nonprofit grading methodologies and suggest caution in assuming equivalence across patient safety metrics. Further research is warranted to clarify how these tools align and to determine their respective roles in evaluating hospital patient safety.

1. Introduction

Patient safety theoretically prioritizes preventing errors, mishaps, and adverse events that can lead to injury, disability, or death in patients during care [1]. Adverse events, or unintended harm to a patient due to care being provided, occur in around 10% of hospital patients, and around half are declared preventable [2].

Patient safety is measured with a variety of metrics, most commonly The Leapfrog Group’s Leapfrog score, the US News and World Report, the Consumer Reports’ Hospital Safety measure, and patient safety culture [1,3,4,5]. This study will primarily focus on The Leapfrog Group’s Leapfrog score and the AHRQ’s Survey on patient safety culture.

In 1999, the National Academy of Medicine, formerly the Institute of Medicine, declared in their report “To Err is Human” the need for every health system to prioritize a culture of patient safety [6]. After this statement, in 2004, the Agency for Healthcare Research and Quality (AHRQ), an operating division of the Department of Health and Human Services in the United States, developed and released the Survey on Patient Safety Culture (SOPS) Hospital Survey 1.0, and in 2019, AHRQ released version 2.0 [5,7]. AHRQ provided a definition of patient safety culture as “The safety culture of an organization is the product of individual and group values, attitudes, perceptions, competencies, and patterns of behavior that determine the commitment to, and the style and proficiency of, an organization’s health and safety management” [5]. This definition implies that patient safety would not rely solely on outcomes but on the values and beliefs of organizations as well. When assessing patient safety culture in hospitals, it has been found that a more positive rating on patient safety culture is associated with reduced mortality, fewer medication errors, fewer readmission, and increased family and patient satisfaction [8].

The Leapfrog Group is a coalition of healthcare purchases formed in November 2000 in effort to make “great leaps forward” in the safety and quality of healthcare in America [9]. The Leapfrog Group is known for releasing the biannual Hospital Safety Grades, formerly known as scores [10]. In 2012, The Leapfrog Group launched their Hospital Safety Grade, aiming to improve transparency in medicine and healthcare delivery [11]. This safety grade accounts for infection, surgical complications, adverse event prevention measures, staffing, and overall safety problems [11]. There is limited research exploring the association between Leapfrog Safety Grades and patient health outcomes. Gonzales and Ghaferi found that there was negligible difference between hospitals’ outcomes based on Leapfrog Safety Grade alone [12].

With the utilization of various patient safety measures for healthcare systems, it is essential to understand their congruence, especially if they are having varying perspectives. Patient safety culture is internally reported by healthcare workers, while the Leapfrog Safety Grade is an external grade provided by health experts. Badr et al. previously assessed and compared various measurement tools across hospital ratings to understand synergies and found that there were vast discrepancies in rating scales [4]. However, Badr et al. did not explore if there were relationships amongst any of the measurement tools utilized.

This study aims to evaluate the relationship between Leapfrog Grades and patient safety culture ratings. Both of these instruments review patient safety within hospitals. The HSOPS considers patient safety culture from an internal workforce perspective, while Leapfrog Safety Grades assesses patient safety from external composited measures. Though the measurement tools explore patient safety from opposing perspectives, for consistent validity, it is believed that they will have alignment. We hypothesize that there will be a higher likelihood of hospitals with a more positive patient safety culture rating to demonstrate better Leapfrog safety ratings. The findings of this study will help us understand the congruence and test the relationship between a federal measurement tool and a private measurement tool for patient safety.

2. Materials and Methods

2.1. Study Design

This cross-sectional analysis examined the association between hospital safety grades from The Leapfrog Group (Leapfrog Scores) and staff perceptions of patient safety culture (HSOPS safety rating) for hospitals in the United States in 2024. The primary objective was to assess whether hospitals with more favorable patient safety culture ratings demonstrated higher Leapfrog Grades.

2.2. Data Sources

This study utilized 2024 data from two sources, the Hospital Survey on Patient Safety Culture Version 2.0 (HSOPS 2.0), administered by the Agency for Healthcare Research and Quality (AHRQ), and the average 2024 Leapfrog Hospital Safety Grade dataset from The Leapfrog Group, and included both the spring and fall grades of 2024. HSOPS 2.0 is a validated survey that captures hospital staff perceptions of safety culture across ten domains, including communication openness, teamwork, and response to error. The primary independent variable was the overall patient safety rating (Item E1), which was measured on a 5-point Likert scale ranging from 1 (Poor) to 5 (Excellent), and was modeled as an ordinal predictor. The Leapfrog Group assigns hospitals a letter grade (A through F) based on publicly reported patient safety performance measures. Leapfrog Grades served as the dependent variable.

The AHRQ Hospital Survey on Patient Safety Culture Version 2.0 comprises 32 survey items that are bundled into 10 composite measures that explore teamwork, staffing, organizational learning, response to error, supervisor support for patient safety, communication about error, communication openness, reporting patient safety events, hospital management, and handoffs; there are two additional survey items that discuss the number of events reported and ask each participant to provide a rating for their organizations’ patient safety culture [7].

The Leapfrog Patient Safety Grades range from A to F and are based on 22 national metrics that are divided into two domains, Process/Structural Measures and Outcome Measures. Process Measures would refer to how often a hospital gives recommended treatment options for conditions, while Structural Measures assess the environment where patients receive care, and Outcome Measures determine what happens when patients receive care. These measures are collected from the Center for Medicare and Medicaid Services’ (CMS) Medicare PSI 90 Patient Safety and Adverse Event survey, CMS, the Leapfrog Hospital Survey, and other supplemental data sources [10].

2.3. Data Linkage and Sample Construction

To link the two data sources, researchers utilized the hospital’s name, state, city, and zip code to manually assign a Leapfrog Grade to each hospital listed in the 2024 HSOPS dataset using publicly available Leapfrog Safety Grade listings. Both the 2024 HSOPS dataset and the 2024 Leapfrog Safety Grade data contain all of these unique identifiers. Leapfrog Safety Grades were assigned to ordinal numeric scale values as follows: F = 1, D = 2, C = 3, B = 4, and A = 5. This ordinal variable served as the dependent variable in the analysis. Hospitals were retained in the analytic sample only if they had complete data for Leapfrog Grades and patient safety ratings (Item E1). Both datasets provided data that was clustered at the hospital level. Hospitals were treated as individual observations.

2.4. Statistical Analysis

We used ordinal logistic regression to estimate the association between Leapfrog Grade and overall patient safety culture rating. Patient safety rating, Item E1, was modeled as an ordinal predictor to reflect the ordered nature of the rating system. Additionally, the dependent variable, Leapfrog Grade, was treated as a 5-level ordinal outcome. The model included hospital-level covariates such as bed size, ownership type (public, nonprofit, or for-profit), and teaching status (yes/no). Model results are reported as odds ratios (ORs) with 95% confidence intervals, which indicate the odds of reporting a higher safety rating associated with each unit increase in Leapfrog Grade. Similarly, additional analyses were conducted on each composite of the HSOPS to assess if there are other potential relationships to consider. All data analyses were conducted in Stata 18 MP.

2.5. Ethical Considerations

This study was reviewed and approved by the Nova Southeastern University Institutional Review Board (2025-448). Because the analysis involved secondary analysis of de-identified, hospital-level data, it was determined to be exempt from human-subject review. All data were stored on encrypted, password-protected servers accessible only to members of the research team.

3. Results

A total of 57 hospitals were utilized in this study. The hospitals included reported both HSOPS safety ratings and Leapfrog Safety Grades in 2024 and differed in size, teaching status, and ownership (Appendix A; Table A1, Table A2, Table A3 and Table A4). Leapfrog Safety Grades were valued using a numeric coding scheme ranging from 1 (F) to 5 (A). The mean Leapfrog Grade was 3.5 (SD = 0.81), with observed values ranging from 2 to 5, indicating variability in hospital safety performance across the sample (Table 1).

Table 1. Distribution of hospital Leapfrog Grades.

Observations	Mean Leapfrog Grade	Standard Deviation	Minimum	Maximum
57	3.5	0.81	2	5

Table 1 showcases the pattern of Leapfrog Grades for hospitals included based on the numeric coding (1 = F and 5 = A).

Patient safety culture ratings were assed using a numeric scale ranging from 1 (Poor) to 5 (Excellent). The mean patient safety culture rating was 3.92 (SD = 0.20), with values ranging from 3.39 to 4.34, reflecting generally high and relatively consistent perceptions of patient safety culture across hospitals (Table 2).

Table 2. Distribution of hospital patient safety culture ratings.

Observations	Mean Patient Safety Culture Rating	Standard Deviation	Minimum	Maximum
57	3.92	0.20	3.39	4.34

Table 2 showcases the pattern of patient safety culture of hospitals based on the numeric coding (1 = Poor and 5 = Excellent).

An ordinal logistic regression model was used to examine the association between patient safety culture ratings and the likelihood of receiving higher Leapfrog Grades while adjusting for hospital characteristics. Patient safety culture rating was not associated with higher Leapfrog Grades (OR = 0.89, p-value = 0.94; 95% CI: 0.04–19.74) (Table 3).

Table 3. Ordinal logistic regression: patient safety culture on Leapfrog Grade likeliness.

Leapfrog Grade	Odds Ratio	Standard Deviation	z	p-Value	95% Confidence Interval
Hospital Mean Patient Safety Culture Rating	0.89	1.41	−0.07	0.94	0.04	19.74
Bed Size	0.67	0.11	−2.39	0.02 *	0.48	0.93
Teaching Status	2.78	1.73	1.65	0.10	0.82	9.41
Ownership Status	0.65	0.37	−0.75	0.46	0.21	2.00

* p-value < 0.05.

Hospital bed size was associated with Leapfrog Grades, with larger hospitals demonstrating lower odds of receiving higher grades (OR = 0.67, p = 0.02; 95% CI: 0.48–0.93) (Table 3). Further analyses utilizing the HSOPS composites did not showcase any significant relationships.

4. Discussion

This study aimed to explore the alignment of higher Leapfrog Safety Grades and a more positive patient safety culture in hospitals in the United States. Based on the results, we found no concordance between the likelihood of higher Leapfrog Grades and safety grades based on a more positive patient safety culture. Additional analyses examining potential reverse causality similarly yielded no significant findings, suggesting that higher Leapfrog Grades were not associated with improved perceptions of patient safety culture, nor vice versa (Appendix A; Table A5). These findings indicate that further research examining the relationship between internal perceptions of patient safety culture and external hospital safety ratings is necessary.

Supplementary analyses demonstrated that hospital bed size was associated with Leapfrog Grades, with larger hospitals tending to have a higher likelihood of receiving lower grades (Appendix A; Table A5), while no other composite demonstrated a significant relationship. This finding aligns with prior work suggesting that hospital structural characteristics may influence rating systems [13,14].

Several factors may explain the lack of a significant relationship between patient safety culture ratings and Leapfrog Grades. This includes limitations of this particular study. One potential explanation relates to constraints within both measurement tools. Leapfrog Grades are composite scores derived from multiple data sources, each of which may introduce variability or discrepancies in how hospital safety is assessed. In contrast, patient safety culture ratings derived from hospitals and included in this study demonstrated limited variability. When examining the distribution of mean Leapfrog Grades by mean patient safety culture ratings, hospitals clustered within a narrow range of approximately 0.5 points on the patient safety culture scale (Appendix A; Table A6). This limited variability may be attributable to characteristics of survey distribution and participation. Notably, the SOPS instrument was developed for a federal agency and is administered as a voluntary survey, which may influence response patterns. It is also possible that hospitals choosing to participate in the patient safety culture survey are those with a greater interest in improving or demonstrating strong patient safety practices. As a result, hospitals with an average or below-average patient safety culture may be underrepresented in this sample. Additionally, response bias, including social desirability bias or authoritative bias depending on survey facilitation, may further contribute to constrained variability in patient safety culture ratings. Moreover, there were a total of 57 hospitals participating in this study. Treating each hospital as individual observations leads to this study having a small sample size, despite the HSOPS dataset being composed of prior-clustered individual responses in each hospital. The lower power associated with the representation of hospitals could potentially explain the lack of concordance between the measurement tools. Another possible explanation for the lack of association is the influence of contextual factors that may affect the relationship between patient safety culture and/or Leapfrog Safety Grades. Contextual factors of each facility, and/or the patients they serve, such as the socioeconomic characteristics of community hospitals, patient case mix, and geographical location can affect staff perceptions of patient safety culture in ways that are not accounted for by the surveys [1]. Lastly, it is critical to understand that survey responses could differ based on the contextual perspective of each respondent during the time of participation.

Patient safety culture surveys and Leapfrog Grades are two commonly used tools to assess patient safety within hospitals in the United States, and in theory, they should reflect related dimensions of safety performance. Failure to reject the null hypothesis in this study may support prior findings that highlight discrepancies across hospital rating systems, particularly when comparing internally focused measures of safety culture with externally derived performance ratings [4]. However, due to this study’s limitations with power in the sample size of hospitals included and limited variability in patient safety culture ratings, we currently cannot curate confident conclusions on survey misalignments.

Contrastingly, previous studies exploring the relationship between patient safety culture and other measures of safety have shown alignment. Sorra et al. found that when exploring the relationship between patient safety culture and patients’ assessment of care received, there was a association between higher overall SOPS scores and higher patient experience scores [15]. Likewise, it has been found that there is an association between patient safety culture and workplace safety culture [16].

5. Conclusions

This study did not observe a statistically significant association between patient safety culture ratings and Leapfrog Hospital Safety Grades. Although both instruments are designed to reflect dimensions of hospital safety performance, these findings suggest that they may capture distinct constructs or operate through different methodological frameworks. Interpretation should remain cautious given the limited sample size and constrained variability in patient safety culture ratings, which may have reduced statistical power to detect meaningful relationships. Future research using larger and more heterogeneous samples is needed to further evaluate the extent to which internal culture-based assessments align with externally derived grading systems. From a policy perspective, stakeholders should be mindful that different safety metrics may not function as interchangeable indicators of performance. For hospital leaders and practitioners, these findings underscore the value of using multiple complementary safety assessment tools to inform quality improvement efforts rather than relying on a single measure to represent overall patient safety.