Q: How are hospitals selected to receive a Leapfrog Hospital Safety Grade?
A: Leapfrog reviewed data for all general acute care hospitals. Some general hospitals did not have sufficient data available, and therefore did not receive a Safety Grade. More than 2,500 hospitals will be receiving a grade. Hospitals excluded from receiving a grade include critical access hospitals, long-term care and rehabilitation facilities, mental health facilities, federal hospitals (e.g. Veterans Affairs, Indian Health Services, etc.) specialty hospitals, including surgical centers and cancer hospitals, free-standing pediatric hospitals, hospitals in US territories, hospitals in Maryland as they do not participate in the CMS Inpatient Payment System (IPPS), and hospitals that are missing data for more than nine (9) process/structural measures or more than five (5) outcome measures.
Q: Is the Leapfrog Hospital Safety Grade biased toward hospitals that voluntarily report to the Leapfrog Hospital Survey?
A: There are no guarantees that a hospital participating in the Survey will score better. If data for one of the measures on the Leapfrog Hospital Survey was not available for an individual hospital, those measures were not included when calculating the hospital’s grade. Similar to grading in a class room setting, the more measures available to grade a hospital, the better chance that hospital has to score higher.
Q: What measures are included in the Leapfrog Hospital Safety Grade?
A: The Leapfrog Hospital Safety Grade utilizes national performance measures from the Leapfrog Hospital Survey, the Agency for Healthcare Research and Quality (AHRQ), the Centers for Disease Control and Prevention (CDC), and the Centers for Medicare and Medicaid Services (CMS) to produce a single composite grade that represents a hospital’s overall performance in keeping patients safe from preventable harm and medical errors. In addition, secondary data from the American Hospital Association’s Annual Survey and HIT Supplement was used to give hospitals as much credit as possible towards their safety grades. The Hospital Safety Grade includes 30 measures, which are all publicly reported by hospital.
The measure set is divided into two domains: (1) Process/Structural measures and (2) Outcome Measures. Each domain represents 50% of the Hospital Safety Grade.
Measures in the process/structural domain include:
- CPOE (Computerized Physician Order Entry) – Leapfrog
- IPS (ICU Physician Staffing) – Leapfrog
- 8 Safe Practices – Leapfrog
- 5 HCAHPS measures – CMS
Measures in the outcomes domain include:
- 8 HACs (Hospital Acquired Conditions) – CMS and Leapfrog
- 7 PSIs (Patient Safety Indicators) – CMS
Hospitals that did not complete the 2016 Leapfrog Hospital Survey will not have Leapfrog results included in their grade. Weights for each measure on the Hospital Safety Grade are redistributed for hospitals with missing data. For full details on the measures and weighting, hospitals can visit the methodology page or contact the help desk at https://leapfrogscore.zendesk.com.
Q: The Hospital Safety Grade incorporates Leapfrog Survey data that is self-reported. How is the self-reported data validated?
A: Like all measures in the grade, those taken from the Leapfrog Hospital Survey were weighted according to strength of evidence, opportunity for improvement, and impact to the patient. This data is only used in the Hospital Safety Grade for hospitals that voluntarily reported to the Leapfrog Hospital Survey. Leapfrog’s Hospital Survey is as reliable and documented a data collection and public reporting tool as any health data collection effort in the United States short of a full-scale accreditation process. As part of Leapfrog’s data collection process, an intensive review of all submitted surveys is performed annually.
Q: The data in the Hospital Safety Grade relies on administrative reports of hospital-acquired conditions. Many experts find this data flawed. Why are the HAC (Hospital-Acquired Conditions) measures included?
A: Leapfrog believes that for consumers, the choice of a hospital is quite literally life and death. As a result, when measures are not perfectly fair to hospitals, the balance should still tip in favor of public reporting because so much is at stake for patients.
CMS has recently updated the rates for hospital-acquired condition measures used in the hospital safety grade including, air embolism, object retrained after surgery, pressure ulcers, and injuries. Leapfrog supports CMS’s decision to continue to report these measures and considers the availability of this information extremely important for hospital transparency. Leapfrog’s panel of top experts also defends the necessity for increased awareness and accountability for HACs, which at this time is only possible by reporting on these existing measures.
Leapfrog is committed to working with AHA and other stakeholders to improve measures and public reporting systems. But they will also continue to stand for a much higher level of transparency than currently exists, beyond what AHA’s current stated positions advocate.
Q: Are there errors in the data? If so, how do you address the errors and how do you prevent them in the future?
A: Leapfrog freely admits human error. Any organization that takes part in data collection and calculation has to anticipate that errors will inevitably arise, and Leapfrog has done so with the highest standards of integrity and transparency. That is why Leapfrog sent all 2500+ hospitals a copy of the source data used in calculating their grade, as well as the grading methodology, five weeks prior to making the grade public. They have corrected the handful of errors reported, but the vast majority of what Leapfrog reported has been substantiated and stands as originally reported.
Q: How old is the data used in the grade? Is it fair to grade hospitals on old data?
A: The Leapfrog Hospital Safety Grade uses the most recent publicly available data. Hospital Safety Grades are updated on a regular basis as new data is made available in order to ensure the grade accurately reflects a hospital’s safety. The Fall Hospital Safety Grades reflect data mostly from 2015.
Leapfrog offers the opportunity for hospitals to update their information as often as every month through the Leapfrog website so it can be reflected in their next Hospital Safety Grade. They use the latest publicly available data and are happy to work with the hospital community in accelerating the national release of data in the future. Leapfrog is also proud to update the grades every six months, more than any other consumer-facing group, to reflect the improvements hospitals made. Further, Leapfrog encourages hospitals to make additional safety data available on their own websites so consumers and community members can get information on safety and process improvement programs that are currently underway at the hospital, and the results of those programs as they become available.
Q: What is unique about the Leapfrog Hospital Safety Grade?
A: The Leapfrog Hospital Safety Grade is the standard for patient safety. It is compiled by the nation’s leading experts on patient safety and is the only hospital safety assessment to be peer-reviewed in the Journal of Patient Safety.
The Grade is 100 percent transparent: unlike any other rating, you can see all the data that was applied to every graded hospital, as well as the entire methodology.
The Leapfrog Hospital Safety Grade was developed under the guidance of the foremost national experts in patient safety. Nine of the leading patient safety researchers voluntarily advised The Leapfrog Group on the methodology for calculating the Hospital Safety Grade. No other ratings system has the advice from this caliber of experts in the medical and academic fields.
The Leapfrog Group’s only job is to serve as a hospital watchdog: The Hospital Safety Grade is run by the nonprofit The Leapfrog Group which has been rating hospitals for more than a decade.
Unlike media-run scorecards, the Hospital Safety Grade does not rely on advertising, sponsorships or other mass-media concerns to operate. Leapfrog asks patients to consider the Hospital Safety Grade as one factor in decision-making regarding hospital care and encourage patients to consult all available hospital ratings and talk with their doctor.
Q: Is the Leapfrog Hospital Safety Grade biased in favor of hospitals in affluent regions?
A: No. Any hospital can afford to be safe. We were very pleased to see a wide range of hospitals serving all patient populations score an A on the Hospital Safety Grade. These included Academic Medical Centers, rural, urban, and suburban hospitals; safety net hospitals; community hospitals; and nonprofit and for-profit hospitals.
The Hospital Safety Grade uses data publicly available at the national level. Where applicable, measures were risk-adjusted.
Most of the data is not risk-adjusted, however, because it measures issues that should apply regardless of the frailty, economic hardship or other special characteristics of the patient. For instance, foreign objects should never be left in a patient after surgery, all hospitals should have a policy that employees must wash their hands, and hospital leadership should always put a priority on safety.
2014 Leapfrog Hospital Survey Submission DeadlineDate: June 30, 2014 / Time: / Location:
Leapfrog Hospital Safety Score Data Snapshot DateDate: August 31, 2014 / Time: / Location:
Lower Your Costs and Improve Outcomes by Promoting Hospital SafetyDate: November 6, 2014 / 12:45 Time: 12:00 / 12:45 Location:
Closure of 2014 Leapfrog Hospital SurveyDate: December 31, 2014 / Time: / Location:
2015 Leapfrog Hospital Survey submission deadline*Date: June 30, 2015 / Time: / Location:
Snapshot Date: August 31, 2015 (for October 2015 release)Date: August 31, 2015 / Time: / Location:
Last day for 2015 Leapfrog Hospital SurveyDate: December 31, 2015 / Time: / Location:
Virtua Leapfrog Press Event/Presentation of Congressional ProclamationDate: February 16, 2016 / 4:00pm Time: 3:00pm / 4:00pm Location: Virtua Marlton Hospital
Courtesy Data Review Period BeginsDate: March 1, 2016 / Time: / Location:
Spring 2016 Hospital Safety Scores Public AnnouncementDate: April 25, 2016 / Time: / Location: