Jan 23, 2020
In this health care podcast, I talk with Karl Bilimoria, MD. Dr. Bilimoria is a surgical oncologist and a VP of quality over at Northwestern Medicine. Plus, he is also a John B. Murphy professor of surgery. The second I heard that Dr. Bilimoria and his colleagues had worked on an initiative to “rate the raters” of hospital and physician quality, I reached out to get him on the show. I had just had about four conversations with various people about the difficulties of judging quality. And I had also had a confounding personal experience visiting a patient at a hospital judged a top hospital by a well-known national rating scale. And this “top” hospital had some readily apparent issues, and I am no expert. That got me wondering about the validity of some of these quality raters.
Given the importance and the need for health care quality transparency, Dr. Bilimoria and his colleagues set out to fill this gap by undertaking a (as mentioned) Rating the Raters process to evaluate and compare probably the major publicly reported hospital quality rating systems in the United States. These include the CMS (Centers for Medicare and Medicaid) Hospital Compare Overall Star Ratings, Healthgrades Top Hospitals, Leapfrog Safety Grade and Top Hospitals, and the U.S. News & World Report Best Hospitals.
Interestingly, that “top” hospital I was in was scored a top hospital by one of the lowest-rated raters.
You can learn more at the New England Journal of Medicine Web site, thesecondtrial.org, and the NEJM Catalyst Web site.
Karl Bilimoria, MD, is a
surgical oncologist and a health services, quality improvement, and
health policy researcher at Northwestern University’s Feinberg
School of Medicine. He is the vice president for quality for the
Northwestern Medicine system. He is also the vice chair for quality
in the Department of Surgery and the John B. Murphy professor of
surgery. His clinical practice is focused on melanoma and sarcoma.
Dr. Bilimoria is the director of the Surgical Outcomes and Quality
Improvement Center of Northwestern University (SOQIC), a center of
50 faculty and staff focused on national, regional, and local
quality improvement research and practical initiatives. He is also
the director of the 56-hospital Illinois Surgical Quality
Improvement Collaborative (ISQIC).
Dr. Bilimoria has published more than 350 scientific articles, including numerous publications in JAMA and the New England Journal of Medicine. His research is funded by the National Institutes of Health, the Agency for Healthcare Research and Quality, Health Care Services Corporation, and numerous others. He was recently listed by Becker’s as one of the “Top 50 Experts Leading the Field of Patient Safety” in the United States.
02:06 The impetus for Dr. Bilimoria deciding to rate the
raters.
03:38 How high the stakes are when considering these ratings.
05:01 Breadth vs depth when choosing how and what to measure among
the rating systems.
05:38 What rating systems the Rate the Raters looked at and
why.
06:11 Who got the best scores as a rating system and who got the
worst?
06:58 Dr. Bilimoria and the Rate the Raters’ qualifications on
rating these rating systems.
07:35 The methodology and criteria that the Rate the Raters came up
with to evaluate these rating systems.
08:01 The six criteria that the Rate the Raters system uses to
evaluate rating systems.
08:39 “At every step, we included the rating systems.”
09:14 The intent behind Rate the Raters.
09:55 Why having grades is a positive for the health care
consumer.
10:41 What conflicts of interest might be of concern among these
rating systems?
12:22 “Notable notes” for these rating systems, and what each
rating system incorporates or doesn’t incorporate, and how these
things affected their rating.
18:22 Creating a gold standard and finding ways to move the field
forward.
22:05 Getting to better data in the short term and the long term by
doing meaningful audits.
24:09 Individual ratings vs institutional ratings and where these
intersect.
26:27 Dr. Bilimoria’s advice to employers and how to pick a rating
system.
27:45 “These rating systems should be interpreted very cautiously,
even the best of them.”
28:03 “Triangulate the data.”
30:07 What Dr. Bilimoria and Rate the Raters are currently working
on.
You can learn more at the New England Journal of Medicine Web site, thesecondtrial.org, and the NEJM Catalyst Web site.
Check out our newest #healthcarepodcast with @kbilimoria. #healthcare #podcast #digitalhealth #healthoutcomes #healthcarequality #qualitymetrics #healthdata
What started #RatetheRaters? @kbilimoria discusses on our #healthcarepodcast. #healthcare #podcast #digitalhealth #healthoutcomes #healthcarequality #qualitymetrics #healthdata
When considering #healthratings, how high are the stakes? @kbilimoria discusses on our #healthcarepodcast. #healthcare #podcast #digitalhealth #healthoutcomes #healthcarequality #qualitymetrics #healthdata
Breadth vs depth in #healthrating systems. @kbilimoria discusses on our #healthcarepodcast. #healthcare #podcast #digitalhealth #healthoutcomes #healthcarequality #qualitymetrics #healthdata
What #healthrating systems did #RatetheRaters look at and why? @kbilimoria discusses on our #healthcarepodcast. #healthcare #podcast #digitalhealth #healthoutcomes #healthcarequality #qualitymetrics #healthdata
How did each #healthrating system score with #RatetheRaters? @kbilimoria discusses on our #healthcarepodcast. #healthcare #podcast #digitalhealth #healthoutcomes #healthcarequality #qualitymetrics #healthdata
What are the #RatetheRaters qualifications in assessing these #healthrating systems? @kbilimoria discusses on our #healthcarepodcast. #healthcare #podcast #digitalhealth #healthoutcomes #healthcarequality #qualitymetrics #healthdata
What evaluating criteria do #RatetheRaters use? @kbilimoria discusses on our #healthcarepodcast. #healthcare #podcast #digitalhealth #healthoutcomes #healthcarequality #qualitymetrics #healthdata
The six criteria to evaluating #healthrating systems. @kbilimoria discusses on our #healthcarepodcast. #healthcare #podcast #digitalhealth #healthoutcomes #healthcarequality #qualitymetrics #healthdata
“At every step, we included the rating systems.” @kbilimoria discusses on our #healthcarepodcast. #healthcare #podcast #digitalhealth #healthoutcomes #healthcarequality #qualitymetrics #healthdata
What’s the intent behind #RatetheRaters? @kbilimoria discusses on our #healthcarepodcast. #healthcare #podcast #digitalhealth #healthoutcomes #healthcarequality #qualitymetrics #healthdata
Are grades a positive for #healthconsumers? @kbilimoria discusses on our #healthcarepodcast. #healthcare #podcast #digitalhealth #healthoutcomes #healthcarequality #qualitymetrics #healthdata
Creating a gold standard. @kbilimoria discusses on our #healthcarepodcast. #healthcare #podcast #digitalhealth #healthoutcomes #healthcarequality #qualitymetrics #healthdata
Individual ratings vs institutional ratings. @kbilimoria discusses on our #healthcarepodcast. #healthcare #podcast #digitalhealth #healthoutcomes #healthcarequality #qualitymetrics #healthdata
“These rating systems should be interpreted very cautiously, even the best of them.” @kbilimoria discusses on our #healthcarepodcast. #healthcare #podcast #digitalhealth #healthoutcomes #healthcarequality #qualitymetrics #healthdata
“Triangulate the data.” @kbilimoria discusses on our #healthcarepodcast. #healthcare #podcast #digitalhealth #healthoutcomes #healthcarequality #qualitymetrics #healthdata