Skip to main content

America's Best Hospitals: A Devastating Critique

America's leading teaching hospitals anxiously await the release of the annual US News hospital ratings. The "winning" hospitals will heavily market their rankings. However, there is now evidence these rankings are little more than a glorified popularity contest, and of little value as a measure of hospital quality. This critique of "America's Best Hospitals" was published in a study led by Dr. Ashwini Sehgal of Case Western Reserve University in the Annals of Internal Medicine.

I suppose reasonable people could disagree about the value of hospital reputations. And I guess it would be hard to object if US News portrayed America's Best Hospitals as little more than a reputation survey. But the problem is that Americas Best Hospitals is seriously misrepresented as something much more than a reputation survey.

It is suggested that the ranking are a complex amalgam of an evaluation of structure, process, and outcomes, based on "hard data." The Sehgal expose shows that these claims, at least in terms of what influences the final rankings, are essentially rubbish. In contrast to how US News portrays their rankings, the rankings are determined almost entirely by a survey of hospital reputation. A read of the US News methodology would suggest a wide array of objective measures of quality determine these rankings. But the Sehgal report shows all these objective measures matter little, and the only factor that really matters much is the reputation survey.

According to the US News methodology report, the reputation survey accounts for only 32.5% of the total score, while objective measures of hospital quality account for 67.5% of the score. This is totally misleading in terms of how the rankings are operationalized. As Sehgal shows, because their is a huge degree of variability in the reputation survey, and comparably mimimal variability in the objective measures, the final ranking are dominated by the popularity contest.

Sehgal shows that scores on the reputation survey are very poorly correlated with the objective measures of quality. Yet, the final rankings are strongly correlated with the reputation survey and minimally impacted by the objective measures. Amazingly, the Sehgal report shows that for the top 20 hospitals in each specialty, the ranking that would have been given based on the reputation survey is identical to the ranking that is supposedly influenced by objective measures 90% of the time.

Hopefully in 2010, US News will fairly portray America's Best Hospitals as a ranking based almost entirely on reputation. To claim that these rankings are significantly influenced by objective measures of quality and "hard data" is misleading.

That at least gives users the option to decide for themselves how meaningful the reputation survey is to them. I suspect it is worth a little, but just a little. I am sure that there are many outstanding hospitals that are not as well known, and many hospitals not nearly as good as their reputation. And of course, the highly publicized US News rankings may self-perpetuate reputations.

The closing comment goes to Dr. Sehgal who noted in an interview:

"There is virtually no relationship between reputation and quality of care. If we used that approach for professional football we'd say, 'Let's just award the Superbowl to the Dallas Cowboys because they have the reputation of being America's team."

Comments

Dan Matlock said…
What a great article! A very clever author indeed. Another great article criticizing the US News rankings (albeit a bit more subtlely) is: Wennberg JE, Fisher ES, Stukel TA, Skinner JS, Sharp SM, Bronner KK. Use of Hospitals, Physician Visits, and Hospice Care during Last Six Months of Life among Cohorts Loyal to Highly Respected Hospitals in the United States. BMJ 2004;328:607-10. This article led Uwe Rheinhardt to say (paraphrased) "How can the best health care in the world cost twice as much as the best health care in the world?"

Popular posts from this blog

Practical Advice for the End of Life: A Podcast with BJ Miller

This week we talk with BJ Miller, hospice and palliative care physician, public speaker, and now author with Shoshana Berger of the book "A Beginner's Guide to the End."

As we note on the podcast, BJ is about as close as we get to a celebrity in Hospice and Palliative Care.  His TED Talk "What Really Matters at the End of Life" has been viewed more than 9 million times.  As we discuss on the Podcast, this has changed BJ's life, and he spends most of his working time engaged in public speaking, being the public "face" of the hospice and palliative care movement.

The book he and Berger wrote is filled to the brim with practical advice.  I mean, nuts and bolts practical advice.  Things like:
How to clean out not only your emotional house but your physical house (turns out there are services for that!)Posting about your illness on social media (should you post to Facebook)What is the difference between a funeral home and mortuaryCan I afford to die?  …

Improving Advance Care Planning for Latinos with Cancer: A Podcast with Fischer and Fink

In this week's GeriPal podcast we talk with Stacy Fischer, MD and Regina Fink, RN, PhD, both from the University of Colorado, about a lay health navigator intervention to improve advance care planning with Latinos with advanced cancer.  The issue of lay health navigators raises several issues that we discuss, including:
What is a lay health navigator?What do they do?  How are they trained?What do lay health navigators offer that specialized palliative care doesn't?  Are they replacing us?What makes the health navigator intervention particularly appropriate for Latinos and rural individuals?  For advance care planning? Eric and I had fun singing in French (yes French, not Spanish, listen to the podcast to learn why).
Enjoy! -@AlexSmithMD




You can also find us onYoutube!



Listen to GeriPal Podcasts on:
iTunes Google Play MusicSoundcloudStitcher

Transcript

Eric: Welcome to the GeriPal podcast. This is Eric Widera.

Alex: This is Alex Smith.

Eric: And Alex, I'm really excited about toda…

The Dangers of Fleet Enemas

The dangers of oral sodium phosphate preparations are fairly well known in the medical community. In 2006 the FDA issued it’s first warning that patients taking oral sodium phosphate preparations are at risk for potential for acute kidney injury. Two years later, over-the-counter preparations of these drugs were voluntarily withdrawn by the manufacturers.  Those agents still available by prescription were given black box warnings mainly due to acute phosphate nephropathy that can result in renal failure, especially in older adults. Despite all this talk of oral preparations, little was mentioned about a sodium phosphate preparation that is still available over-the-counter – the Fleet enema.

Why Oral Sodium Phosphate Preparations Are Dangerous 

Before we go into the risks of Fleet enemas, lets spend just a couple sentences on why oral sodium phosphate preparations carry significant risks. First, oral sodium phosphate preparations can cause significant fluid shifts within the colon …