How good a hospital is St. Mary Mercy Livonia Hospital? Depends on whom you ask.
The Leapfrog Group, a respected nonprofit that promotes patient safety, gave an “A” to this Michigan hospital. The company Healthgrades named it one of America’s best 50 hospitals.
But the Joint Commission, a nonprofit organization that accredits hospitals, and U.S. News and World Report omitted St. Mary from their best hospital lists. Consumer Reports gave it an average safety score of 47 points out of 100, citing high numbers of readmissions, poor communication with patients and excessive use of scans. Medicare, which has a new program rewarding hospitals for meeting certain quality measures, is reducing St. Mary’s payments by a fraction this year.
Evaluations of hospitals are proliferating, giving patients unprecedented insight into institutions where variations in quality can determine whether they live or die. Many have similar names, such as “Best Hospitals Honor Roll,” “America’s Best Hospitals” and “100 Top Hospitals.” Illinois, Florida and other states have created their own report cards. In some places, such as California, there are more than a dozen organizations offering assessments on hospital quality.
But those ratings, each using its own methodology, often come to wildly divergent conclusions, sometimes providing as much confusion as clarity for consumers. Some hospitals rated as outstanding by one group are ignored or panned by another. Ratings results from an individual group can change significantly from year to year.
“We’ve alternatively been labeled the least safe hospital in Maine and the safest hospital in Maine,” said Dr. Douglas Salvador, vice president of quality at Maine Medical Center in Portland.
And the ratings do not always jibe with the views of authorities who oversee hospitals. For instance, UCSF Medical Center has gotten good grades from multiple safety raters even as California public health officials have fined it $425,000 repeatedly for endangering patients.
As ratings multiply, more and more hospitals have something they can boast about. A third of U.S. hospitals—more than 1,600 — last year won at least one distinction from a major rating group or company, according to a Kaiser Health News analysis. In the greater Fort Lauderdale hospital market, 21 of 24 hospitals were singled out as exemplary by at least one rating source. In the Baltimore region, 19 out of 22 hospitals won an award.
“I worry a lot about these ratings,” said Jerod Loeb, executive vice president for health care quality evaluation at the Joint Commission. “They’re all justifiable efforts to provide information, but at the end of the day every single one of them is flawed in some respect. Rather than enlightening, we may be confusing.”
Not A ‘Complete Picture Of The Care’
There are so many report cards on hospitals that the Informed Patient Institute runs a website that grades the raters. Carol Cronin, its executive director, said most report cards are not easy for consumers to use. “A lot of them don’t help users quickly understand which hospital is better than another,” she said.
But many hospitals are eager to trump these distinctions in their marketing. Healthgrades, U.S. News and Leapfrog not only encourage this but also profit from it by charging licensing fees to hospitals that want to advertise their awards. “A hospital cannot buy an award, they must achieve it,” Healthgrades said in a statement.
Healthgrades and Truven Health Analytics, which publishes the 100 Top Hospitals, offer consulting services to hospitals that want to improve their overall performances. Jean Chenoweth, a Truven senior vice president, said the list doesn’t earn Truven any money but it “gives the company a lot of visibility.”
Dr. Andrew Brotman, chief clinical officer at NYU Langone Medical Center in Manhattan, said the fees can be substantial. “Healthgrades, which is one we did well on, charges $145,000 to use this even on the website as a logo, so we don’t do that,” he said. “U.S. News is in the $50,000 range. Leapfrog is $12,500.”
Consumer Reports bars hospitals from using its ratings in marketing, but patients must subscribe to read them online. (Others generally provide free access to ratings on their sites.) The Joint Commission does not charge hospitals that make its top quality list.
A Pew Research Center survey found 14 percent of Internet users consulted online rankings or reviews of hospitals and medical facilities.
Florence Harvey, 70, said when she moved to Washington, D.C., last fall, she picked a health plan and doctor affiliated with Washington Hospital Center after reviewing all the local hospitals rankings on U.S. News’ website. “That was the one that had the best across-the-board ratings,” she said.
But Harvey may be an anomaly. Dr. Peter Lindenauer, a professor with Tufts University School of Medicine based in Springfield, Mass., said the limited research on rankings “suggests they havehad very little impact on patient behavior.”
That’s not surprising since many admissions, such as those due to a heart attack or car crash, have an immediacy that rules out comparison shopping. Also, researchers note, many patients defer to their physicians’ recommendations or go to the hospital where their chosen surgeon has privileges. Still, rating groups say the ratings help keep the pressure on hospital executives to keep quality up.
“Patient safety has to be a priority 24-7,” said Leah Binder, Leapfrog’s president. “The minute it slips off the priority list, that impacts the rating.”
The calculations that go into these ratings are complex. Most hospital assessments synthesize dozens of pieces of data Medicare publishes on its Hospital Compare website, including death rates and the results of patient satisfaction surveys. They also examine other sources and use private surveys to create user-friendly lists or grades, which they display on their websites.
The Joint Commission looks at how frequently patients received recommended treatments, such as flu shots for those with pneumonia. Consumer Reports examines the numbers of patients who die or are readmitted, infection rates and Medicare patient surveys of their experiences. Leapfrog looks at data from its surveys of hospitals, the consistency with which hospitals followed safe surgical practices and frequencies of infections and some types of patient harm. Healthgrades analyzes detailed Medicare records to find death and complication rates for 27 procedures and conditions.
“Ratings and ranking programs certainly offer people information they can use to make their hospital selections, but we don’t recommend relying on any one of them completely,” Jennifer Kennedy, a spokeswoman for St. Mary Mercy, said in an e-mail. “None are able to tell the whole story or paint a complete picture of the care that is delivered.”
Patient safety rankings are based on how frequently correct procedures or errors occur, so hospitals can get good grades even if they have made some egregious errors. Leapfrog gave a “B” to UCSF Medical Center in San Francisco even though California regulators have penalized UCSF eight times for infractions since 2008, most recently for leaving a sponge in one patient and a plastic clip in the skull of another. Those errors cost UCSF $200,000 in fines.
Dr. Josh Adler, the chief medical officer at UCSF, said penalties were partly a result of the hospital’s policy to make sure that all errors are reported to authorities. “The key is that we constantly strive to deliver the highest quality, safest, and most satisfying care, and that we are a learning organization,” he said in an e-mail.
The ratings groups believe the public benefits from the multitude of ratings. Dr. John Santa, who directs Consumer Reports’ health ratings, said consumers benefit from different vantages just as they do for cars or electronic devices, and the competition spurs each rating group to get better. “We think that’s consistent with good science,” Santa said.
Avery Comarow, health rankings editor for U.S. News, agrees. “People go to hospitals for different reasons and priorities,” he said. “I’m not sure there could be a single rating system that can do it all.”
Many of the hospital graders are expanding their awards. Last year, the Joint Commission identified 620 hospitals as “top performers,” up from 406 the previous year. Healthgrades now provides awards for emergency rooms, maternity care, pediatric care, bariatric surgery and gynecology surgery. In 2011, U.S. News started identifying the best hospitals in regions of the country, and identifies 748 hospitals as a “best” hospital in at least one specialty.
Constant Turnover
Much of the quality data is rudimentary, as the science of evaluating hospitals is still in its adolescence. Adding to the confusion is that hospitals can rise and fall from year to year as groups tweak their methods of assessment and as hospital performance shifts, even slightly. A study sponsored by the Commonwealth Fund, a health care philanthropy in New York, found that only 46 percent of hospitals ranked as top performers by Thomson Reuters in 2008 were also winners in 2007. In the Joint Commission’s rankings, 583 hospitals missed being designated a top performer because they fell short on just one of 45 measures.
In some cases, hospitals that have won awards are being penalized financially by Medicare for falling short on the government’s quality assessment. Saint Francis Hospital in Tulsa, Okla., which is losing 0.54 percent of its Medicare payments this year under the government’s quality program, is a Healthgrades Distinguished Hospital for Clinical Excellence and was ranked the second best hospital in Oklahoma by U.S. News.
Some of the hospitals that do the best in the rankings have limited respect for them. Advocate Christ Medical Center in Oak Lawn, Ill., last year received praise from Leapfrog, U.S. News, the Joint Commission, Truven and Healthgrades. But Dr. William Adair, vice president for clinical transformation, says the hospital doesn’t license any of the distinctions. “We’re all made a little bit uneasy, to be frank about it,” Adair says. “Some of these organizations are looking for revenue. It blurs the effectiveness of the ratings processes.”
Still, many hospitals are happy to use the praise. Dr. Brotman from NYU said: “Even though there’s not a hospital executive who won’t tell you that they have a great deal of skepticism about a lot of the methodology, there’s not one who will tell you they don’t want to be on the lists.”
For more information on national ratings of hospitals:
Medicare’s Hospital Compare
This government site publishes dozens of quality measures that private ratings groups use for their own rankings. http://www.medicare.gov/hospitalcompare/ It won’t compare more than three hospitals at a time and it doesn’t synthesize the data into one overall rating.
Consumer Reports
http://www.consumerreports.org/health/doctors-hospitals/hospital-ratings.htm This nonprofit calculates a “safety score” on a 100-point scale based on infection, error and readmission rates as well as good communication from doctors and nurses. You need to subscribe to see the data.
Healthgrades
This private company rates hospitals in up to two dozen specialties and provides some of the Hospital Compare data in a user-friendly format. http://www.healthgrades.com/quality/top-hospitals-2013 Healthgrades also gives out awards and distinctions in various specialties.
The Joint Commission
This accrediting group tells you if a hospital has passed its certification reviews. http://www.qualitycheck.org/consumer/searchQCR.aspx It also gives out awards in areas such as stroke and heart care. Its “top performers” list honors hospitals that follow proper procedures but doesn’t assess whether treatments were successful. http://www.jointcommission.org/accreditation/top_performers.aspx
Leapfrog Hospital Safety Score
http://www.hospitalsafetyscore.org/ This nonprofit group uses information from its own surveys as well as government data to come up with a letter grade for how well hospitals protect patients from unnecessary harm.
Truven Health Analytics 100 Top Hospitals
http://www.100tophospitals.com/
This private company selects best hospitals in five categories based on their size and whether they are teaching institutions or community hospitals. It also picks 15 top hospital systems and best cardiovascular.
U.S. News and World Report http://health.usnews.com/best-hospitals
Alone among raters, this media company surveys specialists for their opinions in choosing the best national and regional hospitals in different specialties. For 12 specialties, reputation makes up a third of the score, with data determining the rest. In four specialties the ratings are entirely based on reputation. U.S. News also ranks best children’s hospitals.
Kaiser Health Newshttp://www.kaiserhealthnews.org/ is an editorially independent program of the Henry J. Kaiser Family Foundation, a nonprofit, nonpartisan health policy research and communication organization not affiliated with Kaiser Permanente.