Wednesday, August 24, 2016
Every year US News and World Report ranks the top colleges and hospitals in the US. Those that are ranked high use these results to market their institutions to potential students and/or patients. I've been asked by a friend to comment on the method that US News and World Report uses to create the annual rankings of hospitals in the US.
According to their FAQ page for these rankings the rankings are based on a summary score from 16 hospital specialties. Twelve of these specialties (cancer, cardiology & heart surgery, diabetes & endocrinology, ear, nose & throat, gastroenterology & GI surgery, geriatrics, gynecology, nephrology, neurology & neurosurgery, orthopedics, pulmonology and urology) have objective health outcomes and are used to rank hospitals within specialties and are factored into the rankings by health outcome data such as survival rates. Another 4 specialties (ophthalmology, psychiatry, rehabilitation and rheumatology) have outcomes that are more subjective so surveys of physician specialists are used to rank hospitals within specialties and factored into overall rankings.
I have two comments on these rankings. The first comment is anytime a ranking is based on a summary score of different measures invariably some information is lost. A hospital can have high ratings in geriatrics, ophthalmology and neurology while another can have similar high rankings in gynecology, cancer and psychiatry. If the rankings for both hospitals on the other measures are the same, the two hospitals can have roughly the same overall rankings. The overall rankings say little about the hospitals relative strengths and weaknesses which are different.
Even within specialties outcomes can vary widely for different diseases. There are many types of diseases that each specialty treats. Overall rankings for specialties can vary widely among different diseases. If one has a specific disease it is better to look at how that hospital treats that disease.
The second comment is that the four subjective specialties that are ranked based on a survey of physicians perceptions of which hospitals are the best at these specialties. Presumably the physicians can rank their hospital first and can choose ones simply based on reputation (eg. Massachusetts General or the Cleveland Clinic). Also the respondents may have no actual firsthand knowledge of the hospital's performance in these specialties. A survey of the patients of the specialties of the respective hospitals may be more accurate but such a survey would not be practical or ethical to conduct.
The video of my presentation at PodCamp can be seen on YouTube and here. The slides are presented below.