"They Said What?": Patient-Initiated Internet Reviews of Physicians

This article was written by Jon Christianson, PhD, Medica Research Institute senior fellow, and James A. Hamilton chair in health policy and management at the School of Public Health at the University of Minnesota.

The amount of publicly available information about physician performance is exploding, and it is taking 2 forms¹: formal reports and patient-initiated ratings. Formal reports, constructed using claims and medical records data, are pushed out to the public on the internet. For instance, Medicare maintains websites that provide basic information on physician characteristics and disclose payments to physicians by pharmaceutical manufacturers. Health plans offer information to their members on quality of care and, less often, on average physician costs for treating specific conditions.^2,3 Voluntary community stakeholder coalitions host public websites that contain quality measures on physicians, clinics, or medical groups.⁴

This information is an important part of the burgeoning “transparency movement,” but surveys find relatively low awareness among consumers that these reports exist. Among those who are aware, only a minority state that they find the information useful.⁵ They say that the measures in the reports are too technical, are not presented in a way that is understandable, and do not address issues that they truly care about. Consequently, consumers continue to seek information about physicians from family, friends, and other sources.^6,7 And, it is increasingly common for them to search for internet sites with patient-initiated ratings.

Patient Report Websites

For a relatively new development, there are now many (upwards of 408) “dedicated” internet sites that invite consumers to provide feedback on their experiences in physician offices. The most popular—as measured by number of visitors—are Healthgrades, Ratemds, Ucomparehealthcare, Vitals, and Zocdoc.⁹ More general sites, such as Angie’s List, also offer opportunities to comment on or rate physicians. Visitors to these sites can express their opinions regarding individual physicians using their own words. On some sites, users are asked to answer a relatively small number of pre-formatted questions in order to generate physician rating scores.⁹

The lure of these sites for consumers is that they can see what others are saying about specific physicians. These unscientific evaluations seem to fill a gap for consumers.

Perhaps information on these sites augments, or even replaces, the traditional “over the back fence” conversations with neighbors about who is a good doctor and who should be avoided. The online process is familiar to consumers who use similar sites when choosing movies, restaurants, automobiles, home remodelers, and the like.⁹ In fact, many consumers turn to these sorts of sites when making purchases, suggesting the value they place on the reported experiences of others. One survey reports that 34% of consumers select physicians based on patient reviews, compared to 9% that use more formal quality ratings.¹⁰

Not surprisingly, patient report websites typically are not held in high regard by the medical profession. For instance, one former president of the American Medical Association recommends that “anonymous online opinions of physicians should be taken with a grain of salt.”³ Physicians react more negatively to patient report websites than to the scientifically based, externally vetted performance measures disseminated by third parties in part because “its personal.”¹¹ Patient comments certainly strike closer to home than, for instance, a measure of the percent of patients seen in the physician’s medical group receiving a recommended vaccination. And, as well as affecting their reputations, negative patient reports might reduce the demand for their services.

Studies on Patient Report Websites

Beyond the fact that they can be disconcerting to physicians, what do we know about patient-report websites? Actually, more than one might suppose. While there is a growing body of research that addresses the limitations, usefulness, and effectiveness of formal reports of physician performance, a similar literature is emerging that focuses on patient report websites.^12,13 These studies have received less attention than they deserve, possibly because most have appeared in physician specialty journals (eg, Ellimoottil et al, 2013)¹⁴ or in the Journal of Medical Internet Research, where readership is limited. However, their findings shed light on the people who use patient report sites, site content, and the validity of physician concerns. The study results are sometimes surprising.

Who Uses Patient Report Websites?

Use of patient report websites is substantial, with over 7 million people a month visiting Healthgrades alone.¹⁵ In general, studies report that users of these websites are disproportionately female, middle aged (45 to 64 years), and college educated. As one might expect, they also are relatively high users of health care.^9,13,16 However, there is some conflicting evidence about age, with one study reporting that middle-aged people are more likely to use the sites,⁹ and another suggesting that younger people are more likely to use them.⁶

These findings, while somewhat sketchy, do not support the notion that patient report websites are principally the domain of young internet-savvy consumers, although people who have a higher opinion of their own internet skills are more likely to be users.¹⁶

Different website characteristics appear to attract different age groups. For example, Zocdoc is more popular with younger consumers. Interestingly, frequency of daily internet use does not seem to be associated with use of patient report websites.¹⁶ Female visitors to rating sites frequently have children, so at least some may be searching for information to use in making decisions about their child’s health care. Thirty percent or more of the visitors to some sites are international, for reasons that are not clear.⁹

How Often Are Physicians Reviewed on Patient Report Websites?

Physicians are unlikely to be rated even once on any given site.³ When a physician is rated, the median number of individual ratings is about 4,¹⁷ and it is very common for physicians to have a single rating.¹³ At first glance, physicians may view this as reassuring—the number of negative reviews must, by definition, be small. However, because of the small number of reviews per physician, a single bad review by a disgruntled patient could receive more attention than is warranted or than it would receive if there were a larger number of ratings.

What Do Patients Typically Say About Physicians?

Understandably, physicians fear that their unhappiest patients will post comments on patient report websites (most sites permit anonymous reviews). They fear that these negative reviews will create a false impression for prospective patients and among the physician’s colleagues as well. But this doesn’t seem to be the case.

Physicians should be reassured by the fact that multiple studies report comments on patient report websites to be mainly positive.^12,9,14 For example, a study of parents who visited these sites seeking information about child health care found positive comments outnumbered negative comments by 3 to 1.¹⁸ This may reflect consumer concerns that their comments are not truly anonymous and that a negative review could result in physicians taking some action against them.¹⁹

When negative reviews do exist, results from one study suggest that consumers are relatively sophisticated in interpreting them. For example, when only a few reviews are posted, and there is a negative review, consumers place more credence on fact-based reviews and tend to discount emotional reviews.²⁰ As one site visitor reported, most reviews “are pretty sober, boring, and don’t sway me one way or the other.”²¹

Is There a Relationship Between Patient Evaluations and Traditional Quality Measures?

Patient reports regarding physicians provide a different perspective than clinical performance measures in formal reports. Postings on patient report sites also differ from measures of consumer experience based on surveys that ask patients to respond to a validated battery of questions. This raises the question of how well patient reports and rankings might correlate with the other 2 types of physician performance measures.¹²

Researchers have found no statistically significant correlation between patient reports and technical measures of quality. For instance, in a recently published study in JAMA Internal Medicine, Gray and colleagues (2015) found “no evidence that physician website ratings were associated with clinical QMs [quality measures].”²² But they did find a small statistically significant association with survey-based patient experience measures. In their work, they did not examine the content of written comments posted by patients. They point out that the lack of association with clinical quality measures is not unexpected or bad. It may simply reflect fundamental differences in the dimensions of physician performance being measured. Some of the same researchers found a positive correlation between patient reports and structural quality measures, such as board certification and malpractices claims.²³

How Should Physicians Respond to Patient Report Websites?

Given the angst physicians have regarding patient report sites it’s reasonable that they might want to take action. However, there is very little they can, or should, do.

Patient-initiated physician ratings and comments are flawed in ways that should discourage consumers from using them exclusively when evaluating physicians. For example, at present the relatively small number of patients rating any one physician suggests that the result may be unreliable and possibly biased. And there is the question of whether the rater is actually one of the physician’s patients.

Organized medicine could seek agreement with website hosts regarding the conditions under which assessments would be revealed (eg, requiring some minimum number of patient-commentators). However, success seems unlikely, as there is a growing thirst for patient initiated information among consumers, and the number of sites appears to be growing, each with its own set of protocols.

Physicians could alert patients to the lack of scientific grounding of information on patient report websites. No doubt this would surprise some patients who might not have known that the sites existed. But this strategy is not likely to discourage patients from seeking information on these sites. It’s likely that patients know the information is not scientifically based but nevertheless feel it has value, much like the value they place on conversations with neighbors.

A third possibility is that physicians could enter the websites to respond to negative comments, much in the manner that manufacturers and restaurants do occasionally. Patient report websites don’t always permit this, however. And because comments are anonymous, it is not possible for physicians to communicate directly with a disaffected patient to address his or her problems.

At the extreme, it is probably not a good strategy for physicians to sue patients over the contents of reviews, as “when a physician sues a patient, it looks like they [sic] have something to hide.”¹⁰ It has been suggested that physicians should monitor the comments on the sites in order to glean something positive from them.¹⁰ For example, there may be information that could lead physicians to restructure work flows or improve staff interactions with patients or sharpen their own communication skills. Monitoring of sites can be a tough prescription for physicians, as they could learn that some of their patients do not hold them in high esteem.

What Does the Future Hold for Patient Report Websites?

Sectors of the economy where consumer service is an essential part of the product provide some insight into what the future holds regarding patient report websites. Looking forward, this experience suggests that physicians can expect more patients to use patient report sites, and that the information on these sites will play an ever larger role in consumer decision-making. The number of patient report sites could increase (some health plans now host sites and encourage members to use them) or decline, as consumers sort through the usefulness of different sites. In either case, the number of patient-generated reports likely will increase, physician rating apps will be easily accessible on smart phones,²⁴ and researchers may be able to mine the data across websites to uncover poor quality of care.²⁵

With growing research on the impact and implications of patient report sites, our understanding of their influence on relations between physicians and patients is likely to increase as well. For this to happen, however, we may all have to master a new language. We will need to fight through the thicket of jargon that seems to be endemic to this developing field of study. In the future, physicians, policymakers, and other observers will need to be conversant in “infodemiology,”²⁶ speak knowingly about the “cloud of patient experience” and “sentiment analysis,”²⁵ and express concerns about “crowd validation.”¹² No matter—it will be worth learning a new language if it allows us to fully exploit this new source of information for its policy and practice implications.

References

1. Shannon D. Publicly reported physician ratings: here to stay but not yet ready for prime time. Physician Exec. 2013:39(1);14-16, 18-20, 22-16.

2. Blue Cross and Blue Shield of Minnesota. Blue Cross and Blue Shield of Minnesota Announces the SmartSelect Health Care Shopping Tool to Help Minnesotans Make More Informed Health Care Choices. Our Company: Newsroom. Retrieved from https://www.bluecrossmn.com/Page/mn/en_US/newsroom-landing-page/News/blue-cross-annouces-smartselect.

3. Lieber R. The Web is Awash in Reviews, but Not for Doctors. Here's Why. The New York Times. Retrieved from http://www.nytimes.com/2012/03/10/your-money/why-the-web-lacks-authoritative-reviews-of-doctors.html?_r=0

4. Christianson JB, Volmar KM, Alexander J, Scanlon DP. A report card on provider report cards: current status of the health care transparency movement. [Research Support, Non-U.S. Gov't]. J Gen Intern Med. 2010:25(11);1235-1241. doi: 10.1007/s11606-010-1438-2.

5. Sinaiko AD, Eastman D, Rosenthal MB. How report cards on physicians, physician groups, and hospitals can have greater impact on consumer choices. Health Aff (Millwood). 2012:602-611. doi: 10.1377/hlthaff.2011.1197

6. Andrews M. Rating Your Doctor Kaiser Health News: Henry J. Kaiser Family Foundation. 2010.

7. Carlin CS, Kralewski J, Savage M. Sources of information used in selection of surgeons. Am J Manag Care, 2013.19(8);e293-300.

8. Kowalcyzk L. Looking for Dr. Right; With More and More Websites Rating Physicians, the Question Is: Can You Trust Them? The Boston Globe. Retrieved from http://www.boston.com/lifestyle/family/articles/2009/06/08/with_more_and_more_websites_rating_physicians_the_question_is_can_you_trust_them/. 2009.

9. Merrell JG, Levy BH 3rd, Johnson DA. Patient assessments and online ratings of quality care: a "wake-up call" for providers. Am J Gastroenterol. 2013:108(11);1676-1685. doi: 10.1038/ajg.2013.112.

10. Rosin T. Consumers Value Objectivity Over Accuracy When Seeking Medical Information: 8 Findings. Becker's Hospital Review. Retrieved from http://www.beckershospitalreview.com/hospital-physician-relationships/consumers-value-objectivity-over-accuracy-when-seeking-medical-information-8-findings.html. 2014.

11. Rosin T. How to Handle Negative Online Reviews. Becker's Hospital Review. Retrieved from http://www.beckershospitalreview.com/hospital-physician-relationships/how-to-handle-negative-online-reviews.html. 2014.

12. Verhoef LM, Van de Belt TH, Engelen LJ, Schoonhoven L, Kool RB. Social media and rating sites as tools to understanding quality of care: a scoping review. [Research Support, Non-U.S. Gov't Review]. J Med Internet Res. 2014:16(2);e56. doi: 10.2196/jmir.3024.

13. Emmert M, Sander U, Pisch F. Eight questions about physician-rating websites: a systematic review. [Review]. J Med Internet Res. 2013:15(2);e24. doi: 10.2196/jmir.2360.

14. Ellimoottil C, Hart A, Greco K, Quek ML, Farooq A. Online reviews of 500 urologists. J Urol. 2013:189(6);2269-2273. doi: 10.1016/j.juro.2012.12.013.

15. Bumpass DB, Samora JB. Understanding Online Physician Ratings. AAOS Now. Retrieved from http://www.aaos.org/AAOSNow/2013/Sep/advocacy/advocacy4/?ssopc=1. 2013.

16. Terlutter R, Bidmon S, Rottl J. Who uses physician-rating websites? Differences in sociodemographic variables, psychographic variables, and health status of users and nonusers of physician-rating websites. J Med Internet Res. Retrieved from http://www.jmir.org/2014/3/e97/. 2014:6(3);e97. doi: 10.2196/jmir.3145.

17. Health Care Team. Hospital and Physician Marketing in the Era of (Nearly) Meaningless Online Ratings. Media Logic. Retrieved from http://www.medialogic.com/health-care-marketing/blog/hospital-physician-marketing-era-nearly-meaningless-online-ratings/. 2014.

18. Hanauer DA, Zheng K, Singer DC, Gebremariam A, Davis MM. Parental awareness and use of online physician rating sites. [Randomized Controlled Trial Research Support, Non-U.S. Gov't]. Pediatrics. 2014:134(4);e966-975. doi: 10.1542/peds.2014-0681.

19. Hanauer DA, Zheng K, Singer DC, Gebremariam A, Davis MM. Public awareness, perception, and use of online physician rating sites. [Research Support, Non-U.S. Gov't]. JAMA. 2014:311(7);734-735. doi: 10.1001/jama.2013.283194.

20. Grabner-Krauter S, Waiguny MK. Insights into the impact of online physician reviews on patients' decision making: randomized experiment. J Med Internet Res. 2015: 9;17(4);e93. doi: 10.2196/jmir.3991.

21. Williams D. The State of Online Doctor Ratings: It's still Early. KevinMD.com. Retrieved from http://www.kevinmd.com/blog/2013/04/state-online-doctor-ratings-early.html. 2013.

22. Gray BM, Vandergrift JL, Gao G, McCullough JS, Lipner RS. Website ratings of physicians and their quality of care. JAMA Intern Med. 2015:175(2);291-3. doi: 10.1001/jamainternmed.2014.6291.

23. Gao GG, McCullough JS, Agarwal R, Jha AK. A changing landscape of physician quality reporting: analysis of patients' online ratings of their physicians over a 5-year period. J Med Internet Res. 2012:14(1);e38. doi: 10.2196/jmir.2003.

24. Bidmon S, Terlutter R, Rottl J. What explains usage of mobile physician-rating apps? Results from a web-based questionnaire. J Med Internet Res. 2014:16(6);e148. doi: 10.2196/jmir.3122.

25. Greaves F, Ramirez-Cano D, Millett C, Darzi A, Donaldson L. Harnessing the cloud of patient experience: using social media to detect poor quality healthcare. [Research Support, Non-U.S. Gov't]. BMJ Qual Saf. 2013:22(3);251-255. doi: 10.1136/bmjqs-2012-001527.

26. Kim M, Jung Y, Jung D, Hur C. Investigating the congruence of crowdsourced information with official government data: the case of pediatric clinics. J Med Internet Res. 2014:16(2);e29. doi: 10.2196/jmir.3078.