Telephone Follow-Up on Medicare Patient Surveys Remains Critical

ABSTRACT

Objectives: Patient experience surveys are essential to measuring patient-centered care, a key component of health care quality. Low response rates in underserved groups may limit their representation in overall measure performance and hamper efforts to assess health equity. Telephone follow-up improves response rates in many health care settings, yet little recent work has examined this for surveys of Medicare enrollees, including those with Medicare Advantage. Our objective was to describe response rates to the 2022 Medicare Consumer Assessment of Healthcare Providers and Systems (MCAHPS) surveys and the completion mode (mail or telephone), overall and by person-level characteristics.

Study Design: Cross-sectional survey.

Methods: Participants were 1,092,434 individuals with Medicare who were selected to receive the 2022 MCAHPS survey in the 50 states and the District of Columbia and who were representative of the Medicare population. Study measures were survey response and completion mode.

Results: The overall response rate was 33.7% (31.3% by mail and 2.3% by telephone), with 6.9% of responses by telephone. Despite the low overall telephone response rate, the phone was used at markedly higher rates by respondents in some groups with lower overall response rates who are thus underrepresented among respondents, including those who were younger than 65 years (eligible for Medicare due to disability: 16.5% of responses by telephone), Black (16.1%), or Hispanic (14.1%) or had limited income and assets (14.6%).

Conclusions: Including a telephone component in the administration of the MCAHPS survey continues to have value because several groups still show a relative preference for survey completion by telephone. Steps should be taken to improve response rates by telephone.

Am J Manag Care. 2025;31(1):In Press

Takeaway Points

• Including a telephone component in the administration of the Medicare Consumer Assessment of Healthcare Providers and Systems survey continues to have value.
• Telephone responses continue to comprise a substantial portion of responses for several underserved groups of Medicare enrollees.
• The telephone mode, although declining in use, is likely to remain one of the important ways to include the experiences of these groups in patient experience surveys.
• Steps should be taken to improve response rates of Medicare enrollees by telephone.
• Efforts to use this mode effectively are likely to be important to measuring progress toward health equity goals.

Patient centeredness is a key dimension of health care quality,¹ which is measured² through standardized, validated patient experience surveys.³ Patient experience survey data are used to compare quality across providers, facilities, and plans and by person-level characteristics, such as race/ethnicity, age, sex, and income or asset status. The latter comparisons assess health equity, defined as the absence of unfair, avoidable, or remediable differences in health and well-being among groups of people.⁴

One criticism of patient experience surveys is that response rates (RRs) are often low and, like survey RRs more generally,⁵ have declined over time.³ Evidence from numerous patient experience surveys indicates that patients with poorer experiences, whose feedback is critical for informing quality improvement, are less likely to respond.^6,7 Survey nonresponse can bias provider comparisons unless appropriate adjustments are made; such adjustments may include case-mix adjustment of scores using factors related to nonresponse.³ Another concern is that RRs often vary by group, with lower rates in underserved groups.⁸ This may result in respondents being unrepresentative of the target population and inadequate sample sizes to assess health equity.

Telephone follow-up of patient experience surveys has been found to improve RRs overall, especially for underserved populations in many health care settings (eg, adult inpatient).^2,9 However, few recent studies have examined telephone follow-up as a component of survey administration with Medicare enrollees, including those with Medicare Advantage (MA).

The Medicare Consumer Assessment of Healthcare Providers and Systems (MCAHPS) surveys measure patient experience among individuals with Medicare. Medicare provides health care insurance to 66 million individuals,¹⁰ including most Americans 65 years and older and those aged 18 to 64 years with certain disabilities.¹¹ MCAHPS patient experience measures are used to publicly report contract-level scores for MA and prescription drug plans (PDPs),¹² as a basis for quality bonus payments for MA contracts, and to compare MA with fee-for-service (FFS) Medicare.¹³ MCAHPS patient experience measures are also publicly reported by race/ethnicity, sex, and rurality.¹⁴

Many CAHPS surveys are fielded using a mail-telephone mode (ie, mail survey with telephone follow-up for mail nonrespondents). Because different members of a survey’s target population may have different propensities to respond by a given mode, such mixed modes tend to have higher RRs than single-mode protocols, including web-only, mail-only, and telephone-only protocols.^6,15 To improve representativeness and sample sizes overall and for health equity work, it is useful to identify groups with low RRs and to understand their preferred survey completion modes.

An analysis of the 2007 fielding of the MCAHPS surveys by Klein et al found a 49% RR among eligible sampled individuals, with 34% responding by mail and 15% by telephone.¹⁶ Adjusted RRs were highest for individuals aged 55 to 80 years and lowest for those aged 18 to 44 years and older than 84 years. Adjusted RRs were higher among White enrollees than among comparable Asian American and Native Hawaiian or other Pacific Islander (AA/NHPI) (16 percentage points [PP] lower), Hispanic (9 PP lower), and Black (7 PP lower) enrollees. Adjusted RRs among individuals with dual eligibility (DE) for Medicare and Medicaid, which is a marker of limited income and assets, were 7 PP lower than among individuals not eligible for Medicaid. Propensity to use one of the available modes varied among respondents, with individuals who were younger, Black, or Hispanic or had DE coverage most likely to respond later by telephone rather than earlier by mail. AA/NHPI respondents were more likely than White respondents to respond by mail than telephone.

As with surveys generally, RRs to MCAHPS surveys have declined over time.¹³ In this article, we update the Klein et al article,¹⁶ describe patterns of response to the 2022 MCAHPS surveys and the completion mode among respondents, and add to the literature by examining whether including a telephone component of survey administration continues to improve completion rates for Medicare enrollees overall, especially for underserved groups.

METHODS

Survey Instruments and Methods

The 2022 MCAHPS surveys included 4 versions, each tailored to a type of Medicare coverage: MA with or without prescription drug coverage (MA-PD and MA-only, respectively), PDP, and FFS.¹³ All surveys were administered using mail-telephone mixed mode.¹⁷ A prenotification letter was followed by up to 2 survey mailings, with telephone follow-up (up to 5 calls) for mail nonrespondents.

Respondents were those who answered at least 1 item evaluating health care. We excluded from all analyses 7745 individuals (0.7% of the sample) who were institutionalized or deceased.

Predictors of Survey Response and Completion Mode

We describe RRs for the 1,092,434 individuals selected to receive the 2022 MCAHPS surveys in the 50 states and the District of Columbia and analyze predictors of the completion mode among the 365,297 respondents. Predictors of survey response and completion mode were person-level characteristics available from administrative sources: survey type (MA, PDP, and FFS), sex, age, a DE or low-income subsidy (LIS) indicator, rurality (metropolitan, micropolitan, and rural), census division, and race/ethnicity (Hispanic and non-Hispanic AA/NHPI, Black, White, and other) (eAppendix 1 [eAppendices available at ajmc.com]).

We used linear mixed models to examine adjusted differences by these characteristics in (1) response propensity among eligible sampled cases and (2) completion mode among respondents (later response by telephone vs earlier response by mail). Models were adjusted for contract, state, and contract-by-state interactions using random effects.

RESULTS

Survey Response

Table 1 describes the eligible sample, respondents, and unadjusted RRs by person-level characteristics. The overall RR was 33.7%. Some groups were underrepresented among respondents due to lower RRs. For instance, 15.2% of the eligible sample but only 9.4% of respondents were younger than 65 years due to those age groups’ below-average RRs (14.3%-24.6%). The eligible sample was composed of 11.2% Black, 10.3% Hispanic, and 4.4% AA/NHPI individuals, but respondents were 8.8% Black, 7.5% Hispanic, and 3.5% AA/NHPI because of below-average RRs for these groups (24.4%-27.4%). Similarly, 31.0% of the eligible sample but only 24.9% of respondents had DE/LIS coverage (27.0% RR for DE/LIS).

Table 2’s first column presents the linear mixed model of survey response. Adjusted RRs had similar patterns to the unadjusted RRs seen in Table 1. Adjusted RRs increased by 22 to 23 PP as individuals’ age increased from 18 to 44 years to 80 to 89 years and then declined by approximately 7 PP for individuals 90 years and older. Adjusted RRs among White enrollees were 10.6 PP, 9.9 PP, and 5.8 PP higher than those of comparable Hispanic, AA/NHPI, and Black enrollees, respectively. Individuals with DE/LIS coverage had an adjusted RR 3.6 PP lower than others. Individuals in metropolitan settings were approximately 2 PP less likely to respond than those in rural settings (P < .001 for all differences in this paragraph).

Completion Mode

Overall, 31.3% of eligible sampled individuals responded by mail and 2.3% by telephone (6.9% of responses were by telephone) (Table 1 and eAppendix 2). In many groups in which overall and mail RRs were below average, the telephone mode represented a larger and substantial proportion of respondents. Among those aged 18 to 64 years, mail RRs were 20.5% or less, and 16.5% or more of responses were by telephone. Mail RRs among Black and Hispanic individuals were low (22.2% and 21.0%, respectively), and 16.1% and 14.1% of responses were by telephone, respectively. Individuals with DE/LIS coverage had a 23.0% mail RR, and 14.6% of responses were by telephone. The AA/NHPI group differed from this pattern; both the proportion of responses by telephone (6.2%) and the mail RR (25.7%) were low.

Table 2, column 2 summarizes the model of response by telephone among respondents. Respondents aged 18 to 54 years were 8.3 to 10.3 PP more likely to respond by telephone than respondents aged 65 to 69 years. Respondents 70 years and older had adjusted rates of telephone response 1 to 2 PP lower than respondents aged 65 to 69 years. Black and Hispanic respondents were the racial/ethnic groups most likely to respond by telephone, with adjusted rates 6.4 PP and 5.9 PP higher than those of White respondents, respectively. In contrast, AA/NHPI respondents were 1.7 PP less likely than White respondents to respond by telephone. After adjustment, respondents with DE/LIS coverage were 3.3 PP more likely to respond by telephone than other respondents (P < .001 for all differences in this paragraph).

DISCUSSION

The 2022 MCAHPS RR (33.7%) was 15.2 PP lower than in 2007 (48.9%).¹⁶ Mail RRs declined by 3.0 PP (34.3% to 31.3%) and telephone RRs declined by 12.3 PP (14.6% to 2.3%). The percentage of respondents by telephone decreased from 29.9% to 6.9%. Given the importance of the telephone mode for representing underserved individuals with Medicare, declining telephone RRs adversely affect representation of many groups.

Despite the large decrease in telephone RRs, the telephone was used by at least 14% of respondents in many groups with below-average RRs, so telephone responses are important for assessing health equity, including the experiences of individuals younger than 65 years (all eligible for Medicare via disability), who are Black or Hispanic, or who have limited income and assets. AA/NHPI enrollees also had a relatively low overall RR and among the lowest telephone RRs. RRs varied by sex and rurality; however, mode preference did not vary greatly by these factors.

Because the telephone remains an important completion mode for some groups, efforts should be made to increase telephone RRs. A study of telephone RRs from 2008 through 2015 found that refusal rates increased only slightly over that period, whereas the rate of not answering survey calls increased by 10 PP for landlines and 24 PP for cell phones,¹⁸ likely due to increased screening of unfamiliar telephone numbers. Survey administrators cannot change this use of screening technology and should focus on improving the salience of prenotification information, improving the quality of telephone contact information through look-up services,¹⁹ distributing follow-up calls throughout the day, and using bilingual interviewers as needed.²⁰ As the Federal Communications Commission works to adopt STIR/SHAKEN caller ID authentication by phone service providers,²¹ survey call centers can help ensure that their outbound calls are not flagged as scam, spam, or robocalls²² and tailor their caller ID names to the target population to increase the likelihood that a call is answered.²³

Adding a web-based component may also increase RRs.²⁴ Recent randomized experiments support the continued use of the telephone mode and have found that adding web as the initial mode to a mail-telephone protocol increases RRs to patient experience surveys.² A hospital CAHPS study found that web-mail-telephone survey administration had a higher RR (36.5%) than both mail-telephone (31.1%) and web-mail (30.6%), indicating that telephone and web increased RRs even when both of the other modes were available.² Both telephone and web modes were associated with higher RRs among racial/ethnic minority groups and younger individuals.² A study in the emergency department setting found a higher RR for web-mail-telephone (30.7%) than mail-telephone (25.3%).²⁵ Another emergency department study did not find a statistically significant difference in RRs between web-mail-telephone (27.3%) and mail-telephone (25.5%) modes, but a web-mail mode had a lower RR (15.3%) than either protocol with the telephone.²⁶ Telephone respondents were more likely than mail or web respondents to be young, Hispanic, Black, and in poorer health and to have lower educational attainment.²⁶

RRs via the web are likely to improve as health care providers collect better email address information.² A 2023 study found that use of the internet at home, possibly a proxy for ease of response by web, has increased among older individuals with Medicare but remains low for some groups (77% overall; < 65% among individuals who are Black, Hispanic, or ≥ 85 years or who have low educational attainment or limited income and assets).²⁷ This may indicate that modes other than the web will remain important in reaching individuals with Medicare.

Limitations

This study was limited to individuals with Medicare, most of whom were older than 65 years; these findings may not be generalizable to other populations.

CONCLUSIONS

As web-first mixed modes enter accountability surveys for measuring patient-centered care, the telephone mode, although declining in use, is likely to remain an important way to include the experiences of underserved groups. Efforts to use this mode effectively for those with Medicare coverage are likely to be important to measuring progress toward health equity goals.

Acknowledgments

The authors would like to thank Katherine Osby and Lauren Lakritz for the preparation of this manuscript.

Author Affiliations: RAND, Pittsburgh, PA (AH, AMH), and Santa Monica, CA (DDQ, NO, JB, MNE); Carnegie Mellon University (AMH), Pittsburgh, PA; CMS (SG), Baltimore, MD.

Source of Funding: This study was funded by a contract from CMS (GS-10F-0275P/75FCMC20F0101).

Author Disclosures: Ms Brown reports that this work was performed as part of contract funding paid to her employer. The remaining authors report no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article.

Authorship Information: Concept and design (MNE); acquisition of data (JB, SG, MNE); analysis and interpretation of data (AH, DDQ, AMH, NO, SG, MNE); drafting of the manuscript (AH, DDQ, JB); critical revision of the manuscript for important intellectual content (AH, DDQ, AMH, NO, SG, MNE); statistical analysis (AH, AMH); provision of patients or study materials (JB); obtaining funding (NO, MNE); administrative, technical, or logistic support (NO, JB, MNE); and supervision (MNE).

Address Correspondence to: Marc N. Elliott, PhD, RAND, 1776 Main St, Santa Monica, CA 90401. Email: elliott@rand.org.

REFERENCES

1. Institute of Medicine. Crossing the Quality Chasm: A New Health System for the 21st Century. The National Academies Press; 2001.

2. Elliott MN, Brown JA, Hambarsoomian K, et al. Survey protocols, response rates, and representation of underserved patients: a randomized clinical trial. JAMA Health Forum. 2024;5(1):e234929. doi:10.1001/jamahealthforum.2023.4929

3. Anhang Price R, Elliott MN, Cleary PD, Zaslavsky AM, Hays RD. Should health care providers be accountable for patients’ care experiences? J Gen Intern Med. 2015;30(2):253-256. doi:10.1007/s11606-014-3111-7

4. Health equity. World Health Organization. Accessed September 12, 2022. https://www.who.int/health-topics/health-equity#tab=tab_1

5. Stedman RC, Connelly NA, Heberlein TA, Decker DJ, Allred SB. The end of the (research) world as we know it? understanding and coping with declining response rates to mail surveys. Soc Nat Resour. 2019;32(10):1139-1154. doi:10.1080/08941920.2019.1587127

6. Elliott MN, Zaslavsky AM, Goldstein E, et al. Effects of survey mode, patient mix, and nonresponse on CAHPS hospital survey scores. Health Serv Res. 2009;44(2 pt 1):501-518. doi:10.1111/j.1475-6773.2008.00914.x

7. Zaslavsky AM, Zaborski LB, Cleary PD. Factors affecting response rates to the Consumer Assessment of Health Plans Study survey. Med Care. 2002;40(6):485-499. doi:10.1097/00005650-200206000-00006

8. Evans R, Berman S, Burlingame E, Fishkin S. It’s time to take patient experience measurement and reporting to a new level: next steps for modernizing and democratizing national patient surveys. Health Affairs Forefront. March 16, 2020. Accessed September 12, 2023. https://www.healthaffairs.org/content/forefront/s-time-take-patient-experience-measurement-and-reporting-new-level-next-steps

9. Luiten A, Hox J, de Leeuw E. Survey nonresponse trends and fieldwork effort in the 21st century: results of an international study across countries and surveys. J Off Stat. 2020;36(3):469-487. doi:10.2478/jos-2020-0025

10. Medicare monthly enrollment. CMS. October 24, 2024. Updated October 31, 2024. Accessed December 20, 2023. https://data.cms.gov/summary-statistics-on-beneficiary-enrollment/medicare-and-medicaid-reports/medicare-monthly-enrollment

11. Medicare program - general information. CMS. Updated September 10, 2024. Accessed September 8, 2022. https://www.cms.gov/Medicare/Medicare-General-Information/MedicareGenInfo

12. Explore your Medicare coverage options. Medicare.gov. Accessed September 27, 2022. https://www.medicare.gov/plan-compare

13. Orr N, Zaslavsky AM, Hays RD, et al. Development, methodology, and adaptation of the Medicare Consumer Assessment of Healthcare Providers and Systems (CAHPS) patient experience survey, 2007-2019. Health Serv Outcomes Res Methodol. 2022;23:1-20. doi:10.1007/s10742-022-00277-9

14. Stratified reporting. CMS. Updated November 12, 2024. Accessed September 8, 2022. https://www.cms.gov/priorities/health-equity/minority-health/research-data/stratified-reporting

15. Price RA, Quigley DD, Hargraves JL, et al. A systematic review of strategies to enhance response rates and representativeness of patient experience surveys. Med Care. 2022;60(12):910-918. doi:10.1097/MLR.0000000000001784

16. Klein DJ, Elliott MN, Haviland AM, et al. Understanding nonresponse to the 2007 Medicare CAHPS survey. Gerontologist. 2011;51(6):843-855. doi:10.1093/geront/gnr046

17. Medicare Advantage and Prescription Drug Plan CAHPS Survey: Quality Assurance Protocols & Technical Specifications.Version 12.0. CMS; 2021. Accessed June 14, 2024. https://www.ma-pdpcahps.org/globalassets/ma-pdp/quality-assurance/2022/ma-pdp-cahps-qapts-v12-complete-manual.pdf

18. Dutwin D, Lavrakas P. Trends in telephone outcomes, 2008-2015. Surv Pract. 2016;9(3):1-8. doi:10.29115/SP-2016-0017

19. Hustedt B, Franklin J, Tate N. Data collection methods used to maximize response rates on NCES postsecondary surveys. New Dir Inst Res. 2019;2019(181):9-20. doi:10.1002/ir.20294

20. Elliott MN, Klein DJ, Kallaur P, et al. Using predicted Spanish preference to target bilingual mailings in a mail survey with telephone follow-up. Health Serv Res. 2019;54(1):5-12. doi:10.1111/1475-6773.13088

21. Federal Communications Commission. Call authentication trust anchor. Fed Regist. 2023;88(118):40096-40121.

22. Edwards GW, Gonzales MJ, Sullivan MA. Robocalling: STIRRED AND SHAKEN! - an investigation of calling displays on trust and answer rates. In: CHI ’20: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. Association for Computing Machinery; 2020:1-12. doi:10.1145/3313831.3376679

23. Peters LE, Zhao J, Gelzinnis S, Smith SR, Martin J, Pockney P. Use of caller ID and text messaging from cell phones to increase response rates in patient surveys. Res Methods Med Health Sci. 2023;4(4):150-155. doi:10.1177/26320843231167496

24. Anhang Price R, Quigley DD, Hargraves JL, et al. A systematic review of strategies to enhance
response rates and representativeness of patient experience surveys. Med Care. 2022;60(12):910-918. doi:10.1097/MLR.0000000000001784

25. Mathews M, Parast L, Tolpadi A, Elliott M, Flow-Delwiche E, Becker K. Methods for improving response rates in an emergency department setting — a randomized feasibility study. Surv Pract. 2019;12(1). doi:10.29115/SP-2019-0007

26. Parast L, Mathews M, Elliott M, et al. Effects of push-to-web mixed mode approaches on survey response rates: evidence from a randomized experiment in emergency departments. Surv Pract. 2019;12(1). doi:10.29115/SP-2019-0008

27. Beckett MK, Haas A, Saliba D, et al. Gaps in internet use narrowed among older adults with Medicare during the COVID-19 pandemic but persist. J Am Geriatr Soc. 2024;72(4):1283-1287. doi:10.1111/jgs.18735