NQF Report Explores Possibility of Adjusting for Social Risk in Performance Measures

The National Quality Forum (NQF) has issued a report detailing the results of its experiment to include social risk factors in its risk adjustment models for performance measures, finding this type of methodology is feasible but not without its challenges.

The report published today is the product of a 2-year trial period in which the NQF temporarily reversed its policy of not accounting for social risk factors when developing the risk adjustment models of the performance measures it endorses. In 2014, an expert panel had recommended that the NQF allow these factors to be included in risk-adjusted scores when appropriate to determine whether they could “ensure fair and accurate comparisons of provider performance.”

Researchers and policymakers have long understood that social risk factors such as education level, Medicaid status, and race/ethnicity impact individuals’ health and health outcomes. Still, there is no clear consensus on how to incorporate these factors into risk adjustment and value-based purchasing, due to worries that it could conceal or exacerbate healthcare disparities and unfairly penalize safety-net providers.

With these concerns in mind, the NQF set out on its 2-year trial run of including social risk factors in measures with a conceptual and empirical rationale and assessing the impact of these factors. Of the 303 performance measures reviewed by the trial, 65 were risk-adjusted measures with a conceptual basis for adjusting for social risk. Ultimately, around a quarter of these were found to have both a conceptual basis and empirical evidence to support adjusting for social risk factors. A few of these 17 measures, followed by the social variables included, are listed below:

• Standardized Mortality Ratio for Dialysis Facilities (race, ethnicity)
• Discharge to Community (marital status)
• 30-day Unplanned Readmissions for Cancer Patients (Phase 3 status as dual eligible for Medicare/Medicaid)
• CAHPS^® Hospice Survey (payer; respondent education; concordance of survey language and respondent’s home language)

The report detailed some roadblocks the NQF encountered while implementing the inclusion of social risk in these measures. First, the guidelines for determining conceptual basis were interpreted differently across different measures and by different reviewers, indicating that this step could use more specificity. The panel also observed wide variations in the impact of each social risk factor on every measure, leading it to recommend that each risk factor—measure combination is carefully examined to determine whether adjustment is appropriate. Finally, NQF encountered difficulties gathering enough granular, patient-level social risk data, suggesting that alternative data sources will be necessary.

Despite these challenges, the NQF determined that using social risk factors in risk adjustment models is indeed feasible and warrants further research. In particular, future experiments may include community factors or modify the adjustment techniques to highlight differences across populations and demonstrate the empirical effects of social risk factors. These efforts will become even more imperative as the healthcare system increasingly rewards providers based on performance while seeking to acknowledge the full impact of healthcare disparities.

“Every stakeholder wants to see the quality of care for all Americans, especially the most vulnerable, improve while ensuring a level playing field for providers in value-based purchasing programs,” said NQF president and CEO Shantanu Agrawal, MD, in a statement announcing the report’s release. “The frequent use of NQF-endorsed measures for payment purposes underscores the importance of ensuring accurate comparisons of providers so that rewards or penalties are fairly assessed and based on true differences in performance.”