Clinical Variable Model More Reliable at Predicting Mortality Among Veterans

Are coding changes responsible for declines in the 30-day mortality rate for hospital admissions among veterans utilizing Veterans Health Administration medical centers (VAMCs)? Which model is more accurate at predicting this measure: a claims-based model or a clinical factor—based model?

Wanting to confirm declines in 30-day mortality from previous studies, investigators set out to answer these questions as they pertained to patients admitted between January 2009 and September 2015 for a primary heart failure (HF), pneumonia, or sepsis/respiratory failure (RF) diagnosis with a pneumonia secondary diagnosis. Previous studies using claims-based models to determine mortality risk have adjusted to account for comorbid conditions, but the study authors expressed worry that changes to diagnostic coding practices may have influenced the results, “particularly because comorbid diagnoses often have flexible clinical criteria.”

Their results appeared in JAMA Internal Medicine. The clinical criteria they used included blood pressure, heart and respiratory rates, oxygen saturation, body mass index, hematocrit, serum glucose, and left ventricular ejection fraction. For 132 VAMCs, there were more admissions for HF than pneumonia (146,924 vs 131,325), most patients were men (98.4% and 97.1%, respectively), and patients with HF tended to be older (mean [SD] age, 71.6 [11.4] vs 70.8 [12.3] years).

Overall, the 30-day mortality rates were 6.45% for HF and 11.22% for pneumonia. Unadjusted, these rates decreased from 6.64% to 6.29% and 11.09% to 10.26%, respectively, from the first to the last years of the study.

The adjusted mortality rates show a much starker difference. Using claims-based data, the mortality rate for HF fell from 6.90% to 6.01% and for pneumonia, from 11.85% to 9.67%. But when clinical factors were adjusted for, the rates only dropped from 6.48% to 6.39% and 11.19% to 10.46%, respectively.

Looking at the rates on a quarterly basis, the clinical models virtually eliminated any improvements in mortality demonstrated by the claims-based models:

• HF: Claims-based rate: —0.051 (95% CI, −0.074 to −0.027) percentage points Clinical factor—based rate: –0.017 (95% CI, −0.039 to 0.006) percentage points
• Pneumonia: Claims-based rate: —0.084 (95% CI, (95%CI, −0.111 to −0.056) percentage points Clinical factor—based rate: –0.026 (95% CI, –0.052 to 0.001)

“The higher performance of clinical risk-adjustment models, as measured by [area under the receiver operating characteristic curve] and goodness-of-fit statistics, compared with those using claims alone implies that the VA should continue allocating resources toward the collection of clinical data, even as it transitions to an alternative electronic medical record,” the investigators concluded. “Among HF and pneumonia hospitalizations, adjusting for clinical covariates attenuated declines in mortality rates identified using claims-based models. Assessments of temporal trends in 30-day mortality using claims-based risk adjustment should be interpreted with caution,”

The authors note that their results should be interpreted with caution because they may not be generalizable to conditions outside of HF and pneumonia.

Reference

Silva GC, Jiang L, Gutman R, et al. Mortality trends for veterans hospitalized with heart failure and pneumonia using claims-based vs clinical risk-adjustment variables [published online December 20, 2019]. JAMA Intern Med. doi: 10.1001/jamainternmed.2019.5970.