Potential of Risk-Based Population Guidelines to Reduce Cardiovascular Risk in a Large Integrated Health System

ABSTRACT

Objectives: We evaluated an alternative way to implement guidelines using an automated risk calculator and risk-based decision tool to calculate patients’ risk of cardiovascular disease (CVD) and recommend therapies. We compared such an approach with traditional guidelines.

Study Design: A retrospective cohort study of 1,506,109 Kaiser Permanente Southern California members 35 years or older.

Methods: We estimated 3-year risks of fatal and nonfatal myocardial infarction and stroke using an independently developed risk calculator, then graphically compared risks with observed outcomes. We used the area under the receiver operating characteristics curve to assess discrimination, and the Hosmer-Lemeshow statistic to test fit. We compared the characteristics and outcomes of populations identified for medication therapy by the risk-based decision tool and traditional guidelines using bivariate statistics.

Results: A risk score was obtained in 72% (1,082,158) of members. The risk calculator was fairly good in discrimination: the area under the curve was 0.774 (95% CI, 0.770-0.779) for myocardial infarction and 0.805 (95% CI, 0.801-0.808) for stroke. Predictiveness and fit was good based on graphical analysis and Hosmer-Lemeshow P <.0001. The risk-based decision tool identified high-risk patients for treatment who were not identified by traditional guidelines (3.80% of all those identified for statins, 3.04% for antihypertensives), as well as low-risk patients who were identified by guidelines (3.80% for statins, 2.51% for antihypertensives).

Conclusions: The risk calculator provided risk estimates in most patients and demonstrated fairly good discrimination and predictiveness. The risk-based decision tool identified high-risk patients for treatment not identified by traditional guidelines, as well as low-risk patients for whom treatment may be unnecessary.

Am J Manag Care. 2016;22(5):e161-e168

Take-Away Points

Automated risk calculator and guideline decision support tools can move clinical care a step closer to systematically individualizing patient therapy and population healthcare.

• The proportion of the population for whom sufficient electronic data were available to make risk estimation was good.
• The risk-based decision tool performed well in discrimination and predictiveness.
• The risk-based decision tool identified high-risk individuals missed by traditional guidelines, and low-risk individuals who were recommended treatment based on traditional guidelines.

Clinical practice guidelines traditionally have specified levels of biomarkers to determine whether or not patients should be treated and to specify treatment goals. For example, the Seventh Report of the Joint National Committee (JNC-7) guideline for cholesterol specifies 3 different risk strata based on LDL cholesterol levels.^1-3 Traditional guidelines can include other risk factors that are typically secondary to the biomarker target, such as gender or smoking status. A notable exception to the traditional approach is the new American College of Cardiology (ACC)/American Heart Association (AHA) cholesterol guideline. Although not explicitly dropping the use of cholesterol treatment goals, these new guidelines endorse the use of a risk calculator to guide treatment.^4,5 Depending on their risk of a cardiovascular event, as estimated by a calculator, patients with high cholesterol levels may not be treated and patients with low cholesterol levels may be treated.

We wished to understand the potential implications of using risk calculators and risk-based decision tools to help risk stratify a population and identify individual patients for treatment. In the first part of our study, we evaluated the population applicability, discrimination, and predictiveness of an independently developed cardiovascular disease (CVD) risk calculator.⁶ The risk calculator was part of a risk-based decision tool under consideration by Kaiser Permanente Southern California (KPSC) for use in place of standard clinical practice guidelines. The validation of the risk calculator against outcomes was considered an essential first step before comparing the risk-based decision tool to traditional care. In the second part of our study, we tested the potential population impact of the risk-based decision tool by comparing actual outcomes in patients selected for possible treatment by the risk-based decision tool versus those selected by traditional clinical practice guidelines.

METHODS

Design, Overview

We conducted a retrospective cohort study of KPSC members with active membership during 2007 to 2010. The ending time point was chosen because at the time of initial analyses, government mortality data were not available past 2010. In the first part of this study, we determined the proportion of an adult population to which the risk calculator could be applied, then we compared predicted and observed outcomes. In the second part of the study, we determined which patients would be selected for medication therapy by the risk-based decision tool and by traditional guidelines, then we compared observed outcomes in these groups. Archimedes, Inc developed the risk calculator and the risk-based decision tool, and was involved in study planning and interpretation, but not in data collection or analysis. No data from KPSC were used to create the risk calculator or the risk-based decision tool evaluated in this study. The study was approved by the KPSC Institutional Review Board.

Setting, Subjects

KPSC is a prepaid, integrated health system with 3.5 million members of diverse race and ethnicity at 14 medical centers and 197 medical offices throughout southern California. Members have similar coverage benefits and a limited range of co-payments for services and medications. Subjects 35 years or older were selected from among health plan members who were alive on December 31, 2007, with no gap in enrollment of more than 45 days in the year prior to that date.

Explanatory Variables: Sources, Measures

The electronic health record (EHR) was used to identify pre-existing conditions using International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) codes assigned during inpatient or outpatient healthcare encounters. We collected data on past cardiovascular risk—related disease history from January 1, 2004 (up to 4 years prior to study outcomes), including diabetes, stroke, myocardial infarction, atrial fibrillation, congestive heart failure, and chronic kidney disease. For 2007, we extracted all the clinical information needed for risk calculation from laboratory and pharmacy data files. Laboratory data included fasting plasma glucose, glycated hemoglobin, low-density lipoprotein cholesterol, high-density lipoprotein cholesterol, total cholesterol, triglycerides, serum creatinine, and measures of proteinuria (eg, microalbumin-to-creatinine ratio). We identified medication dispensing dates for antihypertensives (ie, angiotensin converting-enzyme inhibitors and angiotensin receptor blockers, calcium channel-blockers, thiazide diuretics, beta-blockers), cholesterol-lowering drugs (ie, statins and non-statins), and insulin. We identified additional key risk factors from the EHR, including recent smoking, nonprescription aspirin use, body mass index, and blood pressure measurements.

Outcomes: Sources, Measures

We identified fatal and nonfatal myocardial infarction and cerebrovascular (ie, stroke) events from January 1, 2008, to December 31, 2010. These 3-year outcomes were identified from the EHR using ICD-9-CM hospital discharge diagnosis codes (myocardial infarction: 410; and stroke: 346.6x, 430, 431, 432.x, 433.01, 433.11, 433.21, 433.31, 433.81, 433.91, 434.x, 436). Hospitalization discharge diagnosis codes were collected from claims data if the hospitalization occurred at a non-KPSC facility. The date and cause of death were determined from the California State Death Statistical Master Files and the Multiple Cause of Death Files for all subjects, regardless of health plan membership status.

Risk Calculator and Risk-Based Decision Tool

We predicted CVD risk for all patients using an updated version of a previously developed and described risk calculator.⁶ The models used in the risk calculator were in the form of a proportional hazards survival model and were independently derived using publicly available data sets from the following studies: Framingham, Framingham Offspring, Atherosclerosis Risk in Communities Study (ARIC), and Cardiovascular Health Study (CHS).^2,5-18 The risk calculator was designed to determine the 5-year risk of fatal and nonfatal myocardial infarction, and, as a separate estimate, fatal and nonfatal stroke. Multiple statistical imputation is used to handle missing data when possible. Confidence guardrails prevent presentation of risk information when the data for a particular patient are insufficient for imputation and risk calculation.

eAppendix

The risk-based decision tool is the set of risk calculators and additional models that allow computations of health outcomes and the effects of interventions on risk at a person-specific level. This tool has a software application that utilizes EHR data to create an interactive visual display that quantifies health risks and the benefits of different healthcare interventions for individual patients.⁶ The risk-based decision tool interface can be used to assist collaborative provider-patient decision making in place of standard clinical practice guidelines. The risk-based decision tool sets a threshold of expected benefits and, for each patient, calculates a Health Impact score—the expected gain in 5-year quality-adjusted life-years, depending on whether or not they follow a treatment recommendation (see [available at www.ajmc.com] for details). Information on the expected risks and the outcomes associated with possible therapies is presented to patients and physicians for individualized decision making.⁶ In a population management context, the risk-based decision tool can be used to prioritize patients by the potential benefit of actionable interventions and rank interventions in order of expected benefit. Some of the models for intervention impact on risk were derived independently, and others were derived from risk equations used in the Archimedes Model, which allows computations of health outcomes and effects of interventions on a person-specific level.⁶ The model is validated on a continuous basis to remain relevant and comply with new medical guidelines.

All of these models were derived from and validated against data from multiple studies and public sources, but not data from Kaiser Permanente.

Identification of Patients for Medication Therapy

Traditional guidelines and the risk-based decision tool were each used to identify patients for medication therapy. KPSC traditional guidelines were based on the JNC-7 blood pressure guideline and the Adult Treatment Panel III (ATP III) cholesterol guideline, which incorporated use of the Framingham risk calculator for borderline cholesterol values. These traditional guidelines were converted into an automated algorithm that could be applied to the entire population. For the risk-based decision tool, patients were chosen using a Health Impact score that resulted in the same number of patients being selected for treatment as by traditional guidelines (the “same-size” population threshold).

Our focus for this evaluation was therapy with statins (HMG-Co-A reductase inhibitors) and antihypertensives for CVD event prevention. For both traditional clinical practice guidelines and the risk-based decision tool, a patient was not identified for a medication therapy if they were already on it or there was a history of allergy or prior adverse events based on EHR data.

Statistical Analyses

For the first part of this study, we determined the proportion of patients for whom a risk calculation was possible. The calculation was not performed if the data required by the calculator were not present in the EHR and could not be imputed. For comparisons with actual 3-year outcomes, we linearly interpolated the risk calculations from the risk-based decision tool from 5 to 3 years. We used the area under the receiver operating characteristic curve (AROC) to assess how well the risk calculator sorted those who had an outcome from those who did not (ie, discrimination accuracy). We used graphical methods, as described by Pepe et al,¹⁹ to assess the risk calculator predictiveness across a range of risk percentiles, which provided a visual calibration and goodness of fit assessment. We used the Hosmer-Lemeshow statistic to test risk calculator model goodness of fit.

χ

For the second part of the study, we compared the characteristics of the “same-sized” populations identified for medication therapy by the risk-based decision tool and traditional guidelines using the t test and ² test. Any of 1 or more antihypertensive medications were counted as a single medication recommendation. We determined the outcomes (3-year events/1000 persons) of patients who were identified for possible treatment or not based on the risk-based decision tool, with those identified for treatment or not based on traditional guidelines. We also determined the outcomes in “indeterminate” patients for whom the risk-based decision tool did not calculate a Health Impact score due to insufficient data to calculate risk or to support a treatment recommendation. These indeterminate cases also included patients with medical conditions not modeled (about 5% of all patients): advanced heart failure, late-stage kidney disease, and a CVD event that occurred under age 45. When implementing traditional guidelines, if there was insufficient data to apply the guidelines, patients were assigned to “no treatment” by default since traditional guidelines only specified when to treat, and this approach was consistent with population-based implementation of guidelines.

RESULTS

Risk Calculator Assessment

Table 1

The study cohort consisted of 1,506,109 patients who met inclusion criteria. describes selected characteristics of the study cohort. A risk calculation was possible for 1,082,158 patients (72% of the cohort). The mean (SD) follow-up time for patients in which a calculation was possible was 2.61 (0.86) years.

The risk-based decision tool sorted patients fairly well by their propensity to have a myocardial infarction or stroke: the AROC was 0.774 (95% CI, 0.770-0.779) for myocardial infarction, and 0.805 (95% CI, 0.801-0.808) for stroke. The Hosmer-Lemeshow fit statistic indicated good fit (P <.0001) for each model.

Figure

The shows the predictiveness curves of the risk calculator for fatal and nonfatal myocardial infarction (graph A) and stroke (graph B) for 20 different ranked risk groups. The predicted rates were slightly higher than observed rates for myocardial infarctions in the top 15th risk percentile. The predicted rates for strokes were slightly lower than the observed rates in several high-risk groups, although not the highest-risk group.

Treatment Stratification by the Risk-Based Decision Tool and Traditional Guidelines

Table 1 shows the characteristics of 3 subgroups identified for possible new medication treatment by: 1) the risk-based decision tool but not by traditional guidelines, 2) traditional guidelines but not by the risk-based decision tool, and 3) both. For almost all characteristics, there were statistically significant differences (P <.0001) between the groups. The most notable difference was the older age of patients identified for possible treatment by the risk-based decision tool alone versus those identified by traditional guidelines alone. Using the “same size” population identified for treatment by both approaches, more than 50% of patients to whom medications were recommended by the risk-based guidelines only were 70 years or older. The corresponding percentage was less than 5% for those to whom medications were recommended by traditional guidelines only. Compared with traditional guidelines, the risk-based decision tool also identified proportionally more men, more patients with high-normal blood pressures, and more smokers.

Table 2

shows the percentage of the total cohort eligible for new medication treatment based on the risk-based decision tool, as well as the percentage based on traditional guidelines. The risk-based decision tool made statin recommendations in 34.85% of patients (8.73% for, 26.12% against), and antihypertensive recommendations in 18.20% of patients (4.82% for, 13.38% against). Traditional guidelines made statin recommendations in 9.69% of patients and antihypertensive recommendations in 4.94% of patients, with all others not recommended treatment by default. The CVD event rates per 1000 members among those to whom statins were recommended were similar using either approach, and somewhat higher among those to whom antihypertensives were recommended using the risk-based decision tool (72.0 vs 63.1). The event rates per 1000 members identified for treatment by both the risk-based decision tool and traditional guidelines were greater than the event rates identified by either approach alone.

Although there was overlap in the patients identified for treatment by the risk-based decision tool and traditional guidelines, the risk-based decision tool identified a number of patients who were at high risk, but who were not identified for treatment by traditional guidelines. For statins, the risk-based decision tool identified an additional 3.80% of the population (beyond the 9.69% identified by traditional guidelines) who were not identified for treatment by traditional guidelines but who had a CVD event rate (45.3 events/1000 patients) that was not much less than that of patients to whom treatment was recommended by traditional guidelines (50.1/1000 patients). Similarly, for antihypertensives, the risk-based decision tool identified an additional 3.04% of the population who were not identified for treatment by traditional guidelines, but who had a similar CVD event rate (65.9/1000 patients) to those for whom treatment was recommended by traditional guidelines (63.1/1000 patients).

The risk-based decision tool also made recommendations against therapy for a number of patients for whom treatment was recommended by traditional guidelines. For statins, the risk-based decision tool made recommendations against treatment for 3.80% of the population. These patients had a CVD event rate that was relatively low (15.5/1000 patients), and virtually identical to the risk among patients to whom treatment was not recommended by traditional guidelines (14.3/1000 patients). For antihypertensives, the risk-based decision tool made recommendations against treatment in 2.51% of the population; this group’s CVD event rate (19.9/1000 patients) was actually lower than that of patients to whom treatment was not recommended by traditional guidelines (22.7/1000 patients).

DISCUSSION

We undertook this study to evaluate an alternative way to implement guidelines using an automated risk calculator and decision support tool that captures EHR data to calculate patients’ risk of CVD outcomes, and identifies the impact of potential treatments. Our objectives were to determine its applicability, the usefulness of its predictions, and how patient identification for possible treatment compares with traditional guidelines. In doing so, we hoped to assess the implications of using a risk-based decision tool for both individual and population care.

The risk calculator was widely applicable to the health plan’s general adult population (72% had a risk estimate given 1 year of clinical data) and demonstrated fairly good discrimination and predictiveness. Our results indicate that when tuned to treatment in the same number of patients as traditional guidelines, the risk-based decision tool identifies a group of individuals who are likely to benefit from treatment but who are missed by current guidelines, even when those guidelines include Framingham risk. It also identifies a group of individuals for whom treatment is recommended by current guidelines, but have risks that are sufficiently low enough that they may safely be left untreated.

In a similar type of comparison, the newer ACC/AHA guideline for statin therapy based on a risk calculator was compared with ATP III guidelines with its cholesterol targets supplemented by Framingham risk in an existing community-based cohort.²⁰ This study compared a different risk-based calculator (ACC/AHA) with essentially the same traditional guidelines as used by KPSC. The findings mirror ours in that the calculator had “greater accuracy and efficiency in identifying increased risk of incident CVD and subclinical coronary artery disease”²⁰—similarly, more higher-risk patients were identified. Additionally of note, the risk-based approach of the ACC/AHA guidelines was deemed, in a simulation study, to have an acceptable cost-effectiveness profile.²¹ In this simulation study, the cost-effectiveness of the ACC/AHA risk-based approach varied with risk cut points.²¹ In our study, the Health Impact score was used to introduce the variation of risk to the population affected in place of pure risks.

A significant problem facing all risk-based calculators and decision support tools that use EHR data is that they might not be able to perform the desired calculations, either because of missing data or limitations in patient groups to which they are applicable (eg, late-stage kidney disease). Some of the patients who are excluded from risk models are at higher risk due to relatively infrequent medical conditions; in these situations, decisions are left to the discretion of the treating physician. Hopefully, future risk calculators will address this restricted scope of application. Risk-based decision tools also suffer from the same lack of evidence about treatment as do traditional guidelines. This is especially evident when considering that 72% of patients had a risk calculation, but among patients potentially eligible for a medication, 65% (statins) and 82% (antihypertensives) could not be definitively classified into treatment or no treatment groups.

Limitations

Our study has several limitations, including that the KPSC system may differ in patient outcomes from other health delivery models. Our determination of who would be recommended for treatment was based solely on EHR data and automated guideline implementation rather than clinical practice. Use in practice may increase or reduce the identification of high-risk patients and the differences between approaches. Not all potentially relevant characteristics of a patient, or clinical actions, are captured in the EHR. Additionally, prediction accuracy was not adjusted for subsequent changes in risk factors or clinical treatments. Our results may vary somewhat over shorter or longer follow-up time frames. The risk-based decision tool identified some patients at increased risk who were not identified by traditional guidelines; however, we did not assess the separate contributions of imputation, calculator variables, and estimator accuracy in determining this difference in identification.

CONCLUSIONS

We found that the automated risk calculator can be applied to a large proportion of a population with 1 or more years of EHR data with fairly good discrimination and predictiveness. This finding indicates a useful application to risk stratify large populations. We also found that a risk-based decision tool identified for possible treatment a group of patients at higher risk of CVD events than traditional guidelines, even without any increase in the number of patients treated. There thus exists the potential to further reduce population risk using such a tool. Automated risk calculators combined with decision support can individualize therapy for patients in a way that focuses on overall risk reduction rather than just biological targets. Further exploration of risk-based decision tools—with and without data imputation&mdash;compared with usual practice will help determine the value of such an approach. Ultimately, the use of such tools may depend on implementations that save effort in clinical practice.

Acknowledgments

The authors would like to thank David M. Eddy, PhD, founder and chief medical officer of Emeritus Archimedes, Inc, for his review of the manuscript; and Portia Summers, BS, Kaiser Permanente Southern California, for her help with preparation of the manuscript for submission.

Author Affiliations: Department of Research and Evaluation, Kaiser Permanente Southern California (GI, HZ, AHX, SJJ, SFD, MIE), Pasadena, CA; Archimedes, Inc (MM), San Francisco, CA.

Source of Funding: This study was funded by the Kaiser Permanente Health Plan.

Author Disclosures: At the time of the study IndiGO was a product of Archimedes, Inc, which was a wholly owned subsidiary of Kaiser Permanente. During the study and preparation of this manuscript, Dr Morris had stock options in Archimedes, Inc, was employed by Archimedes, and was inventor on risk stratification patents assigned to Archimedes; he is now affiliated with Symphony Health Performance Analytics, Alpharetta, GA. During the study and preparation of this manuscript, all other study personnel (including all those who obtained and analyzed the data) were employed by the Southern California Permanente Medical Group, which partners with the Kaiser Permanente Health Plan to deliver healthcare to health plan members. Dr Early is now affiliated with the Department of Obstetrics and Gynecology and Women’s Health, Montefiore Medical Center the University Hospital for Albert Einstein College of Medicine, Bronx, New York. The remaining authors report no other potential financial or conflicts of interest related to the subject in this article.

Authorship Information: Concept and design (MM, AHX, SJJ, SFD); acquisition of data (MM, MIE, SJJ, SFD); analysis and interpretation of data (GI, HZ, MIE, AHX, SJJ, SFD); drafting of the manuscript (GI, SFD); critical revision of the manuscript for important intellectual content (GI, HZ, MM, AHX, SJJ, SFD); statistical analysis (HZ, MM, SFD); obtaining funding (SJJ, SFD); administrative, technical, or logistic support (GI, MIE, SJJ, SFD); and supervision (SFD).

Address correspondence to: Stephen F. Derose, MD, MSHS, Department of Research and Evaluation, Kaiser Permanente Southern California, 100 S Los Robles, Pasadena, CA 91101. E-mail address: Stephen.F.Derose@kp.org.

REFERENCES

1. Expert Panel on Detection, Evaluation, and Treatment of High Blood Cholesterol in Adults. Executive summary of the third report of the National Cholesterol Education Program (NCEP) Expert Panel on Detection, Evaluation, And Treatment of High Blood Cholesterol in Adults (Adult Treatment Panel III). JAMA. 2001;285(19):2486-2497.

2. Hard Coronary Heart Disease (10-year risk). Framingham Heart Study website. http://www.framinghamheartstudy.org/risk-functions/coronary-heart-disease/hard-10-year-risk.php. Accessed May 7, 2013.

3. Chobanian AV, Bakris GL, Black HR, et al; Joint National Committee on Prevention, Detection, Evaluation, and Treatment of High Blood Pressure. National Heart, Lung, and Blood Institute; National High Blood Pressure Education Program Coordinating Committee. Seventh report of the Joint National Committee on Prevention, Detection, Evaluation, and Treatment of High Blood Pressure. Hypertension. 2003;42(6):1206-1252.

4. Goff DC Jr, Lloyd-Jones DM, Bennett G, et al; American College of Cardiology/American Heart Association Task Force on Practice Guidelines. 2013 ACC/AHA guideline on the assessment of cardiovascular risk: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines. Circulation. 2014;129(25, suppl 2):S49-S73. doi: 10.1161/01.cir.0000437741.48606.98.

5. Stone NJ, Robinson JG, Lichtenstein AH, et al; American College of Cardiology/American Heart Association Task Force on Practice Guidelines. 2013 ACC/AHA guideline on the treatment of blood cholesterol to reduce atherosclerotic cardiovascular risk in adults: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines. Circulation. 2014;129(25, suppl 2):S1-S45. doi: 10.1161/01.cir.0000437738.63853.7a.

6. Eddy DM, Adler J, Patterson B, Lucas D, Smith KA, Morris M. Individualized guidelines: the potential for increasing quality and reducing costs. Ann Intern Med. 2011;154(9):627-634. doi: 10.7326/0003-4819-154-9-201105030-00008.

7. Hingorani AD, Vallance P. A simple computer program for guiding management of cardiovascular risk factors and prescribing. BMJ. 1999;318(7176):101-105.

8. Anderson KM, Wilson PW, Odell PM, Kannel WB. An updated coronary risk profile. a statement for health professionals. Circulation. 1991;83(1):356-362.

9. Wolf PA, D’Agostino RB, Belanger AJ, Kannel WB. Probability of stroke: a risk profile from the Framingham Study. Stroke. 1991;22(3):312-318.

10. Eddy DM, Schlessinger L. Validation of the Archimedes diabetes model. Diabetes Care. 2003;26(11):3102-3110.

11. Heikes KE, Eddy DM, Arondekar B, Schlessinger L. Diabetes Risk Calculator: a simple tool for detecting undiagnosed diabetes and pre-diabetes. Diabetes Care. 2008;31(5):1040-1045.

12. Stern M, Williams K, Eddy D, Kahn R. Validation of prediction of diabetes by the Archimedes model and comparison with other predicting models. Diabetes Care. 2008;31(8):1670-1671. doi: 10.2337/dc08-0521.

13. Kahn R, Robertson RM, Smith R, Eddy D. The impact of prevention on reducing the burden of cardiovascular disease. Diabetes Care. 2008;31(8):1686-1696. doi: 10.2337/dc08-9022.

14. Brandeau ML. Modeling complex medical decision problems with the Archimedes model. Ann Intern Med. 2005;143(4):303-304.

15. Schlessinger L, Eddy DM. Archimedes: a new model for simulating health care systems—the mathematical formulation. J Biomed Inform. 2002;35(1):37-50.

16. Dziuba J, Alperin P, Racketa J, et al. Modeling effects of SGLT-2 inhibitor dapagliflozin treatment versus standard diabetes therapy on cardiovascular and microvascular outcomes. Diabetes Obes Metab. 2014;16(7):628-635. doi: 10.1111/dom.12261.

17. Schlender A, Alperin PE, Grossman HL, Sutherland ER. Modeling the impact of increased adherence to asthma therapy. PLoS One. 2012;7(12):e51139. doi: 10.1371/journal.pone.0051139.

18. van Herick A, Schuetz CA, Alperin P, Bullano MF, Balu S, Gandhi S. The impact of initial statin treatment decisions on cardiovascular outcomes in clinical care settings: estimates using the Archimedes Model. Clinicoecon Outcomes Res. 2012;4:337-347. doi: 10.2147/CEOR.S35487.

19. Pepe MS, Feng Z, Huang Y, et al. Integrating the predictiveness of a marker with its performance as a classifier. Am J Epidemiol. 2008;167(3):362-368.

20. Pursnani A, Massaro JM, D’Agostino RB, Sr, O’Donnell CJ, Hoffmann U. Guideline-based statin eligibility, coronary artery calcification, and cardiovascular events. JAMA. 2015;314(2):134-141. doi: 10.1001/jama.2015.7515.

21. Pandya A, Sy S, Cho S, Weinstein MC, Gaziano TA. Cost-effectiveness of 10-year risk thresholds for initiation of statin therapy for primary prevention of cardiovascular disease. JAMA. 2015;314(2):142-150. doi: 10.1001/jama.2015.6822.&ensp;

Download PDF: Potential of Risk-Based Population Guidelines to Reduce Cardiovascular Risk in a Large Integrated Health System