Outcome Measures for Oncology Alternative Payment Models: Practical Considerations and Recommendations

ABSTRACT

Objectives: This paper aims to synthesize existing scholarship on quality measures in oncology, with a specific focus on outcome-based quality measures, which are often underutilized. We also present a set of “core outcome measures” that may be considered in future oncology alternative payment models (APMs).

Study Design: Our research consists of a focused literature review, content analysis, and quality measure synthesis and categorization.

Methods: We conducted a focused literature review to generate key evidence on quality measures in oncology. We studied 7 oncology quality assessment frameworks, encompassing 142 quality metrics, and synthesized recommendations using the Center for Medicare and Medicaid Innovation APM toolkit, focusing on outcome measures.

Results: We present 34 outcome-based oncology quality measures for consideration, which are classified into 5 domains: clinical care (eg, hospital and emergency department visits, treatment effectiveness, mortality), safety (eg, infections, hospital adverse events), care coordination (for hospital and hospice care), patient and caregiver experience, and population health and prevention. Both general and indication-specific outcome measures should be considered in oncology APMs, as appropriate. Utilizing outcome-based measures will require addressing multiple challenges, ranging from risk adjustment to data quality assurance.

Conclusions: Oncology care will benefit from a more rigorous approach to quality assessment. The success of oncology APMs will require a robust set of quality measures that are relevant to patients, providers, and payers.

Am J Manag Care. 2019;25(12):e403-e409Takeaway Points

Evidence is scarce on what outcome measures are most suitable and feasible for future oncology payment models. This paper reviews 7 oncology quality assessment frameworks and existing literature on quality measurement in oncology and recommends that:

• Outcome quality measures in future oncology payment models should include domains for clinical care, safety, care coordination, patient and caregiver experience, and population health and prevention.
• Future oncology payment models should draw on the core outcome set but should be tailored to unique care delivery needs and context.
• The inclusion of diverse outcome measures in oncology may help improve care delivery to all patients with cancer.

Several alternative payment models (APMs) are being piloted to address affordability, equity, and quality-of-care challenges in oncology care. Of these arrangements, the Oncology Care Model (OCM), developed by the Center for Medicare and Medicaid Innovation (CMMI), is among the most extensive, covering about 200,000 chemotherapy episodes annually. The OCM relies on multiple quality measures to determine the level of payment for each provider, with the goal of incentivizing higher-quality care in a cost-effective manner. The OCM’s payment design is described in eAppendix A (available at ajmc.com).

Despite innovations in the payment landscape, limited consensus exists about what constitutes indispensable quality measures in oncology. The absence of such consensus may not only limit the development of better payment models, which increasingly link payment to quality of care, but also result in a lack of agreement on how value should be defined (and demonstrated) in an era of innovative, ever more expensive cancer therapies. In 2016, a roundtable of national experts in cancer care and oncology measurement recommended that policy makers “prioritize and develop effective cross-cutting measures that assess clinical and patient-reported outcomes, including shared decision making, care planning, and symptom control” and highlighted an “overreliance on condition-specific process measures.”¹ Disease-specific quality measures have been developed, but there is a lack of consensus on what quality measures ought to be utilized across multiple cancers, especially when measuring clinical outcomes.² As oncology APMs evolve, there are practical considerations in the design and implementation of outcome-based measures.

The development of new quality measures for oncology has been underway for many years. For example, CMS together with America’s Health Insurance Plans (AHIP) and others developed Core Quality Measures in 8 therapeutic areas, including medical oncology, to assess provider performance.³ This initiative included quality indicators focusing on breast cancer, colorectal cancer, prostate cancer, and, more generally, end-of-life care.⁴ It also identified areas for future measure development in oncology (eg, pain control, hospital admission, 5-year cure rates) and highlighted challenges related to data access and measurement as the standard of cancer care progresses, requiring frequent reassessments.⁴ In parallel, the American Society for Radiation Oncology has been working with the American Society of Clinical Oncology (ASCO) to develop measures “for utilization by both organizations in various quality programs and reporting environments.”⁵ However, no comprehensive set of core outcome-based quality measures in oncology has been published.

Research on healthcare quality measures typically differentiates between 2 key categories: process-based measures, which focus on proper reporting and procedure execution, and outcome-based measures, which involve clinical outcomes and patient-reported experience of care. This paper aims to synthesize existing scholarship on quality measures in oncology, with a specific focus on outcome-based quality measures, which are underutilized given their perceived benefits. We also present a set of “core outcome measures” that may be considered in future oncology APMs. Our study does not aim to provide a definitive list but, rather, to present a diverse set of outcome measures most commonly included in quality initiatives and payment models in oncology.

METHODS

Our research consists of a focused literature review, content analysis, and measure categorization, similar to Macefield et al.⁶ First, our focused literature review summarizes key evidence related to quality measures in oncology, with an emphasis on classification, their unique advantages and disadvantages, and the challenges related to implementation in clinical practice. Second, our descriptive analysis of the most commonly used quality measures in oncology draws on a convenience sample of existing payment models and other quality assessment frameworks. Our sample includes 7 oncology quality assessment programs, frameworks, and payment models (also referenced as “oncology quality assessment frameworks”), which encompass 142 quality measures: the OCM by CMMI, the Quality Oncology Practice Initiative by ASCO, the Prospective Payment System—Exempt Cancer Hospital Quality Reporting Program by CMS, the Core Quality Measures Collaborative Core Sets by CMS and AHIP, the Oncology Medical Home program by the Community Oncology Alliance, the Osteoporosis Quality Improvement Registry by the National Osteoporosis Foundation and National Bone Health Alliance, and the Oncology Qualified Clinical Data Registry by the Oncology Nursing Society.

In this paper, we categorize these measures into process- versus outcome-based and analyze their frequency. Given our primary focus on outcome-based quality measures, we decided to expand our literature review to include the evidence base for each of the key outcome measure categories identified. These sources were identified by snowballing from quality measure summaries by CMS and reviewing other relevant literature. In addition, we review and summarize published reports on the impact of emerging oncology APMs on clinical outcomes and spending. We conclude with a synthesis of existing evidence on key outcome-based measures and their appropriateness in future oncology APMs. Finally, we discuss directions for customization and further validation of oncology core outcome measures.

RESULTS

Advantages of Process- and Outcome-Based Measures

Previous scholarship finds that both process- and outcome-based quality measures have advantages and disadvantages (for one such classification, see Table 1⁷).^8-10 For example, it is generally easier to generate actionable feedback based on process-based measures and there is mostly no or limited need for risk adjustment (unlike the case for quality measures such as mortality, for which complex case mix, indication, and disease stage adjustments are often required).⁷ In addition, data collection for process measures is generally faster, can draw on smaller sample sizes, and does not require advanced statistical analysis to yield practical results.⁷

On the other hand, outcome measures are generally based on clinical end points with proven significance in the quality of care. They are more understandable by patients and nonclinicians and are easier to define comprehensively (eg, hospice admissions for at least 3 days prior to death).⁷ Relatedly, an improvement in process measures may be a useful step in care coordination but may not always have an observable effect on improvement in clinical outcomes, especially when included for billing purposes only.¹¹ Given these realities, the Agency for Healthcare Research and Quality regards outcome-based measures as the “gold standard” in quality measurement.¹² Expert groups such as the Healthcare Association of New York State suggest that “regulators and payers should focus on overall performance (outcome measures), and defer the operations and use of process measures for internal quality improvement by healthcare providers.”¹³

Process-based measures dominate the OCM and other oncology quality assessment frameworks, yet outcome-based measures have an important role to play. Outcome-based measures are directly connected to real-world outcomes, ranging from hospital admissions to mortality and patient-reported outcomes (PROs), reflecting what patients and providers care about most.

Outcome Measures in Existing Oncology Quality Frameworks

Of the 142 quality measures from 7 oncology quality assessment frameworks that we reviewed, 80.3% (n = 114) were process-based measures and 19.7% (n = 28) were outcome-based measures. An earlier analysis of the National Quality Measures Clearinghouse found an even lower proportion of outcome-based measures (7.1%) based on a total of 1958 quality indicators from a wide range of therapeutic areas.¹¹ Of those nearly 2000 indicators, only 1.6% were patient-reported outcome measures (PROMs).¹¹

We condensed the 28 outcome-based measures into 23 unique outcome measures by merging identical or near-identical measures and grouping them into 5 categories: (1) admissions and hospital visits (including emergency department [ED] visits), (2) hospice care, (3) mortality, (4) PROs, and (5) adverse events (AEs) (Table 2).

Admissions and hospital visits. Admissions and hospital visits, after risk adjustment, are important indicators of the appropriateness and timeliness of care. Up to 50% of ED visits are related to complications from chemotherapy, which can potentially indicate suboptimal management of the disease and care coordination (ranging from information sharing among providers to education about end-of-life care).¹⁴

The significant variation observed in admission rates and hospital/ED visits between different providers, even when controlling for other factors, has spurred research related to avoidable hospitalizations and appropriateness of care, especially in late-stage cancer care.¹⁵ In 2016, for example, CMS announced the inclusion of inpatient admissions and ED visits for patients receiving outpatient chemotherapy in its Hospital Outpatient Quality Reporting Program.¹⁶ Including hospital visits in payment models aims to “encourage reporting facilities to take steps to prevent and improve management of side effects and complications from treatment.”¹⁷

Hospice care. Although it offers patients, caregivers, and the healthcare system advantages relative to hospital settings, hospice care is generally underutilized. Quality measures related to hospice care may improve the quality of life of patients with late-stage cancer, reduce spending, and reduce burden among providers and caretakers.

Allowing patients with cancer to receive palliative care in a hospice setting is traditionally associated with improvements in quality of life, as well as system efficiencies. Yet, up to 66% of patients with cancer are not enrolled in hospice in the last 30 days of life, and less than 29% are enrolled for at least 2 months (considered appropriate care), based on an international review of 78 studies published between 1998 and 2011.¹⁸ One factor explaining the suboptimal transition to palliative care may be doctors’ tendency to overestimate survival prospects of a patient.¹⁹ Hospice-related measures aim to reduce wasteful spending on care that is unlikely to improve clinical outcomes and to provide patients with a higher quality of life. Specific provisions may be needed for palliative chemotherapy and other treatments that can extend the length of life while in palliative care.

Although outcome-based measures in this domain tend to focus on hospice settings alone, recent discussions suggest that palliative care may improve patient quality of life if initiated earlier in the cancer treatment course.^20,21 In 2012, the National Quality Forum endorsed 14 measures related to end-of-life care, of which several are outcome-based, including “comfortable dying” and bereaved family survey measures.²² Finally, patient- and caregiver-reported outcomes may have a more prominent role to play, as end-of-life care should reflect patient and caregiver preferences.²³

Mortality. Mortality is a common outcome indicator used in both clinical practice and clinical trials, and it may be reported in different ways (eg, patient mortality over a specific period, overall survival, progression-free survival, by the setting of a patient’s death). Additionally, the setting of death (in a hospital vs at home or in a hospice) may play an important role in patients’ quality of life^18,24 and is sometimes used as part of mortality-related quality measures.

PROs. Ranging from pain to social function evaluation, PROs are increasingly used to evaluate appropriateness of care given their ability to reflect patient needs and preferences, which may vary significantly. The FDA issued a PRO-specific guidance in 2009,²⁵ defining PROs as “any report of the status of a patient’s health condition that comes directly from the patient, without interpretation of the patient’s response by a clinician or anyone else,” and stating that in general, “findings measured by a well-defined and reliable PRO instrument in appropriately designed investigations can be used to support a claim in medical product labeling if the claim is consistent with the instrument’s documented measurement capability.”²⁵

PROs are relevant in both early and late disease stages, complementing other measures that address outcomes in a hospital setting. In 2016, a multistakeholder roundtable on improving oncology measurement recommended that PROs be collected “before, during, and after treatment.”²⁶ Although more evidence is needed, PROMs have been studied for use during an initial consultation and during shared decision making regarding patient care, aside from tracking treatment progress and patient satisfaction.²⁷ A 2013 report on PROs by the National Quality Forum indicates that several guiding principles for selecting PROMs should be followed: psychometric soundness, person-centricity, meaningfulness, amenability to change, and implementability.²⁸ Progress in drawing on electronic PROs may catalyze the ability of plans to systematically and reliably collect patient- and caregiver-reported outcomes, especially if they impose minimal burden on staff and patients.²⁹

Nonetheless, the use of PROs has been associated with multiple challenges, including representativeness, inclusion of PROs in medication labels, necessity for both standardized and customizable PROs, and operational and organizational barriers to collecting and analyzing them.³⁰ In addition, PRO collection is often resource-intensive, the validity of disease-specific PROs may be limited, and many PROs lack predictive value.³⁰ Despite these challenges, PROs can be useful tools to obtain insight into patient needs and preferences in order to make better patient-level, as well as policy, decisions and to support further research and development.

AEs. Quality measures based on reporting of AEs aim to lower the number of avoidable incidents, potentially shortening hospitalization length and reducing costs as well as mortality. Initial guidance on the reporting of AEs in oncology trials was published by the Consolidated Standards of Reporting Trials in 2003, and AEs related to oncology care are now understood better.³¹ However, AEs in clinical settings are thought to be significantly underreported, partly driven by voluntary reporting and the use of instruments that may be prone to lower sensitivity.³² Improvements in documentation and electronic reporting are expected to improve the reliability of data about AEs observed in clinical practice (most hospitals do not use electronic health records [EHRs] to “directly measure [or record] patient harm”).³³

If implemented broadly, the magnitude of potential improvements may be significant: Studies have shown that AEs can extend the length of hospitalizations, increase costs of care, and increase mortality up to 2-fold.³⁴

DISCUSSION

Evidence From Early Quality Initiatives in Oncology

Although no comprehensive evaluation of OCM has taken place, limited evidence suggests that previous quality initiatives with outcome-based components have improved care while reducing costs. For instance, during a 2-year pilot in Texas involving 221 oncology patients (Innovent Oncology program by McKesson Specialty Health, Texas Oncology, and Aetna), savings of more than $500,000 were achieved.^35,36 The program has also been shown to improve adherence to clinical pathways and clinical outcomes: Pathway adherence has increased from 63% to 76%; reductions in ED visits, hospital admissions, and hospital days of 48%, 34%, and 44%, respectively, were observed; and average inpatient days decreased from 2.1 to 1.2 days.^32,33 Innovent Oncology based its value-based reimbursement on 3 pillars: (1) Level I Pathways Program (aiming to increase the use of evidence-based treatment guidelines), (2) clinical benchmarking (based on a number of quality indicators), and (3) contract negotiation services.³⁷ Among the quality measures included have been gastrointestinal toxicities, infection, thromboembolic events, pain, and depression.²³

Similarly, an oncology pilot by UnitedHealthcare that drew on episode payments for more than 800 patients with breast, colon, and lung cancer in 5 oncology practices achieved net savings of more than $33 million (a 34% reduction of the predicted total medical cost).³⁸ Some of the key quality measures used by this pilot included ED and hospitalization rates, admissions for cancer symptoms, febrile neutropenia occurrence rate, admissions for treatment-related symptoms, days from last chemotherapy to death, and hospice days for patients who died.³⁸

However, a lack of a counterfactual (via a matched control group, for example) undermines a direct causal link between quality measurement and observed outcomes in these pilots, and more comprehensive evaluations are still needed.

Recommendations for Outcome-Based Measures in Oncology

A synthesis and recommendations for future core outcome sets in oncology are available in Table 3. Measures that are generally seen as being closely tied to the quality of care received by oncology patients were classified into 5 quality domains identified by a CMMI APM toolkit³⁹: clinical care, safety, care coordination, patient and caregiver experience, and population health and prevention. When possible, this set of outcome measures should be tailored to unique patient populations, diseases, providers, or other factors in individual payment models. In addition, some measures, such as hospice care—albeit appropriate for patients with more advanced disease—may not be relevant for patients with curable, early-stage cancer. Future oncology APMs should implement outcome measures relevant to the disease type and stage(s). For a detailed justification and discussion of individual categories and measures, please see eAppendix B.

Collecting outcomes data in all 5 domains of cancer care is fraught with challenges that have been documented in multiple studies. For example, to measure and track outcomes properly, programs often require big data that involve multiple sources, such as EHRs, health insurance claims, and patient/caregiver surveys; however, whether data are complete and accessible and can be translated into clinical practice remains a challenging issue.⁴⁰ Many outcome-based measures rely on administrative claims data, which tend to have a long report lag. Some outcomes data, such as hospice care, may be challenging to access, especially when the patient is transferred from one payer to another. Chung and Basch⁴¹ discuss specific challenges related to collecting and using patient-generated health data (including PROs), ranging from “provider concerns, workflow issues, standardization of patient-generated health data and interoperability of devices/sensors, security and privacy issues” to a “lack of the necessary EHR functionalities and software innovations.” Additionally, statistical challenges related to missing values, highly dimensional data sets, and confounding (bias) require robust statistical approaches that are not yet available in broad clinical practice.⁴² Nonetheless, new approaches are being tested as outcome measures gain support from clinicians, patients, and payers, including a collaborative pilot on establishing a framework to evaluate real-world end points in advanced non—small cell lung cancer led by the Friends of Cancer Research and supported by both public and private stakeholders.⁴³

CONCLUSIONS

As highlighted in this paper, both OCM and other quality initiatives in oncology rely on process- or outcome-based quality measures to determine the quality of care and—in some cases—the level of payment. Given evidence from the literature and an analysis of 7 oncology quality assessment frameworks, we presented a set of outcome-based measures for consideration in future payment models in oncology. Although some measures may be omitted in specific cases, we believe the inclusion of measures related to all 5 domains—clinical care, safety, care coordination, patient and caregiver experience, and population health and prevention—is highly desirable in future oncology APMs. Selective measurement of 1 outcome domain may create perverse incentives for providers to improve performance by underutilizing appropriate care and jeopardize optimal patient outcomes. Where appropriate, indication-specific quality measures should be included to account for quality-of-care complexities associated with individual cancer types and disease stages.

Overcoming hurdles to broader utilization of outcome-based measures in oncology will require a consensus between both payers and providers. These efforts should highlight the benefits of implementing outcome-based measures in oncology APMs (especially relative to the cost of implementation) and solutions to data and evaluation challenges (including risk adjustment and bias control). Future research is also needed to develop best practices for the inclusion and implementation of outcome measures in oncology clinical pathways.⁴⁴ Additional considerations include developing strategies for quality control, dispute resolution, and administrative burden on providers and payers.

Given the steadily increasing costs of oncology care and, in some cases, the availability of multiple high-cost treatment options for patients with cancer, oncology care is in need of a more rigorous approach to quality assessment. The success of emerging oncology APMs will depend on a robust set of quality indicators that are relevant to patients, providers, and payers alike. 

Acknowledgments

The authors thank Jacqueline Vanderpuye-Orgle, Jeffrey Lemay, Zachary Wessler, and Harshali Patel for helpful comments on an earlier draft of this paper. The authors retained full control over research design, analysis, and findings presented herein.Author Affiliations: University of Southern California Schaeffer Center for Health Policy and Economics (JPH), Los Angeles, CA; RAND Corporation (JPH), Los Angeles, CA; Center for the Evaluation of Value and Risk in Health, Institute for Clinical Research and Health Policy Studies, Tufts Medical Center (PJL, PJN), Boston, MA.

Source of Funding: This research was funded by a grant from Amgen Inc via a contract with Tufts Medical Center. Additional research support for Dr Hlávka was provided by the National Institute on Aging of the National Institutes of Health under Award Number R01AG062277. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

Author Disclosures: Dr Lin reports being a consultant or paid advisory board member for Takeda, Otsuka, and Alliance for Aging Research. Dr Neumann reports being a consultant or paid advisory board member for Precision Health Economics, Avexis, AbbVie, Novartis Pharmaceuticals, Research Triangle Institute, Merck, Genentech, Bluebird Bio, GlaxoSmithKline, DePuy, Otsuka, ICON, Indivior, Acorda, and Biogen, and has received lecture fees for speaking at the invitation of AbbVie, Pfizer, Celgene, Sanofi, and GlaxoSmithKline. Dr Hlávka reports no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article.

Authorship Information: Concept and design (JPH, PJL, PJN); acquisition of data (JPH); analysis and interpretation of data (JPH, PJL, PJN); drafting of the manuscript (JPH, PJL); critical revision of the manuscript for important intellectual content (JPH, PJL, PJN); statistical analysis (JPH); provision of patients or study materials (PJL); obtaining funding (PJL); administrative, technical, or logistic support (PJL); and supervision (PJL, PJN).

Address Correspondence to: Pei-Jung Lin, PhD, Tufts Medical Center, 800 Washington St, Box 63, Boston, MA 02111. Email: PLin@tuftsmedicalcenter.org.REFERENCES

1. Valuck T, Blaisdell D, Dugan DP, et al. Improving oncology quality measurement in accountable care: filling gaps with cross-cutting measures. J Manag Care Spec Pharm. 2017;23(2):174-181. doi: 10.18553/jmcp.2017.23.2.174.

2. Gilbert E, Sherry V, McGettigan S, Berkowitz A. Health-care metrics in oncology. J Adv Pract Oncol. 2015;6(1):57-61.

3. Core measures. CMS website. cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/QualityMeasures/Core-Measures.html. Updated July 28, 2017. Accessed May 30, 2019.

4. Consensus core set: medical oncology measures: version 1.0. CMS website. cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/QualityMeasures/Downloads/Medical-Oncology-Measures.pdf. Updated November 23, 2015. Accessed May 30, 2019.

5. Measures. American Society for Radiation Oncology website. astro.org/Daily-Practice/QPP/Measures. Updated January 29, 2018. Accessed May 30, 2019.

6. Macefield RC, Jacobs M, Korfage IJ, et al. Developing core outcomes sets: methods for identifying and including patient-reported outcomes (PROs). Trials. 2014;15:49. doi: 10.1186/1745-6215-15-49.

7. Rubin HR, Pronovost P, Diette GB. The advantages and disadvantages of process-based measures of health care quality. Int J Qual Health Care. 2001;13(6):469-474. doi: 10.1093/intqhc/13.6.469.

8. Sohn W, Resnick MJ, Greenfield S, et al. Impact of adherence to quality measures for localized prostate cancer on patient-reported health-related quality of life outcomes, patient satisfaction, and treatment-related complications. Med Care. 2016;54(8):738-744. doi: 10.1097/MLR.0000000000000562.

9. Hussey PS, Friedberg MW, Anhang Price R, Lovejoy SL, Damberg CL. Episode-based approaches to measuring health care quality. Med Care Res Rev. 2017;74(2):127-147. doi: 10.1177/1077558716630173.

10. Albert JM, Das P. Quality assessment in oncology. Int J Radiat Oncol Biol Phys. 2012;83(3):773-781. doi: 10.1016/j.ijrobp.2011.12.079.

11. Porter ME, Larsson S, Lee TH. Standardizing patient outcomes measurement. N Engl J Med. 2016;374(6):504-506. doi: 10.1056/NEJMp1511701.

12. Types of health care quality measures. Agency for Healthcare Research and Quality website. ahrq.gov/talkingquality/measures/types.html. Published February 2015. Accessed May 30, 2019.

13. Healthcare Association of New York State’s Quality Institute. Measures that matter: moving from measure madness to measures that matter. Healthcare Association of New York State website. hanys.org/quality/clinical_operational_oversight/measures_that_matter/docs/mtm_report.pdf. Accessed May 30, 2019.

14. Kolodziej M, Hoverman JR, Garey JS, et al. Benchmarks for value in cancer care: an analysis of a large commercial population. J Oncol Pract. 2011;7(5):301-306. doi: 10.1200/JOP.2011.000394.

15. Brooks GA, Abrams TA, Meyerhardt JA, et al. Identification of potentially avoidable hospitalizations in patients with GI cancer. J Clin Oncol. 2014;32(6):496-503. doi: 10.1200/JCO.2013.52.4330.

16. CMS proposes hospital outpatient prospective payment changes for 2017. CMS website. cms.gov/newsroom/fact-sheets/cms-proposes-hospital-outpatient-prospective-payment-changes-2017. Published July 6, 2016. Accessed May 30, 2019.

17. Admissions and emergency department visits for patients receiving outpatient chemotherapy. CMS website. cmit.cms.gov/CMIT_public/ReportMeasure?measureRevisionId=672. Accessed May 30, 2019.

18. Langton JM, Blanch B, Drew AK, Haas M, Ingham JM, Pearson SA. Retrospective studies of end-of-life resource utilization and costs in cancer care using health administrative data: a systematic review. Palliat Med. 2014;28(10):1167-1196. doi: 10.1177/0269216314533813.

19. Christakis NA, Lamont EB. Extent and determinants of error in doctors’ prognoses in terminally ill patients: prospective cohort study. BMJ. 2000;320(7233):469-472. doi: 10.1136/bmj.320.7233.469.

20. Temel JS, Greer JA, Muzikansky A, et al. Early palliative care for patients with metastatic non—small-cell lung cancer. N Engl J Med. 2010;363(8):733-742. doi: 10.1056/NEJMoa1000678.

21. Ferrell BR, Temel JS, Temin S, et al. Integration of palliative care into standard oncology care: American Society of Clinical Oncology clinical practice guideline update. J Clin Oncol. 2017;35(1):96-112. doi: 10.1200/JCO.2016.70.1474.

22. Endorsement summary: palliative care and end-of-life care measures. National Quality Forum website. qualityforum.org/WorkArea/linkit.aspx?LinkIdentifier=id&ItemID=70221. Published February 2012. Accessed May 30, 2019.

23. Institute of Medicine. Dying in America: Improving Quality and Honoring Individual Preferences Near the End of Life. Washington, DC: The National Academies Press; 2015.

24. Zhang B, Nilsson ME, Prigerson HG. Factors important to patients’ quality of life at the end of life. Arch Intern Med. 2012;172(15):1133-1142. doi: 10.1001/archinternmed.2012.2364.

25. Guidance for industry: patient-reported outcome measures: use in medical product development to support labeling claims. FDA website. fda.gov/media/77832/download. Published December 2009. Accessed May 30, 2019.

26. Valuck T, Blaisdell D, Dugan D, et al. Improving oncology quality measurement in accountable care. National Pharmaceutical Council website. npcnow.org/system/files/research/download/npc-improving-oncology-quality-measures-final.pdf. Accessed May 30, 2019.

27. Holmes MM, Lewith G, Newell D, Field J, Bishop FL. The impact of patient-reported outcome measures in clinical practice for pain: a systematic review. Qual Life Res. 2017;26(2):245-257. doi: 10.1007/s11136-016-1449-5.

28. Patient-reported outcomes (PROs) in performance measurement. National Quality Forum website. qualityforum.org/WorkArea/linkit.aspx?LinkIdentifier=id&ItemID=72537. Published January 10, 2013. Accessed May 30, 2019.

29. Jensen RE, Snyder CF, Abernethy AP, et al. Review of electronic patient-reported outcomes systems used in cancer clinical care. J Oncol Pract. 2014;10(4):e215-e222. doi: 10.1200/JOP.2013.001067.

30. AMCP partnership forum: improving quality, value, and outcomes with patient-reported outcomes. J Manag Care Spec Pharm. 2018;24(3):304-310. doi: 10.18553/jmcp.2018.17491.

31. Sivendran S, Latif A, McBride RB, et al. Adverse event reporting in cancer clinical trial publications [erratum in J Clin Oncol. 2014;32(8):866. doi: 10.1200/JCO.2014.56.6290]. J Clin Oncol. 2014;32(2):83-89. doi: 10.1200/JCO.2013.52.2219.

32. Classen DC, Resar R, Griffin F, et al. ‘Global trigger tool’ shows that adverse events in hospitals may be ten times greater than previously measured [erratum in Health Aff (Millwood). 2011;30(6):1217. doi: 10.1377/hlthaff.2011.0311]. Health Aff (Millwood). 2011;30(4):581-589. doi: 10.1377/hlthaff.2011.0190.

33. Classen DC, Griffin FA, Berwick DM. Measuring patient safety in real time: an essential method for effectively improving the safety of care. Ann Intern Med. 2017;167(12):882-883. doi: 10.7326/M17-2202.

34. Classen DC, Pestotnik SL, Evans RS, Lloyd JF, Burke JP. Adverse drug events in hospitalized patients: excess length of stay, extra costs, and attributable mortality. JAMA. 1997;277(4):301-306. doi: 10.1001/jama.1997.03540280039031.

35. Wen L, Divers C, Lingohr-Smith M, Lin J, Ramsey S. Improving quality of care in oncology through healthcare payment reform. Am J Manag Care. 2018;24(3):e93-e98.

36. Hoverman JR, Klein I, Harrison DW, et al. Opening the black box: the impact of an oncology management program consisting of level I pathways and an outbound nurse call system. J Oncol Pract. 2014;10(1):63-67. doi: 10.1200/JOP.2013.001210.

37. Hoverman JR, Dronen D, Gomez D, Taylor S. Realizing pay for performance: moving from fee-for-service to value-based reimbursement. Association of Northern California Oncologists website. anco-online.org/P3InnoventANCOWebinar.pdf. Published October 26, 2010. Accessed May 30, 2019.

38. Newcomer LN, Gould B, Page RD, Donelan SA, Perkins M. Changing physician incentives for affordable, quality cancer care: results of an episode payment model. J Oncol Pract. 2014;10(5):322-326. doi: 10.1200/JOP.2014.001488.

39. Alternative payment model design toolkit. CMS website. qpp-cm-prod-content.s3.amazonaws.com/uploads/36/CMS%20QPP%202017%20APM%20Toolkit_2017%2010%2018%20Remediated.pdf. Accessed May 30, 2019.

40. Adibuzzaman M, DeLaurentis P, Hill J, Benneyworth BD. Big data in healthcare—the promises, challenges and opportunities from a research perspective: a case study with a model database. AMIA Annu Symp Proc. 2018;2017:384-392.

41. Chung AE, Basch EM. Potential and challenges of patient-generated health data for high-quality cancer care. J Oncol Pract. 2015;11(3):195-197. doi: 10.1200/JOP.2015.003715.

42. Lee CH, Yoon HJ. Medical big data: promise and challenges. Kidney Res Clin Pract. 2017;36(1):3-11. doi: 10.23876/j.krcp.2017.36.1.3.

43. Establishing a framework to evaluate real-world endpoints. Friends of Cancer Research website. focr.org/sites/default/files/pdf/RWE_FINAL%207.6.18.pdf. Published July 10, 2018. Accessed May 30, 2019.

44. Zon RT, Frame JN, Neuss MN, et al. American Society of Clinical Oncology policy statement on clinical pathways in oncology. J Oncol Pract. 2016;12(3):261-266. doi: 10.1200/JOP.2015.009134.

Download PDF: Outcome Measures for Oncology Alternative Payment Models: Practical Considerations and Recommendations