Using the NQF Measure Incubator to Develop Patient-Reported Outcome Performance Measures in Palliative Cancer Care

Cancer is currently the second leading cause of death within the United States, with 17 million new cases of cancer predicted to be diagnosed in 2016.¹ The lifetime risk of developing cancer among US men is 50%, and 38% among US women. It is estimated that approximately 622,690 people across the United States will die from various forms of cancer in 2016. Despite the statistics, treatments for various forms of cancer have improved; for example, we now have better protocols for the early detection of cancer as well as more effective therapies for treatment. Moreover, since the late 1970s, 5-year survival rates have increased from 50% to almost 70% in 2011.²

Regardless of the advances in medications and positive trends in survival rates, however, the general consensus among cancer professionals is that the quality of cancer care is suboptimal.³ These problems are inherent, largely attributable to a fragmented system that lacks coordination of care, does not ensure access to care, and is inefficient in its use of resources. As a result, some patients fail to receive effective treatments, receive treatments that compound their risk and lower their chance for survival, or have significant side effects and do not improve their chances for survival.⁴ These quality deficiencies are pervasive and found throughout the various phases of cancer care, including diagnosis, initial treatment, survivorship, and palliative care.

To improve the overall quality of cancer care, valid and reliable measures of practice, performance, and patient outcomes must be developed and implemented.⁵ Cancer organizations, such as the American Society of Clinical Oncology (ASCO), the Alliance of Dedicated Cancer Centers, and the American Society for Radiation Oncology, among others, have maintained portfolios and registries of healthcare performance measures that focus on individual cancers, assess specific providers, and address specific processes of care, particularly initial management strategies. Despite the development of approximately150 cancer-related measures, numerous noticeable gaps in the current measurement environment persist, as several aspects of cancer care remain unaddressed. Some of these include,

• Patient assessment and supportive care
• Patient experience of care
• Use of palliative care
• Interdisciplinary and multidisciplinary coordinated care
• Assessment and support of patients and caregivers at the end of life (EOL).⁶

Many professional organizations and specialty societies, that lead rigorous measure development activities on numerous cancer-related topics, recognize the relatively narrow focus of existing cancer quality measures, and the increasing demand for measures from patients, payers, and other organizations to address the shortcomings of the current system.

The dearth of reliable and valid measures is most evident in the area of providing palliative care to cancer patients. Large-scale, post-death surveys demonstrate that a considerable portion of families feels the needs of the patient were not attended to adequately. For example, 18% of hospitalized patients did not consider their pain adequately controlled, and 42% of cancer patients did not receive appropriate levels of analgesia.⁷ While pharmacologic interventions are often the first level of treatment for cancer pain, non-pharmacologic techniques have also proven effective in addressing pain. These may include physical modalities such as exercise, acupuncture, and massage; and psychosocial modalities, such as relaxation techniques, support groups, and family counseling. Studies have demonstrated that the consistent use of an interdisciplinary approach, that combines pharmacologic agents with other forms of treatment, may alleviate the need for invasive interventions, such as intrathecal opioids, and facilitate a safe discharge to the community.⁸

Organizations, such as ASCO, have postulated, that the development and validation of cross-cutting, patient-reported outcome—based performance measures (PRO-PMs), to assess the quality of care for cancer patients in palliative care, is essential. However, a number of persistent problems has hampered the development of such measures:

• Lack of available data to provide sufficient detail for a broad-based assessment of cancer care in palliative settings
• Understanding how to create appropriate symptom and functional status assessments
• Implementation of the measures to ensure that the appropriate information is captured and used to report on the measure
• The time it would take to develop and test these measures.

Additionally, current cancer patients in palliative care are often viewed as too ill to fill in an assessment tool due to cognitive impairment, or may not understand what is required of them.⁹ The validity and reliability of the tool, and the complexity of the PRO-PM to provide a meaningful and useful metric, are also barriers for staff employed at palliative care facilities.¹⁰

Adequately developed PRO-PMs in palliative care can be helpful in identifying a patient’s response to a particular treatment modality, particularly, in terminal cases, when the difference in survival among each of the options is small or non-existent.¹¹ Leveraging a psychometrically validated instrument that is inclusive of patient input across a number of significant domains would be beneficial for providers to understand the needs of their cancer patients. These may include:

• Individual wishes for medical care
• Issues in palliative care that trouble patients
• Priorities for treatment.

Understanding the most important and significant of these needs is crucial for both the development and implementation of patient-reported outcome measures.¹² There are a number of valid instruments that have been developed that measure the quality of life (QoL) in palliative cancer care, such as the European Organization for Research and Treatment of Cancer (EORTC), the Functional Assessment of Cancer Treatment (FACT), and the Palliative Care Quality of Life Instrument (PQLI). Each of these encompasses a short questionnaire that provides patients an ability to voice their opinions and concerns regarding EOL care, and has shown to be a limited burden on patients.^13,14

The National Quality Forum and the Measure Incubator

The National Quality Forum (NQF) is a consensus-building organization that evaluates healthcare performance measures and facilitates a defined process to endorse these measures. Over the past decade, NQF has conducted several projects to evaluate and endorse cancer care measures under the guidance of multi-stakeholder committees that represented payer, consumer, quality improvement, provider, and patient perspectives. Some of these measures include areas such as access and cultural competence; prevention and screening; diagnosis and treatment of breast, colorectal, and prostate cancers; symptom management; and EOL care. In 2008, NQF led a workshop that highlighted key measurement gaps in cancer despite the rapid proliferation of measures across a number of clinical areas. These gaps included many of the topic areas identified by organizations such as ASCO, and emphasized that areas such as shared decision-making, EOL care, care coordination, and patient and family engagement were not adequately represented.¹⁵

In 2012, the Measure Applications Partnership (MAP), a public/private endeavor, enacted as part of the passage of the Affordable Care Act (ACA), and overseen by NQF to guide HHS on the selection of performance measures for federal programs, suggested there was an immediate need to address the fundamental cross-cutting aspects of care that are relevant to all cancer patients throughout the trajectory of their illness. Both NQF and MAP stated that a more comprehensive measurement system that focused on patients, rather than individual providers or distinct care settings, was preferable.

However, the process of designing, testing, and disseminating quality measures to fill these priority gaps is burdensome, costly, and time consuming. On average, measure development and testing take approximately 2 to 3 years. An additional difficulty arises from testing measures, which currently requires implementation across real-world practice sites in order to assure validity and feasibility across multiple clinical sites and vendors. Further, no linear pathway exists to coordinate and invest in measures most relevant for improving health outcomes. As a result, the persistence of measure gaps undermines the quality management enterprise. NQF is addressing the market inadequacies in healthcare measure development and testing through its Measure Incubator: a platform that can shorten the time, and decrease the burden of measure development and enable the focus on novel and needed measures. The Incubator creates the foundation for the next generation of performance measures, including measures of PRO-PMs, measures of resource use and population health, and electronic clinical quality and registry-based measures. It can also accelerate improvement in quality by creating a more collaborative model of measure development that focuses on leveraging the appropriate stakeholders, resources, and data assets, at the time the measure is created, to help complete the development process in an efficient and expeditious manner. This vision mirrors the “incubation” concept used in other contexts to support innovation and transformation of innovative ideas into timely solutions.

Though NQF is playing a facilitative role, the organization is not directly engaging in measure development. NQF also developed strict conflict-of-interest guidelines to prevent any incubated measure to receive preferential status for endorsement. The Incubator is building on the substantial base of priority setting—the result of the MAP and the NQF endorsement process. This new collaborative model can support the development of meaningful, evidence-based measures for a wide range of clinical settings and disciplines. Through greater upstream collaboration with guideline developers and downstream collaboration with stakeholders who support the measure, the development of a measure can be focused where there is the highest likelihood for meaningful improvement in patient outcomes.

The Use of the NQF Measure Incubator to Develop PRO-PMs in Palliative Care

The NQF Measure Incubator can utilize the information gathered through these instruments to help develop robust and valid PRO-PMs by following a pathway outlined in a 2013 report by NQF entitled, “Patient Reported Outcomes (PROs) in Performance Measurement.” This pathway was designed to ensure that the data from the validated instrument is used at the beginning of the process to develop the measure in a manner that is both meaningful to patients and implementable to providers. The pathway consists of the following overarching components:

Identifying a PRO for development.

Each of the tools currently used to assess a patient’s feelings about palliative care focuses on the most relevant and meaningful issues to them, including their priority of choice of treatment for their cancer and their wishes for medical care. Recent studies indicate that physicians must understand the importance of communication in the context of advanced illness, when the achievement of a patient’s specific needs assumes priority over inappropriate prolongation of care. These data could lay the foundation for the development of a PRO-PM in palliative care, as this is one meaningful area that could be developed into a measure using existing survey tools, which would ensure that the most important quality issue is developed into a measure concept.

Each of the referenced tools deals with relevant QoL issues, such as treatment choices and preferred outcomes. Through the use of the Incubator, NQF staff examines the data at the beginning of the development process bringing together the appropriate subject matter experts in palliative care, and the expertise in measurement science from NQF to understand how various interventions affect the outcomes that are important to patients in a comprehensive manner. For example, a 2004 study that used the Palliative Care Quality of Life Instrument or PLQI indicated that cancer patients in palliative care need the support of their family and friends when considering treatment options. Therefore, understanding how the provider brings them into the decision-making process and how that would affect the overall choice of treatment could lead to a PRO-PM that would be relevant to both patients and providers.

Suitability of the PRO-PM for use in palliative care.

Part of the measure incubation process is examining the suitability for the development of PRO-PMs in palliative care, which encompasses multiple concepts, including whether the measure addresses an unmet measurement need, and the instruments are reliable and valid to support the development of a PRO-PM. The data used to both develop and test the measure would come from instruments that have been psychometrically tested and embrace concepts that are meaningful to patients—this would ensure protection against the development of a measure that would not be directly applicable to cancer patients in palliative care. The parameters of the measure would be tightly defined so as to increase its usability in these types of settings and with patients diagnosed with various forms of cancer.

Specifying and testing the measure.

While it should, again, be noted that NQF does not participate in the development of the measure, NQF has promulgated criteria that represent a “gold standard” for both development and use. The NQF Measure Incubator assists measure developers in applying those criteria to ensure that their measures are reliable, valid, and feasible, and can be implemented in the settings for which they are intended. Additionally, given that the measure can be initially tested with data from a validated instrument, it can be determined, early on, if the metric generated from the measure is useful for providers to better understand patient needs. Upon the completion of the process, the organization that funded the measure and the developer can determine if the measure should be considered for NQF endorsement. This is an independent process and does not involve any input from NQF, nor does participation in the Incubator ensure that endorsement is guaranteed.

EBO

The NQF Measure Incubator provides a unique opportunity to allow organizations, measure developers, subject matter experts, and other stakeholder to collectively develop measures that fill existing gaps in quality measurement. By creating a process that is both efficient and cost-effective, the ability to get to “measures that matter” for both patients and providers is not as burdensome or time-consuming. It also allows for the development of PRO-PMs in specific areas, such as palliative cancer care, that lend a voice to patient needs and concerns, and develop much needed quality measures that improve their overall QoL.

Author information

Jason Goldwater, MA, MPA, is a senior director, National Quality Forum.

Nicole Silverman, MBA, is the chief operating officer, National Quality Forum.

Karen Johnson, MS, is a senior director, National Quality Forum.

Rachel Roiland, PhD, is a senior program manager, National Quality Forum.

Address for correspondence

Jason C. Goldwater, MA, MPA

Senior Director

National Quality Forum

1030 15th Street, NW, Suite 800

Washington, DC 20005

E-mail: JGoldwater@qualityforum.org

References

1. Cancer facts and figures 2016. American Cancer Society website. http://www.cancer.org/acs/groups/content/@research/documents/document/acspc-047079.pdf. Accessed May 3, 2016.
2. Spinks TE, Walters R, Feeley TW, et al. Improving cancer care through public reporting of meaningful quality measures. Health Aff (Millwood). 2011;30(4):664-672. doi:10.1377/hlthaff.2011.0089.
3. Smith BD, Smith GL, Hurria A, Hortobagyi GN, Buchholz TA. Future of cancer incidence in the United States: burdens upon an aging, changing nation. J Clin Oncol. 2009;27(17):2758-2765. doi:10.1200/JCO.2008.20.8983.
4. Hassett, MJ, McNiff, KK, Dicker, AP, et al. High-priority topics for cancer quality measure development: results of the 2012 American Society of Clinical Oncology Collaborative Cancer Measure Summit. J Oncol Pract. 2014;10(3):e160-e166. doi:10.1200/JOP.2013.001240.
5. Institute of Medicine (US) and National Research Council (US) National Cancer Policy Board; Hewitt ME, Simone JV, eds. Cancer Care Quality Assurance. In: Ensuring Quality Cancer Care. Washington, DC: National Academies Press; 1999.
6. McGlynn EA, Asch SM, Adams J, et al. The quality of health care delivered to adults in the United States. N Engl J Med. 2003;348(26):2635-2645. doi:10.1056/nejmsa022615.
7. Blayney DW. Defining quality: QOPI is a start. J Oncol Pract. 2006;2(5):203. doi:10.1200/jop.2.5.203.
8. Seow H, Snyder CF, Shugarman LR, et al. Developing quality indicators for cancer end-of-life care: proceedings from a national symposium. Cancer. 2009;115(17):3820-3829. doi:10.1002/cncr.24439.
9. Lorenz KA, Lynn J, Dy SM, et al. Evidence for improving palliative care at the end of life: a systematic review. Ann Intern Med. 2008;148(2):147-159. doi:10.7326/0003-4819-148-2-200801150-00010.
10. Mularski RA, Dy SM, Shugarman LR, et al. A systematic review of measures of end-of-life care and its outcomes. Health Serv Res. 2007;42(5):1848-1870. doi:10.1111/j.1475-6773.2007.00721.x.
11. Bauswein C, Simon ST, Benalia H, et al; PRISMA. Implementing patient reported outcome measures (PROMs) in palliative care—users’ cry for help. Health Qual Life Outcomes. 2011;9:27. doi:10.1186/1477-7525-9-27.
12. Hearn J, Higginson IJ. Development and validation of a core outcome measure for palliative care: the palliative care outcome scale. Palliative Care Core Audit Project Advisory Group. Qual Health Care. 1999;8(4):219-227. doi:10.1136/qshc.8.4.219.
13. Jack B, Hillier V, Williams A, Oldham J. Hospital based palliative care teams improve the insight of cancer patients into their disease. Palliat Med. 2004;18(1):46-52. doi:10.1191/0268216304pm846oa.
14. Baumrucker SJ. The Palliative Care Quality of Life Instrument (PQLI): A new tool for measuring quality of life. Am J Hosp Palliat Care. 2004;21(2):87-89. doi:10.1177/104990910402100203.
15. National Voluntary Consensus Standards for Quality of Cancer Care. Washington, DC: National Quality Forum; 2009.

Download PDF: Using the NQF Measure Incubator to Develop Patient-Reported Outcome Performance Measures in Palliative Cancer Care