Efficiency and Its Measurement: What Practitioners Need to Know

Objective:

To help health professionals understand and evaluate the concept of efficiency and its measurement in practice.

Study Design:

We reviewed conceptual and practical analyses of healthcare efficiency and its measurement and describe our findings.

Methods:

We searched the following 3 sources: the MEDLINE and EconLit databases for articles published from 1990 to 2008 using the keywords efficiency, inefficiency, productivity, and economic profiling; seminal economic studies of efficiency identified in MEDLINE or EconLit or in economics reference materials; and the “gray literature” on efficiency measures developed by private organizations.

Results:

An essential element of efficiency is that healthcare outputs are as large as possible given their inputs. An efficiency measure specifies outputs and inputs appropriate to the goals of the assessment. Participants in the healthcare system have differing perspectives about the goals of efficiency assessments, relevant outputs and inputs, and validity of measures.

Conclusion:

The broad meaning and the value of healthcare efficiency seem uncontroversial, yet any particular application may be confronted with conflicting perspectives and with practical challenges.

(Am J Manag Care. 2009;15(11):842-845)

Efficiency measures are being used with the goal of improving the use of healthcare resources.

• There are important differences in perspective about the meaning of efficiency and the appropriateness of measures.
• The discipline of economics offers a practical framework for defining and measuring efficiency, especially among healthcare providers.
• In implementing and assessing efficiency measures, the perspective of the evaluator must be understood, and healthcare inputs and outputs must be identified.
• If there is substantial variation in quality in a particular context, a measure that does not account for quality could lead to unintended consequences.

Consider a physician whose medical group contracts with several managed care health plans. These plans have asked the group to produce standard measures of clinical quality, and the physician and her colleagues have performed well, even earning bonus payments. But now the physician learns that her practice is also to be evaluated for its “efficiency.” A health plan intends to assess physicians for efficiency in treating acute back pain to tier payments. In addition, the chief executive of the physician’s medical group has announced a program to monitor efficiency within the group. Our physician is left to wonder “Will these assessments accurately reflect meaningful aspects of performance?”

Situations like these are common, and the objective of this research brief is to help health professionals understand and evaluate the concept of efficiency and its measurement in practice. In particular, we describe the potential usefulness of efficiency measures for managerial decision making, emphasize the importance of perspective in defining and evaluating measures, and explore some practical considerations for measurement.

A definition of efficiency and some potential measures

We extensively reviewed efficiency concepts and existing healthcare measures based on searches of the following 3 sources: (1) the ME DLINE and EconLit databases for articles published from 1990 to 2008 using the keywords efficiency, inefficiency, productivity, and economic profiling; (2) seminal economic studies of efficiency identified in MEDLINE or EconLit or in economics reference materials; and (3) the “gray literature” on efficiency measures developed by private organizations. This literature review and our analytic framework are described in an evidence report for the Agency for Healthcare Research and Quality¹; a companion article systematically reviews the efficiency measures that we found.² This research did not require approval from an institutional review board, and none was sought.

In the discipline of economics, a situation is efficient if the only way to make someone better off is to make someone else worse off.³ Inefficiency means that there is a shared opportunity for improved well-being. For example, a medical practice that could provide the same office visits with less nursing labor is able to provide the same care at lower cost. True, some nurses would earn less. Yet as long as revenues from an office visit exceed costs, nurses whose income would otherwise decrease could be fully compensated (at least in theory), while still leaving the practice with lower costs than when nursing labor was used unnecessarily, or wasted.

From the study results, we present the following 4 types of measures that are commonly used and relevant to health professionals operating in managed care environments:

• Productivity. These measures are concerned with the level of services delivered by providers. For example, the Department of Defense measures the outpatient care delivered by primary care physicians in its clinics.⁴ Utilization, as the opposite of productivity, measures the level of input use per service (eg, the number of nursing hours per hospital discharge).
• Cost of service. These measures are concerned with the cost of providing specific health services. An example is the mean cost of a hospital’s discharges, controlling for patient mortality, case mix, and other factors.⁵
• Health outcomes such as mortality have been used to measure clinical quality, in addition to process and structure attributes.⁶ Case mix reflects the complexity and resource intensity of the care provided, which varies with the patient population treated by hospitals.
• Cost of episode. These measures are concerned with the cost of providing an episode of care. Proprietary algorithms from commercial vendors are used to identify and group the health services related to a particular episode.^2,7 The resulting measures are frequently used to assess physicians within a specialty.
• Cost of covered life. These measures are concerned with the cost of providing coverage of health services. An example is total spending on Medicare beneficiaries, controlling for their health status.⁷ This information has been used to identify generalist physicians with large shares of high-spending beneficiaries within their practices. Responsibility for total spending was attributed to these physicians, who were thus deemed inefficient.

These measures pertain to production by healthcare “firms,” with providers producing specific healthcare services and health plans producing coverage of health services. Each measure compares specific healthcare outputs (eg, Medicare coverage) with specific inputs (total Medicare spending [ie, costs]). Some of the inputs are financial in nature (eg, cost), while others are physical (eg, nursing hours).

The promise and peril of measurement

Firms are efficient if their outputs are as large as possible given their inputs or if inputs are as small as possible given outputs. These 2 conditions are equivalent, yet one may be more relevant in a particular situation. For example, a practice that has purchased a computed tomographic (CT) scanner may wish to increase its utilization, while a health plan in a market with ample physician supply may promote primary care over specialty care.

In the context of healthcare firms, true efficiency measures point to opportunities for improved production. Considering physical inputs, measured inefficiency means that a provider or plan should be able to provide healthcare with less labor, supplies, or equipment or to provide more care with the same inputs. Inefficiency based on financial inputs means that a firm can provide care at lower cost (or more care at the same cost) by choosing a different mix of physical inputs. For example, ibuprofen is less costly than a brand-name nonsteroidal anti-inflammatory drug and may be as effective for some patients in treating acute low back pain. In this case, generic substitution is cost-effective and thus efficient in production.

Yet some so-called efficiency measures do not truly measure efficiency. Because the quality of care is important to patients, high-quality care is generally distinct as an output from low-quality care. Only one of the measures in the preceding section (cost of service) incorporated clinical quality (as measured by hospital mortality), just as few of the measures that we found in our study dealt with quality. The AQA Alliance, a consortium of healthcare stakeholders, has defined efficiency of care as a measure of the cost of care associated with a specified level of quality of care.⁸ Therefore, efficiency measures differ from cost measures in accounting for quality, and cost measures are also efficiency measures only if quality is assumed to be uniform. There is strong evidence that quality varies widely across patients and providers.^9,10 This

evidence raises a concern that measurement could create incentives for low-cost but low-quality care or discrimination against high-risk patients and thus health disparities.¹¹ Yet in some settings, quality variation may be limited. For example, recent evidence has shown that cardiac mortality rates were statistically indistinguishable at 38 of 39 New York hospitals analyzed.¹² Furthermore, some health plans whose metrics ignore quality also consider standard quality metrics.^1,2,7 Methods for directly accounting for quality are complex and immature.¹³

The importance of perspective

Stakeholders in the healthcare system include providers, patients, and the health plans and employers who act as their intermediaries. These stakeholders have diverse perspectives about the goals of efficiency assessment and thus relevant inputs and outputs.

Consider a medical practice. This practice seeks to produce its services at the lowest possible cost. (The issue of quality is discussed herein.) One such output could be a diagnostic office visit using a CT scan, with physician hours as the efficiency measure’s input. (Scans would not be an input because the direct use of resources such as electricity in a scan is negligible, notwithstanding the scanner’s high initial cost.) The practice regards itself as efficient if scans are performed with as little labor as possible. Such an assessment could be short term if reorganization of the practice could decrease labor per visit but would take time to implement. For example, improved patient scheduling software could help a physician to diagnose more patients in a day; yet evaluating, implementing, and optimizing software packages are time consuming.

The perspective of a health plan differs. A plan seeks to produce health coverage at minimum cost. Its output could be covered lives, with payments to providers as the input. Even if a medical practice that uses CT scans deemed itself efficient, the practice would be inefficient for the plan in those cases in which another diagnostic is available at a lower reimbursement rate, perhaps from another practice.

As this example suggests, healthcare firms are hierarchically related. Providers deliver services, with some bundling them into episodes of care. Health plans bundle services into episodes of care and these into health coverage. Employers bundle health coverage and wages into compensation packages for their employees. One stakeholder’s output is another’s input. The Figure shows this hierarchy.

Given this interconnectedness, the efficiency of any firm is of interest not only to that firm but also to those above it in the hierarchy. If a medical practice is not efficient from its own perspective, a plan’s cost of covering lives may be higher. But a plan also desires a cost-effective mix of providers and services. A plan that measures its cost of diagnostic visits will judge a practice that provides unnecessary CT scans to be inefficient. Even if the medical practice’s costs of scans are minimized, the plan’s payments for diagnostic visits are not. Interests diverge when one healthcare firm is assessed by another, potentially resulting in disputes about measure validity.

Some practical considerations

Whatever the perspective, various factors affect the usefulness of measurement in practice. There are important tradeoffs in defining and implementing measures. No real-world measure is perfect. We would not suggest that assessments of efficiency are without value. Instead, we explore some of the practical considerations, leaving others to make their own judgments.

To begin with, the production processes of healthcare firms can be complex in their inputs. Hospitals, for example, use different types of labor (nursing, physician, and administrative), supplies (drugs, etc), equipment (beds, etc), and fixed capital (surgical suites, etc). Patient health status can also be viewed as an input. In measuring hospital efficiency, a ratio based on a single type of physical input (eg, nursing labor per discharge) does not deal with the interrelationship among inputs. A hospital with high nursing utilization may seem inefficient only because the hospital chose not to rely on technologies that economize on labor or because its patients are in poor health. Some efficiency measures incorporate multiple inputs, yet their analytic complexity and data requirements can be demanding.1,2,5 In any context, the balance between validity and practicality must be weighed.

Another consideration is the scope of output. Consider the treatment of an episode of back pain. A measure might define an episode narrowly, including only care received within a short window after an acute event. Alternatively, an episode might extend over months as pain recurs or progresses. A health plan could use the broader measure to identify additional opportunities to decrease the cost of coverage with a more cost-effective input mix. On the other hand, more providers are likely to participate in the episode (eg, a physical therapist and a primary care physician). Attributing responsibility to providers becomes more difficult, potentially biasing the assessment and exacerbating conflicts among plan and providers. However responsibility is attributed, a broader definition also results in fewer episodes. The mean cost per episode could be less reliable because costs have a random component that is especially important when few episodes are analyzed.

Conclusions

Consider again the physician who learns that a health plan and group executive intend to assess the efficiency of her practice. Such assessments can reveal opportunities for improvement by reducing waste of inputs or by switching to a more cost-effective input mix. Yet the physician may have a distinctive perspective on these assessments. Where her goals differ from those of the plan or the executive, she may be satisfied with her own performance in meeting her goals, while those assessing her deem her inefficient in meeting theirs. Even where goals are shared, the usefulness of measures can be undermined if, for example, important variation in clinical quality is poorly measured. Nevertheless, it is possible that measurement of efficiency in healthcare production will substantially improve the healthcare system’s overall performance.

Author Affiliations: From the RAND Corporation (JAR, PSH, MCW, PGS, EAM), Santa Monica, CA; the Department of Economics (JAR), Occidental College, Los Angeles, CA; RAND Europe (HD), Cambridge, UK; and the Greater Los Angeles Veterans Affairs Healthcare System (PGS), Los Angeles, CA.

Funding Source: This research was funded by the Agency for Healthcare Research and Quality, contract number 282-00-0005-21.

Author Disclosures: The authors (JAR, PSH, HD, MCW, PGS, EAM) report no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article.

Authorship Information: Concept and design (JAR, PSH, HD, PGS, EAM); acquisition of data (PSH, HD, MCW, PGS); analysis and interpretation of data (JAR, PSH, HD, MCW, PGS); drafting of the manuscript (JAR, PGS); critical revision of the manuscript for important intellectual content (JAR, PSH, MCW, PGS, EAM); provision of study materials on patients (PGS); obtaining funding (PGS , EAM); and supervision (PGS, EAM).

Address correspondence to: John A. Romley, PhD, RAND Corporation, 1776 Main St, PO Box 2138, Santa Monica, CA 90407-2138. E- mail: romley@rand.org.

1. Southern California Evidence-Based Practice Center—RAND Corporation, Santa Monica, CA. Final report: identifying, categorizing, and evaluating health care efficiency measures. AHRQ publication 08-0030. April 2008. http://www.ahrq.gov/qual/efficiency/index.html. Accessed September 11, 2009.

2. Hussey PS, de Vries H, Romley J, et al. A systematic review of health care efficiency measures. Health Serv Res. 2009;44(3):784-805.

3. Reiter S. Efficient allocation. In: Eatwell J, Milgate M, Newman P, eds. The New Palgrave: A Dictionary of Economics. New York, NY: Macmillan & Stockton; 1987.

4. Health Program Analysis and Evaluation Directorate, TRICARE Management Activity (TMA/HPA&E) in the Office of the Assistant Secretary of Defense (Health Affairs) (OASD/HA). Evaluation of the TRICARE program: FY 2007 report to Congress. 2007. http://www.tricare.mil/ocfo/_docsEvaluation_of_the_TRICARE_Program.pdf. Accessed September 11, 2009.

5. Carey K. Hospital cost efficiency and system membership. Inquiry. 2003;40(1):25-38.

6. Donabedian A. Explorations in Quality Assessment and Monitoring: The Definition of Quality and Approaches to Its Assessment. Ann Arbor, MI: Health Administration Press; 1980.

7. Government Accountability Office. Medicare: Focus on Physician Practice Patterns Can Lead to Greater Program Efficiency. Washington, DC: Government Accountability Office; 2007. GAO report GAO-07-307. http://www.gao.gov/cgi-bin/getrpt?GAO-07-307. Accessed July 21,

2008.

8. AQA Alliance. AQA principles of “efficiency” measures. April 2006. http://www.aqaalliance.org/files/PrinciplesofEfficiencyMeasurementApril2006.doc. Accessed September 11, 2009.

9. Fiscella K, Franks P, Gold MR, Clancy CM. Inequality in quality: addressing socioeconomic, racial, and ethnic disparities in health care. JAMA. 2000;283(19):2579-2584.

10. Schuster MA, McGlynn EA, Brook RH. How good is the quality of health care in the United States? Milbank Q. 1998;76(4):517-563.

11. Newhouse JP. Frontier estimation: how useful a tool for health economics? J Health Econ. 1994;13(3):317-322.

12. New York State Department of Health. Adult Cardiac Surgery in New York State: 2002-2004. http://www.health.state.ny.us/statistics/diseases/cardiovascular/. Accessed September 11, 2009.

13. Timbie JW, Normand SL. A comparison of methods for combining quality and efficiency performance measures: profiling the value of hospital care following acute myocardial infarction. Stat Med. 2008;27(9):1351-1370.

Download PDF: Efficiency and Its Measurement: What Practitioners Need to Know