Publication
Article
The American Journal of Managed Care
Economic Profiling of Physicians: Does Omission of Pharmacy Claims Bias Performance Measurement?
Author(s):
Objective: To investigate the degree to which the absence of prescriber identifying information and the absence of pharmacy claims might affect the validity of physicians' economic profiles.
Study Design: The study database consisted of 4 years of claims from a mixed-model health maintenance organization. Using the grouper of Episode Treatment Groups by Symmetry Health Data Systems, Inc, 2 episode databases were created, with and without pharmacy claims included. For each database, the responsibility for defined episodes was attributed to physicians within specialty (1) on the basis of combined professional and prescribing costs and (2) on the basis of professional costs alone.
Methods: Using the different databases and attribution rules, physicians were ranked within specialty on the economic profiling metric, and the various rankings were compared for consistency. Analyses were performed for cardiologists, family practitioners, general surgeons, and neurologists.
Results: The absence of prescriber identifying information appears to have only a small effect on physicians' economic profiles. The absence of pharmacy claims, on the other hand, may affect economic profile performance, but the effects differ by specialty and depend on pharmacy costs as a percentage of episode total costs for the specialty and the correlation of episode costs with and without pharmacy costs included.
Conclusions: Physicians' economic profile rankings are not greatly affected by the presence or absence of prescriber identifying information in pharmacy claims. If pharmacy claims are missing altogether, however, valid economic profiling remains feasible for some clinical specialties but not for others.
(Am J Manag Care. 2006;12:341-351)
Economic profiling of physicians is widespread, and its use by health plans is increasing every year. It is a key element of many pay-for-performance programs, and it provides the most commonly used criterion for partitioning health plan networks into performance tiers.1 By 2006, 3 of the 4 largest health insurers in the United States (United, Aetna, and CIGNA) were all preparing economic profiles on network physicians. In 2005, the Medicare Payment Advisory Commission initiated an evaluation of measures of resource use for inclusion in a Medicare pay-for-performance program.2
To create economic profiles, health plans process their paid claims databases through "episode grouper" software, such as the system of Episode Treatment Groups (ETGs) by Symmetry Health Data Systems, Inc, which aggregates individual members' claims diagnostically and chronologically into episodes of care.3 Examples of types of ETGs include septicemia, gout, acute bronchitis, viral meningitis, congestive heart failure, emphysema, and malignant neoplasm of the prostate. Episodes are defined for short-duration conditions, such as acute bronchitis, and for chronic conditions, such as congestive heart failure. The duration of a chronic episode typically is defined by default as 1 year, although grouper software allows that parameter to be changed by the user. At different times during the period covered by a claims database (eg, 1 year), a patient might experience multiple episodes of the same type (eg, viral skin infection episodes), and at any point in time the patient might be experiencing several different types of episodes (eg, acute bronchitis, congestive heart failure, closed fracture, or dislocation of a lower extremity).
Once episodes are defined, a total actual cost for each episode is calculated by summing the costs from included claims, and an expected cost is determined for each episode, usually as the mean costs of all episodes of the same type (eg, acute sinusitis and type 2 diabetes mellitus). It is common practice among health plans to define an episode's expected cost as being equal to the mean cost of all other episodes of the same type (eg, the same ETG). We have shown elsewhere that risk adjustment, using patient demographics and available person-level risk adjusters (eg, ETGs from Symmetry Health Data Systems, Inc), does not improve the accuracy of ETG mean estimates of episode expected costs.4
cost efficiency
efficiency
efficiency
cost efficiency
efficiency
The costs of defined episodes are attributed to responsible physicians, and each physician's score is calculated as a function of actual costs and expected costs for attributed episodes. In this article, we refer to this score as , but we note that this terminology has been a highly contentious issue. For several years, many health plans have been referring to the measure as , but health economists object to this because they have long used to refer to the cost of resources used in achieving a given outcome or benefit to the patient. In September 2005, a meeting convened by the Ambulatory Quality Alliance and the National Committee for Quality Assurance determined that was an acceptable term for the relative resource use measure described herein, and because it does not control for quality or patient benefit, it must be distinguished from , which controls for outcomes.
For the episode definition and cost estimation process, health plans typically include all categories of claims, including outpatient facility, inpatient facility, professional, and pharmaceutical claims. However, the same is not true for episode attribution, the process by which health plans attribute responsibility for episode costs to individual physicians. Although it would be desirable for episode attribution decisions to consider physicians' influences on total costs, including (for example) the costs of inpatient services, outpatient laboratory use and imaging, therapy sessions, professional fees, and pharmaceutical costs, this does not occur because of limitations of claims databases. Although physician identifiers are always included in professional claims and pharmacy claims usually (but not always) identify prescribing physicians (eg, by Drug Enforcement Administration physician registration numbers), claims for outpatient services and inpatient services normally do not include any fields with which service costs can be associated with ordering physicians. Consequently, episode attribution rules are limited to functions of professional fees and prescribing costs. When the identity of the prescribing physician is unavailable in claims records, only professional fees can be considered. Organizations (eg, the Medicare Payment Advisory Commission) that seek to use Medicare claims for economic profiling of physicians face an additional data limitation. Medicare databases include only Part A (inpatient hospital services) and Part B (outpatient and professional services) claims. It is unclear when claims associated with Medicare's new drug benefit, Part D, will be available, nor is it clear what proportion of the Medicare population will be covered by Part D. No pharmacy claims are available for episode definition and cost calculation, and it is impossible to consider prescribing costs when attributing responsibility for episode costs to physicians.
Among organizations engaged in economic profiling, it is standard practice to use pharmacy claims in the episode definition and cost calculation process and as the source of prescribing costs for episode attribution. However, as already noted, pharmaceutical data are sometimes unavailable. The objective of this study was to investigate whether physician cost-efficiency rankings developed without pharmaceutical information differ from those developed when such data are available. Specifically, we address the following questions: (1) In economic profiling of physicians, does the absence of pharmacy claims affect the accuracy of physician cost-efficiency rankings? (2) In economic profiling of physicians, does the absence of prescriber identifying information in pharmacy claims affect the accuracy of physician cost-efficiency rankings?
METHODS
Data for the study are from a university-owned mixed-model (group and independent practice association) health maintenance organization in southeast Michigan. The database consisted of all professional, outpatient, inpatient, and pharmacy claims for members who were enrolled for the full 12 months of 1999, 2000, 2001, or 2002. Episode grouping was accomplished using the ETGs software by Symmetry Health Data Systems, Inc, and 2 different episode databases were created. With pharmaceutical data included in the episode grouping process, the 4 years of claims produced 674 229 completed episodes, which were divided among 497 combinations of ETGs and sub-ETGs (hereafter referred to as ETGs) and were associated with 104 722 different health maintenance organization members. We refer to this as the with-pharmacy episodes database. With pharmaceutical data excluded, claims were mapped into 650 848 completed episodes, divided among 497 ETGs and associated with 104 578 different health maintenance organization members. We refer to this as the without-pharmacy episodes database. In the study described herein, analyses were restricted to the set of 650 848 episodes that were in both databases.
Current Procedural Terminology
To remove variability associated with service pricing differences among providers, we developed a set of service- specific standardized costs. Standardized costs for professional and outpatient services were developed as the arithmetic means of the actual costs of claims associated with each procedure code for professional claims; for outpatient claims, the Healthcare Common Procedural Coding System code from the Centers for Medicare & Medicaid Services or the local revenue code was used. For pharmacy claims, costs were standardized using the National Drug Code Directory (http://www.fda.gov/cder/ndc/database/default.htm) and the amount dispensed. Standardized hospital inpatient costs were defined on the basis of Medicare diagnosis related group. Details of the cost standardization procedures were previously published.5 Potentially distorting effects of very high cost or very low cost episodes on estimates of physicians' mean costs were reduced by ETG-level Winsorization of episode costs. Separately for each of the 2 episode databases, episode costs were Winsorized to the 2nd percentile for low outliers within each ETG and to the 98th percentile for high outliers within each ETG.6
We examined 2 different pairs of episode attribution rules (ie, 4 rules altogether). First, the responsibility for episode costs was attributed on the basis of combined professional and prescribing costs (1) to physicians who accounted for at least 50% of combined costs and (2) to physicians who accounted for at least 30% of combined costs.5 The 50% and 30% attribution rules are among those commonly used by health plans. These attribution rules have been examined previously.5,7 Note that with the 30% attribution rule, some episodes can be assigned to 2 or more physicians.
baseline
no prescribing costs
no pharmacy
claims
Second, episodes were attributed on the basis of professional costs alone (1) to physicians who accounted for at least 50% of episode professional costs and (2) to physicians who accounted for at least 30% of episode professional costs. Two cost percentage levels (50% and 30%) were included to investigate whether findings would vary based on episode attribution cost percentage. Using these rules and the 2 episode databases already described, we created 6 different analysis sets for each of 4 physician specialties: (1) , using the with-pharmacy episodes database, including (a) episodes attributed to physicians accounting for at least 50% of combined prescribing and professional fees and (b) episodes attributed to physicians accounting for at least 30% of combined prescribing and professional fees; (2) , using the with-pharmacy episodes database, including (a) episodes attributed to physicians accounting for at least 50% of professional fees and (b) episodes attributed to physicians accounting for at least 30% of professional fees; and (3) , using the without-pharmacy episodes database, including (a) episodes attributed to physicians accounting for at least 50% of professional fees and (b) episodes attributed to physicians accounting for at least 30% of professional fees.
standardized cost difference
After responsibility for episode costs was assigned, individual physicians (not groups) were ranked within specialty from the most efficient to the least efficient on the basis of the , which is the standardized difference between the mean actual cost and the mean expected cost for the sample of episodes managed by the physician. Using Zk to represent the standardized cost difference for the kth physician,
N
where ˆyk is the mean actual cost and ˆyk is the mean expected cost associated with the kth physician's set of episodes, s is the standard deviation of episode expected costs, and k is the number of episodes assigned to the kth physician. As previously shown,8 this measure is less likely than measures that do not adjust for sample size to incorrectly identify smaller-sized panels as being high or low outliers.
The levels of agreement between cost-efficiency rankings and rankings derived from no prescribing costs and no pharmacy claims data sets were measured using the weighted k statistic described by Landis and Koch,9 who indicate that the appropriate interpretation of the weighted k values, which vary from 0.0 to 1.0, would be 0% to 20% (slight agreement), 21% to 40% (fair agreement), 41% to 60% (moderate agreement), and at least 61% (substantial agreement).
For each of the 4 data years, health plan physicians in each specialty were partitioned into thirds on the basis of standardized cost differences (with cost efficient ranked as 1, average cost ranked as 2, and cost inefficient ranked as 3) to correspond to health plans' partitioning of network physicians into cost-efficiency performance tiers. Weighed k analyses were also performed with specialty physicians partitioned into 4 subgroups and 5 subgroups. Patterns of results were similar to those based on 3 subgroups and are not presented herein, but they are available from the author. Then, the weighted k value was used to assess the level of agreement between the following pairs of tier rankings: (1) baseline and no prescribing costs and (2) baseline and no pharmacy claims. Based on previous findings, we profiled and ranked cardiologists having at least 20 attributed episodes, family practitioners having at least 120 attributed episodes, and general surgeons and neurologists having at least 25 attributed episodes.10
RESULTS
Across all 650 848 episodes considered in our analyses, pharmaceutical costs represented 22% of episode total costs. The percentage varied greatly across ETGs, however. In Table 1, data are summarized for each of the 22 major practice categories (MPCs) into which Symmetry Health Data Systems, Inc has organized its ETGs, showing the number of ETGs that comprise the MPC and the percentage of episode total costs represented by pharmacy claims. For example, in MPC 1 (infectious diseases), the smallest percentage was 0.5% and occurred with the 2984 episodes in ETG 0007 (infectious disease signs and symptoms). The maximum percentage was 80.6%, for the 101 episodes in ETG 0001 (AIDS). The weighted mean pharmacy cost percentage across the 6 ETGs in MPC 1 was 18.9%. Nine other MPCs, including MPC 2 (endocrinology) with 26 ETGs and MPC 4 (psychiatry) with 13 ETGs, had larger weighted mean cost differences (47.8% and 53.5%, respectively) than MPC 1. Of the 24 208 episodes in MPC 2, almost one third (7297 episodes) were in ETG 0047 (hyperlipidemia), for which pharmacy costs represented 72.2% of episode total costs. For an additional 3891 type 2 diabetes mellitus episodes in this MPC, pharmacy costs averaged 61.6% of episode total costs. However, for 4 ETGs in MPC 2, pharmacy costs represented less than 1% of episode total costs.
Descriptive statistics for episode expected costs and for physicians' standardized cost difference scores are given in Table 2 for each specialty (2002 data, using the 30% attribution rule). The levels of agreement between baseline cost-efficiency rankings and rankings derived from no prescribing costs and no pharmacy claims data sets are given in Table 3 for each specialty in each of the 4 years. Because pharmacy costs represent high percentages of episode total costs (at least in some MPCs), the levels of agreement in Table 3 are surprising. Regardless of the episode attribution rule, in all 4 years the agreement between baseline and no prescribing costs rankings is perfect for general surgery and is perfect or "substantial," as interpreted by Landis and Koch,9 for the other 3 specialties. The agreement between baseline and no pharmacy claims rankings is perfect or substantial in all 4 years for general surgeons and cardiologists and in 3 of the 4 years for neurologists. However, for family practitioners, the agreement between baseline and no pharmacy claims rankings is only moderate (except in 1999, using the 30% attribution rule). General patterns of weighted k values in Table 3 indicate that the levels of agreement for each of the paired rankings are strongest for general surgeons, weakest for family practitioners, and stronger for cardiologists than for neurologists. There are no systematic differences in the levels of agreement between the 2 episode attribution rules for any of the specialties.
r
Because pharmacy costs represent 22% of overall episode total costs, why does exclusion of these costs from episode attribution and episode definition and costs have a small effect on physicians' cost-efficiency rankings? Baseline rankings could be similar to no prescribing costs rankings and to no pharmacy claims rankings if physicians' economic profiles were based on ETGs for which pharmaceutical costs were small. Even if profiles included ETGs with significant pharmacy costs, baseline rankings could still be similar to no prescribing costs rankings if physicians' episode-level professional fees were highly correlated with combined professional and prescribing costs. For no pharmacy claims rankings to be similar to baseline rankings when profiles include ETGs in which pharmacy costs are significant, the following 2 conditions would have to be met: (1) episode attributions would have to be similar and (2) episode costs with pharmacy costs included would have to be correlated with episode costs with pharmacy costs excluded. In Tables 4 through 7, for each of the 4 specialties considered in this study, we list the ETGs on which specialty profiles are based, and, for each of these ETGs, we show the percentage of the specialty's episodes included in that ETG, the mean percentage of episode cost represented by pharmacy costs, the level of correlation (Pearson product moment correlation) between episode costs with and without pharmacy costs included, and the percentage of episodes for which physicians' episode attributions change when prescribing costs are omitted from the attribution calculations. Data in Tables 4 through 7 reflect episode attributions made using the 30% rule. The ETGs identified were the same and the data patterns were similar when episodes were attributed using the 50% rule. Tables related to 50% attribution rules are available from the author.
r
r
Cardiologists' profiles (Table 4) were composed of episodes from 10 ETGs for which pharmacy costs average 14.0% of total costs. To assess how exclusion of pharmacy claims might affect cost-efficiency scores, we look at the columns in Table 4 that are related to the mean episode total costs. In 9 of the ETGs in cardiologists' profiles, the correlations between episode costs with and without pharmacy costs were at least 0.90, indicating that even if cost differences were large the effects on the relative cost-efficiency scores would be minimal. For ETG 0047, data would seem to suggest that the effects of omitting pharmacy claims might be significant, because the correlation was only 0.32 and pharmacy costs represented 72.2% of episode total costs. However, this ETG accounted for only 1.7% of cardiologists' episodes. Therefore, similar to the other 9 ETGs, the effects from the omission of pharmacy claims in hyperlipidemia cases would be small. For cardiologists, if pharmacy claims were available but did not identify the prescribing physician, 7.1% of episodes in ETG 0264 would be attributed to different physicians if the attributions were based on the 30% rule. For other ETGs, the percentages of episodes for which attributions change are smaller (eg, 1.7% in ETG 0274 and 1.2% in ETGs 0265 and 0281). Although attribution changes occur in 7 of the 10 ETGs comprising cardiologists' profiles, the effects of these changes are minimal. Although 7.1% of episodes in ETG 0264 are affected by attribution changes, this ETG accounts for only 0.6% of episodes used to construct cardiologists' profiles, so attribution changes represent only 0.04% of cardiologists' episodes. Across all 10 ETGs, using the 30% rule, 1.2% of cardiologists' episodes are attributed to different physicians if the prescribing physician is not identified in pharmacy claims. When episodes are attributed to physicians using the 50% rule, only 0.5% of episodes are affected when the prescriber identifying information is unavailable.
r
As Table 5 indicates, family practitioners' profiles are based on 13 ETGs, with ETG 0794 (routine examination) accounting for 24.0% of all family practice episodes. Pharmacy costs average 28.3% of episode total costs across the 13 family practice ETGs and exceed 50% of episode total costs in 5 ETGs and exceed 40% of episode total costs in 7 ETGs. Pharmacy costs represent higher percentages of episode total costs in family practice ETGs than in cardiology ETGs (Table 4), and the correlations between episode costs with and without pharmacy costs are lower in family practice ETGs. Nevertheless, the correlations exceed 0.70 in 8 of the 13 family practice ETGs and exceed 0.50 in all except ETG 0047, which accounts for only 1.9% of family practice episodes.
Using the 30% rule, when prescriber identifying information is omitted from attribution calculations, family practitioner attributions change for 4.8% of episodes in ETG 0332 and for 4.0% of episodes in ETG 0333 (Table 5). Using the 50% rule, more than 5% of attributions are changed in ETGs 0332, 0333, and 0335. However, the overall percentage of family practitioner episodes affected by attribution changes is small for both attribution rules (1.5% using the 30% rule and 2.3% using the 50% rule). As shown by the weighted k data in Table 3, these attribution changes have only a small effect on family practitioner rankings.
r
In our database, general surgeons' profiles were based on episodes from 9 ETGs (Table 6), and in these ETGs pharmacy costs averaged only 5.7% of episode total costs. Pharmacy costs were minor in all of the ETGs except ETG 0678 (minor inflammation of skin and subcutaneous tissue), for which they represented 42.3% of episode total costs. However, in this ETG the correlation between episode total costs with and without pharmacy costs was 0.96, so general surgeon cost-efficiency scores were unaffected by the omission of pharmacy claims. Not surprisingly, the omission of prescriber identifying information in pharmacy claims has no effect on general surgeon episode attributions using the 30% rule or the 50% rule.
r
r
r
The ETGs comprising neurologists' profiles are listed in Table 7. Pharmacy costs averaged 20.1% of neurologists' episode total costs in these ETGs, and ETG 0150 (inflammation of the central nervous system, without surgery) appears to be the most problematic for consistency of neurologists' profiles with and without pharmacy costs included. For ETG 0150, pharmacy costs represented 74.6% of episode total costs, and episode costs without pharmacy costs demonstrated an correlation of only 0.35 with episode costs with pharmacy costs included. However, because this ETG accounted for only 3.6% of neurologists' episodes, its effect on neurologists' profiles would be small. Two other potentially problematic ETGs are 0152 (epilepsy, without surgery) and 0168 (common migraine headache). In each of these ETGs, pharmacy costs represented more than 60% of episode total costs. The correlation between episode costs with and without pharmacy costs included for ETG 0152 was 0.92, so omission of pharmacy costs in this ETG would have little effect on physicians' cost-efficiency scores. However, ETG 0168 included 11.1% of neurologists' episodes, and the correlation was low at 0.64. The absence of prescriber identifying information in pharmacy claims also had a small effect on neurologists' profiling results. Using the 30% attribution rule for neurologists, the physicians identified as being responsible for 7.7% of episodes in ETG 0167 and for 4.6% in ETG 0169 would change if the prescriber identifying information were unavailable. However, these ETGs together represent less than 6% of neurologists' episodes. When the prescriber identifying information is omitted from the attribution calculations, across all 14 ETGs the physician attribution is changed for only 0.5% of neurologists' episodes when the 30% rule or the 50% rule is used.
DISCUSSION
In economic profiling of physicians, the importance of prescriber identifying information in pharmacy claims is determined by the fraction of episode attribution decisions that would be affected by the use of that information. Among the 4 specialties examined in this study, the percentages of episode attribution changes range from 0.0% for general surgery to 1.5% for family practice when episodes are attributed to physicians based on the 30% rule. When episode attribution is based on the 50% rule, attribution changes occur for fewer than 0.5% of cardiology, general surgery, and neurology episodes, while 2.3% of family practice episodes are affected. The results presented herein do not identify a threshold percentage at which episode attribution changes might begin to affect physicians' cost-efficiency rankings. However, because the levels of agreement between pairs for family practitioner rankings, while still substantial, are lower than those for the other 3 specialties, it appears that cost-efficiency rankings might be negatively affected by changes in physician attribution for as few as 1% of episodes.
The absence of pharmacy claims is a greater threat to the validity of physicians' economic profiles than the absence of prescriber identifying information. Not only can episode attribution decisions be affected by the absence of pharmacy claims, but episode costs can be understated as well. Whether either of these negative effects occurs depends on the following 2 conditions, both of which must be present: (1) across episode types (ETGs), the weighted mean differences in episode total costs when pharmacy costs are and are not included must be substantial and (2) across types of episodes affected, episode costs that include pharmacy costs must not be highly correlated with episode costs that do not include pharmacy costs.
r
But what does "substantial" mean in the first of these conditions, and what does "highly correlated" mean in the second? Pharmacy costs as a percentage of episode total costs ranged from 2.2% to 72.2% (mean, 14.0%) for cardiology, from 0.0% to 72.2% (mean, 28.3%) for family practice, from 0.3% to 42.3% (mean, 5.7%) for general surgery, and from 0.6% to 74.6% (mean, 20.1%) for neurology. The weighted mean correlations across all ETGs for the 4 specialties were 0.975 for cardiology, 0.796 for family practice, 0.995 for general surgery, and 0.903 for neurology. The patterns of the weighted k statistics in Table 3 for the 4 specialties correspond perfectly to the weighted mean of the pharmacy cost percentages and the correlations. The specialty with the lowest mean pharmacy cost percentage and the highest correlation rate is general surgery, which consistently has the highest weighted k statistics, and the specialty with the highest mean pharmacy cost percentage and the lowest correlation rate is family practice, which consistently has the lowest weighted k statistics. Weighted k statistics for cardiology and for neurology also correspond to their relative mean pharmacy costs and correlation rates. Because the patterns are similar, we are not able to say that the mean pharmacy costs or the mean correlations are more influential in determining whether cost-efficiency rankings will be affected by the absence of pharmacy claims. Both must be considered important.
Are these findings generalizable to specialties other than those examined in this study? We believe that the findings should not be generalized to other specialties. Table 1 indicates that pharmacy costs accounted for large proportions of episode total costs in some ETGs, and that some specialties (eg, endocrinology and psychiatry) may treat larger volumes of these high pharmacy cost episodes than other specialties. However, Table 1 indicates that pharmacy costs average 28.0% of episode total costs in cardiology, while Table 4 indicates that, among episodes attributed to cardiologists in this study, pharmacy costs represent only 14.0% of episode total costs. For any specialty, including endocrinology and psychiatry, it is impossible to determine solely on the basis of data such as those in Table 1 whether the omission of pharmacy claims or prescriber identifying information would affect cost-efficiency rankings. As noted herein, such effects depend on pharmacy costs as a percentage of episode total costs and the correlation of episode costs with and without pharmacy costs included. Therefore, determining whether the omission of pharmacy costs would substantially affect the relative cost-efficiency rankings within a particular clinical specialty will require analyses such as those presented herein.
Are the findings generalizable to other settings? The analyses described herein were performed using claims from a moderately small university-affiliated health plan. However, because we have no reason to believe that the clinical management of patients in this health plan is different from that elsewhere, we would expect to see similar findings when comparable analyses are performed using other health plan data sets.
From the Institute for Health Policy, Edmund S. Muskie School of Public Service, University of Southern Maine, Portland.
This study was supported by grant 047789 from the Robert Wood Johnson Foundation Changes in Health Care Financing and Organization program.
Address correspondence to: J. William Thomas, PhD, Institute for Health Policy, Edmund S. Muskie School of Public Service, University of Southern Maine, 509 Forest Ave, Ste 200, Portland, ME 04101-9300. E-mail: jwthomas@usm.maine.edu.
