Use of Critical Pathways to Improve Efficiency: A Cautionary Tale

Background: Critical pathways are healthcare management plans that specify patient goals and the sequence and timing of actions necessary to achieve these goals with optimal efficiency. More than 80% of hospitals in the United States use critical pathways for at least some of their patients. Unfortunately, critical pathway effectiveness in improving clinical efficiency is unclear.

Objectives: To assess whether critical pathways have been successful in reducing patient length of stay (LOS) and resource utilization in our tertiary-care academic medical center.

Study Design: A before-and-after observational study using multivariate linear regression analyses.

Methods: We identified all critical pathways initiated in our medical center between 1993 and 1996 in which at least 50 adult patients would be evaluated in the year preceding and succeedingpathway implementation; 13 pathways satisfied these inclusion criteria. Using a before-and-after design, multivariate linear regression was used to evaluate each pathway’s effect on average monthly LOS and resource utilization after adjusting for case mix and secular trends.

Results: Three of the 13 pathways were associated with a statistically significant immediate decrease in inpatient LOS: acute myocardial infarction (20.7% decrease; P = .001), cesarean section (14.6% decrease; P = .03), and kidney transplantation (24.5% decrease; P = .003). Only 1 pathway, percutaneous transluminal coronary angioplasty (PTCA), produced a statistically significant decrease in LOS slope (a decrease of 5.2% per month; P = .001). Two pathways were accompanied by a statistically significant immediate reduction in ancillary resource utilization: kidney transplantation (26.4% decrease; P = .001) and community-acquired pneumonia (21.8% decrease; P = .002). Only the PTCA pathway produced a statistically significant decrease in resource utilization slope during the 12- month follow-up period (a decrease of 8.4% per month; P < .001).

Conclusions: Although some pathways did reduce LOS or resource utilization or both, most pathways reduced neither. Because substantial resources must be expended on pathway development, implementation, and maintenance, future efforts should be placed on further evaluating the effectiveness of critical pathways and understanding the reasons behind their success or failure before additional resources are consumed for this management strategy.

(Am J Manag Care 2003;9:758-765)

More than 80% of hospitals in the United States use critical pathways as care strategies for at least some patients.¹ Critical pathways are management plans that specify goals for patients and the corresponding ideal sequence and timing of staff actions to achieve those goals with optimal efficiency.² Implementation of critical pathways, developed for myriad diseases and admitting diagnoses, usually requires a substantial time commitment by a multidisciplinary development team.² Such endeavors are largely regarded as worthwhile by healthcare managers who have "embraced critical pathways as a method to reduce variation in care, decrease resource utilization, and potentially improve healthcare quality."³

The major goal of a critical pathway is to maximize clinical efficiency of care by reducing patient length of stay (LOS) and resource utilization. By closely coordinating care by both nursing and medical staff from the time a patient is admitted until hospital discharge, pathways are theoretically well suited to minimize waste and inefficient care. Unfortunately, evaluation of the effectiveness of critical pathways has been inadequate.^4,5 The University of Michigan Medical Center, a tertiary-care academic medical center, has been using critical pathways for more than a decade. To evaluate the effectiveness of critical pathways at that center, we assessed whether these clinical management plans have been successful in reducing patient LOS and resource utilization.

METHODS

Setting

The University of Michigan Medical Center, an 872-bed hospital with a level-1 trauma center, is a primary referral hospital in Southeastern Michigan. Critical pathways have been developed and widely implemented for various disorders and admitting diagnoses at the University of Michigan Medical Center since 1991.

Pathway Development and Implementation

Pathway development at the University of Michigan Medical Center entailed a multidisciplinary process, usually led by either a physician or nurse with special expertise or interest in the disease process focused on by the pathway. For each pathway, 4 steps usually occurred. (1) A series of meetings were held, often during the course of several months, in which evidence from both published literature and expert opinion were used to develop a final version of the pathway for use in patients with the diagnosis of interest. (2) The pathway was then disseminated to the nursing and medical staff providing clinical care to patients with the diagnosis under consideration. (3) In addition to educating the nursing and medical staff about the specifics of the pathway, copies of the pathway were available in each clinical area for future reference. (4) A person was appointed as pathway "leader" so that modifications could be made in a coordinated manner as needed in response to new information. For a few pathways, the clinical staff occasionally used the pathway as an orientation tool for new house officers or nurses. In their final versions pathways were generally formatted as Gantt charts, thereby outlining the suggested processes of care using a time-task matrix.² Of note, no type of incentive-financial or otherwise-was offered to encourage pathway use.

Data

We identified all critical pathways initiated in our medical center between 1993 and 1996 in which at least 50 adult patients were evaluated in the year preceding and succeeding pathway implementation. Thirteen pathways satisfied inclusion criteria. The diagnoses (or procedures) covered by these pathways were the following: acute myocardial infarction, acute pancreatitis, asthma, breast surgery, cesarean section (C-section), community-acquired pneumonia (CAP), hip arthroplasty, kidney transplantation, knee arthroplasty, liver transplantation, lung lobectomy, percutaneous transluminal coronary angioplasty (PTCA), and radical nephrectomy. We used ICD-9 procedure or diagnosis codes to identify patients who would be considered for management by each pathway. As we were primarily interested in the effectiveness of pathways as a management method, we included patients whether or not they were actually managed on the particular pathway. (If this were a clinical intervention, our approach could be termed an intention-to-treat analysis.)

We excluded patients transferred into our hospital for continued inpatient care, as most such patients likely would not have been managed by a pathway. We also excluded LOS outliers, defined as those patients with a LOS more than 3 standard deviations above the geometric mean for their corresponding diagnosis-related group (DRG). We excluded outliers for 2 reasons. First, we believe that such patients are fundamentally different from the population we wished to generalize about and are not part of the target population for the application of critical pathways. Specifically, outlier patients tend to have very severe or complex illness (or occasionally social issues) requiring a completely different management approach than that used in most routine cases. The pathways concept itself is not suited for addressing extreme situations; indeed, the strategy is based on clinical experience with routine cases. Second, even if we thought extreme outliers in this study were part of the same population as the other cases, there are statistical reasons to assess them differently. In situations in which extreme skew exists, a variety of techniques are commonly used. We believe that outright deletion did not provide a substantially different result from the down-weighting that would have occurred with the other statistical techniques that we would have had to use if we had left the DRG outliers in the sample (e.g., median or quartile regression).

Analysis

Using a before-and-after design, we evaluated each pathway's effect on patient LOS and resource utilization (as determined by University of Michigan-derived standardized relative value units⁶). The unit of observation was the average monthly LOS or resource utilization, adjusted for case mix. We constructed separate linear regression models for each pathway using each outcome (LOS and resource utilization) and included patient data 1 year before and 1 year after pathway implementation.

To account for secular trends, 5% of patients with other diagnoses but in the same general discipline as pathway patients (e.g., surgery, cardiology) were randomly selected to serve as controls. In this manner, we were able to assess both secular trends occurring specifically among pathway patients as well as overall ecological effects occurring in the hospital. We used regression models to evaluate whether any change in LOS or resource utilization could be detected after pathway implementation. Specifically, we looked for both a 1-time immediate change and a change in the slope of utilization over time occurring after the pathway was implemented.

Any changes observed were compared with changes in the non-pathway diagnoses as a control group, and only those changes above and beyond those seen in non-pathway diagnoses were considered as potentially attributable to the pathway. Thus our regression models included variables identifying whether an observation was pathway or non-pathway (PATH), the month of the observation (MONTH), and whether the observation was from before or after pathway implementation (PERIOD). The interaction between pathway and period tested the hypothesis that a 1-time decrease (or increase) in utilization occurred after implementation. The 3-way interaction between PATH, MONTH, and PERIOD tested the hypothesis that there was a change in the slope of utilization over time after pathway implementation that was above and beyond any background ecologic changes occurring in non-pathway diagnoses. The full model included all possible 2-way interactions and the 3-way interaction. However, if the 3-way interaction was not significant it was dropped from the model. Thereafter, any non-significant 2-way interactions were dropped. The 2-way interaction testing the hypothesis of the 1-time change in utilization was retained in the models regardless of its statistical significance.

To simplify interpretation of the results of the regression model, Figures 1 and 2 represent the final model developed using the above strategy for each diagnosis. We used a natural log transformation so that the distribution of the dependent variables would more closely approximate the normal distribution. Although dramatic changes in patient case mix are atypical within a 24-month period at our facility, we nevertheless adjusted for such changes using the DRG case-mix index. Our analyses therefore account for both secular trends and case-mix changes within groups.

P

A pathway was considered successful if a statistically significant decrease, immediate or gradual, was noted in 1 of the outcome measures. Statistical significance for all analyses was based on a 2-sided < .05. Analyses were conducted with SAS version 6.08 (SAS Institute Inc, Cary, NC). RESULTS Overall, 5 of the 13 pathways (38%) were associated with a statistically significant decrease in LOS, resource utilization, or both, after controlling for secular trends and case-mix changes (Table). Evaluating pathway effects stratified by outcome measure, however, provided additional important information.

Pathway Effects on Length of Stay

P

P

P

P

No statistically significant increase in LOS was noted after pathway implementation. Three of the 13 pathways were associated with a statistically significant immediate decrease in LOS: acute myocardial infarction (20.7% decrease; = .001; Figure 1a), C-section (14.6% decrease; = .03; Figure 1e), and kidney transplantation (24.5% decrease; = .003; Figure 1h). Only 1 pathway, PTCA (Figure 1l), produced a statistically significant decrease in LOS slope (5.2% decrease per month; = .001). Of note, the PTCA pathway was not associated with an immediate decrease in LOS. For most of the 13 pathways (69%), no significant decrease in LOS was noted either immediately or during the course of the 12-month follow-up period.

Pathway Effects on Resource Utilization

P

P

P

P

Two pathways were accompanied by a statistically significant immediate reduction in ancillary resource utilization: kidney transplantation (26.4% decrease; = .001; Figure 2h) and CAP (21.8% decrease; = .002; Figure 2f). Again, only the PTCA pathway (Figure 2l) produced a statistically significant decrease in resource utilization slope during the 12-month follow-up period (8.4% decrease per month; < .001). In the asthma pathway (Figure 2c), a statistically significant increase was noted in resource utilization immediately after pathway implementation (17.7% increase; = .04). Similar to the findings with LOS, most of the pathways (77%) did not significantly decrease resource utilization either immediately or during the 12-month follow-up period.

P

P

Importantly, if we had not accounted for secular trends, the pathway for C-section would have appeared to reduce resource utilization by 45.2% (< .001). However, after including ecological changes within our regression models, the effect of the pathway was no longer statistically significant (18% reduction in resource utilization; = .13).

Graphical Display of Pathway Effects

Figures 1 and 2 depict how each of the 13 pathways affects LOS and resource utilization, respectively. In general, if a pathway was associated with a significant decrease in LOS it was also associated with a significant decrease in resource utilization. As can be seen, however, most of the time pathway implementation was statistically unrelated to either LOS or resource utilization. Figures 1 and 2 do reveal a marked heterogeneity of pathway effectiveness across the different diagnoses and procedure. Describing a few of the pathway results in detail clarifies the varied effects of the pathways.

For the LOS evaluation, the pathways for acute myocardial infarction (Figure 1a), C-section (Figure 1e), and kidney transplantation (Figure 1h), were associated with an immediate decrease in LOS following the intervention but no gradual effect (i.e., the intervention changed the intercept but not the slope). The PTCA pathway (Figure 1l) reveals no immediate effect of the intervention but a substantial gradual effect (i.e., the intervention changed the slope but not the intercept). This example highlights the need to allow for gradual effects in the evaluation process and for evaluating the effect of a pathway several months after implementation. The plots in Figure 2 evaluating resource utilization are roughly similar to the plots for the corresponding pathways in Figure 1.

DISCUSSION

We evaluated the effectiveness of critical pathways to improve clinical efficiency in 13 clinical areas in our institution. Our study revealed the following important finding: although some pathways did reduce LOS, resource utilization, or both, most pathways reduced neither. Our findings add to the growing, but still limited, number of studies that have evaluated critical pathways while also considering the secular trends that have been reducing LOS and resource utilization independent of pathway use. Whereas several earlier uncontrolled studies7 that did not consider secular trends have found critical pathways to be effective in increasing efficiency, 2 recent studies have not found critical pathways to be effective once ecological trends and other quality improvement strategies are considered.^8,9 Given the decrease in LOS observed during the past decade in US hospitals, it is important that investigators evaluating pathway effectiveness adjust for this downward trend. Indeed, when we estimated national ecological trends using the national Healthcare Cost and Utilization Project database ( http://www.hcup.ahrq.gov/HCUPNet.asp ), which was funded by the Agency for Healthcare Research and Quality, we found substantial similarity between our data and the secular trends found in the population as a whole.

Pearson and colleagues⁸ performed a before-and-after evaluation of 5 surgical pathways implemented between 1993 and 1997 in their tertiary-care hospital. Although each pathway significantly decreased postoperative LOS (from 3% to 21% depending on the pathway), similar reductions in LOS were observed for 3 of the procedures at surrounding hospitals not using pathways. ⁸ Based on their findings, Pearson and colleagues⁸ "dropped" plans to develop pathways for additional conditions. In their study of Medicare beneficiaries at 32 Connecticut hospitals, Holmboe and colleagues⁹ evaluated the effectiveness of a critical pathway for patients hospitalized with acute myocardial infarction. Compared with patients in hospitals without an acute myocardial infarction critical pathway but using other quality improvement strategies (e.g., physician opinion leaders and peer review), patients hospitalized at the 10 pathway hospitals had similar lengths of stay.⁹ Additionally, patients in hospitals using critical pathways did not have increased use of proven medical therapies (e.g., aspirin, beta-blockers, reperfusion) or an improved survival compared with patients hospitalized at medical centers not using an acute myocardial infarction pathway.⁹ However, a recent study by Marrie and colleagues¹⁰ from Canada focusing on a critical pathway for CAP found that pathway to be substantially more efficient than standard care. Specifically, this 19-center randomized trial found that hospitals randomized to the critical pathway had significantly reduced average LOS (8.2 days vs. 9.6 days) and duration of intravenous antibiotics (4.6 days vs. 6.3 days), without worsening quality of life or increasing the rate of adverse events.¹⁰

What are the implications of our findings in the context of previous work performed in this area? Because substantial resources must be expended on pathway development and implementation,⁵ pathway effectiveness should be clearly demonstrated before additional resources are consumed for this management strategy. The confidence placed in critical pathways as a management tool to coordinate a large number of diagnoses and procedures to enhance clinical efficiency should perhaps be re-appraised. However, although the majority of pathways we evaluated were not effective, some clearly were. In fact, the PTCA pathway, in particular, was markedly successful in decreasing both LOS and resource utilization. Thus, increased efforts should be made to focus on those diagnoses or procedures that are most likely to benefit from the application of a critical pathway.

Evaluating the reasons behind the success or failure of an individual pathway is complex. When considering the results of our evaluation, we identified some possible explanations for why some pathways were successful while most were not. Importantly, these explanations were not based on a priori hypotheses and thus serve primarily for hypothesis generation. Pathways for a few surgical procedures were successful. Specifically, the pathways for C-section and kidney transplantation reduced LOS or resource utilization significantly more than secular trends would have predicted at our hospital. We were unable to determine why these pathways were successful while pathways for the other surgical procedures were not. One hypothesis is that kidney transplantation and C-section are relatively routine procedures, thus lending themselves to closely coordinated care. Additionally, perhaps if we had larger sample sizes, some of the other surgical pathways would have led to statistically significant decreases in LOS (e.g., knee arthroplasty) or resource utilization (e.g., hip arthroplasty). Most of the remaining unsuccessful surgical pathways, however, had very little effect on LOS or resource utilization, indicating that lack of power would be an unlikely explanation for our findings. Yet even if some of these pathway effects were statistically significant, often the modest effect sizes would not justify their use. Pathways are, after all, practical management tools, and their purpose is to obtain clinically meaningful effects, not merely statistically significant ones.

Pathways for cardiology-related domains also appeared to be successful. We have a potential explanation for these findings. During the past several years at our institution, much effort has been expended in improving the quality of care patients with cardiovascular disorders receive. Indeed, the cardiology leaders of these quality improvement efforts at our institution have published their successful efforts widely.^11-13 Thus, having well-respected opinion leaders spearhead quality improvement initiatives in a focused clinical area likely accounts for the success of the cardiology-related pathways in our hospital. However, pathways intended for 2 medical diagnoses (asthma and pancreatitis) were not effective, perhaps due to their notably variable clinical courses and high degree of patient-specific management strategies.

Given our results and those of others who have evaluated critical pathways, how should we approach the important issue of rational resource use in a hospital environment? We think that a "one-size-fits-all" management approach should be treated skeptically, regardless of its theoretical appeal. To choose the appropriate management technique, decision makers must understand the underlying processes and key points of care that are to be managed, and these appear to differ by medical procedure and even by hospital. For example, if LOS and ancillary resource consumption are the management targets, what is the "potential for control" of these targets for a given class of patients? What is the distribution of ancillary service costs (e.g., operating room, recovery room, radiology, and laboratory) for a class of patients? How many of these ancillary costs are potentially controllable? Who determines the utilization of these ancillary services? Are there national benchmarks that can be used to inform these decisions? If opportunities for controlling these ancillary service components are available, what are the key steps in the care process that are either "rate-limiting" or clinically discretionary? These questions highlight the complexity of decision making and imply that the widespread adoption of a particular management technique is unlikely to achieve universal success. Healthcare managers may thus be better served by choosing among a selection of management techniques for the approach most likely to be successful.

Our study should be considered in the context of several important limitations. First, this was an observational study using a before-and-after design. It was not an experiment with prospective evaluation, although we did adjust for secular trends and case mix. Nevertheless, other possible confounding factors could theoretically affect our findings. Second, we limited our analyses to only those pathways with enough patients to provide a stable estimate of effect. Perhaps pathways with smaller numbers of patients may be systematically more effective. We have no reason to think this assumption is true, however. Third, our analyses were performed at a single site, namely a tertiary-care referral hospital. Thus, our findings may not necessarily be indicative of pathway effectiveness in different clinical settings. Fourth, in an attempt to adjust for secular trends, we may have underestimated the effect of a pathway on either LOS or resource utilization. Despite our belief that adjustment for secular changes is the appropriate way to measure pathway effectiveness given a before-and-after study design, this method of analysis does bias our findings toward the null hypothesis (i.e., no difference between pathway and non-pathway patients). Finally, because we included patients whether or not they were actually managed by the particular pathway, the method of analysis is again biased toward the null. We were primarily interested in the effectiveness-not efficacy-of pathways as a management tool, as the utility of pathways to enhance efficiency in a carefully selected minority of patients within a certain clinical area is not their purpose.

Despite these limitations, however, our study reveals that it would be prudent to proceed cautiously with the development and implementation of critical pathways. Despite the theoretical benefit of such management tools, pathway effectiveness is still largely unproven. Although earlier studies found these tools to be effective, more recent studies that have considered secular trends have been much more cautionary. Our evaluation reveals that although some pathways significantly reduced LOS and resource utilization, most did not. Future efforts should therefore be placed on critically evaluating the effectiveness of critical pathways and understanding the reasons behind successful and unsuccessful strategies.

Acknowledgments

We thank Maureen Thompson, RN, for providing additional information about several of the specific pathways evaluated in this study.

Am J Med.

1. Pearson SD. Et tu, critical pathways? [letter; comment]. 1999;107:397-398.

Ann Intern Med.

2. Pearson SD, Goulart-Fisher D, Lee TH. Critical pathways as a strategy for improving care: problems and potential. 1995;123:941-948.

3. Every NR, Hochman J, Becker R, Kopecky S, Cannon CP. Critical pathways: a review. Committee on Acute Cardiac Care, Council on Clinical Cardiology, American Heart Association. Circulation. 2000;101:461-465.

Am J Med.

4. Weingarten S. Critical pathways: what do you do when they do not seem to work? [letter; comment]. 2001;110:224-225.

5. Darer J, Pronovost P, Bass EB. Use and evaluation of critical pathways in hospitals. Eff Clin Pract. 2002;5:114-119.

Ann Intern Med.

6. McMahon LF Jr, Creighton FA, Bernard AM, Pittinger WB, Kelley WN. The integrated inpatient management model. A new approach to clinical practice. 1989;111:318-326.

7. Campbell H, Hotchkiss R, Bradshaw N, Porteous M. Integrated care pathways. BMJ. 1998;316:133-137. 8. Pearson SD, Kleefield SF, Soukop JR, Cook EF, Lee TH. Critical pathways intervention to reduce length of hospital stay. Am J Med. 2001;110:175-180.

Am J Med.

9. Holmboe ES, Meehan TP, Radford MJ, Wang Y, Marciniak TA, Krumholz HM. Use of critical pathways to improve the care of patients with acute myocardial infarction. 1999;107: 324-331.

10. Marrie TJ, Lau CY, Wheeler SL, Wong CJ, Vandervoort MK, Feagan BG. A controlled trial of a critical pathway for treatment of community-acquired pneumonia. CAPITAL Study Investigators. Community-Acquired Pneumonia Intervention Trial Assessing Levofloxacin. JAMA. 2000;283:749-755.

JAMA.

11. Mehta RH, Montoye CK, Gallogly M, et al. Improving quality of care for acute myocardial infarction: the Guidelines Applied in Practice (GAP) Initiative. 2002;287:1269-1276.

12. Eagle KA, Gallogly M, Mehta RH, et al. Taking the national guideline for care of acute myocardial infarction to the bedside: developing the guideline applied in practice (GAP) initiative in Southeast Michigan. Jt Comm J Qual Improv. 2002;28:5-19.

Arch Intern Med.

13. Mehta RH, Das S, Tsai TT, Nolan E, Kearly G, Eagle KA. Quality improvement initiative and its impact on the management of patients with acute myocardial infarction. 2000;160:3057-3062.

Download PDF: Use of Critical Pathways to Improve Efficiency: A Cautionary Tale