Treatment Decisions for Complex Patients: Differences Between Primary Care Physicians and Midlevel Providers

Objectives:

To examine whether treatment change for diabetic patients presenting with elevated blood pressure (BP) differed between physicians and midlevel providers (nurse practitioners [NPs] and physician assistants [PAs]) and to determine reasons for any observed differences.

Methods:

Analyses were part of a prospective cohort study of 1169 diabetic patients with scheduled visits to 92 primary care providers (PCPs) in 9 Midwest Veterans Health Administration facilities presenting with a triage BP of ≥140/90 mm Hg. We analyzed predicted probabilities of treatment change by provider type.

Results:

The PCPs included 64 physicians, 21 NPs, and 7 PAs. Patients treated by physicians and midlevel providers did not differ in their mean visit BP, number of chronic conditions, age, or number of BP medications. Controlling for current and past BP readings and patient characteristics, physicians were significantly more likely than midlevel providers to initiate a treatment change for elevated BP at a visit (53.8% vs 36.4%; P = .001). After controlling for additional visit-specific factors, practice style, measurement, and organizational factors, physicians were still more likely to initiate a treatment change (52.5% vs 37.5%; P = .02).

Conclusions:

Midlevel providers were significantly less likely than physicians to change BP treatment for diabetic patients with multiple chronic conditions presenting with elevated BP at a single visit. We could not find good explanations for this difference. Given the expanding role of midlevel providers in delivering primary care to complex patients, we need to understand whether these treatment change differences lead to long-term differences in BP control.

(Am J Manag Care. 2009;15(6):373-380)

Probability of treatment change by provider type was analyzed for diabetic patients with multiple chronic conditions who presented with elevated blood pressure (BP) at a single visit to a Veterans Health Administration physician or midlevel provider (nurse practitioner or physician assistant).

• Midlevel providers were significantly less likely than physicians to initiate a treatment change for elevated BP, although it was not clear why.
• Whether treatment change at a single visit could lead to long-term differences in BP control needs to be investigated.
• The reasons for this difference are unclear even though we examined a broad array of explanatory variables.

Incorporating midlevel providers—nurse practitioners (NPs) or physician assistants (PAs)—into primary care practice is an effective way to boost revenue, improve patient access, and give physicians more time to see patients with complex conditions.^1-3 Since 1990, the number of PAs has grown from 24,000 to more than 64,000, while the number of NPs has more than quadrupled from 30,000 to more than 125,000.^4,5 Today, 26 states allow NPs to practice independently without physician oversight.⁶ However, many leading organizations do not support NPs practicing independently until there is further evidence regarding patient outcomes.^7-9

Previous studies have suggested that the quality of care delivered by midlevel providers is equal to that of physicians.^10-14 Studies evaluating processes of care for diabetes care (eg, glycosylated hemoglobin [A1C], urine microalbumin, and low-density lipoprotein [LDL] test ordering; treating high lipid levels) also have shown similar results.^15,16 However, the strongest body of evidence supporting lack of differences between physicians and midlevel providers is derived from studies assessing care for patients who are not clinically complex,¹⁷ provision of preventive care services,¹⁸ or delivery of care under the direct or delayed supervision of physicians.^19,20 The Veterans Health Administration (VHA), in response to ongoing pressure over the past decade to expand access to primary care,²¹ mandated a 200% increase in midlevel providers.^22,23 According to most recent estimates, approximately 30% of VHA primary care providers (PCPs) are midlevel.^24-27

VHA patients are complex, with poorer heath status, more medical conditions requiring intensive management, and higher use of medical resources than the general US population.^28-33 Most NPs and some PAs in the VHA system practice more or less independently, or under delayed physician supervision, and manage patients requiring high levels of decision-making complexity. Few studies have examined differences in practices among physicians and midlevel providers requiring decision making for complex patients with multiple chronic conditions.

The current study had 2 research objectives:

1. To examine whether the probability of treatment change for diabetic patients presenting with an elevated blood pressure (BP) measurement differed between physicians and midlevel providers.

2. To determine whether patient, provider, measurement, and/or organizational characteristics explained any of the observed differences.

METHODS

This analysis is part of a larger prospective cohort study (the ABATe [Addressing Barriers to Treatment for Hypertension] study) at 9 VHA facilities in 3 Midwestern states examining patient, clinician, and organizational factors that contribute to inadequate treatment of elevated BP among patients with diabetes with scheduled primary care visits.³⁴ The study protocol was approved by the institutional review boards of all participating facilities.

Study Setting and Design

Facilities. The 9 facilities varied in size and structure, with 3 large academic-affiliated medical centers, 1 large nonacademic medical center, and 1 large and 4 small communitybased outpatient clinics.

Primary Care Providers. We invited all nonresident PCPs with patient care responsibility at least 2 half days per week to participate in the study. Of the eligible 126 providers approached, 104 (71 physicians, 25 NPs, 8 PAs) consented to participate, for an overall recruitment rate of 83%. Once we excluded providers who had left their facilities by the time recruitment started, our final sample had 92 PCPs (64 physicians, 21 NPs, 7 PAs).

Patients. Research assistants approached 1556 patients who presented for a scheduled visit with a participating PCP and whose lowest triage BP was >140/90 mm Hg. In addition to the triage BP level, patients had to meet the following inclusion criteria at screening: confirmed a diagnosis of diabetes, confirmed receiving diabetes care from the PCP, and identified the VHA facility as their main site of care. We excluded patients with impaired decision-making ability (eg, dementia, traumatic brain injury) or terminal disease and patients who resided in a nursing home or who did not speak English. Of the 1556 approached, 213 patients were found not to be eligible and 1169 provided written informed consent to participate in the study (87% of those approached and eligible; median facility-level recruitment rate, 89%). We enrolled a median of 14 patients per PCP (range 1-16).

Data Sources

Data for our analyses came from 5 sources, including patient and provider surveys (see the eAppendix available at www.ajmc.com). Providers completed a baseline survey that assessed BP management practice style, general provider characteristics, and number of patients seen in clinic per half day. Providers also completed a brief visit survey after each patient visit within the same clinic session that the provider saw the patient (99% completion rate). The visit survey asked whether medications were changed, reasons if no changes were made, and what patient issues were discussed during the visit. Patient characteristics were obtained from a baseline patient survey (91% completion rate). Review of electronic medical record—documented actions taken by the provider regarding BP management (including repeat BP measurement) at the enrollment visit. Finally, patient age and comorbidity (using International Classification of Diseases, Ninth Revision codes in the year prior to the enrollment visit), prescribed medications and their dosages, BP values, and patient medication adherence were obtained from VHA automated data sources.^35,36

Variables

Dependent Variable: Treatment Change in Response to Blood Pressure Elevation. Our main dependent variable was whether or not there was a treatment change by the provider in response to the elevated triage BP. We considered treatment change to have occurred if (1) the PCP indicated on the visit survey (n = 1151) or in the medical record (n = 18) if a visit survey was not completed that he or she added a medication or increased the dose of an existing medication; or (2) if the PCP documented in the medical record that he or she intended to bring the patient back for BP follow-up within 4 weeks.

Independent Variables and Covariates. The Hypertension Clinical Action Model describes successful clinical action in response to an elevated BP as being influenced by clinical uncertainty about the true BP values, competing demands, prioritization such as unrelated comorbidities, medication-related factors such as adherence, and organizational factors.³⁴ Using this model as our motivating framework, we examined specific patient, provider, visit, and organizational factors that we hypothesized could explain differences in BP treatment change between midlevel providers and physicians (Figure).

Baseline patient characteristics included both sociodemographic and clinical variables. We included the lowest systolic blood pressure (SBP) and the lowest diastolic blood pressure (DBP) taken by the triage staff at the enrollment visit, and the patient’s mean SBP in the year before the enrollment visit. The SBPs used to calculate mean SBP in the prior year included readings from the emergency department but not inpatient stays. We included all outpatient SBPs as they better reflect a patients’ overall BP severity. We also included patient’s age, race, sex, and number of chronic conditions using a VHA Health Economics Resource Center adaptation of the Agency for Healthcare Research and Quality Clinical Classification Software,^32,37,38 and the number of BP medication classes the patient was on before the enrollment visit.

Our patient-specific factors included 3 variables that could play a role in treatment change at a visit. Adherence to BP medications was assessed by using Steiner’s refill gap measure calculated as a single measure for all BP medications prescribed.^39-41 We also included 2 competing demand factors: (1) whether the provider reported on the visit survey that he or she discussed 1 or more discordant conditions with the patient at the visit (eg, conditions pathophysiologically unrelated to hypertension and diabetes⁴²) and (2) whether the provider reported on the visit survey that he or she discussed medications or adherence issues with the patient during the visit.

Third, we included an organizational factor that might influence treatment change by provider type: the time available to see a patient (assessed by a question on the baseline provider survey, which asked for the average number of patients the provider saw in a half-day clinic).

We included several factors related to provider practice style: provider experience (defined as the number of years that the provider had been practicing), provider knowledge of SBP targets for patients with diabetes (based on reporting a target SBP of <130 mm Hg vs a higher target), and provider prioritization of BP (defined as a provider’s willingness to wait longer than 4 weeks vs less than 4 weeks to follow up a BP level that was slightly above the provider’s goal).

Finally, we included 2 variables related to clinical uncertainty regarding whether the enrollment visit triage BP reflected the patient’s true BP value: (1) when the provider indicated in the medical record a repeat BP lower than 140/90 mm Hg or (2) when the provider recorded that the patient’s home BP was adequate. We defined a priori the lowest triage BP done by a nurse with an electronic cuff as the unbiased measure of a patient’s BP on the day of a visit. PCPs idiosyncratically chose to repeat a BP measurement, usually with a manual cuff, and not surprisingly were much less likely to intensify therapy when they documented that the BP was less than 140/90 mm Hg.³⁴ The literature suggests that these measurements are less accurate and less reliable than nursing BP measurements done with an electronic cuff.34 Like reports of well-controlled home BP by patients, it is difficult to know how to interpret these alternative assessments of meeting BP targets, but it is important to control for the use of these alternative BP assessments in comparing intensification rates between provider types.

We conducted sensitivity analyses to further examine differences by provider type in treatment intensification in several ways. First, we incorporated patients’ reports of a problem adding another medication to control their BP (from the baseline survey) to the full model. Second, we examined differences by provider type regarding treatment change for elevated BP with only medication intensification (without inclusion of follow-up within 4 weeks) as the dependent variable. Third, we examined whether patients already being on high doses of BP medications contributed to lack of treatment change. Fourth, we evaluated differences in treatment change controlling for providers’ propensity to intensify treatment. We also examined whether patients of midlevel providers and physicians differed in other patient characteristics that could possibly be related to differences in therapy intensification. We also compared reasons midlevel providers and physicians provided from among choices presented on the visit survey for not intensifying medications among the 635 patients whose therapy was not intensified at the visit.

Statistical Analyses

We constructed a 3-level multivariate model (with patients clustered within provider clustered within site).⁴³ Using logistic regression models, we first examined the association of provider type (physician vs midlevel) with treatment change (BP medication intensification at the visit or planned prompt follow-up within 4 weeks to reassess BP). We controlled for baseline patient sociodemographic and clinical characteristics only (enrollment visit BP and mean SBP in the prior year, age, race, sex, number of chronic conditions, number of BP medication classes). In a second model, we additionally controlled for provider factors (years in practice, provider prioritization of BP, goals for BP), visit factors (discussion of medication adherence or discordant health condition at visit, patient medication adherence gaps), organizational factors (number of patients seen per half day), and BP measurement factors (home BP and repeat visit BP) that could further explain differences by provider type. For clarity of presentation, we calculated a probability of treatment change by provider type for a typical patient (white, male, and age 56-75 years, with all other variables held at their mean). The provider and site random effects were set to zero, which means that the predicted probability is for that of a median provider at the median site seeing a white, male, 56- to 75-year-old patient, with all other characteristics set to their mean population values apart from the provider type (physician vs midlevel). All analyses were conducted using Stata software, version 10.0 (StataCorp, College Station, TX).

RESULTS

Baseline Characteristics

Patient Characteristics. Baseline patient demographic and clinical characteristics by provider type are presented in Table 1. There were no differences between patients seen by midlevel providers and physicians with respect to mean SBP at the enrollment visit (about 154 mm Hg) and prior year SBP (about 145 mm Hg). There also were no differences in patients’ age, race, sex, or number of comorbid conditions. Patients seen by midlevel providers had a higher mean DBP (79.4 vs 77.4 mm Hg; P = .01) and had been prescribed, on average, 2.0 BP medications compared with 2.2 BP medications for patients seen by physicians.

In terms of visit-specific factors, there were no differences in patient medication adherence gaps or discussion of discordant conditions at the visit. However, midlevel providers were more likely to report discussing medications and/or compliance at the visit (14% vs 9%; P = .01).

Provider Characteristics. There were no differences between midlevel providers and physicians in years of practice, knowledge of BP management in diabetes, or prioritization of BP control for diabetes patients.

Organizational and Measurement Factors. Physicians were more likely to report seeing 6 or more patients per halfday clinic than midlevel providers (84% vs 61%; P = .01). There were no differences by provider type in recording that the patients’ home BP was adequate or that a repeat BP was less than 140/90 mm Hg.

Treatment Change

Overall, 573 of 1169 (49%) patients had a treatment change for elevated BP at the visit. Most of those patients had a medication started and/or dose increased (511/573), and the remaining 62 had a documented plan to follow up BP within 4 weeks. The likelihood of treatment change increased with higher levels of triage SBP and DBP and higher levels of mean prior year SBP. Age, race, and sex were not associated with the likelihood of treatment change.

Predicted probabilities for treatment change by provider type are presented in Table 2. In model 1, after controlling for current/past BPs and patient sociodemographic and clinical characteristics, physicians were significantly more likely to initiate a treatment change at the visit for an elevated BP (53.8% vs 36.4%; P = .001). In model 2, after controlling for patient sociodemographic and clinical characteristics, visitspecific factors, provider practice style, measurement, and organizational factors, physicians were still significantly more likely to initiate a treatment change at the visit (52.5% vs 37.5%; P = .02).

Sensitivity Analyses

After adjusting for patients’ reports of a problem adding another medication to control their BP in the full model, physicians were still more likely than midlevel providers to initiate treatment change for elevated BP (53.8% vs 36.9%; P = .01). Similarly, physicians were more likely than midlevel providers to intensify treatment for elevated BP even after adjustment for patients already being on high doses of BP medications and providers’ propensity to intensify treatment change (52.5% vs 37.6%; P = .02; and 52.6% vs 37.3%; P = .02, respectively).

To assess whether the definition of treatment change in our model altered our findings, we repeated our analyses with only medication intensification (without inclusion of followup within 4 weeks) in the dependent variable. The differences between physicians and midlevel providers with respect to initiation of treatment change remained large and significant (45.6% vs 26.5%; P <.001).

When we examined other patient characteristics that could possibly be related to differences in therapy intensification, we found no differences by provider type in rates of patient home BP monitoring (62% vs 57%; P = .118), nor of patient report of side effects related to BP medications (P = .22).

Reasons for Lack of Treatment Change

To further elucidate potential reasons for the observed differences in intensification rates, we examined the reasons PCPs provided for not intensifying treatment that we collected immediately after each visit (Table 3). The mean number of responses regarding not intensifying therapy for elevated BP did not differ between physicians and midlevel providers (physicians 1.50 ± 0.80; PAs/NPs 1.51 ± 0.81). Physicians were more likely than midlevel providers to indicate as a reason for not intensifying therapy that their patients were unwilling to change or add medication (8% vs 3%; P = .03). Midlevel providers listed “other reasons” for not intensifying therapy more often than physicians (19% vs 14%; P = .05). The category “other reasons” included prescribing diet and exercise, providing home BP monitors, referring patient to a BP clinic, or deferring to a PCP outside the VHA system.

DISCUSSION

Midlevel providers were significantly less likely to change BP treatment for diabetic patients with multiple chronic conditions even after controlling for a number of patient, provider, and organizational characteristics. Specifically, differences persisted even after we controlled for clinical uncertainty about the true BP value and for several visit, patient, provider, and organizational factors. Results did not change significantly when we examined differences by provider type in medication intensification, rather than in medication intensification and follow-up combined. Indeed, the fairly comprehensive set of potential explanatory variables that we assessed did not seem to account for any of the difference in intensification rates between physician and midlevel providers.

Outcomes for patients cared for by NPs have been found to be comparable to those for patients cared for by physicians.^18-22,44,45 Hopkins and colleagues18 conducted a welldesigned controlled trial randomly assigning 1300 (214 diabetic) patients presenting to the emergency department to midlevel providers or physicians; they measured outcomes of patient satisfaction, health status, and utilization and found no differences by provider type in patients’ A1C level or BP at 6-month follow-up. However, this study was conducted in a single city, included an exclusively Hispanic (>90%) population, and had a small sample of providers—only 7 NPs and 17 physicians. The more recent study by Ohman-Strickland and colleagues¹⁵ on diabetes care quality found that several processes of care for diabetes such as checking glycemic and lipid levels were more frequent in family practices that employed NPs than in all physician practices. However, their analyses were conducted at the practice level (with confounding organizational characteristics), did not assess differences at the provider level, and did not adjust for important patient sociodemographics. Additionally, details on the role of midlevel providers within the pratice and whether midlevel providers functioned independently or had a specific role within the practice were not provided.

Although much of the attention in comparative studies has focused on patient outcomes, there is a growing body of evidence that NPs and physicians also differ in the preventive and treatment strategies they use during patient encounters. Analyses of the National Ambulatory Medical Care Survey found that midlevel providers had more visits for “nonillness” care and more emphasis on therapeutic/preventive care.^46,47 Using medical record reviews of patient encounters, other investigators have found that physicians focused care delivery on secondary prevention, while midlevel providers tended to provide more education and counseling.²³ Because control of BP among diabetes patients falls into the category of secondary prevention, this difference in focus is one possible explanation of the difference in treatment change rates that we observed.

Guidelines for comparative studies call for a randomized controlled trial or studies that use several sources of data conducted at multiple sites. To our knowledge, ours is the largest multisite study with the most clinically detailed data sources to examine treatment intensification by provider type for medically complex patients. There are, however, some limitations to our study. Although there is a national consensus that BP in high-risk patients (eg, patients with diabetes) is undertreated, it is impossible to know what the right level of treatment change is for these complex patients. While we found that midlevel providers changed treatment less often than physicians at a single visit, it is possible that physicians in this study were overtreating these patients. However, we did choose a higher cut point for elevated BP (140/90 mm Hg) compared with that recommended by the American Diabetes Association⁴⁸ (SBP <130 mm Hg), which more than 80% of providers surveyed thought should be the systolic target for patients with diabetes. So there is little question that the presenting BP needed attention. Second, it could be that midlevel providers were more appropriately taking into account medication side effects and contraindications that were not fully measured in our study. However, we assessed patient reported concern for medication side effects overall and for BP medications in particular using items adapted from the Horne scale⁴⁹ and found no significant differences in patient-reported medication side effects by provider type. Third, this study was designed to look at treatment change at one visit. A different picture may emerge when we examine treatment change over time. It is possible that midlevel providers intensify less frequently but as effectively in the long run or that they more effectively address adherence issues. Finally, our study was conducted in the VHA and hence may not be generalizable to other healthcare settings.

We found that midlevel providers were significantly less likely to change BP treatment for diabetic patients with multiple chronic conditions presenting with elevated BP at a single visit. However, we lack a potential explanation for these differences, despite examining a broad array of possible explanatory variables based on a comprehensive conceptual model. Thus, it is difficult to consider any specific policy changes based on this finding. We can say that given the expanding role of midlevel providers in delivering primary care to complex patients, we need to better understand whether differences in BP treatment change by provider type at a single visit lead to long-term differences in BP management and control.

Acknowledgments

The authors wish to acknowledge the invaluable contributions of our recruitment coordinator (Claire Robinson, MPH) and the efforts of our research assistants (Stacey Hirth, BS, Susan Jaeger, MPH, Madhavi Diwanji, MBA, Janice Thomson, MSW, Caroline Solomon, BA, and Diana Newman, DPA), who worked to recruit patients for this study; our site principal investigators (Drs David Aaron, Martin Bermann, and Ketan Shah), without whom this study could not have taken place; and the many providers and patients who participated in the study.

Author Affiliations: From the Roudebush VA Medical Center (US), Indianapolis, IN; the Division of General Medicine & Geriatrics (US), Indiana University School of Medicine, Indianapolis; the Center of Excellence (EAK, MK, BJZ-F, RH, TPH), Department of Veterans Affairs, Health Services Research & Development Service, Ann Arbor, MI; and the Department of Internal Medicine (EAK, BJZ-F, TPH), University of Michigan, Ann Arbor.

Funding Source: This work was supported by a research grant from the Department of Veterans Affairs, Health Services Research & Development Service (IIR 02-225). This work also was supported in part by Michigan Diabetes Research and Training Center Grant P60DK-20572 from the National Institute of Diabetes and Digestive and Kidney Diseases, National Institutes of Health. Dr Zikmund-Fisher is supported by a career development award from the American Cancer Society (MRSG-06-130-01-CPPB). The funding sources were not involved in the conduct or analysis of this study. The funding agreements ensured the authors’ independence in designing the study, interpreting the data, and publishing the report. The views expressed in this article are those of the authors and do not necessarily reflect the position or policy of the Department of Veterans Affairs, Indiana University, or the University of Michigan.

Author Disclosure: The authors (US, EAK, MLK, BJZ-F, RGH, TPH) report no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article.

Authorship Information: Concept and design (US, EAK, MLK, BJZ-F, TPH); acquisition of data (EAK, MLK); analysis and interpretation of data (US, EAK, MLK, BJZ-F, RGH, TPH); drafting of the manuscript (US, MK, RH, TPH); critical revision of the manuscript for important intellectual content (EAK, MLK, BJZ-F, RGH, TPH); statistical analysis (MLK, RGH, TPH); provision of study materials or patients (US); obtaining funding (EAK, BJZ-F, TPH); administrative, technical, or logistic support (MLK); and supervision (EAK, TPH).

Address correspondence to: Usha Subramanian, MD, MS, Roudebush VA Medical Center, 250 University Blvd, IF 122, Indianapolis, IN 46202. E-mail: usubrama@iupui.edu.

1. Brown SA, Grimes DE. Nurse Practitioners and Certified Nurse-Midwives: A Meta-Analysis of Studies on Nurses in Primary Care Roles. Washington, DC: American Nurses Publishing; 1993.

2. Dial TH, Palsbo SE, Bergsten C, Gabel JR, Weiner J. Clinical staffing in staff- and group-model HMOs. Health Aff (Millwood). 1995;14(2):168-180.

3. Hooker RS, Berlin LE. Trends in the supply of physician assistants and nurse practitioners in the United States. Health Aff (Millwood). 2002;21(5):174-181.

4. American Academy of Physician Assistants. Home page. http://www.aapa.org/. Accessed July 24, 2008.

5. American Academy of Nurse Practitioners. AANP information. http://www.aanp.org/AANPCMS2/AboutAANP/AANP Information. Accessed July 24, 2008.

6. Cooper RA, Henderson T, Dietrich CL. Roles of non-physician clinicians as autonomous providers of patient care. JAMA. 1998;280(9):795-802.

7. Physician assistants and nurse practitioners [position paper]. American College of Physicians Task Force on Physician Supply. Ann Intern Med. 1994;121(9):714-716.

8. Regents discuss midlevel providers, Oregon Health Plan and GME. ACP-ASIM Observer. March 2000.

9. Family physician workforce reform. As approved by the 2006 Congress of Delegates. Recommendations of the American Academy of Family Physicians. 2006. http://www.aafp.org/online/en/home/policy/policies/w/workforce.html. Accessed March 6, 2009.

10. Sox HC Jr. Quality of patient care by nurse practitioners and physicians’ assistants: a ten year perspective. Ann Intern Med. 1979;91(3):459-468.

11. Mundinger MO, Kane RL, Lenz ER, et al. Primary care outcomes in patients treated by nurse practitioners or physicians: a randomized trial. JAMA. 2000;283(1):59-68.

12. Lenz ER, Mundinger MO, Kane RL, Hopkins SC, Lin SX. Primary care outcomes in patients treated by nurse practitioners or physicians: 2 year follow-up. Med Care Res Rev. 2004;61(3):332-351.

13. Horrocks S, Anderson E, Salisbury C. Systematic review of whether nurse practitioners working in primary care can provide equivalent care to doctors. BMJ. 2002;324(7341):819-823.

14. Brown SA, Grimes DE. A meta-analysis of nurse practitioners and nurse midwives in primary care. Nurs Res. 1995;44(6):332-339.

15. Ohman-Strickland PA, Orzano AJ, Nutting PA, et al. Quality of diabetes care in family medicine practices: influence of nurse-practitioners and physician’s assistants. Ann Fam Med. 2008;6(1):14-22.

16. Roumie CL, Grogan EL, Falbe W, et al. A three-part intervention to change the use of hormone replacement therapy in response to new evidence. Ann Intern Med. 2004;141(2):118-125.

17. Curzio JL, Rubin PC, Kennedy SS, Reid JL. A comparison of the management of hypertensive patients by nurse practitioners compared with conventional hospital care. J Human Hyperten. 1990;4(6):665-670.

18. Hopkins SC, Lenz ER, Pontes NM, Susan SX, Mundinger MO. Context of care or provider training: the impact on preventive screening practices. Prev Med. 2005;40:718-724.

19. Carter AJ, Chochinov AH. A systematic review of the impact of nurse practitioners on cost, quality of care, satisfaction and wait times in the emergency department. CJEM. 2007;9(4):286-295.

20. Wilson IB, Landon BE, Hirschhorn LR, et al. Quality of HIV care provided by nurse practitioners, physician assistants, and physicians. Ann Intern Med. 2005;143(10):729-736.

21. Kizer KW, Fonseca ML, Long LM. The veterans healthcare system: preparing for the twenty first century. Hosp Health Serv Adm. 1997;42(3):283-298.

22. Kizer KW. Prescription for Change. Washington, DC: Department of Veterans Affairs; 1997.

23. Lynn MM, Achtmeyer C, Chavez C, Zicafoose B, Therien J. The evolving role of advanced practice nursing within the New Veterans’ Health Administration. Health Care Manage Rev. 1999;24(4):80-93.

24. Primary Care Management Module (PCMM). Enhancements for Direct Primary Care: User’s Guide. January 2003. www.va.gov/vdl/documents/Clinical/Pri_Care_Mgmnt_Module_(PCMM)/sd_53_277.um-final.doc. Accessed June 1, 2009.

25. Department of Veterans Affairs, Veterans Health Administration. Primary Care Direct Patient Care Time. November 3, 2006. VHA directive 2006-060. http://www1.va.gov/vhapublicationsViewPublication.asp?pub_ID=1505. Accessed March 6, 2009.

26. Huang PY, Yano EM, Lee ML, Chang BL, Rubenstein LV. Variations in nurse practitioner use in Veterans Affairs primary care practices. Health Serv Res. 2004;39(4):887-904.

27. Running A, Calder J, Mustain B, Foreschler C. A survey of nurse practitioners across the United States. Nurse Pract. 2000;25(6 part 1):15-16, 110-116.

28. Mrus JM, Leonard AC, Yi MS, et al. Health-related quality of life in veterans and nonveterans with HIV/AIDS. J Gen Intern Med. 2006;21(S5):S39-S47.

29. Kazis LE, Miller D, Clark J, et al. Health-related quality of life in VA patients: results of the Veterans Health Study. Arch Intern Med. 1998;158(6):626-632.

30. Greenfield S, Rogers WH, Mangotich M, Carney MF, Tarlov AR. Outcomes of patients with hypertension and non-insulin-dependent diabetes mellitus treated by different systems and specialties: results from the Medical Outcomes Study. JAMA. 1995;274(8):1436-1444.

31. Rogers WH, Kazis LE, Miller DR, et al. Comparing the health status of VA and non-VA ambulatory patients: the veterans’ health and medical outcomes studies. J Ambul Care Manage. 2004;27(3):249-262.

32. Yu W, Ravelo A, Wagner TH, et al. Prevalence and costs of chronic conditions in the VA health care system. Med Care Res Rev. 2003;60 (3 suppl):146S-167S.

33. Payne SM, Lee A, Clark JA, et al. Utilization of medical services by Veterans Health Study (VHS) respondents. J Ambul Care Manage. 2005;28(2):125-140.

34. Kerr EA, Zikmund-Fisher BJ, Klamerus ML, Subramanian U, Hogan MM, Hofer TP. The role of clinical uncertainty in treatment decisions for diabetic patients with uncontrolled blood pressure. Ann Intern Med. 2008;148(10):717-724.

35. Cowper DC, Hynes DM, Kubal JD, Murphy PA. Using administrative databases for outcomes research: select examples from VA Health Services Research and Development. J Med Syst. 1999;23(3):249-259.

36. Render ML, Welsh DE, Kollef M, et al. Automated computerized intensive care unit severity of illness measure in the Department of Veterans Affairs: preliminary results. SISVistA Investigators. Scrutiny of ICU Severity Veterans Health Systems Technology Architecture. Crit Care Med. 2000;28(10):3540-3546.

37. Clarke P, Kelman C, Colagiuri S. Factors influencing the cost of hospital care for people with diabetes in Australia. J Diabetes Complications. 2006;20(6):349-359.

38. Kilbourne AM, Cornelius JR, Han X, et al. Burden of general medical conditions among individuals with bipolar disorder. Bipolar Disord. 2004;6(5):368-373.

39. Steiner JF, Koepsell TD, Fihn SD, Inui TS. A general method of compliance assessment using centralized pharmacy records. Description and validation. Med Care. 1988;26(8):814-823.

40. Steiner JF, Prochazka AV. The assessment of refill compliance using pharmacy records: methods, validity, and applications. J Clin Epidemiol. 1997;50(1):105-116.

41. Choo PW, Rand CS, Inui TS, et al. Validation of patient reports, automated pharmacy records, and pill counts with electronic monitoring of adherence to antihypertensive therapy. Med Care. 1999;37(9):846-857.

42. Kerr EA, Piette JD. Impact of comorbid chronic conditions on diabetes care. Diabetes Care. 2006;29(3):725-731.

43. Snijders TAB, Bosker RJ. Multilevel Analysis: An Introduction to Basic and Advanced Multilevel Modelings. Thousand Oaks, CA: Sage Publications; 1999.

44. Sackett DL, Spitzer WO, Gent M, Roberts RS. The Burlington randomized trial of the nurse practitioner: health outcomes of patients. Ann Intern Med. 1974;80(2):137-142.

45. Hooker RS, McCaig LF. Use of physician assistants and nurse practitioners in primary care, 1995-1999. Health Aff (Millwood). 2001;20(4):231-238.

46. Moody NB, Smith PL, Glenn LL. Client characteristics and practice patterns of nurse practitioners and physicians. Nurse Pract. 1999;4(3):94-103.

47. Lin SX, Hooker RS, Lenz ER, Hopkins SC. Nurse practitioners and physician assistants in hospital outpatient departments, 1997-1999. Nurs Econ. 2002;20(4):174-179.

48. Clinical practice recommendations 2005. Diabetes Care. 2005; 28(suppl 1):S1-S79.

49. Horne R, Weinman J. Patients’ beliefs about prescribed medicines and their role in adherence to treatment in chronic physical illness. J Psychosom Res.1999;47(6):555-567.

Download PDF: Treatment Decisions for Complex Patients: Differences Between Primary Care Physicians and Midlevel Providers