Patient Need and Provider Referrals to Diabetes Self-management Education

ABSTRACT

Objectives: This study measured the likelihood of adult patients with diabetes being referred to diabetes self-management education (DSME) when “in need” according to clinical guidelines and identified which types of clinical need predict a greater likelihood of provider referral to DSME.

Study Design: This repeated cross-sectional analysis utilized patient electronic health records (EHRs) and a statewide health information exchange database to examine a 7-year panel (2010-2016) of adult patients with diabetes. Our analytic sample included 8782 adult patients with diabetes with a total of 356,631 encounters.

Methods: Fixed-effects linear probability models with clustered robust standard errors estimated the association between patients’ need for DSME and likelihood of being referred to the service. Models controlled for patients’ health status, prior utilization, encounter setting, comorbidity risk scores, the state’s expansion of Medicaid, and the count of accredited DSME program sites in the community.

Results: Most patient encounters indicated at least 1 type of need for DSME, but less than 7% of those encounters with a documented need resulted in a provider referral. In regression analysis, clinical indicators of need increased the likelihood that patients would be referred to DSME. Patients exhibiting multiple types of need were most likely to be referred to DSME.

Conclusions: Although findings indicate that patient need for DSME does improve the likelihood of being referred, provider referral rates were significantly lower than anticipated. Future research should explore barriers to clinical guideline adherence and whether clinical decision support in EHR systems can facilitate provider referrals.

Am J Manag Care. 2021;27(6):e201-e207. https://doi.org/10.37765/ajmc.2021.88669

Takeaway Points

The Algorithm of Care for diabetes self-management education (DSME) identifies critical points in patient care when providers should refer patients “in need” to DSME. However, no research has explored whether these clinical indicators of need increase the likelihood of receiving a referral. This article:

• is the first application of clinical guidelines to electronic health records (EHRs) to examine need-based referrals to DSME;
• demonstrates that although clinical indicators of need increase the likelihood of referral, only a small proportion of patients in need are being referred; and
• identifies clinical decision support in EHR systems as a potential facilitator of provider DSME referrals.

National standards of care for diabetes recommend that primary care providers assess patients’ need for diabetes self-management education (DSME) at diagnosis, during annual condition assessments, at the onset of complicating factors, and following events that affect the continuity of care.^1-3 Receipt of DSME in accordance with accepted standards is associated with improved patient outcomes,^1,3-6 yet DSME is largely underutilized by patients in need.^1,7-13

Contrary to national standards for DSME delivery, providers frequently do not refer patients to DSME services.^14-16 In fact, instead of referring at the recommended intervals, providers report viewing DSME as a “last resort” following major glycemic crises or when traditional clinical treatment fails.^14,17 Moreover, evidence suggests that clinical need is not a key factor in providers’ referral behavior.¹⁸ It is unclear whether providers’ classification of need for DSME aligns with established standards of care.

This study explores providers’ DSME referrals. We are particularly interested in understanding at which time points in patient care providers consider patients “in need” of DSME. Using the Algorithm of Care framework^1,3 to identify patients in need, we measure the association between patients’ clinical need for DSME and provider referrals during patient encounters. Additionally, because state laws and regulations generally limit DSME referral authority to licensed physicians, we test the robustness of our findings by examining the impact of patient need on referrals during physician encounters.

METHODS

Setting

Indiana, the Midwestern state in which this study was set, has a well-established and long-standing mandate that extends DSME benefits to all publicly and privately insured patients with diabetes.¹⁹ Under the state’s mandate, insurance coverage for DSME is triggered at diagnosis, following a change in health status, and when reeducation is recommended by providers.¹⁹ This reimbursement provision grants physicians the authority and flexibility to apply the DSME Algorithm of Care as recommended. This state is also the home of one of the oldest and largest community health information exchanges (HIEs) and urban safety-net health systems in the United States, Eskenazi Health.²⁰ Like other safety-net health systems, Eskenazi Health largely serves patients who have government-funded insurance and lower socioeconomic status.²¹ Moreover, the health system has had a highly rated Association of Diabetes Care & Education Specialists (ADCES)–accredited DSME program since 2014, which offers colocated DSME in federally qualified health clinics, as well as in the hospital setting. This facilitates the DSME referral process at all points of care rather than limiting it to primary care visits. Finally, the health department for the county in which the health system is located has had a multisite ADCES-accredited DSME program since 2011.

Data Sources

The primary data sources were the health system’s electronic health record (EHR) and the community HIE database. The EHR includes structured referral data in a computerized order entry system from every provider within the health system, as well as unstructured provider notes. The HIE stores EHRs that include information on patient demographics, diagnoses, laboratory results, provider orders, and encounters from 25,000 physicians, 106 hospitals, 110 clinics, and numerous other health organizations in the state.²² These data provide the added benefit of tracking patients across health systems over time.

Sample

The study panel included adults (18 to 64 years) with a type 2 diabetes diagnosis (before or during the study period) who had at least 1 provider encounter each year in the Eskenazi system between 2010 and 2016. The final sample included 8782 distinct patients with a total of 356,631 encounters.

Outcome of Interest

Our primary outcome of interest was an indicator of whether a patient encounter resulted in a DSME referral. Referrals were identified from orders placed using a computerized order entry system and unstructured provider notes using natural language processing techniques.^23,24 Referrals were then linked to encounters by patient and dates. These notes and orders are the best direct measure of provider referrals.

Determinant of Interest

The primary determinant of interest was an indicator of patient need for DSME at the time of the encounter. Patient need was measured based on the criteria set in the Algorithm of Care framework.^1-3 Patients were considered in need if they met any of these criteria:

Initial diagnosis. Patients should first receive DSME at the time of initial diagnosis. In the absence of a specific International Classification of Diseases, Ninth Revision (ICD-9) or Tenth Revision (ICD-10) code for onset of diabetes, we employed an identification schema informed by a classification algorithm noted in the existing literature.²⁵ Using the extracted EHR and HIE data, we identified the first date when any ICD-9 or ICD-10 code for type 2 diabetes was notated in each patient’s record from 2010 to 2016. ICD codes included all variations of 249.x and 250.x for ICD-9 and all variations of E08.x, E09.x, E10.x, E11.x, and E13.x for ICD-10. That encounter was then coded as an initial diagnosis only if, at any encounter prior to that encounter date, the patient’s glycated hemoglobin A_1c (HbA_1c) had never exceeded 7.0% (mmol/mol), there were no provider referrals to diabetes care services, and provider notes did not include any indicator of type 2 diabetes or prediabetes.

Annual condition assessment. Among patients with existing diabetes, referral to DSME should occur when annual assessments indicate that the condition is not being properly managed. Therefore, patients were considered in need at the encounter if their HbA_1c level exceeded 7.0% (mmol/mol), systolic blood pressure (BP) was greater than 140 mm Hg or diastolic BP was greater than 90 mm Hg, cholesterol ratio was 5.0 or higher, or body mass index was 30 or more.² Because dates of the laboratory results often did not match encounter dates, we used the most recent measure before the encounter as indicators of improper management of the condition at the time of the encounter.

Complicating factors. Because DSME provides patients with the skills and tools necessary to manage the condition in their everyday lives, the onset of complicating factors that alter patients’ day-to-day routine signals a need for reeducation. These complicating factors include a newly diagnosed comorbid condition, social issues, or financial strain. Patients were classified as being in need of DSME if any of these complicating factors were present at the time of the encounter. To determine if patients received a new diagnosis of congestive heart failure, hypertension, renal failure, or depression (all of which patients with diabetes are at increased risk of developing²) at the time of the encounter, we compared ICD-9/ICD-10 codes at the time of the encounter with admitting and discharge diagnoses at previous encounters. If these conditions were not notated in the patient’s record at previous encounters, the patient was classified as having a new diagnosis of a comorbid condition. We then utilized patient appointment records, provider notes, and service orders to determine whether the patient was experiencing social issues or financial strain at the time of the encounter. Mentions of homelessness, food insecurity, or family issues, as well as referrals to or appointments with a social worker, financial counselor, or medical-legal partnership, were considered indicators of social or financial strain.

Transitions in care. Events signaling a transition in care also may necessitate a DSME referral. These events typically include a transition from inpatient care or a change in insurance status. For a transition in care, patients were considered in need of DSME at the outpatient encounter immediately following an inpatient stay for a diabetes-related complication. Changes in insurance status from uninsured to insured and insured to uninsured were determined using the payer at the time of the encounter. Newly insured patients who previously were listed as self-pay at all prior visits were considered to have had a change in insurance status. Self-pay patients who were listed as insured at all prior visits were also considered to have had a change in insurance status. Patients meeting either of these criteria were considered in need of DSME.

Analysis

Frequencies, percentages, and means were used to describe the variables of interest. A fixed-effects linear probability model with clustered robust standard errors estimated the association between patients’ evaluated need for DSME and receipt of a referral to DSME. We controlled for patient and community factors associated with referral ordering behaviors²⁶ and access,²⁷ including health status, prior utilization, and comorbidity risk scores, which were calculated using the widely accepted Elixhauser measure of comorbidity when using administrative data.²⁸ We also controlled for the enactment of the Affordable Care Act, the state’s expansion of Medicaid, and the count of accredited DSME program sites on site and in the community. Analyses were conducted for all encounters, outpatient encounters, and hospital-based (ie, inpatient and emergency department) encounters. Lastly, the state’s insurance mandate limits reimbursement to DSME that is ordered by a licensed physician.¹⁹ This health system’s EHR system includes a digital authorization process that allows physicians to finalize DSME referrals from other providers. However, many health systems do not have these capabilities. Therefore, to improve the generalizability of our findings, we repeated the modeling strategy and measured association between patient need and DSME referrals but limited the panel to physician encounters (8120 patients; 93,986 encounters) and grouped by encounter setting. All models were checked for consistency using a logistic model.

RESULTS

Of the 356,631 patient encounters that occurred between 2010 and 2016, nearly two-thirds of those encounters indicated a need for DSME (Table 1). Most commonly, encounter records indicated a transition in care or insurance coverage (38.0%) or co-occurring types of need (15.6%). Additionally, on average, patients’ comorbidity scores indicated an increased risk of hospitalization and mortality.²⁸ Most encounters occurred in outpatient settings and were covered by some type of health insurance.

Despite indicators of need for DSME being present at most patients’ encounters, less than 7% of those encounters resulted in a provider referral (Table 2). Stratifying by type of need, less than 10% of encounters with newly diagnosed patients (7.7%) and patients experiencing transitions in care/insurance (3.1%) resulted in a referral to DSME. Encounters with patients who exhibited multiple types of need had the highest percentage of provider referrals (13.1%), followed by encounters with patients not meeting target levels (12.1%) and encounters with patients experiencing complicating factors (11.6%). Unadjusted models show that all indicators of clinical need for DSME were associated with whether a provider issued a referral (P ≤ .01).

Patient Need and Provider Referrals to DSME, by Encounter Setting

All encounters. Controlling for other factors at the time of the encounter, each type of need was associated with the likelihood of receiving a referral to DSME (Table 3). A new type 2 diabetes diagnosis, having off-target levels, the onset of complicating factors, transitions in care and insurance coverage, and multiple types of need increased the likelihood of being referred to DSME (P ≤ .01). Encounters with patients who were older or with higher comorbidity scores were less likely to result in a referral (P ≤ .01). Both paying for the encounter out of pocket and seeing a physician increased the likelihood of receiving a DSME referral (P ≤ .01). Lastly, having more DSME program sites available in the community was associated with an increased likelihood of receiving a DSME referral (P ≤ .01).

Outpatient encounters. All relationships observed among all encounters held for outpatient encounters. Exhibiting any type of need during outpatient encounters increased the likelihood of receiving a DSME referral (P ≤ .01). Additionally, encounters with older and sicker patients were associated with a decreased likelihood of resulting in a DSME referral at the time of the encounter (P ≤ .01). Lastly, self-pay encounters, physician encounters, and encounters occurring when more DSME programs were available in the community were more likely to result in a DSME referral (P ≤ .01).

Hospital-based encounters.Among hospital-based encounters, a new type 2 diabetes diagnosis, having off-target levels, the onset of complicating factors, transitions in care and insurance coverage, and multiple types of need all increased the likelihood of being referred to DSME (P ≤ .01). Age remained associated with a decreased likelihood of referral (P ≤ .01), but there was no significant relationship between comorbidity and referral. Encounters covered by Medicare (P ≤ .05) and paid out of pocket (P ≤ .01) were associated with an increased likelihood of referral. Physician encounters and encounters occurring when more DSME programs were available in the community also remained more likely to result in a DSME referral (P ≤ .01).

Subanalysis Results: Physician Encounters

Of the 93,986 encounters with physicians, 29% resulted in referrals. Findings from the subanalysis were mostly consistent with the primary models (Table 4). With respect to indicators of need, all types of evaluated need remained positively associated with receiving a referral during all encounters and outpatient encounters (P ≤ .01). Only having off-target levels, the onset of complicating factors, and multiple types of need significantly increased the likelihood of receiving a referral during hospital-based encounters. Also consistent with the previous models, encounters with older patients were consistently less likely to result in a DSME referral (P ≤ .01) and encounters paid out of pocket were consistently more likely to result in a DSME referral (P ≤ .01), irrespective of encounter setting. The relationship between the likelihood of referral and other factors varied by setting.

DISCUSSION

Clinical guidelines for DSME recommend that providers refer patients to the service when indicators of need are present.³ Our findings indicate that although most encounters indicated some type of need for DSME and being in need increases the likelihood of being referred, only a small proportion of encounters with indicators of need actually result in a referral. Additionally, the relationship between being in need and the likelihood of referral was consistent across encounter settings, although the increase was generally higher among outpatient encounters. Moreover, the effect size of need on referral is relatively small (regardless of setting), indicating that providers’ decisions to refer are indeed driven by factors other than evaluated need.¹⁸ The lack of referrals for those indicated by clinical guidelines is not surprising given the documented poor translation of evidence-based guidelines into practice.^29,30

Having insurance does not appear to improve the likelihood of being referred to DSME, even in a state with a favorable policy environment for DSME reimbursement. In fact, patients who were likely to pay for DSME out of pocket were more likely to be referred. This contradicts previous research that suggests that out-of-pocket costs deter patients from participating in DSME.^9,14,17,31 It is possible that the higher likelihood of referrals for self-pay patients in this population was due to the presence of the public health department’s accredited program, which offers DSME free of charge to patients with diabetes in the county. Fortunately, our findings suggest that having more program sites is associated with a greater likelihood of referral. It is also possible that providers were more likely to refer self-pay patients to DSME due to the heightened importance of self-management when financial barriers limit access to health care.

Seeing a physician during an encounter increased the likelihood of being referred to DSME, not a surprising finding given that the state limits reimbursable DSME to claims resulting from physician referrals. Interestingly, although most indicators of patient need for DSME did improve the likelihood of referral among physician encounters, that increase was often smaller than what we observed in the primary models, which included all provider encounters. These findings could be due to the limited contact time during physician-patient visits,^32-34 which restricts physicians’ ability to thoroughly review all clinical information in patients’ health records (which are not often organized to support effective decision-making),^35,36 as well as to consult clinical guidelines and identify available diabetes management resources in the area. It is possible that the expansion of EHR capabilities to include clinical decision support could improve provider referrals to DSME.^36-39 Of note, a new diagnosis and transitions in care were not associated with referrals during physician encounters in hospital-based settings. This could also be attributed to poor clinical decision support.

Limitations

Data limitations could have potentially affected our findings. First, although the HIE database captures patient data across health systems, referrals and provider notes regarding referrals were unavailable outside the health system. As patients move and/or change insurance for any reason, they may have to change provider networks. However, different health systems often use different EHR systems, which may or may not be compatible with one another. It is possible that referrals were issued by other providers outside the health system but not captured in the HIE database. Next, the county health department offers a free course through its accredited DSME program. Because there is no claim or reimbursement, a provider referral is not required. To account for this, we included provider notes that mention recommending DSME to the patient as a measure of “referrals,” rather than limiting our analysis to orders and billing. Next, our modeling strategy linked referrals to a single encounter by date. However, it was possible that the referral was the attending provider’s response to clinical indicators noted at multiple encounters. Moreover, the same patient was often referred to DSME during multiple encounters in the year. These could result in overstated findings. Finally, we had limited data on the individual providers, such as years in practice and training in diabetes management. Therefore, we were unable to conduct an in-depth analysis of provider referral behaviors and patterns.

CONCLUSIONS

Consistent with clinical guidelines, patient need for DSME does improve the likelihood of being referred to the service. However, provider referral rates were low despite clinical indicators of need in patients’ EHRs. Future research should explore providers’ EHR capabilities and determine whether clinical decision support improves provider referrals among patients in need. Moreover, to facilitate targeted outreach, providers must have the tools necessary to identify patients in need of DSME who have been referred but have yet to enroll. Therefore, future research should explore ways in which EHR systems can identify patients in need of DSME who have not enrolled.

Author Affiliations: Department of Population Health, Dell Medical School, University of Texas at Austin (BLB-P), Austin, TX; Department of Health Policy and Administration, Pennsylvania State University (YS), State College, PA; Department of Health Policy and Management, Indiana University Richard M. Fairbanks School of Public Health (JRV), Indianapolis, IN.

Source of Funding: This project was supported by grant number R36HS026352 from the Agency for Healthcare Research and Quality. The content is solely the responsibility of the authors and does not necessarily represent the official views of the Agency for Healthcare Research and Quality.

Author Disclosures: Dr Brown-Podgorski received an R36 dissertation grant from the Agency for Healthcare Research and Quality. Dr Vest provided consulting to the Indiana Health Information Exchange and the NY eHealth Collaborative. Dr Shi reports no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article.

Authorship Information: Concept and design (BLB-P, YS, JRV); acquisition of data (BLB-P, JRV); analysis and interpretation of data (BLB-P, YS); drafting of the manuscript (BLB-P); critical revision of the manuscript for important intellectual content (YS, JRV); statistical analysis (BLB-P, JRV); obtaining funding (BLB-P, JRV); administrative, technical, or logistic support (JRV); and supervision (JRV).

Address Correspondence to: Brittany L. Brown-Podgorski, PhD, MPH, Department of Population Health, Dell Medical School, University of Texas at Austin, 1601 Trinity St, Bldg B, Austin, TX 78712. Email: brb239@pitt.edu.

REFERENCES

1. Powers MA, Bardsley J, Cypress M, et al. Diabetes self-management education and support in type 2 diabetes. Diabetes Educ. 2017;43(1):40-53. doi:10.1177/0145721716689694

2. American Diabetes Association. Standards of Medical Care in Diabetes—2018. Diabetes Care. 2018;41(suppl 1):S1-S159.

3. Beck J, Greenwood DA, Blanton L, et al; 2017 Standards Revision Task Force. 2017 National Standards for Diabetes Self-Management Education and Support. Diabetes Educ. 2018;44(1):35-50. doi:10.1177/0145721718754797

4. Norris SL, Engelgau MM, Narayan KV. Effectiveness of self-management training in type 2 diabetes: a systematic review of randomized controlled trials. Diabetes Care. 2001;24(3):561-587. doi:10.2337/diacare.24.3.561

5. Johnson TM, Richards J, Churilla JR. Care utilization patterns and diabetes self-management education duration. Diabetes Spectr. 2015;28(3):193-200. doi:10.2337/diaspect.28.3.193

6. Robbins JM, Thatcher GE, Webb DA, Valdmanis VG. Nutritionist visits, diabetes classes, and hospitalization rates and charges: the Urban Diabetes Study. Diabetes Care. 2008;31(4):655-660. doi:10.2337/dc07-1871

7. Powell MP, Glover SH, Probst JC, Laditka SB. Barriers associated with the delivery of Medicare-reimbursed diabetes self-management education. Diabetes Educ. 2005;31(6):890-899. doi:10.1177/0145721705283039

8. Duncan I, Birkmeyer C, Coughlin S, Li QE, Sherr D, Boren S. Assessing the value of diabetes education. Diabetes Educ. 2009;35(5):752-760. doi:10.1177/0145721709343609

9. Shaw K, Killeen M, Sullivan E, Bowman P. Disparities in diabetes self-management education for uninsured and underinsured adults. Diabetes Educ. 2011;37(6):813-819. doi:10.1177/0145721711424618

10. Li R, Shrestha SS, Lipman R, Burrows NR, Kolb LE, Rutledge S; CDC. Diabetes self-management education and training among privately insured persons with newly diagnosed diabetes—United States, 2011-2012. MMWR Morb Mortal Wkly Rep. 2014;63(46):1045-1049.

11. Kolb LE, Lipman RD. Under-utilization of diabetes self-management education. AADE Pract. 2015;3(4):14-17. doi:10.1177/2325160315576338

12. Strawbridge LM, Lloyd JT, Meadow A, Riley GF, Howell BL. Use of Medicare’s diabetes self-management training benefit. Heal Educ Behav. 2015;42(4):530-538. doi:10.1177/1090198114566271

13. Diabetes: objectives. HealthyPeople.gov. Accessed February 4, 2019. https://www.healthypeople.gov/2020/topics-objectives/topic/diabetes/objectives

14. Peyrot M, Rubin RR, Funnell MM, Siminerio LM. Access to diabetes self-management education: results of national surveys of patients, educators, and physicians. Diabetes Educ. 2009;35(2):246-263. doi:10.1177/0145721708329546

15. Sarkar U, Piette JD, Gonzales R, et al. Preferences for self-management support: findings from a survey of diabetes patients in safety-net health systems. Patient Educ Couns. 2008;70(1):102-110. doi:10.1016/j.pec.2007.09.008

16. Kent D, D’Eramo Melkus G, Stuart PMW, et al. Reducing the risks of diabetes complications through diabetes self-management education and support. Popul Health Manag. 2013;16(2):74-81. doi:10.1089/pop.2012.0020

17. Peyrot M, Rubin RR. Access to diabetes self-management education. Diabetes Educ. 2008;34(1):90-97. doi:10.1177/0145721707312399

18. Mehta S, Mocarski M, Wisniewski T, Gillespie K, Narayan KMV, Lang K. Primary care physicians’ utilization of type 2 diabetes screening guidelines and referrals to behavioral interventions: a survey-linked retrospective study. BMJ Open Diabetes Res Care. 2017;5(1):e000406. doi:10.1136/bmjdrc-2017-000406

19. Health insurance coverage laws for diabetes self-management education and training. The Policy Surveillance Program. Updated through May 1, 2017. Accessed February 4, 2019. https://www.lawatlas.org/datasets/diabetes-self-management-education-laws

20. About us. Indiana Health Information Exchange. Accessed August 8, 2020. https://www.ihie.org/about-us/

21. Institute of Medicine. America’s Health Care Safety Net: Intact but Endangered. The National Academies Press; 2000.

22. Vest JR, Grannis SJ, Haut DP, Halverson PK, Menachemi N. Using structured and unstructured data to identify patients’ need for services that address the social determinants of health. Int J Med Inform. 2017;107:101-106. doi:10.1016/j.ijmedinf.2017.09.008

23. Chandrasekaran V, He J, Chase MR, Bhandari A, Frederick C, Dexter P. Extracting information from electronic health records using natural language processing – knowledge discovery from unstructured information. Presented at: Machine Learning in Healthcare Conference; August 19, 2017; Boston, MA.

24. Imler TD, Ring N, Crabb DW. Medical, social, and legal risks to predict alcoholic liver disease using natural language processing and advanced analytics. Gastroenterology. 2015;148(4 suppl 1):S499. doi:10.1016/S0016-5085(15)31674-7

25. Kudyakov R, Bowen J, Ewen E, et al. Electronic health record use to classify patients with newly diagnosed versus preexisting type 2 diabetes: infrastructure for comparative effectiveness research and population health management. Popul Health Manag. 2012;15(1):3-11. doi:10.1089/pop.2010.0084

26. Shortell SM, Anderson OW. The physician referral process: a theoretical perspective. Health Serv Res. 1971;6(1):39-48.

27. Aday LA, Andersen R. A framework for the study of access to medical care. Health Serv Res. 1974;9(3):208-220.

28. Elixhauser A, Steiner C, Harris DR, Coffey RM. Comorbidity Measures for Use with Administrative Data. Med Care. 1998;36(1):8-27. doi:10.1097/00005650-199801000-00004

29. Cabana MD, Rand CS, Powe NR, et al. Why don’t physicians follow clinical practice guidelines? JAMA. 1999;282(15):1458-1465. doi:10.1001/jama.282.15.1458

30. Grol R, Grimshaw J. From best evidence to best practice: effective implementation of change in patients’ care. Lancet. 2003;362(9391):1225-1230. doi:10.1016/S0140-6736(03)14546-1

31. Nam S, Chesla C, Stotts NA, Kroon L, Janson SL. Barriers to diabetes management: patient and provider factors. Diabetes Res Clin Pract. 2011;93(1):1-9. doi:10.1016/j.diabres.2011.02.002

32. Rui P, Okeyode T. National Ambulatory Medical Care Survey: 2016 national summary tables. CDC. Accessed May 16, 2019. https://www.cdc.gov/nchs/data/ahcd/namcs_summary/2016_namcs_web_tables.pdf

33. Gottschalk A, Flocke SA. Time spent in face-to-face patient care and work outside the examination room. Ann Fam Med. 2005;3(6):488-493. doi:10.1370/afm.404

34. Mechanic D, McAlpine DD, Rosenthal M. Are patients’ office visits with physicians getting shorter? N Engl J Med. 2001;344(3):198-204. doi:10.1056/NEJM200101183440307

35. Sittig DF, Wright A, Osheroff JA, et al. Grand challenges in clinical decision support. J Biomed Inform. 2008;41(2):387-392. doi:10.1016/J.JBI.2007.09.003

36. Koopman RJ, Kochendorfer KM, Moore JL, et al. A diabetes dashboard and physician efficiency and accuracy in accessing data needed for high-quality diabetes care. Ann Fam Med. 2011;9(5):398-405. doi:10.1370/afm.1286

37. Bright TJ, Wong A, Dhurjati R, et al. Effect of clinical decision-support systems: a systematic review. Ann Intern Med. 2012;157(1):29-43. doi:10.7326/0003-4819-157-1-201207030-00450

38. O’Connor PJ, Sperl-Hillen JM, Rush WA, et al. Impact of electronic health record clinical decision support on diabetes care: a randomized trial. Ann Fam Med. 2011;9(1):12-21. doi:10.1370/afm.1196

39. Cebul RD, Love TE, Jain AK, Hebert CJ. Electronic health records and quality of diabetes care. N Engl J Med. 2011;365(9):825-833. doi:10.1056/NEJMsa1102519