Dosage and Outcomes in a Complex Care Intervention

ABSTRACT

Objectives: The purpose of our study was to assess the relationship between intervention dosage and hospital utilization outcomes among patients with complex health and social needs enrolled in a care management program. We highlight the importance of measuring patient engagement and intervention dosage in program evaluation.

Study Design: We performed a secondary analysis of data collected between 2014 and 2018 as part of a randomized controlled trial of the Camden Coalition’s signature care management intervention. Our analytical sample consisted of 393 patients.

Methods: We calculated a time-invariant cumulative dosage rank based on the number of hours spent by care teams working with or on behalf of patients and then divided patients into low- and high-dosage groups. We applied propensity score reweighting to compare hospital utilization outcomes between patients in these 2 groups.

Results: Compared with patients in the low-dosage group, those in the high-dosage group had a lower readmission rate at 30 (21.6% vs 36.6%; P < .001) and 90 (41.7% vs 55.2%; P = .003) days post enrollment. The difference between the 2 groups at 180 days post enrollment was not statistically significant (57.5% vs 64.9%; P = .150).

Conclusions: Our study elucidates a gap in how care management programs for patients with complex health and social needs are evaluated. Although the study shows an association between intervention dosage and care management outcomes, patients’ medical complexity and social circumstances are among the factors that can attenuate the dose-response relationship over time.

Am J Manag Care. 2023;29(6):293-298. https://doi.org/10.37765/ajmc.2023.89370

Takeaway Points

We observed an association between intervention dosage and hospitalization rates after intervention enrollment. This relationship was attenuated over time.

• In care management programs for individuals with complex needs, more intense engagement early in the intervention may help stabilize patients in the community, potentially averting hospital readmissions in the weeks following a hospital discharge.
• Stable staffing structures and reliable funding sources to support them are necessary to sustain the effort required to effectively engage patients with complex needs over time.
• Evaluations of care management programs should prioritize measurement of essential aspects of engagement, including care continuity and trust.

Overlapping medical, behavioral health, and social needs such as unstable housing, food insecurity, and lack of transportation can lead to high rates of potentially avoidable hospital use.^1-3 Given that inpatient stays are significant drivers of health care costs and readmission rates are considered indicators of quality, the success of interventions serving individuals with complex health and social needs, referred to as complex care programs, is often measured in terms of changes in hospital use.^4-8 However, studies of the effect of complex care programs on hospital use patterns have yielded inconsistent results—and the programs themselves vary widely in their approaches, patient populations, and settings.^9-12

To continue building the evidence base for what constitutes a successful complex care program, a more comprehensive understanding of what works, for whom, and how is required.¹³ The dose-response concept—a metric common in medical research but rarely used in assessments of health interventions that aim for behavior change—can help us understand the impact that a complex care program can have.^14,15 Intervention dosage has been operationalized in programs as a single one or some combination of the following elements: duration (ie, the length of time of an intervention), frequency (ie, the number of contacts made within the intervention over a set period), amount (ie, the length of contact between program staff and participants), and breadth (ie, the number and types of intervention elements delivered).^14-18 Using dosage metrics can help make the case for increased programmatic resources over longer periods of time, elucidate the effect of direct contact through face-to-face and telephonic meetings, and support changes to staffing models.^19-21

There is evidence that intervention dosage is associated with health outcomes in care and case management programs. Research has found intervention dosage to be related to positive changes in hemoglobin A_1c levels, birth weight and gestational age outcomes, and access to community resources to address social needs.^18-22 Other evidence suggests that the relationship between dosage and outcomes is not linear or unidirectional. For example, a randomized controlled trial of a set of case management models for substance use disorder treatment showed worse family problems and legal status outcomes with a higher intervention dose, which the authors stated was related to more complex patients requiring more support.¹⁴

Factors influencing intervention dosage include program design, patient engagement, operational capacity, and the quality of the relationship between care team members and patients. Patient engagement, related to a person’s ability and motivation to actively participate in treatment or care, plays a critical role in whether an intervention is delivered as intended.^23-25 As a component of engagement, activation pertains to the knowledge, skills, and confidence needed to manage one’s own health and health care.²³ Patients who feel that they can exert influence over their health are more willing and able to take independent actions to manage their health care.^26,27 Motivation is another component of engagement: Patients motivated by personal goals are easier to engage and more likely to adhere to a treatment plan, and a lack of motivation has been identified as a reason for a patient not succeeding in a care management program.^23,28,29

Although there are patients who enter care with a high level of activation and motivation, by design, complex care programs try to strengthen these qualities by focusing on the elements of health improvement that are most meaningful to the patient.^23,25 This fundamental task of identifying patient goals allows complex care team members to foster patient engagement through interactions aimed at building authentic, trusting relationships. Many care managers view fostering a patient’s motivation to engage in care via interpersonal relationships as the most important—and most time-consuming—aspect of their work.²³ In turn, complex care patients report that they are motivated to engage because of high levels of interaction and rapport with care team members, nonjudgmental and caring dispositions among team members, and the sense of security and support gained from care team members’ willingness to meet them in their homes or other comfortable community spaces rather than in a health care setting.^25,29-31 Thus, patient engagement and staff effort are intertwined influencers of intervention dosage.

Patients are likely to receive a low intervention dose when staff cannot or do not consistently engage them or prioritize what is meaningful to them, when patients’ life events interfere with engagement no matter the level of staff effort, or when a mix of staff, operational, and patient characteristics prevent the intervention from being delivered as intended. In contrast, complex care interventions that can be delivered as intended may accomplish desired results when high intervention dosage is achieved through an effective program model that fosters engagement and patient goal attainment. Our aim is to contribute knowledge relevant to the operation and evaluation of care management programs by assessing dosage-outcome associations in a single care management program for patients with complex health and social needs.

METHODS

Study Design

Our results are based on a post hoc secondary analysis of a 2-armed randomized controlled trial conducted between June 2014 and September 2017. Trial data were collected through October 2018. That research showed no impact of the Camden Coalition’s signature care management intervention on hospital readmissions 180 days after study enrollment. A total of 800 residents of Camden, New Jersey, and surrounding zip codes who met the following criteria were eligible for the intervention and participated in the trial: 2 or more hospitalizations in a 6-month period, 2 or more chronic conditions, and evidence of unmet social needs. The 400 patients in the intervention arm of the trial received intensive care management services as part of the intervention; the 400 individuals in the standard care arm received standard discharge planning from the hospital with no follow-up from Camden Coalition care teams. The published trial report provides full details of study design and outcomes.³² The parent study was approved by the trial researchers’ institutional review board and the institutional review boards of the 3 hospitals from which patients were discharged. All study participants provided written informed consent. This secondary analysis was covered under the original institutional review board approvals because it did not involve additional contact or data collection from or about participants.

Study Setting

Enrollment staff met eligible patients at hospital bedside to invite them to participate in the trial. Patients who agreed to participate completed a preenrollment questionnaire and were subsequently randomly assigned into the control or treatment arm of the trial. For those randomly assigned into the treatment arm, the intervention began with patient-centered care planning and goal setting prior to hospital discharge. Upon discharge, patients began working with an interprofessional care team, which included a registered nurse, a social worker, and community health workers. Operational milestones included meeting patients at home or in the community within 5 days of discharge, scheduling follow-up appointments with primary or specialty care within 7 days of discharge, meeting patients at home or elsewhere in the community at least once a week throughout the intervention, and helping navigate clinical and social services as needed. Patients graduated when they reached their self-defined goals and exhibited stable health status, typically around 4 months after enrollment. Patient status was deemed incomplete if they were lost to follow-up, died, relocated out of the city, or declined further services.

A total of 393 patients enrolled in the intervention arm of the clinical trial were included in the current study. Seven patients who could not be identified in all-payer hospital claims data were excluded. Patients who completed the intervention, were lost to follow-up, died within the study time frame, or exited the intervention before graduation for some other reason were included in the analysis.

The data sources for the clinical trial and for the present study were all-payer claims data from 4 regional hospital systems, the Camden Coalition Health Information Exchange, information from a semistructured preenrollment survey completed by the patient at the hospital upon agreement to participate in the clinical trial, and a care coordination database. The integrated claims data covered 6 months prior to and 6 months post enrollment for each patient. These data were at the hospital utilization level and included admission and discharge dates along with International Classification of Diseases, Ninth Revision and Tenth Revision diagnostic codes. The preenrollment survey provided socioeconomic information, and the health information exchange added additional demographic information and a record of each study participant’s index admission. Care coordination encounter records included information from in-person meetings and phone calls with patients as well as clinical and social needs coordination activities carried out by care team staff on behalf of the patients.

Statistical Analysis

The primary objective of this study was to investigate the relationship between intervention dosage, measured in time spent by care team staff working directly with or on behalf of patients, and postenrollment hospitalizations after adjusting for other factors that could be associated with the outcomes. Because hospitalization rates and intervention dosage are functions of time length, we created a relative dosage rank measure that is invariant with respect to time and then categorized patients into high- and low-dosage groups. The details of our method for calculating dosage are in eAppendix A (eAppendices available at ajmc.com).

We used propensity score reweighting to compare hospital utilization outcomes between the 2 dosage groups. To predict dosage group, we applied a gradient-boosting machine-learning model, which is a tree-based classification algorithm.^33,34 Twenty-nine variables covering demographic, social, and clinical measures were used as independent variables in the model to predict dosage group. Variables were treated as continuous unless they were intrinsically categorical. The complete list of variables included in the model is in eAppendix Table 1.

We then used weighted logistic regression models to estimate 30-, 90-, and 180-day hospitalization rates with dosage group as the treatment variable and other covariates as controlling variables. The weights used in the logistic regression models were the inverse propensity score weights; the statistical significance level was set at α = .05.

RESULTS

Table 1 displays the characteristics of the full, unadjusted cohort by dosage group. The study sample consisted of 259 patients in the high-dosage group and 134 patients in the low-dosage group. As shown in Table 1, the high- and low-dosage groups were similar across many of these characteristics, but significant differences were observed for education, social complexity, and the mean number of days spent in the hospital during the admission at which patients consented to participate in the trial (ie, the index admission). After reweighting, the differences between the 2 groups on these characteristics were reduced but remained statistically significant for education (eAppendix Table 2).

Table 2 shows the mean number of attempts and successful engagements for patients in the high- and low-dosage groups along with mean dosage hours. An engagement attempt was coded successful if the patient participated in the encounter either in person or on the phone and there were documented staff hours for that encounter. If the patient was not present for a scheduled visit, call, or appointment (ie, a no-show), an engagement attempt was considered unsuccessful. Over the first intervention week, the mean number of attempted engagements was 27% higher for high-dosage vs low-dosage group patients, and successful engagements were 66% higher. The mean number of engagement attempts dropped for the full cohort during the second intervention week compared with the first, reflecting intentionally greater effort during week 1, and this drop was sharper for patients in the low- vs high-dosage group (51% vs 39%). That patients who ultimately experienced a lower dose of the intervention received a lower number of engagement attempts suggests some influence of staffing features on dosage, and the lower proportion of successful engagements for this group may point to the influence of patient characteristics and/or circumstances. The number of dosage hours was twice as high for patients in the high- vs low-dosage group throughout the intervention, beginning in week 1.

Table 3 presents the unadjusted and reweighted associations between dosage and hospitalization outcomes. Among the 259 patients in the high-dosage group, 21.6% had a hospitalization within 30 days of intervention enrollment vs 36.6% among patients in the low-dosage group (absolute difference, –15.0; 95% CI, –24.5 to –5.4; P = .003). At 90 days after enrollment, 41.7% of patients in the high-dosage group had a hospitalization compared with 55.2% among those in the low-dosage group (absolute difference, –13.5; 95% CI, –23.9 to –3.2; P = .012). The higher hospitalization rate among low-dosage group patients compared with high-dosage group patients was not significantly different at 180 days post enrollment (absolute difference, –7.4; 95% CI, –17.5 to 2.7; P = .150). The results based on reweighted data are similar.

DISCUSSION

In this study, patients with complex health and social needs who received a relatively higher dose of a care management intervention had lower postenrollment hospitalization rates compared with patients who received a lower dose, a finding that aligns with those of other studies showing differences in outcomes based on intervention dosage level.^18-20 Because the dosage-outcome relationship that we observed was attenuated over time, our results suggest that higher dosage early in the intervention may help avert hospitalizations as care teams work to stabilize patients in the community, but the medical requirements of long-term chronic diseases, often combined with social complexity and shortcomings within the health care system, can overtake gains made through care management when it comes to utilization over a longer period. Other research has shown a weakening over time of the relationship between dosage and intervention outcomes.³⁵

Neither patient motivation nor activation was measured in this study, and these attributes may have played a role in both dosage received and hospital utilization outcomes. Participants in the intervention were enrolled at the hospital bedside because of the belief that hospitalization is a catalytic moment—a pause in day-to-day responsibilities and distractions that allows individuals to reflect on their health and personal goals. However, assessing motivation and ability to engage in care management at that moment is complicated. Many patients are discharged from the hospital and remain engaged. For others, the return to normal life brings challenges that demand attention, which may weaken their ability to stay engaged.³⁵ As a result, evaluation of complex care programs should prioritize measurement of essential aspects of engagement, including care continuity and trust.⁹ An assessment tool to measure intervention readiness or patient self-efficacy, such as the Patient Activation Measure,²⁶ could help programs determine which patients may require more effort early in the intervention as well as help evaluators tease out the effect of intervention dosage vs patient disposition and life circumstances on outcomes. We caution that such measures should not be used to exclude patients from the care that they need but rather as tools to guide intervention strategy and strengthen program evaluation. This is an especially important point considering the systemic factors, such as poverty and racism, that affect access to and engagement in care.^36-38

Our analysis further suggests that the amount of staff effort needed to stabilize patients in the community after a hospitalization is likely to vary based on patient characteristics and circumstances. Indeed, other research has shown a positive relationship between the social complexity of patients in care management programs and staff effort.^14,21 The reality of staffing an intensive, person-centered model coupled with this new understanding that a higher dose of the intervention is associated with positive outcomes creates an image of the volume of effort needed to make even short-term changes in outcomes for patients with complex health and social needs. Stable staffing structures in high-touch care management programs and reliable funding sources to support them are necessary to sustain the effort required to effectively engage patients over time.

Although many complex care programs are evaluated on how well they reduce utilization, programmatic goals often focus on self-efficacy, connection to benefits and services, and stability. This means that evaluations focusing solely on health care utilization and costs can leave out significant insight into the impact these programs can and do have on quality of life, self-esteem, healing from psychological trauma, and human connection. This is an especially critical consideration for evaluations of complex care programs that serve patients in disadvantaged communities whose experiences of racism and other forms of structural disadvantage may lead to distrust of institutions, especially health care.³⁶

Limitations

The main limitation of this study is that we focused on only 1 arm of the randomized controlled trial; therefore, it is not possible to claim a causal relationship between intervention dose and readmission outcomes. Although we adjusted for possible differences among patients in the high- and low-dosage groups using propensity score reweighting, there are unobserved, confounding characteristics that may have affected our results. We deprioritized a reanalysis of the intervention’s treatment effect because our intent was to directly assess the relationship between dosage received and outcomes for quality improvement purposes. Analyses aimed at providing causal estimates based on dosage status would benefit from the application of an instrumental variable method to take advantage of the randomized data.

CONCLUSIONS

This article contributes knowledge regarding the application of the dose-response concept to the evaluation of health interventions that aim for behavior change. Although the study provides evidence of a relationship between intervention dosage and outcomes, patients’ medical complexity and social circumstances are among the factors that may attenuate this relationship over time. Findings from this study highlight the importance of robust, high-touch interventions for patients with complex needs and illuminate a gap in how these interventions are evaluated. Assessing patient motivation and activation and identifying programmatic and systemic barriers to delivering an intervention as intended are crucial and yet rarely included in evaluation protocols. To deepen our understanding of how complex care interventions can affect patient health and well-being, we must take these factors into consideration when designing, implementing, and evaluating care management programs.

Acknowledgments

The authors would like to acknowledge Aaron Truchil, Evelyne Kane, Jesse Gubb, Kathleen Noonan, Kelly Craig, Kenneth Coburn, Mary Naylor, and Meredith Matone for their critical review of an earlier draft of this manuscript and Lisa Miller for her insight and editorial suggestions. They would also like to recognize the Camden Coalition care teams for their dedication to the patients they serve.

Author Affiliations: Camden Coalition (DW, QY, TK, MA-F), Camden, NJ; now with Phreesia (TK), Wilmington, DE; now with Rutgers University-Camden Senator Walter Rand Institute for Public Affairs (MA-F), Camden, NJ.

Source of Funding: None.

Author Disclosures: Ms Kuruna is now employed by Phreesia, which acquired Insignia Health in 2021, which has the exclusive rights to the Patient Activation Measure. The remaining authors report no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article.

Authorship Information: Concept and design (DW, QY); acquisition of data (QY); analysis and interpretation of data (DW, QY); drafting of the manuscript (DW, QY, TK, MA-F); critical revision of the manuscript for important intellectual content (DW, QY, TK, MA-F); statistical analysis (QY); administrative, technical, or logistic support (TK, MA-F); and supervision (DW).

Address Correspondence to: Dawn Wiest, PhD, Camden Coalition, 800 Cooper St, 7th Floor, Camden, NJ 08102. Email: dwiest@camdenhealth.org.

REFERENCES

1. Humowiecki M, Kuruna T, Sax R, et al. Blueprint for complex care. Camden Coalition National Center for Complex Health and Social Needs. Updated March 2019. Accessed June 15, 2021. https://www.nationalcomplex.care/wp-content/uploads/2019/03/Blueprint-for-Complex-Care_UPDATED_030119.pdf

2. Schulman M, Thomas-Henkel C. Opportunities for complex care programs to address the social determinants of health. Center for Health Care Strategies. February 2019. Accessed June 15, 2021. https://www.chcs.org/media/TCC-SDOH-022119.pdf

3. Hasselman D. Innovative complex care management programs: common themes from the super-utilizer summit. Center for Health Care Strategies. October 2013. Accessed May 18, 2020. https://www.chcs.org/resource/super-utilizer-summit-common-themes-from-innovative-complex-care-management-programs/

4. Liang L, Moore B, Soni A. National inpatient hospital costs: the most expensive conditions by payer, 2017. Healthcare Cost and Utilization Project statistical brief No. 261. July 2020. Accessed May 18, 2021.
https://www.ncbi.nlm.nih.gov/books/NBK561141/

5. Fischer C, Lingsma HF, Marang-van de Mheen PJ, Kringos DS, Klazinga NS, Steyerberg EW. Is the readmission rate a valid quality indicator? a review of the evidence. PloS One. 2014;9(11):e112282. doi:10.1371/journal.pone.0112282

6. Benbassat J, Taragin M. Hospital readmissions as a measure of quality of health care: advantages and limitations. Arch Intern Med. 2000;160(8):1074-1081. doi:10.1001/archinte.160.8.1074

7. Ness D, Kramer W. Reducing hospital readmissions: it’s about improving patient care. Health Affairs. August 16, 2013. Accessed April 15, 2019. https://www.healthaffairs.org/do/10.1377/hblog20130816.033808/full/

8. Janson IA, Foster TL, Goodman MR. An outpatient critical care transition clinic model reduces admissions/readmissions in medically complex patients. Am J Manag Care. 2021;27(9):e301-e307. doi:10.37765/ajmc.2021.88742

9. Williams BC, Fendrick AM. Tailoring complex care to patients’ needs: myths, realities, and best next steps. Am J Manag Care. 2022;28(2):47-50. doi:10.37765/ajmc.2022.88750

10. Thapa BB, Li X, Galárraga O. Impacts of community-based care program on health care utilization and cost. Am J Manag Care. 2022;28(4):187-191. doi:10.37765/ajmc.2022.88862

11. Powers BW, Modarai F, Palakodeti S, et al. Impact of complex care management on spending and utilization for high-need, high-cost Medicaid patients. Am J Manag Care. 2020;26(2):e57-e63. doi:10.37765/ajmc.2020.42402

12. Komaromy M, Bartlett J, Gonzales-van Horn SR, et al. A novel intervention for high-need, high-cost Medicaid patients: a study of ECHO Care. J Gen Intern Med. 2020;35(1):21-27. doi:10.1007/s11606-019-05206-0

13. Pourat N, Chen X, Tsugawa Y, et al. Intersection of complexity and high utilization among health center patients aged 18 to 64 years. Am J Manag Care. 2022;28(2):66-72. doi:10.37765/ajmc.2022.88751

14. Huber DL, Sarrazin MV, Vaughn T, Hall JA. Evaluating the impact of case management dosage. Nurs Res. 2003;52(5):276-288. doi:10.1097/00006199-200309000-00002

15. Rowbotham S, Conte K, Hawe P. Variation in the operationalisation of dose in implementation of health promotion interventions: insights and recommendations from a scoping review. Implement Sci. 2019;14(1):56. doi:10.1186/s13012-019-0899-x

16. Voils CI, Chang Y, Crandell J, Leeman J, Sandelowski M, Maciejewski ML. Informing the dosing of interventions in randomized trials. Contemp Clin Trials. 2012;33(6):1225-1230. doi:10.1016/j.cct.2012.07.011

17. Huber DL, Hall JA, Vaughn T. Dose of case management interventions. Lippincotts Case Manag. 2001;6(3):119-126. doi:10.1097/00129234-200105000-00006

18. Dorr DA, Wilcox A, Jones S, Burns L, Donnelly SM, Brunker CP. Care management dosage. J Gen Intern Med. 2007;22(6):736-741. doi:10.1007/s11606-007-0138-z

19. Goyal NK, Hall ES, Meinzen-Derr JK, et al. Dosage effect of prenatal home visiting on pregnancy outcomes in at-risk, first-time mothers. Pediatrics. 2013;132(suppl 2):S118-S125. doi:10.1542/peds.2013-1021J

20. Manian N, Wagner CA, Placzek H, Darby BA, Kaiser TJ, Rog DJ. Relationship between intervention dosage and success of resource connections in a social needs intervention. Public Health. 2020;185:324-331. doi:10.1016/j.puhe.2020.05.058

21. Martinez Z, Koker E, Truchil A, Balasubramanian H. Time and effort in care coordination for patients with complex health and social needs: lessons from a community-based intervention. J Interprof Educ Pract. 2019;15:142-148. doi:10.1016/j.xjep.2019.03.002

22. Slaughter JC, Issel LM. Developing a measure of prenatal case management dosage. Matern Child Health J. 2012;16(5):1120-1130. doi:10.1007/s10995-011-0840-7

23. Dryden EM, King K, Touw S. Facilitators and barriers to successful engagement in a complex care management program: patient and care manager perspectives. J Health Care Poor Underserved. 2019;30(2):789-805. doi:10.1353/hpu.2019.0056

24. Madden EF, Kalishman S, Zurawski A, O’Sullivan P, Arora S, Komaromy M. Strategies used by interprofessional teams to counter healthcare marginalization and engage complex patients. Qual Health Res. 2020;30(7):1058-1071. doi:10.1177/1049732320909100

25. Parry C, Kramer HM, Coleman EA. A qualitative exploration of a patient-centered coaching intervention to improve care transitions in chronically ill older adults. Home Health Care Serv Q. 2006;25(3-4):39-53. doi:10.1300/J027v25n03_03

26. Hibbard JH, Stockard J, Mahoney ER, Tusler M. Development of the Patient Activation Measure (PAM): conceptualizing and measuring activation in patients and consumers. Health Serv Res. 2004;39(4 pt 1):1005-1026. doi:10.1111/j.1475-6773.2004.00269.x

27. Hibbard JH, Greene J. What the evidence shows about patient activation: better health outcomes and care experiences; fewer data on costs. Health Aff (Millwood). 2013;32(2):207-214. doi:10.1377/hlthaff.2012.1061

28. Kimerling R, Lewis ET, Javier SJ, Zulman DM. Opportunity or burden? a behavioral framework for patient engagement. Med Care. 2020;58(2):161-168. doi:10.1097/MLR.0000000000001240

29. Komaromy M, Madden EF, Zurawski A, et al. Contingent engagement: what we learn from patients with complex health problems and low socioeconomic status. Patient Educ Couns. 2018;101(3):524-531. doi:10.1016/j.pec.2017.08.019

30. Grinberg C, Hawthorne M, LaNoue M, Brenner J, Mautner D. The core of care management: the role of authentic relationships in caring for patients with frequent hospitalizations. Popul Health Manag. 2016;19(4):248-256. doi:10.1089/pop.2015.0097

31. Hudon C, Chouinard MC, Pluye P, et al. Characteristics of case management in primary care associated with positive outcomes for frequent users of health care: a systematic review. Ann Fam Med. 2019;17(5):448-458. doi:10.1370/afm.2419

32. Finkelstein A, Zhou A, Taubman S, Doyle J. Health care hotspotting—a randomized, controlled trial. N Engl J Med. 2020;382(2):152-162. doi:10.1056/NEJMsa1906848

33. Ayyadevara VK. Gradient boosting machine. In: Ayyadevara VK. Pro Machine Learning Algorithms: A Hands-On Approach to Implementing Algorithms in Python and R. Apress; 2018:117-134. doi:10.1007/978-1-4842-3564-5_6

34. Zhang Y, Haghani A. A gradient boosting method to improve travel time prediction. Transp Res Part C Emerg Technol. 2015;58(B):308-324. doi:10.1016/j.trc.2015.02.019

35. Heerman WJ, Sommer EC, Qi A, et al. Evaluating dose delivered of a behavioral intervention for childhood obesity prevention: a secondary analysis. BMC Public Health. 2020;20(1):885. doi:10.1186/s12889-020-09020-w

36. Fleming MD, Shim JK, Yen IH, et al. Patient engagement at the margins: health care providers’ assessments of engagement and the structural determinants of health in the safety-net. Soc Sci Med. 2017;183:11-18. doi:10.1016/j.socscimed.2017.04.028

37. Armstrong K, Ravenell KL, McMurphy S, Putt M. Racial/ethnic differences in physician distrust in the United States. Am J Public Health. 2007;97(7):1283-1289. doi:10.2105/AJPH.2005.080762

38. Yearby R. Racial disparities in health status and access to healthcare: the continuation of inequality in the United States due to structural racism. Am J Econ Sociol. 2018;77(3-4):1113-1152. doi:10.1111/ajes.12230