Pervasiveness and Clinical Staff Perceptions of HPV Vaccination Feedback

ABSTRACT

Objectives: This study describes the use of data-based feedback, such as human papillomavirus (HPV) vaccination rates, to advance HPV vaccination uptake in pediatric and family medicine clinics.

Study Design: A survey of primary care clinical staff in the US who provided HPV vaccination to children aged 9 to 12 years (N = 2527; response rate, 57%).

Methods: The primary outcome was a mutually exclusive categorical variable that described the type of quality metrics for which providers received feedback in the past year: HPV vaccine, other pediatric vaccinations, other quality metrics, or none. Secondary outcomes were provider perceptions of HPV vaccine feedback helpfulness and their comfort with colleagues seeing their HPV vaccination rates. Logistic models adjusted for clinical staff and clinic characteristics.

Results: Only 36.2% (n = 916) of respondents received HPV feedback. Feedback on HPV vaccination rates was more likely in nonrural clinics (OR, 2.03; 95% CI, 1.38-2.99), clinics in systems of 5 or more (OR, 1.81; 95% CI, 1.38-2.36), and in clinics serving 50 or more children per week (OR, 3.08; 95% CI, 2.03-4.66). Hispanic, Latino, or Spanish (OR, 1.54; 95% CI, 1.00-2.36) and Black or African American clinical staff (OR, 2.12; 95% CI, 1.44-3.12) were more likely than White clinical staff to find HPV vaccine feedback helpful. Relative to pediatricians, family medicine clinical staff were less comfortable with colleagues seeing their HPV vaccination rates (OR, 0.70; 95% CI, 0.57-0.87).

Conclusions: Clinical staff seldom receive feedback about HPV vaccination in primary care.

Am J Manag Care. 2024;30(11):In Press

Takeaway Points

Current research provides limited evidence of the extent to which direct provider feedback is used to improve human papillomavirus (HPV) vaccination rates. This study found that clinicians seldom receive such feedback in primary care. The results of this study contribute to managed care considerations by:

• Quantifying the reported use of HPV vaccination feedback
• Identifying the types of clinics that most lack HPV vaccination feedback
• Providing insight into how providers interact with HPV vaccination feedback

Feedback, defined as data for primary care professionals on specific quality metrics, provides an opportunity for clinical staff to track aspects of their care that can be enhanced. Studies have shown that provider feedback effectively increases care quality,^1-4 particularly when baseline metrics are low.¹

There is evidence that provider feedback can also be effective in increasing human papillomavirus (HPV) vaccination.^3,4 Despite being highly safe and effective in preventing 6 types of HPV-related cancers, HPV vaccination initiation and completion lag behind other childhood and adolescent immunizations in the US. Healthy People 2030 aims to increase HPV vaccination completion to 80% by age 15 years.⁵ The method of provider assessment and feedback also influences improvements in HPV vaccination rates. For example, although direct provider feedback has been associated with significant increases in HPV vaccination rates,^3,4 general information from health educators to providers about their patient populations did not yield such results.^6,7 Two of 6 recommended HPV programs in the National Cancer Institute’s Evidence-Based Cancer Control Programs include provider feedback.^8,9

Despite evidence that provider feedback is effective for increasing HPV vaccination,^3,4 little is known about how much feedback is used in practice. There is also a dearth of information on what types of clinics and clinical staff are likely to use feedback or how clinical staff characterize their experiences with it. This descriptive study aims to address these gaps in the literature by sharing the results of a national survey of clinical staff involved in HPV vaccination of patients aged 9 to 12 years and their reported perceptions of, experiences with, and frequency of receiving feedback.

METHODS

Survey Participants

From May through July 2022, the Improving Provider Announcement Communication Training team conducted a national, online survey of clinical staff working in primary care clinics that provided HPV vaccination to children (N = 2527; American Association for Public Opinion Research Response Rate 3, 57%). Prior to fielding the survey, items were tested for readability, comprehension, and face validity with 8 local providers in North Carolina. WebMD Market Research recruited participants through their Medscape Network. Respondents were (1) certified to practice in the US; (2) practiced as a physician, physician assistant, advanced practice nurse (including nurse practitioner), registered nurse, licensed practical/vocational nurse, medical assistant, or certified nursing assistant; (3) worked in pediatrics, family medicine, or general medicine specialties; and (4) had a role in HPV vaccination for children aged 9 to 12 years.

Outcome Variables

Respondents were provided the following guidance: “The next questions are about giving primary care professionals data about their performance on specific quality metrics. We will call this feedback.” Respondents were asked, “In the past year, which quality metrics did you receive feedback about?” and to check all that apply from the following list: “HPV vaccination rates,” “other pediatric vaccination rates,” “other pediatric quality metrics,” “other adult quality metrics,” or “none of these.” We recoded responses into a mutually exclusive categorical outcome variable: HPV vaccine, other pediatric vaccinations (but not HPV vaccine), other quality metrics (either other pediatric or other adult quality metrics but not HPV vaccine or other pediatric vaccinations), or none.

Two secondary outcomes captured clinical staff opinions about feedback on HPV vaccination rates. Respondents were asked, “How helpful would receiving feedback be for increasing your HPV vaccination rates?” We coded a binary indicator as 1 for “very” or “extremely helpful” and 0 for “not at all,” “slightly,” or “moderately helpful.” Respondents were also asked, “How comfortable would you be with your colleagues seeing your HPV vaccination rates?” We coded a binary indicator as 1 for “very” or “extremely comfortable” and 0 for “not at all,” “slightly,” or “moderately comfortable.” We recoded the response categories as described because our focus was respondents who definitively found feedback helpful or who were comfortable with colleagues seeing their HPV vaccination rates vs those who did not find feedback helpful or who were comfortable under certain conditions. The eAppendix Table (eAppendix available at ajmc.com) reports frequencies for the raw outcome categories.

Covariates

Several covariates were included in 3 categories: clinical staff experience with vaccine feedback, clinic characteristics, and respondent characteristics. Clinical staff experience with feedback was determined by whether clinical staff had received feedback on quality metrics for their region, health system, clinic, patients, and/or other patients or providers. Clinic characteristics included rurality (rural [reference] or nonrural), specialty (pediatrics [reference], family medicine, or other), system size (not in a system [reference], system of 1-4 clinics, or system of ≥ 5 clinics), practice type (solo practice [reference], group practice, hospital or academic institution, federally qualified health center or community health center, or other), number of providers (1 [reference], 2-5, 6-10, ≥ 11), percent of children using the Vaccines for Children program (< 25% [reference], 25%-49%, 50%-74%, 75%-100%, or not sure), and the number of children aged 9 to 12 years seen in a typical week (0-9 [reference], 10-24, 25-49, or ≥ 50). Covariates for respondent characteristics included medical training (physician [reference], physician assistant, advanced practice nurse, nurse, or assistant), gender (woman [reference], man, or another gender), race/ethnicity (White [reference]; Hispanic, Latino, or Spanish; Black or African American; Asian; other or prefer not to say; or multiple), and years in practice (continuous). Race/ethnicity categories representing race as a social construct were self-reported and were included to explore potential disparities in feedback receipt and attitudes.

Statistical Analysis

We used a multinomial logistic regression model for the primary outcome of feedback type that controlled for the clinic and respondent characteristics previously described. We also estimated logistic regression models for each of the secondary outcomes: helpfulness of feedback and comfort with colleagues seeing feedback. These logistic regressions controlled for respondent experience with feedback and clinic and respondent characteristics described above. We report exponentiated coefficients and 95% CIs calculated using robust SEs. All statistical analyses were performed using Stata 17.0 (StataCorp LLC). This study was approved by the University of North Carolina at Chapel Hill Institutional Review Board.

RESULTS

Feedback Experience

Only 36.2% (n = 916) of respondents reported ever receiving feedback on HPV vaccination rates (Table 1). Feedback measured at the clinic level was the most common form of feedback (eg, 37% of those receiving feedback on HPV vaccination). The distributions of all covariates, except respondent gender and race/ethnicity, were significantly different across feedback types (Table 1).

In adjusted analyses, feedback was associated with several clinic characteristics. Feedback on HPV vaccination was more likely in nonrural clinics (relative risk ratio [RRR], 2.03; 95% CI, 1.38-3.01) (Table 2). HPV vaccine feedback also varied by specialty, as family medicine clinical staff (RRR, 1.33; 95% CI, 1.02-1.74) and other specialty clinical staff (RRR, 1.91; 95% CI, 1.31-2.80) were more likely than pediatric clinical staff to receive such feedback on HPV vaccination. Relative to clinics outside a health care system, clinics in systems were more likely to receive all feedback types (eg, clinics in a system of 5 or more clinics were more likely to receive HPV vaccine feedback: RRR, 1.81; 95% CI, 1.39-2.37). More providers per facility was associated with an increased likelihood of receiving HPV vaccine feedback but not other feedback types (eg, the risk for HPV feedback compared with no feedback was higher for clinics with 11 or more providers relative to clinics with a single provider: RRR, 2.04; 95% CI, 1.21-3.44). Clinic size, as measured by the number of children aged 9 to 12 years seen in a typical week, was positively associated with the likelihood of receiving feedback for HPV vaccination and other pediatric vaccinations relative to no feedback. For example, relative to clinics that saw 0 to 9 children per week, clinics that saw 50 or more children per week were most likely to receive feedback for HPV vaccination (RRR, 3.08; 95% CI, 2.01-4.71).

The only respondent characteristic that was consistently associated with feedback type was years in practice. Clinical staff with more years in practice were less likely to receive feedback on other pediatric vaccination (RRR, 0.99; 95% CI, 0.97-1.00) and other quality metrics (RRR, 0.98; 95% CI, 0.97-1.00).

Helpfulness of HPV Vaccination Feedback

Most clinical staff did not believe that HPV vaccination feedback would be helpful, with negative responses ranging from 57% among those receiving HPV vaccination feedback to 72% for those not receiving any type of feedback (Table 1). In adjusted analyses, those receiving HPV vaccination feedback were more likely to believe HPV vaccination feedback was helpful (eg, relative to those receiving HPV vaccination feedback, those with no feedback were less likely to believe feedback was helpful: OR, 0.57; 95% CI, 0.45-0.72) (Table 3). Only 1 clinic characteristic was associated with viewing feedback as helpful: family medicine clinical staff were more likely than pediatrics staff to find HPV vaccination feedback helpful (OR, 1.37; 95% CI, 1.11-1.68). Three respondent characteristics were associated with belief of the helpfulness of HPV vaccination feedback: men were more likely than women to believe that HPV vaccination feedback would be helpful (OR, 1.42; 95% CI, 1.15-1.76), and Hispanic, Latino, or Spanish clinical staff (OR, 1.54; 95% CI, 1.00-2.36) and Black or African American clinical staff (OR, 2.12; 95% CI, 1.42-3.15) were more likely than White clinical staff to find HPV vaccination feedback helpful.

Comfort With Public HPV Vaccination Feedback

Most clinical staff felt they would be comfortable with their colleagues seeing their HPV vaccination rates, with positive responses ranging from 59% for those receiving feedback for other quality metrics to 70% among those receiving HPV vaccination feedback (Table 1). In adjusted analyses, those receiving HPV vaccination feedback continued to be comfortable with that feedback being made public to their colleagues (eg, relative to those receiving HPV vaccination feedback, those with no feedback were less likely to be comfortable with colleagues seeing their HPV vaccination rates: OR, 0.68; 95% CI, 0.53-0.85) (Table 4).

Two clinic characteristics were statistically significantly associated with higher likelihood of comfort: specialty and number of providers. Relative to pediatric clinical staff, family medicine clinical staff were less comfortable with colleagues seeing their HPV vaccination rates (OR, 0.71; 95% CI, 0.57-0.87), while respondents from clinics with 6 to 10 providers were most comfortable (OR, 1.73; 95% CI, 1.14-2.62). Several respondent characteristics were significantly associated with comfort. Compared with physicians, advanced practice nurses were the only clinical staff who were significantly more comfortable with colleagues seeing their HPV vaccine rates (OR, 1.32; 95% CI, 1.01-1.73). Asian clinical staff (OR, 0.67; 95% CI, 0.52-0.86) and those who did not identify a race/ethnicity (OR, 0.64; 95% CI, 0.44-0.93) were less comfortable with colleagues seeing their HPV vaccine rates than White respondents. Finally, more years in practice were positively associated with comfort (OR, 1.01; 95% CI, 1.01-1.02) (Table 4).

DISCUSSION

Our results indicate that although 77% of clinical staff reported receiving feedback on at least 1 quality metric, only 36% reported receiving feedback on HPV vaccination rates. The use of HPV vaccination feedback was more likely in nonrural clinics, family medicine and other specialty clinics, clinics in a health care system, and larger clinics (measured by number of providers and patients). Clinical staff were skeptical that HPV vaccination feedback would be helpful; those with experience with HPV vaccination were the least skeptical. Most clinical staff were comfortable with their HPV vaccination rates being made public to their colleagues, and those with experience receiving HPV vaccination feedback were the most likely to be comfortable. These findings suggest there is substantial opportunity to increase the use of HPV vaccination feedback and that clinical staff experience with HPV vaccination feedback may improve their opinions of it.

Existing literature supports the usefulness of HPV vaccination feedback.^3,4 HPV provider feedback is important for improving rates of initiating and completing the HPV vaccine series⁶ and for encouraging providers to routinely recommend the vaccine.¹⁰ Additionally, pairing HPV vaccination quality improvement programs with provider feedback can increase the reach of said programs.¹¹ Although proven useful, HPV vaccination feedback may be low because HPV vaccination itself is not as frequent as other routine vaccinations. In some reporting systems, HPV vaccination is combined with other adolescent vaccinations.

Best practices for administering assessments and feedback include having a supervisor or colleague as the source,^12-14 providing feedback more than once,^13-15 delivering verbal and written feedback,¹³ and including specific targets and an action plan.^13,15 Based on the theory of conditional cooperation, making feedback public to the team can advance its effect. The theory of conditional cooperation states that people are more apt to participate when they know that others are as well.¹⁶ Thus, feedback that is readily available to the team may improve individual performance. Importantly, an acknowledgement of both individual and organizational performance is an ideal environment for optimal performance improvement.¹⁷ For example, supporting quality improvement programs with performance feedback bolsters the motivational effects of team-based rewards because the feedback advances the team’s understanding of how to collaborate and progress.¹⁷ Our results indicate that a large portion of respondents did not believe that HPV vaccination feedback was extremely helpful. Experiences with ineffective feedback that did not include the best practices that we have described could provide an explanation for these perceptions.

This study’s main contribution to the literature on HPV vaccination feedback is to document the prevalence of its use among a national sample of frontline clinical staff. Our survey included all clinical staff with a role in HPV vaccination, including nurses and nursing and medical assistants. Prior literature related to HPV vaccination has typically been restricted to physicians. We also focus on the clinical staff’s perspective and knowledge of feedback in their clinics, rather than reports from administration or management. This is crucial, as feedback works only if frontline clinical staff are aware of it.

Limitations

This study has limitations to consider. Self-reports of feedback received in the past year may suffer from recall bias. Although our survey included a large, nationwide sample of clinical staff with a high response rate, it may not be fully representative of the targeted US workforce. For example, our survey had relatively low representation of rural clinics (9% of the sample). We compared demographics from our sample recruited from the Medscape panel with the Current Population Survey’s 2022 Annual Social and Economic Supplement by occupation.¹⁸ Whereas age, sex, and race are largely comparable, our sample skews slightly younger and has more women for physicians, more men for advanced practitioners and nurses, fewer White individuals for physicians and advanced practitioners, and more White individuals for nurses and nursing staff. Finally, clinical staff experience with vaccine feedback depicted the level at which clinical staff received feedback, but not the modality of feedback distribution (eg, email vs team huddle) or the quality of the feedback. The ways in which clinical staff receive feedback, and the reliability of said feedback, could indicate barriers to the effectiveness of HPV vaccination feedback. This presents an opportunity for future research to explore methods of feedback distribution and to identify the methods that are associated with improved vaccination rates.

CONCLUSIONS

This study provides a necessary addition to the literature on HPV vaccination feedback, its current prevalence, and how primary care providers interact with it. As clinics and systems implement new audit and feedback protocols and quality improvement programs to improve HPV vaccination rates, the responses elicited in this study can be applied to develop more appropriate and effective interventions. Based on our results, HPV vaccination feedback is acceptable at a more granular, or individual, level and special efforts for inclusion should be made to ensure feedback reaches rural clinics, pediatric providers, and smaller clinical settings. There also may be opportunities to tie feedback metric achievement to incentives through pay-for-performance programs to increase their impact. Future research can add to the developments in this study by assessing the modalities used and pathways through which feedback on HPV vaccination rates is disseminated, the perceived appropriateness and penetration of feedback modalities and content to clinical staff, and application of content for quality improvement planning and activities.

Author Affiliations: Department of Health Policy and Management, Gillings School of Global Public Health, University of North Carolina at Chapel Hill (JAL, JGT), Chapel Hill, NC; Lineberger Comprehensive Cancer Center, University of North Carolina at Chapel Hill (KB, JGT), Chapel Hill, NC; Department of Health Behavior, Gillings School of Public Health, University of North Carolina at Chapel Hill (TLQ), Chapel Hill, NC.

Source of Funding: Research reported in this publication was supported by the National Cancer Institute of the National Institutes of Health under award number P01CA250989. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

Author Disclosures: The authors report no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article.

Authorship Information: Concept and design (JAL, KB, JGT); acquisition of data (KB, TLQ, JGT); analysis and interpretation of data (JAL, TLQ, JGT); drafting of the manuscript (JAL, KB, TLQ, JGT); critical revision of the manuscript for important intellectual content (JAL, KB, TLQ, JGT); statistical analysis (JAL); provision of patients or study materials (KB); obtaining funding (JGT); administrative, technical, or logistic support (JAL, KB); and supervision (KB, JGT).

Address Correspondence to: Justin G. Trogdon, PhD, Department of Health Policy and Management, Gillings School of Global Public Health, University of North Carolina at Chapel Hill, 1101-B McGavran-Greenberg Hall, CB #7411, Chapel Hill, NC 27599-7411. Email: justintrogdon@unc.edu.

REFERENCES

1. Sebek KM, Virkud A, Singer J, Pulgarin CP, Schreibstein L, Wang JJ. Preliminary evaluation of a comprehensive provider feedback report. J Pract Manag. 2014;29(6):397-405.

2. Bentz CJ, Bayley KB, Bonin KE, et al. Provider feedback to improve 5A’s tobacco cessation in primary care: a cluster randomized clinical trial. Nicotine Tob Res. 2007;9(3):341-349. doi:10.1080/14622200701188828

3. Oliver K, Frawley A, Garland E. HPV vaccination: population approaches for improving rates. Hum Vaccin Immunother. 2016;12(6):1589-1593. doi:10.1080/21645515.2016.1139253

4. Rand CM, Schaffer SJ, Dhepyasuwan N, et al. Provider communication, prompts, and feedback to improve HPV vaccination rates in resident clinics. Pediatrics. 2018;141(4):e20170498. doi:10.1542/peds.2017-0498

5. HPV vaccination. Cancer Trends Progress Report. National Cancer Institute. Updated March 2024. Accessed November 19, 2023. https://progressreport.cancer.gov/prevention/hpv_immunization

6. Perkins RB, Zisblatt L, Legler A, Trucks E, Hanchate A, Gorin SS. Effectiveness of a provider-focused intervention to improve HPV vaccination rates in boys and girls. Vaccine. 2015;33(9):1223-1229. doi:10.1016/j.vaccine.2014.11.021

7. Irving SA, Groom HC, Stokley S, et al. Human papillomavirus vaccine coverage and prevalence of missed opportunities for vaccination in an integrated healthcare system. Acad Pediatr. 2018;18(suppl 2):S85-S92. doi:10.1016/j.acap.2017.09.002

8. Give teens vaccines. Evidence-Based Cancer Control Programs. National Cancer Institute. Updated March 29, 2023. Accessed November 19, 2023. https://ebccp.cancercontrol.cancer.gov/programDetails.do?programId=26197507

9. DOSE HPV: development of systems and education for HPV vaccination. Evidence-Based Cancer Control Programs. National Cancer Institute. Updated March 29, 2023. Accessed November 19, 2023.
https://ebccp.cancercontrol.cancer.gov/programDetails.do?programId=25930477

10. Lake PW, Kasting ML, Christy SM, Vadaparampil ST. Provider perspectives on multilevel barriers to HPV vaccination. Hum Vaccin Immunother. 2019;15(7-8):1784-1793. doi:10.1080/21645515.2019.1581554

11. Gilkey MB, Parks MJ, Margolis MA, McRee AL, Terk JV. Implementing evidence-based strategies to improve HPV vaccine delivery. Pediatrics. 2019;144(1):e20182500. doi:10.1542/peds.2018-2500

12. Briss PA, Rodewald LE, Hinman AR, et al. Reviews of evidence regarding interventions to improve vaccination coverage in children, adolescents, and adults. The Task Force on Community Preventive Services. Am J Prev Med. 2000;18(suppl 1):97-140. doi:10.1016/S0749-3797(99)00118-x

13. Ivers N, Jamtvedt G, Flottorp S, et al. Audit and feedback: effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev. 2012;2012(6):CD000259. doi:10.1002/14651858.CD000259.pub3

14. Veloski J, Boex JR, Grasberger MJ, Evans A, Wolfson DB. Systematic review of the literature on assessment, feedback and physicians’ clinical performance: BEME Guide No. 7. Med Teach. 2006;28(2):117-128. doi:10.1080/01421590600622665

15. Hysong SJ. Meta-analysis: audit and feedback features impact effectiveness on care quality. Med Care. 2009;47(3):356-363. doi:10.1097/MLR.0b013e3181893f6b

16. Frey BS, Meier S. Social comparisons and pro-social behavior: testing “conditional cooperation” in a field experiment. Am Econ Rev. 2004;94(5):1717-1722. doi:10.1257/0002828043052187

17. Blumenthal DM, Song Z, Jena AB, Ferris TG. Guidance for structuring team-based incentives in healthcare. Am J Manag Care. 2013;19(2):e64-e70.

18. Questionnaires. United States Census Bureau. Revised November 19, 2021. Accessed November 19, 2023. https://www.census.gov/programs-surveys/cps/technical-documentation/questionnaires.html