Chatbot Systems Using AI May Feasibly Provide Access to Information on Infectious Diseases

Information on current vaccines and infectious diseases could be communicated accurately, completely, and persuasively by chatbot systems that utilize artificial intelligence (AI), according to a study published in Digital Health. The researchers believe these systems could be used to persuade people to act in support of their health.

At the time of the study, the COVID-19 pandemic had killed more than 984,000 individuals in the United States alone, with only 69.9% of all the eligible population acquiring the vaccine due to vaccine hesitancy and resistance. The main reasons for vaccine hesitancy include lack of complete and correct information about the vaccines and their level of effectiveness, as well as a lack of trust in the vaccines and suspicions about the rapid development.

As such, a priority for public health is to provide responses to queries about the COVID-19 vaccines and the disease itself while also providing primary care. Chatbots using AI could represent a cost-effective way to deliver direct responses to questions and concerns. This study aimed to describe the design and deployment of an AI COVID-19 vaccine promotion chatbot and assess the accuracy and engagement with the chatbot engagement.

The chatbot was able to deliver messages via SMS through a textbot and on websites through a webbot. The chatbot was aimed at populations at risk of disparities in COVID-19 outcomes, such as those with lower income and those of ethnic minority groups. The researchers partnered with 5 health care systems in the state of Colorado, 3 of which served patients in the Denver metropolitan area and 2 of which served those outside the Front Range; 75% of patients receiving care at these clinics were uninsured and 62% were non-White.

All health care delivery systems created an SMS message that informed their patients of the available chatbot, and 2 partnering health care systems embedded a web-based version of the chatbot system onto their website.

The chatbot was built by categorizing anticipated “intents,” or topics that the researchers believed individuals would want to ask about COVID-19. There were 51 intents that were generated for the initial start of the system, and intents were added as more information on questions was collected. New intents were added as more information about COVID-19 vaccines came out. At least 18 variations on queries for each intent were collected after crowdsourcing participants provided potential questions that could be asked about COVID-19.

Response messages were developed for English and Spanish. All answers were validated with credible sources, including the CDC, the World Health Organization, the FDA, The New York Times, and the Colorado Department of Public Health and Environment.

Message content was updated with new COVID-19 vaccine guidance, which led to 55 different versions of the textbot and 33 versions of the webbot by the end of the study. The chatbot was launched on April 19, 2021, and was active until March 22, 2022. In that time, 2479 users interacted with the chatbot and generated 3994 questions. A mean (SD; median) of 1.8 questions (1.8; 5) were created by each user, ranging from 1 to 38 questions.

Most frequently asked questions included ones about boosters, asked 648 times. Where to get a vaccine (601 times) and COVID-19 symptoms, testing, and reporting (464 times) were the next most frequent questions. The total accuracy of the chatbot was 74.8% on average, with the chatbot starting at 54.0% accuracy and ending on 91.1% accuracy. The average reaction time for the system to provide an answer was 0.199 seconds.

The researchers concluded that chatbot systems were a potentially useful and feasible tool that could help patients access complete and persuasive information about infectious diseases and their vaccines. They plan to conduct a quasi-experimental study of the chatbot’s efficacy in increasing vaccination rates.

Reference

Zhou S, Silvasstar J, Clark C, Salyers AJ, Chavez C, Bull SS. An artificially intelligent, natural language processing chatbot designed to promote COVID-19 vaccination: a proof-of-concept pilot study. Digit Health. Published online March 5, 2023. doi:10.1177/20552076231155679