USE CASES/PILOTS/METHODOLOGIES

A Digital Intervention to Reduce Disparities in Well-Child and Immunization Completion in Community Health

Nivedita Mohanty, MD¹ ; Ta-Yun Yang, MS¹ ; Jennifer Morrison, APN, MPH¹ ; Tania Hossain, MD, MPH² ; Andrea Wilson¹ ; Abbey Ekong¹

¹AllianceChicago, Chicago, Illinois, USA; ²Heartland Health Centers, Chicago, Illinois, USA

Keywords: chatbot, CHEC-UP, immunization, telehealth, well-child visits

Abstract

Objective: Well-child visits and immunizations among children in the United States declined at the onset of COVID-19, and vulnerable populations have been disproportionately affected. We tested an innovative mechanism to use chatbots to engage caregivers in evidence-based preventive care for children.

Design: The Child Health Engagement and Coaching Using Patient-centered Innovation (CHEC-UP) project was developed and implemented as a clinical quality improvement pilot in primary care pediatrics. Artificial intelligence (AI)-enabled chatbots were used to personalize messages and facilitate appointment scheduling over 5 months. The chatbot included a campaign to simultaneously send texts to multiple individuals whose children were within 2 weeks of a recommended wellness visit and vaccines. From the text message, recipients launched either an English- or Spanish-language chatbot from their smartphone and were guided through a predefined automated conversation that provided age-specific education, asked predefined questions, and provided guidance based on recipient answers.

Setting: The pilot study was conducted at a community health center in Chicago, Illinois, that serves roughly 10,500 children, and whose parents or guardians are 82% racial and/or ethnic minorities.

Participants: We targeted outreach to 250 English- and Spanish-speaking families with children of 0–17 years for proactive outreach using chatbots promoting well-child visit completion and up-to-date immunization status. Initially, a special emphasis was placed on the 0–2 age group as the first 2 years represent a critical time for primary prevention of vaccine-preventable diseases.

Interventions: Intervention focused on pre-visit engagement by launching an AI-enabled chatbot to deliver personalized messages and facilitate appointment scheduling via mobile devices. An additional novel component of CHEC-UP entailed disseminating evidence-based anticipatory guidance prior to an appointment.

Results: Chatbots facilitated a relative increase in well-child visits and immunizations by 27% in the in tervention group who engaged with the chatbot. Well-child visits and immunizations in the intervention group demonstrated an absolute increase of 13% compared to the usual care group. Survey results, and patient and clinician interviews revealed a high level of satisfaction with the chatbot. Patients also identified future use cases for chatbots to improve health and well-being.

Conclusions: Engaging patients with chatbots improved vaccination and well-child uptake. Patients were highly satisfied with chatbot engagement. By engaging patients and caregivers, chatbots present the potential to proactively engage patients in care and optimize vaccination uptake and realize one of societies’ greatest public health achievements: decreasing the spread of communicable diseases.

 

 

Well-child visits are an important venue for counseling, as well as vaccine administration and documentation. The idea for the CHEC-UP (Child Health Engagement and Coaching Using Patient-centered Innovation) project was conceived as part of Health Resources and Services Administration challenge to improve vaccination rates.¹

Access disparities still exist in national pediatric initiatives despite technological advances. The use of artificial intelligence (AI) chatbots has grown exponentially in many industries and is gaining rising acceptance from clinicians on its merits for patient engagement.² High-level adoption of smartphones in community health underscores the role that smartphones can provide as a vehicle to engage patients in health services such as preventive care visits.³ Additionally, smartphone-based patient engagement around immunizations has demonstrated improvements in immunization status in children.⁴

However, despite advances in the use of mobile technology, regional vaccination programs, and public health guidance, vulnerable communities still experience barriers to well-child visits and optimal uptake of immunizations.⁵ These disparities in access have been amplified by the COVID-19 pandemic along with other barriers to care, including transportation limitations, stress, and economic hardship.⁶

Within months of the COVID-19 pandemic, Centers for Disease Control and Prevention (CDC) data showed declining rates of vaccinations and well-child visits among pediatric populations.⁷ The CDC reported that a decline in vaccination coverage might leave young children and communities vulnerable to vaccine-preventable diseases such as measles.⁸ Well-child visits are not only an important venue for vaccine administration but also for supporting families and identifying a variety of health needs. Providers screen for developmental delays, and parents can raise concerns about physical concerns, behavior, and mental health and receive personalized guidance on healthy nutrition, exercise, and safety.

The CHEC-UP project’s goals aimed to reduce disparities in well-child care and immunization completion by leveraging chatbots in a community health center (CHC). AllianceChicago’s approach optimized patient-centered digital technology to coach and promote engagement with parents around well-child and vaccination visits while providing trusted, evidence-based recommendations from the CDC’s Developmental Milestones. ⁹

AllianceChicago is a national, practice-based research network consisting of more than 50 CHCs across 19 states. The CHEC-UP project was conceived as a pilot quality improvement project to address the need in underserved communities to establish a streamlined online process for engaging families interactively, scheduling children for well-child visits, and providing anticipatory guidance. The project was designed to facilitate a timely, patient-centered connection to the patient’s pediatric care team at a CHC. The CHEC-UP digital outreach entails three ways that the patient’s care team connects via a HIPAA-compliant chatbot by a smartphone—communicating with the caregiver before the child’s visit, proactively sharing evidence-based anticipatory guidance materials, and facilitating easy scheduling.

CHCs served as patient-centered medical homes that offer comprehensive, culturally responsive care to meet the health and social needs of entire families.¹⁰ The CHCs rapidly adopted digital health technology during the public health emergency to create new channels for patient access, education, and engagement to connect with patients and to deliver primary care services.¹¹ The CHEC-UP project created an opportunity to evaluate the relationship between digital health tools and specific health outcomes such as immunization status and preventive health visit completion. CHEC-UP provided an all-in-one, patient-centered technology engagement solution that combines innovative chatbot technology and coaching to: 1) remind parents of upcoming well-child visits and immunizations at their medical home; 2) promote dialogue around age-based recommendations from the CDC; and 3) facilitate easy appointment scheduling that aligned with patient preferences.

Methods

Populations Served

AllianceChicago provides shared infrastructure for health information technology, research, innovation, quality improvement, and clinical collaboration across 50 CHCs. The CHC participants in the network represent rural and urban geographies and collectively serve nearly 160,000 children. The CHEC-UP project targeted families with children 0–17 years of age for proactive outreach using chatbots promoting well-child visit completion and up-to-date immunization status. Initially, a special emphasis was placed on the 0–2 age group as the first 2 years represent a critical time for primary prevention of vaccine-preventable diseases.¹² Additionally, this age group has been prioritized for national quality measurement related to a health system’s performance on ensuring the pediatric populations receive timely childhood immunizations.¹³ After appreciating a decline in visits across age groups, the age range was expanded.

The population engaged in CHEC-UP included families who are English or Spanish speaking. AllianceChicago implemented CHEC-UP at one of the partners CHCs in Chicago, which serves roughly 10,500 children, and 82% of patients or guardians are racial and/or ethnic minorities. Around 30% of patients are best served in a language other than English. Nearly 99% of patients served are at or below 200% of the federal poverty guideline. AllianceChicago delivered the intervention to a total of 249 families in the intervention group and 250 families in the control group for comparison. The sample size of 250 was selected after factoring in feasibility during the 5-month period and considerations from past projects; these past projects also required a component of technology design, customization, testing, implementation, and iterative modification. The 250 patients in the control cohort were randomly selected to be age matched cohorts that were similar based on the demographic characteristics of active patients at the same CHC.

Methods

AllianceChicago’s multidisciplinary team reflected expertise in pediatric clinical care in CHCs, primary care innovations, data science, user-centered design, and consumer-facing patient engagement technology. From the period of May 24, 2021 to November 30, 2021, Plan-Do-Study-Act (PDSA) improvement cycles were leveraged to implement and evaluate outcomes. In keeping with the learning health system framework, a process of continuous evaluation of quantitative and qualitative data was used to make modifications to the project. A multi-disciplinary team of clinicians, user-centered designers, and AI technology development experts collaborated to develop messages and decision trees customized by the ages corresponding to vaccinations and well-child visits due. A population health approach was employed to identify CHC patients who were due for immunizations and/or well-child, and an eligibility list was developed. The list was curated from data and analytic tools provided and hosted by AllianceChicago on behalf of the CHC. The analytic tools included clinical and administrative data obtained from electronic health records (EHRs), such as demographics, appointment information, and immunization completion data. A schedule of message deployment was developed for the patients due for visits and immunizations during the pilot period, with the messages deployed from the chatbot platform. Outcomes data on well-child visit and immunization completion were extracted from AllianceChicago’s data and analytics platform for analysis. Surveys to assess the experience with chatbots were designed and deployed via a message delivered via chat message. The team continuously monitored analytics on engagement with the chatbot and outcomes of well-child visits and immunizations to improve the intervention. Results were analyzed across three groups: (1) Usual Care, (2) Intervention group where the text-based message was deployed, (3) Intervention and Engagement group where the families who received the text message launched and engaged with the chatbot.

Digitally Enabling Intervention Outreach

AllianceChicago implemented the chatbot system to reduce disparities in well-child and immunization completions among vulnerable communities. Caregivers of pediatric patients were contacted as part of the intervention, which involved using a text-based communication system to launch an AI-enabled chatbot to deliver personalized messages and facilitate appointment scheduling via a parent’s mobile device. Ultimately, informed by patient feedback, the intervention format was expanded to include email outreach as well.

The AllianceChicago team customized pre-built workflows using the chatbot design studio to create template messaging for patient outreach. This messaging was tailored to the specific age groups and adapted as needed. The chatbot technology allowed for easy customization, and it did not require any advanced coding experience. The chatbot was deployed on a weekly to bi-weekly basis as the team adjusted to the clinic and patient feedback.

The chatbot assisted families in the network by: forwarding reminders via text and email of upcoming well-child visits and immunizations in their preferred language; initiating dissemination of evidence-based anticipatory guidance from the CDC before the visit to enrich dialogue; and facilitating easy appointment scheduling by notifying reception of patients indicating interest in scheduling an office visit.

AI-Based Chatbot Enables Human-Centered Patient Communication

Proactive smartphone-based outreach to parents provided reminders of upcoming well-child visits and shared anticipatory guidance from the CDC. Interventions such as text message reminders have proved effective at improving immunization rates.¹⁴ CHEC-UP augmented the concept of “reminder systems” through the chatbot while making encounters interactive, educational, and engaging while alleviating cumbersome aspects of appointment scheduling. A particularly novel aspect of CHEC-UP is the ability to provide an all-in-one communication solution for real-time, bi-directional messaging; future capabilities might include, for example, video or phone visits and EHR integration, thus eliminating the need for multiple engagement tools.

An additional advantage of the chatbot communication was that it did not require the download of an app. In prior work, AllianceChicago identified that app downloads can present a technology barrier to patient engagement at CHCs. To provide flexibility with end-user preferences, the chatbot is accessible from any smart device or desktop computer. For CHEC-UP, the team used the chatbot’s AI engine to customize algorithms in Spanish and English for well-child and immunization content. The chatbot engaged patients in their primary language, which contributed significantly to enabling participating CHCs to improve care quality, outcomes, patient experience, and satisfaction around pediatric care.

Coaching and Patient Engagement Using Anticipatory Guidance

Coaching involves purposeful and motivational conversations to engage patients in their health. Pediatric providers in the United States have confidence in the evidence-based guidance provided through trusted entities such as the CDC and the American Academy of Pediatrics; most practitioners share anticipatory guidance handouts with parents during well-child visits. These handouts provide valuable information about nutrition, development, safety, and expectations. An office visit can be a stressful time for parents to process information in the detailed handout. Office visits are time-limited, and having vital signs checked or receiving immunizations can be upsetting to young children.

CHEC-UP changed this dynamic by proactively engaging parents at their convenience, providing anticipatory guidance before the family arrives. Coaching in preparation for the visit with trusted information has the potential to educate, improve confidence, and promote shared decisions between the parent and provider when they are together. Using the chatbot technology to share recommendations allowed parents to digest valuable information about their child’s growth and development outside of the stress and pressure of an office visit.

Learning Health Systems Framework

Throughout the project, continuous improvement was driven by data analysis and clinic and patient stakeholder feedback. Patient families were vital members of the learning team. CHEC-UP leveraged chatbot technology to collect ongoing feedback to strengthen the intervention. A learning health system framework approach allowed for feedback and modifications to the project as needed in weekly meetings of the project team.¹

The feedback chatbot was developed to increase patient engagement and learn about the end-user experience and satisfaction. Based on patient families and CHC clinician feedback, the age group of patients receiving the intervention was expanded from 0–2 years to 0–17; email distribution was introduced to the workflow; messaging was simplified and personalized in both English and Spanish; an option for patient scheduling was introduced; and the timing of chatbot deployment was altered throughout the course of the project. Utilizing data allowed for continuous monitoring and modification of the intervention approach.

Additionally, patients who initially interacted with the chatbot were sent a follow-up chatbot message to request their participation in an interview. Questions for patient interviews were geared to help the team better understand factors influencing engagement and future opportunities to improve the use of chatbots. Examples of questions included: What made you click on the chatbot to open it? Were you hesitant to open the chatbot? How could we avoid this in the future and make patients feel more comfortable? Were there any struggles in using the chatbot? Do you have any ideas on how to improve the chatbot? Do you like the use of chatbots at your Health Center? Would you be more or less likely to open a chatbot in the future after this experience? If you could access the chatbot at any time on your Health Centers website to ask a question would that be helpful?

Sustainability of CHEC-UP

The pilot enabled the real-world workflow design, technical implementation, and rigorous evaluation of CHEC-UP to identify the most promising practices that could be both sustained and scaled beyond the project period. With the project deemed successful, CHEC-UP has the potential for operational efficiencies gained by outreach automation and fewer revenue losses attributable to no-shows or missed appointments.

Financial sustainability for the chatbot technology has the potential to be achieved from: 1) cost savings from automated workflows instead of reliance on staff, 2) revenue from billable well-child visits, and 3) quality payments for performance on quality measures focused on immunizations. Additionally, because the platform is an all-in-one solution with virtual visit and messaging capabilities, it eliminates costs currently incurred by CHCs for separate text messaging and telehealth solutions.

Human and Animal Participants

This pilot project was conducted as a quality improvement activity to enhance evidence-based clinical care, and, thus, based on guidelines outlined by the Department of Health and Human Services was not deemed to be human subjects research.

Results

CHEC-UP Outcomes Prove Successful Use of Chatbots to Improve Health Disparities

During the 5-month implementation period, key metrics heralded positive outcomes. Across the intervention group of 249 patients, 28% percent of patients/families engaged with the chatbot (intervention and engagement group); furthermore, this group experienced a 27% relative increase in well-child visit and immunization completion. The cohort of patient/families who engaged with the chatbot demonstrated a 13% absolute increase in immunizations and well-child visit completion. The cohort of patients receiving the text message to launch the chatbot (intervention group) demonstrated an absolute increase of 8% in the intervention group when compared to usual care, demonstrating they were more likely to schedule and attend a well-visit when compared to the control group—even if they did not engage with the chatbot.²

There was observed variation in outcome when looking at specific age segments: for children 0–11 years of age, opening the chatbot resulted in a 30% relative increase in well visits and immunizations compared to baseline; for older adolescents in the 12–17-year-old age group, the chatbot did not result in any difference in immunizations or well visit attendance.

Fig. 1. Learning health systems framework.

Continuous Improvement

Throughout the project, the AllianceChicago team conducted continuous improvement using data analysis based on clinic and patient stakeholder feedback as well as direct feedback from staff at the CHC who represented clinical and operations-related roles. The AllianceChicago team leveraged chatbot technology to collect ongoing feedback from patient families, and whose input was considered vital to strengthening the intervention.³

The team’s quantitative and qualitative evaluation will inform its evolving use case strategy for future iterations, drive intervention for long-term success, and generate a knowledge base for the future patient engagement needs of pediatric patients and their families served by CHCs.

Discussion

The pilot project highlighted a practical intervention that led to an improvement in desired clinical outcomes of improved vaccination rates and well-child care by implementing a chatbot.

The CHEC-UP project successfully demonstrated how chatbots can be used innovatively to improve health disparities in a patient-centered manner. The highest impact in the pilot project was observed in children ages 0 to 11 years of age. There was no significant impact observed in older adolescents aged 12 to 17 years. One factor for the difference may be that older children have much less frequent vaccination visits and well-child visits compared to younger aged children where immunizations are administered more frequently. Older adolescents may already had visits outside of the pilot period. Additionally, well-child visits for younger children occur more frequently to coincide with key milestones and requirements such as for Women Infant and Children documents and school and sports physicals. The results may have been impacted the short pilot period and a smaller number of participants in the adolescent age range.

Fig. 2. CHEC-UP (Child Health Engagement and Coaching Using Patient-centered Innovation) results.

Similar to CHEC-UP, using pre-visit chatbots for other clinical use cases was received favorably by patients as described by a qualitative study of prenatal patients.¹⁴ Studies conducted using research methodologies across a larger sample of patients would be beneficial to understand; specifically, the identification of components of education delivered successfully outside of an office visit and the preferred format that best engages patients.

The use of chatbots has the potential to be transformational in many ways. It can provide a more engaging mechanism of outreach to patients and can align with preferences of how patients receive information. While this project was conducted as a quality improvement project, research studies also suggest that text-based interventions can be effective in improving immunization rates. In a study of influenza vaccination, 27.1% of the intervention group who received text messages had completed influenza vaccination compared with 23% in the usual care group.¹⁵

In addition to going beyond routine applications of text messaging such as appointment reminders, the CHEC-UP project focused on conversations to understand specific needs such as assistance with scheduling the appointment. Patients specifically requested assistance with scheduling that could be facilitated with clinic staff. Additionally, patients expressed high satisfaction with receiving health education materials in advance of the visit. The workflow implemented through CHEC-UP to ensure preventive services are completed could potentially be leveraged for other clinical areas in primary care, such as cancer screening, chronic care management, and social determinants of health screening.

Innovation

Digital health presents the opportunity to reimagine new patient-centered engagement outside of the traditional health setting that recognizes the patient’s need for convenience and flexibility. Making a pivot to digital health challenges health systems to rethink the notion of point of care—beyond the physical walls of a clinic and outside of the time constraints of an office visit to better manage the delivery of health services. Traditionally, providers and patients consider the point of care occurring at a hospital or clinic. The emergence of digital solutions challenges this perception by engaging patients, wherever they are using smartphone technology and in a format that feels human and highly bidirectional.

Outreach-related pediatric care to increase engagement and wellness can be impactful to public health, providers, and patients. This is especially true in populations where needs may be complex due to systemic and structural barriers. These populations are at risk for further disparities as demonstrated during the pandemic; for example, regularly scheduled childhood immunizations and well-child visits nationally were at an all-time low.

Fig. 3. End-user feedback to use of chatbots.

Provider organizations that provide health access and services to disadvantaged populations are looking for proactive solutions including digital outreach to overcome some of the structural barriers, for example, transportation and language experienced by their patient communities. Leaders of public health organizations in addition to hospitals, health systems, medical practices, post-acute care facilities, and other organizations can benefit from the use of chatbots to improve the patient experience, engagement, compliance, and outcomes.

Conclusion

The CHEC-UP project demonstrated success as an innovative approach to address declines in immunization rates and well child completion in community health. The pediatric health outcomes of immunization rates and well child visit attendance improved during a 5-month pilot period. The project’s use of chatbots demonstrates responsiveness to the needs frequently articulated by patients served by CHCs, such as timely access to care teams and easier scheduling processes. By leveraging streamlined smartphone technology and approachable human-centered communications, the chatbot alleviates barriers. CHEC-UP also introduces opportunities to improve parents’ confidence and participation in their child’s health by equipping them with targeted education regarding their child’s development in order to make knowledgeable decisions. By engaging patients and guardians, chatbots present the potential to optimize vaccination uptake and realize one of societies’ greatest public health achievements: decreasing the spread of communicable diseases. Future study of digital health’s impact using research methodologies across a greater sample size and different populations is needed to better understand how interventions such as chatbots correlate with clinical and public health outcomes.

Contributors

Nivedita Mohanty, MD, is a project lead for CHEC-UP; she directed the project’s study design, implementation, and evaluation. She wrote the original draft, with subsequent reviewing and editing, and approved the submitted version. Jennifer Morrison, APN, MPH, directed the project study design, implementation, and evaluation; reviewed and edited the manuscript; and approved the submitted version. Tania Hossain, MD, is a project lead for CHEC-UP at Heartland Health Centers, directed the project’s study design and implementation, and reviewed and edited with the approval of the submitted version. Abbey Ekong contributed to study design, implementation, and evaluation, and reviewed and edited with the approval of the submitted version. Ta-Yun Yang, MS, conducted statistical analysis for CHEC-UP data evaluation, with reviewing and editing, and approval of submitted version. Andrea Wilson contributed to reviewing and editing, and approval of the submitted version.

Acknowledgments

We wish to express our deepest gratitude to the following participants for their invaluable contributions to the study. We are sincerely thankful for Jeffrey Panzer, MD (Alliance Chicago and Heartland Health Centers), Melissa Briano (Heartland Health Centers), and other clinical staff for their research, technical, and other assistance in the conception and implementation of the CHEC-UP project at Heartland Health Centers. We recognize and profoundly thank the Health Resources and Services Administration (HRSA) and the P4 Innovation Challenge for their financial contributions, which made this pilot project possible. We appreciate QliqSOFT’s generous contributions providing the Quincy chatbot.

References

1. HRSA Maternal & Child Health. Submission requirements for the promoting pediatric primary prevention (P4) challenge. Health Resources & Services Administration. Available from: https://mchb.hrsa.gov/funding/challenge-competitions/p4-challenge/submission-requirements-promoting-pediatric-primary [cited 25 October 2022].
2. Brandtzaeg PB, Følstad A. Chatbots: changing user needs and motivations. Interactions. 2018;25(5):38–43. doi: 10.1145/3236669
3. Broderick A, Haque F. Mobile health and patient engagement in the safety net: a survey of community health centers and clinics. Issue Brief (Commonw Fund). 2015 May;9:1–9. PMID: 26040018.
4. Dumit EM, Novillo-Ortiz D, Contreras M, Velandia M, Danovaro-Holliday MC. The use of eHealth with immunizations: an overview of systematic reviews. Vaccine. 2018;36(52):7923–8. doi: 10.1016/j.vaccine.2018.06.076
5. Samuels RC, Ward VL, Melvin P, Macht-Greenberg M, Wenren LM, Yi J, et al. Missed appointments: factors contributing to high no-show rates in an urban pediatrics primary care clinic. Clin Pediatr. 2015;54(10):976–82. doi: 10.1177/0009922815570613
6. McNally VV, Bernstein HH. The effect of the COVID-19 pandemic on childhood immunizations: ways to strengthen routine vaccination. Pediatr Ann. 2020;49(12):e516–22. doi: 10.3928/19382359-20201115-01
7. Patel Murthy B, Zell E, Kirtland K, Jones-Jack N, Harris L, Sprague C, et al. Impact of the COVID-19 pandemic on administration of selected routine childhood and adolescent vaccinations—10 U.S. Jurisdictions, March-September 2020. MMWR Morb Mortal Wkly Rep. 2021 Jun 11;70(23):840–5. doi: 10.15585/mmwr.mm7023a2
8. Bramer CA, Kimmins LM, Swanson R, Kuo J, Vranesich P, Jacques-Carroll LA, et al. Decline in child vaccination coverage during the COVID-19 pandemic—Michigan Care Improvement Registry, May 2016–May 2020. MMWR Morb Mortal Wkly Rep. 2020 May 22;69(20):630–1. doi: 10.15585/mmwr.mm6920e1
9. Zubler JM, Wiggins LD, Macias MM, Whitaker TM, Shaw JS, Squires JK, et al. Evidence-informed milestones for developmental surveillance tools. Pediatrics. 2022 Mar 1;149(3):e2021052138. doi: 10.1542/peds.2021-052138
10. Hu R, Shi L, Sripipatana A, Liang H, Sharma R, Nair S, et al. The association of patient-centered medical home designation with quality of care of HRSA-funded health centers. Med Care. 2018;56(2):130–8. doi: 10.1097/MLR.0000000000000862
11. Demeke HB, Pao LZ, Clark H, Romero L, Neri A, Shah R, et al. Telehealth practice among health centers during the COVID-19 pandemic—United States, July 11–17, 2020. Morb Mortal Wkly Rep. 2020;69(50):1902. doi: 10.15585/mmwr.mm6950a4
12. CDC: catch up on well-child visits and recommended vaccinations. Available from: https://www.cdc.gov/vaccines/parents/visit/vaccination-during-COVID-19.html [cited 12 September 2022].
13. eCQI Resource Center (healthit.gov): childhood immunization status. Available from: https://ecqi.healthit.gov/ecqm/ec/2021/cms117v9 [cited 12 September 2022].
14. Urbina C, Muyalde CM, Gudmundsson S. Patient and provider response to a prenatal pre-visit chatbot. Human Genetics Theses 96. 2021. Available from: https://digitalcommons.slc.edu/genetics_etd/96 [cited 25 October 2022].
15. Stockwell MS, Kharbanda EO, Martinez RA, Lara M, Vawdrey D, Natarajan K, et al. Text4Health: impact of text message reminder–recalls for pediatric and adolescent immunizations. Am J Public Health. 2012;102(2):e15–21. doi: 10.2105/AJPH.2011.300331

Copyright Ownership: This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, adapt, enhance this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0