Health IT
David McCune, MD, MPH1,2; James Pellegrin, MS1; Anshul Sachdeva1; Roxana Cham, MD1; Jessica Sollaccio, MPH1; Sandra Giramahoro Coyne, MPH1; Mark Stewart, MS
Corresponding Author; dmccune@98point6.comObjective: Reducing antibiotic overuse is a point of emphasis of the Centers for Disease Control and Prevention. Inappropriate use of antibiotics is cited as a key driver of antibiotic resistance. Despite this, both telemedicine and traditional in-person medical care struggle to meet national guidelines. This study evaluates antibiotic prescribing practices at 98point6, a Seattle-based provider of real-time, text-based primary care. This paper reviews a novel combination of machine learning with a physician-led virtual platform and smartphone interface to exceed published benchmarks for the avoidance of antibiotics in the treatment of bronchitis.
Design: This retrospective cohort study looks at patients ages 18–64 who were evaluated and treated using real-time, text-based care. Cases in which patients were diagnosed with “acute bronchitis/bronchiolitis” between December 1, 2019 and November 30, 2020 were analyzed. Visits were categorized by whether systemic antibiotics were or were not given.
Setting: This telemedicine provider provides service to patients ages 1 and above in all 50 states and the District of Columbia. The cohort that is present was drawn from that population.
Participants: A total of 1,238 consecutive patients ages 18–64 who presented for medical care during the time period.
Interventions: This is a retrospective cohort. No direct intervention was undertaken. Patients received standard care for the evaluation and treatment of upper respiratory symptoms.
Main Outcome Measures: The results are presented as descriptive statistics with demographic information and compared to published reports of appropriate antibiotic use for the treatment of bronchitis.
Results: Of the 1,238 visits with a bronchitis/bronchiolitis diagnosis, 99 (8.0%) were treated with prescribed systemic antibiotics. The rate of antibiotic avoidance of 92.0% compares favorably with published benchmarks from the National Committee for Quality Assurance and is significantly lower than rates in previously published samples for telemedicine, primary care, and urgent care.
Conclusion: Real-time text-based care resulted in a high rate of adherence to the established standard for the treatment of bronchitis, greatly exceeding benchmarks and published results from both telemedicine and in-person medical practice.
Keywords: Antibiotic Stewardship, Bronchitis, Machine Learning, Primary Care, Telemedicine
Citation: Telehealth and Medicine Today 2021, 6: 272 - http://dx.doi.org/10.30953/tmt.v6.272
Copyright: © 2021 The Authors. This is an open access article distributed in accordance with the Creative Commons Attribution NonCommercial (CC BY-NC 4.0) license, which permits others to distribute, adapt, and enhance this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See http://creativecommons.org/licenses/by-nc/4.0.
Published: 18 November 2021
Corresponding Author: David McCune, dmccune@98point6.com
Antibiotic-resistant infections are on the rise in the United States and have become a population health crisis, accounting for at least 2.8 million infections and 35,000 deaths yearly according to the Centers for Disease Control and Prevention (CDC).1 Reducing inappropriate antibiotic use can play a critical role in helping address this problem. Several recent publications have highlighted inappropriate antibiotic use by ambulatory care providers in both brick-and-mortar and telemedicine settings. Overall, antibiotic-resistant infections are estimated to cost the United States more than $20 billion annually in direct healthcare costs and another $35 billion in lost productivity.2
A 2016 study of in-person care by Fleming-Dutra et al.3 estimated that approximately 50% of antibiotic prescriptions for acute respiratory conditions are unnecessary. Telemedicine’s track record is no better; Uscher-Pines et al.4 found that more than 83% of the patients of a large telemedicine provider with a diagnosis of bronchitis were prescribed systemic antibiotics. The same authors published an updated paper examining a larger dataset of telemedicine visits that showed systemic antibiotics were still prescribed in approximately 57% of bronchitis/bronchiolitis cases.5
Acute bronchitis is a self-limited infection of the upper airway. It is characterized primarily by a cough that can last up to 6 weeks. It is among the most common outpatient diagnoses, representing as much as 10% of ambulatory healthcare visits.6 The majority of cases are caused by a viral infection,7 and the recommended treatment for bronchitis is supportive care without antibiotics. The inappropriate use of antibiotics in bronchitis increases cost,8 causes avoidable complications,9 and is a factor in increasing antibiotic resistance.10 This makes the appropriate treatment of bronchitis an easily measured and clear benchmark for clinical quality. In fact, “avoidance of antibiotic treatment” in this condition is tracked by the National Committee for Quality Assurance (NCQA) as part of its Healthcare Effectiveness Data and Information Set (HEDIS), a widely used performance improvement tool.11 Despite the national attention for avoiding antibiotics in bronchitis, the US healthcare system struggles with this goal. In 2019, the most recent data available, NCQA-reported rates of antibiotic avoidance are still low; commercial preferred provider organizations (PPOs) and health maintenance organizations (HMOs) avoided antibiotics for only 40.2 and 41.2% of patients aged 18–64, respectively.11
This paper uses the term “virtual clinic” to describe the primary care provided by 98point6, a Seattle-based company delivering medical care using a smartphone application (app). The service is on-demand, with patients using the app without any requirement for an appointment. The clinic here differs from traditional video telemedicine in that the patient interview and medical care are conducted using text-based messaging. Some states require a brief video interview to establish care during a patient’s first visit, but more than 98% of the remainder of doctor–patient communication is completed via real-time, text-based messaging. This method is distinct from store-and-forward, with no delay of secure in-app messages exchanged by doctor and patient.
A patient visit begins with an automated interview, in which the chief complaint is expressed using natural language. The service uses artificial intelligence and machine learning technology to provide an initial assessment, asking questions that are relevant to the concern that was entered by the patient. After the automated interview, the app connects the patient with a primary care physician who completes the evaluation and makes any treatment recommendations or medication orders. A care plan is sent to the patient’s phone at the conclusion of the visit.
The clinic uses a program of antibiotic stewardship that is rooted in evidence-based practices encouraged by the CDC, American College of Physicians (ACP) and the American Academy of Family Practitioners (AAFP). The guidelines are tailored to the unique care modality and specify infectious disease conditions that do or do not warrant an antibiotic prescription. They are then paired with an in-house Clinical Quality Assurance (CQA) process in which peer audits are performed for visits, with the goal of tracking, monitoring, and ensuring adherence to clinical guidelines; feedback from CQA is used as a mechanism to support adherence. Because the virtual clinic is a text-based service, the entire digital encounter is saved and assessed for areas of improvement. Automation and machine learning readily support the culture of antibiotic stewardship as extending to all respiratory infections.
Patients who presented with upper airway symptoms were eligible for assessment. Patients with symptoms consistent with bronchitis were screened for potential serious illness, and those with shortness of breath or painful and/or rapid breathing were referred for in-person evaluation. Additional screening questions evaluated patients for sinusitis, bacterial pharyngitis, pertussis, pneumonia, and influenza before considering bronchitis. Patients with no urgent symptoms and symptoms lasting 1–3 days were classified as upper respiratory illness, most likely viral. Patients with cough lasting 4–21 days in duration were classified as acute bronchitis, and those who experienced symptoms for longer than 21 days were classified as chronic bronchitis or other illnesses, based on individual evaluations.
We analyzed data from adult patients (between the ages of 18–64) diagnosed with acute bronchitis and presenting for care from December 1, 2019 through November 30, 2020. Only “completed encounters,” in which a physician provided a diagnosis and care plan, were reviewed in this analysis. “Noncompleted encounters” (e.g., when a patient abandoned the encounter before a diagnosis had been reached) were discarded from the denominator to avoid falsely decreasing antibiotic prescribing rates. ICD-10 diagnostic categories were used to group the diagnoses. Bronchitis was defined by the ICD-10 codes J20.0 (acute bronchitis due to Mycoplasma pneumoniae, n = 4), J20.8 (acute bronchitis due to other specified organisms, N = 11), and J20.9 (acute bronchitis, unspecified, n = 1,223) for a total of 1,238 cases during the period. Demographic information for the patients is presented in Table 1.
Cases of bronchitis were evaluated for the administration of prescriptions as part of the visit, classified as “no systemic antibiotic” or “systemic antibiotic.” Systemic antibiotics were subdivided as “broad spectrum” or “narrow spectrum” using a previously published classification scheme.12 The rates of treatment are reported in Table 1.
Previously published rates of antibiotic avoidance for bronchitis were evaluated for comparison. The NCQA distributes rates of antibiotic avoidance for 2019 as 41.2% for commercial HMO, 40.2% for commercial PPO, and 52.3% for Medicaid HMO. In an extensive review of prescription practices for upper respiratory illness from 2015 to 2016, Shi et al.5 report antibiotic avoidance rates in bronchitis of 43% (telemedicine), 40% (primary care provider), and 36% (urgent care).
To assess the quality of care, all patients receive an emailed survey 3 days after their visit. One question in the survey is relevant to the accuracy of the diagnosis. The patient is asked if they feel better, worse, the same or “not applicable” compared to when they came for their visit. While this measure is not a direct assessment of diagnostic accuracy, it does provide information on the patient’s subjective impression of the virtual care and subsequent post-visit experience. We present the responses in Table 2.
As part of the comparison with previously published results, we assumed that prescription rates follow a binomial distribution with the distribution mean equal to the prescription rate. We calculate a 95% confidence interval around the mean for these binomial distributions by approximation with normal distributions. If the confidence intervals do not overlap, we can reject the null hypothesis H0 (H0 = the virtual clinic antibiotic prescription rates for bronchitis are equal to prescription rates from telemedicine/PCP/urgent care). Additionally, we calculate a binomial exact test to compare the clinic binomial distribution to the published rates for nontreatment of bronchitis in the sample published by Shi et al.5 Following this, a binomial exact test was used to compare the mean rate of appropriate nontreatment of bronchitis at the clinic, telemedicine, primary care provider, and urgent care. The probability that the true means of the samples overlap was calculated.
Internal counsel determined the study was exempt from Institutional Review Board review under 45 CFR 46.104(d)(4). The conduct of this study and all procedures conformed to the principles described in the Declaration of Helsinki as revised in 2013.
The adherence to the standard of antibiotic avoidance for patients seen by the virtual clinic and diagnosed with bronchitis in the calendar year 2020 is dramatically higher than rates reported in national surveys such as NCQA. In addition, the mean rate of adherence to the standard is much higher than recently reported rates of adherence for telemedicine, primary care providers, and urgent care.
A direct statistical comparison between the sample populations reported in the work of Shi, et al. must begin with a demographic comparison of the two populations. In both cases, the telemedicine population and the virtual clinic population included a majority of female patients. The most common age groups in both samples were in the 18–34 and 35–44 years-old groups. As illustrated in Table 3, the demographics of the groups are similar.
| Virtual clinic (2020) | 2018 paper—telemedicine | 2018 paper—PCP | 2018 paper—urgent care | |||||
| Count | % | Count | % | Count | % | Count | % | |
| Overall | 1,238 | — | 39,974 | — | 1,084,056 | — | 212,837 | — |
| Gender | ||||||||
| Female | 769 | 62.1 | 25,271 | 63.2 | 680,629 | 62.8 | 130,061 | 61.1 |
| Male | 469 | 37.9 | 14,703 | 36.8 | 403,427 | 37.2 | 82,776 | 38.9 |
| Age group | ||||||||
| 18–34 | 464 | 37.5 | 12,979 | 32.5 | 360,129 | 33.2 | 92,369 | 43.4 |
| 35–44 | 351 | 28.4 | 12,637 | 31.6 | 248,321 | 22.9 | 53,185 | 25.0 |
| 45–54 | 252 | 20.4 | 9,213 | 23.0 | 250,865 | 23.1 | 39,714 | 18.7 |
| 55–64 | 171 | 13.8 | 5,145 | 12.9 | 224,741 | 20.7 | 27,569 | 13.0 |
| Demographic comparison between bronchitis visits during 1/1/2020 – 12/31/2020 with unmatched populations reported in 2018 Zhuo Shi et al.5 | ||||||||
Finally, to test whether the results were statistically significant, we calculated the 95% confidence interval around the rates of antibiotic avoidance in bronchitis at the virtual clinic [91.08% 92.92%] compared to traditional telemedicine [41.73% 44.34%], PCP [39.74% 40.26%], and urgent care [35.42% 36.58%]. Based on these calculations, there was a statistically significant difference between the antibiotic avoidance rates of the clinic compared to other published treatment methods. Additionally, a binomial exact test comparison to the clinic prescription rates demonstrated a p-value of p < 0.001 (p < 2.2e-16 indistinguishable from 0) to reject the null hypothesis that the distributions are equal. These comparisons are presented in Table 4.
| Healthcare provider | Antibiotic avoidance rate (%) | 95% confidence interval (normal approximation) | p-value for binomial exact test comparison with virtual clinic |
| Virtual clinic | 92.0 | [91.08% 92.92%] | — |
| Traditional telemedicine | 43.0 | [41.7% 44.3%] | p < 0.001* |
| Urgent care | 36.0 | [35.4% 36.6%] | p < 0.001* |
| PCP | 0.0 | [39.5% 40.6%] | p < 0.001* |
| *Demographic comparison between bronchitis visits during 1/1/2020 – 12/31/2020 with unmatched populations reported in 2018 Zhuo Shi et al.5 | |||
The responses to the post-visit survey are compared to the general population of patients at the clinic during the same time period. Of the patients diagnosed with acute bronchitis, 9.77% responded. Among respondents, overall satisfaction, as measured by NPS, was slightly higher than the general population (72.0 vs. 68.5). The percentage reporting that their condition had worsened was similar (5.0% vs. 5.2%) and the percentage reporting that their condition was unchanged was higher (26.5% vs. 17.3%).
Overprescribing of antibiotics among ambulatory healthcare providers has become a population-health crisis, resulting in multidrug-resistant organisms and numerous avoidable adverse medication reactions. According to the CDC, at least 30% of antibiotics prescribed in the United States are unnecessary.1
Antibiotic stewardship, the judicious and appropriate use of antibiotics, is a critical responsibility of all members of the healthcare system. Yet, despite clear and unambiguous guidelines, providers consistently fail to meet the goals for the treatment of bronchitis. Bronchitis is an illness that will resolve without antibiotics in the majority of cases. Because of the risk of antibiotic resistance, altered host microbial flora and supra infections such as Candida, it is generally the case that the administration of antibiotics has the potential to do more harm than good. Nevertheless, the NCQA benchmark for antibiotic avoidance remains mired below 50%, despite more than a decade of attention to the issue.
Clearly, measures beyond guidelines are called for, and the method of care delivery provided by the virtual clinic represents an important step forward in this process. Since the modality of text-based messaging, rather than in-person or video interviews, allows the entire conversation between patient and provider to be captured and analyzed, a greater opportunity exists to review the relevant data for the diagnosis and apply the appropriate practice standard. The technology supports the physicians by eliciting relevant information in the automated interview and including it in the initial assessment. This makes it easier to adhere to the recommended practice standards; through this, it also reduces provider variability in practice. When combined with the CQA program and an unambiguous practice standard, the platform for delivering primary care has produced a rate of adherence to the nontreatment of bronchitis that is more than double the rate in published samples and national benchmarks.
There are limitations with this comparison. One of the challenges with making a diagnosis of bronchitis is the degree to which the diagnosis can be shown to be accurate. It is possible to miss more serious illness for which antibiotics would be beneficial. As described above, the practice standard applied in the clinic used a well-accepted case definition for bronchitis and also applied screening criteria to refer patients at high risk of pneumonia or other complicated illness for in-person evaluation.
A survey after a visit has limited utility. It is subject to responder bias, which tends to select for patients who had either very positive or negative experiences. Within these limits, the survey results do not suggest that patients treated for bronchitis had an inferior patient experience or greater tendency to have their condition worsen.
Additionally, the period of time during which the data presented by Shi et al.5 was 2015–2016, compared to 2019–2020 in the sample from the virtual clinic. It is possible that the prescribing practices at the institutions involved in the analysis have improved in the intervening years. Additionally, the comparison between the clinic and the other treatment modalities assumes normality in the distribution of patients. Given the size and national distribution of the samples, this is reasonable, but there may be bias that affects the comparison. The samples are not matched based on age, gender, or other demographic factors, and it is possible that this introduced bias into the comparison. The diagnostic practices may differ between the clinic and the institutions that are compared. Furthermore, additional areas for evaluation remain. Future work will include more detailed examination of other types of upper respiratory illness, including viral URI, sinusitis, and otitis.
Despite these challenges, the rate of appropriate antibiotic nonuse for the diagnosis of bronchitis at the virtual clinic is dramatically higher than previously reported rates and national benchmarks. By leveraging a clinical quality system in combination with machine learning that promotes antibiotic stewardship, the clinic demonstrated appropriate use of antibiotics at more than double the rate of traditional telemedicine and in-person care.
We acknowledge, with deep gratitude, the contributions made to this project and manuscript by Tess Forte, Alex Koh, and Travis Thompson.
David McCune, the primary and corresponding author, was responsible for the background research and writing the manuscript. James Pellegrin was primarily responsible for statistical analyses. Anshul Sachdeva collected and collated data from 98point6 encounters. Roxana Cham provides leadership for clinical quality at 98point6 and provided guidance for the inception and interpretation of the study. Jessica Sollaccio provided critical revision of the project. Sandra Giramahoro Coyne contributed to revision and final approval of the manuscript. Mark Stewart contributed to revisions of the manuscript, data collection, and analysis.
No funding was acquired or necessary for this study.
The authors declare no potential conflicts of interest and acknowledge all authors were employed at 98point6 during the writing and at publication of the article.