David McCune,1 Erin Graf,1 Jo Masterson,2 Amanda Cuda1
Affiliations: 198point6, 701 5th Ave, Suite 2300, Seattle, WA 98104; 22Morrow, Inc., 12020 113th Ave NE Suite 295, Kirkland, WA 98034
Corresponding Author: David McCune, Email: dmccune@98point6
Keywords: Acceptance and Commitment Therapy, Anxiety, App, Artificial Intelligence, Behavioral Health, Machine Learning, mHealth, Pain, Primary Care, Stress, Telemedicine, Tobacco, Virtual, Weight
Category: Research
This pilot program tested the feasibility of connecting patients receiving medical care from 98point6 (a virtual healthcare platform that leverages artificial intelligence and machine learning to deliver primary care through a text-based, smartphone interface) with the 2Morrow app (a behavioral health platform that utilizes acceptance commitment therapy [ACT] to address key chronic disease drivers and conditions). It also evaluated the potential for future clinical research using this methodology.
Materials and methods: Adult patients of 98point6 who expressed one of several behavioral health concerns (tobacco addiction, pain, stress/anxiety, or weight management) were evaluated by a board-certified primary care physician. They were subsequently given the recommendation to download the 2Morrow app and use one of the ACT-based programs at no cost. De-identified data were collected about 2Morrow app usage. Surveys were sent to patients via email at 7 and 28 days post-recommendation to collect information about their experience using the app.
Results: From April to October 2018, 98point6 and 2Morrow conducted a pilot program to offer the 2Morrow app to patients who expressed concern about any of the following topics: tobacco addiction, pain, stress/anxiety, or weight management. 98point6 providers recommended the 2Morrow app to 28 patients, 16 of whom downloaded the app and used at least one 2Morrow program. Five patients used multiple modules within the 2Morrow app. De-identified data are presented regarding those patients who used the app and which modules were accessed and with what frequency. Four patients completed the post-recommendation user surveys.
Conclusions: The 98point6 virtual healthcare patients showed good adherence to the recommendation for ACT delivered through a mobile app. Patients followed through on physician recommendations at rates equivalent to similar published studies. A clinical trial to measure patient satisfaction, compliance, and outcomes is feasible. The ability to treat several behavioral health concerns commonly seen in primary care using an entirely digital healthcare plan deserves additional study.
Mental illness, unhealthy behavior, and behavioral health (collectively termed as “BH”) conditions are common. The 2016 National Survey on Drug Use and Health indicated a diagnosis of “any mental illness” to be prevalent in 44.7 million Americans.1 In particular, tobacco addiction, emotional stress, concerns about weight, and complaints of somatic pain make up a substantial portion of primary care patient visits.2 Despite these statistics, successful screening and treatment of BH concerns is challenging in a primary care setting, and one study found that only 41% of adults with mental illness received treatment in the past year.3
Smoking is the leading preventable cause of death, and quitting greatly reduces the risk of tobacco-related illnesses.4 Assisting patients in quitting tobacco is an important role for primary care, and physician involvement in patient efforts to stop smoking is beneficial.5 One common model is the “5 A’s” (ask, advise, assess, assist, and arrange), in which physicians ask patients about tobacco use, advise to stop smoking, assess readiness to quit, assist in obtaining resources, and arrange follow-up evaluation.6 An alternative strategy—ask, advise, connect—showed a higher success rate, and this approach readily lends itself to a digital care model.7
Anxiety disorders of all types are increasing in incidence, including in younger adults.8 Recent data from the National Institute of Mental Health indicate that the incidence of treatment for any anxiety disorder in the previous year is >20% for adults aged 18–59 years.9 Self-reported stress and anxiety are important components of the diagnosis of anxiety disorders.10
Pain also has a high prevalence, with more than 126 million Americans reporting pain in the past 3 months and 11.3% reporting chronic daily pain.11 The management of pain is a complex issue, and while it is not traditionally considered a behavioral health issue, evidence suggests that outcomes are improved when both pharmacological and nonpharmacological approaches are used. According to the Centers for Disease Control and Prevention (CDC), nonpharmacologic therapy (including behavioral therapy) and nonopioid pharmacologic therapy are the preferred treatment for managing chronic pain,12 yet behavioral therapy is seldom offered to patients. For most chronic pain sufferers, the absence of pain is not a likely outcome, and helping patients set realistic treatment and outcome goals for pain and function is important.13 Behavioral therapy can help people learn better coping skills that increase valued activity and improve quality of life.14
Finally, multiple behavioral health conditions can present at the same primary care visit. Patients with pain tend to have stress, or their stress can exacerbate their pain.15 Weight gain is a common concern with tobacco cessation,16 and stress can have a negative effect on stopping smoking.17 For these reasons, the flexibility to treat one or more behavioral health concerns is a desirable feature of any digital behavioral health intervention.
Acceptance commitment therapy (ACT) is a third-wave cognitive behavioral therapy (CBT) model that is especially helpful in dealing with unwanted or unpleasant thoughts, feelings, or urges.18 This therapy encourages patients to change health behaviors and improve coping skills and resilience. There are more than 200 randomized control trials published on ACT in a variety of focus areas. In particular, ACT has shown evidence of benefits in the treatment of tobacco addiction,19 stress and anxiety,20 pain,21 and weight management.22 ACT engages with brief interventions that providers use in primary care settings,20 making the approach a good fit for a virtual primary care model.
Virtual healthcare has been recommended for greater use in the evaluation and treatment of behavioral health.23 However, its employment has been impeded by technical challenges and slow implementation. Smartphone-based mental health apps represent a unique opportunity to expand the availability and quality of mental health treatment. The number of smartphone apps focused on mental health has rapidly increased: a 2015 World Health Organization (WHO) survey found more than 15,000 apps, with 29% focused on mental health diagnosis, treatment, or support.24 Despite this, there are limited data on effectiveness. A 2013 review identified more than 1,500 depression-related apps in commercial app stores but only 32 published research papers on the subject.25
98point6 is a Seattle-based start-up that connects patients to board-certified primary care physicians from wherever they are (at home, on the bus, or at work) via a private, text-based experience. By addressing common issues associated with primary care delivery through a service that combines quality, accessibility, and affordability, the company aims to inspire early and frequent use that leads to better overall health in individuals and populations. 98point6 is available to patients from any location in the United States, with the ability to use voice or video if necessary.
2Morrow Inc. has created a digital health app that includes behavioral science-based modules for self-treatment using ACT. The modules target tobacco cessation, stress and anxiety management, non-narcotic pain management, and weight loss. Clinical trials have shown a beneficial effect of using the app, including in a randomized, controlled trial of tobacco cessation.26 Research is ongoing in weight management and pain management. While there are no studies published using the novel method of virtual healthcare provider recommending app-based therapy, a study of web-based tobacco cessation teaching had a follow-up rate of 58.4% at 3 months.27 Another study that offered patients a choice of a telephone “quitline” or web-based intervention showed that 48% of patients followed through on the recommendation, and an additional 29% of those did not interact with the service.28
This feasibility study was a prospective internal pilot designed to test connecting 98point6 patients with other digital tools and resources. 98point6 provides treatment for a broad spectrum of acute care issues; however, there is a need for additional resources to fully address behavioral health conditions. In this pilot, physicians recommended the use of the 2Morrow app in order to provide a wider breadth of support for issues seen within the 98point6 patient population.
Participants were adults who initiated an encounter with 98point6 for any issue regarding their health. They reported their primary concern in their own words, and they were asked a series of questions relevant to that concern. The patient was then connected to the 98point6 physician, who continued the patient interview. The following process was used to evaluate patients and recommend the 2Morrow app:
After the visit, the patient received an electronic copy of the Care Plan that included a link to the 2Morrow pilot. They then exited the 98point6 app and registered on a pilot registration webpage. After registration, participants needed to download the 2Morrow app. Note that participants did not need to enter any personal information such as a name or email but were assigned a unique username and password to activate the program.
Targeted advertising was used in an attempt to inform patients about the availability of care at 98point6, but there was no mechanism at the time to identify the rate of conversion from advertisement to 98point6 app download. The pilot ran from April 9 to October 8, 2018.
The 98point6 service is available to adult patients in the United States through individual subscription or an employer-sponsored health plan. For this pilot, 98point6 physicians identified patients for recommendation of the 2Morrow app. Patients were considered appropriate for recommendation if they expressed a desire to quit smoking or if they indicated a concern with pain, stress/anxiety, or weight management. If the physician determined that the patient was a candidate for app-based ACT, the patient was given a medical care plan that included instructions to download the 2Morrow app.
The primary aim of the pilot program was to assess the feasibility of the recommendation program between 98point6 and 2Morrow. Additional endpoints included patient satisfaction with the recommendation process and the 2Morrow app. Descriptive statistics are presented.
During the course of the pilot, 98point6 had access to the following de-identified usage statistics from 2Morrow via a website dashboard: number of 98point6 patients who downloaded the app, which modules used, and the progress made by patients in each module. 98point6 surveyed patients at 7 and 28 days after the visit to determine satisfaction with the recommendation by 98point6, the usefulness of the app in addressing their health concern, and overall app experience.
Institutional Review Board review and approval was determined by an internal counsel as not required for this study. No attempt was made to conduct research as defined by HHS regulations as “a systematic investigation, including research development, testing and evaluation, designed to develop, or contribute to generalizable knowledge.”
A total of 28 patients were assessed as appropriate candidates for the 2Morrow recommendation. Of the 28 patients, 16 (57%) downloaded and used the 2Morrow app. Five (18% of the total recommended or 31% of those using the app) patients used multiple modules. The average age of patients who received a recommendation was 36 years, and the median age was 33 years. Patient demographics are presented in Table 1, and patient usage rates are presented in Table 2. A breakdown of modules used by the five patients who used multiple modules is given in Table 3.
Table 1. Patient demographics.
| Variables | Number | Percentage |
|---|---|---|
| Gender | ||
| Male | 13 | 46 |
| Female | 15 | 54 |
| Age (year) | ||
| 18–24 | 2 | 7 |
| 25–34 | 14 | 50 |
| 35–44 | 6 | 21 |
| 45–64 | 6 | 21 |
| 65+ | 0 | 0 |
Table 2. Patient usage rates.
| Patient Use Category | Module Name | |||
|---|---|---|---|---|
| Smoking Cessation | Chronic Pain Management | Weight Management | Stress Management | |
| Recommendation (n) | 9 | 7* | 4 | 9* |
| Usage (n)** | 5 | 2 | 7 | 7 |
*One patient was recommended the use of two modules (chronic pain and stress).
**Usage rates reflect some patients who used multiple modules.
Table 3. Multiple module use breakdown*.
| Patient Number | Smoking Cessation | Chronic Pain Management | Weight Management | Stress Management |
|---|---|---|---|---|
| 1 | X | X | ||
| 2 | X | X | ||
| 3 | X | X | X | |
| 4 | X | X | ||
| 5 | X | X | X |
*Because data were de-identified and unique patient identifiers were not used, it was not possible to determine what module the physician recommended to these patients. (X = indicates the patient used the module. Blank indicates no use of the module).
Surveys were sent to patients at 7 and 28 days after the recommendation. Four surveys were returned. Within this limited data set, two of the four patients reported an excellent experience, and two of the four patients reported an unsatisfactory experience.
In an increasingly on-demand, digitally empowered culture, consumer expectations have changed around convenience and immediate delivery of services, including medical care. As more patients become accustomed to virtual healthcare options, there is a need for well-validated assessments of this trend. Due to its prevalence and ongoing treatment challenges in the primary care setting, behavioral health is a worthwhile area for studying the incorporation of virtual healthcare and mHealth applications.
This pilot project demonstrated the feasibility of referring patients from one virtual healthcare portal to an mHealth application as a treatment for BH concerns of smoking cessation, pain management, stress/anxiety, and weight management. The 98point6 patients who received the recommendation to use 2Morrow downloaded and activated the app at a 57% rate. Despite having to navigate a complicated process (no system integration was done for this pilot), the follow-through rate on the recommendation was comparable to the reported adherence achieved in a large, web-based tobacco cessation program.28 This rate of compliance was achieved using a prototype process that required very few resources. Additional investigation is warranted.
The high rate of successful follow-through in the project is encouraging, particularly in light of the fact that there were several steps in the process during which a patient could abandon it. The compliance with the recommendation to use 2Morrow in this setting suggests that physician engagement in the process of using an mHealth app can play an important role. In addition, five of 16 patients who downloaded the app used more than one module. This is consistent with the literature that suggests a high degree of behavioral health comorbidity. An mHealth solution that can be used for multiple BH concerns is useful in this setting. As a point of comparison, the percentage of 98point6 patients who followed the recommendation was similar or superior to compliance rates in other published studies of web-based tobacco-cessation interventions.27,28
Several limitations of this pilot must be noted. First, the results are specific to this setting: the pilot was only designed to assess the feasibility of recommending an mHealth app to patients receiving virtual care from 98point6 to 2Morrow. Therefore, the results cannot be generalized to other settings.
Second, there was potential for bias. The entry criteria were not standardized. Eligibility was based on patient self-report of health concerns and the clinical judgment of the referring physician. There was no blinding and no control group. Without formal entry criteria, it is possible that the subset of patients who received the recommendation were more predisposed to follow-through than a general population.
Third, the sample size was small, and the pilot was of sort duration. We found that 28 referrals were sufficient to conclude that the process is feasible. However, this sample size limits conclusions about effectiveness. In addition, the follow-up process was limited to 28 days. This was insufficient to assess the potential for long-term benefits of using the program.
Fourth, research indicates that physician involvement in quitting smoking can have a beneficial effect. This pilot included no physicians for patient follow-up. This may also be reflected in the low rate of return of the post-visit surveys.
Finally, the process required the physician to bring together information from multiple sources to complete a successful recommendation. This may have decreased the number of successful recommendations.
The authors conclude that the process of using virtual primary care with 98point6 to refer a patient to a digital therapeutic is feasible, but the limitations discussed above must be addressed in any future work. A clinical trial to address these limitations is under development. It will include clearer inclusion and exclusion criteria, a longer assessment period, better defined follow-up survey procedures, and a technically smoother referral process. The follow-up trial will measure the clinical and cost outcomes of this process and will make generalizable conclusions.
In general, this project supports the feasibility of future studies that would combine virtual healthcare with specialized mHealth applications. The compliance rate occurred despite using a process that was not fully automated. As noted in the review of literature, the speed of development of digital innovation makes effective evaluation of mHealth apps a challenge. Using virtual healthcare for the entire process of identification, enrollment, treatment, and follow-up reduces infrastructure costs and the time required to complete the project. Future clinical trials should expand on this capability.
Authors’ contributions: David McCune, MD, MPH, is the primary and corresponding author and was responsible for the study design. Erin Graf is the secondary author and was responsible for data collection and analysis through the 98point6 platform and the study design. Jo Masterson was responsible for data collection through the 2Morrow platform, study design, and manuscript editing. Amanda Cuda, MD, MPH, is the secondary author, primary referring physician, and was responsible for the study design.
Acknowledgments: We acknowledge the following people for their contributions to this project: David H. Dickey, MD, Andrew T. Le, MD, Diana P. Lewis, MD, Thomas M. McCloy, MD: all 98point6 physicians (who recommended the 2Morrow program); J. Claire Carbary, JD, CIPP-US: 98point6 Legal Counsel (who performed legal review and assisted with project design and copyediting); Linda Pourmassina, MD: 98point6 Product Physician (who assisted with project design and supplementary material drafting/editing); Gaurai Uddanwadiker, MHA, and Tommi Rainey, MHA: 98point6 Clinical Operations (who assisted with patient tracking, reporting, and follow-up survey results write-up); Tess Forte and Jenna Walcott: 98point6 Marketing/Copyediting; the rest of the 98point6 team; and the rest of the 2Morrow, Inc., team.
Funding Statement
This study was funded by 98point6 and 2Morrow, Inc. The patients were provided with access to 2Morrow’s service at no cost.
Financial and non-Financial Relationship and Activities
The authors declare no potential conflicts of interest and acknowledge all authors were employed at 98point6 during the writing and at publication of the article.
| 1. | Ahrnsbrak R. Substance abuse and mental health services administration. Key substance use and mental health indicators in the United States: Results from the 2016 National Survey on Drug Use and Health (HHS Publication No. SMA 17-5044, NSDUH Series H-52). Rockville, MD: Center for Behavioral Health Statistics and Quality, Substance Abuse and Mental Health Services Administration; 2017. |
| 2. | Gillespie CD, Hurvitz KA, Centers for Disease Control and Prevention (CDC). Prevalence of hypertension and controlled hypertension—United States, 2007-2010. MMWR Suppl. 2013 Nov 22;62(3):144–8. |
| 3. | Center for Behavioral Health Statistics and Quality. Behavioral health trends in the United States: Results from the 2014 National Survey on Drug Use and Health (HHS Publication No. SMA 15-4927, NSDUH Series H-50). 2015. https://www.samhsa.gov/data/sites/default/files/NSDUH-FRR1-2014/NSDUH-FRR1-2014.pdf |
| 4. | West R. Tobacco smoking: Health impact, prevalence, correlates and interventions. Psychol Health. 2017 Aug 3;32(8):1018–36. |
| 5. | Lancaster T, Stead L. Physician advice for smoking cessation. Cochrane Database Syst Rev. 2004 Oct 18;(4):CD000165. |
| 6. | Siu AL, U.S. Preventive Services Task Force. Behavioral and pharmacotherapy interventions for tobacco smoking cessation in adults, including pregnant women: U.S. Preventive Services Task Force Recommendation Statement. Ann Intern Med. 2015 Oct 20;163(8):622–34. |
| 7. | Vitrine JI, Sheet S, Cao Y, et al. Ask-advise-connect: A new approach to smoking treatment delivery in health care settings. JAMA Intern Med. 2013 Mar 25;173(6): 458–64. |
| 8. | Mujtaba R, Olson M, Han B. National trends in the prevalence and treatment of depression in adolescents and young adults. Pediatrics. 2016 Dec 1;138(6):e20161878. |
| 9. | Alegria M, Jackson JS. Kessler RC, Takeuchi D. Collaborative Psychiatric Epidemiology Surveys (CPES), 2001–2003 [United States]. Ann Arbor, MI: Inter-university Consortium for Political and Social Research [distributor]; 2016. https://doi.org/10.3886/ICPSR20240.v8 |
| 10. | Bjelland I, Dahl AA, Haug TT, Neckelmann D. The validity of the hospital anxiety and depression scale. An updated literature review. J Psychosom Res. 2002 Feb;52(2):69–77. |
| 11. | Nahin RL. Estimates of pain prevalence and severity in adults: United States, 2012. J Pain. 2015 Aug;16(8):769–80. |
| 12. | Dowell D, Haegerich TM, Chou R. CDC Guideline for Prescribing Opioids for Chronic Pain — United States, 2016. MMWR Recomm Rep 2016;65 (No. RR-1): 1–49. DOI: http://dx.doi.org/10.15585/mmwr.rr6501e1. |
| 13. | Veehof MM, Oskam M-J, Schreurs KMG, Bohlmeijer ET. Acceptance-based interventions for the treatment of chronic paIn: A systematic review and meta-analysis. PAIN. 2011 Mar 1;152(3): 533–42. |
| 14. | Vowles KE, McCracken LM. Acceptance and values-based action in chronic paIn: A study of treatment effectiveness and process. J Consult Clin Psychol. 2008;76(3):397–407. |
| 15. | Bair M. Association of depression and anxiety alone and in combination with chronic musculoskeletal pain in primary care patients. Psychosom Med. 2008 Oct;70(8):890–7. |
| 16. | Aubin H-J. Weight gain in smokers after quitting cigarettes: Meta-analysis. BMJ. 2012(345):e4439. |
| 17. | Abrantes A. The role of negative affect in risk for early relapse among low distress tolerance smokers. Addict Behav. 2008 Nov;33(11):1394–401. |
| 18. | Hayes SC, Strosahl KD, Wilson KG. Acceptance and commitment therapy: An experiential approach to behavior change. New York, NY, US: Guilford Press; 1999, xvi, 304. |
| 19. | Mak YW, Loke AY. The acceptance and commitment therapy for smoking cessation in the primary health care setting: A study protocol. BMC Public Health. 2015 Feb 7 [cited 2019 Jan 25];15. Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4389806/ |
| 20. | Forman EM, Herbert JD, Moitra E, Yeomans PD, Geller PA. A randomized controlled effectiveness trial of acceptance and commitment therapy and cognitive therapy for anxiety and depression. Behav Modif. 2007 Nov;31(6):772–99. |
| 21. | Dahl J, Wilson KG, Nilsson A. Acceptance and commitment therapy and the treatment of persons at risk for long-term disability resulting from stress and pain symptoms: A preliminary randomized trial. Behav Ther. 2004;35(4):785–801. |
| 22. | Lillis J, Kendra KE. Acceptance and commitment therapy for weight control: Model, evidence, and future directions. J Contextual Behav Sci. 2014 Jan;3(1):1–7. |
| 23. | Langarizadeh M, Tabatabaei MS, Tavakol K, et al. Telemental health care, an effective alternative to conventional mental care: A systematic review. Acta Inform Med. 2017 Dec;25(4):240–6. |
| 24. | Anthes E. Mental health: There’s an app for that. Nature. 2016 Apr 7;532(7597): 20–3. |
| 25. | Martínez-Pérez B, de la Torre-Díez I, López-Coronado M. Mobile health applications for the most prevalent conditions by the World Health Organization: review and analysis. J Med Internet Res. 2013 Jun 14;15(6):e120. |
| 26. | Zeng EY, Heffner JL, Copeland WK, Mull KE, Bricker JB. Get with the program: Adherence to a smartphone app for smoking cessation. Addict Behav. 2016 Dec 1;63:120–4. |
| 27. | Graham AL, Papandonatos GD, Cha S, et al. Improving adherence to smoking cessation treatment: Intervention effects in a web-based randomized trial. Nicotine Tob Res. 2017 Mar 1;19(3):324–32. |
| 28. | Japuntich SJ, Zehner ME, Smith SS, et al. Smoking cessation via the internet: A randomized clinical trial of an internet intervention as adjuvant treatment in a smoking cessation intervention. Nicotine Tob Res. 2006 Dec;8 Suppl 1:S59–67. |
Copyright Ownership: This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, adapt, enhance this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0.