ORIGINAL RESEARCH

Sustainable Virtual Care in Ontario’s Health System: A Quality Metrics Comparison

Jakarinya Mangalamoorthy, MD¹; William Cherniak, MD, MPH²; Brian Geller, BSc, MD, MBA³; Ryan Tam, MD, MPH⁴

¹Rocket Doctor Research Assistant, Toronto, Ontario, Canada; ²Division of Emergency Medicine, Department of Family & Community Medicine, University of Toronto; Rocket Doctor, Founder & CEO, Toronto, Ontario, Canada; ³Clinical Associate Professor, University of Saskatchewan, Saskatoon, Saskatchewan, Canada; Department of Family Medicine, Rocket Doctor Medical Director, Toronto, Ontario, Canada; ⁴Department of Medicine, University of Toronto, Toronto, Canada; Rocket Doctor Medical Advisor, Toronto, Ontario, Canada

Keywords: direct-to-patient virtual visit, direct-to-physician virtual platform, quality of care, telemedicine, virtual care

Abstract

Objective: The lack of pre-existing public virtual care platforms in Ontario at the start of the COVID-19 pandemic led to the generation of many private companies employing physicians in corporate practice groups. The quality of care provided remains unknown. This study assesses the quality of care provided by physicians through a virtual care platform that focuses on independent physicians providing coordinated care within the public sector.

Methodology: A prospective self-directed survey was distributed to Ontario patients who received virtual care visits through the Rocket Doctor platform between October 18, 2022 and November 30, 2022. Quality indicators were selected and benchmarked against Health Quality Ontario’s (HQO) quality framework. Additional indicators of quality were measured using internal platform metrics.

Results: A total of 2276 patients completed the survey out of 14,324 who completed virtual care encounters. Among the respondents, 40% reported using physicians on Rocket Doctor as their primary care provider. Compared with in-person primary care clinics, patients were able to book appointments on the same day or the next day, 74% versus 41% of the time. In addition, the platform was able to book an appointment on the same day or the next day with a specialist 86% of the time. Physicians were able to resolve or provide follow-up on 94% of encounters, with only 1% of patients referred to the emergency department.

Conclusion: Quality of care metrics overall met and surpassed comparable HQO indicators. Services provided were more timely, efficient, and effective. High-quality virtual care can be effectively delivered through a private technology platform built on public health insurance and integrated into Canada’s healthcare system.

Plain Language Summary

Delivery of care through a virtual platform in Canada is novel and expanded rapidly in response to the COVID-19 pandemic. While the medical care provided is publicly funded, private companies manage the virtual care platforms. The quality of care provided in this unique, novel system was currently unknown and much needed to help inform Canada’s health policies on virtual care. This patient survey study looked at the quality of care provided on one of these virtual care platforms and compared it with provincial quality standards. Patients were able to be seen effectively, timely, and efficiently above provincial quality standards. Virtual care platforms can help provide high-quality clinical care and sustainably integrate into the Canadian healthcare system.

 

 

In 2020, Canada lacked a federally or provincially coordinated framework for reimbursement of virtual physician services, and there were no universal healthcare information technologies (HITs) to support virtual care services delivery.^1–3 This is in contrast to the United States, where continuous implementation of the 2009 HITECH Act and 21st Century Cures Act established a framework and infrastructure for virtual care delivery.⁴

This lack of a coordinated framework and HIT resulted in limited virtual care adoptions before the pandemic, with less than 1% of all patient visits conducted virtually before March 2020 and a push in the fall of 2019 by the Canadian Medical Association to begin modernization of the healthcare system.^5–7

Despite the lack of coordinated frameworks and HIT investment, effective and sustainable virtual care models did exist in Canada. The Ontario Telemedicine Network (OTN) has been established for over a decade providing acute and chronic care to rural and indigenous communities across many rural and remote regions of Ontario.⁸ In the Spring of 2020, temporary COVID-19 pandemic physician billing codes were introduced, suddenly allowing the emergence of virtual care for all.^9,10

The demand for services and the lack of HIT infrastructure created an extreme need for patients and providers. This caused a vacuum for private companies to create novel care delivery models. In Ontario, as of 2022, approximately 10% of virtual care was provided through private companies.1-3 Some companies have been noted to charge additional platform fees, and debate has been ongoing around the role of these policies in the Canadian healthcare system.¹⁰

Since the completion of this study in December 2022, rates of remuneration for virtual care visits have been reduced by up to 50% in Ontario.¹¹

While retrospective studies analyzing billing codes and usage data have been published from blended public/private virtual walk-in clinics, corporate entities, and individual family doctor offices,^12–14 to our knowledge, no studies have been conducted on the quality of care provided by independent physicians on a dedicated direct-to-patient virtual platform, entirely supported by the public sector.

We explored how a publicly funded clinical care model conducted on a privately supported virtual care platform performed compared with provincial standards for quality-of-care metrics before the implementation of funding cuts in Ontario in December 2022.

Methods

A prospective self-directed survey was distributed to 14,322 Ontario patients who received virtual care from physicians by a Canadian digital health company (Rocket Doctor Inc.) between October 18, 2022 and November 30, 2022.

Rocket Doctor is a digital health platform and marketplace that intelligently matches patients with the best clinician for their needs while utilizing AI-fueled software, a unique support system, and remote medical devices. Like a Shopify for providers, the company enables doctors with the tools and resources they need to build and run successful independent virtual practices and coordinate a system of care with a single patient-facing brand identity, largely directed to patients on Medicaid in the United States and in rural/remote communities across Canada.

The platform provides direct-to-patient and direct-to-physician access to complete a clinical encounter through video, audio, and/or in-person care. Patient visits in this study were fully virtual and remunerated under the single-payer provincial health insurance that had been expanded under COVID-19 policies. Fees for technological access were taken from physicians.

Unlike traditional telemedicine, Rocket Doctor empowers physicians to run their own virtual practices, such that it is not a walk-in clinic. In this way, physicians can order blood work and imaging, make specialist referrals, book follow-up visits, and repeatedly see their patients. Rather than sourcing locums while away, the platform provides flexibility in virtual primary care, with hundreds of colleagues supporting one another.

Health Quality Ontario (HQO) utilizes the six domains of framework to generate indicators to measure and track the quality of care delivered from the hospital, outpatient, and to long-term care settings in the province.¹⁵ A review of the HQO indicators comparable to virtual care was conducted and used as benchmarks to measure the quality of care provided by Physicians on Rocket Doctor.

The indicators selected were measured through internal platform statistics and a post-encounter patient survey. Surveys were e-mailed to patients seen in Ontario within an hour after every patient encounter. Internal platform statistics were extracted by data queries from software engineers utilizing a SQL database, with engineers blinded to the objectives of the study. The University of Toronto Research Ethics Board approved the study protocol (#43064.)

Results

There were 14,324 virtual patient encounters completed in Ontario during the study period, with 2276 (16%) patients who completed the survey. The technology platform automatically sent surveys once after a visit was completed, with no subsequent follow-up to patients to complete the survey.

Seven indicators were selected, with three comparable to HQO’s indicators. Compared with 2019, 44% more patients reported not having a family physician or other primary care provider for regular check-ups in 2022 (Figure 1). A total of 35% of patients reported using Rocket Doctor as their primary care provider (Figure 2).

Fig. 1. Comparison of having a family doctor or other primary care provider between Health Quality Ontario (HQO) in 2019¹⁶ and Rocket Doctor Services (RDS) in 2022.

Fig. 2. Use of physicians on Rocket Doctor as a primary care provider (PCP).

Compared with HQO in-person primary care clinics,¹⁶ Rocket Doctor booked an appointment on the same day or the next day—74% versus 41% of the time (Figure 3). Rocket Doctor connected patients with physicians 32% of the time within 8 hours and 74% within 24 hours from the time of patient request (Figure 4). Specialist appointments with patients were booked on the Rocket Doctor platform 86% of the time, the same or the next day (Figure 5).

Fig. 3. Comparison between time to primary care appointment on the same day or the next day when sick: Health Quality Ontario (HQO) in 2019¹⁶ versus Rocket Doctor Services (RDS) in 2022.

Fig. 4. Time to all appointments on Rocket Doctor platform.

Fig. 5. Time to specialists appointments on the Rocket Doctor platform. ED: emergency department; PCP: primary care physician

Rocket Doctor resolved or provided follow-up on 94% of encounters, with 1% referral to the emergency department (Figure 6). The most common referrals to specialists on the Rocket Doctor platform included psychiatry, dermatology, gastroenterology, gynecology, and urology (Figure 7).

Fig. 6. Patient survey asking where patients went after being seen on the Rocket Doctor platform

Fig. 7. Specialist referrals through the Rocket Doctor platform

Discussion

As a novel concept in a previously unexplored area of healthcare delivery, there are limitations and considerations when interpreting these data. Foremost, HQO has not conducted its quality metrics since 2019, and therefore, comparisons have the added uncertainty of the impact of the COVID-19 pandemic. Access to the healthcare system since the pandemic is unlikely to have improved and favors the validity of the comparisons made.

A decrease in family physicians was seen in our study population. In addition, the use of virtual care as a replacement for a family physician is a trend, particularly in the population of patients included in this study.

The ability of virtual care to improve access to care through timeliness to appointments and specialists was noticed and consistently found across different healthcare systems and delivery models in the literature.^{12–14,17,18} The response rate in this study is lower than traditional research project expectations. However, response rates depend heavily on the population, follow-up, and survey type.¹⁹ Given the novelty of this study population, a benchmark against general online survey response rates might be a more comparable standard.

Compared with other online survey-only response rates, a response rate of 10%, while low, is within the expected range, particularly given the lack of incentive and multiple follow-ups often offered to respondents in in-person studies, which were thought to be potentially biased toward more positive responses.¹⁹ We believe that in light of the sample size of 14,322 visits on the platform during the study, the results reported here are representative of the larger population across the province.

The novelty of the platform engaging a direct-to-patient model means that there is neither any comparison data from other platforms at the time of publication of this study nor any available data on the total virtual visits completed in the province. Finally, race and ethnicity data were not routinely collected on the platform and are not a standard metric tracked in self-reporting online surveys in Canada.

Conclusion

Rocket Doctor had a virtual care platform model in Ontario during the pandemic providing access to over 10,000 publicly funded patient visits and hundreds of providers a month. It provided superior timeliness, patient-centered, effective, and efficient markers of quality care based entirely on publicly funded health insurance when benchmarked to existing health system indicators.

Continuation to develop similar health information technology solutions with the existing Ontario healthcare system should be prioritized as the Canadian system continues to cope with increasing barriers to healthcare access post-COVID-19 pandemic.^20,21 Although the system has changed in Ontario, this model remains sustainably operational in British Columbia and Alberta, where publicly funded virtual care has been matched to in-person care.

Contributors

Jakarinya Mangalamoorthy conducted the literature review, ethics, study design, and manuscript preparation. William Cherniak contributed to the study design, analysis, manuscript preparation, and ethical considerations. Brian Geller was responsible for study analysis and manuscript preparation. Ryan Tam contributed to the study conception, design, manuscript preparation, and analysis, in addition to the literature review and ethical considerations.

Acknowledgments

We would like to acknowledge the engineering team, in particular Gene Ryuzaki and Anthony Poon, for providing their voluntary time to extract the various data required for this study. We would like to acknowledge Freddy Drucquer for his voluntary time in designing the graphics used in the study.

References

1. Patterson PB, Roddick J, Pollack CA, Dutton DJ. Virtual care and the influence of a pandemic: necessary policy shifts to drive digital innovation in healthcare. Healthc Manage Forum. 2022 Jun 30;35(5):272–8. doi: 10.1177/08404704221110084
2. Canadian Institute for Health Information. Virtual care in Canada | CIHI [Internet]. www.cihi.ca. [cited 2022 Oct 21]. Available from: https://www.cihi.ca/en/virtual-care-in-canada
3. Canada Health Infoway. Canadian digital health survey 2021: health innovation & artificial intelligence | Canada Health Infoway [Internet]. www.infoway-inforoute.ca. 2021 [cited 2022 Oct 21]. Available from: https://www.infoway-inforoute.ca/en/component/edocman/resources/reports/benefits-evaluation/4018-canadian-digital-health-survey-2021-health-innovation-artificial-intelligence
4. Gettinger A, Zayas-Cabán T. HITECH to 21st century cures: clinician burden and evolving health IT policy. J Am Med Inform Assoc. 2021 Feb 12;28(5):1022–5. doi: 10.1093/jamia/ocaa330
5. Bhyat R, Hagens S, Bryski K, Kohlmaier JF. Digital health value realization through active change efforts. Front Public Health. 2021 Oct 12;9: 741424. doi: 10.3389/fpubh.2021.741424
6. Canadian Medical Association. Virtual care in Canada: discussion paper [Internet]. Canadian Medical Association; 2019 [cited 2023 Jul 11]. Available from: https://www.cma.ca/virtual-care-canada-discussion-paper
7. Canadian Medical Association. Virtual care in Canada: progress and potential report of the virtual care task force [Internet]. Canadian Medical Association; 2022 Aug [cited 2023 Jul 11]. Available from: https://www.cma.ca/sites/default/files/2022-02/Virtual-Care-in-Canada-Progress-and-Potential-EN.pdf
8. Brown EM. The Ontario Telemedicine Network: a case report. Telemed J e-Health. 2013 May;19(5):373–6. doi: 10.1089/tmj.2012.0299
9. Webster P. Virtual health care in the era of COVID-19. Lancet. 2020 Apr;395(10231):1180–1. doi: 10.1016/S0140-6736(20)30818-7
10. Canadian Medical Association. Understanding public and private health care [Internet]. Canadian Medical Association. 2023 [cited 2023 Aug 20] Available from: https://www.cma.ca/our-focus/public-and-private-health-care/understanding-public-and-private-health-care
11. Government of Ontario. Bulletin 221203—virtual health care in Ontario | OHIP INFOBulletins 2022 | ontario.ca [Internet]. www.ontario.ca; 2023 [cited 2023 Jul 11]. Available from: https://www.ontario.ca/document/ohip-infobulletins-2022/bulletin-221203-virtual-health-care-ontario
12. Stamenova V, Chu C, Pang A, Tadrous M, Bhatia RS, Cram P. Using administrative data to explore potentially aberrant provision of virtual care during COVID-19: retrospective cohort study of Ontario provincial data. J Med Internet Res. 2021 Sep 7;23(9):e29396. doi: 10.2196/29396
13. Kiran T, Green ME, Strauss R, Wu F, Daneshvarfard M, Kopp A, et al. Virtual care and emergency department use during the COVID-19 pandemic among patients of family physicians in Ontario, Canada. JAMA Netw Open. 2023 Apr 28;6(4):e239602. doi: 10.1001/jamanetworkopen.2023.9602
14. Ryan BL, Brown JB, Freeman TR, Richard L, Stewart M, Meredith L, et al. Virtual family physician care during COVID-19: a mixed methods study using health administrative data and qualitative interviews. BMC Prim Care. 2022 Nov 25;23(1):300. doi: 10.1186/s12875-022-01902-9
15. U.S Department of Health and Human Services. Six domains of health care quality [Internet]. Agency for Healthcare Research and Quality; 2022 [cited 2023 Jul 11]. Available from: https://www.ahrq.gov/talkingquality/measures/six-domains.html
16. Health Quality Ontario. Indicator library—Health Quality Ontario (HQO) [Internet]. www.hqontario.ca. [cited 2022 Oct 21]. Available from: https://www.hqontario.ca/System-Performance/Measuring-System-Performance/Indicator-Library
17. Zhou X, Snoswell CL, Harding LE, Bambling M, Edirippulige S, Bai X, et al. The role of telehealth in reducing the mental health burden from COVID-19. Telemed J e-Health. 2020 Mar 23;26(4):377–9. doi: 10.1089/tmj.2020.0068
18. Dawson AZ, Walker RJ, Campbell JA, Davidson TM, Egede LE. Telehealth and indigenous populations around the world: a systematic review on current modalities for physical and mental health. MHealth. 2020 Mar 17;6:30. doi: 10.21037/mhealth.2019.12.03
19. Wu MJ, Zhao K, Fils-Aime F. Response rates of online surveys in published research: a meta-analysis. Comput Hum Behav Rep. 2022 May;7(2):100206. doi: 10.1016/j.chbr.2022.100206
20. Winsa P. Why Ontario is reducing doctors’ payments for one-off virtual appointments and what it could mean for your health care. Toronto Star [Internet]. 2022 Oct 18 [cited 2022 Oct 21]. Available from: https://www.thestar.com/news/gta/why-ontario-is-reducing-doctors-payments-for-one-off-virtual-appointments-and-what-it-could/article_1b5d2c0f-8b4a-5c96-86e6-76da71b38114.html?
21. Casey L. Ontario and its doctors reach virtual-care billing deal, but only for specialists [Internet]. CBC News. The Canadian Press; 2022 [cited 2022 Oct 21]. Available from: https://www.cbc.ca/news/canada/toronto/ont-virtual-care-1.6625700

Copyright Ownership: This is an open-access article distributed in accordance with the Creative Commons Attribution Non-Commercial (CC BY-NC 4.0) license, which permits others to distribute, adapt, enhance this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0