Dr. Jing Xu, Ph.D.1 ; Dr. Hanadi Y. Hamadi, Ph.D.1
; Dr. Kristen K. Hicks-Roof, Ph.D., RDN, LDN, FAND2
; Dr. Robert J. Zeglin, Ph.D., MA, BS, CST, NCC, LMHC3
; Ms. Chloe E. Bailey, MHA, BHA1
; Dr. Mei Zhao, Ph.D.1
1Department of Health Administration, Brooks College of Health, University of North Florida, Jacksonville, Florida; 2Department of Nutrition & Dietetics Brooks College of Health, University of North Florida, Jacksonville, Florida; 3Department of Public Health, Brooks College of Health, University of North Florida, Jacksonville, Florida.
Objective: During the coronavirus disease 2019 (COVID-19) pandemic, many other health providers needed to rapidly adopt telehealth services to ensure continuity of patient care, without the opportunity to extensively evaluate the usability of the adopted technology. Therefore, this study aims to examine health professionals’ telehealth usability during COVID-19 in Florida.
Design: This cross-sectional study employed the Telehealth Usability Questionnaire (TUQ) to licensed healthcare providers in Florida in June 2020.
Setting and Participants: A total of 399,660 selected health professionals with Florida licensure were recruited from open-access Florida healthcare to participate in a Qualtrics web-based survey. A total of 1,868 health professionals completed the survey. Multiple linear and mixed regression models were applied to analyze the overall and subdomain scores from TUQ.
Main Outcome Measures: Telehealth usability.
Results: The analysis of the overall TUQ score showed younger, female healthcare professionals, and participants who reported an increase in telehealth usage during pandemic had a significantly higher overall TUQ score. Compared with the score from physicians and nurses, the scores from the mental health group and social work group were significantly higher, while the score rehabilitation group was significantly lower. Analysis of the subdomain scores was consistent with the overall scores.
Conclusion: The findings from this study indicate that the health professionals’ telehealth usability is related to age, gender, and the change of telehealth usage during the COVID-19 pandemic. While pandemics represent only one possible impetus for the healthcare system to swiftly switch to telehealth platforms, each profession should consider providing adequate resources to accommodate the need for change.
Keywords: COVID-19, Florida, Health Professionals, Telehealth, Telehealth Usability, Telemedicine
Citation: Telehealth and Medicine Today 2021, 6: 270 - http://dx.doi.org/10.30953/tmt.v6.270
Copyright: © 2021 The Authors. This is an open access article distributed in accordance with the Creative Commons Attribution NonCommercial (CC BY-NC 4.0) license, which permits others to distribute, adapt, and enhance this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See http://creativecommons.org/licenses/by-nc/4.0.
Published: 18 November 2021
Competing interests and funding: No funding was received from any source for preparing this article for publication.
Corresponding Author: Hanadi Y. Hamadi, Email: h.hamadi@unf.edu
At the end of December 2019, patients were admitted to hospitals in Wuhan, Hubei Province, China, with symptoms of pneumonia of unknown origin. This was the first time the new coronavirus disease 2019 (COVID-19) was encountered, where reports predicted the early onset of a potential outbreak of this coronavirus.1 By March 11th, the World Health Organization declared COVID-19 a global pandemic.2 States, counties, and cities nationwide issued stay at home orders, while many businesses and restaurants are closed for the most part to flatten the curve and avoid a surge of patients flooding healthcare facilities across the country.3 By July 29th of 2020, Florida became a new epicenter for COVID-19 at 441,977 total confirmed cases with 68.1% of cases occurring within the month of July.4 As a precaution and ways to mitigate the spread of COVID-19, many hospitals and healthcare providers limited and even halted medically unnecessary and elective procedures5–7 and access to care.8–10 Therefore, there is a dire need to examine the ability of health providers to use these technologies to improve their care delivery remotely. During the start of the pandemic, telemedicine visits were found to have increased by 683% in the months of March and April alone with a staggering 700 additional virtual visits a day.
Telehealth technologies define a vast array of activities that allow the delivery of care and the interaction of provider-to-patient or provider-to-provider through synchronous (phone and video) and asynchronous (store and forward such as patient portals) communication and virtual agents (telemonitoring through wearable devices).11 Currently, there is a great need for telehealth services during these trying times as many organizations have expanded their telehealth capabilities to serve the patients to the best of their ability while keeping them safe at home.12 It allows for services that have been traditionally conducted by acute care hospitals and health provider offices during face-to-face outpatient visits to be delivered to underserved communities and populations.13 By December of 2016, The Agency for Health Care Administration reported that less than half (44.8%) of hospitals in Florida were found to have telehealth services available to their patients.14
However, telehealth services are not new, more recently, the biggest need for these types of services is to manage chronic diseases.12 An estimated 100 million Americans suffer from chronic diseases such as heart disease, cancer, stroke, diabetes, arthritis, asthma, hypertension or high blood pressure, emphysema, bronchitis, depression, anxiety, and others. These chronic diseases account for 75% of healthcare expenditures.15 In Florida, 46.7% of all Medicare enrollees ages 65 and more reported having at least four chronic conditions, indicating that the large aging population within the state is contributing to the higher proportion of chronic conditions.16 Traditionally, these illnesses are managed by the patient visiting the primary care offices episodically; however, using telehealth services would offer more benefits such as a reduction in hospitalizations, readmission, length of stay, and costs.17,18 Telehealth also offers opportunities for consistent communication between patients and clinicians with the use of tools such as smartphones, tablets, or computers that possess webcams.19 Furthermore, telehealth technologies can be extremely effective when it comes to reducing the cost of healthcare by saving an estimated $19–$121 per virtual visit. These savings come from reduced operational overhead and technology adaption, which is of great necessity during a pandemic that has created a healthcare recession that resulted in over 1.4 million healthcare jobs lost in April 2020 and 30,000 jobs in January 2021.20,21
Furthermore, in March 2020, Centers for Medicare & Medicaid Services (CMS) has provided mass waivers for telehealth reimbursements, allowing for healthcare facilities to provide these services for new and established patients during times of public health emergencies. Additionally, these emergency regulations promote broader access to patients than ever before and allow more providers to take part in conducting their care through virtual work.22 One of the telehealth’s largest obstacles before the pandemic was regarding certain state governments requiring health providers to hold a license in all states they provide care or consultative services in, whether virtually or in-person. Yet, the recent emergency legislation has bypassed most states’ requirements, with some mandating only an emergency credential application to perform virtual consultations in the new COVID-19 era.23 This act has significantly broadened accessibility for health providers and patients. For example, many surveyed physicians have reported that the lifted telehealth restrictions have eased their workload more efficiently, leading to increased care quality and greater satisfaction with newly adopted technology.24 Ultimately, these policies support the promise of telehealth with its proven effectiveness to connect more patients with the quality care they deserve and open the door to have telehealth and make its place in the future of healthcare.
There is an increasing need to use telehealth technologies and increase policy incentivizing its use now more than ever during the COVID-19 pandemic. Yet, it is important to also examine the usability of these sudden adoptions or utilizations of telehealth technologies. Usability refers to a product’s ability to be used by specified users to achieve specific goals such as efficiency, effectiveness, and satisfaction within a particular context of use, which can provide evaluation and overall improved effectiveness of both the technology and delivered services.25 Telehealth’s usability is an important issue for many patients particularly the elderly and those with chronic conditions, such as cardiovascular conditions, who may need to use telehealth services more frequently.26 It is imperative that the operational interface and end-user experience are easy to use for those who will require more access to these platforms, especially in times of a pandemic when sudden adjustments to virtual care may be the only option for nonemergency patients.27 Therefore, the objective of this paper is to report on health professionals’ telehealth usability in a COVID-19 hotspot area.
This study was a cross-sectional survey disseminated online using Qualtrics™.
The Telehealth Usability Questionnaire (TUQ)28 was implemented as part of the survey. Final survey questions included a modified version of the TUQ, including adjusted language from patient-focused to provider-focused. Five experts in healthcare reviewed and approved the modified survey for face validity. The TUQ is designed to evaluate the usability of telehealth implementation and services, including 21 questions covering six subdomains: usefulness (three questions), ease of use & learnability (three questions), interface quality (four questions), interaction quality (four questions), reliability (three questions), and satisfaction and future use (four questions). Each question is assessed by the participants on a seven-point Likert scale (1 = strongly disagree and 7 = strongly agree), in which a higher score consistently indicates a more positive answer to each question. The overall composite score is calculated as an arithmetic mean of all questions with nonmissing answers. The composite score for each subdomain is calculated in the same fashion using the questions from the specific domain. At the beginning of the survey, participants gave anonymous consent to use their responses for research purposes. This study was deemed exempt by the Institutional Review Board (IRB) at the researchers’ University.
Health professionals licensed in Florida were invited to participate in this study. Participation in this study was voluntary, there were no cash incentives for completing this survey. The following healthcare professionals were selected to complete the survey: medical doctor, osteopathic physician, advanced practice registered nurse, registered nurse, physician assistant, physical therapist, occupational therapist, registered dietitian, psychologist, licensed mental health counselor, and licensed clinical social worker. Specific professions were selected based on the availability of the Florida Healthcare Practitioner Data Portal.29 A total of 399,660 surveys were disseminated via email in June 2020, including an initial and one follow-up email. Participants completed a demographic profile including primary profession, years as a licensed provider, gender, age, ethnicity, and geographic location of the practice.
All data summaries and statistical analyses were performed using the SAS software version 9.4 (SAS Institute, Cary, NC).30 In the analysis, the nine individual professions were categorized into five groups based on the practice area: physician and nurses (medical doctor, nurse, and physician assistant), rehabilitation (physical therapist and occupational therapist), mental health (licensed mental health counselor and licensed psychologist), social work (social w1orker), and dietetics (registered dietitian).
The analysis was conducted for the overall TUQ score first, and then for the subdomain scores. The overall score was analyzed using a multiple linear regression model; the scores from six subdomains were analyzed using a linear mixed model. In both models, profession group, gender, age, and the change of telehealth usage during the COVID-19 pandemic were included as fixed covariates. In the subdomain analysis, the correlations between residual errors from the same participants on different subdomains were accounted for by the specified covariance structure. The Kenward and Roger method will be used for the calculation of the denominator degrees of freedom.31,32
A total of 1,868 healthcare professionals are included in the sample. Table 1 summarizes the demographic information including profession group, gender, years of experience, and the change of telehealth usage during the pandemic. In all the survey participants, 78% are female healthcare professionals, indicating a higher likelihood for females to use telehealth compared with their male counterparts. A total of 1,722 healthcare professionals were included in the analysis due to missing values in the response variables.
For the overall TUQ score, profession group, gender, age, and the change of telehealth usage during the COVID-19 pandemic all had a significant impact (Table 2). Specifically, younger and female healthcare professionals participating in the survey had a significantly higher overall TUQ score versus elder and male participants. In addition, individuals who reported to have increased usage during the pandemic also had a significantly higher score. The differences in the overall TUQ scores among different profession groups were also significant. Compared with the score from physician and nurses (estimated average = 4.21), the scores from the mental health group (4.78) and social work group (4.62) were significantly higher and the score from the dietetics group (4.39) was higher but not significant, while the score rehabilitation group (3.93) was significantly lower.
The scores from all six subdomains demonstrated a similar pattern to the overall score across the profession groups from the highest to the lowest, indicating the reliability and consistency of the questionnaire (Table 3). It is worth mentioning that the absolute scores were varying from different subdomains. Specifically, usefulness, ease of use and learnability, satisfaction, and future use domains received the highest scores, and reliability domain received the lowest score, while interface quality and interaction quality domains were in the middle for each profession group. It is also estimated that the female participants scored significantly higher in three of the six subdomains: interface quality, interaction quality, and reliability.
The findings from this study indicate that the health professionals’ telehealth usability in Florida was significantly related to health professional categories, age, gender, and the change of telehealth usage during the COVID-19 pandemic. Compare with physicians and nurses, mental health professionals and social workers reported higher telehealth usability, while rehabilitation clinicians reported much lower usability.
Mental health professionals in the sample, including clinical mental health counselors and psychologists, evidenced the highest TUQ scores. These professions have a lengthy history utilizing telehealth modalities, with discussions of best practices around such beginning in earnest in the 1990s.32–34 As such, professionals in these fields are likely to receive at least some training in telehealth practice.35,36 For example, the Council for Accreditation of Counseling and Related Education Programs (CACREP) requires counseling programs to include training on “technology-assisted [counseling] relationships”.37 Technology-assisted relationships, including telehealth, are therefore not anomalous for most mental health professionals.38–40 This may be why a recent study by McClellan and Florell41 noted that 89% of mental health clinicians in their sample had favorable or neutral attitudes toward telehealth. The results of the current study add helpful context to this, showing that mental health professionals have significantly more favorable views of telehealth than other medical professionals.
Mental health professionals also evidenced higher scores on all TUQ subdomains. The highest subdomain scores were in relation to usefulness and intention to use in the future. The typical tasks of mental health clinicians almost exclusively require less hands-on contact than other professionals, and therefore, may find the telehealth platform less hindering to their normal work duties. Mental health counselors and psychologists providing primarily psychotherapy can, with notably few exceptions, provide the same services via telehealth platforms that they can in face-to-face settings.42 This fact may more easily assuage any concerns these clinicians have about the usability of telehealth platforms.
Services provided by physicians, nurses, and rehabilitation clinicians require a lot of hands-on examinations, and it would be harder for these groups to practice completely virtually. However, with better telehealth tools, these health providers are able to address this gap in care delivery. Examples of such tools found to be conductive to improve care delivery include the use virtual tools that allow for both communication and data sharing among health providers and between health provider and patient,43 and for health organizations to partner with local communities to create a community telehealth network to better serve rural and vulnerable populations.44 Furthermore, there is a need to ensure that telecommunication infrastructures continue to grow as the demand for telehealth services increases.45 Nevertheless, both hospitals and healthcare providers have created innovative alternatives during COVID-19. For example, individual specialties and subspecialties developed virtual examination protocols and having patients proceed through sets of physical activities and manipulating readily available props in the home.46 Studies have shown that clinicians can successfully use several elements of a standard musculoskeletal assessment to evaluate patients with mild low back pain47 and knee pain48 in countries where telehealth is widely available. Studies also indicate that a systematic virtual examination can aid in triaging and managing common musculoskeletal conditions.49 These best practice protocols are still evolving with more use of telehealth services.
Worth noting, psychiatrists, who are trained physicians with an additional psychiatry rotation, were coded as physicians in the present study. Their work, which is largely hands-off medication consultations, can be completed more easily via telehealth modalities than many of their physician colleagues. Telepsychiatry has proliferated over the past decade,50 especially during the COVID-19 pandemic.38,51 As such, psychiatrists may represent a bit of an anomaly within the physician community, evidencing the flexibility and benefits that the other mental health professionals in the study did.
Younger and female healthcare professionals participating in this study showed significantly higher telehealth usability, which is consistent with the literature that telemedicine use among physicians was more prominent among younger doctors. Physicians 45 or younger reported they were practicing some form of telehealth at a higher rate than physicians 46 or older.52 Of course, this trend may change due to the abrupt adaptations needed from the COVID-19 pandemic. In April 2020, already nearly half of physicians using telehealth, up from just 18% in 2018.52 At the same time, the findings on the female healthcare professionals are corresponding with the reports that women and younger adults are more likely to use telehealth services.53 Motivation strategies for older and male healthcare professionals to use telehealth services should be something the healthcare leaders to consider in the future.
Lastly, the healthcare professionals who reported to have increased usage during the pandemic showed significantly higher telehealth usability. This is understandable since practice makes perfect. This is also consistent with previous studies that healthcare providers engaged in more telehealth activities in the realms of telemedicine and there was a significant shift in their perceptions, indicating greater openness and willingness to adopt telehealth services.54 The COVID-19 pandemic forced an array of healthcare providers to quickly adapt and find ways to provide their usual services by using online tools. However, we should also realize the challenges related to telehealth services regarding efficiency, safety, quality, outcomes, as well as the adequacy of existing regulations from both the federal and state governments.
Connolly, Miller55 highlighted the relationship between acceptance of telehealth and the use of telehealth. In light of the COVID-19 pandemic, and especially the subsequent need for a rapid transition to telehealth, it is prudent for training programs in all health professions to consider including at least some training in telehealth, following the model set by the mental health fields, to promote acceptance of the modality of care. Pandemics represent only one possible impetus for the healthcare system to swiftly switch to telehealth platforms; each profession should consider providing their trainees with the best possible chance of successfully making such a switch. The remaining concerns among all healthcare professions surround the platform’s technical stability.
This study has several limitations to consider. Healthcare professionals were contacted through their registered emails with the Florida Department of Health; likewise, healthcare facilities may have blocked these emails containing external URLs, with concerns of spam or cybercrimes. Additionally, a 1% response rate was less than preferred. This sample may not be generalizable to the entire 400,000+ healthcare professionals practicing in Florida. Lastly, this study was a cross-sectional study, only capturing data from one point in time, thus not capturing the changes across the span of the COVID-19 pandemic. Future research should evaluate how health professional’s telehealth usability changes over time.
The United States healthcare system has faced incredible demands for both in-person care and adaptations to virtual care. Educators and policymakers must consider the importance of the education and training of different modalities of care (e.g., telehealth). Telehealth can provide easy access to high-quality care regardless of geography, work, or personal constraints, but healthcare professionals need to be provided the tools for success. It is important for policymakers and health administrators to view telehealth as a vital tool in future pandemic planning and response.56