ORIGINAL RESEARCH
A Prioritization Model for Emergency Telepsychiatry Service Evaluation and Selection
Jacqueline Sandling BA1 , Kathleen Carrothers MS, MPH2 and David Svec MD, MBA3
1Georgetown University School of Medicine, Washington, DC, USA; 2Stanford Health Care Tri-Valley, Pleasanton, California, USA; 3Department of Medicine, Stanford School of Medicine, Stanford, California, USA
Keywords: administration, cost analysis, emergency department, psychiatry, telemedicine
Abstract
Background: Reducing the length of stay (LOS) is an important way for hospitals to improve emergency department (ED) costs and outcomes. Psychiatric patients represent a challenge to reducing LOS as the scarcity of psychiatric specialists leads to longer LOS. Previous literature describes the unique solutions employed by different hospitals across the United States. but does not offer methods for evaluating or selecting a solution that can be applied to other hospitals.
Methods: A mixed methods exploratory sequential design was retrospectively registered in order to build and subsequently test a contextually appropriate evaluation framework. Firstly, interviews with hospital staff were conducted in a qualitative phase, the results of which were used to develop an evaluation framework as a quantitative instrument. This evaluation framework was subsequently tested using a large sample of observational ED case data from one community hospital, as well as pricing data resulting from market research on psychiatry services. This information, along with projected return on investment, was aggregated to create a holistic model for evaluating different telepsychiatry service options and selecting the one with the best fit.
Results: Our methodology identified eight key factors that captured the overall difficulty of implementation and benefits associated with each service option. These factors were used to develop a prioritization model that identifies the one psychiatric service for improving psychiatric LOS and best fitting the hospital’s overall priorities and operations.
Conclusion: The prioritization model created in this study was instrumental in selecting the solution for reducing LOS in a way that best meets the needs of patients and hospitals. This model may be applied to other hospitals in the United States to provide a holistic review and direct comparison of opportunities.
Citation: Telehealth and Medicine Today © 2023, 8: 412 - https://doi.org/10.30953/tmt.v8.412
Copyright: © 2023 The Authors. This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, adapt, enhance this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0.
Received: February 26, 2023; Accepted: April 2, 2023; Published: April 28, 2023
Funding Statement: The authors received no financial support for the research, authorship, and/or publication of this article.
Financial and Non-Financial Relationships and Activities: The authors confirm no relevant conflicts or completing of interest. One author has invested in an outpatient ADHD startup, which does not interact with inpatient care or EDs.
Corresponding Author: Jacqueline Sandling; js5155@georgetown.edu
Plain Language Summary
Reducing the length of stay (LOS) is an important way for hospitals to improve emergency department (ED) costs and outcomes. The scarcity of psychiatric specialists leads to longer LOS for psychiatric patients in particular. Previous literature describes different approaches by hospitals but does not provide generalizable methods for evaluating or selecting a solution. For this study, a mixed methods exploratory sequential design was retrospectively registered in order to build and subsequently test a contextually appropriate, generalizable evaluation framework. Our methodology identified eight key factors associated with each service option: meeting patient needs; meeting staff needs; partnership viability; future opportunities; revenue; initial implementation; logistics; financial cost. The framework was tested using a large sample of observational ED case data from one community hospital and pricing data from psychiatry services on the market. The impact of psychiatric boarding is felt not only by psychiatric patients but also by other ED patients, ED care providers, and hospital networks. To our knowledge, this is the first study to propose a generalizable method for evaluating multiple telepsychiatry service options and selecting the one best fit for the specific needs of patients and hospitals. The resulting framework is applicable to EDs in the United States that are considering starting a telepsychiatry service to decrease psychiatric patients’ long lengths of stay.
Reducing the LOS is an important way for hospitals to improve cost efficiency and health outcomes in their ED. One issue faced by hospital EDs occurs when a patient presents with mental health concerns but there is no specialist to lead the diagnosis, treatment, and disposition.1 The extended waiting time between the patient’s arrival and treatment is known as psychiatric boarding. Psychiatric involuntary holds (IPHs), which are initiated when a person is considered a danger to themselves or others, contribute to boarding when the availability of psychiatric facilities is limited. Protocols for involuntary holds vary subtly by jurisdiction,2 and in California, IPHs last up to 72 h.3 Boarding in the ED creates several problems: increased stress and delayed mental health treatment for psychiatric patients; worsened ED overcrowding; delayed treatment for other ED patients; and ultimately a loss of ED revenue.4
Previous literature outlines efforts by individual institutions to leverage telepsychiatry within the ED to decrease LOS.5–16 Two nationally representative surveys found that one in five EDs used telepsychiatry instead of an in-person psychiatrist. Most EDs using telepsychiatry reported that telepsychiatry was the only emergency psychiatry service available to them, suggesting it plays a critical role in access to mental health services.6 Advances in technology have incorporated interactive synchronous videoconferencing into telemedicine, allowing it to be a critical tool in the delivery of mental health care. It has demonstrated its ability to increase access and quality of care and, in some settings, to do so more effectively than treatment delivered in person.16
Despite the wealth of literature focused on how telepsychiatry services have been implemented in individual institutions, there is a scarcity of literature proposing an actionable, reproducible method for selecting a telepsychiatry service partner based on institutional priorities.17–21 In addition, only a portion of the existing literature addresses the financial costs associated with psychiatric coverage.6–9,12,22 The available models focus primarily on realized costs post-implementation without guidance for estimating costs prior to implementation.7,22
As a result, hospitals implementing an emergency telepsychiatry service have limited resources available to guide their decision-making process. The intent is for the generated framework’s results to guide hospitals’ decisions on how to implement a new ED psychiatric coverage service, and so far, it has been successful in the case of one community hospital in California.
Background
The rationale behind creating an actionable, reproducible method for selecting a telepsychiatry service is that a generalizable approach would directly benefit hospital administrators and clinical leaders seeking a solution for their psychiatric patients, regardless of geography or other factors. Because there is a need to standardize the synthesis of quantitative data based on qualitative information, this study advances the decision-making process for selecting psychiatric services for the ED setting using an exploratory sequential design. The first phase of the study was a qualitative exploration of reasons for and outcomes of long psychiatric lengths of stay. From this initial exploration, the qualitative findings were used to generate a quantitative evaluation framework that can be administered to a larger sample. To test the evaluation framework, quantitative case data and costs associated with psychiatric care were collected. To our knowledge, this is the first study to propose a generalizable method for evaluating multiple psychiatry service options and selecting the one best fit for the patients’ and hospital’s specific needs.
Methods
The mixed methods exploratory sequential design consists of three distinct phases: collection and analysis of qualitative data, a development phase of translating the qualitative findings into a quantitative approach, then finally, implementation and testing of the quantitative measures. This study was an intervention-development variant of exploratory sequential design, meaning that the qualitative data helped the researchers to develop an intervention that would work with the participants and be meaningful to them. An illustration of the procedures is found in Figure 1.
Fig. 1. This study was conducted according to the steps in this procedure diagram.
Research consent was deemed unnecessary because the project was determined by the Stanford Institutional Review Board panel IRB-98 not to meet the definition of human subjects research as defined in federal regulations 45 Code of Federal Regulations (CFR) 46.102 or 21 CFR 50.3. The study took place at one community hospital in the Bay Area because it was borne out of a city improvement project at that single site. In the United States, 84% of all hospitals are community hospitals.23 Focusing on one site allowed a comprehensive investigation of the causes, scope, and solutions of an issue they commonly experience.
Qualitative Interview Phase
Informed by a grounded theory approach, the qualitative phase was conducted in a manner most aligned with the hospital’s operational practices. There were two rounds of semi-structured interviews to identify causes for and the impact of psychiatric boarding. The first round was conducted with ED clinicians and staff who treat psychiatric patients directly. The second round was with administrators who manage ED projects and finances. Participants were selected using purposeful sampling to hone in on those who manage ED psychiatric patients and are most directly impacted by long lengths of stay.
There were 11 first-round interview participants, including ED administrative and medical directors; the Social Services Director and staff; three hospitalists; and nursing officers. The second round of interviews was conducted with six administrators from Compliance, Finance, Project Management, and Business Development teams. Interviews were conducted from February 4, 2020 to March 17, 2020. Initial outreach to participants was via email and interviews took place either in person at the community hospital or via the Zoom virtual meeting platform. No one else was present besides the participants and researchers. No one dropped out or refused to participate.
Team member JS led the 45-min interviews. Team members DS and KC attended all interviews except those with physician hospitalists and half of the social workers because of scheduling conflicts. Although repeat interviews were not formally conducted, all interview participants were actively involved in later conversations about selecting and implementing a new psychiatric coverage. Data saturation was reached when in interviews, no new data were revealed, and, specifically, participants answered, “Who else should we interview for this project?” with no new names. All interview participants had prior working relationships with DS and KC but not JS, and they understood the purpose of the interviews was to create a psychiatric service in the ED. See Appendix 1 and Appendix 2 for interview protocols.
The hospital’s business development team confirmed that for the purpose of getting a new service approved, JS, KC, and DS did not require specific training for conducting interviews. The audio or visual recording was not used to collect the data, which is reliant upon what was captured in field notes typed during the interviews by JS, who also analyzed the interview data.
Four thematic domains emerged, as seen in Table 1: the causes of psychiatric boarding, and the impacts that boarding has on patients, providers, and the hospital overall. In summary, interview participants reported that because the ED did not staff a psychiatrist or mental health expert, the ED could neither directly treat nor remove IPHs, which would require a psychiatrist evaluation. Instead, social workers had to coordinate a transfer to a psychiatric facility for all psychiatric holds to be evaluated and lifted.
When asked about patient flow and patterns of changes in lengths of stay, multiple ED clinicians mentioned that ED LOS typically increased for all cases during peak hours. The cited reason was that ED saw its highest case volumes during peak hours.
Quantitative Evaluation Framework
A qualitative thematic analysis of these interviews was conducted to identify the causes and consequences of long psychiatric lengths of stay. It also clarified which consequences were of higher priority to different staff and administrators, such as the importance of mean LOS for psychiatric patients or the revenue generated by a new ED service within its first year.
After causes were identified, only solutions that directly addressed these causes were considered. All possible solutions involved implementing a new psychiatric service in the ED, which called for creating a quantitative instrument to assess the degree to which each possible service minimized long LOS and its consequences. Then, integrative data analysis was conducted to design a quantitative evaluation framework to identify the best psychiatric service option, as informed by the qualitative results. The research team also chose to create an evaluation framework that would adapt to reflect the different consequences of interest for different stakeholders.
This framework was refined through an expert review conducted by hospital administrators familiar with the ED, including patient volume, operations, and finances (Table 2).
Because the quantitative features were grounded in qualitative data from interviewing hospital staff and administrators, all quantitative features were designed according to the hospital administrator’s perspective. For instance, the time horizon selected should reflect the typical approach taken by the hospital’s business development team to appeal to decision-makers. This approach mirrors those that other studies used for post-implementation ROI calculations,7,22 and this alignment allows for actionable pre-post analyses.
This study incorporated general financial considerations suggested by previous literature, such as the costs of purchasing technological devices.6–9,12,22 Specific costs were identified in consultation with the Finance Department to make the framework useful and relevant for hospital stakeholders. To calculate the answers for variables related to financial costs and viability, a return on investment (ROI) model was used. The ROI was projected for 5 years, the time horizon that reflected the typical approach taken by the hospital’s business development team.
For calculating ROI, the expected return comprised two sources identified in the qualitative phase: direct costs that are currently incurred and will be avoided post-implementation and new revenue from treating ED patients who would have left without being seen (LWBS). Two direct costs associated with delays in psychiatric care were identified: sitters and funded transportation to psychiatric facilities for uninsured patients. These costs were multiplied by the psychiatric case volume and expected LOS improvement rate to represent direct cost savings. New revenue from treating patients with the time saved from psychiatric cases was calculated as the mean contribution margin per ED case multiplied by the number of additional patients that could be treated. This revenue captured the financial impact to the hospital. The financial investment required for each service option was estimated using pricing structures supplied by service options. Each pricing structure comprised two fee types: one-time implementation fees, including the purchase of equipment, and monthly fees to cover ongoing staff and technical support.
The results from the initial qualitative exploration were used to build the evaluation framework, called a prioritization model. Results from the prioritization model helped answer the mixed methods question, “How can clinician and administrators’ accounts of problems related to ED psychiatric lengths of stay help design an instrument so the instrument best fits the needs of those ED psychiatric patients?” Similar to the ROI calculation, the prioritization model was designed to answer this question in a way that was familiar and easily understandable to stakeholders.
Specifically, the prioritization model categorizes each feature as either a “benefit” or “implementation difficulty” and then scores each psychiatry service option on how well it aligns with the hospital’s needs. Because the qualitative phase revealed that stakeholders had varying priorities, the prioritization model allows for categories to be weighted to reflect each feature’s importance according to the hospital’s organizational knowledge. For instance, a category with a weight of 2 is twice as important as another category with a weight of 1.
Inputs and themes from the two rounds of interviews were used to identify eight different categories important to have in any psychiatry service: five categories of benefits and three categories representing implementation difficulty. For all categories, higher scores were favorable. As seen in Table 2, each category was further broken down into more specific components that could be directly answered with either a number or Yes/No.
Components with percent values were converted into quintiles, with negative percent values assigned a score of 0. For example, an ROI of 65% was scored 4. Other numeric values were converted into a percent of the maximum component value and assigned a quintile score. For example, if the greatest LOS improvement across all service options was 4.6 h, then the service option offering an improvement of 1.4 h would be assigned a score of 2. For Yes/No questions, “Yes” was assigned a score of 3 and “No” was assigned a score of 0.
Component scores were averaged to calculate a category score. Each category score was then multiplied by weights determined by the hospital. Weights (0–2) were assigned to each category based on perceived importance. The weighted scores were then summed to create an overall “Benefit” score and “Implementation Difficulty” score per service option. Since a higher score is better across benefits and implementation difficulty, these two can be summed together to find the one service option with the highest total score, indicating the best fit. The ability to change category weights allows for generalizability: if another hospital is evaluating programs that improve staff recruitment and retention, for instance, that hospital can weigh “meets staff needs” more heavily in their own model.
Quantitative Test Phase
For the quantitative phase, an observational cross-sectional study was conducted to compare lengths of stay for psychiatric patients and non-psychiatric patients. The ED cases occurred from January 1, 2019 to February 29, 2020, at the same the San Francisco Bay Area community hospital’s ED studied in the qualitative phase. The same institution was selected to see how the evaluation framework translated specific qualitative themes into quantitative metrics. To minimize selection bias, no ED cases from that time were excluded.
A psychiatric ED case was defined as a case that began with an IPH and ended with a transfer to a psychiatric facility or both. Because this analysis used ED case data generated from electronic medical records that were also used for billing and legal purposes, measurement error was not a concern, and no case data were missing, including patient identifying information, timestamps, discharge information, and costs. The only information that required verification was the home addresses of patients who were homeless, which social workers confirmed.
The LOS is the time between the patient’s arrival and discharge, and psychiatric LOS improvement means shortening it to equal the mean LOS for non-psychiatric cases. Subgroup analyses were conducted and demographic data were collected to test patterns and themes noticed by participants in interviews from the qualitative phase.
To quantify the relative severity of psychiatric boarding during peak hours, the mean and median lengths of stay were calculated. To investigate when patients were waiting the longest throughout their ED visit, the mean time spent in each step of the typical ED patient journey was also calculated: Arrival time to Roomed; Roomed to Attending; Attending to Attending Disposition; Attending Disposition to Departure. To eliminate potential confounding by hospital inpatient bed vacancy, a subgroup analysis was conducted on psychiatric patients, excluding those who were admitted to inpatient.
Results were also analyzed for psychiatric and non-psychiatric subgroups. For psychiatric patients on an IPH, their discharge location was also analyzed to investigate the discharge and transfer patterns described by interview participants in the qualitative phase. Calculations were done for all times and peak time to quantify the difference observed by ED staff in the qualitative phase. The standard deviation was calculated to estimate how dispersed the lengths of stay were in relation to the mean.
A “clearance rate” is the percent of patients whose IPH’s are removed after evaluation out of all patients with IPH’s in the ED. Data from other telepsychiatry programs suggested that access to psychiatric care in the ED could lead to clearance rates of 25% – 80%.5,8,24,25
Sensitivity analyses were conducted to assess changes in outcomes according to the factors that interview participants in the qualitative phase deemed to be the most salient for LOS.
Telepsychiatry vendors and market solutions were found by two mechanisms. Firstly, an online search was conducted using these search terms:
“telepsychiatry” OR “psychiatry” OR “telemedicine psychiatry” AND “emergency” OR “hospital” OR “emergency consultation” OR “acute” AND “service” OR “vendor” OR “company”
Searches were repeated with the names of the geographical region and state to minimize selection bias that may exclude local providers whose search engine optimization is less robust than larger, national vendors. Secondly, opportunities within the hospital network were sought to leverage local psychiatrists and mental health experts. Vendors supplied information on their business and pricing models both by written and verbal communication. Requesting written documentation, which required approval from each vendor’s respective legal team, minimized information bias.
Results
There were 41,876 ED cases identified, 875 of which were psychiatric and 41,001 were non-psychiatric. Table 3 lists the demographic: age range, gender, geography, and insurance payor for this ED site.
| Psychiatric cases (n = 875) | Non-psychiatric cases (n = 41,001) | |
| Age (years) | ||
| 0–17 | 186 | 6,442 |
| 18–34 | 350 | 8,348 |
| 35–44 | 124 | 5,116 |
| 45–54 | 112 | 5,132 |
| 55–64 | 63 | 5,232 |
| 65+ | 40 | 10,731 |
| Geography | ||
| Within primary service area | 575 | 30,987 |
| Outside primary service area | 191 | 9,477 |
| Homeless | 109 | 537 |
| Gender | ||
| Female | 397 | 21,427 |
| Male | 478 | 19,570 |
| Undisclosed | 0 | 4 |
| Payer | ||
| Private | 324 | 14,966 |
| MediCal* | 300 | 10,178 |
| Medicare** | 103 | 10,572 |
| Dept of corrections*** | 35 | 251 |
| Other | 22 | 2,029 |
| Uninsured | 91 | 3,005 |
| Case data are shown by age range, gender, geography, and insurance payor. Race and ethnicity data were not available for these specific ED cases; geography has been provided instead. ED: emergency department. *MediCal: The California Medicaid program that provides medical services to low-income people at little or no cost; **Medicare: The U.S. federal health insurance program for: people who are age 65 years or older, certain younger people with disabilities, and people with end-stage renal disease. ***Department of Corrections: those patients who were incarcerated. | ||
As seen in Table 4, the opportunity for improvement was sizable: mean LOS for psychiatric cases was 8.54 h longer than for non-psychiatric cases. The distribution of LOS by IPH and Transfer to Psychiatric Facility status is illustrated in Figure 2. The ED’s schedule was then categorized into peak and non-peak hours. The greatest volumes of patients arrive during peak hours, experiencing longer lengths of stay and greater risk of leaving without being seen by a provider. A subgroup analysis of peak-hour cases was also done to reflect the fluctuations in case volume that participants reported in interviews. Case data confirmed that more patients arrived per hour during peak hours, and these patients experienced longer lengths of stay and greater risks of leaving without being seen by a provider.
| Psychiatric cases | Non-psychiatric cases | All cases | |
| All time | (n = 875) | (n = 41,001) | (N = 41,876) |
| Mean (h) | 11.98 | 3.44 | 3.62 |
| Median (h) | 8.75 | 3.07 | 3.12 |
| Standard deviation | 9.92 | 2.14 | 2.84 |
| Peak hours (10 am – 8 pm)* | (n = 498) | (n = 25,224) | (n = 25,722) |
| Mean (h) | 12.69 | 3.65 | 3.82 |
| Median (h) | 8.71 | 3.32 | 3.37 |
| Standard deviation | 10.87 | 2.14 | 2.88 |
| *Length of stay summary statistics are shown by ED case type for all times of day and for peak time (10 am – 8 pm [10:00 – 20:00]). | |||
Fig. 2. Length of stay for emergency department psychiatric cases by involuntary psychiatric hold (IPH) and transfer to psychiatric facility criteria. For Transfer to Psychiatric Facility without IPH, the median length of stay = 8.5 h and mean length of stay = 10.7 h. For Transfer to Psychiatric Facility with IPH (n = 560), the median length of stay = 9.5 h, and mean length of stay = 12.7 h. For All IPHs (n = 700), the median length of stay = 8.9 h, and the mean length of stay = 12.3 h.
The mean time spent in each step of the typical ED patient journey that did not end in admission to inpatient is also shown in Table 5. The analysis of ED discharge locations for psychiatric patients confirmed that the majority of those on IPHs were discharged to psychiatric facilities; these results can be seen in appendix Table 6.
Ten possible psychiatric service options were found through this study’s search method. Four were within the community hospital’s network and all offered telepsychiatry. Six services provided revenue management and eight services offered reconciliation of psychiatrist professional fees. A breakdown of savings and costs included in calculating ROI is presented in Figure 3. The ROI was considered as one factor in the prioritization model. Because this study was conducted from the perspective of the hospital administrator who is tasked with implementing a new psychiatric service to combat long lengths of stay, all data on costs, times, and vendors were sampled from resources that hospital administrators at this community hospital trust and use on a daily basis: namely, the hospital’s Finance Department and electronic medical record reports, as well as and vendors’ representatives. This minimized sampling uncertainty relative to the purpose of the evaluation framework.
Fig. 3. Sources of costs and savings used to calculate return on investment (ROI). This diagram shows the breakdown of financial information used to calculate projected costs and savings and, ultimately, return on investment. *Indicates a cost that might be calculated differently for other hospitals.
The relative weights used by the hospital are shown in Table 7. Based on the priorities stated by stakeholders at this community hospital, two models were created: a community engagement and financial model.
| Category | Financial model | Community engagement model | |
| Benefits | Meets patient needs | 2 | 2 |
| Meets staff needs | 0* | 0* | |
| Partnership viability | 1 | 2 | |
| Future opportunities | 1 | 2 | |
| Revenue | 2 | 1 | |
| Implementation difficulty | Initial implementation | 0* | 1 |
| Logistics | 0** | 1.5 | |
| Financial cost | 2 | 1 | |
| Weights applied to categories in each model to represent different versions of one hospital’s priorities. *Categories where all service options had the same score; **categories that were not priorities for the hospital but are important for hospitals to consider generally. | |||
Community engagement and financial model scores are presented in Table 8 for each of the 10 service options explored. The mean benefit score in the community engagement model was 11.8 (SD = 2.8) and the mean implementation difficulty score was 5.7 (SD = 1.3). Option J achieved both the highest benefit score of 16.0 and the highest implementation difficulty score of 7.0, making Option J the preferred partner in this model. The mean benefit score in the financial model was 10.8 (SD = 3.1) and the mean implementation difficulty score was 4.7 (SD = 2.5). Option A received the highest benefit score in this model (16.2), while Option B achieved the highest sum score of 20.0 (benefit = 14.0; implementation difficulty = 6.0). Both A and B ranked high in prioritization using the financial model.
| Service | Tele-psychiatry or in-person? | Community engagement model | Financial model | ||||
| Benefit | Implementation difficulty | Total score | Benefit | Implementation difficulty | Total score | ||
| Option A | Tele | 14.0 | 4.5 | 18.5 | 16.2 | 3.0 | 19.2 |
| Option B | Tele | 10.7 | 7.0 | 17.7 | 14.0 | 6.0 | 20.0 |
| Option C | Tele | 9.7 | 5.5 | 15.2 | 12.0 | 5.0 | 17.0 |
| Option D | Tele | 9.7 | 5.0 | 14.7 | 12.0 | 2.0 | 14.0 |
| Option E* | Tele | 14.7 | 6.0 | 20.7 | 10.8 | 6.0 | 16.8 |
| Option F | Tele | 8.0 | 3.0 | 11.0 | 8.7 | 0.0 | 8.7 |
| Option G* | Tele | 14.3 | 5.5 | 19.8 | 10.2 | 5.0 | 15.2 |
| Option H* | Tele | 12.0 | 7.0 | 19.0 | 7.5 | 8.0 | 15.5 |
| Option I | Both | 8.7 | 6.0 | 14.7 | 5.7 | 4.0 | 9.7 |
| Option J* | Both | 16.0 | 7.0 | 23.0 | 11.0 | 8.0 | 19.0 |
| Range of possible scores | 0–35.0 | 0–17.5 | 0–52.5 | 0–30.0 | 0–10.0 | 0–40.0 | |
| Standard deviation | 2.8 | 1.3 | 3.5 | 3.1 | 2.5 | 3.8 | |
| Outcomes for each psychiatric service option evaluated using the two prioritization models. *in-network status. | |||||||
Discussion
This study used a mixed methods exploratory sequential design to propose a structured way of evaluating ED psychiatry service options to decrease psychiatric lengths of stay.
Qualitative Interview Phase
Interviews from the qualitative phase suggested that there were three main causes of psychiatric boarding: Inability to remove involuntary psychiatric holds in the ED when appropriate; low vacancy rates at inpatient psychiatric facilities; and an inability to consult with a psychiatrist to triage patients’ needs according to urgency. Interviews also suggested three distinct entities that were negatively impacted: patients; ED staff; and the hospital as an institution. Some impacts were subjective, like feelings of frustration, and others were objective and could be confirmed with quantitative data, such as delays in care or loss of revenue.
To check for representativeness, interviews were conducted with multiple interview participants of the same role or position at the hospital, for example, interviewing three physicians instead of one until data saturation was reached. Because KC and DS were employed by and typically spent the majority of their working hours at the hospital, the effects of a researcher as an external observer inserting themselves into the hospital’s operations and environment were minimized. To test the preliminary conclusions reached about the causes of psychiatric boarding, night shift social workers, who were reported by other interview participants to experience the most extreme cases of boarding, were interviewed.
The accuracy of accounts was also verified by triangulating information among multiple participants, investigators, and data collection methods. Throughout the study, the research team shared its evolving description of causes and impacts of psychiatric boarding with interview participants.
Because the hospital’s work culture was not for meetings to be audio or visual recorded, interview responses were captured by note only. Although a coding tree was not agreed upon, JS’s findings from each interview were compared against KC’s whenever possible to minimize the bias stemming from the effects of the site on the researcher. In addition, a comprehensive summary document was written by JS to capture the causes and impacts highlighted by interview participants, which were confirmed by KC and DS.
The COnsolidated criteria for REporting Qualitative research (COREQ) checklist was used and criteria except for those regarding coding were fulfilled. Future studies should incorporate the full checklist.
Quantitative Evaluation Framework
The integration results suggest that the evaluation framework is an effective tool for deciding which psychiatric coverage service option to implement. The need to implement psychiatric services in the ED to combat boarding is reflected in previous literature leveraging telepsychiatry within the ED to decrease LOS.5–16 Important features for an emergency psychiatry service include the ability to meet patients’ needs, patient-centeredness, smooth processes and operations, strategic alignment, affordability, and integration of care.20,26,27 This evaluation framework provides some flexibility for different hospitals to use weights that emphasize some criteria over others and addresses these topics with a method that can apply to multiple hospitals operating with different resources, regulatory landscapes, reimbursement strategies, and health system infrastructure.
The ROI values used in the Prioritization Model were conservative in the following ways: firstly, the definition of “psychiatric case” excluded cases where LOS was unlikely to improve. Secondly, direct costs excluded the costs of non-sitter staff, bus tickets, and other transient costs. Thirdly, new revenue from non-peak hour cases was not incorporated, because it is not confirmed that patients are at risk for LWBS in non-peak hours. Fourthly, it was assumed that all psychiatric cases required a full evaluation to shorten their LOS. In November 2020, the County implemented a policy where a consultation with a psychiatrist would likely suffice for placing and removing IPHs. Since consultations are faster than evaluations, it is possible that even greater improvements in LOS could be captured. The Consolidated Health Economic Evaluation Reporting Standards (CHEERS) checklist was used and criteria for single study-based evaluation and model-based evaluation were fulfilled.
Quantitative Test Phase
To test the evaluation framework, quantitative case data and costs associated with psychiatric care were collected. The main outcome of the observational cross-sectional study, mean LOS, was 248% longer for psychiatric than non-psychiatric cases, verifying the severity of boarding that participants observed in their interviews. Furthermore, the analysis of time spent in each step of an ED journey revealed that psychiatric patients spent 109% longer in the “Attending to Attending Disposition” phase, that is, after being seen by an attending but before receiving a discharge plan. The longer amount of time needed to create a treatment and discharge plan validated the perspective of interview participants in the qualitative phase that longer lengths of stay resulted from an inability to consult with a psychiatrist to triage patients’ needs according to urgency and an inability to remove involuntary psychiatric holds in the ED when appropriate.
Furthermore, psychiatric patients spent 1,333% more time in the “Attending Disposition to Departure” phase, that is, after receiving a discharge plan but before being discharged. This difference, 8.6 h versus 0.6 h, was due to the fact that many psychiatric patients were discharged to psychiatric facilities instead of home or other locations. These wait times confirmed interview participants’ concerns that longer lengths of stay were causing delays in psychiatric patients’ care and less availability of ED beds, which resulted in delays in all other ED patients’ care and greater frustration for both patients and providers alike.
Because ROI and Prioritization depended on both ED peak case volume and improvement in LOS due to removing IPHs, sensitivity analyses were conducted to assess the impact of changes in the two factors for each service option. Specifically, ED peak case volume was estimated at low, medium, and high levels; and psychiatric cases with LOS improvement were estimated at 25%, 50%, and 75%.
The research team presented results regarding lengths of stay and recommended two possible service options, depending on which priorities the hospital chose to pursue. Because the analysis was found to be comprehensive, hospital leadership had no follow-up questions for additional data and was able to select one service option according to the one model that reflected its priorities more accurately.
To confirm construct validity, a subgroup of ED staff and administrators reviewed the quantitative results and confirmed that they were meaningful indicators of the degree to which each service option would address psychiatric LOS and its consequences. These reviewers’ positive feedback about the quantitative findings suggested that the eight categories appropriately captured the features and criteria necessary to address boarding and its consequences.
This study’s main limitation is its single-site nature. As seen in Table 9, the ED site in this study had similar patient age and gender breakdowns compared with other EDs in the same county. However, a lower percentage of its patients identified as Black, Hispanic, and/or on MediCal insurance (28% vs. 9% Black; 28% vs. 18% Hispanic; 37% vs. 26% on MediCal).
| Age | All Alameda County EDs | Community hospital |
| 0–9 | 41,306 | 2,011 |
| 10–19 | 34,125 | 1,997 |
| 20–29 | 66,427 | 3,493 |
| 30–39 | 71,233 | 4,138 |
| 40–49 | 55,991 | 3,374 |
| 50–59 | 56,294 | 3,293 |
| 60–69 | 49,192 | 2,610 |
| 70–79 | 33,860 | 2,171 |
| 80+ | 25,358 | 1,926 |
| Race | ||
| Asian | 58,003 | 4,155 |
| Black | 120,615 | 2,204 |
| Multiracial | 9,935 | 578 |
| NHPI | 5,703 | 86 |
| White | 102,518 | 11,374 |
| Other | 129,469 | 6,505 |
| Unknown | 7,619 | 111 |
| Ethnicity | ||
| Hispanic | 123,106 | 4,563 |
| Non-Hispanic | 302,780 | 20,303 |
| Other / Unknown | 7,976 | 147 |
| Gender | ||
| Female | 227,547 | 13,032 |
| Male | 206,209 | 11,980 |
| Other / Unknown | 106 | 1 |
| Payer | ||
| MediCal* | 162,207 | 6,542 |
| Medicare** | 85,218 | 5,102 |
| Other payer | 16,249 | 1,405 |
| Private Coverage | 138,703 | 9,868 |
| Self-pay | 31,481 | 2,096 |
| Case data are shown by age range, race, ethnicity, sex, and insurance payer. *MediCal: The California Medicaid program that provides medical services to low-income people at little or no cost; **Medicare: The U.S. federal health insurance program for people who are of 65 years or older, certain younger people with disabilities, and people with end-stage renal disease. NHPI: Native Hawaiian and Pacific Islander. | ||
When other sites choose to use the ROI Model and Prioritization Model, they should account for these differences in patient population in addition to other considerations, such as different regulatory landscapes and opportunities within-network. With that in mind, the authors propose that other hospitals evaluating telepsychiatry ED services will still be able to substitute their own costs and weights to select the service option best fitting their specific needs. Scores were not received from the ED over a significantly long period of time to examine stability. This would be addressed in a follow-up study when more time has elapsed.
After selecting a psychiatric service option to implement, from January 1, 2022 to February 28, 2023 the ED utilized psychiatric consults in 486 cases, or a mean of 35 cases per month. The telepsychiatry services have since been implemented in the hospital’s inpatient units as well, which has been used in 140 cases or a mean of 10 per month. These numbers suggest that despite observed decreases in ED utilization due to the COVID-19 pandemic, telepsychiatry can play an important role in delivering psychiatric care in EDs and other hospital environments. In addition, the following were out of this project’s scope but are expected to improve with increased access to psychiatry and decreased psychiatric boarding: patient satisfaction; indirect costs from provider productivity and satisfaction; ability to meet The Joint Commission standards or other quality metrics for accreditation29 and reputation within the community. Future studies may evaluate these outcomes. Lastly, the STROBE checklist for cross-sectional studies was used.
The intent of this phase was for the generated framework’s results to guide the hospital’s decision on how to implement a new ED psychiatric coverage service. Few studies have proposed a generalizable method that can apply to other hospitals. One challenge previously cited is the diversity in health care settings.19,23 This study suggests a framework that may be used even when costs and priorities vary, allowing for a direct comparison of options. To the authors’ knowledge this is the first proposal for an actionable, reproducible method for selecting a telepsychiatry service partner based on institutional priorities.17–22
Conclusions
The concerns with psychiatric boarding in the ED are well documented as the impact is felt not only by psychiatric patients but also other ED patients, ED care providers, and hospital networks. Many hospitals have published their own efforts to tackle psychiatric boarding. To our knowledge, this is the first study to propose a generalizable method for evaluating multiple psychiatry service options and selecting the one best fit for the patients’ and hospital’s specific needs. The resulting framework is applicable to EDs in the United States that are considering starting a telepsychiatry service to decrease psychiatric patients’ long lengths of stay. Future research includes applying this framework to additional ED sites; conducting follow-up studies; and investigating the framework’s impact on patient satisfaction, provider productivity and satisfaction, meeting quality metrics, and overall hospital reputation.
Contributors
DS and KC contributed to the conceptualization of this study. All authors designed the study. JS and KC performed material preparation, data collection, and analysis. JS conducted literature searches and prepared the manuscript. DS and KC provided ongoing feedback and consultation during the manuscript preparation stages. All authors read and approved the final manuscript.
Acknowledgments
The authors wish to acknowledge all the interview participants for their cooperation and time; Lisa Shieh for guiding the research process; and John Borghi for assisting literature searches.
Supporting Information
- Prioritization Model.xlsx: This file is the prioritization model proposed in this study. The purpose of this model is for individuals to enter their own data and use the findings to guide evaluations of emergency psychiatric coverage service options. As a result, there is no data in this file.
- ROI Model.xlsx: This file is the ROI model used in this study. The purpose of this model is for individuals to enter their own data and use the calculated savings and costs to guide evaluations of emergency psychiatric coverage service options. As a result, there are no data in this file.
Availability of Data and Materials
Data on ED case volume from January 1, 2019 to February 29, 2020 are available from the corresponding author on reasonable request and with permission of the facility.
The pricing model data that support the findings of this study are from the third party vendors, but restrictions apply to the availability of these data, and so they are not publicly available. A list of vendors is available from the authors upon reasonable request.
The Prioritization and ROI Models discussed in this published article are found in the supplementary files Appendix 3. Prioritization Model and Appendix 4. ROI Model.
References
- Meyer JD, McKean AJS, Blegen RN, Demaerschalk BM. Emergency department telepsychiatry service model for a rural regional health system: the first steps. Telemed J e-Health. 2019;25(1):18–24. doi: 10.1089/tmj.2017.0293
- Fariba K, Gupta V. Involuntary commitment [Internet]. Treasure Island, FL: StatPearls Publishing; 2021. Available from: https://www.ncbi.nlm.nih.gov/books/NBK557377/ [cited 25 February 2023].
- Article 1: detention of mentally disordered persons for evaluation and treatment [5150 – 5155]. California Legislative Information 2019. Available from: leginfo.legislature.ca.gov [cited 23 December 2020].
- Black AD, Kenny B, Dym A. Addressing psychiatric boarding within the emergency department. American Academy of Emergency Medicine; 2022. Available from: https://www.aaem.org/resources/publications/common-sense/issues/featured-articles/novdec-2022 [cited 6 April 2023].
- Rachal J, Sparks W, Zazzaro C, Blackwell T. Highlight in telepsychiatry and behavioral health emergencies. Psychiatr Clin North Am. 2017;40(3):585–96. doi: 10.1016/j.psc.2017.05.014
- Freeman RE, Boggs KM, Zachrison KS, Freid RD, Sullivan AF, Espinola JA, et al. National study of telepsychiatry use in U.S. Emergency Departments. Psychiatr Serv. 2020;71(6):540–6. doi: 10.1176/appi.ps.201900237
- Thomas JF, Novins DK, Hosokawa PW, Olson CA, Hunter D, Brent AS, et al. The use of telepsychiatry to provide cost-efficient care during pediatric mental health emergencies. Psychiatr Serv. 2018;69(2):161–8. doi: 10.1176/appi.ps.201700140
- Hyer T, Zeller S. Behavioral healthcare in ED improved with telepsychiatry. Hosp Peer Rev. 2017;42(4):42–4. Available from: https://pubmed.ncbi.nlm.nih.gov/29996014/ [cited 6 April 2023].
- States leverage telepsychiatry solutions to ease ED crowding, accelerate care. ED Manag [Internet]. 2015;27(2):13–7. Available from: https://pubmed.ncbi.nlm.nih.gov/25688413/ [cited 6 April 2023].
- Holton A, Brantley T. Telepsychiatry in North Carolina: mental health care comes to you [Internet]. In: Rash M, editor. NC Statewide Telepsychiatry Program. Raleigh, NC: North Carolina Center for Public Policy Research; 2014. Available from: https://ncstep.ecu.edu/ [cited 6 April 2023].
- Reliford A, Adebanjo B. Use of telepsychiatry in pediatric emergency room to decrease length of stay for psychiatric patients, improve resident on-call burden, and reduce factors related to physician burnout. Telemed J e-Health. 2019;25(9):828–32. doi: 10.1089/tmj.2018.0124
- Narasimhan M, Druss BG, Hockenberry JM, Royer J, Weiss P, Glick G, et al. Impact of a telepsychiatry program at emergency departments statewide on the quality, utilization, and costs of mental health services. Psychiatr Serv. 2015;66(11):1167–72. doi: 10.1176/appi.ps.201400122
- Saeed S. Using telepsychiatry and health technologies to provide evidence-based care [Internet]. College of Nursing. North Carolina Statewide Telepsychiatry Program (NC-STeP); 2020. Available from: https://nursing.ecu.edu/ [cited 6 April 2023].
- Zhong C, Freeman RE, Boggs KM, Zachrison KS, Gao J, Espinola JA, et al. Receipt of telepsychiatry and emergency department visit outcomes in New York State. Psychiatr Q. 2021;92(3):1109–27. doi: 10.1007/s11126-021-09886-y
- Natafgi N, Childers C, Pollak A, Blackwell S, Hardeman S, Cooner S, et al. Beam me out: review of emergency department telepsychiatry and lessons learned during COVID-19. Cur Psychiatry Rep. 2021;23(11):72. doi: 10.1007/s11920-021-01282-4
- Mishkind M, Boyce O, Krupinski E, Barrett R, Caudill R, Chiu A, et al. Resource document on best practices in synchronous videoconferencing-based telemental health [Internet]. APA Resource Document. Washington, DC: American Psychiatric Association; 2022. Available from: https://www.psychiatry.org/getmedia/49f487ae-7b83-416a-8812-b2b25c717f6a/Resource-Document-Telemental-Health-Best-Practices.pdf [cited 6 April 2023].
- van der Goes DN, Edwardson N, Rayamajhee V, Hollis C, Hunter D. An iron triangle ROI model for health care. ClinicoEcon Outcomes Res. 2019;11(11):335–48. doi: 10.2147/CEOR.S130623
- Augenstein J, Marks JD, Brown A. Estimating the ROI of telehealth programs [Internet]. New York, NY: Manatt Health Strategies, LLC; 2019. Available from: https://www.manatt.com/Manatt/media/Documents/Articles/TelehealthROIWhitePaper.pdf [cited 6 April 2023].
- Kramer GM, Shore JH, Mishkind MC, Friedl KE, Poropatich RK, Gahm GA. A standard telemental health evaluation model: the time is now. Telemed J e-Health. 2012;18(4):309–13. doi: 10.1089/tmj.2011.0149
- Arkwright BT, Leslie M, Light M. Telehealth finance variables and successful business models. Telehealth Med Today. 2019;4. doi: 10.30953/tmt.v4.140
- Cahill A, Matsui EC. Telehealth in the post pandemic era. Telehealth Med Today. 2021;6(4). doi: 10.30953/tmt.v6.291
- Hilty DM, Serhal E, Crawford A. A telehealth and telepsychiatry economic cost analysis framework: scoping review. Telemed J e-Health. 2022;29(1). doi: 10.1089/tmj.2022.0016
- American Hospital Association. Fast facts on U.S. hospitals, 2019 | AHA [Internet]. American Hospital Association; 2022. Available from: https://www.aha.org/statistics/fast-facts-us-hospitals [cited 25 February 2023].
- Telepsychiatry program eases patient crowding in the ED, expedites mental health services to patients and providers. ED management: the monthly update on emergency department management [Internet]. 2013 [cited 2023 Apr 6];25(11):121–4. Available from: https://pubmed.ncbi.nlm.nih.gov/24195141/
- Saeed SA, Jones K, Muppavarapu K. The impact of NC Statewide Telepsychiatry Program (NC-STeP) on cost savings by reducing unnecessary psychiatric hospitalizations during a 6½ year period. Psychiatr Q. 2021;93:527–36. doi: 10.1007/s11126-021-09967-y
- Mishkind MC, Doarn CR, Bernard J, Shore JH. The use of collaboration science to define consensus outcome measures: a telemental health case study. Telemed J e-Health. 2013;19(6):455–9. doi: 10.1089/tmj.2013.0069
- Shore JH, Mishkind MC, Bernard J, Doarn CR, Bell I, Bhatla R, et al. A lexicon of assessment and outcome measures for telemental health. Telemed J E-Health [Internet]. 2014 ;20(3):282–92. Available from: https://pubmed.ncbi.nlm.nih.gov/24476192/ [cited 25 October 2022].
- 2021 Hospital Emergency Department—characteristics by facility (Pivot Profile) [Excel]. Sacramento, CA: Department of Health Care Access and Information, Healthcare Analytics Branch; 2021. Available from: https://data.chhs.ca.gov/dataset/hospital-emergency-department-characteristics-by-facility-pivot-profile [cited 4 April 2023].
- Managing patient flow: a shared responsibility. The Joint Commission—The Source [Internet]. 2019;17(5):11. Available from: https://store.jcrinc.com//assets/1/14/TS_17_2019_05.pdf [cited 6 April 2023].
Appendix 1: Interview Protocol for Clinicians
Current State
⃘ When a patient needs psychiatric care, what is supposed to happen? ■ What community partners do we route patients to, and have these relationships changed meaningfully in the past few years? ⃘ What actually happens? ⃘ How does the current system of care impact other hospital services? ⃘ What is the perceived risk for not having a reliable solution? ⃘ What is the impact on your staff for not having a reliable solution? (Emotional, wellness) ■ How many staff? ⃘ What is working well and needs no improvement?
⃘ Leave the ED without being treated? ■ Leave because the patient has a psychiatric concern and SHC-VC’s psychiatric care would take a long time? ■ Leave because the patient has a non-psychiatric concern but psychiatric patients are boarded in the bed they need? ⃘ Need psychiatric care of any kind? ■ Of them, how many do you think received satisfactory care? Unsatisfactory care? ⃘ Require a psychiatric evaluation for an involuntary psychiatric hold? ⃘ Require referral to a community partner? ⃘ Need a language other than English? ⃘ Are under age 18? ⃘ Are uninsured or may be exposed to financial toxicity?
⃘ What is the total avoidable length of stay, and what type of resources are needed to care for patients during their stay? ■ What is the cost of this care? ⃘ Where is psychiatric care delivered? (Which specific rooms, in case we need to confirm Wi-Fi coverage) ⃘ How many patients are admitted inpatient who need psychiatric care? ■ Referred to other facilities? ■ Discharged home? ⃘ What is the response call time for psychiatric care when provided? ■ Are there any gaps in coverage? How big are those gaps? ■ When those care gaps occur, what happens? ⃘ What is the response evaluation time for psychiatric care when provided? ■ Are there any gaps in coverage? How big are those gaps? ■ When those care gaps occur, what happens? ⃘ What staff members are involved in psychiatric care currently? ⃘ Are there times of the day, week, year, etc. when psychiatric care is needed the most? ⃘ Are there any laws/regulations that currently impact your work in the ED related to psychiatric care? Priorities
⃘ Is in-person coverage a top priority, or is telemedicine a viable option? ⃘ Any concerns / requests / advice Project Planning
|
Appendix 2: Interview Protocol for Administrators
Current State
⃘ When a patient needs psychiatric care, what is supposed to happen? ⃘ What actually happens? ⃘ What is the perceived risk for not having a reliable solution? ⃘ What is working well and needs no improvement? ⃘ What rules or regulations impact psychiatric patients’ care journeys?
⃘ Require a psychiatric evaluation for an involuntary psychiatric hold? ⃘ Are not on an involuntary psychiatric hold, but need psychiatric care that is dictated by other laws or regulations? ⃘ Need a language other than English? ⃘ Are under age 18? ⃘ Are uninsured or may be exposed to financial toxicity?
⃘ What is your department’s workflow for involuntary psychiatric holds? ⃘ If there is an issue with your department, how is it raised and how much time does it take to resolve? ⃘ Which other staff does your department work most closely with? ⃘ How much do psychiatry services typically cost your department? ⃘ What do you see as the pros and cons of telemedicine, institution-salaried physicians, hiring private physicians? ⃘ How much has the hospital invested for similar projects or services in the past 5 years?
⃘ What metrics or evaluations are necessary for budget approval? ⃘ What % improvement or ROI would be considered successful? ⃘ What is the typical submission and approval process like? ⃘ What other recent projects that can serve as references for success? ⃘ How are the following valued in a proposal for your department?Boarding cost per hour ■ Sitter cost per hour ■ Improvement in length of stay inpatient and ED ■ Clearing involuntary psychiatric holds ■ ED care ■ Consultation, including chart review, care plans, Diagnosis and treatment ■ Psychiatrist, 1 h session (with relevant CPT codes) Priorities
Project Planning
|
Appendix 3: Scoring Criteria Methodology – Notes
The USER gives each service a raw score for on each question on the ‘Scoring Criteria - Benefits’ and Scoring Criteria - Implementation Difficulalty’ sheets. Higher scores are favorable.
Components with percent values will be converted into quintiles using the Scoring Table below, with negative percent values assigned a score of 0. For example, an ROI of 65% would be given a score of 4.
Other numeric values are converted into a percent of the maximum component value, and assigned a quintile score. For example, if the greatest LOS improvement across all service options is 4.6 hours, then the service option offering an improvement of 1.4 hours would be assigned a score of 2.
For Yes/No questions, Yes is assigned a score of 3 and No is assigned a score of 0.
| Scoring Table | ||
| Lower Bound | Upper Bound | Score |
| 81% | 100% | 5 |
| 61% | 80% | 4 |
| 41% | 60% | 3 |
| 21% | 40% | 2 |
| 10% | 20% | 1 |
| NA | 0% | 0 |
Prioritization Methodology – Notes
Component scores are averaged to calculate a category score.
The USER enters weights for each category to assign relative importance. Greater weights indicate more importance.
The weighted scores will be summed to create an overall “Benefit” score and “Implementation Difficulty” score per service option.
Since a higher score is better across benefits and implementation difficulty, the two will be summed together to find the one service option with the highest total
The USER may duplicate ‘Prioritization Model’ sheet to create multiple models and compare their outcomes by varying the prioritization weights.
All USER input cells are highlighted in yellow.
Appendix 4: Required User Input
USER must create 1 new sheet per service option modeled by the ‘Template’ sheet
All USER input cells are highlighted in yellow.
On ‘Summary’ Sheet, USER can enter sheets’ names to create a summary of all service options’ ROI values
On ‘Reference - Hourly ED Arrivals’ Sheet, USER must enter data on the volume of ED cases arriving each hour: psychiatric, IPH, and non-psychiatric
The number of patients arriving per hour and the peak hours associated with LOS improvement are used to calculate new revenue
For each service option sheet, the USER must input baseline information for their model into the Assumptions box
Operating Cases 1-3 show differences in ROI depending on which patient populations use the service: IPH’s only, psychiatric transfers, etc.
For each service option sheet, the USER must fill out the sensitivity analysis by changing assumptions for ED capacity during Peak Hours and Psychiatric cases with LOS improvement
The sensitivity analysis shows the difference in outcomes according to the ED capacity during Peak Hours and Psychiatric cases with LOS improvement
All USER input cells are highlighted in yellow.