Mental Health Symptom Reporting to a Virtual Triage Engine Prior to and During the COVID-19 Pandemic

Abstract

Objective: To examine patient-user symptom reporting to an AI-based online virtual triage (VT) and care-referral engine to assess patterns of mental health symptoms (MHS) reporting prior to and during the COVID-19 pandemic. Methods: The frequencies of 11 MHS reported through VT were analyzed over three time intervals: one year prior to the WHO declaring a global COVID-19 emergency; from pandemic declaration to a mid-point in US vaccine distribution/uptake; and one year thereafter. Results: A total of 4,346,987 VT encounters/interviews presenting somatic and MHS occurred, increasing over time and peaking in the COVID-19 post-vaccine interval with 2,257,553 encounters (51.9%). In 866,218 encounters (19.9%), at least one MHS was reported. MHS reporting declined across subsequent time intervals, was lowest in the COVID-19 post-vaccine period (19.1%), and slightly higher in the pre-pandemic and COVID-19 pre-vaccine intervals (p = 0.05). The most frequently reported symptoms were anxiety, sleep disorder, general anxiety, irritability, and nervousness. Women reported anxiety less often and nervousness and irritability more often. Individuals aged 60+ years reported anxiety and nervousness less frequently, insomnia and sleep disorder more often than individuals 18–39 and 40–59 years old, and sleep disorder more often than those aged 40–59 years in all periods (all p = 0.05). Conclusions: Overall VT usage for somatic and mental health symptom reporting and care referral increased dramatically during the pandemic. VT effectively screened and provided care referral for patient-users presenting with MHS. Virtual triage offers a valuable additional vehicle to detect mental health symptoms and potentially accelerate care referral for patients needing care.

1. Introduction

The COVID-19 pandemic was a global event that was associated with an elevated prevalence of mental health problems, including anxiety, depression, and psychological distress [1,2]. A systematic literature review found that the prevalence of mental health disorders in diverse populations increased from 17.0% to 56.0% during the pandemic, with anxiety and depression reported most frequently [3]. The effects of the pandemic on mental health likely manifested in various ways, with the degree of mental health challenges varying by the stage of the pandemic and by demographic and social factors [3,4]. The World Health Organization (WHO) reported that in the first year of the pandemic, anxiety and depression increased globally by 25.0% [5]. Meta-analyses of populations from Europe, USA, South America, and China confirmed high levels of depression and anxiety (20.0–35.0%) during the first two years of the pandemic [6,7,8]. Individuals experienced mental health issues 30 days after testing positive for COVID-19 compared to uninfected historical controls [9]. People contracting SARS-CoV-2 experienced an array of mental health disorders after the acute pandemic phase, with high incidence of anxiety, depression, and stress-related and adjustment disorders [9].

It is intuitive that the pandemic increased the incidence of mental health symptoms (MHS) and illness. Implementation of lockdown restrictions and quarantine measures isolated people from friends and family, and negatively impacted social support networks and psychological resilience [10]. Restrictive disease control measures also undermined the global economy, increasing worldwide unemployment. Thus, COVID-19 not only generated mental health consequences directly, but the personal financial struggles it created led indirectly to their higher incidence [11,12]. Certain demographic groups were affected more than others, including vulnerable populations such as the elderly, who faced elevated risk of hospitalization and death [1,13,14,15]. The elderly suffered socially, financially, and emotionally, as well as physically [16]. Globally, more than 80.0% of COVID-19 deaths were among individuals 60 years and older [17], and a survey of individuals 50–80 years old found that 18.3% experienced a mental health decline, 18.9% reported depression, and 28.3% reported anxiety [18].

Vaccine availability significantly improved mental health, reducing the reporting of depression, anxiety, and feelings of hopelessness [19,20,21]. This study examined patient MHS reporting to an AI-based online virtual triage (VT) and care-referral engine (or “symptom checker”) to assess demographic differences in MHS reporting across the evolving phases of the pandemic. This is the first report in the literature on MHS reporting to a VT engine during the pandemic. An advantage of AI-based VT is its utility in healthcare and public health settings where clinician resources are stretched. Whether as a standalone fully automated platform or integrated within a human triage workflow, VT is highly cost-effective and improves care acuity alignment [22,23]. Another potential utility for VT in public health is surveillance and engagement of users during public health emergencies, such as communicable disease outbreaks. VT can detect and refer patients for needed treatment earlier and facilitate contact tracing or other public health disease control interventions in both crisis and non-crisis settings. For serious acute conditions, VT can enable early detection of patient-users presenting with specific symptoms suggesting an evolving serious pathological condition with high morbidity/mortality and reduce care delays and avoidable serious morbidity or death [24,25].

The COVID-19 pandemic caused major global disruptions, with clinical and population impacts on mental health that continue to be analyzed. However, research specifically addressing the sensitivity of AI-based virtual triage technology in detecting mental health symptoms and providing care referral during such a unique crisis remains sparse. Understanding patterns of MHS reporting can help optimize VT systems for future pandemics or other large-scale emergencies that disrupt healthcare delivery. This study aims to address this gap by examining the ability of VT to detect MHS during distinct phases of the pandemic, thereby informing the development of scalable and accessible mental health screening tools for crisis settings.

2. Methods

2.1. Study Objectives

The objective of this analysis was to assess trends in MHS reporting to a VT and care-referral engine before and during two phases of the COVID-19 pandemic—prior to and after the availability of a vaccine. The research question sought to determine if VT is a sensitive method for detecting mental health issues, and to compare trends reported before and during two phases of the pandemic—pre- and post-vaccination.

2.2. Study Design

The frequency of reporting for 11 MHS to an online free VT and care referral engine, Symptomate, was analyzed during three pandemic periods by patient-user demographics including age, gender, and language. Symptom reporting was evaluated during three discrete time periods: prior to the declaration by WHO of a global COVID-19 emergency, from the declaration of a pandemic until a mid-point in US vaccine distribution and population uptake, and for a year after this point in time. Frequencies of MHS reporting by patient-user age, gender, and language were assessed. This study used only de-identified, anonymous patient data, and all analyses were completed in the aggregate. All VT patient-users conveyed consent for their data to be used in the aggregate for analyses.

2.3. Setting and Description of Intervention/Virtual Triage Engine Utilized

The Infermedica Symptomate VT engine is designed for general public use and completes evidence-driven analyses informed by over 800 diseases, 1500 symptoms, and 200 risk factors. Leveraging AI, machine learning, and natural language processing, VT evaluates symptoms reported by patient-users, suggesting the most probable conditions matching the presentation and history, and refers the patient-users to the most clinically appropriate and safest care. The technology identifies potential somatic or mental health symptoms that warrant further professional evaluation. There are no prescribed interview pathways, and in light of additional information, the VT AI explores various clinical queries and hypotheses (as physicians do). The VT interview concludes with an analysis of the reported symptoms and a recommendation to pursue one of four levels of care acuity: self-care, consult a primary care or specialist physician on an outpatient basis, proceed to an ED, or call an ambulance for ED transport. Data were extracted from VT episodes engaged by patient-users of the Symptomate VT application, a standalone VT engine available in 24 languages. Over 18 million Symptomate evaluations have been completed since 2012.

2.4. Sample Selection and Inclusion/Eligibility Criteria

During a 35-month period between February 2019 and December 2021, 4,346,987 VT encounters/interviews were completed by adults (age ≥ 18 years). Study participants were selected based upon the following eligibility criteria: (1) completed encounters with at least one reported MHS and (2) reporting incidence equal to or greater than 1.0% of the total remaining sample size. This yielded a study sample size of n = 866,219. Data for these analyses were drawn exclusively from completed interviews where all demographic variables were fully recorded. As a result, there was no need to employ any data imputation methods, nor were any cases excluded due to missing data. The dataset analyzed consists solely of complete virtual triage encounters.

2.5. Data Captured and Analyses Completed

Analyses were performed on a dataset of eligible Symptomate users to assess if the level of reported MHS differed statistically and meaningfully between the three specified pandemic time periods. Completed VT encounters/interviews during the three study periods were examined for the reporting of one or more of 11 MHS including acute anxiety and/or general anxiety; nervousness or weepiness; insomnia and/or sleep disorder; agitation; irritability; stress-related gastric symptoms; suicidal thoughts and/or intent; fear of dying; and feelings of hopelessness.

The entire study period was divided into the following sub-periods or time intervals: (1) interval I was the COVID-19 pre-pandemic period that included eligible VT encounters/interviews conducted between 1 February 2019 and 29 January 2020; (2) interval II began after the WHO declaration of COVID-19 as a global public health emergency on 30 January 2020 and prior to the first release of vaccines in the US through to 13 December 2020; and (3) interval III, a post-vaccine pandemic period, included interviews conducted between 14 December 2020 and 13 December 2021, when the US government declared that 50% of vaccine eligible Americans had been vaccinated with an initial dose of one of the available mRNA vaccines.

The dataset was examined for significant differences among demographic variables (gender, age, and language of VT encounter completion). Given large intrinsic data variation, a sampling weights method was utilized to make demographic groups comparable across the three time periods of interest, and included the following steps: (1) the number of encounters for each unique combination of groups was calculated; (2) the proportion of encounters for each group across time periods were compared; (3) time interval I was set as the reference period; (4) weights were calculated for time intervals II and III to match the proportion of encounters in the reference period; (5) the weights were applied to encounters in time intervals II and III so that they were directly comparable; and (6) subsequent statistical analyses were performed on the weighted data. Sample weights used are shown in

Table A1. Sampling Weights Used Across Time Interval Groups.

Gender Group	Age Group	Language Group	Time Interval I Weight for Each Group	Time Interval II Weight for Each Group	Time Interval III Weight for Each Group
Female	18–39	English	1.0	1.2	1.3
Female	40–59	English	1.0	1.3	1.4
Female	60+	English	1.0	1.3	1.3
Male	18–39	English	1.0	0.1	1.2
Male	40–59	English	1.0	1.0	1.4
Male	60+	English	1.0	0.1	1.6
Female	18–39	Other language	1.0	0.8	0.3
Female	40–59	Other language	1.0	0.7	0.2
Female	60+	Other language	1.0	0.7	0.3
Male	18–39	Other language	1.0	0.1	0.4
Male	40–59	Other language	1.0	0.9	0.4
Male	60+	Other language	1.0	0.9	0.5
Female	18–39	Spanish	1.0	0.1	0.1
Female	40–59	Spanish	1.0	0.2	0.2
Female	60+	Spanish	1.0	0.1	0.1
Male	18–39	Spanish	1.0	0.2	0.2
Male	40–59	Spanish	1.0	0.2	0.1
Male	60+	Spanish	1.0	0.2	0.2
Female	18–39	Polish	1.0	1.5	2.3
Female	40–59	Polish	1.0	1.5	1.9
Female	60+	Polish	1.0	1.5	2.3
Male	18–39	Polish	1.0	1.5	2.5
Male	40–59	Polish	1.0	1.4	2.0
Male	60+	Polish	1.0	1.4	2.6
Female	18–39	German	1.0	1.0	0.8
Female	40–59	German	1.0	0.9	0.6
Female	60+	German	1.0	1.0	0.6
Male	18–39	German	1.0	0.1	0.9
Male	40–59	German	1.0	0.9	0.6
Male	60+	German	1.0	1.0	0.7

.

Table A2. Z-Test Values for Mental Health Symptoms Reported.

Mental Health Symptom	Pre-Pandemic Interval vs. COVID-19 Pre-Vaccine Interval	Pre-Pandemic Interval vs. COVID-19 Post-Vaccine Interval	COVID-19 Pre-Vaccine Interval vs. COVID-19 Post-Vaccine Interval
Anxiety	−22.5	6.6	36.0
Sleep disorder	72.8	74.9	−7.9
General anxiety	1.3	33.3	35.6
Nervousness or weepiness	−10.8	21.3	37.7
Agitation	−6.5	7.6	16.8
Insomnia	−18.8	−16.5	4.9
Irritability	5.8	−83.5	−102.4
Gastric symptoms, stress-related	−28.5	−58.4	−33.8
Feeling of hopelessness	−46.4	−21.7	35.8
Fear of dying	−16.3	−68.9	−64.5
Suicidal intent/thoughts	47.6	62.8	6.8

conveys Z-test values for each of the MHS reported. Conducting statistical analyses on the weighted dataset ensured that the demographic composition of the dataset was balanced across time intervals, allowing for a more accurate comparison of MHS over time that was unaffected by intrinsic demographic differences in the sample. Z-tests for each MHS were adjusted to the weighted samples by taking into account unequal contributions of each observation based on its weight. Thus, the variance of the weighted proportions and standard errors were adjusted for heteroscedasticity in the Z-tests formulas [26].

The sampling weights method was used rather than propensity score matching because study demographic data are limited to three variables (gender, language group, and age group). Thus, logistic regression in propensity score matching would employ the same set as used with the sampling weights. The net analytic value of this would be similar, but would introduce unneeded methodological complexity.

For each specified time interval, the aggregate number of encounters reporting each of the MHS was determined and analyzed by demographic variables to evaluate trends by patient-user gender, age, and language used. Differences in symptom reporting during the three pandemic study time intervals were evaluated for statistical significance. A Z-test with a significance level of p = 0.05 was used to compare each time interval’s pattern of MHS reporting. A p-value of 0.05 was used as a threshold for statistical significance. Data calculations, analyses, and statistical tests were completed using Google Sheets (Online Spreadsheet Editor).

3. Results

A total of 4,346,987 VT encounters/interviews occurred across the entire study period and all three time intervals. Overall VT use or encounter volume, including all somatic as well as mental health concerns, increased substantially from interval I, with 864,087 encounters to 1,225,387 in the second interval after COVID-19 had been declared a global public health emergency, and then again almost doubling to 2,257,553 encounters during interval III, when a vaccine was widely distributed (

Table 1. Mental Health Symptom Reporting Among All Reported Symptoms by COVID-19 Pandemic Time Interval.

Mental Health Symptom	Pre-Pandemic Interval (Percent Reported in Encounters During Time Interval I)	COVID-19 Pre-Vaccine Interval (Percent Reported in Encounters During Time Interval II)	COVID-19 Post-Vaccine Interval (Percent Reported in Encounters During Time Interval III)	Total Encounters in Which Symptom Was Reported (Percent Reported in All Encounters Across All Time Intervals)
Anxiety 1	122,760 (14.2%)	187,900 (15.3%)	314,176 (13.9%)	624,836 (14.4%)
Sleep disorder 2	97,225 (11.3%)	101,169 (8.3%)	191,968 (8.5%)	390,362 (9.0%)
Irritability	20,351 (2.4%)	27,363 (2.2%)	98,876 (4.4%)	146,590 (3.4%)
General anxiety 3	31,547 (3.7%)	44,325 (3.6%)	65,872 (2.9%)	141,744 (3.3%)
Nervousness or weepiness	29,112 (3.4%)	44,719 (3.6%)	65,647 (2.9%)	139,477 (3.2%)
Agitation	26,343 (3.0%)	39,298 (3.2%)	65,152 (2.9%)	130,793 (3.0%)
Insomnia	23,082 (2.7%)	38,201 (3.1%)	68,239 (3.0%)	129,523 (3.0%)
Gastric symptoms, stress-related	10,359 (1.2%)	20,624 (1.7%)	50,052 (2.2%)	81,035 (1.9%)
Feeling of hopelessness	6485 (0.8%)	17,743 (1.4%)	22,935 (1.0%)	47,163 (1.1%)
Fear of dying 1	2691 (0.3%)	5583 (0.5%)	25,700 (1.1%)	33,975 (0.8%)
Suicidal intent/thoughts	9406 (1.1%)	6269 (0.5%)	10,357 (0.5%)	26,032 (0.6%)
Total Encounters reporting 1+ mental health symptom	187,365 (21.7%)	247,160 (20.2%)	431,693 (19.1%)	866,218 (100.0%)

Notes: ¹ Includes anxiety associated symptoms (excessive fear of separation, fear of social interactions, anxiety attack, fear of losing control, specific phobias such as thanatophobia, i.e., fear of dying, hydrophobia, agoraphobia, fear of action, fear of going insane, and general anxiety). ² Includes sleep disorder associated symptoms (decreased need for sleep, prolonged nighttime sleep, insomnia, sleep attacks, restless sleep, waking up during the night, waking up early, somnambulism, and sleep onset and wake time later than desired). ³ General anxiety is a symptom subset of anxiety. Time periods are interval I—COVID-19 pre-pandemic outbreak period (1 February 2019 to 29 January 2020); interval II—COVID-19 pre-vaccine pandemic period (30 January 2020 to 13 December 2020); and interval III—COVID-19 post-vaccine pandemic period (14 December 2020 to 13 December 2021).

).

Reporting of at least one MHS occurred in 866,218 encounters (or 19.9%). The percentage of encounters where one or more MHS were reported decreased over serial time intervals: it was highest in the pre-pandemic period, with 21.7%, it decreased to 20.2% during the COVID-19 pre-vaccine period, and then to 19.1% post-vaccine (p = 0.05) (Figure 1). The percentage of patient-users reporting MHS was significantly higher in the pre-vaccine COVID-19 than in the post-vaccine period (p = 0.05; 95% CI [0.025, 0.027]) (Figure 1).

As seen in Table 1, seven symptoms were reported more frequently during the COVID-19 pre-vaccine (interval II) compared to interval I prior to the outbreak (p = 0.05): anxiety 95% CI [−0.012, −0.010]; nervousness or weepiness 95% CI [−0.003, −0.002]; agitation 95% CI [−0.002, −0.001]; insomnia 95% CI [−0.005, −0.004]; stress-related gastric symptoms 95% CI [−0.0052, −0.0045]; and feeling of hopelessness and fear of death 95% CI [−0.0073, −0.0067]. Seven symptoms were reported less frequently during the COVID-19 post-vaccine period compared to the COVID-19 pre-vaccine period (p = 0.05): anxiety 95% CI [−0.0013, −0.0015]; general anxiety 95% CI [0.0066, 0.0074]; nervousness or weepiness 95% CI [0.007, 0.008]; agitation 95% CI [0.003, 0.004]; insomnia 95% CI [0.0006, 0.0013]; and feelings of hopelessness 95% CI [0.004, 0.005]. Reported more frequently before the pandemic (interval I) than after the pandemic had begun (p = 0.05) were: sleep disorder 95% CI [0.0029, 0.0031]; irritability 95% CI [0.001, 0.002]; and suicidal thoughts 95% CI [0.0055, 0.0060] (Table 1 and Figure 2).

All differences in MHS reporting between the three time intervals were statistically significant (p = 0.05), with the exception of general anxiety.

Table 2. Most Frequent Mental Health Symptoms by Percentage of Encounters with Reported Mental Health Symptoms and COVID-19 Pandemic Time Interval.

Mental Health Symptom	Pre-Pandemic Interval	COVID-19 Pre-Vaccine Interval	COVID-19 Post-Vaccine Interval	All Encounters and Intervals
Anxiety 1	65.5%	76.0%	72.8%	72.1%
Sleep disorder 2	51.9%	40.9%	44.5%	45.1%
General anxiety 3	16.8%	17.9%	15.3%	16.4%
Nervousness or weepiness	15.5%	18.1%	15.2%	16.1%
Agitation	14.1%	15.9%	15.1%	15.1%
Insomnia	12.3%	15.5%	15.8%	15.0%
Irritability	10.9%	11.1%	22.9%	16.9%
Gastric symptoms, stress-related	5.5%	8.3%	11.6%	9.4%
Feeling of hopelessness	3.5%	7.2%	5.3%	5.4%
Fear of dying 1	1.4%	2.3%	6.0%	3.9%
Suicidal intent/thoughts	5.0%	2.5%	2.4%	3.0%

Notes: ¹ Includes anxiety associated symptoms (excessive fear of separation, fear of social interactions, anxiety attack, fear of losing control, specific phobias such as thanatophobia, i.e., fear of dying, hydrophobia, agoraphobia, fear of action, fear of going insane, and general anxiety). ² Includes sleep disorder associated symptoms (decreased need for sleep, prolonged nighttime sleep, insomnia, sleep attacks, restless sleep, waking up during the night, waking up early, somnambulism, and sleep onset and wake time later than desired). ³ General anxiety is a symptom subset of anxiety. Time periods are interval I—COVID-19 pre-pandemic outbreak period (1 February 2019 to 29 January 2020); interval II—COVID-19 pre-vaccine pandemic period (30 January 2020 to 13 December 2020); and interval III—COVID-19 post-vaccine pandemic period (14 December 2020 to 13 December 2021).

shows the most frequent MHS reported by pandemic time interval. Across time intervals, the most frequently reported MHS were anxiety, sleep disorder, general anxiety, nervousness or weepiness, and irritability. Suicidal intent/thoughts and fear of dying were reported least often. The largest increases occurred in anxiety between intervals I and II (10.5%), and in irritability between intervals I and III (12.0%). The largest decreases in sleep disorders occurred from intervals I to II (11.0%).

Demographically,

Table 3. Virtual Triage Use by Patient-User Gender and Age and COVID-19 Pandemic Time Interval.

Gender of Patient-User	Pre-COVID-19 Encounters	COVID-19 Pre-Vaccine Encounters	COVID-19 Post-Vaccine Encounters	Total Encounters
Gender
Female	603,635 (13.9%)	856,072 (19.7%)	1,577,157 (36.3%)	3,036,864 (69.9%)
Male	260,412 (6.0%)	369,315 (8.5%)	680,396 (15.7%)	1,310,123 (30.1%)
Gender Totals 1	864,047 (19.9%)	1,225,387 (28.2%)	2,257,553 (51.9%)	4,346,987 (100.0%)
Age
18–39	646,347 (14.9%)	916,646 (21.1%)	1,688,754 (38.8%)	3,251,747 (74.8%)
40–59	176,791 (4.1%)	250,724 (5.8%)	461,914 (10.6%)	889,429 (20.5%)
60+	40,909 (0.9%)	58,017 (1.3%)	106,886 (2.5%)	205,812 (4.7%)
Age Totals 1	864,047 (19.9%)	1,225,387 (28.2%)	2,257,553 (51.9%)	4,346,987 (100.0%)

Notes: ¹—Due to the rounding of interview weights, rows might not sum up to the exact number as totals. Time periods are interval I—COVID-19 pre-pandemic outbreak period (1 February 2019 to 29 January 2020); interval II—COVID-19 pre-vaccine pandemic period (30 January 2020 to 13 December 2020); and interval III—COVID-19 post-vaccine pandemic period (14 December 2020 to 13 December 2021).

shows that the sample was almost 70% female and young (three-fourths aged 18–39 years), reflecting a similar gender and age distribution to that observed in the Symptomate patient-user database [27].

With respect to MHS reporting by gender, women reported anxiety less often than men, but nervousness or weepiness and irritability more often across all time intervals (p = 0.05). Patient-users aged 40–59 years reported anxiety less frequently and insomnia and sleep disorder more often than younger individuals across all time intervals (p = 0.05). Those aged 60+ years reported anxiety and nervousness less frequently and insomnia and sleep disorder more frequently than younger individuals across all time intervals, but reported sleep disorder more often than those aged 40–59 years (all p = 0.05).

Symptomate does not capture national location of patient-user, but the language of the user is recorded. As shown in

Table 4. Patient-User Language by COVID-19 Pandemic Time Interval.

Language of Patient-User	Pre-COVID-19 Encounters	Pre-Vaccine COVID-19 Pandemic Encounters	Post-Vaccine Encounters	Total Encounters
English	644,006 (14.8%)	913,326 (21.0%)	1,682,637 (38.7%)	3,239,969 (74.5%)
Polish	103,460 (2.4%)	146,726 (3.4%)	270,317 (6.2%)	520,503 (12.0%)
German	52,014 (1.2%)	73,766 (1.7%)	135,900 (3.1%)	261,680 (6.0%)
Spanish	18,413 (0.4%)	26,113 (0.6%)	48,109 (1.1%)	92,635 (2.1%)
Other languages	46,154 (1.1%)	65,455 (1.5%)	120,590 (2.8%)	232,199 (5.3%)
Total 1	864,047 (19.9%)	1,225,387 (28.2%)	2,257,553 (51.9%)	4,346,987 (100.0%)

Notes: ¹—Due to the rounding of interview weights, rows might not sum up to the exact number as totals. Time periods are interval I—COVID-19 pre-pandemic outbreak period (1 February 2019 to 29 January 2020); interval II—COVID-19 pre-vaccine pandemic period (30 January 2020 to 13 December 2020); and interval III—COVID-19 post-vaccine pandemic period (14 December 2020 to 13 December 2021).

, English was used among almost three-fourths, followed by Polish (12.0%) and German (6.0%), all p = 0.05. We suspect that a substantial majority of English language patient-users are based in the USA. In VT encounters, Polish speakers reported less agitation than English speakers across all pandemic intervals (p = 0.05), and reported higher rates of nervousness or weepiness and sleep disorders across periods, with increased anxiety levels during the COVID-19 pre- versus post-vaccine period. Conversely, German speakers reported lower frequencies of anxiety and general anxiety than English speakers across all time intervals (all p = 0.05).

4. Discussion

Overall VT use increased from the pre-COVID interval I to interval II (pandemic pre-vaccine), and then again to interval III, after vaccine availability. MHS reporting decreased in a stepwise fashion from the pre-COVID period to interval III (post-vaccine). Other factors may have contributed to a decline in MHS reporting after vaccines were introduced, including the re-opening of public spaces, schools, and businesses and the restoration of both remote and in-person work opportunities. This study did not examine these issues, which limits the interpretability of the data presented. However, a beneficial impact of COVID-19 vaccination on mental health status was also reported by Chaudhuri [19], Bilge [20], and Chen [21].

Other study limitations include the extent to which VT users do not represent the general population, with higher use among females and younger individuals. However, Xiong et al. [2], Chen et al. [12], and Xie et al. [9] also reported higher prevalence of MHS among young females during the pandemic. VT users are self-selected, potentially biasing the sample towards individuals with more severe symptoms. The large sample size and statistical power of these analyses (866,219) lend confidence with respect to the generalizability of our findings.

The extent to which growth in VT encounters represents unique new patient-users versus repeat visits by returning users is unknown. However, as lockdowns and precautions in healthcare facilities continued, the use of telemedical and virtual healthcare services’ delivery grew substantially. This may be reflected in the doubling of VT use that peaked during interval III. Easy, around the clock access to on demand virtual telecare as the pandemic progressed could have increased VT use. Factors other than public adaptation to telemedical care services may be responsible for growth in VT use over the study period. As unemployment increased, lapsed health insurance coverage could have inspired greater VT use.

Younger patient-users (18–39 years) experienced anxiety more than their elders, who reported more insomnia and sleep disorders. Other substantial pandemic dislocations, such as employment and income disruption, may have more negatively impacted younger versus older individuals. Older adults exhibited a higher incidence of sleep disturbances. Min Du et al. also found older adults reported a high prevalence of poor sleep quality (47.1%) and sleep problems (26.9%) during COVID-19 [28]. Ohayon et al. reported age-related declines in sleep quality, including slow-wave and REM sleep [29]. Pandemic loneliness, limited social interaction, health concerns, and uncertainty likely exacerbated sleep problems among older adults [30].

While MHS reporting decreased once a vaccine was available, we did not observe an increase from the pre-COVID period to interval II, when the pandemic was spreading rapidly and causing many deaths. Less intuitive is higher MHS reporting before the pandemic began than during either COVID-19 time intervals. One explanation derives from a large survey-based study which demonstrated very high levels of stress reported by Americans in 2019. Mass gun shootings, healthcare access, climate change, changing abortion laws, immigration, and the upcoming 2020 presidential election were major stressors among Americans prior to the pandemic [31].

The limitations of an AI-driven reporting tool for mental health include the challenge of effectively detecting the spectrum of often clinically nuanced mental health issues. Unlike somatic health issues, where symptoms are strongly indicative or pathognomonic of certain pathologies, rarely is one or a few MHS associated strongly with particular ailments. While VT may be quite sensitive in early detection of MHS, and effective in care referral, it may offer less specificity in identifying a precise mental health disorder.

5. Conclusions

Virtual triage use increased substantially during the COVID-19 pandemic and was effective in detecting MHS. Patient-users were comfortable reporting MHS to an automated AI-based VT system. MHS reporting decreased slightly from the pre-pandemic to the pre-vaccine pandemic period, and again after vaccines were available. Anxiety and sleep disorders were the most commonly reported MHS followed by nervousness/weepiness, agitation, and insomnia. VT may offer an additional vehicle to detect MHS and accelerate referral of patients to reduce care delays.