Awareness and Use of Virtual Clinics following the COVID-19 Pandemic in Saudi Arabia

Studies have shown that virtual clinics enjoyed high use and high patient satisfaction during the COVID-19 pandemic. Thus, virtual clinics are expected to be the new normal mode of receiving care after the COVID-19 pandemic. This study aimed to assess public awareness and use of virtual clinics following the pandemic and identify factors associated with virtual clinic use. Methods: A cross-sectional design was employed in which data were collected via a structured online questionnaire based on the Technology Acceptance Model (TAM) domains: perceived usefulness, perceived ease of use, and social influence. Participants were selected based on the non-probability sampling of convenience. Univariate, bivariate, and binary logistic regression models were used for analysis. Results: A total of 405 responses were received; of those, 286 (70.6%) were aware of the existence of virtual clinics and 99 (34.6%) were post-pandemic users. Among users, 50% used virtual clinics more than two times, 72% used virtual clinics to seek care for themselves, with the vast majority using it via voice calls (83.8%), and for visits to the family medicine clinic (55%). Young adults, females, single adults, those with a higher level of education, the employed, and those with lower income were more likely to use virtual clinics (p < 0.05). The logistic regression model showed that 20% of the variation in virtual clinic use was explained by perceived usefulness and perceived use (p < 0.001). Conclusion: This study showed high awareness of virtual clinics among the population following the COVID-19 pandemic, with one-third being active users. Age, gender, marital status, education, income, employment status, perceived usefulness, and ease of use are associated with virtual clinics’ awareness and use. Considering those factors is important when planning for sustained use of e-health and virtual care.


Introduction
The healthcare system is evolving at a rapid pace as healthcare providers look for ways to increase healthcare quality and accessibility while decreasing the overall cost. A virtual clinic is a digital technology that allows remote and real-time interaction between patients and doctors through telephone or video calls for diagnosis, examination, and medical assessment [1]. Virtual care can be defined as any remote interaction between patients and healthcare providers, using any form of communication or information technology, to facilitate or maximize the quality and effectiveness of patient care [2].
Several studies [1][2][3][4] have discussed the benefits of virtual clinics, such as saving time and money on in-person visits and reducing the cost of transportation. In addition, convenience and accessibility are key for patients living in remote areas or those who cannot afford to take time off work. Others found virtual care useful in preventing the spread of infections by keeping patients out of the waiting room. Thus, virtual clinics have gained a high level of satisfaction and trust among the population, despite the service being relatively new [5]. McGrail et al. indicated that 93% stated that the virtual visit was of Healthcare 2022, 10, 1893 2 of 10 high quality and 91% indicated that the virtual visit was useful in resolving their health issues, indicating high patient satisfaction [6]. Additionally, others indicated that, during the pandemic, the quality of virtual clinic service was comparable to actual office visits with the additional benefit of fast access to care [7,8].
As part of the Saudi vision 2030 framework released in 2017, the path for digital health transformation initiatives was paved and the use of informatics tools has bloomed since the COVID-19 pandemic [9]. Specifically, the use of virtual clinics has increased dramatically since the onset of the pandemic in 2020 as one of the strategies to support public health control measures in reducing the risk of disease transmission and minimizing exposure to health facilities [10]. The delivery of virtual clinics in Saudi Arabia is provided through several mechanisms. Firstly through hospital appointments, where the doctor determines whether or not the patient requires his presence, or if the patient wishes to obtain the service remotely. Secondly through the Sehhaty mobile application, where the patient can directly book an appointment if the service is available in the healthcare facility. Since February 2022, virtual clinics expanded with the launching of "Seha Virtual Hospital", which is considered the first virtual hospital in the Middle East and the largest virtual hospital in the world. The hospital supports a total of 130 hospitals around the country and provides specialized virtual clinics such as psychiatry, cardiac, endocrine, and diabetes clinics [11]. In addition, virtual multidisciplinary committees composed of multiple consultants and experts from various health facilities were grouped and enabled to communicate and provide their specialized opinions based on the patient's case. The capacity of the hospital during the year reaches over 400 thousand beneficiaries. Given the rising expectations to continue adopting virtual care, there are limited data that explore public utilization of virtual care following the pandemic. This study evaluates public awareness and use of virtual clinics following the pandemic and identifies the factors affecting the actual use among the general adult population. Understanding those factors will help inform the digital strategy as it matures and support the design of plans for sustained virtual clinic use. This has the potential to improve access and continuity of care to patients.

Study Theoretical Model
Many theories exist in social science to predict human behavior, such as the Health Belief Model [12], Social Cognitive Theory [13], Theory of Reasoned Action [14], and Theory of Planned Behavior [15]. However, the Technology Acceptance Model (TAM) is one of the well-known theories to predict human behavior in the adoption of new technologies, such as virtual clinics [16,17]. Based on TAM, perceived usefulness (PU) and perceived ease of use (PEU) are the two main predictors of the intention of technology use. PU refers to the level to which someone thinks that it would be beneficial to use new technology in terms of providing timely and low-cost care. PEU refers to the level to which someone thinks that it would be effortless to use new technology. TAM suggests that, the higher the PU and PEU of a technology a person has, the higher the probability of that person's intent to use the technology.
In addition, other variables from the literature are found to be associated with technology use such as social influence (SI) and demographic variables. SI refers to the level to which someone thinks that others, particularly his/her friends and acquaintances, believe that he/she should use a new system [18]. In Saudi Arabia, there was a significant association between SI and the use of technology in the health sector, including electronic health records [19], electronic triage systems [20], and mobile healthcare services [21]. Furthermore, a study from Pakistan showed that SI affected the public use of telemedicine [18]. Similarly, researchers from China proved that patients' adoption of telemedicine was influenced by their social environments [22]. Therefore, we measured the SI and its impact on the use of virtual clinics. Moreover, scholars found that the use of telemedicine was determined by the user's demographics involving age [23,24], gender [23,24], marital status [23], income [23,25], and education level [25]. Thus, this study also assessed the influence of demographic variables on the use of virtual clinics.

Study Design and Population
This paper presents a cross-sectional study conducted in March 2022 among the general adult population aged 18 years or older residing in Saudi Arabia. Participants were selected using a nonprobability sampling technique, namely convenience sampling. To achieve a 95% confidence interval with a 5% margin of error, the minimum required sample size for large populations was 384 [26].

Survey Instrument
A structured online anonymous questionnaire was used for data collection. The questionnaire consisted of 31 questions organized into three sections. The first section included two questions about awareness (aware, unaware) and use (user, non-user) of virtual clinics, with seven follow-up questions asked to users (i.e., frequency of visits, type of visit, the visitor, type of clinic, type of healthcare facility, mode of communication, and the device used in their last visit to the virtual clinic). The second section included twelve questions informed by elements from the Technology Acceptance Model, with four questions per construct: PU [18,27,28], PEU [18,28], and SI [29]. Questions were adopted from previous studies with few modifications to suit the context and measured on a fivepoint Likert scale, ranging from "strongly disagree" = 1 to "strongly agree" = 5. The sum of all points answered per construct was calculated and analysed, where a higher score of a construct indicates a greater likelihood of using virtual clinics, and vice versa. The third section of the questionnaire included demographic questions such as gender, nationality, age, marital status, residential region, education level, occupational status, monthly income, and the existence of chronic diseases (Supplementary file S1).

Instrument Validation
Linguistic validity was conducted using the translate-translate back methodology, in which two independent certified translation offices translated the questionnaire from English to Arabic and back into English [30]. Face and content validity of the questionnaire was ensured by consultation with expert faculty members at Imam Abdulrahman Bin Faisal University (IAU) and then pilot-tested with a sample of individuals from the general public of Saudi Arabia (i.e., the intended sample) [30]. The internal consistency reliability was tested through Cronbach's alpha (PU = 0.823, PEU = 0.807, and SI = 0.859). The testing revealed that each factor had a Cronbach's alpha higher than 0.8, indicating a good internal consistency [31].

Statistical Analysis
Univariate analysis was conducted using frequencies and percentages for categorical variables, means, SD, and ranges for continuous variables [32]. Skewness and kurtosis criteria were used to test the normality of the continuous variables and showed a normal distribution [32]. Bivariate analysis was conducted using Chi-square tests and Fisher's exact tests to examine the factors affecting virtual clinic use. Multivariable analysis using binary logistic regression was performed for model testing. All analyses were completed using IBM SPSS Statistics (Version 26.0. Armonk, NY, USA: IBM Corp) [33]. A p-value less than 0.05 was considered statistically significant.

Descriptive Statistics (Univariate Analysis)
A total of 405 individuals participated in this study. Most participants were females (n = 299, 73.8%), between 18 and 30 years of age (n = 287, 70.9%), not married (n = 280, 69.1%), Saudi citizens (n = 388, 95.8%), and residing in the Eastern province of Saudi Arabia (n = 293, 72.3%). More than half of the respondents indicated that they were unemployed (n = 249, 61.5%) and the average monthly income in the study group was less than 2801.12 USD (n = 239, 59%). Almost half of the study participants had a bachelor's degree (n = 192, 47.4%) and most of the participants indicated that they did not have any chronic disease (n = 348, 85.9%) ( Table 1).

Virtual Clinic Users
Among users, 49.5% used virtual clinics more than two times. In addition, in their last visit to virtual clinics, 54.5% of users reported that they went for an initial consultation, 72.7% used virtual clinics to seek care for themselves, and most visits were to the family medicine clinic (55.6%). Furthermore, 76.8% of users went for a governmental hospital in their last virtual clinic visit, with the vast majority using it via voice calls (83.8%) via their cellphones (94%) (See Table 2).  Table 3 shows the factors that influence the utilization of virtual clinics. Gender was significantly associated with virtual clinic use (x 2 = 6.037, p = 0.014). The majority of females reported that they have not used virtual clinics before (n = 154, 69.1% of all females). Age was significantly associated with the actual use of virtual clinics (x 2 = 11.323, p = 0.01). Among adults between 18 and 30 years old, 143 (71.1%) individuals were non-users. Marital status also significantly influences respondents' use (x 2 = 10.714, p = 0.001). Most unmarried individuals were non-users of virtual clinics (n = 141, 71.6% of all unmarried participants). Participants' educational level significantly influences their virtual clinic use (x 2 = 12.349, p = 0.006). Those with a bachelor's degree or lower tend to be non-users. In addition, occupational status was also found to significantly influence virtual clinic utilization (x 2 = 10.969, p = 0.003). Most unemployed individuals have never used a virtual clinic (n = 128, 72.7% of all unemployed participants), while 46% of the employed participants indicated that they have used a virtual clinic at least once (n = 46). Furthermore, monthly income was significantly associated with the actual use of virtual clinics (x 2 = 12.501, p = 0.014). Participants with a monthly income of less than 2801.12 USD were more likely to be non-users (n = 118, 70.7%). On the other hand, 60.6% of those with a monthly income between 8403.92 and 11,204.48 USD were actual users of virtual clinics (n = 20). Nationality, living region, and chronic diseases did not have any significant influence on participants' actual use of the virtual clinic (Table 3). Table 4 shows the TAM domains associated with the actual use of the virtual clinic. The table shows that all domains are significantly related to the actual use of the virtual clinic (PU: t = 2.558, p = 0.011; PEU: t = 6.627, p < 0.001; and SI: t = 3.707, p < 0.001). All users of the virtual clinics have on average higher scores in all domains.

Interpretation of Multivariable Analysis
Using a binary logistic regression, two models were tested to predict the actual use of virtual clinics ( Table 5). The first model contains PU and PEU as the independent variables. As shown in Figure 1, this model explained 20.4% of the actual use of virtual clinics, and the beta for both variables was significant. clinic (PU: t = 2.558, p = 0.011; PEU: t = 6.627, p < 0.001; and SI: t = 3.707, p < 0.001). All users of the virtual clinics have on average higher scores in all domains.

Interpretation of Multivariable Analysis
Using a binary logistic regression, two models were tested to predict the actual use of virtual clinics ( Table 5). The first model contains PU and PEU as the independent variables. As shown in Figure 1, this model explained 20.4% of the actual use of virtual clinics, and the beta for both variables was significant.  The second model contains PU, PEU, and SI as the independent variables. As shown in Figure 2, this model explained 20.8% of the actual use of virtual clinics, and the beta for the SI variable was not significant. As there was no major increase in the explanatory power (R2) after the addition of SI, the effect of SI was not significant. The second model contains PU, PEU, and SI as the independent variables. As shown in Figure 2, this model explained 20.8% of the actual use of virtual clinics, and the beta for the SI variable was not significant. As there was no major increase in the explanatory power (R2) after the addition of SI, the effect of SI was not significant. Healthcare 2022, 10, x 8 of 11 Figure 2. Model 2: The model includes perceived usefulness, perceived ease of use, and social influences as independent variables. The presented model explained 20.8% of the actual use of virtual clinics. However, the beta for the SI variable was not statistically significant. * p < 0.05; ** p < 0.001.

Discussion
The study's main objective was to assess public awareness and use of virtual clinics following the pandemic and to identify factors associated with virtual clinic use. The findings showed that 70.6% of participants were aware of the existence of virtual clinics and 34.6% of participants were post-pandemic users. Young adults, females, single adults, those with a higher level of education, the employed, and those with lower income were more likely to use virtual clinics. Those factors were in accordance with previous literature that shows an association between using virtual care and sociodemographic characteristics such as age [23,24], gender [23,24], marital status [23], income [23,25,34], and education level [25].
User-friendly digital tools and interfaces play an important role in the use of virtual care [35]. The perceived usefulness and ease of use reported in this study were significant predictors of virtual clinic use. Our findings are consistent with Kamal et al. (2020), who found that ease, satisfaction, and comfort in using virtual care devices are associated with such use [18]. Yet, social influence was not a significant predictor of the use of virtual clinic use in our model. This finding contradicts the results of an earlier study reporting that social influence was a strong explanatory factor for the use of electronic health record systems in Saudi Arabia [19]. However, this contradiction could be due to the different time frames in which both studies were conducted as well as the intended sample.
As usefulness and ease of use were reported to be key factors affecting virtual clinic use, healthcare leaders must focus on those features in their future improvement plans. It is essential to note that most reported worldwide barriers include digital infrastructure limitations and technology-specific issues such as limited internet connectivity or speed as well as risks related to data security [36]. Other patient-specific barriers include a lack of access to proper communication equipment and patients' low digital health literacy [35,37]. Those barriers can be eliminated by focused policy to strengthen the digital infrastructure and engage all relevant stakeholders to adequately adopt this technology. In addition, other strategies are needed to enhance the user interface design, improve communication between providers and patients, and teach both providers and patients the technological skills required to access virtual clinics. Incentives can also be considered to overcome barriers and encourage using virtual care, which can take the form of free or lower fee-for-service for patients and monetary incentives for providers. Most importantly, regulating virtual clinic platforms and developing standalone virtual clinic practical guidelines are essential to govern providing and receiving virtual care [2].
This study is the first to measure public awareness and the use of virtual clinics following COVID-19 in Saudi Arabia, and to identify factors affecting the use of virtual clinics in light of the Technology Acceptance Model. However, it has limitations. First, the

Discussion
The study's main objective was to assess public awareness and use of virtual clinics following the pandemic and to identify factors associated with virtual clinic use. The findings showed that 70.6% of participants were aware of the existence of virtual clinics and 34.6% of participants were post-pandemic users. Young adults, females, single adults, those with a higher level of education, the employed, and those with lower income were more likely to use virtual clinics. Those factors were in accordance with previous literature that shows an association between using virtual care and sociodemographic characteristics such as age [23,24], gender [23,24], marital status [23], income [23,25,34], and education level [25].
User-friendly digital tools and interfaces play an important role in the use of virtual care [35]. The perceived usefulness and ease of use reported in this study were significant predictors of virtual clinic use. Our findings are consistent with Kamal et al. (2020), who found that ease, satisfaction, and comfort in using virtual care devices are associated with such use [18]. Yet, social influence was not a significant predictor of the use of virtual clinic use in our model. This finding contradicts the results of an earlier study reporting that social influence was a strong explanatory factor for the use of electronic health record systems in Saudi Arabia [19]. However, this contradiction could be due to the different time frames in which both studies were conducted as well as the intended sample.
As usefulness and ease of use were reported to be key factors affecting virtual clinic use, healthcare leaders must focus on those features in their future improvement plans. It is essential to note that most reported worldwide barriers include digital infrastructure limitations and technology-specific issues such as limited internet connectivity or speed as well as risks related to data security [36]. Other patient-specific barriers include a lack of access to proper communication equipment and patients' low digital health literacy [35,37]. Those barriers can be eliminated by focused policy to strengthen the digital infrastructure and engage all relevant stakeholders to adequately adopt this technology. In addition, other strategies are needed to enhance the user interface design, improve communication between providers and patients, and teach both providers and patients the technological skills required to access virtual clinics. Incentives can also be considered to overcome barriers and encourage using virtual care, which can take the form of free or lower fee-forservice for patients and monetary incentives for providers. Most importantly, regulating virtual clinic platforms and developing standalone virtual clinic practical guidelines are essential to govern providing and receiving virtual care [2].
This study is the first to measure public awareness and the use of virtual clinics following COVID-19 in Saudi Arabia, and to identify factors affecting the use of virtual clinics in light of the Technology Acceptance Model. However, it has limitations. First, the sampling technique was nonprobability sampling, which limits our ability to generalize the results of this paper because of the lack of random selection of participants. Furthermore, the sample size was unequal as it included nearly two-thirds of females, and most participants were aged 18-30 years old and lived in the eastern province. This could be because the study questionnaire was carried out by authors located in the eastern province. In addition, data were distributed using an online survey on social media, which limited measuring the response rate and making the data subject to technological literacy bias.

Conclusions
The study showed high awareness of virtual clinics among the population following the COVID-19 pandemic, with one-third being active users. The study shed light on factors associated with virtual clinic use, which need to be considered to sustain the use of virtual care. Investing in enhancing the design of digital tools and platforms to be user-friendly and easy to use is important to engage patients in using virtual clinics, which has the potential to improve accessibility and continuity of care. Informed Consent Statement: Participants provided their agreement in the questionnaire to participate in this study. Therefore, participants explicitly provided their consent prior to completing the questionnaire. Participation was entirely voluntary and the responses were anonymous for the participant and their confidentiality was protected. Additionally, this paper follows the Helsinki principles and STROBE guidelines.